case studies · data and analytics

Data and analytics, move by move.

We've moved warehouses, rebuilt pipelines that ran on a scheduler nobody owned, and fixed dashboards that disagreed with finance. The reports kept running the whole time.

6 case studies5 migrations1 upgrade12 platforms
featured case study

Apache Airflow 2 to Airflow 3

Told without the client's name or anything that could identify them.

what happened

A retailer's data engineering team was still running self-managed Airflow 2 after its end of life on 22 April 2026. Custom operators opened SQLAlchemy sessions against the metadata database. Old DAGs still used SubDAGs and execution_date, and CI pipelines triggered runs through /api/v1. We got the cluster onto the latest 2.x release and ran Ruff's AIR rules, which turned the work into a list. Custom code moved onto the Airflow Python client with API tokens. DAGs then moved in batches to a parallel Airflow 3 environment. For each batch we paused the schedules on the old side and checked the catchup and cron timetable changes. Once the last batch checked out, the Airflow 2 environment was retired.

what the client got
  • The scheduler runs a supported Airflow 3 releaseWe got production DAGs onto Airflow 3. The api-server and dag-processor run as separate services, and they're defined as code with the auth and plugin configuration.
  • Task code no longer touches the metadata databaseWe moved custom operators and callbacks onto the Airflow Python client with API tokens, as Airflow 3 requires. None of them opens a SQLAlchemy session any more.
  • No job ran twice or got skipped at cut-overWe checked every batch against the catchup_by_default change and the new CronTriggerTimetable default. Once a batch was live on 3, its Airflow 2 schedules got paused.
Read the full case study
after / airflow-2-to-3 delivered
beforeApache Airflow 2.x afterApache Airflow 3
01 · CALLERS 02 · AIRFLOW 3 03 · STORAGE AND TARGETS CI pipelines: API token CI pipelines API token Monitoring tools: read-only token Monitoring tools read-only token Airflow 3 api-server: /api/v2 · auth manager Airflow 3api-server /api/v2 · authmanager Scheduler: TaskGroups · Assets Scheduler TaskGroups ·Assets dag-processor: separate service dag-processor separateservice Custom operators: Airflow Python client Customoperators Airflow Pythonclient Metadata database: migrated copy Metadata database migrated copy Data platforms: Snowflake · Databricks Data platforms Snowflake · Databricks
every case study

All 6 in data and analytics

Read any of them and you get the story, the before-and-after architecture, the plan phase by phase and the results.

what goes wrong

Risks that keep coming up

One risk from each of these moves, and the case study that shows how we handled it when it hit.

  1. Silent schedule changes ↗

    catchup_by_default is now False, and bare cron strings now use CronTriggerTimetable unless create_cron_data_intervals is set. DAGs that depend on data_interval values can run with different dates.

  2. UCX is a Labs project ↗

    UCX comes from Databricks Labs without formal support or SLAs. It does most of the inventory and table work well, but plans can't assume it covers everything.

  3. Logic with no SQL equivalent ↗

    Java transformations, stored procedure calls, dynamic lookup caches and update strategies with complex conditions need redesigning if they move to dbt. A line-by-line translation won't work.

  4. Undersizing below F64 ↗

    A smaller F SKU looks cheaper until every Free-license viewer needs a Pro licence. Viewer counts go into the sizing decision from the start.

  5. LOD and table calculations ↗

    Tableau's FIXED, INCLUDE and EXCLUDE expressions and table calculations have no one-to-one DAX form. Naive translation gives numbers that are close but wrong.

  6. Quiet semantic differences ↗

    Teradata compares strings case-insensitively by default and SET tables silently drop duplicate rows. Snowflake does neither. Converted SQL can run cleanly and still return different numbers.

01Is staying on Airflow 2 a real risk?

For a self-managed install, yes. Airflow 2 reached end of life on 22 April 2026, so security fixes have stopped. Managed services set their own dates, which we check for your provider.

02Can we keep some tables in the Hive metastore for now?

Yes. Hive metastore federation lets Unity Catalog govern tables still in Hive while you finish. It's a bridge, so don't plan to live on it.

03Is PowerCenter going out of support?

It depends on your version. Informatica publishes end-of-support dates per version in its support portal. We look yours up before setting the timeline.

04Do Pro or Premium Per User licences change?

No. Pro, PPU and embedded EM and A SKUs aren't part of this retirement. Only per-capacity P SKUs are.

05Do we need Fabric capacity, or is Power BI Pro enough?

Usually it comes down to viewer count and data size. Pro works per user. On F64 or larger, someone with a Free license can read content in those workspaces. For a large audience that changes the math.

other areas

More case studies by area

The same approach, applied to other parts of the estate.

sources 8 sources linked last checked 2026-10-20 content fingerprint 729e65d5
  1. 01Apache Airflow: Supported versionsairflow.apache.org ↗
  2. 02Apache Airflow: Upgrading to Airflow 3airflow.apache.org ↗
  3. 03Microsoft Learn: Migrate your account to UC-only workspaces (Azure Databricks)learn.microsoft.com ↗
  4. 04Databricks docs: Use the UCX utilities to upgrade your workspace to Unity Catalogdocs.databricks.com ↗
  5. 05Databricks: AI governance with Unity AI Gatewaydatabricks.com ↗
  6. 06Databricks: What's new with Unity Catalog at Data + AI Summit 2026databricks.com ↗
  7. 07Salesforce: Salesforce completes acquisition of Informatica (18 Nov 2025)salesforce.com ↗
  8. 08Informatica: PowerCenter to Cloud Modernizationinformatica.com ↗
Reviewed by the Cohash team Product names are trademarks of their owners. how we handle facts and data →
next step

Facing one of these moves?

Tell us what you're on now and where you need to be. We'll come back with how we'd run the move.

useful to bring
  • The platform and the version you're on
  • Where you need to get to, and by when
  • What's custom: rules, plugins, integrations
  • Any audit finding or deadline driving it