Skip to content
Yash Singh Ramgadiya

Case study

Organisation
Burda Luxury, Asia Pacific

Redshift-to-DuckDB Analytics Modernisation

A platform modernisation programme moving suitable analytical workloads from Amazon Redshift to DuckDB, targeting approximately 40% lower Redshift infrastructure cost, with production-versus-migrated validation and approval gates.

~40%Target: lower Redshift infrastructure cost

What changed

Migration decisions are gated by production-versus-migrated metric validation rather than infrastructure parity alone.

Data PlatformsCost OptimisationModernisation
Redshift-to-DuckDB migration and validation flowWorkloads are assessed, migrated, then compared production-versus-migrated using absolute delta, percentage delta and metric-level checks. An approval gate precedes cutover. The programme targets approximately 40% lower Redshift infrastructure cost; this is a target, not an achieved saving.Workload assessmentMigrationProduction vs migrated comparisonAbsolute deltaPercentage deltaMetric-level checksApproval gateCutover
Redshift-to-DuckDB migration and validation flow. ~40% lower Redshift infrastructure cost - target

Context

The analytics platform runs on Amazon Redshift. As transformation layers and semantic models grew, so did infrastructure cost, while many workloads did not need a large distributed warehouse.

Challenge

Reduce cost without degrading performance or governance, and without disrupting the reporting and analytics products that depend on the platform.

My Role

I lead the programme: setting the cost target, defining which workloads are candidates for DuckDB, and sequencing the migration with the analytics engineering team.

Approach

  • Profile workloads to identify those suited to DuckDB's in-process analytical engine.
  • Migrate suitable workloads incrementally while keeping Redshift for what genuinely needs it.
  • Validate every migrated workload against production: absolute delta, percentage delta and metric-level checks, with approval gates before cut-over.
  • Preserve semantic-layer governance so consumers see no change in definitions.

Architecture / Analytics Design

A hybrid model in which DuckDB serves suitable analytical workloads while Amazon Redshift remains for workloads that require it, with the semantic layer abstracting the engine from consumers. Production-versus-migrated comparison runs at metric level and gates approval.

Business Impact

The programme is targeting approximately 40% lower Redshift infrastructure cost. This is the programme target and is stated as such.

Technology

  • Amazon Redshift
  • DuckDB
  • dbt
  • Semantic layer

Key Lessons

  • Not every analytical workload needs a distributed warehouse.
  • A semantic layer makes engine migration far less disruptive for consumers.
  • Metric-level validation with approval gates is what makes a migration safe to trust.

Related work