Layering, the semantic layer & CI/CD

You now have all the dbt building blocks. This final lesson assembles them into the patterns that make a project understandable, governed, and production-grade: a layered architecture, a semantic layer for trustworthy metrics, and a CI/CD workflow that ships changes safely. We'll end with the 2026 landscape — dbt Fusion and the SQLMesh alternative — so you know where the field is heading.

Layering: staging → intermediate → marts

A pile of models with arbitrary ref()s becomes spaghetti fast. The durable fix is a layered convention: every model belongs to one of three layers, and data flows strictly downhill through them.

We'll name the layers on ShopFlow (ShopFlow — see Meet ShopFlow):

• Staging — one model per source table, doing only light cleanup: rename columns to a standard, cast types, fix obvious nulls. No joins, no business logic. stg_orders, stg_customers, stg_products, stg_order_items — one per raw.* table. Think "make each raw table clean and consistent." Usually materialized as views (cheap, always live).
• Intermediate — the messy middle: join staging models together, apply business logic, build reusable pieces that aren't yet a final product. int_customer_orders (the per-customer rollup feeding dim_customer's lifetime_value). Often ephemeral or views. Not exposed to end users.
• Marts — the polished, business-facing products: the fact and dimension tables (Chapter 4) that analysts and dashboards actually query. ShopFlow's fact_sales, dim_customer, dim_product, dim_date. Usually materialized as tables (queried constantly → pay at build, not query).

This is the medallion architecture

If you've heard of medallion architecture (bronze/silver/gold — common in the Databricks/lakehouse world, Chapter 10), dbt's layers are the same idea under different names:

Medallion	dbt layer	ShopFlow contents
Bronze	(raw sources)	raw.orders, raw.customers, raw.products, raw.order_items — untouched as loaded
Silver	staging + intermediate	stg_orders … stg_order_items, int_customer_orders — cleaned, conformed, joined
Gold	marts	fact_sales, dim_customer, dim_product, dim_date — business-ready facts & dimensions

The durable principle behind both: data gets progressively cleaner and more business-shaped as it flows through named layers, and each layer has one clear job. Bronze/silver/gold and staging/intermediate/marts are two vocabularies for that one idea. Knowing they're the same thing lets you read either ecosystem's docs without confusion.

The semantic layer: one definition of every metric

Here's a problem layering alone doesn't solve. At ShopFlow, finance computes "revenue" one way; the sales dashboard computes it another; a data scientist writes a third query — one forgets to exclude cancelled orders, another sums order_amount instead of line_revenue. Three numbers, three definitions, endless "why don't these match?" meetings. The cause: metric logic is scattered across BI tools and ad-hoc queries, each re-implementing the definition.

A semantic layer fixes this by defining each metric once, in one governed place, so every tool asks the semantic layer for "revenue" and gets the same computation. In dbt this is MetricFlow (the engine behind the dbt Semantic Layer). You define metrics in YAML on top of your marts — here, ShopFlow revenue as the sum of line_revenue on fact_sales:

# models/marts/_semantic.yml
semantic_models:
  - name: sales
    model: ref('fact_sales')
    entities:
      - {name: order, type: primary, expr: order_id}
    dimensions:
      - {name: order_date, type: time, type_params: {time_granularity: day}}
    measures:
      - {name: revenue, agg: sum, expr: line_revenue}

metrics:
  - name: total_revenue
    type: simple
    type_params: {measure: revenue}     # revenue = sum(line_revenue)

Now ShopFlow's total_revenue has one definition: sum(line_revenue) over fact_sales. A BI tool, a notebook, or an API call requests total_revenue (optionally sliced by order_date or any dimension), and MetricFlow generates the correct SQL on the fly. You don't pre-build every possible slice; you define the metric and its dimensions, and the warehouse computes the requested cut. The durable win: governed metrics — a single source of truth for what each business number means — so every ShopFlow dashboard agrees. (Looker pioneered this idea with LookML; the dbt Semantic Layer makes it tool-agnostic.)

dbt in CI/CD: what makes it production-grade

Everything so far runs the same whether you type commands by hand or a robot does. CI/CD (Continuous Integration / Continuous Deployment, Chapter 8 and the Cloud guide) is what turns a dbt project from "scripts I run" into "a reviewed, automatically-tested, safely-promoted production system." Three dbt-specific features make this work.

Slim CI with state:modified

When you open a pull request changing two models, you do not want CI to rebuild all 800 models — slow and expensive. Slim CI rebuilds only what changed. dbt compares your branch against a stored manifest (a JSON snapshot of the last production build) and, with the selector state:modified+, builds only the modified models and their downstream children (the + means "and everything downstream"):

dbt build --select state:modified+ --defer --state ./prod-artifacts

This tests exactly the blast radius of your change — the models you touched plus everything they could break — and nothing else.

Deferral (--defer)

But a changed model ref()s unchanged upstream models that CI didn't build. Where do those come from? Deferral is the answer: with --defer, any ref() to a model not built in this run resolves to the production version instead. So CI builds your 2 changed models in a temporary schema, and their references to the other 798 point at the real production tables. You test your change against real upstream data without rebuilding the world.

Environment promotion

The same dbt project runs against different environments by swapping the target (the profile idea): a dev schema where you experiment, a CI schema for pull-request checks, and prod where the real tables live. A change flows dev → CI (slim, deferred, tested) → merge → prod. The project is identical across all three; only the connection target differs. This is "build once, promote through environments" applied to data — the discipline that makes dbt trustworthy at scale.

Slim CI and deferral are the mechanics of running dbt on a pull request; the test strategy that runs inside that machinery — unit tests on transform logic, integration tests on a staging schema, and data-diffing a PR's output against production — is covered in Chapter 11's pipeline-testing lesson.

:::tip Slim CI + defer is the whole game The combination — build only state:modified+, --defer everything else to prod — is what lets a 1,000-model project run a fast, cheap, complete CI check on every pull request. Memorize this pairing; it's the most-asked dbt production question in interviews and the thing that separates a toy project from a real one. :::

The 2026 landscape: Fusion and SQLMesh

dbt's mechanics are durable; the engine is evolving fast. Two names to know:

• dbt Fusion — a 2025/2026 rewrite of dbt's engine in Rust (replacing the older Python core). The headline gains are speed and, more importantly, static analysis: Fusion actually parses and understands your SQL before running it, enabling column-level lineage (which specific column feeds which downstream column, not just which table), earlier error-catching, and faster compilation. The mental model from this chapter is unchanged — it's still compiled SQL on a DAG — Fusion just makes the compiler far smarter.
• SQLMesh — the most credible alternative to dbt. It embraces the same "transform with SQL on a DAG" philosophy but adds, natively, things dbt bolts on: built-in column-level lineage, automatic detection of breaking vs non-breaking changes, true blue-green environments via virtual data marts, and stronger handling of incremental backfills. It's smaller in adoption but worth knowing as the named challenger — the concepts transfer directly because they share the same paradigm.

The durable takeaway: the paradigm — version-controlled, tested, layered SQL transformation on a DAG with governed metrics — is settled. Which engine compiles it (Python dbt, Rust Fusion, SQLMesh) is the dated detail that will keep churning. Learn the paradigm; swap the engine when the market does.

Why it matters

A dbt project earns trust through three patterns. Layering (staging → intermediate → marts, the same idea as bronze/silver/gold medallion) gives every model one clear job and a clean flow from raw to business-ready. The semantic layer (MetricFlow / the dbt Semantic Layer) defines each metric once so every dashboard agrees on what "revenue" means. And CI/CD — slim CI with state:modified+, --defer to production, and promotion through dev/CI/prod environments — is what makes dbt production-grade rather than a pile of scripts. In 2026, dbt Fusion (Rust, static analysis, column-level lineage) and SQLMesh (the alternative) are evolving the engine, but the paradigm you learned this chapter is the durable part. You can now read any dbt project, justify every materialization, reason about incremental loads, and describe the modern data stack end to end.

Next: Chapter 7 checkpoint →