What Is the Silver Layer?

The Silver Layer is the second tier of the Medallion Architecture — the transformation layer that converts raw Bronze records into clean, validated, business-conforming data that downstream Gold transformations and analytics can trust. Silver is where data quality enforcement happens: records that don't meet quality standards are rejected (written to a quarantine table for investigation) rather than propagated to Gold where they would corrupt business metrics.

Unlike the Bronze Layer (which preserves data exactly as received), the Silver Layer applies business-defined transformation logic: deduplication of duplicate events, schema standardization (consistent column names and types across sources), null handling, and — most importantly — CDC state management (maintaining the current state of source operational database tables from a stream of INSERT/UPDATE/DELETE events).

Silver Layer: CDC Current-State Pattern

The most important Silver Layer pattern for operational data is the CDC current-state table: a Silver Iceberg table that maintains the present state of a source operational database table by applying CDC change events via MERGE INTO.

Implementation:

  1. Bronze table contains a log of all CDC events (INSERT, UPDATE, DELETE) with operation type and full row image
  2. Scheduled Spark or dbt job reads new Bronze CDC events since the last run
  3. MERGE INTO applies events to Silver current-state table: INSERTs become new rows, UPDATEs replace existing rows, DELETEs remove rows
  4. Silver table always reflects the current state of the source — analysts can query 'what is the current status of order #12345?' against Silver, not Bronze
Silver Layer CDC Current State Pattern diagram
Figure 1: Silver CDC current-state pattern — MERGE INTO transforms CDC events into current-state Iceberg tables.

Silver Layer Maintenance

Silver tables require consistent maintenance because frequent MERGE INTO operations accumulate many small files and delete files:

  • Compaction: Merge small data files and delete files created by MERGE INTO operations. Run daily or after large batch MERGE operations.
  • Z-Ordering: Sort data by common query filter columns (customer_id, region, order_date) for improved data skipping. Run weekly for heavily queried tables.
  • Snapshot expiry: Remove snapshots older than the retention window (typically 7 days for Silver). Each MERGE INTO creates a new snapshot; retaining thousands accumulates metadata overhead.
Silver Layer Maintenance Schedule diagram
Figure 2: Silver layer maintenance — compaction, Z-ordering, and snapshot expiry cadences.

Summary

The Silver Layer is the trust-building tier of the Medallion Architecture — the transformation stage where raw Bronze data is transformed into the clean, validated, business-conforming datasets that Gold aggregations and analyst queries depend on. Implemented as Apache Iceberg tables with regular MERGE INTO for CDC current-state maintenance and compaction for read performance, Silver tables are the reliable middle tier that separates raw ingestion complexity from business analytics simplicity. High-quality Silver tables are the most important investment a data engineering team can make for downstream analytical trust.