Week 3 · Activation
The pipeline says success
Green runs, wrong numbers. Audit the data, not the dashboard.
The brief
Wednesday, 07:50. Sofia forwards you a screenshot. The revenue tile on the new prototype jumped +189,500 overnight, then dropped back the next morning.
Raj is defensive and has a point: every run of gold_sales_metrics succeeded. Then you look at the run log and find three straight days in June where it didn't, and one morning in July where crm_deals_ingest processed 30% of the usual rows and still reported success.
Lena, quietly: "Reps close deals in the CRM days after the contract is signed. Sometimes they never move the stage at all."
This is Activation week: input ports, transforms, output ports, and SLOs. The first three exist. The fourth is where trust lives. Nobody is going to audit this data for you.
This week: write the data-quality checks a Data PM would insist on before launch. Duplicates, stale stages, missing dates, freshness, failed-run streaks, and row-count anomalies. Window functions do the heavy lifting.
Lessons
Read
Activation: Input Ports, Transforms, Output Ports, SLOs (Week 3)
A data product activates once four pieces exist: input ports, transformation steps, output ports, and SLOs — ideally declared, not hand-built.
Data Quality Dimensions (and how to write an SLO)
Six dimensions — completeness, uniqueness, timeliness, consistency, validity, accuracy — each becomes an SLO once you name the measurement, threshold, and check time.
Medallion Architecture: Bronze, Silver, Gold
A three-layer pattern for structuring a data lakehouse: raw ingestion (Bronze), cleaned/augmented (Silver), business-level aggregates (Gold).
Self-Service Infrastructure (SSI)
Ready-to-use resources (policy, monitor, workflow, contract, service, secret, compute) that let engineers declare intent instead of building plumbing from scratch.