Governance as Code: Automating Data Quality in the Cloud

Governance as code means expressing data quality rules, access policies, retention requirements, and lineage expectations as version-controlled artefacts that pipelines enforce automatically. A schema change fails CI until the contract is updated. A pipeline that breaks a quality rule is blocked before consumers see bad rows.

#Why Manual Governance Fails

What this means for you: if your governance lives in a spreadsheet and your changes ship daily, the gap between the two is your actual risk position.

Governance that lives in spreadsheets, wiki pages, chat threads, or quarterly review meetings never stays aligned with production. The rules exist. Nothing enforces them at the moment a change is made.

Meanwhile the documentation describing how things should be drifts further from how things actually are:

• A developer renames a column and nothing downstream is consulted.
• A source system starts emitting nulls in a field that had been reliably populated for years.
• An analyst is granted access "temporarily", and the grant outlives the reason for it.

The failure pattern is structural rather than cultural. Manual checks fail on three axes at once:

• Too late. They run after data has landed and reports have shipped.
• Too infrequently. Audits are quarterly; changes are daily.
• Too far from the change. Reviewers lack the context needed to judge what is actually in front of them.

The fix is the one software engineering already applied to infrastructure and to security. Stop describing the desired state in documents. Declare it in code that machines verify on every change.

#What Governance as Code Actually Means

What this means for you: four categories of governance intent become reviewable files in a repository, next to the pipelines they govern.

The access a person holds becomes derivable from the repository.

That property is the one auditors ask for, and it is the one a spreadsheet can never provide. Once these artefacts exist as code, everything the software lifecycle offers arrives with them: peer review of policy changes, a history of who changed what and why, rollback of a bad rule, and environment promotion of governance alongside the pipelines it governs.

The vocabulary here matters. Data governance as a discipline predates all of this; what changes is only where the rules live and what evaluates them.

#Shift-Left Data Quality: Testing Before Merge, Gating Before Deploy

What this means for you: the cheapest place to catch a breaking change is a code review, and that is a scheduling decision more than a tooling one.

Traditional data quality operates at the right-hand end of the lifecycle — dashboards that flag bad data after it lands. Shift-left moves the checks to where changes originate.

At the pre-merge stage, a pull request that modifies a pipeline or a model runs contract tests in CI. Three questions get asked mechanically:

• Does the output schema still match the published contract?
• Do transformations still pass their rule assertions against representative test data?
• Does the change break a declared downstream dependency?

A failing check blocks the merge. The conversation about a breaking change then happens between engineers in a code review, rather than between a steward and an angry report consumer a week later.

At the deployment stage, CD gates verify governed properties in the target environment before a release goes live. Policy files are applied and verified, quality rules execute against a staging snapshot, and a release that would violate a contract objective is held.

A contract violation caught in CI costs a review comment. The same violation caught in production costs an incident, a backfill, and a little of the business's trust in the platform.

This is where a data quality platform earns its keep. Our DQ Sentinel service operates exactly this layer — contract validation, rule execution and quality gating wired into pipelines a team already runs — and the monitoring discipline that sits beside it covers what to watch once the gates are live.

#Where Policy Decisions Are Evaluated

What this means for you: the engine you choose matters far less than the number of moments at which it is consulted.

A family of policy-as-code engines exists to separate policy definition from policy enforcement. They share a shape: policies are written in a declarative language, stored in version control, and evaluated at decision points. May this pipeline write to this dataset? May this role read this column unmasked? Does this schema change satisfy the contract? The engine returns allow, deny, or warn, and the platform acts on the verdict.

The important design choice is where those decision points sit. Mature setups evaluate policy at three moments: at review time as advisory checks in CI, at deployment time as hard gates in CD, and at runtime as continuous evaluation against live systems.

What happens at that third point once an estate gets large — machine-assisted classification, context-aware access decisions, compliance checks against live systems — is a discipline of its own, covered in automating governance decisions at runtime. This article stays with the delivery pipeline.

#Data Contracts: Schema, Semantics, and Service Levels

What this means for you: the contract is the artefact that turns a schema change from a surprise into a negotiated, versioned event.

The data contract is the workhorse of governance as code. A useful one covers three layers that schemas alone miss: the structure (fields and types), the semantics (what each field means, who owns it, how sensitive it is), and the service levels (the freshness, completeness and availability the producer commits to).

contract: orders_daily
version: 2.3.0
owner: commerce-data-team
consumers: [finance-reporting, demand-forecast]
schema:
  - name: order_id
    type: string
    constraints: [not_null, unique]
  - name: order_total
    type: decimal(12,2)
    semantics: "Gross order value incl. tax, in settlement currency"
    constraints: [not_null, ">= 0"]
  - name: customer_email
    type: string
    classification: pii
    masking: hash_for_non_privileged
quality_slos:
  freshness: "available by 06:00 UTC daily"
  completeness: "row count within 5% of source system"
  null_rate: "order_total null rate = 0"
changes:
  breaking: require_major_version_and_consumer_signoff
  additive: allowed_with_minor_version

Everything in that file is mechanically checkable. CI validates producer output against the schema and constraints. The deployment gate verifies the quality objectives are being met in staging. The changes block makes schema evolution a negotiated event with a version number attached.

Because consumers are named in the contract, the platform knows who to notify and whose sign-off to require when a breaking change is proposed.

#Drift Detection and Enforcement in CD

What this means for you: merge-time checks verify intent. Only drift detection verifies reality, and reality is what a regulator asks about.

Production systems drift, and they do it quietly:

• A source starts sending a new enum value nobody declared.
• A manually applied access grant bypasses the policy repository entirely.
• A table's freshness degrades from hours to days without crossing an alert threshold.

The continuous-delivery half of governance as code closes that gap with two mechanisms.

Drift detection continuously compares the declared state — contracts, policies and rules in the repository — against the observed state of actual schemas, actual grants and actual quality metrics. Divergence raises an alert with the diff, which is the same mental model as infrastructure drift detection applied to governance artefacts. Declared lineage is what makes an undeclared edge detectable at all.

Enforcement decides what happens on divergence, and the honest answer is graduated. New rules start in observe mode to build a baseline, move to warn so violations surface in CI and dashboards, and only then to enforce, where they block the merge, hold the deployment, or quarantine the data.

#Governance Artefacts and Where They Are Enforced

#A Rollout Playbook: Audit, Codify, Gate, Enforce

What this means for you: four stages, in order. Teams that attempt them simultaneously produce artefacts nobody trusts.

The cultural shift matters as much as the tooling, and it is the part that survives a reorganisation.

Governance stops being a team you ask permission from and becomes a property of the platform.

The same instinct underpins platform engineering and DataOps culture, and the enabling capability usually sits with the team that already owns DevOps and delivery automation.

Where governed data feeds AI systems, the evidence these artefacts produce is exactly what the regulatory operating model above them needs as input. What the data itself must look like first is set out in what an AI-ready foundation requires, and the resulting controls are what your security and compliance function will have to defend.

#Frequently Asked Questions

#How is governance as code different from writing data quality tests?

Quality tests are one artefact class within it. Governance as code extends the same version-and-enforce treatment to access policies, retention rules, data contracts, and lineage expectations. Critically, it wires all of them into enforcement points in CI and CD, so the repository both describes and controls the governed state rather than sitting alongside it as documentation that ages.

#Which artefact should we codify first?

Data contracts for the two or three datasets that feed board-level or regulated reporting. They have obvious owners, obvious consumers, and an obvious cost of failure, which makes the first enforcement conversation easy to win. Access policies usually follow, because they carry the clearest audit value. Retention and lineage declarations are worth deferring until the first two are stable.

#Do we need a data contract for every dataset?

No, and attempting one for everything is the most common way these programmes stall. Contracts earn their overhead where a producer-consumer boundary carries real risk: datasets feeding financial reporting, regulated data, and interfaces between teams. Internal scratch tables and exploratory datasets do not need contracts until somebody depends on them, at which point the dependency itself is the trigger.

#Will blocking merges on governance checks slow engineering down?

Blocking on unbaselined rules will, which is exactly why the observe-warn-enforce progression exists. Once the rules are tuned, gates speed teams up on net: breaking changes are caught in review where they cost minutes, instead of in production where they cost incident response, backfills, and re-earned trust. Measure the change in lead time before and after.

#Should governance artefacts live with the pipelines or centrally?

Keep contracts and quality rules with the pipeline code that produces the data, so a change to one forces review of the other in the same pull request. Cross-cutting policies — access, retention, classification standards — usually live in a shared repository owned by the platform team and consumed by every pipeline's CI. The split follows ownership rather than convenience.

---

Unolabs is a Data and AI first engineering consultancy, headquartered in the United Kingdom with engineering operations in Pune and active engagements across the UK, Australia, and Hong Kong. We help enterprises build the architectural foundation for autonomous AI execution — governed data platforms, semantic intelligence, and agentic systems that enterprises can stand behind.

If you are deciding which governance rules to codify first, or how to gate them without stalling delivery, book a discovery call.