Grid-Scale Data Engineering for the Energy Transition.
We build the pipelines that move grid data across the IT/OT boundary safely: one-way, CIP-respecting replication out of SCADA historians, AMI head-end and MDM extracts at 15-minute interval granularity, and OMS/GIS feeds landed as governed, CIM-aligned data products for load forecasting and outage intelligence. The same engineered paths carry water-network telemetry — WIMS historian tags and DMA flow and pressure loggers — for water and wastewater utilities.
Fragmented Silos
Legacy Utility systems and disconnected feeds.
CIP-Respecting Ingestion Paths
VEE-Aware Interval Pipelines
Production Reality
SCADA Historian + AMI/MDM Pipelines
SCADA Historian + AMI/MDM Pipelines
One-Way CIP-Respecting Data Paths
Interval Data at Meter Scale
Industry-Specific Friction Points
Trapped OT Signals
The highest-value telemetry lives in SCADA historians inside the electronic security perimeter, where analytics tooling cannot safely reach. Without an engineered outbound path, load research and outage models run on stale monthly extracts instead of live grid signals.
Interval Data at Meter Scale
A million-meter AMI estate generates roughly 96 million interval reads a day at 15-minute granularity. Head-end and MDM systems were built for billing, not analytics — so late reads, estimation flags, and re-reads silently corrupt downstream load models.
Meter-to-Grid Identity Gaps
Meter, transformer, and feeder identifiers rarely reconcile across the AMI head-end, the GIS connectivity model, and the ERP asset master, so engineers cannot roll interval loads up to the transformer that actually serves them.
How the Utility delivery flow works
This technical flow diagram reveals how Unolabs treats Utility data to deliver governed, production-ready outputs.
Source Layer
OT/AMI Ingest
Replicating SCADA historian tags and VEE-processed interval reads through one-way, CIP-respecting collection paths.
Industry Logic
Identity Resolution
Reconciling meter, transformer, and feeder identifiers against the GIS network model and the ERP asset master.
Load Shaping
Aggregating 15-minute interval reads into transformer and feeder load profiles with weather-normalised baselines.
Activation
Grid Serve
Publishing CIM-aligned grid data products that feed load forecasting, DER studies, and outage analytics without touching operational systems.
How the work is engineered for Utility
CIP-Respecting Ingestion Paths
We engineer one-way replication from SCADA historians and the AMI head-end through the CIP DMZ, so analytics consumes live OT data without opening a single inbound connection into the electronic security perimeter.
VEE-Aware Interval Pipelines
We build interval-data pipelines that respect MDM validation, estimation, and editing status — late and re-issued reads are versioned rather than overwritten, keeping load research consistent with billing and settlement.
Grid Identity Resolution
We reconcile meter, premise, transformer, and feeder identities across AMI, GIS, OMS, and the ERP asset master into one connectivity-aware spine for load and outage engineering.
Where the Real Work Is
A Read Is a Claim, Not a Fact
An interval read can change days after it lands. The MDM issues estimates when a meter goes quiet, replaces them when the re-read arrives, and edits values during billing exception review. Pipelines that treat each read as immutable quietly diverge from the billing system of record. We model reads bitemporally — the interval the energy was consumed and the moment the value became known — so a load-research query and a billing reconciliation can each ask for the version of truth they need.
Head-End APIs Were Sized for Billing
AMI head-end and MDM interfaces were designed to hand a billing system one clean file per cycle, not to serve an analytics platform asking for everything, hourly. Vendor APIs throttle aggressively, page awkwardly, and share capacity with operational functions like remote disconnects and on-demand pings. Extraction has to be engineered around those limits — scheduled windows, incremental watermarks, and retry budgets that back off before the head-end does — because a backfill that degrades meter operations is a career-limiting incident.
Historian Tags Are an Archaeology Project
A SCADA historian accumulates decades of tag names shaped by whoever commissioned each substation — the same measurement spelled five ways across the territory. Compression and exception settings tuned for operator trending also distort raw values in ways analytics must understand before trusting them. The unglamorous core of the work is a maintained tag dictionary: every tag mapped to an asset in the registry, with units, scaling, and deadband behaviour documented so downstream models stop guessing.
Visible work products, not vague advice
Each deliverable is designed to be used by Utility architects, engineers, data owners, and operations teams after the engagement ends.
How We Measure Grid Pipelines
Every metric below is baselined jointly in the first weeks of the engagement against your own systems — we measure movement from your starting point, not against invented industry numbers.
Interval completeness at close
Percentage of expected interval reads landed and queryable within an agreed window after interval close, tracked per feeder class so rural comms gaps do not hide behind urban averages.
VEE flag fidelity
Share of reads arriving with validation, estimation, and editing status intact, reconciled against MDM counts each billing cycle — the pipeline should never silently launder an estimate into a measurement.
Meter-to-transformer match rate
Percentage of active meters resolvable to an energised transformer and feeder path, with the unresolved remainder visible in a stewarded exception queue rather than dropped.
Cost per million reads processed
Ingestion and processing cost attributed per million interval reads, trended over time so growth in the meter estate does not translate linearly into platform spend.
Frequently Asked Questions
Do you extract raw head-end reads or VEE-processed reads from the MDM?
For anything billing-adjacent, VEE-processed reads from the MDM — validation and estimation flags are part of the data, and analytics should see exactly what billing saw. Raw head-end reads are added only where lower latency genuinely matters, and they are labelled pre-VEE so no downstream model mistakes them for settled values.
What happens when a read is corrected after we've already consumed it?
Corrections are expected, not exceptional. Each read carries both its interval timestamp and the time the value became known, so a late re-read lands as a new version instead of an overwrite. Downstream marts recompute the affected windows on their next run, and consumers can query either the current best value or the value as known on any prior date.
Do the pipelines keep running through a major storm event?
They are designed for exactly that day. Outage-relevant signals — meter last-gasp events and OMS feeds — get priority lanes, while bulk interval collection backs off under head-end pressure and catches up through automated backfill afterwards. The runbook defines these modes in advance, so storm behaviour is a rehearsed procedure rather than an improvisation.
How an engagement starts
Bring your meter count, read frequency, and one feeder whose data you distrust to a discovery call — we trace that feeder's path from head-end to model, flag where VEE status or identity is being lost, and put the findings in writing before you commit to anything.
Interested in the full industry blueprint?
We have deeper technical documentation for Data Engineering & Integration for Utility in the Utility sector.
Bring one feeder with flaky AMI reads. We'll walk its path from meter to model.