Trust
Methodology
How gridIQ sources, processes and delivers Australian energy market data. Every number on the platform traces to a primary source, a documented pipeline and a versioned calculation.
Data Sources
gridIQ ingests data directly from the primary publisher in each case. We do not redistribute third-party aggregations or derived datasets. gridIQ is not affiliated with AEMO, DCCEEW or the CER.
Source
Description
NEM dispatch prices (5-min), pre-dispatch forecasts, generation (SCADA), rooftop PV (ASEFS), FCAS prices, interconnector flows, dispatch constraints, market notices, generator bids, STPASA, MTPASA, IRSR auction results, generation information workbook.
Wholesale Electricity Market balancing prices (30-min) and generation mix for Western Australia.
National Greenhouse Account emission factors, versioned per financial year. Used for location-based and market-based Scope 2 calculations.
Clean Energy Regulator weekly clearing-house prices for LGC and STC certificates. ACCU spot prices and QCMR (Quarterly Carbon Market Report) publication data.
72-hour weather forecasts (temperature, wind speed, solar irradiance) for all six market regions. Free public API, no API key required.
Pipeline Cadence
gridIQ runs 22 automated pipelines on Vercel Cron. Each pipeline fetches from its primary source, parses the response, deduplicates against existing records and persists to PostgreSQL. All pipelines are monitored via heartbeat endpoints with alerting on missed runs.
Pipeline
Cadence
Description
NEM Dispatch Prices
Every 5 min
Spot prices, pre-dispatch forecasts, rebid detection and CPT matview refresh for NSW, VIC, SA, QLD, TAS.
NEM Generation Mix
Every 30 min
Large-scale SCADA generation output by fuel type, carbon intensity and market scores.
Rooftop PV
Every 30 min
ASEFS distributed solar PV estimates for NEM regions.
WEM Prices + Generation
Every 30 min
Western Australia balancing prices and generation mix.
STPASA
Every 2 hours
Short-term reserve adequacy (6-day horizon) and generator outage register.
Weather Forecasts
Every 3 hours
72-hour temperature, wind speed and solar irradiance for all six regions.
Generator Bids
Daily
AEMO bid data with a 4-day publication lag. Feeds rebid detection and bidder strategy analysis.
Gas Prices
Daily
STTM (Sydney, Adelaide, Brisbane) and DWGM (Victoria) gas hub prices.
Daily Price Aggregation
Daily
Rolls dispatch-interval prices into daily summaries.
AI Narratives
Daily (06:30 AEST)
14 regional and thematic narratives generated by Sonnet from live market data.
The Grid Brief
Daily (07:00 AEST)
Six-region price strip and overnight/day-ahead digest delivered by email.
Certificate Prices
Weekly
CER LGC and STC weekly clearing-house prices (Saturday AEST publication).
IRSR Auctions
Weekly
AEMO SRA settlement residue auction results; new tranches are auto-discovered and imported.
MTPASA
Weekly
3-year DUID availability and retirement-tagged generators.
Constraint Aggregation
Weekly
90-day binding frequency analysis across all dispatch constraints.
Watt's Week Newsletter
Weekly (Mon 07:00 AEST)
AI-synthesised weekly digest of NEM and WEM activity.
Monthly Price Aggregation
Monthly
Rolls daily prices into monthly summaries.
Peer Benchmarks
Monthly
Percentile computation across consented sites (minimum 10 per category).
Contract Performance
Monthly
Per-contract monthly delivered MWh, variance, VWAP, effective strike and settlement estimate.
ACCU Publication Detection
Monthly
HEAD-checks CER QCMR workbook URLs and alerts admin on new releases.
Generator Information
Quarterly
AEMO Generation Information workbook: nameplate capacity, fuel type, commissioning and closure dates.
Alert Processing
Every 5 min
Evaluates user alert configurations against live data and dispatches email or webhook notifications.
Live pipeline status is available on the system status page.
NEM12 Handling
gridIQ natively parses AEMO NEM12 (Meter Data File Format) files. The parser is a fault-tolerant state machine that never throws; parsing errors and warnings are collected into the result and surfaced in the UI rather than rejecting the upload.
- Multi-NMI support. A single NEM12 file can contain interval data for multiple National Metering Identifiers. Each NMI is parsed and persisted independently.
- E1 + B1 netting. Estimated reads (E1) and basic/substituted reads (B1) are both ingested. Quality flags are preserved per interval so downstream calculations can distinguish measured from estimated data.
- Quality flags: A, S, F, V. Each interval carries the AEMO quality flag (A = actual, S = substituted, F = final substituted, V = verified). Scope 2 calculations note the flag composition of the underlying data.
- Interval lengths. 5-minute, 15-minute and 30-minute interval data are all supported. The parser reads the IntervalLength field from the 200-record and adjusts accordingly.
- AEST / AWST handling. NEM12 timestamps use fixed-offset local time (no daylight saving adjustment). AEST regions use UTC+10; WA1 uses UTC+8. The parser applies the correct offset based on the NMI-to-region mapping.
DCCEEW Factor Versioning
Scope 2 emission calculations depend on the Department of Climate Change, Energy, the Environment and Water (DCCEEW) National Greenhouse Account emission factors. These factors are published annually and sometimes restated retroactively.
- Version stamping. Every Scope 2 calculation records the DCCEEW factor version that was applied. If the factors are updated, historical calculations retain the version they were computed with.
- Append-only restatement. When DCCEEW publishes updated factors, gridIQ adds the new version without overwriting prior calculations. Users can recalculate against the updated factors, producing a new result alongside the original.
- Audit trail. The factor version, calculation timestamp and input data hash are stored with every Scope 2 result. This supports the DCCEEW version audit trail required for ASRS-defensible disclosure.
AEMO Format Changes
AEMO occasionally changes the structure of NEMWEB CSV files: adding columns, renaming fields or changing ZIP archive layouts. gridIQ handles this through a combination of defensive parsing and operational monitoring.
- Fault-tolerant parsing. The AEMO CSV parser reads by column header name rather than ordinal position. Added columns are ignored; missing optional columns fall back to defaults. This absorbs most non-breaking format changes without code updates.
- Heartbeat monitoring. Every pipeline pings an external heartbeat monitor on successful completion. If a pipeline fails to report within its tolerance window (e.g. 15 minutes for a 5-minute pipeline), the on-call team is alerted.
- Data freshness checks. The /api/health endpoint and the daily admin digest both monitor data freshness. Stale data triggers an alert before it reaches users.
- Notification. When a breaking format change is detected, the admin team is notified via the daily digest and Sentry error tracking. Affected pipelines are patched and redeployed, typically within one business day.
What We Don't Do
gridIQ is a data and analytics platform, not a modelling service, advisory firm or regulated entity. Understanding our boundaries is as important as understanding our capabilities.
- We do not model the market. gridIQ reports what happened and what AEMO forecasts will happen. We do not produce proprietary forward-market price views or predict future prices.
- We do not provide regulated advice. Nothing on gridIQ constitutes financial, legal or energy-market advice. Verify Scope 2 outputs with a qualified professional before regulatory submission.
- We are not affiliated with AEMO. gridIQ is an independent product of High Impact Group Pty Ltd. AEMO is the source of our market data; we have no commercial, licensing or partnership relationship with AEMO.
- We do not cover Scope 1 or Scope 3. gridIQ is a Scope 2 specialist. We do not track fuel combustion (Scope 1), value-chain emissions (Scope 3) or supplier engagement.
- We do not operate below the meter. gridIQ works at the grid level and the site meter level. We do not provide behind-the-meter optimisation, battery dispatch, HVAC control or any form of load control.
- We do not provide forward-market price views. PPA backtest and contract performance tools use historical settlement data. Forward-market price views are typically provided by your broker or advisory counterparty.