From Sensors to Decisions: Real-World Energy Data Pipelines

Learn how a NSW energy distributor moved from month-long BI queues to live AI-powered analytics, real architecture and trade-offs included.

Rajat Naik from Essential Energy walks through how they ditched month-long BI queues by deploying AI-powered analytics for 50+ stakeholders, while Pravin Varma from Databricks covers the operational technology data pipelines that make it work. You'll see the actual architecture choices, governance patterns, and trade-offs they made in a regulated environment, from Unity Catalog semantic layers to row-level security for natural language queries. This is hands-on, lessons-learned content aimed at data engineers, architects, and analysts working in energy and utilities, though the patterns translate to any data-heavy, mission-critical operation. Expect practical depth: how they shipped without waiting for perfect data models, why sub-second query performance over 1.4 TB matters for compliance teams, and what it takes to stream sensor data reliably at scale.