Job Overview
Job Title: IoT Data Pipeline Engineer
This role builds and maintains the full device-to-cloud data flow for connected equipment. The engineer owns how machines send telemetry, how the cloud processes it, how it is stored, and how customers see it. The work starts at Azure IoT Hub ingestion and ends at customer dashboards and APIs.
Core responsibility is turning raw device messages into reliable, usable information. Devices send binary or CBOR telemetry, alerts, and status updates. The engineer designs the ingestion path, validates data, orders events correctly, and prevents duplication or loss during bad cellular connectivity. The system must handle online, offline buffering, reconnect bursts, and large backfills without breaking reports.
The pipeline includes IoT Hub, server processing, storage, indexing, and reporting layers. The engineer builds Azure Functions or equivalent processing services that decode payloads, apply device rules, detect state changes, and produce normalized records. They implement idempotent processing so repeated messages never corrupt state. They maintain sequence tracking and acknowledgement handling so devices and server stay synchronized.
Storage design is a major part of the job. The engineer structures operational data in Cosmos DB for current machine state and asset history, and indexes time series data into Elasticsearch for visualization and analytics. They separate current state tables from event history and implement retention policies and rollups. High resolution telemetry is summarized intelligently so storage cost and bandwidth stay controlled while faults remain detectable.
The role includes device twin and configuration management. The engineer defines desired versus reported properties, versioning rules, and how devices recover configuration after reboot. They implement safe rollout logic so firmware and sampling changes propagate predictably across fleets.
Reporting is not an afterthought. The engineer prepares data models optimized for dashboards, alerts, and customer APIs. They ensure alarms, DTCs, runtime, load, and performance metrics remain accurate even when messages arrive late or out of order. They also design summary generation so customers receive meaningful reports instead of raw data floods.
Reliability matters more than raw speed. The system must survive intermittent connectivity, duplicate packets, partial uploads, and firmware mismatches. The engineer builds retry queues, dead-letter handling, replay capability, and monitoring so failures are detectable and recoverable.
Required skills include strong backend development in C#, Python, or Node, cloud architecture in Azure, event-driven processing, distributed systems behavior, and time-series data handling. Experience with IoT Hub, message sequencing, telemetry normalization, and large scale logging systems is expected. Familiarity with embedded devices and constrained networks is important because design decisions must account for real hardware limits.
Success in this role means machines can lose signal for hours, reconnect, upload thousands of records, and still produce correct dashboards and reports without manual cleanup. The engineer is responsible for making the data trustworthy.
Job Types: Full-time, Part-time, Contract
Pay: $97,765.31 - $117,738.88 per year
Expected hours: 40.0 per week
Work Location: In person
