Real-Time IoT Data Logging with InfluxDB and Grafana

How DIGIT built an industrial monitoring pipeline ingesting 50k+ sensor readings per minute with sub-second query latency and automated alerting.

Majid Hussain· Founder & CEO, DIGIT7 min read

Industrial IoT monitoring has a different performance profile from typical web applications: write volume is enormous, data is always time-series, and queries are always range-based. A relational database is the wrong tool. Here is the stack DIGIT uses.

Why InfluxDB

InfluxDB is purpose-built for time-series data. Writes are optimised for high-throughput append-only workloads. Range queries across millions of data points run in milliseconds because the storage engine is sorted by timestamp. Retention policies automatically expire old data, keeping storage costs predictable.

For the industrial monitoring system we built — ingesting 50,000+ sensor readings per minute from 200+ industrial machines — InfluxDB handled the write load on a single 4-core node with CPU utilisation under 30%.

Architecture

Sensors → MQTT broker (Mosquitto) → Node.js subscriber → InfluxDB → Grafana.

The Node.js subscriber is the critical component. It consumes MQTT messages, batches them into InfluxDB line protocol writes of 500–1,000 points, and flushes every 500ms. Batching is essential — individual point writes at 50k/min would overwhelm any system. Batched writes at 100 batches/min are trivial.

Alerting

Grafana Alerting evaluates rules against InfluxDB queries on a 1-minute cadence. When a sensor value crosses a threshold (temperature above 85°C, pressure below 2 bar), Grafana fires a webhook to our notification service, which sends SMS and WhatsApp alerts to the on-call engineer.

We set two thresholds: a warning threshold at 80% of the critical level (early warning, email only) and a critical threshold (SMS + WhatsApp + PagerDuty). The two-level system dramatically reduced alert fatigue compared to the single-threshold approach the client had previously.

Data Retention

We keep raw data for 30 days, 5-minute aggregates for 1 year, and hourly aggregates forever. InfluxDB's continuous queries compute the aggregates automatically. This reduces storage from ~500GB/year (raw) to ~8GB/year (aggregated long-term) while preserving the ability to answer any historical capacity planning question.

Related Articles

Built by DIGIT

Need help building something like this?

DIGIT has shipped 1,000+ projects across web, mobile, AI and cloud. Let's talk about yours.