Data generated by IoT devices is being consumed by an ever-growing number of downstream systems — real-time analytics, model training, AI agent decision-making. The familiar engineering playbook is to let MQTT handle device connectivity and Kafka handle the data pipeline, gluing the two together with a bridge layer. This architecture works well enough in traditional IoT scenarios. But when the number of downstream consumers grows from one to five, when the same data needs to feed both a training pipeline and a real-time inference service simultaneously, the cracks in the bridge layer become hard to ignore: unpredictable latency, silent data loss, broken backpressure propagation.

The root cause is not the bridge layer itself, but the positioning of the MQTT Broker — designed as an ingestion layer, eager to push data out the moment it arrives, with no built-in durability or stream consumption capabilities. Every new downstream use case means another bridge link and another operational blind spot.

This talk shares architectural lessons from RobustMQ, an open-source unified message broker built in Rust. The core idea: a unified storage layer that natively supports both MQTT and Kafka protocols — data written by devices over MQTT can be consumed directly via the Kafka protocol, with no bridging, no data copying, one message, one copy, consumed by any protocol.

We will share: • Real pain points of MQTT + Kafka bridge architectures in multi-consumer IoT scenarios • Key architectural decisions in evolving the MQTT Broker from “ingestion layer” to “data layer” • The unified storage layer design: one copy of data, consumed natively by multiple protocols • The full edge-to-cloud pipeline: offline buffering, auto-sync, minimal deployment footprint • Current progress, known limitations, and future direction

This is an ongoing architectural exploration, not a success story. RobustMQ is fully open-source under the Apache 2.0 License. We look forward to exchanging real-world experience with engineers who are tackling the same IoT data plumbing problems, and we hope to eventually become an Apache Software Foundation top-level project, giving back to the open-source community.

https://robustmq.com/

https://github.com/robustmq/robustmq

Speakers:

Lobo Xu: RobustMQ Auther

Author of RobustMQ, former senior engineer of ByteDance, former senior engineer of Tencent Cloud. Former Head of Kafka/RabbitMQ technology at Tencent Cloud message Queue. Author of RobustMQ and Apache Kafka/RocketMQ/Pulsar Contributor is active in the message queuing and Serverless open source communities. Deep product, technical and commercial understanding and vision of MQ and Serverless. He is the author of the Geek Hour columns “Breaking Down Message Queues in Depth” and “Rust in Action: Writing the Next Generation of Cloud-Native Message Queues” and co-author of the technical book “The Road to Foundational Software: Enterprise Practices and Open Source Strategies.”