MQTT is now the go-to protocol for low-latency, TCP-connected devices. For good reasons: it is easy to understand and relatively simple to implement on the device side. Outside of the TLS library, the protocol is easy to understand, and many great open-source client libraries exist.
The hardest client-side work is often not MQTT itself. It is getting the connection retry policy, offline behavior, and reconnection logic right.
Look at the server side of the connection: delivering a full-featured, resilient, and scalable MQTT broker means dealing with a very different class of problems.
A broker is a distributed system
The first challenge is maintaining a large number—potentially millions—of long-lived TCP connections. That means managing memory, file descriptors, kernel resources, disconnection detection, and cleanup.
Long-lived connections also create operational constraints. You need to distribute connections across nodes, drain them during maintenance, survive reconnect storms, and protect the TCP stack from slow clients that can consume disproportionate memory and other resources.
Then there is state. MQTT brokers may need to track:
- Subscriptions and shared subscriptions
- Retained messages
- Offline queues and persistent sessions
- Inflight QoS messages and acknowledgements
- Last Will messages and session expiry
Those data structures need to be replicated, partitioned, or coordinated across the nodes in an MQTT cluster.
QoS 1 and QoS 2 add another distributed-computing problem. QoS 1 is simpler than QoS 2, but the broker must manage inflight messages, acknowledgements, and session state.
QoS 2 is where things become much harder. Providing its guarantee in a distributed cluster is difficult during network partitions, failover, and node rebalancing. Even when QoS 2 is implemented correctly, it solves a narrow MQTT delivery problem. It does not give your complete backend exactly-once processing since the transaction is limited to the MQTT level.
Once all of this works at scale, more operational concerns appear, including rolling MQTT server software updates across the broker cluster without disrupting the fleet.
The hardest features also create lock-in
What is interesting about this gloomy picture is that the difficult features are also the ones brokers support differently.
QoS 2 may be unsupported, partially supported, or offered with different operational tradeoffs depending on the vendor. The same applies to queuing, persistent sessions, retained messages, shared subscriptions, and clustering behavior.
That is normal. These are hard distributed-systems problems, and each vendor has made its own tradeoffs.
So is there a better way to scale MQTT while keeping your usage vendor-agnostic?
For many IoT platforms, especially telemetry-heavy ones, I think the answer is yes. My first advice to customers considering MQTT is: do not use its complex corners unless you truly need them.
Make application guarantees explicit
Do not rely on MQTT as your end-to-end ordering or deduplication mechanism. MQTT provides ordering guarantees in specific situations, but a complete system includes reconnects, retries, multiple publishers, multiple topics, backend processing, and storage. It is safer to make ordering and deduplication explicit.
Add a random UUID-style message ID and a timestamp—or, when strict per-device ordering matters, a monotonic device sequence number—to every published message. The processing or storage layer can then detect duplicates and restore the order that the application actually cares about.
The same principle applies to retained messages, persistent sessions, and Last Will messages: use them deliberately, but do not let the broker become a hidden application-state store unless that is intentional.
If you limit MQTT usage to a small surface, you gain much more freedom in choosing a broker:
- Authenticate and connect.
- Publish status to a device-specific topic such as
dev/{deviceId}/telemetry. - Subscribe to one device-specific topic such as
dev/{deviceId}/commands. - Use QoS 0 or QoS 1 with an explicitly defined acknowledgement boundary.
This also simplifies MQTT scaling. The system becomes closer to a WebSocket-style pipeline. Edge nodes still manage connection counts, slow clients, keepalive, authentication, authorization, and the QoS 1 acknowledgement policy, but they can remain mostly stateless compared with a full MQTT broker.
There is now little or no distributed MQTT state to replicate. You may still need a small bus between nodes—Redis/Valkey Pub/Sub, PostgreSQL LISTEN/NOTIFY, or something similar—to propagate connection and command-routing information. But you eliminate most of what a full-featured broker vendor needs to coordinate.
Telemetry in, commands out
Each MQTT server node—which perhaps can no longer be called a full broker—accepts device connections, decodes MQTT packets, and forwards published telemetry over HTTP or another internal protocol to the backend responsible for processing and storage.
For QoS 1, the acknowledgement point must be a conscious decision. Acknowledging immediately at the edge gives lower latency but means the application must accept a loss window if that edge node fails. Acknowledging only after durable handoff provides a stronger guarantee at the cost of latency and more edge state.
In the other direction, the backend needs a path to the MQTT node connected to a particular device. One option is to send the command to any node and have it forward the request to the correct node.
For low command volume, you can even broadcast commands to the cluster and let the node currently connected to the device deliver them. For larger systems, keep a small connection registry that maps deviceId to the current server node.
This registry is not free, but it is much smaller and easier to reason about than replicated subscriptions, retained messages, queues, and sessions.
Scale MQTT by reducing its responsibilities
For many IoT platforms, the best way to scale MQTT is to do on the server side what MQTT already did for clients: simplify the feature set, and therefore the number of problems the protocol layer must solve.
Do you really need QoS 2, strict in-order delivery, and persistent sessions? I have rarely seen an MQTT application where all of them were mandatory, or where the backend could not provide the required behavior more explicitly.
This is not an argument against full-featured brokers. If your product genuinely needs broker-managed sessions, offline delivery, or complex subscription semantics, buying a mature implementation is usually more sensible than rebuilding those distributed-systems features yourself.
But if your actual contract is telemetry in and commands out, treating the broker as a durable application platform adds complexity and coupling that you may not need.
How would you implement it?
This is something I will probably open-source one day, but the basic idea is simple: take the packet decoder and encoder from an open-source MQTT library, put them on top of a scalable TCP and TLS server using Go, Rust, Java, or whatever fits your team.
The protocol parsing is the straightforward part. Production quality still requires strict packet validation, bounded buffers and queues, authentication and authorization, keepalive enforcement, backpressure, and protection against reconnect storms. But these are problems common to any TCP server. The benefit is that these responsibilities are explicit and limited.
I help companies design and scale IoT platforms, especially around MQTT, device communication, fleet management, and backend architecture.