Optimizing MQTT Protocol IoT The Most Efficient Data Transfer Protocol

IoT workers must have many headaches because the system is sluggish, sometimes data is received and sometimes it is not, especially when having to handle thousands of sensor devices. The problem is often not in the hardware but in the communication protocol. I once struggled with a smart home project where the device responded slowly because of mistakes in protocol selection and configuration. After many times of "paying the price", I realized that MQTT protocol IoT data transfer protocol is the real thing, but I have to know how to optimize it. This article is Pham Hai's experience to help you turn your IoT system into the fastest, lightest and most stable.

Some "small but powerful" MQTT optimization tricks to help your IoT system run smoothly

Optimizing MQTT in IoT systems requires fine-tuning QoS, payload compression, standard topic design and reasonable Keep-Alive configuration to reduce the load on the network and devices.

At Pham Hai, we always emphasize that using MQTT is not difficult, but using it smoothly when scaling to tens of thousands of devices is another story. Below are practical combat techniques you should apply immediately.

Choose the appropriate QoS (Quality of Service) level: Don't abuse QoS 2 when not necessary

MQTT QoS levels include 0, 1, and 2; Choosing the right level balances reliability and bandwidth.

Many beginners often set the default QoS 2 (Exactly once) to be sure. But in reality, QoS 2 takes up to 4 hand-shake steps, increasing latency and consuming terrible bandwidth. My experience is to only use QoS 2 for extremely important control commands (like unlocking the door, recharging).

As for data from temperature and humidity sensors sent continuously every 5 seconds, QoS 0 (At most once) or QoS 1 (At least once) is more than enough. This helps take full advantage of low bandwidth for IoT devices, avoiding local network congestion.

Payload optimization: Data format (JSON, Protobuf) and data compression to reduce bandwidth

Instead of sending bulky JSON, using Protobuf or MessagePack helps reduce the payload size to a minimum during data transfer.

The JSON format is very readable for humans, but it contains too many extra characters (brackets, commas, repeated field names). When transmitting data over mobile networks (3G/LTE), each byte is charged.

• Sử dụng Binary format: Tụi mình thường chuyển sang dùng Protocol Buffers (Protobuf) hoặc CBOR. Kích thước gói tin có thể giảm từ 30-50% so với JSON.
• Nén dữ liệu: Nếu gói tin lớn, hãy cân nhắc nén GZIP trước khi gửi.
• Mã hóa payload: Kết hợp với payload encryption (mã hóa nội dung) ở cấp độ ứng dụng để gói tin vừa siêu nhẹ vừa an toàn tuyệt đối.

Make smart use of Retained Messages and Last Will and Testament (LWT) mechanics

Retained Messages saves the last message on the Broker, while LWT notifies the device's sudden disconnection status to increase reliability.

Bạn có bao giờ bực mình khi mở app lên phải chờ thiết bị gửi dữ liệu mới biết trạng thái hiện tại? Hãy dùng cờ Retained = True khi Publish. MQTT Broker sẽ lưu lại tin nhắn cuối cùng này và gửi ngay cho bất kỳ Subscriber nào mới kết nối.

And LWT is the "will" of the device. When a network node suddenly loses power, Broker will automatically send an LWT message to let the entire system know that node is offline. This feature greatly contributes to the reliability of the monitoring system.

Scientific Topic structure design: Easy to manage, easy to expand and optimize performance for Broker

Topic structure should go from overview to detail, avoid using wildcards (#, +) at the root to not overload Broker.

Đừng đặt Topic kiểu tùy hứng như phongkhach/nhietdo. Hãy chuẩn hóa theo dạng phân cấp logic.

Tuyệt đối tránh việc Subscribe vào gốc # (ví dụ đăng ký nhận mọi thứ trên hệ thống), vì nó ép Broker phải định tuyến hàng triệu tin nhắn, gây nghẽn cổ chai ngay lập tức.

Reasonable Keep-Alive configuration to balance between maintaining connection and saving device power

Keep-Alive periodically sends ping packets to keep the TCP connection open; Setting the time too short will waste the device's battery.

The default Keep-Alive is usually 60 seconds. This means that every 60 seconds, the device wakes up and sends a PINGREQ packet. With battery-powered microcontrollers (like ESP32, STM32), this drains the battery extremely quickly.

If the network is stable, you can push Keep-Alive to 300 seconds or 600 seconds. Mastering how to deploy MQTT for IoT devices at the Keep-Alive tuning stage is the secret to helping our smart agriculture projects run on battery for years instead of months.

Understand exactly what MQTT is in 5 minutes

MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol, operating according to the Pub/Sub model, designed specifically for low-bandwidth and unstable networks.

To answer the question What is MQTT in IoT, just imagine it as a super-speed post office. Although the MQTT protocol's history began in 1999 by IBM for oil pipeline monitoring, as of 2026, it remains the gold standard. If you are a complete beginner, skimming through IoT Internet of Things for Beginners will give you an overview before diving into this protocol.

MQTT is not Client-Server, it is Publish/Subscribe (Pub/Sub)

Unlike HTTP calling APIs directly, MQTT completely separates the sender and receiver through the intermediate Publish/Subscribe model.

In the traditional client/server architecture, the Client must know where the Server is to send a Request and wait for a Response. But MQTT uses the Publish/Subscribe model. The sending device (Publisher) and the receiving device (Subscriber) do not know each other. They communicate through an intermediary. This asynchronousmessaging protocol mechanism is ideal for Machine-to-Machine (M2M)networks.

The main components cannot be ignored: Client, Broker, Topic

The main components of the MQTT protocol include Client (device/app), Broker (central server) and Topic (message classification topic).

Thành phần chính của giao thức MQTT xoay quanh 3 trụ cột:

• MQTT Client: Có thể là bất cứ thứ gì, từ một con chip ESP8266 bé xíu đến một server backend hoành tráng. Nó có thể đóng vai trò Publisher (người gửi) hoặc Subscriber (người nhận).
• MQTT Broker: Trái tim của hệ thống, nhận tin nhắn và phân phối chúng đến đúng nơi.
• Topic: Giống như "địa chỉ nhà" hoặc "kênh phát thanh" để Broker biết phải giao tin nhắn cho ai.

MQTT activity flow: Publisher sends news, Broker distributes, Subscriber receives news

The way the MQTT protocol works starts with the Client connecting TCP/IP to the Broker, then Publish or Subscribe to Topics.

Cách hoạt động của giao thức MQTT diễn ra rất mượt mà: Đầu tiên, Client mở một kết nối TCP/IP tới Broker. Sau đó, một cảm biến nhiệt độ (Publisher) gửi bản tin "30 độ C" vào topic home/temp. Broker nhận được, tra cứu xem ai đang đăng ký (Subscribe) topic home/temp này. Cuối cùng, Broker đẩy "30 độ C" đến app trên điện thoại của bạn (Subscriber). Không có polling liên tục, không lãng phí tài nguyên.

Why is MQTT the "star" in the IoT/M2M data transmission village?

The advantages of MQTT in IoT data transmission lie in its byte-level compactness, microcontroller resource savings, and unlimited scalability.

When looking for an effective IoT data transfer solution, MQTT is always at the top of the list. For example, in web development, you often hear about REST. You can learn more What is REST API and RESTful standard design to see the difference. REST API is great for the web, but when brought down to embedded systems, the HTTP header is too cumbersome, wasting resources uselessly compared to MQTT.

Unbelievably light: MQTT header is extremely small, saving maximum bandwidth for unstable networks such as 2G, 3G, LoRaWAN

MQTT has a header of only 2 bytes, helping it run smoothly on low-resource devices and weak mobile networks.

Did you know that the header of an HTTP packet can be up to hundreds of bytes? Meanwhile, the fixed part in the MQTT header is only 2 bytes. For systems using 2G/3G or NB-IoT sims, this overhead reduction saves tens of MB of capacity per month for each device, significantly reducing operating costs.

Reliable: With 3 QoS levels, MQTT ensures message sending and receiving even in intermittent network conditions

Thanks to the automatic storage and resending mechanism of QoS 1 and 2, MQTT does not lose data when the network connection is temporarily dropped.

Driving into a tunnel, losing 3G signal is normal. But with QoS 1 or 2, your messages are not thrown away. Broker or Client will save it in cache and automatically send it back as soon as there is signal. This is the advantage of MQTT in IoT data transmission that is completely superior to using pure UDP.

Flexible and easy to expand: Pub/Sub model allows adding or removing devices (Clients) without affecting the entire system

The separation between Publisher and Subscriber helps the system easily expand from 10 devices to 10,000 devices without having to rebuild and rebuild.

Let's say you want to add a temperature display to your old system. You do not need to edit the sensor code. You just need to code that screen to connect to Broker and Subscribe to the correct Topic and you're done. Industrial MQTT applications (such as SCADA, smart factories) absolutely love this because of its excellent plug-and-play.

Security for MQTT - It cannot be overlooked

MQTT protocol security requires applying TLS/SSL transmission encryption, device authentication, and strict ACL authorization.

By default, MQTT sends data in plain-text (unencrypted). If someone intercepts the packet, they will see all your system parameters. MQTT security is vital. At Pham Hai, we always ask customers to apply the following 3 layers of security.

Encrypt transmission with TLS/SSL: Required to prevent data eavesdropping

Using TLS/SSL encryption (port 8883) helps protect data from Man-in-the-Middle attacks on the internet environment.

Never use port 1883 in production environments. Always configure Broker to run on port 8883 with TLS/SSL encryption. It will create a safe tunnel. Even if hackers can sniff the network, they will only see a jumble of characters. In return, the device will use a little more RAM to process the certificate.

Client authentication: Username/Password and Client Certificates to ensure the correct device is connected

Device authentication via Username/Password or X.509 certificate prevents unauthorized connections to the Broker system.

device authentication is the second layer of defense. Broker needs to know "Who are you?". The basic way is to give each device a Username/Password pair. More professionally, especially when connecting to IoT cloud platforms (such as AWS IoT, Azure IoT), you are required to use Client Certificates (X.509 digital certificates) for two-way authentication (Mutual TLS).

Access permissions (ACLs): Set Publish/Subscribe permissions on each Topic for each Client

Access control list (ACL) limits the rights of each Client, not allowing them to read/write data of other devices.

ACL (Access Control List) là chốt chặn cuối cùng. Cảm biến nhiệt độ phòng khách thì chỉ được phép Publish lên topic phongkhach/nhietdo, tuyệt đối không được quyền Subscribe vào topic cuachinh/khoa để ra lệnh mở cửa. Việc phân quyền bảo mật giao thức MQTT chặt chẽ giúp khoanh vùng rủi ro nếu một thiết bị không may bị hack.

Choosing to deposit gold: Which MQTT Broker should I use?

The choice of popular MQTT broker depends on the project size, from the lightweight Mosquitto to the powerful EMQX for enterprise clusters.

Introducing popular Brokers: Mosquitto, HiveMQ, EMQX and the advantages and disadvantages of each type

Mosquitto is suitable for small projects, while HiveMQ and EMQX provide powerful cluster capabilities for systems with millions of connections.

Among the popular MQTT broker, there are 3 names you must know:

• Mosquitto: Huyền thoại viết bằng C, siêu nhẹ, chạy mượt trên cả Raspberry Pi. Hợp cho dự án cá nhân hoặc gateway nhỏ.
• EMQX: Viết bằng Erlang, sinh ra để chịu tải khủng khiếp. Hỗ trợ cluster cực kỳ mạnh mẽ cho hàng triệu kết nối.
• HiveMQ: Viết bằng Java, enterprise-grade, giao diện quản lý cực kỳ trực quan và hỗ trợ plugin tốt.

To program on the device, you can use open source MQTT client libraries like Paho (supports C, Python, Java...) which are very stable.

Broker in the cloud or self-hosted (on-premise): When to choose which solution?

Self-hosting Broker helps control data 100%, while using Cloud Broker saves operating costs and easily expands scale.

If your project requires strict internal data security (such as factory, medical), host Broker yourself on a private server (on-premise). On the contrary, if you make IoT products sold internationally, use Managed MQTT Broker services on Cloud. They take care of all the infrastructure, you just need to focus on making the app.

MQTT versions and when to use MQTT over WebSockets

Understanding the differences between MQTT and WebSockets versions helps you better architect cross-platform communication systems.

Quick comparison of MQTT v3.1.1 and v5.0: What is worth upgrading to v5.0?

MQTT v5.0 adds many advanced features such as Shared Subscriptions, Message Expiry and more detailed error reporting compared to v3.1.1.

When comparing MQTT versions, v3.1.1 has been the industry standard for many years. But MQTT v5.0 was born and brought with it valuable improvements. It supports "Shared Subscriptions" (allowing multiple workers to process messages from one topic to balance the load), "Message Expiry Interval" and more detailed error return codes. At Pham Hai, we started migrating new projects to v5.0 to take full advantage of these features.

What is MQTT over WebSockets? Why is it needed for real-time web applications?

MQTT over WebSockets allows web browsers to connect directly to Broker, ideal for real-time monitoring dashboards.

Web browsers (Chrome, Safari) cannot open plain TCP connections. So what is MQTT over WebSockets? It is a wrapper. MQTT packets are encapsulated into WebSockets frames (usually running port 8083 or 8084). Thanks to that, you can write a dashboard website using ReactJS, connect directly to Broker and draw real-time dancing temperature charts without going through a backend server.

In short, MQTT protocol IoT data transfer protocol is an extremely powerful and effective tool, but that power is only maximized when you really understand and know how to optimize it. Don't just use MQTT at a basic level, apply optimal techniques from choosing QoS, designing Topic to security. That's the way to build a professional, stable and ready for future expansion IoT system. My experience is: a good system starts from a well-configured protocol.

What difficulties are you facing when implementing MQTT? Or do you have any good optimization tips you want to share? Leave a comment below, me and everyone can discuss!

Lưu ý: Các thông tin trong bài viết này chỉ mang tính chất tham khảo. Để có giải pháp tốt nhất, vui lòng liên hệ trực tiếp với chúng tôi để được tư vấn cụ thể dựa trên nhu cầu thực tế của bạn.