As it happens to me from time to time, today I ended up sleepless1 at 2 am with nothing to do. So after contemplating life, the universe and everything for a while I decided to put my time to a good use by playing with MQTT.
In case you haven’t heard it before, the name MQTT stands for MQ Telemetry Transport, and it is a publish/subscribe protocol aimed at sensor networks and embedded devices. MQTT is an application (layer 7) protocol that uses TCP or other reliable transports and is standardized at the wire format level (in this aspect it is similar to AMQP, where one client library can connect to any AMQP broker, instead of requiring vendor-specific clients like JMS or ODBC/JDBC).
The operations implemented by MQTT are the bare minimum for this kind of service:
- Connect / Disconnect
- Subscribe to a topic
- Unsubscribe from a topic
- Publish a message to a topic
There is no concept of “Queue” in MQTT. All messages sent to a topic are dispatched to all subscribers, which is very confusing since the two first letters of the protocol’s name are precisely “MQ”2. MQTT does not impose any particular format on the message data, so it can handle JSON, XML and binary formats equally well. The simplicity plus the lack of restrictions make it ideally suited for use in embedded systems like IoT devices.
The simplest way to try MQTT these days seems to be to use Mosquitto, an Eclipse.org project that implements a full MQTT server. Luckily for me several people have done the hard work of packaging Mosquitto as a Docker image, so the only thing I had to do was pull the image and wire up some folders, and I was up and running. This is the command I used to do that:
docker run -it -p [server_ip]:[server_port]:1883 \ -p [server_ip]:[server_port]:9001 \ -v /storage/docker/work/mqtt/config:/mosquitto/config \ -v /storage/docker/work/mqtt/data:/mosquitto/data \ -v /storage/docker/work/mqtt/log:/mosquitto/log \ --restart=always --name mosquitto eclipse-mosquitto
By default the image will listen on ports 1883 (MQTT) and 9001 (MQTT over Websocket). I had to remap those ports because I already have other things listening there. I also mapped host directories to the configuration, data and log directories in the container, to make them persistent3.
Playing with the Python client
Connecting to an MQTT server is surprisingly straightforward using the Paho client for Python, and the no-nonsense design of the API makes it very easy to work with topics and messages. With just a handful of lines of code I was able to publish and consume simple messages:
import paho.mqtt.client as paho import threading import time # ------------ SUBSCRIBER&amp;amp;nbsp;CODE ----------------- # Callback function for every received message def processMessage(client, userdata, msg): print("Message from " + msg.topic + ": " + str(msg.payload)) # Create receiver client = paho.Client() client.connect("[SERVER_IP]", [SERVER_PORT], 60) client.on_message = processMessage client.subscribe("/test/topic") # Start receiver on a separate thread loopThread = threading.Thread(target=client.loop_forever) loopThread.start(); # ------------ PUBLISHER&amp;amp;nbsp;CODE ----------------- # Create publisher publisher = paho.Client() publisher.connect("[SERVER_IP]", [SERVER_PORT], 60) # Publish 3 messages for i in range(3): publisher.publish("/test/topic", "Message number " + str(i)) time.sleep(0.5) # Clean up publisher.disconnect() client.disconnect() print("End")
Since by default MQTT doesn’t use SSL I was able to capture and examine the contents of the MQTT conversation between clients and broker. I was surprised by how compact the protocol actually is. The message headers are almost nonexistent and the data is packed so as to fit as much as possible in a single TCP segment or Link Layer frame.
Here are some Wireshark captures of both the publisher and the subscriber. I annotated the TCP stream with the different segments exchanged between clients and server.
First, here is the exchange when the publisher connects to the broker and pushes three messages (client to server messages are in red, where server to client messages are in blue):
And here is the client side of it, with the three messages being pushed to the client by the MQTT broker:
1 Go ahead, make the “Sleepless in Seattle” joke. You know you want to.
2 Apparently MQTT was once part of IBM’s MQ series of products. Hence the prefix.
3 I know I could have used Docker volumes, but this works better with my backup strategy.