# MQTT

## Summary


MQTT is a **publish/subscribe** messaging transport protocol. It has low complexity, small code footprint and consumes low network bandwidth for messaging. The lightweight protocol and small packet size support makes it suitable for applications such as Machine to Machine (M2M) and Internet of Things (IoT). 

The protocol runs over network protocols like TCP/IP that provide ordered, lossless, bidirectional connections. MQTT-SN v1.2, MQTT for Sensor Networks (formerly known as MQTT-S), is a version of the protocol targeted for embedded devices on non-TCP/IP networks, such as Zigbee. Connectionless network transports such as User Datagram Protocol (UDP) are not suitable because packets may be lost or arrive out of order. The protocol enables transmit of messages either in 1-to-1 or 1-to-n configuration. The messaging transport is agnostic to the content of the payload.

## Discussion

### What's the expansion for MQTT?

The published OASIS standard does not give any expansion of MQTT. 

MQTT used to mean *MQ Telemetry Transport*, where MQ is sometimes mistaken for Message Queue. In fact, MQ referred to IBM's MQ Series of products. Today, MQTT is not an acronym and it doesn't use traditional message queues. 


### How does the MQTT protocol work?

MQTT uses **Publish/Subscribe** model or pattern. Publishers publish messages. Subscribers receive messages if they have subscribed to the same. However, publishers and subscribers don't communicate to each other directly. There's an intermediate central **Server** or **Broker** that mediates in between. Broker filters all incoming messages and distributes them to subscribers accordingly. Publishers and subscribers are also called **Clients**.

Subscribers subscribe to messages that are of interest to them. Such filtering could be topic-based, content-based, or type-based. Topics could be organized in a hierarchy. 

With the publish/subscribe pattern, the publisher and subscribers don't need to know each other. They are not dependent on each other for initialization and synchronisation. Message delivery is one-to-many and can scale with respect to subscribers without burdening publishers.

It will also be apparent that MQTT operates unlike message queues. Messages cannot get stuck in a queue if no one reads them. In message queues, only one consumer can read the message. Message queues are also defined explicitly but in MQTT topics can be created on the fly. 


### What are the core features of MQTT?

The core features of MQTT are: 

    + Simple to implement: This translates to small code and memory footprint for constrained devices.
    + Lightweight and network-bandwidth efficient: Payload must be transferred without significant overhead.
    + Handle abnormal client disconnection: Using what is called **Last Will and Testament (LWT)**, if a client is not sending anything for some time, broker will automatically publish a message. For example, this can be useful to inform subscribers that an IoT device (publisher) is not reachable.
    + Transport protocol is payload data agnostic: Data can be binary, JSON or any other format.
    + Provides a Quality-of-Service data delivery: QoS can be selected based on the needs of the application.
    + Supports continuous session awareness: Sessions can be continued even across intermittent network connections. This implies clients need not subscribe again when they reconnect to server.
    + **Retained messages**: The last published message (with retained flag set to true) is stored at the broker so that new subscribers can immediately obtain *last known good value* rather than wait for the next update from publisher.

### Why can't I use HTTP instead of MQTT?

Altough popular on the web, HTTP is not suitable for small data packets typical in IoT. HTTP is document-centric whereas MQTT is data-centric. HTTP is a complex protocol. It's textual and therefore parsing HTTP is not trivial. MQTT is binary and encoding/decoding MQTT packets is a lot easier. Because of its publish-subscribe model, data distribution is one-to-many in MQTT whereas in HTTP it's limited to one-to-one. 

An MQTT PUBLISH message header is 2-4 bytes; CONNECT message header is 14 bytes; HTTP header can be up to 1KB. A C implementation of an MQTT client can be as small as 30KB. 

MQTT uses less battery power, can send more messages per hour and send them more reliably than HTTPS. MQTT uses more battery power to initiate a connection but this quickly compensated by gains as the connection stays open longer. 

In fact, there are alternatives to MQTT that may be more suitable candidates for comparison than HTTP: AMQP, CoAP, XMPP, DDS, STOMP. 


### What was MQTT designed for and where is it popularly used today?

IBM originally designed MQTT protocol for communication between oil pipeline stations over low bandwidth satellite links. It was later used for applications such as home automation and message broadcasting to ferries. Other areas where it's used include connected cars, asset tracking, smart metering, process control in manufacturing. 

MQTT gained traction in IoT space due to its lightweight protocol. One popular example of its usage is by Facebook for its Facebook Messenger application. 


### What are the control packets supported in MQTT?

Control packets can be summarized in the following groups: 

    + **Connection Management**: CONNECT, CONNACK, DISCONNECT, PINGREQ, PINGRESP
    + **Subscription Management**: SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK
    + **Message Delivery**: PUBLISH, PUBACK, PUBREC, PUBREL, PUBCOMP
    + **Authentication**: AUTHAUTH packet was introduced in MQTT version 5.0. PINGREQ is sent from client to server to indicate that client is alive though not sending any other control packet. 


### What sort of Quality of Service (QoS) does MQTT offer?

MQTT protocol defines three levels of QoS for message delivery: 

    + **QoS 0**: At most once delivery. Receiver does not sent a response. Sender does not retry and hence deletes the message locally as soon as it's sent. Message arrives at the receiver once or not at all.
    + **QoS 1**: At least once delivery. Receiver acknowledges the PUBLISH packet using the PUBACK packet.
    + **QoS 2**: Exactly once delivery. Neither loss nor packet duplication is acceptable. This QoS level adds protocol overhead and specifies many rules for both sender and receiver. Packets PUBREC, PUBREL and PUBCOMP are involved instead of PUBACK used in QoS 1.Sender will retry on unacknowledged packets only if client reconnects without a clean start and session is present. 


### Does MQTT support security?

MQTT is a transport protocol that concerns message transmission. So, security features such as privacy of data, authentication and authorization of users, etc. have to be taken care by the developer. MQTT specification provides guidance about possible implementations such as TLS. Security-related aspects that need to be considered could be related to geography, industry or regulations. 

TLS/SSL security and payload encryption can be used. The IANA has assigned port 8883 for MQTT over SSL, or secure MQTT over TCP or UDP. Incidentally, port 1883 is for MQTT over TCP or UDP We should note that SSL is not exactly lightweight and adds significant overhead. 

Since Version 3.1 of MQTT protocol, there are two provisions: 

    + Add username and password in the CONNECT packet
    + Include return codes in CONNACK packet to report security issuesIn Version 5.0, MQTT adds the AUTH control packet. This can be used to implement complex authentication methods such as SCRAM, Kerberos or OAuth. 

There is a supplemental publication that focuses on MQTT protocol's integration within the NIST Framework for Improving Critical Infrastructure Cybersecurity, simply called *Cybersecurity Framework*. 


### What are the limiting factors of MQTT to consider while using it?

Here are some limiting factors of MQTT:

    + The structure of the published data is not known to the subscriber. This information should be shared or known beforehand. When topic-based filtering is used, publisher and subscribers must know the right topics to use.
    + The delivery of a message is not confirmed as there is no automatic confirmation from the recipient. Moreover, the publisher also does not know if there are any subscribers.
    + MQTT uses TCP, which needs complex handshaking and is not easy on memory and power requirements. Standard states that TCP is a suitable transport and UDP is not. MQTT can also use TLS or WebSocket.
    + Broker is centralized. This is not only a single point of failure but also can limit scalability. However, real-world implementations overcome this by managing a **distributed broker cluster** that clients see as a single logical broker. Examples of such cluster implementations are HiveMQ, VerneMQ, and EMQ.
    + MQTT has no built-in security. Developers need to implement security on their own.

### What software implementations of MQTT are available?

Several MQTT client libraries are available based on different languages and for supporting varied devices. Broker or server implementations and tools are also available from different vendors as noted on MQTT GitHub Wiki. 

Eclipse Mosquitto, a MQTT broker implementation, is about 120kB and requires 3MB RAM for 1000 clients connected. 

Eclipse Paho client offers a number of features: automatic reconnect, offline buffering, message persistence, WebSocket or TCP support, blocking or non-blocking API, and high availability. For embedded devices of limited resources, Paho offers an ANSI standard C compliant library. 

RabbitMQ server uses AMQP instead of MQTT but it offers a plugin for MQTT, making AMQP interoperable with MQTT. 

## Milestones

1999

MQTT is developed by Andy Stanford-Clark (IBM) and Arlen Nipper (Arcom, later Eurotech). It's used to send sensor data gathered from an oil pipeline. Data is sent to a SCADA host system via low-bandwidth links. 

Aug  
2010

IBM releases MQTT V3.1 and makes it available to the public. They define MQTT as,

> A lightweight broker-based publish/subscribe messaging protocol designed to be open, simple, lightweight and easy to implement.

Aug  
2011

Facebook announces in a blog post that their *Facebook Messenger* is using MQTT. Without it, they experienced long latency in the order of seconds. With MQTT, it's down to hundreds of milliseconds. 

Nov  
2011

IBM and Eurotech donate MQTT code to the Eclipse Foundation. A month later, Eclipse Foundation releases open source Java and C client code for MQTT. Client-side MQTT code is under the project named **Eclipse Paho**. Today client code from Paho is available in a number of languages. 

Oct  
2014

MQTT Version 3.1.1 is officially approved as an OASIS Standard. In June 2016, this is published by ISO as ISO/IEC 20922:2016. 

Feb  
2015

As MQTT broker implementation, open source **Eclipse Mosquitto 1.4** is released, the first release by the Eclipse Foundation. Version 1.0 of Mosquitto was available as early as 2012. The growth of MQTT can be attributed to the open source nature of Eclipse Paho and Mosquitto. 

Apr  
2016

Created four years earlier by Feng Lee, version 1.0 of **EMQ (Erlang MQTT Broker)** is released. Written in Erlang/OTP, EMQ is open source, distributed, and massively scalable. Messages are routed through clusters that apply *Topic Trie Match* and *Routing Table Lookup*. 

May  
2018

OASIS releases **MQTT Version 5.0**. It supercedes the previous standard. Eclipse Paho 1.4.0 is expected to support this by June 2018 in C and Java. V5.0 support from Eclipse Mosquitto is expected by August 2018. It's expected that the latest version will go into production systems by Q4 2018. Main changes due to V5.0 are summarized by a HiveMQ blog post.