Home Automation With MQTT and Home Assistant — TechTalk Gist
By: Anuradha Wickramarachchi
Tags: Software Development, Engineering, IoT, Home Automation, Home Assistant

Practical Considerations

Quality of service

QoS 0

Fire and forget approach. The MQTT broker will not care if the message is actually received or not by all the subscribers.

Eg:

DHT (digital humidity temperature) sensors that transmit climate information every 10s

Presence sensors (PIR) that check presence every 5 seconds and transmit the status

Suitable for scenarios where missing a single message or point of data is not significant. Or power is constrained and network overhead is preferred at a minimum.

QoS 1

The messages are guaranteed to be sent at least once. This implies that messages may be sent several times.

Suitable when you expect all the readings to be received without loss. This also assumes that in case, duplicate data can be removed by other means (by using timestamps, etc).

Weather monitoring, where time-series information is vital. Packet losses can occur due to network issues. Hence, application layer delivery Acks needed.

QoS 2

Delivery is guaranteed to be just once. No duplicates or lost messages.

Parking lots need to monitor the number of vehicles entered to show customers about available parking slots. Can have parallel vehicle entry from multiple locations. No assumptions about time are possible.

Deep ocean monitoring systems where timestamps might not be reliable. In such cases, messages must be received just as many times as the events occur.

In QoS 1 and 2. Messages are queued by the clients for sending, often a bad choice for memory-constrained devices such as Arduinos or ESPs.

Network usage and power consumption

Network connections can be reliable yet, the IoT devices can be in states that make them incapable of being in a stable connection. For example, device meshes (like ZigBee or ESP) can change topology with movement. Some devices can restart periodically due to OTA (over the air updates). Or the user might decide to plug out the smart bulb from the Dining room and move it to the kitchen!.

In such cases, there must be a means by which the communicated messages can be made persistent. In other words, if a light is switched on, and mains go off for some reason, the bulb should still be able to know that it should come on when it comes online again. We achieve this using the retain flag.

Turn the retain flag on, the broker will keep the last message with the retain flag, indefinitely. This can be cleared using the — remove-retained argument.

Real Example

The flow of a typical MQTT device is as follows.

Click — smart home app button for light L1.

Send the following payload to MQTT broker under topic arduino-esp/light/L1/set

{

“state”: “ON”,

“brightness”: 255

}

Now the message is sent to the subscribing bulb (or controller) under the same topic with the same payload.

Now the device turns the bulb relay to a closed position (bulb turns on) and sends the following payload to topic arduino-esp/light/L1/state.

{

“state”: “ON”,

“brightness”: 255

}

When this is received by the smart home app, it will turn the colour of the button to yellow and brightness level to the set amount.

Why is this done so?

There could be several users controlling the brightness/light. At the same time, there could be failures in this operation due to the many issues we discussed. Therefore, the bulb controller itself must share its status to the broker to be sure.

One way of doing this is the publishing state every 10 or 20 seconds. And to be more responsive, an immediate message could be sent when the state is changed. Periodic messages could be expensive and can avoid devices going to deep sleep. We can use only the immediate acknowledgements for updates. But for that, we need a QoS above 0. Otherwise, the controlling app may be unresponsive. This comes to a trade-off between how active the device is, and its capability to be in deep sleep with WiFi or any other kind of service.

Practical Example with Home Assistant — Home Assistant IO

Screenshot from Reddit

Steps you need to follow before you can start implementing this example!

Install Home Assistant

Install MQTT broker plugin. Link: MQTT

ESP32 with Arduino IDE and PubSub library (MQTT library)

ESP32 MQTT Publish-Subscribe with Arduino IDE (Image source+Tutorials)

Try the following!

Image Source: Link

More elegant approach with Home Assistant! You have a nice UI and Mobile Apps. It is open source too!

Home Automation with Home Assistant and MQTT

Home Assistant can discover MQTT enabled devices automatically and register in the application. MQTT Discovery

Discovery message template for a light

{

“~”: “homeassistant/light/kitchen”,

“name”: “Kitchen”,

“unique_id”: “kitchen_light”,

“cmd_t”: “~/set”,

“stat_t”: “~/state”,

“schema”: “json”,

“brightness”: true

}

“~” is a placeholder to be placed at its occurrence in values of this JSON object.

This message has to be sent to the discovery topic of Home Assistant, which is;

homeassistant/light/kitchen/config

Everything in the JSON object is configurable by the device itself. The discovery topic must be strictly followed. Otherwise, some hacks are needed.

Steps:

The device (light) will publish the above JSON to the discovery topic.

Home Assistant will add a bulb controller to its UI

Rename it as you wish, say “Dining room light”

The above bulb has brightness control too and it runs with a JSON payload.

When the user clicks the button to switch the bulb, the home assistant will publish a payload to topic “~/set” (determined by cmd_t which expands to command_topic). “~” will be replaced by “homeassistant/light/kitchen”.

Then light will publish its new state to “~/state” which is determined by the “stat_t” which expands to “state_topic”.

JSON keys are shortened as some devices operate with limited memory and string processing power. It also saves network packet usage, needed bandwidth, RAM usage and internal buffer sizes. Some networks have limited packet sizes or MTUs (Max Transmit Units).

Advanced Automations with NodeRED and Home Assistant

Install: Home Assistant Community Add-on: Node-RED

NodeRED is a virtual wiring tool for MQTT (or any other) messages. You can transform one MQTT message to another or several others. An example as follows.

PIR sensors detect home as empty for 5 hours.

Turn off all electrical appliances except for vitals (Fridge, Cameras, etc) (state: LOCK)

Now the gate sensor detects an open message from the user

Exit the home LOCK state as follows

Turn on the Garage light

Turn on AC and cool the house

Turn the geezer on

Open blinders of the bedroom to get some sunlight

Play some music

If needed the exit LOCK routine can have a different path, say if someone returns home from the front door without a car. All this is 100% compliant with Home Assistant and can be done with Arduino IDE and ESP8266 or ESP32 MCUs.

Why should one do this? (Just for knowledge)

Privacy is a massive concern in the modern world. Ubiquiti like companies provide cheap wifi solutions to cover entire houses. Therefore, the limitations of devices such as Philips Hue, Google Home or Alexa will no longer be a problem. You anyway got WiFi. ZigBee devices are expensive and limited in variety. (Philips Hue Bulb: Rs 5000+ per bulb). It is a closed standard. However, ESP32 is less than Rs 2000 (even cheaper on Ali) and can automate a Huge Chandelier with little to no effort. Zero Access to the Internet needed. The home assistant was released as an OS to run on Raspberry. The open-source mobile app (Android and iOS) can run within the local network. The quality and stability of these applications are far better than Google and Alexa apps.

Limitations of MQTT

It is a SPOF (Single Point of Failure).

MQTT needs to be run with a watchdog (to restart if it crashes).

Poorly configured MQTT can be vulnerable to intruders.

MQTT brokers need to be online always. This is not the case in BLE beaconing devices. Hence needs to be running in a mains powered device.

Routing is IP based. This essentially means that only a limited number of devices can connect in a given subnet. Needs a DHCP server to assign IPs. This is not the case in ZigBee or ESP-Mesh.

Considered not suitable for IIoT (Industrial IoT) applications. One major reason is the QoS support. QoS is honoured only when both subscribers and publishers are at the same level of QoS. In MQTT the same topic can have several subscribers with different QoS levels. This makes it difficult for the publisher to conclude delivery and the overhead is high for communication.

This is an application layer messaging protocol. This makes it difficult to route data in a customised manner. For example, a faraway device without a wifi connection cannot be reached by asking a near-by device to relay data. However, there are network layer messaging applications that support WiFi mesh networks that overcome this. (Painless Mesh Arduino)

Ways to overcome these limitations

Securing with SSL (MQTTS)

Use bridging devices that connect to MQTT brokers.

The bridging device can make a mesh of other devices and relay messages between external devices and the mesh. Philips Hue bridge does something similar by connecting the IP network and ZigBee bulbs. This can be achieved fully using owagner/hue2mqtt: Gateway between a Philips Hue bridge and MQTT and official Philips Java API library.

This is indeed a bulky article. However, I believe this might as well be complete for someone whos looking out to get themselves into home automation with Home Assistant.