All of us tinkermen eventually end up working on a home monitoring/automation system sooner or later. And that's for me the big background project at the moment.

I have already some of the pieces in several stages of readiness but I was lacking an overall view of the system as a whole. My initial approach was to store everything in a MySQL database and develop a web application to graph the time series values. The plan was to deploy a Wireless Sensor Network with an undefined number of sensors transmitting their information over Zigbee. An Xbee coordinator, laying on a Sparkfun Xbee Explorer USB pluged to the server, will receive the information and a python script will decode the packages and store the message in the database.

image

I was planning to design the database based on the physical building blocks (device, sensor, value) and then provide a REST API (I am a REST fanboy) to consume the data.

Then I thought about storing the information from other services I'm using like Efergy Engage (I will talk about that in another post). Unfortunately Efergy does not provide an API for their products. I've asked them a couple of times and they said that that was not priority but I think that will go against what they feel their business is so I don't think they will ever open it. Anyway it was not hard to look at the requests they are doing at their online graphics app and I wrote a little script that logs in and makes a single request to read the data for the last 24 hours and stores it in the database.

And then I thought that it won't harm start graphing all that data in Cosm.com or an RRD tool. After all it will take me some time to put the UI I wanted in place. But then where should I plug that? Do I have to create a consumer for the REST API that just pushes data to Cosm.com with a cron every minute? Or should the Xbee driver and the efergy script dump the information to MySQL and to Cosm.com? In the second option MySQL is a simple data backup storage and there are a plethora of drivers performing various tasks. Moreover, if I want more consumers in the future I will have to add code to all those drivers… The first option seems more mantainable and decouples the several components of the network. MySQL becomes the center of the network and clients will pull data from it on regular basis using the REST API. No real-time data here, thou. And each component will have to support my custom API.

And then one day, reading the news feeds in my phone on my way to work, I stumbled on a post in Robbert Henkkens blog (on of my favourites) about publishing sensor data using MQTT. MQTT (Message Queue Telemetry Transport) is a lightweight publish/subscribe messaging protocol aimed to machine-to-machine communication. It was created at IBM in 1999 but it has an open license and it's royalty free. The infrastructure requires of a broker that receives and dispatches messages and a number of clients. Each message has a topic and a content (and a header that can be as short as two bytes). Clients can publish or subscribe to any number of topics (you can define access control rules) and specify I message to post on a topic when they disconnect. There are 3 QoS levels and subscribers can request messages they lost while disconnected.

It took me a few days to realize that that was the solution I needed to decouple the components of my network. I could work on each component of my network in a isolated way: a publisher or consumer of MQTT messages, no matter what or who is on the other side of the communication. There are a number of MQTT brokers available, including Mosquitto, an open-source project that provides a daemon broker, C library, python library, and command line utilities for publishing and subscribing which is perfect for me. Besides I would be using a (de facto) standard protocol, no more custom APIs or other solutions. Even Cosm.com supports MQTT (although it does not support the full specification )!

This is how my MQTT-centered WSN might look like:

image

I am already adapting what I have to this pattern. There are still some issues I have to address, like how to store the sensor data (MySQL is not the best option for sure) and define a topic naming convention or how to publish data from dumb sensors under the right topic. Robert suggests a couple of solutions for this last problem. I think the republishing topics option is the one I like the most but this is a topic (ehem) for another post.

"Home monitoring system" was first posted on 11 November 2012 by Xose Pérez on tinkerman.cat under Projects and tagged home network, mqtt, wsn.