XBee to MQTT gateway

So far I’ve posted about hardware and theoretical stuff like network architecture or naming conventions. I think it’s time to move to the software side.

The core of the sensor network I’m deploying at home is the Mosquitto broker that implements MQTT protocol. It manages the messaging queue, listening to messages posted by publishers and notifying the subscribers.

I’ve been working in parallel to have at least some pieces in place to get and store information from the pulse counter sensor. These are an XBee to MQTT gateway and a couple of consumers: one storing info into a MySQL database and another one pushing it to cosm.com.

I want to introduce you the first piece: the xbee2mqtt daemon. It’s already available on github under GPL v3 license. It publishes the messages received by an XBee radio to the Mosquitto instance. The radio must have a Coordinator API firmware loaded. Right now the gateway understands frame IDs 0x90 and 0x92 which account for “Zigbee received packet” (i.e. data sent through the serial link of the transmitting radio) and “Zigbee IO data sample” (that’s an automatic sample of selected analog and/or digitals pins on the transmitting radio).

I’ve tailored the daemon to my needs, but trying to be as generic as possible. The design is based on small components:

  • The “xbee” component takes care of the connection to the radio and the packet parsing.
  • The “router” maps xbee addresses/ports to MQTT topics.
  • The “processor” pre-processes values before publishing them.
  • The “mqtt” component takes care of the message publishing.
  • And the XBee2MQTT class (which extends Sander Merechal’s fabulous daemon class) glues everything together.

You can read the code to get a full insight of what it does but I’d like to explain here some decisions I’ve taken.

I’ve abstracted the message source to an address and a port. The address is the 8 bytes physical serial number of the radio (SH and SL) and the port is the pin (adc0, adc1,… dio1, dio2,…). 0x90 packets are mapped to virtual ports. The sender can define the name of the virtual port (like “battery:4460\n”) or otherwise a generic name will be used (“serial” by default).

The routing is a basic functionality. As I already explained in my previous post about topic naming conventions I think the mapping should be done in the gateway because no other component should have to know about the physical structure of the wireless (XBee) network. So the xbee2mqtt daemon maps all the messages to MQTT topics with semantic meaning. You can also allow default topic mapping which will publish any message received by an undefined address/port combination to a topic like /raw/xbee/<address>/<port>.

The processor uses a strategy pattern to pre-process any incoming value. I will be using this to do some calculations on the adc7 value the XBees report (that’s the voltage monitor pin) to convert it to the real voltage the batteries are providing.

All the components have been designed so they can be injected to any code that depends on them. This is a common pattern (dependency injection) that favours decoupling and provides a clean way to define strategies at runtime, for instance when mocking components in the unit tests.

As always, comments are more than welcome!

CC BY-SA 4.0 XBee to MQTT gateway by Tinkerman is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

10 thoughts on “XBee to MQTT gateway

  1. Pingback: Smartmeter pulse counter (4) | Tinkerman

  2. Scott

    I’ve been researching MQTT applications for my home automation plans. Regarding the Xbee2MQTT code. It appears that only data moving from the Xbee module to MQTT is being handled for now. If this is so, are you planning on incorporating data flow from MQTT to Xbee modules?

    Reply
    1. xose Post author

      Yes, data goes only from the XBees to the MQTT network for the current version. But I’m planning to develop the reverse functionality soon. For my next project I will add a couple of notification LEDs and make them blink with the Xbee sleep cycle and I want to switch the blinking on and off based on MQTT messages.

      The idea is to use the same pattern but reversed. For instance: imagine the scenario where pin DIO7 in radio ABCDEF is configured as digital output and is being mapped to /raw/xbee/ABCDEF/dio7 and remapped to /home/led/status. I want to be able to send a message from whatever client in the network to /home/led/status/set with a 0 or a 1 to change the status of the pin. It shouldn’t be hard to implement…

      Reply
    2. xose Post author

      Hi Scott
      I’ve added some basic communication from MQTT to remote radios. It basically allows you to set a digital output pin LOW or HIGH publishing a 0 or a 1 to a certain topic, just the way I defined in my previous comment.
      I’ve also changed a bit the way mappings were defined, so please review your configuration if you have already started doing something.

      Reply
    1. xose Post author

      Upgrading the python library was not in my priorities. Anyway, in the change log it says “The Python library is now written in pure Python and so no longer depends on libmosquitto.”, so if I upgrade de code to use the newest Mosquitto Python library it won’t be using the native C library but a parallel python implementation… Having parallel implementations is not a good idea (usually) but the new implementation supports SSL and Python 3. On the other hand I will love to see unicode support fixed. Don’t know, maybe I could give it a try.

      Reply
  3. Martin Jarvis

    Xose,
    This is a really interesting project. I’m working on something very similar using node.js. I managed to get your code installed on MacOS but could not get it to respond to XBee data. Hoping to diagnose the serial connection, I also tried to get the xbee2console to work, but the code was quitting due to problems with the log method. Is it possible to enable debug mode in deamon to see more information about what it is doing?
    Regards
    Martin

    Reply
    1. xose Post author

      Thank you Martin.

      The xbee2mqtt project is under a major refactoring and it’s not stable yet (mental note: tag stable revisions in the repo). The problem with the log method is happening because I have migrated the daemon to the built-in logging facility in python (http://docs.python.org/2/library/logging.html) but the xbee2console script has not been updated accordingly yet. I’ll try to push an update as soon as possible.
      If you have copied the default configuration file for the daemon it is already in debug mode (logging_level = 10, set it to 20 for production). Everything should be logged to the stdout file (/tmp/xbee2mqtt.log by default).
      The most common problem is permissions. Post here the error and I’ll try to help you.

      Xose

      Reply
  4. smas99

    sorry i want to ask..
    how to implementation xbee s2 (in API mode) with arduino uno in mqtt?
    i want to publish n subscribe sensor data
    thanks

    Reply
    1. Xose Pérez Post author

      No problem in asking 🙂

      MQTT is a protocol that works over TCP, you you will need something else aside an Arduino. You have two options (at least):

      • You can use an XBee in coordinator mode plugged to a computer (it can be a RaspberryPi or similar) via USB and a python daemon running in the computer decoding de XBee API messages and forwarding them using MQTT. This is the approach explained in this post. You can also read this other post: [http://tinkerman.cat/home-monitoring-system/](http://tinkerman.cat/home-monitoring-system/).
      • You can also use an Ethernet or Wifi shield with the [PubSubClient library](https://github.com/knolleary/pubsubclient). Check the documentation of the library for compatible hardware.
      Reply

Leave a Reply (all comments are moderated, be patient)