Monthly Archives: November 2012

Smartmeter pulse counter (2)

This week I have had some spare time – well, I should say I’ve borrowed some time from my sleep – to work on the smart-meter pulse counter setup.

In my last post about this I analized the signal from my photocell sensor. My conclusion was that the signal was clean and neat, even before throwing in a schmitt trigger to make it more “digital”. I have found out that it is also very dependent on the environmental light so I good isolation is a must for the sensor.

I had already prepared a sensor probe with a 3.5mm audio jack on one end and the photocell on the other. But the photocell is aligned with the cable, which is quite rigid and I am concerned about how to attach it to the smart meter case. For now I will leave it like that and do some real scenario tests. Since it is an external sensor I can change it whenever I want.

Finally I have also added an RC filter to the sensor input. The previous data suggests it is not necessary but I needed to do some tests with a faked signal source (a mini button) and it won’t harm.

Aside from these things I’ve done some very important steps forward. I have a final schema for the board.

There are some points worth noticing:

  • I’ve added a ceramic capacitor across VDD and GND as the Xbee documentation recommends. I didn’t have the 1uF and 8.2pF ones but I have seen some boards with the *standard* 100nF.
  • There is a voltage divider for the battery monitoring on the top right corner.
  • On the bottom you can see the voltage divider with the photocell in series with the RC filter
  • I’m wiring DIO9 *and* DTR to the Arduino, this way I can switch from Xbee-driven to Arduino-driven from code (more on this later).
  • Finally I decided to wire DOUT to the Arduino (I’m using SoftwareSerial, so RX and TX pins are digital 5 and 6 respectively) Not any more. I am using hardware serial on the final version…

This setup allows me to choose between two very different approaches: I can make Arduino drive the sleep cycle by changing DTR pin (Xbee in mode SM1 or “pin sleep”) or I can make the Xbee control the sleep cycle (Xbee in mode SM4/5 or “cycle sleep”) triggering an Arduino interrupt  when it awakes.

The first option lets me send a report every N pulses. In my case 4 pulses equal 1Wh. Lets say I want to report chunks of 5Wh, that’s 20 pulses. That means a report every 4.5 to 90 seconds depending on the current load (200W to 4kW range), being 60 seconds (300W) the average.

With the second option I can report average power consumption every minute. The Arduino counts the pulses and when the Xbee awakes it asks the Arduino (i.e. triggers an interrupt) to calculate and report the average power consumption over the last minute. For instance, if the pulse count is 28, that’s 7Wh over the last minute or an average of 420W.

I prefer the later because the reports are evenly spaced which means they are somewhat easier to graph and the graph is more meaningful: if you graph the power in the Y axis and time in the X axis the area below the power line is the energy (watts * hour). With the former the data should be post-processed to get some useful information like average Wh per day.

I’ve already done some tests but, so far, I have not been able to program the Xbee to follow an accurate sleep cycle. To configure the sleep period you can use SP and SN, the first one represents the time in tens of milliseconds, the second is a multiplier. So with SP=0x03E8 (1000 in hex) and SN=0x06 that’s 60 seconds. The problem is that the awake period has it’s own counter, ST, which is reset “each time serial or RF data is received”. That introduces a random time in the calculation which translates into a slight shift in the reports.

Option B would be to use some RTC to awake the Xbee with an alarm trigger. But I would rather avoid adding more hardware to the sensor.

Well, still a lot of things to figure out… more to follow 🙂

Home monitoring system

All of us tinkermen eventually end up working on a home monitoring/automation system sooner or later. And that’s for me the big background project at the moment.

I have already some of the pieces in several stages of readiness but I was lacking an overall view of the system as a whole. My initial approach was to store everything in a MySQL database and develop a web application to graph the time series values. The plan was to deploy a Wireless Sensor Network with an undefined number of sensors transmitting their information over Zigbee. An Xbee coordinator, laying on a Sparkfun Xbee Explorer USB pluged to the server, will receive the information and a python script will decode the packages and store the message in the database.

Home WSN version 1

I was planning to design the database based on the physical building blocks (device, sensor, value) and then provide a REST API (I am a REST fanboy) to consume the data.

Then I thought about storing the information from other services I’m using like Efergy Engage (I will talk about that in another post). Unfortunately Efergy does not provide an API for their products. I’ve asked them a couple of times and they said that that was not priority but I think that will go against what they feel their business is so I don’t think they will ever open it. Anyway it was not hard to look at the requests they are doing at their online graphics app and I wrote a little script that logs in and makes a single request to read the data for the last 24 hours and stores it in the database.

And then I thought that it won’t harm start graphing all that data in or an RRD tool. After all it will take me some time to put the UI I wanted in place. But then where should I plug that? Do I have to create a consumer for the REST API that just pushes data to with a cron every minute? Or should the Xbee driver and the efergy script dump the information to MySQL *and* to In the second option MySQL is a simple data backup storage and there are a plethora of drivers performing various tasks. Moreover, if I want more consumers in the future I will have to add code to all those drivers… The first option seems more mantainable and decouples the several components of the network. MySQL becomes the center of the network and clients will pull data from it on regular basis using the REST API. No real-time data here, thou. And each component will have to support my custom API.

And then one day, reading the news feeds in my phone on my way to work, I stumbled on a post in Robbert Henkkens blog (on of my favourites) about publishing sensor data using MQTT. MQTT (Message Queue Telemetry Transport) is a lightweight publish/subscribe messaging protocol aimed to machine-to-machine communication. It was created at IBM in 1999 but it has an open license and it’s royalty free. The infrastructure requires of a broker that receives and dispatches messages and a number of clients. Each message has a topic and a content (and a header that can be as short as two bytes). Clients can publish or subscribe to any number of topics (you can define access control rules) and specify I message to post on a topic when they disconnect. There are 3 QoS levels and subscribers can request messages they lost while disconnected.

It took me a few days to realize that that was the solution I needed to decouple the components of my network. I could work on each component of my network in a isolated way: a publisher or consumer of MQTT messages, no matter what or who is on the other side of the communication. There are a number of MQTT brokers available, including Mosquitto, an open-source project that provides a daemon broker, C library, python library, and command line utilities for publishing and subscribing which is perfect for me. Besides I would be using a (de facto) standard protocol, no more custom APIs or other solutions. Even supports MQTT (although it does not support the full specification )!

This is how my MQTT-centered WSN might look like:

Home WSN version 2

I am already adapting what I have to this pattern. There are still some issues I have to address, like how to store the sensor data (MySQL is not the best option for sure) and define a topic naming convention or how to publish data from dumb sensors under the right topic. Robert suggests a couple of solutions for this last problem. I think the republishing topics option is the one I like the most but this is a topic (ehem) for another post.