Tag Archives: code

Counting events with Arduino and PCF8583

Hey!

I’ve been away for some time. It’s not that I had stopped tinkering, but work and laziness have kept me away from the blog. During these months I have been working mostly on a new weather station (yes, yet another weather station or YAWS). The project was a step forward in a lot of aspects, from carpentry to remote reporting hacking cheap chinese routers and 3G dongles, from new libraries for Arduino to a bidirectional HTTP to MQTT bridge in node.js…

The result has been kind of a fail… mostly due to improper casing, humidity has been a major enemy. Anyway there are quite a few aspects to talk about, so I’ll try to write a series of posts about different hardware and software components involved in the project.

Counting the wind and rain

Two of these components share the same interface: when an event happens a pulse is triggered. The anemometer is a cheap plastic model I bought from todoelectronica.com. It’s specifications state than a 10km/h wind correlates with 4 pulses per second, so it is a simple pulse counter.

Anemometer

Anemometer

The rain gauge is a little more tricky… I bought a wireless rain gauge at ebay but I really didn’t need the wireless feature, it was just a convenient model to hack. The sensor has a seesaw with two small compartments where rain drops fall when passing through the top hole. The seesaw has a small magnet in the middle. When the water weight moves the seesaw down the magnet closes a reed switch triggering the pulse. The water evacuates and the other compartment starts receiving the water.

Wireless Rain Gauge

Wireless Rain Gauge

I hacked the rain sensor disabling all the circuitry and soldering a couple of wires to the reed switch with a load resistor in series. By “disabling the circuitry” I mean cutting the lines to the switch to avoid spurious currents to create noise but leaving the circuit, so it fits nicely inside the sensor.

I decided to use a standard interface for the cables, a 3.5 stereo jack (a mono jack would have been enough but I didn’t have any around). They plug into the main board of the weather station and the signals go to the event counter pins of a couple of PCF8583.

The PCF8583 is very similar to the PCF8563 but it adds several nice features: hardware configurable I2C address with 2 different addresses available (you can throw 2 of these IC in your project without mush hassle), 240 bytes of free RAM, more alarm options and an event counter mode.

According to the datasheet, “The event counter mode is used to count pulses externally applied to the oscillator input (OSCO left open-circuit).”. The count is stored as BCD values in 3 registers. BCD stands for Binary-coded decimal, the most common implementation is to split a byte into 2 groups of 4 bits and use each group to represent a decimal number from 0 to 9. So, for instance, 37 would be encoded as “0011 0111”. Thus, 3 registers or 3 bytes allow a maximum of 1 million events. This IC is very convenient for any project where you want to remove the responsibility of counting events from your microprocessor, and it would have been a very good choice for my smartmeter pulse counter.

To easily access the IC features I have written an Arduino library for the PCF8586. The library is not complete, there are some methods to be implemented yet. But the core time and event counter methods are already there.

To library comes with a simple exemple of usage (I have to add more examples) but to use it as an event counter you will only have to write something like:


#include

// declare an instance of the library for IC at address 0xA0
// (A0 pin connected to ground)
PCF8583 counter(0xA0);

void setup {

    // configure serial link to debug
    Serial.begin(9600);

    // configure PCF8586 to event counter mode and reset counts
    counter.setMode(MODE_EVENT_COUNTER);
    counter.setCount(0);

}

void loop {

    // report the events every second
    Serial.println(counter.getCount());
    delay(1000);

}

Issues

There are a couple of issues to take into consideration.

First: the counter counts double. That’s it: for every pulse it looks like it counts the rising AND the falling edges so you end up with double the counts. I have not been able to find a reason for this, nor even in the datasheet. I have tested it with a controlled set up, cleaning the input with an RC filter and a Schmidt Trigger and it counts exactly double it should…

Second: the maximum external clock frequency in event counter mode is rated to 1MHz. In the real world of my projects noise is an issue with frequencies way bellow that mark, so I only accept sensible values depending on what I’m measure. For instance, the above anemometer sends 4 pulses per second for a 10km/h wind. Fortunately, where I live there are no hurricanes and I have not checked but I doubt that there have ever been gusts of more than 150km/h. That sets a maximum of 60Hz. So in my code I have a check to not trust any value over 60 counts per second.

Final note

Finally, there are some other libraries for the PCF8586, like this one by Ryan Mickfor Netduino or this other one by Eric Debill for Arduino, that are worth checking as a reference.

As always, any comment will be more than welcomed.

It’s nice to be back 🙂

The Rentalito

“Say Hello to The Rentalito!”

That was the first message The Rentalito displayed almost 2 years ago. The Rentalito means something like “small rental monitoring thingy” (free translation from “rental” and “ito”, which is the suffix for “small” in Spanish). It was born to display the rentals in a VOD project my team was working at that time. The Rentalito displayed total rentals, rentals by day and it beeped every time a new rental showed up in the database displaying the movie that was just rented.

The Rentalito at Work

In the beginning it was funny and kind of a party every time it beeped. The time between beeps was quite spaced but it was just the beginning, you know. But as the project went on and the rentals didn’t reach the required pace my team was moved to other projects and shortly after I left the company and The Rentalito came back home with me.

The Rentalito uses two 32×16 RG Dot Matrix displays by SureElectronics. They cost about $32 each on the official store, I bought it on ebay. They use a Holtek HT1632C driver and there are a bunch of libraries (in github, in google code) and examples on using these displays with Arduino. I grabbed one that suited my needs (it works, it’s easy to use, and has scrolling text functionality) and added a couple of functionalities. You can check it out here.

The first version of The Rentalito was based on an Arduino UNO with an Ethernet shield and the display was wired with jumper wires to the shield headers. Quite messy. So every thing was hidden from view with a cardboard (!!) box with a sticker on it designed by the team designer (yeah, it’s a VHS video tape). Everything was held together with a sheet of plywood, some screws and spacers.

The code was not better looking but it worked. The Arduino requested the data to a PHP web application that was running on my laptop, it queried the production database to get the latest data and built a message in a proprietary format that was sent back to the Arduino, there it got parsed and displayed. The server application used a SQLite database to store aggregated data and custom messages that could be scheduled based on date and time (like “Good morning” before 10am or “Today is Einar’s birthday!!” when it was Einar’s birthday). The response could contain more than one message. Each message had title and body (or just body for a full screen message) and some control codes (text color, beep,…). The system worked fine but it was not very flexible.

The Rentalito goes WIFI

In its second life The Rentalito went WIFI. Using a Roving Networks RN-XV module and a custom Arduino shield that had a male header to plug the cable from the display. Tidier.

Rentalito Schematics

Rentalito Schematics

Software side I replaced the Ethernet library with WiFlyHQ by harlequin-tech. The library is really good, easy to use and very configurable but it doesn’t implement the Ethernet Client API so it is not a drop-in replacement for the stock Ethernet library.

The Rentalito goes MQTT!!

The last evolution of The Rentalito has been integrating into my home network as a MQTT client using excellent Nick O’Leary’s PubSubClient library for Arduino. But to make it work  I had to switch to a WIFI library that uses the Ethernet Client API. After testing some existing libraries I chose Sparkfun’s Wifly Shield library, basically because it worked.

Although there are several points for improvement in the code (mainly the ht1632c library) the approach is much more flexible now. The main problem is some ugly incompatibility between the ht1632c library and the WiFly library which causes the connection to hang sometimes when a new message arrives while refreshing the display. It’s probably due to the fact that I am using SoftwareSerial library instead of hardware serial. Anyway, if not using scrolling messages the display works for hours without problems and even if there is a problem there is code to reconnect and keep going. The code is available on github.

On the outside the new version has also seen some changes. I finally got rid of the wooden sheet and then cardboard box replacing them by two sheets of methacrylate which make it look cooler (my opinion at least) and give it  some support to stand over a flat surface.

The Rentalito perspective

The Rentalito perspective

The Rentalito close view

The Rentalito close view

The Rentalito displaying the current power consumption

The Rentalito displaying the current power consumption

The Rentalito has suffered different metamorphosis for the past months and I am sure this won’t be it’s last reincarnation.

Decoding 433MHz RF data from wireless switches

[Update 2013-03-01] I have added more documentation on the codes these remotes use in a different post.

I’m starting to move towards not only gathering information but also acting. My first project in this subject will be controlling some lights and the house heaters. So last week I visited the urban market of “Els Encants” in Barcelona and bought some very cheap wireless outlets.

Two different remotes

I bought two sets of three wall plugs, each set with it’s own remote. They all transmit in the 433MHz frequency and I already had a set of transmitter and receiver for that frequency so as soon as I had some time I started trying to intercept and reproduce the codes the remotes were sending.

Sample outlets from each set plus remotes

Sample outlets from each set plus remotes

In the image above you can see an outlet and the remote for each of the sets. The left one is branded “Noru” and each outlet is rated 3kW (good for the heaters) and it features auto switch off time (1, 2, 4 or 8 hours). The remote can control a maximum of 3 outlets and apparently it is programmable, since you first have to sync the outlets with the remote.

The right one is branded “Avidsen”, also 433Mhz but rated 1000W, just below the consumption of my house electrical heaters, but good to control lights and other appliances. It’s got the very common dip switch to sync the remote and up to five outlets. There are 32 different channels available. So if your lights switch on and off randomly maybe you neighbour is using the same channel you are, then you better change the channel.

Available libraries for Arduino

I started reading documentation about the protocol these devices use and found out there is some very useful information out there. In fact there are even a couple of libraries for Arduino. The first one is called RemoteSwitch and it is a little old, it has not been ported to Arduino 1.0 but if you are like me you will keep a copy of Arduino 0023 just for this kind of situations.

The second library is called RCSwitch and I have to say it is a neat piece of code. It has been ported to the Raspberry Pi, although the port is not as updated as the original Arduino library.

My first tests with the RemoteSwitch library were encouraging. The Show_received_code sketch dumped the codes for the Avidsen remote one by one. I though: if it can decode it, it should be able to recreate it. And it worked from scratch. Good!

But by then I knew I wanted to use the newer library. There were several reason for this: it is being actively developed, it supports more protocols, the code is much more elegant and object oriented and it has a port to RPi, which I plan to use as a central gateway soon. So I checked which one of the RCSwitch protocols matched the one I had successfully used with RemoteSwitch and started doing some more tests…

Slightly different protocol

Here was when things started to get complicated. The thing did not work. So I spent a couple of hours studying the code for both libraries, decoding the codes the RemoteSwitch library had dumped before and trying to find the difference. Until I found it: RCSwitch.cpp, line 239, that ‘0’ should be a ‘1’… and everything started working again. Very good! I started a thread in the library forum to find out whether this is a bug or a slightly different protocol.

Index: RCSwitch.cpp
===================================================================
--- RCSwitch.cpp	(revision 219)
+++ RCSwitch.cpp	(working copy)
@@ -284,7 +284,7 @@
         if (sGroup[i] == '0') {
             sDipSwitches[j++] = 'F';
         } else {
-            sDipSwitches[j++] = '0';
+            sDipSwitches[j++] = '1';
         }
     }

By the way, the protocol of these things is pretty interesting. It’s worth a look at Sui’s post to get an overview of the implementation. The tri-bit concept is really awkward.

Using the Bus Pirate and OLS Logic Analyser

Then I moved to the other set of outlets. These are rated 3000W so I plan to use them to control my house heaters, which is the main reason for all this work. I followed the same steps, starting with getting the codes with the Show_received_code sketch. But weird enough the library was only able to decode some of the button presses… Only the SET button for outlet #1, the ON and OFF buttons for outlet #2, the ALL OFF button or the 2, 4 and 8H timeout buttons seemed to work.

This time it was going to be harder, since I didn’t even have all the codes. Well, a good opportunity to use my Bus Pirate!

Bus Pirate to the rescue!

Bus Pirate to the rescue!

So I plugged the RF receiver to the Bus Pirate and launched the OLS Logic Analyser to capture the signal from the remote.

You don’t have to configure anything to use the Bus Pirate as a (low speed) logic analyser. But since I wanted to power the radio receiver with the BP I had to enable the Power Supply mode. To do so you have to open a terminal session, type ‘?’ to get a prompt, select one of the modes that allow enabling the power supply typing ‘m’ and selecting the mode (like UART, for instance) and then type ‘W’ (uppercase to enable, lowercase to disable). Then you can close the session and it will keep the power on the 5V and 3V3 lines as long as it is plugged to the computer. Mind you have to free the port so the logic analyser software can use it. I had problems doing it with screen or minicom, but it worked great with picocom.

After some tests with the Avidsen remote (I knew what the codes were so I could compare the signal output with the actual code) I started getting the signals for each and every button in the Noru remote.

The image below shows the signal for the ON button for the outlet #1.

Signal for the #1 ON button of the Noru remote

Signal for the #1 ON button of the Noru remote

Now, since the RemoteSwitch library was able to decode some of the signals, the protocol could not be that different. So I started to decode manually all the signals applying the same protocol. The signal is a series of 12 tri-bits plus a sync-bit. For the Avidsen-like remotes there are 3 different tri-bit values (logically), they are called 0, 1 and F, for “floating”. Each tri-bit has a pulses shape. The following tables describes the pulses:

Tri-bit Pulses
0 short high + long low + short high + long low
1 long high + short low + long high + short low
F short high + long low + long high + short low

The long pulses are about 3 times the length of the sort ones. The overall period is a characteristic of each remote. There is also a trailing high pulse followed by a long low which is called “sync bit”.

A fourth tri-bit?

Decoding the signals from the Noru remote I found out that there was a fourth tri-bit value (well maybe I should call them tetra-bits now). In fact it is obvious since there is a forth option for an alternate sequence of 4 highs and lows. I’ve named the new tetra-bit X (for unknown, but also after my name :P). The full table for the Noru remotes is:

Tretra-bit Pulses
0 short high + long low + short high + long low
1 long high + short low + long high + short low
F short high + long low + long high + short low
X long high + short low + short high + long low

Now the previous image for the ON#1 button can be decoded as 1F000001FFX0S. With a small patch I could make this work with the RCSwitch library. The library cannot create the code but you can feed it to the sendTriState method to generate the signal.

Index: RCSwitch.h
===================================================================
--- RCSwitch.h	(revision 219)
+++ RCSwitch.h	(working copy)
@@ -106,6 +106,7 @@
     void sendT0();
     void sendT1();
     void sendTF();
+    void sendTX();
     void send0();
     void send1();
     void sendSync();
Index: RCSwitch.cpp
===================================================================
--- RCSwitch.cpp	(revision 219)
+++ RCSwitch.cpp	(working copy)
@@ -441,6 +441,9 @@
         case '1':
           this->sendT1();
         break;
+        case 'X':
+          this->sendTX();
+        break;
       }
       i++;
     }
@@ -561,6 +564,16 @@
 void RCSwitch::sendTF() {
   this->transmit(1,3);
   this->transmit(3,1);
+}
+
+/**
+ * Sends a Tri-State "X" Bit
+ *            ___   _
+ * Waveform: |   |_| |___
+ */
+void RCSwitch::sendTX() {
+  this->transmit(3,1);
+  this->transmit(1,3);
 }

 /**

And this is a sample code for Arduino that switches on and off outlet #1 every 2 seconds.

#include <RCSwitch.h>

RCSwitch mySwitch = RCSwitch();

void setup() {

  Serial.begin(9600);

  // Transmitter is connected to Arduino Pin #11
  mySwitch.enableTransmit(11);

  // Optional set pulse length.
  mySwitch.setPulseLength(302);

  // Optional set protocol (default is 1, will work for most outlets)
  mySwitch.setProtocol(1);

  // Optional set number of transmission repetitions.
  mySwitch.setRepeatTransmit(6);

}

void loop() {
    mySwitch.sendTriState("1F000001FFX0");
    delay(2000);
    mySwitch.sendTriState("1F000001FFFX");
    delay(2000);
}

Again, comments are more than welcome!

XBee to MQTT gateway

So far I’ve posted about hardware and theoretical stuff like network architecture or naming conventions. I think it’s time to move to the software side.

The core of the sensor network I’m deploying at home is the Mosquitto broker that implements MQTT protocol. It manages the messaging queue, listening to messages posted by publishers and notifying the subscribers.

I’ve been working in parallel to have at least some pieces in place to get and store information from the pulse counter sensor. These are an XBee to MQTT gateway and a couple of consumers: one storing info into a MySQL database and another one pushing it to cosm.com.

I want to introduce you the first piece: the xbee2mqtt daemon. It’s already available on github under GPL v3 license. It publishes the messages received by an XBee radio to the Mosquitto instance. The radio must have a Coordinator API firmware loaded. Right now the gateway understands frame IDs 0x90 and 0x92 which account for “Zigbee received packet” (i.e. data sent through the serial link of the transmitting radio) and “Zigbee IO data sample” (that’s an automatic sample of selected analog and/or digitals pins on the transmitting radio).

I’ve tailored the daemon to my needs, but trying to be as generic as possible. The design is based on small components:

  • The “xbee” component takes care of the connection to the radio and the packet parsing.
  • The “router” maps xbee addresses/ports to MQTT topics.
  • The “processor” pre-processes values before publishing them.
  • The “mqtt” component takes care of the message publishing.
  • And the XBee2MQTT class (which extends Sander Merechal’s fabulous daemon class) glues everything together.

You can read the code to get a full insight of what it does but I’d like to explain here some decisions I’ve taken.

I’ve abstracted the message source to an address and a port. The address is the 8 bytes physical serial number of the radio (SH and SL) and the port is the pin (adc0, adc1,… dio1, dio2,…). 0x90 packets are mapped to virtual ports. The sender can define the name of the virtual port (like “battery:4460\n”) or otherwise a generic name will be used (“serial” by default).

The routing is a basic functionality. As I already explained in my previous post about topic naming conventions I think the mapping should be done in the gateway because no other component should have to know about the physical structure of the wireless (XBee) network. So the xbee2mqtt daemon maps all the messages to MQTT topics with semantic meaning. You can also allow default topic mapping which will publish any message received by an undefined address/port combination to a topic like /raw/xbee/<address>/<port>.

The processor uses a strategy pattern to pre-process any incoming value. I will be using this to do some calculations on the adc7 value the XBees report (that’s the voltage monitor pin) to convert it to the real voltage the batteries are providing.

All the components have been designed so they can be injected to any code that depends on them. This is a common pattern (dependency injection) that favours decoupling and provides a clean way to define strategies at runtime, for instance when mocking components in the unit tests.

As always, comments are more than welcome!