Category Archives: Analysis

20160820_102049x

New firmware for the Slampher

Some weeks ago I received a parcel from Itead. Previously, I had written about the Sonoff and they were kind enough to send me two more of their home automation products for me to review: the S20 Smart Socket I wrote about two weeks ago and the Slampher.

Slampher box

The Slampher comes in a simple cardboard box with no documentation at all… just visit their wiki!

The Slampher is kind of a Sonoff RF that sits before any light bulb with an E27 screw. As you can see in the header pic of this post it adds quite some length to the group. It’s a bit bulky and might not fit in every lamp. Off course the board layout is different from the Sonoff and it uses a JST131U-800D 1A triac instead of a relay to switch the bulb. Aside from that they are equivalent.

Continue reading

S20 Smart Switch

S20 Smart Socket

Since I discovered the Sonoff I’ve been thinking about embedding it inside a switch. I started looking for old power meters, timers,… I had at home but the Sonoff is a bit too long. Why didn’t they design a square board? I event bought a bulky Kemo STG15 case with socket.

Next I decided to design my own board. It is meant to be the “official” hardware for the ESPurna project so it’s called ESPurna too. It’s opensource hardware and available at the ESPurna project repository at Bitbucket. I have some boards already for the first iteration (version 0.1). They are mostly OK but I’m already working on a revision.

But then ITead’s released their S20 Smart Socket. It’s the Sonoff in a wall socket enclosure. Almost 100% what I wanted. And at 11.70€ it’s hard to beat. There are other wifi smart sockets available, mainly Orvibo and BroadLink (an SP2 Centros should be arriving home anyday now) but ITead’s is cheaper and you can easily re-flash it. Just solder a 4 pins header, connect it to your FTDI programmer, hold the S20 button, connect the programmer to your computer and flash. Done.

OK, not so fast. Why would I do that? Why would I change the stock firmware?

The answer for me is a mixed up of philosophy and practicity. But you are right. Let’s go step by step.

Continue reading

Sonoff TH 10A/16A

ITead’s Evolution

I was not the first to arrive at the party but since I discovered the ESP8266 I’ve been enjoying it. Then I stumbled upon the Sonoff and dude was I amazed. They are cheap and so very hackable you cannot help buying them, tear them open and customize them.

Sure they are not CE or UL compliant, yet. My previous post about adding a custom RF module to a Sonoff HT got some visibility as it was published at hackaday.com. Most of the commentors there  where concerned about safety measures in the device. The truth is that early Sonoffs looked more like products for the DIY market, not for end customers.

But ITead’s home automation product line is evolving quite fast. It all started with several Sonoff models (with or without RF, with or without temperature and humidity sensors, with or without AC/DC transformer) and the Slampher I’ll be reviewing it soon.

Continue reading

Decoding 433MHz RF data from wireless switches

[Update 2013-03-01] I have added more documentation on the codes these remotes use in a different post.

I’m starting to move towards not only gathering information but also acting. My first project in this subject will be controlling some lights and the house heaters. So last week I visited the urban market of “Els Encants” in Barcelona and bought some very cheap wireless outlets.

Two different remotes

I bought two sets of three wall plugs, each set with it’s own remote. They all transmit in the 433MHz frequency and I already had a set of transmitter and receiver for that frequency so as soon as I had some time I started trying to intercept and reproduce the codes the remotes were sending.

Sample outlets from each set plus remotes

Sample outlets from each set plus remotes

In the image above you can see an outlet and the remote for each of the sets. The left one is branded “Noru” and each outlet is rated 3kW (good for the heaters) and it features auto switch off time (1, 2, 4 or 8 hours). The remote can control a maximum of 3 outlets and apparently it is programmable, since you first have to sync the outlets with the remote.

The right one is branded “Avidsen”, also 433Mhz but rated 1000W, just below the consumption of my house electrical heaters, but good to control lights and other appliances. It’s got the very common dip switch to sync the remote and up to five outlets. There are 32 different channels available. So if your lights switch on and off randomly maybe you neighbour is using the same channel you are, then you better change the channel.

Available libraries for Arduino

I started reading documentation about the protocol these devices use and found out there is some very useful information out there. In fact there are even a couple of libraries for Arduino. The first one is called RemoteSwitch and it is a little old, it has not been ported to Arduino 1.0 but if you are like me you will keep a copy of Arduino 0023 just for this kind of situations.

The second library is called RCSwitch and I have to say it is a neat piece of code. It has been ported to the Raspberry Pi, although the port is not as updated as the original Arduino library.

My first tests with the RemoteSwitch library were encouraging. The Show_received_code sketch dumped the codes for the Avidsen remote one by one. I though: if it can decode it, it should be able to recreate it. And it worked from scratch. Good!

But by then I knew I wanted to use the newer library. There were several reason for this: it is being actively developed, it supports more protocols, the code is much more elegant and object oriented and it has a port to RPi, which I plan to use as a central gateway soon. So I checked which one of the RCSwitch protocols matched the one I had successfully used with RemoteSwitch and started doing some more tests…

Slightly different protocol

Here was when things started to get complicated. The thing did not work. So I spent a couple of hours studying the code for both libraries, decoding the codes the RemoteSwitch library had dumped before and trying to find the difference. Until I found it: RCSwitch.cpp, line 239, that ‘0’ should be a ‘1’… and everything started working again. Very good! I started a thread in the library forum to find out whether this is a bug or a slightly different protocol.

Index: RCSwitch.cpp
===================================================================
--- RCSwitch.cpp	(revision 219)
+++ RCSwitch.cpp	(working copy)
@@ -284,7 +284,7 @@
         if (sGroup[i] == '0') {
             sDipSwitches[j++] = 'F';
         } else {
-            sDipSwitches[j++] = '0';
+            sDipSwitches[j++] = '1';
         }
     }

By the way, the protocol of these things is pretty interesting. It’s worth a look at Sui’s post to get an overview of the implementation. The tri-bit concept is really awkward.

Using the Bus Pirate and OLS Logic Analyser

Then I moved to the other set of outlets. These are rated 3000W so I plan to use them to control my house heaters, which is the main reason for all this work. I followed the same steps, starting with getting the codes with the Show_received_code sketch. But weird enough the library was only able to decode some of the button presses… Only the SET button for outlet #1, the ON and OFF buttons for outlet #2, the ALL OFF button or the 2, 4 and 8H timeout buttons seemed to work.

This time it was going to be harder, since I didn’t even have all the codes. Well, a good opportunity to use my Bus Pirate!

Bus Pirate to the rescue!

Bus Pirate to the rescue!

So I plugged the RF receiver to the Bus Pirate and launched the OLS Logic Analyser to capture the signal from the remote.

You don’t have to configure anything to use the Bus Pirate as a (low speed) logic analyser. But since I wanted to power the radio receiver with the BP I had to enable the Power Supply mode. To do so you have to open a terminal session, type ‘?’ to get a prompt, select one of the modes that allow enabling the power supply typing ‘m’ and selecting the mode (like UART, for instance) and then type ‘W’ (uppercase to enable, lowercase to disable). Then you can close the session and it will keep the power on the 5V and 3V3 lines as long as it is plugged to the computer. Mind you have to free the port so the logic analyser software can use it. I had problems doing it with screen or minicom, but it worked great with picocom.

After some tests with the Avidsen remote (I knew what the codes were so I could compare the signal output with the actual code) I started getting the signals for each and every button in the Noru remote.

The image below shows the signal for the ON button for the outlet #1.

Signal for the #1 ON button of the Noru remote

Signal for the #1 ON button of the Noru remote

Now, since the RemoteSwitch library was able to decode some of the signals, the protocol could not be that different. So I started to decode manually all the signals applying the same protocol. The signal is a series of 12 tri-bits plus a sync-bit. For the Avidsen-like remotes there are 3 different tri-bit values (logically), they are called 0, 1 and F, for “floating”. Each tri-bit has a pulses shape. The following tables describes the pulses:

Tri-bit Pulses
0 short high + long low + short high + long low
1 long high + short low + long high + short low
F short high + long low + long high + short low

The long pulses are about 3 times the length of the sort ones. The overall period is a characteristic of each remote. There is also a trailing high pulse followed by a long low which is called “sync bit”.

A fourth tri-bit?

Decoding the signals from the Noru remote I found out that there was a fourth tri-bit value (well maybe I should call them tetra-bits now). In fact it is obvious since there is a forth option for an alternate sequence of 4 highs and lows. I’ve named the new tetra-bit X (for unknown, but also after my name :P). The full table for the Noru remotes is:

Tretra-bit Pulses
0 short high + long low + short high + long low
1 long high + short low + long high + short low
F short high + long low + long high + short low
X long high + short low + short high + long low

Now the previous image for the ON#1 button can be decoded as 1F000001FFX0S. With a small patch I could make this work with the RCSwitch library. The library cannot create the code but you can feed it to the sendTriState method to generate the signal.

Index: RCSwitch.h
===================================================================
--- RCSwitch.h	(revision 219)
+++ RCSwitch.h	(working copy)
@@ -106,6 +106,7 @@
     void sendT0();
     void sendT1();
     void sendTF();
+    void sendTX();
     void send0();
     void send1();
     void sendSync();
Index: RCSwitch.cpp
===================================================================
--- RCSwitch.cpp	(revision 219)
+++ RCSwitch.cpp	(working copy)
@@ -441,6 +441,9 @@
         case '1':
           this->sendT1();
         break;
+        case 'X':
+          this->sendTX();
+        break;
       }
       i++;
     }
@@ -561,6 +564,16 @@
 void RCSwitch::sendTF() {
   this->transmit(1,3);
   this->transmit(3,1);
+}
+
+/**
+ * Sends a Tri-State "X" Bit
+ *            ___   _
+ * Waveform: |   |_| |___
+ */
+void RCSwitch::sendTX() {
+  this->transmit(3,1);
+  this->transmit(1,3);
 }

 /**

And this is a sample code for Arduino that switches on and off outlet #1 every 2 seconds.

#include <RCSwitch.h>

RCSwitch mySwitch = RCSwitch();

void setup() {

  Serial.begin(9600);

  // Transmitter is connected to Arduino Pin #11
  mySwitch.enableTransmit(11);

  // Optional set pulse length.
  mySwitch.setPulseLength(302);

  // Optional set protocol (default is 1, will work for most outlets)
  mySwitch.setProtocol(1);

  // Optional set number of transmission repetitions.
  mySwitch.setRepeatTransmit(6);

}

void loop() {
    mySwitch.sendTriState("1F000001FFX0");
    delay(2000);
    mySwitch.sendTriState("1F000001FFFX");
    delay(2000);
}

Again, comments are more than welcome!

Storing and publishing sensor data

Now that I have started to monitor some magnitudes at home, like power consumption or the front door opening, I have to do something with this information. The sensor information can be useful for two purposes, mainly:

  • analysing it to know more about your environment or your life-style patterns (like when and how you spend the money you spend on energy)
  • taking real-time actions based on events from your sensors (open a light when you pass by a dark corridor at night, receive a notification when someone enters your house,…)

To analyse the information you will first have to store it in some way you could retrieve it later, graph it, summarize it, perform different time range roll ups or moving averages or detect time patterns,… whatever. Even if the goal of the sensor information is to trigger events (watch out: the door sensor is running out of batteries!) you will probably want to have it stored somewhere and save a log of actions as well.

So, where to store this information? You have different options available:

  • Relational databases (RDBMS) like MySQL or PostgreSQL. If you have some programming background this is pretty easy, we are all used to store data on a MySQL or similar. But mind the tables will GROW very quickly. A sensor reporting every minute will create more than half a million entries a year.
  • NO-SQL databases like MongoDB or Cassandra. They are popular and everyone wants to use them. Cassandra is often used to store time-series data like server load or log files and the Internet is full of information about how to store it, retrieve it, partition it,… MongoDB has also some nice features like capped collections.
  • Round-robin databases. They have been around for a while. RRDtool version 1.0 was born in 1999 and MRTG is even older. Data is stored in circular buffers, you can define the size of the buffer and the step or how often a new value is inserted into the buffer. When the buffer is full old data gets overwritten and this happens over and over. So you have another buffer for aggregated data (like daily data) and another one for even larger scale data (like weekly data) and so on.
  • Time series databases. And finally there are a number of databases that have been specifically designed to store time-series data: fast inserts, fast range lookups, roll ups,…

Now, some companies are providing time series storage services online. They provide a public API, graphing capabilities and some of them even roll ups or data aggregation. They have been born around the Internet of Things hype. Maybe the most popular is Cosm (formerly Pachube) but it’s not the only one: TempoDB, Nimbits or ThingSpeak are very good options.

Cosm has been around for a while and it is still the most used by the IoT community. AFAIK, there is no restriction on the number of data points you can store for free if you keep your data “public”. Data is structured in “feeds” and “datastreams”. They have an API you can use to push data individually or in batches. You can also use it to backup the data you have in Cosm. Their charts are nice looking but they have predefined time ranges that automatically aggregate your data so you have little control on what and how you want to graph. They have a number of goodies like triggers, graph builder (to insert your Cosm charts in your site), tags, apps, localization based search,…

TempoDB is a relatively new player. It’s free up to 5 million data points per database, that’s roughly 9.5 years for a 1 samples/minute data. You can have several “series” of data per database. Their API is really good and they have a number of clients including a python one :^). Their charts might not be as “pretty” as those from Cosm but you have full control on the data you are visualizing: date range, interval and aggregation functions, and they are FAST (Cosm can be really sluggish sometimes). Your data is private, so there is no sense in having a search tool. They have tags and I’ve been told they are about to release a notification service.

Nimbits has a desktop-like interface base on ExtJS. It is basically a paid service (their free quota is 1000 API requests per day, which is OK for 1 sensor reporting at a rate of 1 samples every 2 minutes) and it costs $20/2 million requests (roughly $20 per year if you have 4 sensors reporting at a 1 sample/minute rate).

ThingSpeak is very similar to Cosm, data is presented in a “channel” where you can see multiple charts for different magnitudes (they call them “fields”), a map and even a video from Youtube. It’s open source, so you can clone the code from Github and install it in you server. Their web service is limited to 1 API request every 15 seconds per channel, which is OK as long as you group all the data from one channel in a single request (that’s it if you have more than one field per channel). For each field you have a lot of options to define your chart: time range, time scale, different aggregation functions, colors, scale,…

The information in this post is not very detailed but I hope it can be an introduction to anyone who is looking for ways to store or publish sensor data. Right now I’m using Cosm and TempoDB, as well as storing all my data in a local MySQL database (not the best option but it works for now). There are plenty of options I still have to explore. In my next post I will talk about the daemon I’m using to push data to Cosm and TempoDB, in the meantime you can check the code on Github.