Manage and persist settings with Embedis

For months I've been searching for a settings manager for my Arduino and ESP8266 projects. There are basically two issues to take care of:

• On memory settings or** how do you access to the configuration values** from your code
• Persistence or how do you store the configuration across reboots

For the first issue common solutions in high level languages are key-value solutions in the form of containers or hash arrays. But C++ is not so high level and the ATMega328 or the ESP8266 are nothing more than microcontrollers so resources are limited. Common patterns are simple variables, container classes or structures. They all have the same problem: they are mostly static, if you want to add a new setting you have to change the code and flash it to your controller.

For the second issue some microcontrollers have some type of EEPROM, like the ATMega328 in the Arduinos. The ESP8266 doesn't have one so it needs an external SPI Flash memory. Using the** file system** partition in the ESP modules to store a configuration file with all the project settings is feasible. It is persistent across reboots but it gets deleted when you upload a new file system binary. You could use an external memory or make use of “reserved” memory space for you own purposes like Peter Scargill proposes in a recent post.  But if you are using Arduino Core for ESP8266 it already defines a memory layout that reserves a **4Kb block to emulate EEPROM** and provides a library to access it that it's (mostly) API compatible with the Arduino EEPROM library.

Meet Embedis

The Embedis library is the magic trick that solves all the issues above. It provides a key-value type access to your settings, you can set (both updates or creates), get and delete them, list them, query if a certain key exists,… and you can persist them in different storage solutions, in particular in the (emulated) EEPROM.

The library is really easy to use. Just initialize an embedis object and throw this code in your setup and loop methods and you are good to go (if you don't need console management you can forget the loop part).

Embedis embedis(Serial);

void setup() {

    ...

    EEPROM.begin(SPI_FLASH_SEC_SIZE);
    Embedis::dictionary( F("EEPROM"),
        SPI_FLASH_SEC_SIZE,
        [](size_t pos) -> char { return EEPROM.read(pos); },
        [](size_t pos, char value) { EEPROM.write(pos, value); },
        []() { EEPROM.commit(); }
    );

    ...

}

void loop() {

    ...

    embedis.process();

    ...

}

The previous code works on ESP8266 devices, if you are using an Arduino you don't need any parameter in the “begin” method but you will need to define a proper size for the dictionary.

Then you just have to get and set your settings and they are automatically persisted to EEPROM:

String ssid;
Embedis::get("ssid", ssid);

... (the user changes the ssid from the web interface, for instance) ...

Embedis::set("ssid", ssid);

The library stores the settings in the EEPROM starting from the end of it backwards. So you can still use the beginning of the memory to store your own values (relay statuses, for instance) provided that you use less than the available EEPROM size between both.

Goodies

The library comes with some nice bonus features, like console management and custom commands. You can modify all your settings from your terminal connection or eventually create your own commands to manage your wifi, reboot the board or execute any administrative options on your device.

Some of the built in commands let you manage your key-value database directly from console or publish o subscribe to topics. This last feature could be used, for instance, to enable specific debug messages. Instead of simply dump debug messages to console we can:

<del>Serial.println("[WEBSERVER] Request /index.html");
</del>Embedis::publish("webserver", "[WEBSERVER] Request /index.html");<del>
</del>

The message won't be displayed unless you subscribe to the “webserver” topic from the console with:

subscribe webserver

You will then receive only the messages for that topic (and not the “wifi” or “sensor” ones, for instance).

Credits

The Embedis library is an open source, community supported project, initially developed by David Turnbull, Tom Moxon and PatternAgents, LLC as part as the thingSoC initiative.

I'm not associated in any way to Embedis, thingSoC or PatternAgents, nor I had any previous contact with them. My intention with this post is to give credit to a tool that has solved me a problem in a way that I find both elegant and efficient. A perfect match.

"Manage and persist settings with Embedis" was first posted on 16 September 2016 by Xose Pérez on tinkerman.cat under Analysis, Tutorial and tagged arduino, atmega328, eeprom, embedis, esp8266, spiffs.