20. February 2017

How to send command from Python via MQTT to RGB LED connected to ESP8266

I’ve described how to send server load as a number to MQTT in the previous article. The number could be then translated via Node-RED to command for LampESP with RGB LED. The result is simple. LED indicates server load by displaying different colors.

The other option is to deliver color command directly from the server using Python.

Just install paho-mqtt:

pip install paho-mqtt

Here is small snippet of Python code (publish_server_load.py):

#!/usr/bin/env python3

import paho.mqtt.publish as publish
import os

color = 'red'
load = os.getloadavg()[0]
if load < 0.7:
    color = 'black'
elif load < 1.5:
    color = 'blue'
elif load < 3:
    color = 'green'
elif load < 7:
    color = 'orange'

publish.single('/server/monitoring/command', color, hostname='iot.sinusgear.com')

ESP should listen to /server/monitoring. Code of LampESP 0.3 is available at GitHub.

Put this code into crontab

* * * * * /usr/local/bin/publish_server_load.py

If you’re using virtualenv the command should be:

* * * * * /opt/my-python-env3/bin/python /usr/local/bin/publish_server_load.py

11. February 2017

Display server load on Linux Mint desktop delivered via MQTT

In previous article I wrote how to subscribe to MQTT topics and display also timestamp with color. There was one MQTT topic which represented server load and it demonstrates that MQTT could be used for server monitoring.

Here is an example how to measure server load and publish it as MQTT topic. Open crontab:

crontab -e

Add following line:

* * * * * /usr/bin/mosquitto_pub -t /server/reactor/load -m `cat /proc/loadavg | sed -e 's/ .*//g'`

Cron will invoke the command every minute. Command will publish data to MQTT topic /server/reactor/load (note: reactor is the name of my server).

We can subscribe at a workstation to the topic and execute some commands. mosquitto_sub will keep running in the background and we can use xargs to invoke a new command once the new message arrives.

Following example shows how to display notification on Linux Mint – Cinnamon using

mosquitto_sub -t '/server/reactor/load' -h iot.sinusgear.com | xargs -d$'\n' -L1 sh -c 'notify-send "Server load: $0"'

The result at workstation:

The only downside is that notifications are spamming desktop every minute. It does not make sense to notify admin when server load is too low. We can add a little bit of logic and display notifications about load which exceeded a certain threshold.

mosquitto_sub -t '/server/reactor/load' -h iot.sinusgear.com | xargs -d$'\n' -L1 /bin/zsh -c 'if (( $(echo "$0 > 1.2" | bc -l ) )); then notify-send "Server load: $0"; fi'

Note: just be aware that message from MQTT is not sanitized. Run such command only in trusted environment or add type check of value received from MQTT.

Enjoy 🙂

5. February 2017

Colorful log from MQTT with timestamp

The easiest way to subscribe to all messages on MQTT server like Mosquitto is following command:

mosquitto_sub -v -t '/#'

The only problem is that result is just plain text without timestamp which makes it harder to spot some messages.

The solution to the problem is quite simple. Just pipe output of subscription command to xargs which will inject date and to ccze which will add colors. Here is the command:

mosquitto_sub -v -t '/#' | xargs -d$'\n' -L1 sh -c 'date "+%d.%m.%Y %T $0"' | ccze -m ansi

Result:

30. January 2017

LampESP 0.3 – small improvements

I’d like to announce small updates to LampESP code.

The first feature is that you can send commands with color via MQTT to RGB LED. It’s easy, just send color name to topic /parent/host/command. Available colors are:

  • blue
  • green
  • orange
  • red
  • white

Or you can send just off to turn the led off.

The other small feature is a configuration for PIR sensor. I moved the configuration of PIR uptime minimal limit to configuration. You can set a number of seconds which represents how long it will take to turn off the light. The configuration could be set by URL:

http://IP-address/config?pir_up_interval=30

Here you can see a configuration of ESP8266 with Photocell sensor, PIR and 3W LED powered by power bank.

One interesting thing is small breadboard ZY. Dimensions are just 5×5 and it’s very useful for small configurations like this one.

Enjoy 😉

1. December 2016

ESP8266 WiFiManager gotchas

Thanks to PlatformIO it is very easy to add further functionality from libraries to the code for ESP8266. When you need to install library, just start Library Manager from ide and type:

pio lib install WiFiManager

PlatformIO will resolve dependencies and download all necessary stuff. Even better option is to add dependency to platformio.ini file:

[env:d1_mini]
platform = espressif8266
board = d1_mini
framework = arduino
lib_deps =
  ArduinoJson
  esp8266_mdns
  PubSubClient
  WiFiManager

PlatformIO will manage the installation after you save the file. That is very neat.

Then new problems occurred when I started with integration of WiFiManager. Do not take me wrong. WiFiManager is very handy library, but you may experience some tricky issues.

The first problem that I hit was that WiFiManager interface was not visible at 192.168.4.1 port 80. ESP was apparently running in AP mode for configuration, but there was no web interface. This issue was caused by this line in my code:

ESP8266WebServer server(80);

I had web server in the main code before I merged the code from WiFiManager. Web server was serving simple REST API for controlling relay. Unfortunately this construct blocked the port so the AutoConnect web server. The server was not able to bind the port. Solution was simple. It was sufficient to instantiate web server after WiFi was working.

The second problem was that AutoConnect from WiFiManager was always turning in AP mode. I was hunting the problem for several hours without any result. One idea was that WiFiManager is not saving the password. That was not true. Other idea was that the memory is corrupted and reformat would help. Unfortunately it didn’t. I gave up.

That solved the problem. Seriously. I gave up and that solved the problem. I let the device running and then ding. It was online. For some strange reason ESP8266 was not able to establish connection with particular WiFi router at first shot. Then it turned on AP mode and after timeout of Config Portal it connected to the WiFi network. Remedy? Just shorten ConfigPortalTimeout:

wifiManager.setConfigPortalTimeout(60);

The third problem was that Config Portal was not displaying information stored as extra custom parameters. I stored there host name of MQTT broker and other options so they won’t be hardcoded. The problem was caused by WiFiManagerParameter constructed in global scope. Issue was similar the first problem with web server.

Solution was to move construction of WiFiManagerParameter to the method after loading configuration values from file config.json stored on file system.

File configFile = SPIFFS.open("/config.json", "r");
...
WiFiManagerParameter custom_mqtt_host("mqtt_host", "MQTT host", configMqttHost, 40);

After resolving these issues WiFiManager is working and it is possible to set values for configuration via Config Portal and there is no need to hardcode them anymore.

Original code is stored at GitHub – LampESP branch v0.1. The new version is at Github – LampESP branch master. I made also small refactoring and the functionality was divided into smaller files which resembles modules for better code maintenance.

27. November 2016

Controlling relay connected to ESP8266 via MQTT from Android

After successful implementation of MQTT into Lamp Relay with ESP8266 there was one remaining challenge. Communication of Android Lamp App with ESP8266 via MQTT. It was little bit harder than I expected, because examples were not very clear.

Sending commands to MQTT broker was relatively easy. There was just a catch. It is necessary to declare the MQTT service in AndroidManifest.xml:

<service android:name="org.eclipse.paho.android.service.MqttService">
</service>

Thanks to tutorial from HiveMQ I was able to send messages to MQTT broker.

The problem was the opposite direction. How to receive messages?

Well, there is support in the same MQTT library for topic subscriptions. The issue is that incoming message is being processed on background thread and it’s not possible to update UI. It took me some time to realize the source of problem. I was hunting ghosts. I thought that there is some problem in ImageButton or TobbleButton.

Correct solution was to move communication to Android service class. Then bind the service so UI can send commands to the service and the last part was to add broadcast receiver to pass message from service to UI.

Here is small schema which describes delivery of message from broker to app.

mqtt-broker-to-android

It was necessary to make also small update in ESP8266 in order to tell Mosquitto to persist the state of a lamp (the last bool parameter):

mqttClient.publish(topicName("relay"), "off", true);

The result is nice. I can turn on the lamp via web interface and all connected Android apps are updated and user can see the state of the lamp.

lamp-app-mqtt

The code for Android app is available at Github – LampApp. Previous code with REST API was moved to branch v0.1. The code for ESP8266 module is also at Github – LampESP. Feel free to customize the app. You can find instructions in README.md.

I see the next challenge for LampApp: display lamp control widget at lock screen. Maybe I’ll be more successful than the last time.

22. November 2016

IoT and waiting for the sun

It took me some time to implement all features to ESP8266 controlling relay of my lamp.

smart-home-lidl-salt-lamp

Thanks to examples from PlatformIO I was able to use all neat features in one code:

You can find complete code at github repo LampESP. Just replace HOST_NAME_HERE and ROOM_NAME_HERE with your MQTT server and your room name.

Once you deploy code to ESP8266 you can use Over-the-air update (OTA). Just change the value in platformio.ini and set upload_port to IP address of your device.

The implementation allows control via simple GET API by accessing the web of device or LampAndroidApp.

As I wrote few weeks ago, my goal was to add MQTT. I’ve installed Mosquitto and connected the device to the server.

It is possible to watch states of device is possible via MQTT client:

mosquitto_sub -v -h HOST_NAME_HERE -t '/#'

Here is example how to change states of relay:

mosquitto_pub -h HOST_NAME_HERE -t '/home/ROOM_NAME_HERE/command' -m 'on'
mosquitto_pub -h HOST_NAME_HERE -t '/home/ROOM_NAME_HERE/command' -m 'off'

I had an idea about next cool feature: turn on/off lamp based on sunset and sunrise. I found very simple and elegant solution that I would like to share with you.

Download Sunwait and compile it. The program just starts and waits until the time of sunrise or sunset based on geographic coordinates. You can also specify offset (how long before or after event) should program end.

Now you can put the sunwait into your local cron table and chain it with command that should be executed after sunrise/sunset occurs.

Here is example how to configure cron to turn on the lamp 15 minutes before sunset and turn it off 15 minutes after sunrise.

crontab -e
10 15 * * * /usr/bin/sunwait sun down -0:15:00 49.230738N, 16.576802E; /usr/bin/mosquitto_pub -h HOST_NAME_HERE -t /home/ROOM_NAME_HERE/command -m "on"
4 15 * * * /usr/bin/sunwait sun up +0:15:00 49.230738N, 16.576802E; /usr/bin/mosquitto_pub -h HOST_NAME_HERE -t /home/ROOM_NAME_HERE/command -m "off"

The lamp is now working. The only problem is the Android app. I thought that adding MQTT client to Android app will be as easy as adding it to ESP8266, but it’s far from truth. There are still some challenges 😉

19. November 2016

Programming ESP8266 in Go language? Not yet. November 2016

I was wondering if it would be possible to use Go language to write code for ESP8266.

The very first search result seemed to be the right choice – Gobot.

I read the instructions and typed:

go get -d -u github.com/hybridgroup/gobot/...

After a while Go asked for password to git.eclipse.org. I had no account there so I registered the account. And again:

go get -d -u github.com/hybridgroup/gobot/...

New error:

package git.eclipse.org/gitroot/paho/org.eclipse.paho.mqtt.golang.git: cannot download, git.eclipse.org/gitroot/paho/org.eclipse.paho.mqtt.golang uses insecure protocol

Go was trying to get repository from Eclipse using unsecure HTTP. Luckily there was feature request to add switch for accessing insecure repositories by Go get – issue #9637. The solution was just to add “-insecure” parameter:

go get -insecure -d -u github.com/hybridgroup/gobot/...

New error:

# cd .; git clone git.eclipse.org/gitroot/paho/org.eclipse.paho.mqtt.golang .../projects/gobot/src/git.eclipse.org/gitroot/paho/org.eclipse.paho.mqtt.golang.git
fatal: repository 'git.eclipse.org/gitroot/paho/org.eclipse.paho.mqtt.golang' does not exist
package git.eclipse.org/gitroot/paho/org.eclipse.paho.mqtt.golang.git: exit status 128

Yay, that’s cute. I’ve searched through issue tracking system in hybridgroup/gobot and the answer was issue #339.

The workaround is to change line 4 in github.com/hybridgroup/gobot/platforms/mqtt/mqtt_adaptor.go to:

"github.com/eclipse/paho.mqtt.golang"

Ron Evans recommended just to switch to dev branch.

Thanks to Go integration with Git it was very easy:

cd src/github.com/hybridgroup/gobot
git checkout dev
go get -d -u github.com/hybridgroup/gobot/...

Only after this sequence of steps from magic dance Gobot was ready to build application on my computer.

My next step was attempt to build MQTT ping. Easy? Just grab a code from gobot.io documentation, paste it to file and run it.

go run mqtt-ping.go

Another funny error:

# command-line-arguments
./mqtt-ping.go:11: undefined: gobot.NewGobot
./mqtt-ping.go:13: undefined: "github.com/hybridgroup/gobot/platforms/mqtt".NewMqttAdaptor

The documentation on the web is from master. Ou, but I switched to dev branch. What a simple mistake. I grabbed the code from github repo everything was ok.

cp  ./github.com/hybridgroup/gobot/examples/mqtt_ping.go .
go run mqtt_ping.go

The application was able to connect to Mosquitto Broker and send messages.

The last step would be to deploy it to ESP8266. Unfortunately this is not possible yet. The platform is not supported by Gobot and there is issue with plaftom-request #301.

Ok, never mind. It was interesting experiment and I can see potential of Go for programming IoT.

What will be my next experiment? Developing the code for ESP8266 using PlatformIO.

  • Where’s the fish?

  • Translations

  • Further info

  • Twitter

    Follow @jurajmichalek on twitter.

  • Comments

  • Tags

  • Topics