Building a Home Energy Dashboard with Node-RED, MQTT, InfluxDB and Grafana

I recently started building a local energy monitoring dashboard for my home solar setup. The goal was to take live data from my Solis inverter, process it in Node-RED, publish it through MQTT, store it in InfluxDB, and visualise it in Grafana.

This post outlines the first working version of that setup, and is a continuation of the initial retrieval of data from Solis into NodeRed

The Overall Architecture

The final data path looks like this:

Solis inverter
  ↓
Modbus TCP
  ↓
Node-RED
  ↓
MQTT / Mosquitto
  ↓
InfluxDB
  ↓
Grafana

The idea is to keep MQTT as the central message bus. Node-RED reads and decodes the inverter data, publishes useful values to MQTT, and also writes those values into InfluxDB for historical storage. Grafana then reads from InfluxDB to provide dashboards and charts.

User Presence Detection using Node-Red and Unifi

A big challenge in Home Automation is implementing a reliable mechanism for presence detection - figuring out who's home, and tailoring logic accordingly.

Here's how I do it using Node-Red, Unifi and MQTT.

Many approaches to presence detection rely on interacting with the device itself, either by way of software running on the mobile handset, or having the mobile ping or poll an external service. None of these approaches really appealed to me as they are fiddly, rely on handset configuration & maintenance and need to be reset every time someone gets a new phone.

The approach I've taken here is to rely on the fact that all the families' mobile phones are configured to connect to home WiFi (to save on data charges!). This method interrogates the home network to see what phones are present - a good indication that the person is home.

In short, I have a Node-Red flow that polls the network on a schedule, looks for named devices, and updates presence status in MQTT based on that.

I could have used a database, or flat file storage for storing state, but as I have an MQTT broker running anyway, it's a convenient and flexible approach for me.

The Grand Plan

Welcome to Mediaserver8, a place where I document my adventures in home automation and media serving - mostly for my own reference and future sanity!

I'm going to keep this post on top and maintain the overview schematic at Gliffy to offer an introduction to my HA system as it evolves. (Here's a direct link to a scalable version of this diagram).

Card

Read on for a detailed breakdown of this setup...

unRAID as a software dev. platform

unRaid continues to amaze.

I find myself in the middle of a spare-time software development project that quickly grew from a personal home automation idea to a quite public 'when's it ready' kind of thing. It's the development of an Alexa Skill to allow spoken commands to an Amazon Echo to control Squeezebox devices. (see Hab-Tunes.com for the project details).

openHAB debugger

Having previously set up the infrastructure for connecting my legacy Comfort system to openHab, I've started working with openHab items, sitemaps and rules to respond to sensor status updates. In doing so, I've ben working with scripts in openHAB and have been a bit frustrated by the level of debugging available. With MQTT and Node-Red in place, I wrote a simple facility to easily trace code objects and values.

Seeking Comfort

Embarking on the great home automation project of 2016, I wanted to get my openHAB installation talking to my legacy Comfort system.

I'd installed the comfort system 15 years ago, (it's a Comfort Pro), and it's been very reliable if somewhat basic since. At it's core, it's a home automation system that can function as a security system, telephone answering machine, lighting controller, infra-red blaster and a number of other tasks.

It's got a range of modular input and output systems and in my case, connects to the outside world via an RS232 serial interface. (USB and Ethernet options are also available but I don't have them). The system can be programmed and works on the basis of zones/inputs and responses so can be set up to, say, switch a light on if a particular sensor is tripped after dark.

I have window and door sensors connected up as well as a few PIRs as well as an X-10 interface for lighting control. There's a doorphone at the front door and the system is connected to the telephone line. This means that if we're not at home and the doorbell rings, we can set the system to call a mobile and we can speak to the person at the door. Neat.

Comfort has been updated over the years and version 2 has been out for a while with support for more modern protocols etc. but as an upgrade would require me to replace the entire system, I didn't bother.

Now with a renewed interest in home automation and a decision to pursue openHAB as the core, I needed to find a way to get Comfort and openHab linked up. I really don't want to throw out a perfectly good alarm system and have to rebuild it all again with new components just to get the same functionality. My ambition is to reuse and recycle what I have if it's fit for purpose.

MQTT bi-directional communications in PERL

In attempting to interface openHab with my Comfort system, I needed to adapt an existing Perl linux driver for Comfort to work in Windows. The idea is that a small Perl program manages serial communications with the Comfort system and uses the IoT protocol MQTT to pass messages back and forth with openHab.

In doing so, I found that there was a very limited amount of information on integrating MQTT with Perl. There is a minimal MQTT interface for Perl that does the trick but the sample code was not obvious to me and there's very little else out there.

I've written a vey basic Perl <-> MQTT program to show how basic input and output is achieved.