Deploy and configure Node-Red

This how-to explains how qbee.io can help to deploy Node-Red applications to distributed remote devices. Node-Red is an extremely powerful tool and we will only give some basic insights into what and how you can use this to gain very secure and granular control over your distributed SCADA or operational technology (OT) deployments. qbee.io will enbale Node-Red to do its magic behind firewalls, NATs, mobile networks or other real industrial developments lifting it from the lab to the real work. And yes, Node-Red is stable enough to run in production (and qbee is governing it while doing so).

Tip

qbee.io and Node-Red work extremely well together. In this tutorial we will focus on the following tasks:

  • get control over Node-Red settings
  • allow system wide credential access
  • enable automatic updates of flows on remote target devices
  • allow templating of Node-Red flow properties
  • create a watchdog layer around your deployment

This is all done on a Raspberry Pi (RPI) as this is widely available. But it will work on any other platforn as well.

The first thing you want to do is to adjust your settings file. On Linux the settings.js file is copied from the /usr/lib/node-modules/node-red/ folder to your .node-red folder in your home directory when you start node-red, and there is no settings.js file present. This file contains a lot of options. Here we will only look at the ones that are relevant for describing our base use case. There are very good tutorials on the internet how to tweak and adjust additional settings. The settings file itsself can also be explored in its latest version on github.

Tip

We recommend to base your settings.js file on the one that you get from your version of node-red (the path defined above), but it should be compatible across versions as well.

Warning

There is one setting in Node-Red called credentialSecret: "a-secret-key". Once you set this property, do not change it - doing so will prevent node-red from being able to decrypt your existing credentials and they will be lost. It seems that if this is set once it cannot be changed easily. We circumvent this problem by playing out a defined settings.js and the according flows_creds.json to all devices.

Let's get started. Please install Node-Red on your development machine and start Node-Red.

This will copy a settings.js file into your .node-red folder. Open this and edit the following items: Remove the trailing "//" to activate this the following. This basically removes the dependency of the device name out of the flow name and makes life easier.

//flowFile: 'flows.json',

Note

The following change encrypts the credentials used in your blocks. This is very useful, but consider to do this in a second step. If you set this to false everything will be saved in clear text and things might be easier.

Do the same with the following and insert your secret key.

//credentialSecret: "a-secret-key",

Thus this line changes into

credentialSecret: "my-personal-secret-key",

Then save this file again and restart Node-Red. Now it is time to fire up Node-Red and create a basic flow. We will build a small weather station that measures the quality of the weather (string) and the temperature (number). An inject node, a function node and a debug node is needed for each. This is how the flow will look:

qbee-node-red-flow1

The complete flow can be seen here and be copied into your Node-Red editor:

[{"id":"3bd341c4.4b018e","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"df6b029a.8abbd8","type":"inject","z":"3bd341c4.4b018e","name":"weather","topic":"weather","payload":"good","payloadType":"str","repeat":"10","crontab":"","once":true,"onceDelay":0.1,"x":171.5,"y":175,"wires":[["f7d0552a.308d48"]]},{"id":"f7d0552a.308d48","type":"function","z":"3bd341c4.4b018e","name":"weather station","func":"var payload = msg.payload;\n\nmsg.payload=\"The weather is \"+msg.payload;\n\nreturn msg;","outputs":1,"noerr":0,"x":373.5,"y":175,"wires":[["1c683a18.c636ce"]]},{"id":"1c683a18.c636ce","type":"debug","z":"3bd341c4.4b018e","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":586.5,"y":174,"wires":[]},{"id":"908d10c2.c9368","type":"inject","z":"3bd341c4.4b018e","name":"temp","topic":"temp","payload":"12","payloadType":"num","repeat":"10","crontab":"","once":true,"onceDelay":0.1,"x":160,"y":256,"wires":[["c46cd335.f411a8"]]},{"id":"c46cd335.f411a8","type":"function","z":"3bd341c4.4b018e","name":"temp station","func":"var payload = msg.payload;\n\nmsg.payload=\"The temperature is \"+msg.payload;\n\nreturn msg;","outputs":1,"noerr":0,"x":365,"y":255,"wires":[["b56ad264.a1b36"]]},{"id":"b56ad264.a1b36","type":"debug","z":"3bd341c4.4b018e","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":590,"y":254,"wires":[]}]

The weather and temperature are hard-coded for now. The relevant injectors and functions look as follows:

qbee-node-red-weather

qbee-node-red-temperature

qbee-node-red-function

Please run the flow locally and you can change both weather or temperature and see this in the debug output.

Tip

This flow is a static flow and this could be played out to a large number of devices. How this is done will be explained below. But we don't want to bore you and add another interesting step.

Adding the magic - qbee templating

qbee has a templating engine that can identify any value in any configuration file and make it accessible as a key-value pair. This is done by using Mustache annotation with double curly brackets around a variable. So the "good" weather turns into the variable {{weather}} and the 12 degrees turn into {{temp}}.

Let's revisit the flow and change into qbee's templating format with {{weather}} and {{temp}}.

qbee-node-red-weather-mustache

qbee-node-red-temperature-mustache

Since we change the number into a string we get a warning message. Please ignore this for now and deploy anyhow. This writes the flows.json. Alternatively you copy the flow code from the Node-Red UI into a browser.

qbee-node-red-warning

Please copy and rename or use the new templated node red code to create a file called

flows.tmpl

Now you upload the settings.js and the flows.tmpl to the qbee file manager. The default directory needs to be used. This is software-management/config-templates.

Tip

The qbee file manager does not work with hidden directories like .node-red yet. Please copy the files to a non-hidden location and upload from there.

Now everything is prepared to play out files and configuration to real target machines. In many cases this works but for the following pathes we rely on Node-Red running form the /home/pi user directory. If you eventually find an empty flow it could be coming from root. The following tip explains how to get a clean and updated installation on your RPI.

Installing Node-Red on the Raspberry Pi

The easisest and most recommended way to get a working Node-Red installation is to follow the recipe on this page. If you have an installation from before just delete your .node-red directory as user pi. Then run the command form the web page: bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)

In the devices tab of qbee we create a tree structure with one or more devices to experiment. In this case we define two groups "warm" and "cold" that should display each their weather based on the same universal flow. Any device we ever move into the warm group will then automatically be configured to adhere to the universal flow with the values for warm while all devices being moved into the cold group will get a cold setting. For a reference about the qbee tree concept please look here.

qbee-node-red-targets

Now we can start defining the configuration to achieve what we want. Go into the Configure menu of qbee and navigate to the warm group in the selector on top.

qbee-node-red-configuration

In software->software-management please enable this for this group level and select the following:

Software package "nodered"

Tip

This downloads and installs the standard node-red package from the device's existing repository. If you rather want to install a specific version that you control or your repository does not have node-red available you can upload this as a .deb package with standard debian naming conventions to software-management/packages Then that version is installed and used.

Then add a configuration file template. Use

flows.tmpl

and the configuration file location is

/home/pi/.node-red/flows.json

/// note Obviously this path is device and user specific. Please adjust accordingly if you are not running user pi on a standard RPI.

Then on the right hand side add two key value configuration boxes and use (one on each line)

weather : warm
temp : 25

Then add another configuration file template on the left with

settings.js

and

/home/pi/.node-red/settings.js

Please save and commit and wait one run interval of the qbee agent. In the log files you will see that the both the flows.tmpl file was transformed into the flows.json with the appropriate values, that the settings file was played out and that nodered was installed and/or restarted automatically.

Tip

Any change in configuration causes qbee to restart the service thus making configuration valid automatically.

qbee-node-red-deployed

Did it work? Well, let's see by going to the remote device and press on the debug pane. Now there should be a weather state and a temperature. There is a manual for remote web access here.

Ok, then let's do the same thing for group cold. It is the same setup as above. Obviously the weather should be "cold, ugly, or whatever weather you dislike". qbee AS, the company behind qbee.io is from Norway. So we are quite used to anything cold weather might bring.

qbee-node-red-cold

Tip

At this point in time it is possible to define groups of devices running node-red and provide them with a common flow. The templating variables will get set through the qbee UI. Any change is logged in the audit trail of qbee so it is possible to see which user has set something and what it actually was.

qbee-node-red-change-audit

Let's make things slightly more complicated by piping this data out to an MQTT channel. We are sometimes using CloudMQTT for MQTT services. There you can also get a free MQTT account for testing.

Use a Node-Red MQTT output node and configure it to send data to your MQTT server. Then deploy. Obviously, if you do this on your local machine your values will look like this because they are not defined:

qbee_cloudmqtt

But the good thing is that you have a stable connection. Introducing the MQTT block with the credentials (and pressing deploy) triggers the creation of a new flows_creds.json file that uses the credentialSecret introduced in the beginning to hash the credentials. In addition flows.json of course is updated as well. Now we can go into the file manager and delete our flows.tmpl. Define the new flows.json as flows.tmpl and upload together with the flows_creds.json. Then introduce the flows_creds.json to the configuration engine for both groups as shown here.

qbee-node-red-with-credentials-file

Every device that is moved in each of the groups will now receive the according specification and the MQTT messages will come in with the different weather conditions and temperatures:

qbee-node-red-MQTT

As a final remark it can be said that groups of devices can be configured with the same flow and specific properties can be changed as needed. This could be different update settings (fast on ethernet, slower for devices on mobile network), machine configurations or anything else. By moving a device into the correct group it will receive the correct settings. Any change is registered in the audit log. This records also the values. If anything changes the node-red process is restarted automatically and this is recorded in the logs as warning.

Tip

A watchdog layer was mentioned initially. The good thing with qbee is that the software-management part will automatically monitor the service (in this case nodered) and restart either if a change occurs or if the process is down.