Templating MQTT connection information

Here we show templating with software management. Please also check out file distribution examples!

Templating is one of the key features of qbee. It can be used within software management and file distribution to achieve configuration templating, to trigger scripts, to control script parameters and do much more. Most people will use this from file distribution but it is also part of software management.

A customer has build a wireless sensor network based on a proprietary radio protocol called TinyMesh. The radio receiver is integrated into an industrial Linux controller based on a Debian Linux. In this case it is actually an industrial Raspberry Pi with a 4G modem. The collected sensor data is transmitted via MQTT over a 4G mobile network into the Tingsense IoT platform. On the cloud platform a MQTT broker is running to receive the sensor information. The MQTT application is delivered remotely to a fleet of devices through an over the air software update mechanism. Then the application is configured through the built-in templating engine in qbee With regards to the MQTT server the following considerations were taken into account:

  • Currently the third party CloudMQTT service is used. Each final customer has a dedicated MQTT instance to isolate the different MQTT streams. At a later stage the qbee customer considers to operate an own MQTT server.

  • All TinyMesh gateways for different customer should be operated within one interface but connect to the different CloudMQTT instances based on a per customer basis

  • One TinyMesh package and configuration file should be installed. It should be possible to remotely update this

qbee solves this in a simple way.

Currently, the latest TinyMesh software is version tinymesh-agent_0.0.3_all.deb. It installs a configuration file into the path /etc/tinymesh-agent/tinymesh-agent.conf.

This file contains information about the MQTT server. A simplified version of the config file is shown here.

[broker]
username = "my_username"
password = "James Bond"
server = "customer_a.mqtt.someserver.com"
port = 8883
tls = 0

[LPWAN system]
system-id = 12

In order to be able to template this Mustache notation is used. All relevant information that should be accessible through a UI is templated with double curly brackets.

[broker]
username = "{{username}}"
password = "{{password}}"
server = "{{server}}"
port = 8883
tls = 0

[LPWAN system]
system-id = {{system-id}}

What to template

In this (simplified example) both username and password as well as server URL and system-id can be set through the qbee UI. This allows to build a tree with devices for different customers that can use the same installation package but will receive customized information based on their location in the tree. In addition it is also possible to adjust the system-id individually (this is setting the radio channel) in order to use different channels in the same building.

qbee-customer-tree

This allows to granularly access and configure the devices on each tree level in the qbee UI. The following image shows how the key-value pairs are defined. These will be transformed into individual valid configuration files for each device.

qbee-mqtt-templating

Template only the key-value pair you need

Template any key-value pairs in any configuration file. Even if the file has many 100 lines it is possible to expose only the relevant ones. These can be defined in the UI.

Receive audit and log entries about any change made to the system

Any change to the key-value configuration will give a log entry about which device has received and activated the new configuration. In addition also an audit entry is made about who made a change and what the change is.

Here is a log entry about two Raspberry Pi devices receiving new templating information, writing the new log file and automatically restarting the application to make the log file change valid. Two devices received the new configuration and user Carsten initiated this on 19-10-2019 @ 11:39:11.

qbee-mqtt-software-management

Updating the software version and recreating the config file

If the application has a software update a new version of the app can be uploaded to the qbee file manager and defined as new software package. In this case this could be tinymesh-agent_0.0.4_all.deb. This would overwrite the config file during the installation but qbee will automatically use the template and correct this as soon as the application is updated. Thus the individual configuration can survive software package update cycles without needing to put this into a specific place.

Templating allows you to work with universal spare devices

If the customer needs to send a spare part because a gateway is broken the gateway can be send from an intermediate storage facility. As soon as the device is powered up it appears in the qbee tree and can be moved into the correct position in the tree. It will then automatically receive the correct configuration and of course the latest version of the software.

What else can be mentioned about this customer case? Additional reasons for the customer to choose qbee were:

  • new software packages can be tested and rolled out in a staged way starting with test, QA and then roll out per customer. This reduces the deployment and upgrade risk substantially

  • the 4G mobile network sometimes lost connection. Using the qbee watchdog this is easily fixed without needing any additional development on the device

  • if a new vulnerability in any of the libraries on the device is detected this is will be flagged immediately. Apart from that update of the firmware can be kept to a minimum.

  • during testing of new application functions the packages could be deployed to the remote test devices. These devices transmit all their metrics and even process information. This makes it possible to improve and quality check the application in the field very easily. In one case this helped the customer to detect a memory leak in the app.