In this blog post we present a complete out of the box and ready to use solution on how to read Modbus data from a device where qbee is installed and visualize that data locally. You can also change the configuration such that you send the Modbus data to a different cloud service. The solution consists of three standalone deb packages that are custom built. Each package comes with its configuration files which are templatized. This means that some settings are exposed as key-value pairs in the qbee.io configuration engine. This makes it very easy to change behaviour for a single device or a group of devices (maybe your customers have different requirements or different Modbus devices). Additional configuration can also be exposed and this is a blueprint of how to do this with other packages as well. We provide the entire source code in our GitHub repository and you can interchange the parts to your liking.
In the following we present the architecture and then give an overview of the custom built packages which are based on the open source Python libraries modbus4mqtt and http-plot-server.
At the end of this blog post we show how to set up the configuration using qbee.io’s software management functionality and illustrate how to use GitHub actions to build and distribute those packages. But they can also be used as packages in the file manager for simplicity.
The architecture
Below we show the underlying architecture of our solution.
The solution consists of three custom built deb packages, namely:
- plotserver
- mqtt2plotserver
- modbus4mqtt
Which we describe in more detail below. The general idea is that we have independent parts for data acquisition and visualization, namely modbus4mqtt and plotserver.
The modbus4mqtt package can be configured to read the desired modbus registers in a specified interval and send the data to a specified MQTT broker.
The plotserver package accepts HTTP POST requests to obtain data and visualizes the data on a configurable port (here 8080) for a configurable retention period (per default 100 data points).
As an interface we have built a small utility that we call mqtt2plotserver which subscribe to a configurable topic of a specified MQTT Broker and sends that data as HTTP POST requests to the specified plot server. The data transfer is done through a locally running MQTT broker. We can use unencrypted traffic as we are not sending any MQTT data over the internet.
The reason for using three different packages instead of a single one that goes directly from Modbus to a visualization tool is that we can make use of ready to use open source third party libraries which we simply transform in configurable system services. We can easily exchange blocks according to the requirements without having to rewrite a monolith software.
Package 1: Plot server
The plotserver package uses the http-plot-server Python package and turns it into a system service. Using qbee.io’s software management functionality (explained below) you don’t have to deal with monitoring the service. qbee.io makes sure it is running. Furthermore, the package comes with a configuration file that is templatized and the key-value pairs are exposed in the qbee.io UI. The config and package files are available in our GitHub repository.
Package 2: MQTT to plot server
This package is completely custom built and serves as an interface between the modbus4mqtt module and the plotserver module. It is essentially a simple Python file that connects to an MQTT broker and subscribes to a specified topic. The MQTT and plot server settings are exposed to a configuration file.
We made the mosquitto MQTT broker a dependency of this package as we want to us a local running MQTT broker. For simplicity and ease of presentation we used a passwordless set up but it is no problem to define MQTT users.
Package 3: Modbus to MQTT
This is the core module of the application and based on the publicly available Python package modbus4mqtt developed by tjhowse. The package abstracts the pymodbus library to run with Modbus configuration files and send the data in a certain format to a specified MQTT broker. We have built a wrapper around this library to run it with a configuration file instead of command line parameters and run it as a system service. The code and configuration files are available in our GitHub repository. Certain parts of the configuration files are exposed in the qbee.io UI.
qbee.io's software management
We deploy the the 3 debian packages using qbee.io’s software management. This way the system services are taken care of without further ado and we can deploy templated configuration files. That means if we make a change in one of the parameters the service is restarted automatically with the new parameters without having to execute the necessary commands manually.
The package dependencies are installed automatically on package distribution and we get a feedback of the installation process in qbee.io’s log page.
Below you can find the full configuration json containing all three packages from above.
{
"enabled": true,
"items": [
{
"package": "https://d5af23a0.delivery.rocketcdn.me/modbus-mqtt-plotserver/plotserver/plotserver_0.0.1_all.deb",
"config_files": [
{
"config_template": "https://d5af23a0.delivery.rocketcdn.me/modbus-mqtt-plotserver/plotserver/param.cfg.template",
"config_location": "https://d5af23a0.delivery.rocketcdn.me/etc/plotserver/param.cfg"
}
],
"parameters": [
{
"key": "host",
"value": "0.0.0.0"
},
{
"key": "port",
"value": "8080"
},
{
"key": "max-points",
"value": "100"
}
]
},
{
"package": "https://d5af23a0.delivery.rocketcdn.me/modbus-mqtt-plotserver/mqtt2plotserver/mqtt2plotserver_0.0.1_all.deb",
"config_files": [
{
"config_template": "https://d5af23a0.delivery.rocketcdn.me/modbus-mqtt-plotserver/mqtt2plotserver/config.yml.template",
"config_location": "https://d5af23a0.delivery.rocketcdn.me/etc/mqtt2plotserver/config.yml"
}
],
"parameters": [
{
"key": "mqtt_topic",
"value": "qbee/Temperature"
},
{
"key": "plot_server_port",
"value": "8080"
}
]
},
{
"package": "https://d5af23a0.delivery.rocketcdn.me/modbus-mqtt-plotserver/modbus4mqtt/modbus4mqtt_0.0.1_all.deb",
"config_files": [
{
"config_template": "https://d5af23a0.delivery.rocketcdn.me/modbus-mqtt-plotserver/modbus4mqtt/config.yml.template",
"config_location": "https://d5af23a0.delivery.rocketcdn.me/etc/modbus4mqtt/config.yml"
},
{
"config_template": "https://d5af23a0.delivery.rocketcdn.me/modbus-mqtt-plotserver/modbus4mqtt/temp_sensor.yml",
"config_location": "https://d5af23a0.delivery.rocketcdn.me/etc/modbus4mqtt/temp_sensor.yml"
}
],
"parameters": [
{
"key": "MQTT_HOSTNAME",
"value": "localhost"
},
{
"key": "MODBUS_CONFIG_FILE",
"value": "https://d5af23a0.delivery.rocketcdn.me/etc/modbus4mqtt/temp_sensor.yml"
}
]
}
],
"version": "v1"
}
You can modify this configuration to your needs, for example to only include the Modbus data acquisition package in case you are using an external MQTT broker. Or you could exchange the plot server in case you want to use a different data target.
Package creation using GitHub actions
Our GitHub repository contains an actions workflow file to build the three deb packages inside a GitHub runner using the fpm action and then upload them together with the templatized configuration files to the qbee.io file manager. In addition we distribute the above shown software management configuration file to the given group of devices.
However, you can apply the software management configuration manually in qbee.io itself and build the deb packages locally. We recommend using the following Docker container such that you don’t have to deal with the installation process on your personal computer. The fpm commands for building the deb packages are available in the workflow file of our repository.
