The second part of our PLC tutorial explains how to automate PLC deployment and build a continuous integration / continuous delivery (CI/CD) pipeline. As an example we use OpenPLC and qbee. This blog post is building upon using a Raspberry Pi as PLC with OpenPLC. A PLC or programmable logic controller is still a base building block of most IIoT industry 4.0 use cases. However, deploying and automating a large number of PLCs efficiently is still a challenge for many industrial companies.
In the following paragraphs we will investigate and solve these PLC use cases that are associated with automated PLC deployment with CI/CD:
- Automatic roll-out and installation of a PLC runtime on a fleet of edge devices
- Enable auto-start of PLC programs without manual interaction
- Continuous delivery of PLC programs to the edge
- How to build a complete CI/CD pipeline for PLCs with Github as an example
- Using qbee’s advanced key-value templating to customize PLC programs based on group membership – adjust for PLC port changes or other variables
The automated roll-out and installation of a PLC on a remote device has been shown here. In that case we used the built-in qbee VPN to access the OpenPLC webserver remotely and we manually uploaded our “hello-world.st” program. It was also important to configure the PLC webserver. We had to select the hardware platform (defined as “Raspberry PI”) as well as to enable automatic PLC program run on start-up. So in reality this was not a fully automated installation.
Continuous delivery of PLC programs to the edge
We identified three shortcomings that will be addressed here:
- Configure the OpenPLC webserver to run PLC programs automatically
- Define the hardware platform
- Distribute and run the PLC program “blank_program.st”
After the first installation OpenPLC keeps your login credentials as well as all the other information in an sql-lite database. So we need to configure one OpenPLC instance manually as described below and save all settings. Then we copy the file ~/OpenPLC_v3/webserver/openplc.db from the device and upload it to the qbee file manager into the “openplc” folder. In the video below we use the qbee-connect VPN to both get remote access to the web application and we use a SFTP tool to remotely download the file. This makes the whole operation very convenient.
Now the sql-lite database is added to the qbee file manager. We continue to build on the “Hello-World.st” example from OpenPLC. We just rename it to blank_program.st to comply with what software we configured for autostart. Again, the base example is changed to use a different port (makes it work for the RPI) and we call the program blank_program.st, both as file name and in the source code:
PROGRAM blank_program
VAR
my_button AT %IX0.2 : BOOL;
lamp AT %QX0.0 : BOOL;
END_VAR
VAR
TOF0 : TOF;
END_VAR
TOF0(IN := my_button, PT := T#2000ms);
lamp := TOF0.Q;
END_PROGRAM
CONFIGURATION Config0
RESOURCE Res0 ON PLC
TASK task0(INTERVAL := T#20ms,PRIORITY := 0);
PROGRAM instance0 WITH task0 : blank_program;
END_RESOURCE
END_CONFIGURATIONNow we use the qbee file distribution from the previous post. Here we defined a script for automatic installation. What we do next is to define a second file distribution that exchanges the sql-lite database on the device with the one that we configured and downloaded. Thus, all devices will have the same password and settings. qbee is also monitoring this file. So if anyone manually changes anything on any edge device it will be reverted back by the qbee agent since it operates state based (please refer to the qbee documentation for more information).
As a last step we upload a shell script called “update-openplc.sh” that compiles any new blank_program.st automatically by calling a script available in OpenPLC. As explained previously, the qbee agent understands if a new file is uploaded to the file manager. Then it will be transported to all devices in scope and the script below is invoked and run through the “command to run” bash /home/pi/update-openplc.sh which will be executed after the file distribution.
#!/usr/bin/env bash
cd /home/pi/OpenPLC_v3/webserver
systemctl stop openplc
/home/pi/OpenPLC_v3/webserver/scripts/compile_program.sh blank_program.st
systemctl start openplc
exit 0The shell script stops the PLC, compiles the .st file and starts the PLC again. Since it is configured to automatically start the blank_program.st this is exactly what will happen. The file distribution will then look like this.
Let’s take a short recap. We install OpenPLC, we provide a pre-defined database (with correct credentials and setup) and then we distribute a “blank_program.st” to the remote device, compile it there and run it. qbee reports this as a log event and even allows to inspect the output of the PLC compiling by pressing on “show logs” (as can be seen below). If we now replace the “blank_program.st” with a new version in the file manager this will be distributed and run on all devices in scope. The installation and database copy operation will not take place again since nothing in the involved files has changed.
How to build a complete CI/CD deployment pipeline for PLCs with Github as an example
So how can we turn a PLC deployment into a continuous integration / continuous delivery (CI/CD) solution? This is very easy to do. qbee offers a REST-API and this allows to upload files automatically to the file manager which then triggers the new file distribution with the new script execution. In our documentation we have a few use case doing this with doing CI/CD with Github. In that example a Github runner is triggered any time a new file is pushed to production. The runner then uses the qbee API to securely transport the file to the file manager and from there it is distributed further to the devices and the according script is executed. So here we have build a complete CI/CD pipeline for PLCs that allows very reliable and flexible operation of large numbers of programmable logic controllers. If you use Jenkins or any other tool you can just use the API to connect to qbee. Please reach out if you need help to build this.
Using key-value templating to customize PLC programs based on group membership
So far we have enabled you to deliver and run PLC files on thousands of devices with OpenPLC. The larger your deployment base will grow the more inhomogeneous it will probably become. With qbee you can build logical group of devices and these can have completely different configurations (for example different “*.st” files) or we can use a common file and use a technique called templating.
Templating is about making certain parts of a script or a code file accessible through mustache notation (this is basically a double curly bracket in which the {{key}} is then later defined by a value). Please refer to this article for further information.
In our demo PLC program we use a switch to trigger a LED to turn on for a certain period of time. In order to demonstrate key-value templating in a PLC context we use the time value as an example. But any other part can be used as well. If you have devices with different pinout you can easily define the output or input port as a template variable. In order to make the on-time of the lamp adjustable we expose this in the code as seen below. We replace the on-time of 2000ms with the mustache expression {{on-time}}. We could have just as well used %IX0.2 as {{input-pin-switch}}:
PROGRAM blank_program
VAR
my_button AT %IX0.2 : BOOL;
lamp AT %QX0.0 : BOOL;
END_VAR
VAR
TOF0 : TOF;
END_VAR
TOF0(IN := my_button, PT := T#{{on-time}});
lamp := TOF0.Q;
END_PROGRAM
CONFIGURATION Config0
RESOURCE Res0 ON PLC
TASK task0(INTERVAL := T#20ms,PRIORITY := 0);
PROGRAM instance0 WITH task0 : blank_program;
END_RESOURCE
END_CONFIGURATIONWe call this file “blank_program.tmpl” to indicate that it is a template and upload it to the qbee file manager. The ending is just for readability, it will be written as indicated in the destination path. There it needs to be written as “blank_program.st” as defined in our OpenPLC settings. But this file is first created on the target edge device and the qbee agent will replace the key with the value (in this case 4000ms). So the device will see a “blank_program.st” with the correct value in there. The next thing we need to do is to change the last file distribution and allow templating as seen below:
By default the qbee agent runs every 5 minutes. The next time this happens it will see that there is a new configuration. The agent will apply this and run a PLC program with 4s turn on time. In the video below you see the output of the PLC for 1s and then we changed it to 4s. The update happened fully automated and the PLC obviously restarted showing a 4s length in the second part of the video.
Summing up this blog post it shows how easy it is to turn a simple PLC or even OpenPLC into a fully automated PLC deployment with a continuous integration / continuous delivery system that can be deployed to thousands of units and maintained very easily. I would like to thank Thiago from OpenPLC for starting such a nice project and also for answering some questions I stumbled upon.
