Skip to content

File distribution

Overview

All qbee.io configuration management options can get accessed through the "Configuration" function. Please see the "Configuration management" section for additional information.

This is one of the most complex configuration policies. Please also refer to the question marks behind the form items to get additional information. File Distribution allows you to perform a multitude of different tasks:

  • Ensure that one or more files are distributed to the devices. If the file gets changed on the device it will be replaced with the original one defined as source file from file manager.

  • Re-distribute one or more files if they change in file manager

  • Convert a configuration file into a templating file and do key-value templating. This allows for example to substitue different parameters in config files depending on which group the devices belong to.

  • Create any kind of script, play it out through file distribution and run it

  • Trigger run-once actions

Basic file distribution

In general any file that should be distributed can be uploaded through the file manager. In the file distribution UI it is important to define the correct source and destination path. This is always the full path. A file is then played out to the according destination. This file has a checksum. Any changes to the file will be detected by the qbee agent and causes the file to be overwritten with the original version from the source path. This effectively restores any changes to the original version automatically as well as that it allows to play out new updated files.

Info

Any change to the file on the local device will be reverted by the next run of the qbee agent.

Warning

Please be aware that qbee agent checks if a file checksum on the target machine is different than the file checksum in the qbee file manager. This can consume a lot of bandwidth if either one of the following cases appears:

  • A file on the target changes between qbee agent runs
    This causes qbee to download the file again and again
  • A file is written to a temporary directory
    If a file is donwloaded to /tmp on the target and /tmp is a temporary directory that is cleaned after every reboot then the qbee agent will download the file again and again after each reboot. So, when using file distribution we recommend to put installation files in a permanent directory to avoid extensive downloads.

The first time a file is played out or any time there is a change of the file on the device or in the key-value configuration the "command to run" is executed.

Info

It is possible to run chained commands like && :
command A && command B

It is possible to run a command in the background with & :
command A &

It is possible to run complete scripts. They need to be distributed first. Usually they are written as non executable. Therefore you can execute them with bash or you need to do a chmod
bash /usr/local/bin/my_script.sh

It is possible to run python scripts with python or python3:
python3 /usr/local/bin/run-me.py

If files are played out they might need a chmod or chown command as "command to run" to receive the correct file permissions or ownership. Please do not use chmod on a templated file:

chmod 755 myfile.txt

chown pi myscript.sh -> changes ownership for user

chown pi.pi myscript.sh -> changes ownership for user and group to "pi"

Sometimes it makes sense to run the script in the background and pipe output messages to /dev/null. This can be done as follows: Use "Command to run" as bash /usr/local/bin/cpu_temp.sh > /dev/null 2>&1 &. This runs the script in the background and the appendix > /dev/null 2>&1 & pipes bash output to /dev/null.

Templating and chmod will NOT work

If templating is used you CANNOT do a chmod on that file. This will cause a recurring execution. So it is not possible to template a script, do a chmod on the same file and then execute it. It is also not possible to template a config file and do chmod. Please use a cpcommand or the technique described here. This is an example for a config file: cp /some/temporary/collection/location/monitrc /etc/monit/monitrc && /bin/chmod 0700 /etc/monit/monitrc

Files are played out in sequential order

Multiple files can be played out. They will be played out in sequential order. So it is possible to play out a tarball and then play out an installation script in the second file distribution and run this with the second "command to run" definition. Only after that the first file has been played out the second file will be downloaded and run.

One-off file distribution for files that will be changed on the target device

Sometimes it is needed to deliver a file to a target device that can be changed on the target without being re-deployed and over-written. We call this a one-off file distribution. The recommended way to solve this is to deliver a file "file1_tmp" to the device. Then deliver a "script.sh" that checks if the file "file1" exists on the device. If so it does not do anything. Otherwise it will copy "file1_tmp" to the correct location as "file1". After that it will never re-distribute the file even if "file1" is changed locally. Just deploying "file1" directly would result in a new file distribution and an overwrite process any time the file is changed on the device. This can also be solved with a command to run:

[ ! -f /home/pi/text.txt ] && cp -p /home/pi/text-base.txt /home/pi/text.txt 
Please see here for an example.

Adjusting file ownership - how to change files from being owned by root

The qbee agent will distribute files with the owner root. The correct onweship and file permissions need to be set with chmod and chown commands. Using chown pi.pi myscript.sh sets both user and group ownership to pi.

Running scripts / software in a user context

Sometimes it is important to run a software or script in a user context and not as root. This can be done as follows through the command to run box (here for user "pi"):

chown pi.pi /home/pi/my-application && sudo -u pi /home/pi/my-application

and here as a service

sudo -u pi node-red-restart

Advanced file distribution with templating

One very useful feature of qbee.io is to expose parts of regular config files for remote configuration through key-value pairs. This allows for many different use cases. Instead of pushing out different configuration files for different customers you can use the same basic config file and then configure some configurations individually. One example could be that different customers need to get directed to different MQTT servers. This can be very easily done. Just take the original configuration file and expose any accessible parameters through mustache notation - two double brackets as displayed here {{server}}. A thorough example can be found here

Original config file configuration.conf:

[broker]
username = "my_username"
password = "James Bond"
server = "customer_a.mqtt.someserver.com"
port = 8883
tls = 0

[LPWAN system]
system-id = 12

Now it is possible to access one or all configuration options of this file remotely. Please copy this file. Let’s call it configuration.conf.tmpl. Then all configuration parameters that are in scope need to be exposed with {{your_key}} to expose them as key-value in the UI.

[broker]
username = "{{username}}"
password = "{{password}}"
server = "{{server}}"
port = 8883
tls = 0

[LPWAN system]
system-id = {{system-id}}

This file can now be played out. It is then possible to select a specific group of devices from the group selector and assign that group a different mqtt server or a different user name or password.

Tip

Structure your device groups in a way that it fits your configuration needs. For example
RPI → Customers → Customer_A → Test
RPI → Customers → Customer_A → Production
Then you can use different MQTT servers for testing and production.

qbee-file-distribution-embedded-device2

Please note that the "Command to run" gets executed any time a parameter is changed. Usually this command would restart the application. This in return reloads the configuration file and allows the new configuration to become active. In the following example you see the feedback the system has generated on a successful reconfiguration of a node-red flow file:

qbee-file-distribution-embedded-device3

Tip

Only expose the parameters that you really are interested in. They can be anywhere in a long configuration file. This way you can expose the three or four relevant parameters within a configuration file with many hundred lines.