Automatic Node-RED flow deployment with GitHub

github-badge nodered-badge

In short

In this example we show how to use GitHub to deploy a Node-RED flow to a fleet of Raspberry Pis. The flow are distributed and managed including versioning control. The process starts by pushing the changes of your Node-RED flow to your GitHub repository. Then, using GitHub actions, the flow is placed into your qbee.io file manager with the qbee API. From there on you can distribute and run it on the group of devices you have specified with file distribution.

This is a very simple Node-RED flow used in this example:

flows.json
    [
    {
        "id": "117f088e.84eb97",
        "type": "tab",
        "label": "Flow 1",
        "disabled": false,
        "info": ""
    },
    {
        "id": "d75ee7b0.0775c8",
        "type": "inject",
        "z": "117f088e.84eb97",
        "name": "weather",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "10",
        "crontab": "",
        "once": true,
        "onceDelay": 0.1,
        "topic": "weather",
        "payload": "very nice",
        "payloadType": "str",
        "x": 171.5,
        "y": 175,
        "wires": [
            [
                "d1e24e2f.b5a2"
            ]
        ]
    },
    {
        "id": "d1e24e2f.b5a2",
        "type": "function",
        "z": "117f088e.84eb97",
        "name": "weather station",
        "func": "var payload = msg.payload;\n\nmsg.payload=\"The weather is \"+msg.payload;\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 373.5,
        "y": 175,
        "wires": [
            [
                "e7b731c2.2f7f2"
            ]
        ]
    },
    {
        "id": "e7b731c2.2f7f2",
        "type": "debug",
        "z": "117f088e.84eb97",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "x": 586.5,
        "y": 174,
        "wires": []
    }
]

This flows.jsonneeds to automatically be delivered to any remote embedded devices in scope.

This is the automatic workflow:

  • use git as for version control management
  • setup a GitHub runner (using GitHub actions) to build our code once a push is triggered
  • copy the new flows.json to the qbee.io file manager via API calls
  • use the qbee.io file distribution to upload the new file to a list of devices and to restart Node-RED

How to create a repository on GitHub

Check out the official documentation on how to create a repository on GitHub.

Once a repository is set up, we sepcify the GitHub secrets as shown in the following screenshot. We define the qbee login mail address as QM and the password as QP.

There we specify our qbee.io username and password, as we do not want them to be exposed.

Now GitHub actions need to be configured. These actions execute the script defined in the YAML file on the runners. Runners can be utilized through GitHub (both as a free or paid service) or you can set up runners on your local machine or in any other data center. In order to create a GitHub action please use the "set up a workflow yourself").

The script we used is the following:

node-red-flow-distribution.yml
name: Automated Node-RED flow file distribution

on:
 push:
    branches: [ main ]
 pull_request:
    branches: [ main ]

jobs:  


 build:
    runs-on: ubuntu-latest
    env:
        FLOWNAME: flows.json  

    steps:
    - uses: actions/checkout@v2

    - name: install jq
      run: |
        sudo apt-get install jq

    - name: copy to qbee file manager with jwt authentication
      run: |
        successful_status_code='200'
        auth='false'
        i=0
        token=""

        while [[ "$auth" != "true" && $i -lt 10 ]]
        do
            echo "try number $((i++))"
            output=$(curl --request POST -sL --url 'https://www.app.qbee.io/api/v2/login' \
          --header "Content-Type: application/json" \
          -d "{\"email\":\"${{ secrets.QM }}\",\"password\":\"${{ secrets.QP }}\"}"\
          -w "\n{\"http_code\":%{http_code}}\n")

            http_code=$(echo $output | jq -cs | jq -r '.[1].http_code')
            echo $http_code
            tokenValue=$(echo $output | jq -cs | jq -r '.[0].token')
            echo $tokenValue

            if [ "$http_code" != "$successful_status_code" ]

            then
                echo $tokenValue
            #uncomment next line to do nothing
            #: 
            else
                auth='true'

                apiOutput=$(curl -i --request "DELETE" -d "path=/$FLOWNAME" -H "Content-type: application/x-www-form-urlencoded" \
                   --url 'https://www.app.qbee.io/api/v2/file'\
                   --header 'Authorization: Bearer '"$tokenValue")
                echo "API output is:\n$apiOutput"

                apiOutput=$(curl -i --request POST -H "Content-Type:multipart/form-data" -F "path=/" -F "file=@$FLOWNAME" \
                   --url 'https://www.app.qbee.io/api/v2/file'\
                   --header 'Authorization: Bearer '"$tokenValue")
                echo "API output is:\n$apiOutput"


            fi
        done

Comments on the yaml file

  • this action is triggered on pushes to the main branch
  • the runner is a virtual machine with an ubuntu OS as we specified ubuntu-latest
  • Now the full code is checked out by this command actions/checkout@v2
  • We use the jq package for JWT authentication and this is installed in this step
  • the output is placed into the root directory of your qbee.io file manager (c.f. file distribution utilizing the API)

You cannot see any credentials in the action output due to the "secret" function:

Finally, we distribute our files to the remote devices as usual with the internal file distribution.

As a "command to run" we use sudo -u pi node-red-restart or whatever command you use to restart Node-RED.

Automatic deployment

Using this workflow, every time you push your Node-RED flow changes to the repository the new flow will be replaced within the qbee.io file manager. If the flow file differs from the one previously placed in the file manager, then the qbee.io file distribution is triggered along with the run command that you provided. This distributes the flows to all devices in scope and restarts Node-RED.

Hence, all your edge devices are updated by a simple git push :)