Skip to content

Create, install and run python packges using Github

In short

In this example we show how to use Github to create a python package (using pip) from various (python) source files. The created package file (*.tar.gz) is then installed and run on your fleet of Raspberry Pis.

The process starts by pushing the changes of your source code to your Github repository. Then, using Github actions, your source code is packaged and finally the package is placed into your file manager. From there on you can distribute and install it on the group of devices you have specified.

We provide a sample test script that uses the newly installed package. As a convenient feature this script is also distributed along with the package.

If we want to build an executable (called hi-pi) for our Raspberry Pi we can use the following code

The way of packaging is inspired by this tutorial.

The package folder structure

To build our own python package called py_qbee_tst we need a folder structure as follows:

File structure


where the source files of the package are located in the subdirectory py_qbee_tst/. In the bin/ directory we have placed a sample script that is installed along with the package. The gives details about the package to be built.

To build our package py_qbee_tst for the installation with pip we can simply run the command

    python3 sdist
which creates a *.tar.gz file containing the package. This package can now be distributed to any target system and being installed via
    pip3 install py_qbee_tst-0.1.tar.gz
We are going to automate this process via Github as described in Automatic workflow using Github. First we present the toy python project.

The package source files to be built

Our toy project consists of two python files containing implemented routines and and file collecting all relevant routines.

def hello():
    return (u'Hello from')

def bye():
    return (u'Goodbye from')

from .hello import *
from .bye import *

The package setup file

In order to create a package, we define a setup file.

from setuptools import setup

    description='test package to run on',
    author='qbee AS',

Testing the installed package

For testing purposes we ship a script along with the package that is placed in the bin/ directory.

#!/usr/bin/env python3

import py_qbee_tst as qbee
print( qbee.hello() )
print( qbee.bye() )

Automatic workflow using Github

However, we aim to setup an automatic workflow, where we

  • use git as for version control management
  • setup a Github runner (using Github actions) to build our code once a push is triggered
  • copy the created package to the file manager via API calls
  • use the file distribution to the distribute (and install) the package on a list of devices

How to create a repository on Github

Check out the official documentation on how to create a repository on Github.

Once a repository is set up, before creating a runner up Github secrets as shown in the following screenshot. github-secrets There we specify our username and password, as we do not want them to be exposed.

Once that is done, we create a Github actions which runs our setup script on the runners provided by Github (or you can set up your own runners). Github actions can be created as follows (we use "set up a workflow yourself")


The script we used is the following:

name: Upload Python Package

    branches: [main]


    runs-on: ubuntu-latest

    - uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
        python-version: '3.x'
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install setuptools wheel
    - name: Build
      run: |
        python sdist 
        echo `ls dist/`
    - name: copy pip package to qbee file manager
        APPDIR: dist/
        APPNAME: py_qbee_tst-0.1.tar.gz
      run: |
        curl --digest -u ${{ secrets.QM }}:${{ secrets.QP }} -i -X "DELETE" -d "path=/$APPNAME" -H "Content-type: application/x-www-form-urlencoded"
        curl --digest -u ${{ secrets.QM }}:${{ secrets.QP }} -i -X POST -H "Content-Type:multipart/form-data" -F "path=/" -F "file=@$APPDIR$APPNAME"

Comments on the yaml file

  • this action is triggered on pushes to the main branch
  • the runner is a virtual machine with an ubuntu OS as we specified ubuntu-latest
  • on that virtual machine (freshly created for you on every single run) the code from your repository is checked out using the actions/checkout@v2 option
  • python3 along with the necessary dependencies is set up such that the code can be packaged
  • the actual packaging is done via python sdist
  • the output is placed into the root directory of your file manager (c.f. file distribution via API)

Note that your credentials aren't revealed as we use Github secrets to encrypt them as you can see in the action output


Package installation in using file distribution

Finally, we distribute our package file to the remote devices as usual with the file distribution. github-pythoon-filedist

Note that the package installation is very simple. Using pip3 (since we install it for python3) we do not need to unpack the created *.tar.gz package. Via the command

    pip3 install /home/pi/src/py_qbee_tst-0.1.tar.gz -U
the package py_qbee_tst with version 0.1 is installed globally and the flag -U causes an update if the package is already installed. The test script which was placed in the bin directory of your source code folder structure is also installed globally in the directory /usr/local/bin. Therefore we add the run command | sudo -u pi tee /home/pi/src/py_out.txt
to the above install command. We pipe the output to tee with the user pi to create an output file with owner pi. As we can see from the output the installation and test script worked. github-python-output Compare with the script presented in Testing the installed package

Automatic deployment

Using this workflow, every time you push your code changes to the repository the package file will always be replaced within the file manager. Upon a new file upload the file distribution is triggered along with the run command that you provided (which is the package install along with the test script).

Hence, all your edge devices are updated by a simple git push :)

Package verison

For ease of presentation we hard-coded the installation of version 0.1 of our custom py_qbee_tst package. Of course one could change the setup such that always the newest version is installed replacing the older one.