Package: tufup
tufupA simple software updater for stand-alone Pythonapplications.Application updates and TUFHere's what a basic update-cycle for aself-updating applicationcould look like:the app's development team create a new release, and make it available on a serveran older version of the app, out in the wild, contacts the server to check for updatesthe old app finds the new release, downloads it, and installs itgoto 1.The principle is relatively simple, as long as you don't consider the security risks involved.Unfortunately, in the real world,security cannot be neglected: You don't want to install untrusted software on your system.So, how can we make sure our update-cycle is secure? This is where things get quite complicated.Luckily,python-tuf, the reference implementation forTUF(The Update Framework) takes care of this complexity. If used properly, TUF ensures a high level of security for your update system.That's why thetufuppackage is built on top ofpython-tuf.It is highly advisable to read theTUF documentationbefore proceeding.QuickstartThe easiest way to understand howtufupworks is by example. A minimal example of an application that usestufupcan be found in the companion repository:tufup-exampleThe example repository shows how to integrate thetufupclient into your app, and it shows you how to set up atufupupdate-repository.NOTE: Although the tufup-example repository uses PyInstaller to bundle an application,tufupcan be used withanytype of application bundle, even plain python scripts.Questions and IssuesIf you have questions abouttufup, or need help getting started, please start anew Q&A discussion, or post a question onStack Overflow.If you encounter bugs or other problems that are likely to affect other users, please create anew issuehere.Some backgroundThetufuppackage was inspired byPyUpdater, and uses a general approach to updating that is directly based on PyUpdater's implementation.NOTE:tufupis completelyindependentof PyUpdater. In fact,tufupwas created as a replacement for PyUpdater, given the fact thatPyUpdater has been archived and is no longer maintained.However, whereas PyUpdater implements acustomsecurity mechanism to ensure authenticity (and integrity) of downloaded update files,tufupis built on top of the security mechanisms implemented in thepython-tufpackage, a.k.a.tuf.
By entrusting the design of security measures to the security professionals,tufupcan focus on high-level tools.Althoughtufsupports highly complex security infrastructures, see e.g.PEP458, it also offers sufficient flexibility to allowapplicationdevelopers to tailor the security level to their use case.
For details and best practices, refer to thetuf docs.Based on the intended use, thetufuppackage supports only the top-level roles offered bytuf. At this time we do not support delegations.NOTE: Whereas PyUpdater is tightly integrated with PyInstaller,tufupis completelyindependentof the type of packaging solution.
At its core,tufupsimply moves bundles of files from A to B, securely, regardless of how these bundles were created.
A bundle may consist of a simple python script, a PyInstaller bundle, a PEX package, or any other collection of files and folders.OverviewBorrowing TUF terminology, we distinguish between arepo-side (repository) and aclient-side (application).Below you'll find a list of the basic steps that occur in an application update cycle.
Steps covered bytufuparehighlighted.On therepo-side, the appdevelopermodifies the application codecreates a new application archive file and corresponding patch filesigns the resulting files cryptographicallydeploys these files to a serverOn theclient-side, theapplicationchecks for updatesdownloads update filesapplies the update files (i.e. installation)Thetufpackage is used under the hood to check for updates and download update files in a secure manner, sotufupcan safely apply the update.
See thetuf docsfor more information.Archives and patchesInternally,tufupworks witharchives(gzipped bundles of files and folders) andpatches(binary differences between subsequent archives).
Each archive, except the first one, has a corresponding patch file.Archive filenames and patch filenames follow the pattern<name>-<version><suffix>wherenameis a short string that mayonlycontainalphanumeric characters,underscores, andhyphens,versionis a version string according to thePEP440specification, andsuffixis either'.tar.gz'or'.patch'.BEWARE:whitespaceis NOT allowed in the filename.Patches are typically smaller than archives, so the tufupclientwill always attempt to update using one or more patches.
However, if the total amount of patch data is greater than the desired full archive file, a full update will be performed.If this sounds confusing, don't worry: it is all handled internally.How updates are created (repo-side)When a new release of your application is ready, the following steps need to be taken to enable clients to update to that new release:Create an application archive for the new release.Create a patch from the current archive to the new archive.Add hashes for the newly created archive file and patch file to thetufmetadata.Sign the modifiedtufmetadata files.Upload the new target files, i.e. archive and patch, and the updated metadata files, to the update server.Thetufup.repomodule and thetufupCLI provide convenient ways to streamline steps 1 to 4, based on thetufbasic repo example.
Step 5 is not covered bytufup, as it depends on the implementation.The signed metadata and hashes ensure both authenticity and integrity of the update files (seetuf docs).
In order to sign the metadata, we need access to the private key files for the applicabletufroles.How updates are applied (client-side)By default, updates are applied by copying all files and folders from the latest archive to the current app installation directory.Here's what happens during the update process:The latest archive is either downloaded in full, as described above, or it is derived from the current archive by applying one or more downloaded patches.Once the latest archive is available on disk, it is decompressed to a temporary directory.A default install script is then started, which copies the new files and folders from the temporary directory to the current app installation directory. On Windows, this script is started in a new process, after which the currently running process will exit.Alternatively, you can specify a custom install script to do whatever you want with the new files.The default install script accepts an optionalpurge_dst_dirargument, which will causeALLfiles and folders to be deleted from the app installation directory, before moving the new files into place.
This is a convenient way to remove any stale files and folders from the app installation directory.WARNING: Thepurge_dst_diroption shouldonlybe used if the app is properly installed in itsown separatedirectory.
If this is not the case, for example if the app is running from the WindowsDesktopdirectory, anyunrelatedfiles or folders in this directory will also be deleted!App versions and release channelsWhen adding an application bundle to yourtufuprepository, you need to specify an app version string.
This version string is used in the archive filename, and must bePEP440compliant (internally we usepackaging.version.Version).Thetufupclient inspects these version strings to determine if updates are available.By default, when thetufupclient looks for updates, it only includes final releases.
Pre-releases are filtered out, unless you explicitly specify a "pre-release" channel.
Refer to theClient.check_for_updates()method for details:Ifpreis specified, pre-releases are included, down to the specified level.
Pre-release identifiers follow the PEP440 specification, i.e.'a','b', or'rc', for alpha, beta, and release candidate, respectively.For example, suppose your latest final-release is1.3.0, and your latest pre-release is2.0.0a3.
An app in the field still has old version1.0.0.
If this app checks either the default channel, the release-candidate ('rc') channel, or the beta ('b') channel, it finds version1.3.0available.
If the app checks the alpha channel ('a'), it finds2.0.0a3.Just to be clear:tufupassumes a typical linear release history without branching, so0.0 < 0.1a < 0.1b < 0.1rc < 0.1rc1 < 0.1 < ...Migrating from other update frameworksHere's one way to migrate from another update framework, such aspyupdater, totufup:Addtufupto your main application environment as a core dependency, and movepyupdaterfrom core dependencies to development dependencies.Replace allpyupdaterclient code (and configuration) in your application by thetufupclient.Initialize thetufuprepository, so the root metadata fileroot.jsonexists.Modify your PyInstaller.specfile (from PyUpdater) to ensure that theroot.jsonfile is included in your package.Build, package, and sign usingpyupdater, and deploy to your server, as usual.
This ensures that yourpyupdaterclients currently in the field will be able to update to the newtufupclient.From here on, new updates will be deployed usingtufup.If you want to enable a patch update from thepyupdaterversion to the newtufupversion, extract the latest PyUpdater archive and add the resulting bundle to thetufuprepository.To skip patch creation, just create a new app bundle and add that to thetufuprepository.BEWARE: Keep thepyupdaterrepository in place as long as necessary to allow all clients to update.From now on, build, package, sign and deploy usingtufup, as described elsewhere in this document.Platform supportThetufup.clienttools are aimed primarily atWindowsandmacOSapplications, whereas thetufup.repotools are platform independent, astufup.repois just a thin layer on top ofpython-tuf.Althoughtufup.clientcould also be used for Linux applications, those are probably better off using native packaging solutions, or solutions such as Flatpak or Snapcraft.
Read thePython packaging overviewfor more information.Platform dependence fortufup.clientis related to file handling and process handling during the installation procedure, as can be seen intufup.utils.platform_specific.
A custom, platformdependent, installation procedure can be specified via the optionalinstallargument for theClient.update()method.