Package: tailucas-pylib
About The ProjectOverviewThis package was created by factoring out many reusable code artifacts from myvarious projectsover a number of years. Since this work was not a part of a group effort, the test coverage is predictably abysmal :raised_eyebrow: and Python documentation notably absent :expressionless:. For each of my projects, which derive from a common Docker applicationfound here, this package provides some specific and relatively powerful features to enable rapid offshoots for new ideas. Here is what functionality this package provides:aws.init.py: If AWS environment variables are set, global boto and boto3 session objects are instantiated.aws.metrics: CloudWatch metric helper.aws.swf: Client-side interface tobotoflowwith interfaces specific to a number of prior projects. Examples of this use are in respectiveswfbranches of some projects. This mechanism has been since replaced with message passing viaRabbitMQ.init.py: A lot of module bootstrap with a pitiful attempt to support unit testing. At application startup, a choice is made between logging to a configured logger or to the terminal if one exists, to support a scenario where the application is run interactively. A standard application configuration file is loaded with a Python ConfigParser and then a credentials client is loaded based on1Password. The choice of 1Password is motivated by their secrets automation features and because I happened to use their service already. I have been tempted to try Bitwarden also but there has not been a need yet to create this abstraction. Next, theSentryclient is loaded with whatever extras the given application might specify. Then Python builtins areabused to store some globals for use throughout the package and application.app: A few of my applications useZeroMQfor "lockless" IPCs between Python threads and so it became apparent that some helpers were needed to support common patterns.ZmqRelayis one such example where an application thread can be used to receive some message on which processing is done, and then forward it to another ZeroMQ channel. The main thread logic is contained with a context manager which handles failures and shutdown gracefully.app.bluetooth: Bluetooth helper functions that make use ofhcitooll2ping.app.data: Simple utility to make a common IPC or network payload usingMessagePack.app.datetime: Date and timestamp manipulation with time zone normalization.app.handler: A useful Python context manager to handle a variety of failure conditions.app.process: Simple signal handler with interaction with application shutdown tracking.app.rabbit: A fairly good example of aRabbitMQapplication framework with internal IPC handoff toZeroMQchannels for inter-thread communication.app.threads: A useful thread nanny with shutdown debugging and metric support.app.zmq: Useful helper functions for aZeroMQenabled application.Handy stand-alone tools:config_interpol: By making creativeabuse of Python's ConfigParser, this tool designed for the command-line will take a configuration file with variables that are automatically substituted with with either an overlay configuration or environment variables by the same name and output the interpolated configuration. A good example of this tool being used ishere.cred_tool: Useful to fetch an item from1Password.yaml_interpol: A script useful to generate docker-compose YAML output from templates. A crude example isherewhich fetches application configuration from a 1Password vault. Note that application runtime secrets are only in memory and loaded ininit.py. I happen to use 1Password to also store application key-value pairs for use in docker-compose templates.Package StructureA crude attempt to visualize the package structure withpyreverse.(back to top)Built WithTechnologies that help make this package useful:Also:CronitorMessagePackBotoflow(back to top)Getting StartedHere is some detail about the intended use of this package.PrerequisitesAPythonproject or runtime environment. Since this project is already initialized withPoetrydependency management, I recommend that you continue to use it. Beyond the Python dependencies defined in thePoetry configuration, the package init carries hardcoded dependencies onSentryand1Passwordin order to function. Unless you want these and are effectively extending mybase project, you're likely better off forking this package and cutting out what you do not need.InstallationThis project is intended to be used as aGit submodulebecause it is not yet structured or built to be vended from thePython Package Index. One of mycompanion projectsillustrateshow easilythe project can be added to the dependency closure of a project using Poetry which also greatly diminished the need to publish to online package indexes.(back to top)UsageI havevarious projectsthat use this tool chain. For example, myBase Projectwhich can be run stand-alone but also serves as myDocker base imagefrom which other projects are derived.(back to top)LicenseDistributed under the MIT License. SeeLICENSEfor more information.(back to top)AcknowledgmentsTemplate on which this README is based(back to top)