Package: recruitment
recruitmenta lib for sourcing actionpacked entities capable of getting the job doneOverviewThis code provides abstractions (mostly housedhereat time of writing this) that support unified and robust interaction with cloud services.
TheBrokerconcept allows for the recasting of methods provided by cloud integration SDKs (e.g.boto) into an interface of your choosing. TheCommlinkconcept houses method bindings defined by theBroker.interfacewhile theConsumer,Publisher, andAgententities implement the bound interface withactionpacked resilience 💥Some TermsWhen dealing with AWS data storage services, data is eitherpublishedorconsumed.
This library presents a flexible API for doing just that.
The primary entities are a type ofJobas follows:ConsumerPublisherThis list outlines the main types used:TypeDescriptionConfigselects an interface to bind; holds credentialsCommlinkhosts the bound interfaceContingencydescription of how to handle failureCoordinatornamespace for describing how to do workJobtop-level scope for executing workThe following diagrams the relationship between the types:This one zooms-in on theBroker:There also exists anAgenttype (not pictured) capable of both consumingandpublishing by requiring injection of both aforementionedJobtypes upon construction.
Work done (say by invoking.consumeor.publish), is encapsulated as anEfforttype.
The culmination of that work can be found under the eponymous attribute of anEffortinstance.EffortDescription.culminationoutcome from retrying.initial_attemptfirst attmept.final_attemptlast attempt.attemptsall attempts.retriesattempts - initial_attemptAttempts are returned asResulttypes for convenience (seeherefor more about that type).UsageSay you'd like to pull files from s3; just follow these steps:Define aConfigconfig=Config(service_name='s3',# can also pass Broker.s3region_name='somewhere-in-the-world',access_key_id='s3curityBadge!',secret_access_key='p@ssw0rd!',endpoint_url='some-computer.com',)Build theJobconsumer=Consumer(Coordinator(Commlink(config),Contingency))Simple as that.
Give it a try.
Being that aConsumerwas built, above, the.consumemethod is available.
Similar can by done with aPublisher.ContingenciesThings can go wrong and when they do, it may be helpful to try again.
Passing aContingencyto aCoordinatoris how you do that.
The class, alone, can be passed for some default behavior or it can be instantiated with the params.
Themax_retriesparam is self-expanatory as it governs the maximum number of retries that will be attempted.fromactionpack.actionsimportCallfromactionpack.utilsimportClosurecallback=Call(Closure(print,'did a thing!')Contingency(max_retries=3,reaction=callback)Thereactionparam is a bit more nuanced.
If anActionis passed, it's guaranteed to be performed after the original job is completed.
This feature is great for logging or notifying other processes of the what has occurred.DevelopmentSetupBuild scripting is managed vianoxfile.
Executenox -lto see the available commands (set theUSEVENVenvironment variable to view virtualenv-oriented commands).
To get started, simply runnox.
Doing so will installrecruitmenton your PYTHONPATH.
Using theUSEVENVenvironment variable, a virtualenv can be created in the local ".nox/" directory with something like:USEVENV=virtualenv nox -s recruitment-venv-install-3.10.All tests can be run withnox -s testand a single test can be run with something like the following:TESTNAME=<tests-subdir>.<test-module>.<class-name>.<method-name> nox -s testCoverage reports are optional and can be disabled using theCOVERAGEenvironment variable set to a falsy value like "no".Coming Soon...Sometimes you'd like to resume work or even automate remediation.
In such cases, you could serialize, then persist progress locally for some other process to work from later.
This sort of design would facilitate the closed loop for ensuring whatever work tasked eventually gets done without error.The picture, above, demonstrates a fail-safe apparatus where aPublisherpublishes messages to some cloud backend and record failures to local disk when encountered. TheAgentlives in a separate execution context and can re-publish failed messages.