Package: chrisjen
Why chrisjen?Named after the biggest badass inThe Expanse, who knew how to get things done,chrisjenprovides an accessible, stable foundation for designing pure Python project workflows with straight-forward configuration files or simple scripting. The primary goal ofchrisjenis to provide a lightweight, intuitive, powerful, extensible framework for constructing and implementing Python project workflows.chrisjenunderstands that you want to start designing your project without spending countless hours designing the framework for your project. Indeed, I createdchrisjenso that I could skip the preliminaries in my own Python projects. File management, configuration settings, and workflow design are already implemented inchrisjen's easy-to-use system.Intuitivechrisjenputs all of the essential components for a Python project under one roof, using consistent naming conventions and structures. You start with anIdea(typically in the form of a(n) ini, toml, json, or Python file, but you can use a Python dict as well). Then, the rest of the project is automatically created for you. If you want to manually change, iterate, or otherwise advance through the stages of your project, that can easily be done entirely through theProjectclass using intuitive attributes likeoutline,workflow, andsummary.All of the nitty-gritty details of a project are handled through the consistent interface of the projectManager. Themanagerattribute of aProjectinstance directs all file management through its filingclerkand asset access and creation through itslibrarian. And, if you include all of the necessary information in your initialIdea, you do not have to concern yourself with even those simple interfaces.chrisjenstrives to get out of your way and has an easy, short learning curve. Unlike most other workflow packages,chrisjendoes not require learning a new scripting language and it mayb e used entirely from the command line without knowing any Python. For example, this is part of an .ini configuration file for a data science project derived from one used inchrisjen's unit tests:[general]
seed = 43
conserve_memory = True
parallelize = False
gpu = False

[files]
source_format = csv
interim_format = csv
final_format = csv
analysis_format = csv
test_data = True
test_chunk = 500
export_results = True

[wisconsin_cancer_project]
wisconsin_cancer_workers = analyst, critic
wisconsin_cancer_design = kanban

[analyst]
design = contest
analyst_steps = scale, split, encode, sample, model
scale_techniques = minmax, robust, normalize
split_techniques = stratified_kfold, train_test
encode_techniques = target, weight_of_evidence, one_hot, james_stein
sample_techniques = none, smote
model_techniques = xgboost, logit, random_forest
model_type = classify
label = target
default_package = sklearn

[critic]
design = waterfall
critic_steps = shap, sklearn
critic_techniques = explain, predict, report
data_to_review = testYou do not even have to worry about selecting all of the available options and specifications becausechrisjenincludes intellgent defaults and sometimes infers selctions from other options selected. For example, if one of your project workers did not have a "design" setting,chrisjenwould use thewaterfall design, the basic sequential design pattern used in project management.PowerfulTo faciliate workflow construction,chrisjencomes with the most common and useful workflow structures. While straightforward, some of these workflows are otherwise tedious and can be difficult to implement.chrisjendoes all of that work for you.chrisjenis particularly well-suited for comparative and conditional projects where you want to identify the best strategy or average results among multiple options. Among the workflow designs provided out-of-the-box are:Waterfall: the simplest workflow in project management which follows a pre-planned rigid structureKanban: a sequential workflow with isolated stages that produces deliverables for the following stage to useScrum: flexible workflow structure that requires greater user control and interventionContest: evaluates and selects the best workflow among several based on one or more criteriaPert: workflow that focuses on efficient use of parallel resources, including identifying the critical pathAgile: a dynamic workflow structure that changes direction based on one or more criteriaLean: an iterative workflow that maximizes efficiency based on one or more criteriaSurvey: averages multiple workflows based on one or more criteriaFlexiblechrisjenemphasizes letting users design their projects from a range of options. These choices can be provided in another package or added on the fly. The entire package is designed to allow users to alter the structure and framework if a user so desires.chrisjen's framework supports a wide range of coding styles. You can create complex multiple inheritance structures with mixins galore or simpler, compositional objects. Even though the data structures are necessarily object-oriented, all of the tools to modify them are also available as functions, for those who prefer a more functional approaching to programming.ContributingThe project is highly documented so that users and developers can makechrisjenwork with their projects. It is designed for Python coders at all levels. Beginners should be able to follow the readable code and internal documentation to understand how it works. More advanced users should find complex and tricky problems addressed through efficient code.Similar Projectsairflow: Apache's workflow tool that is likely the market leader. It requires substantial overhead and has a learning curve but offers the greatest extensibility for non-Python workflow components and support for continuous, always-on workflows.jetstream: similar DAG workflow structures in pure Python with a greater emphasis on loading workflows from disk.luigi: Spotify's workflow tool with much greater overhead and support for controlling workflow nodes outside of Python.pathos: supports parallel workflow construction with heterogenuous computing resources. Among other features, it includes drop-in replacements for Python'spickleandmultiprocess, calleddillandmultiprocessing, respectively.