Package: airflow-windmill
WindmillDrag'n'drop web app to manage and create Airflow DAGs. DAGs are described
usinga JSON "wml" file, which can be transpiled into a Python DAG file and
pushed to a configured git repository.Front end is built using React on TypescriptBack end is built using Flask on Python 3.6+Getting StartedInstall withpip install airflow-windmillAirflow is expected to be installed on the system. This allows Windmill to run with arbitrary versions of AirflowOtherwise it can be packaged with windmill usingpip install airflow-windmill[airflow]. The version is defined inpyproject.tomlRunwindmill initto create a local Windmill projectcd windmill-projectRunwindmill runfrom this folder to run the app locallyMVP Required FeaturesFront-End FeaturesDynamic OperatorsMenu DropdownsLoad Operators from AppFormat operator display into classesSearch functionality for operatorsBasic operator level propertiesImplement DAG level propertiesNew DAG FunctionalityParameter TooltipsRender arbitrary viewport windows for New/Save/Load etcOverwrite/Save prompt on NewDAG renaming and save functionalityOpen dag from menuMake save/load more efficient by removing non-essential valuesSwitch nav menu to iconsAdd convert DAG callAdd hotkeys to menu functionsMake input/output nodes more clearCheck if file already exists on renamePrompt save if there are nodes on openFix loss of state on refresh bugPut File details in File BrowseMake Flask Backend URI configurableAdd a last saved time to NavBarAdd error handling to backend callsOnly save local state if validAdd testsGet task descriptions from Operator listXSS and injection vulns?Ctl+Shift+F FIXMEBack-End FeaturesGenerate Operator ListsCLI to start Web and Front EndGenerate DAG SpecCLI to create new windmill projectCLI to start windmill from a windmill projectImplement windmill-dev startSave/Load Windmill Files functionalityGet default valuesPull parameters from parent classesMove airflow dependency as extraConvert WML into Python DAGAPI Endpoint to trigger WML -> DAGMake sure that nodes are being put in right order using portsEdge cases for WML -> DAGAllow repeated/weird dag/task ids (e.g. 123)Get WML owner and last-modified details during wml listAllow custom operatorsStrategy for Python Opjects (e.g. callables) - allow either a import ref or an inline statementBackport existing Python DAGs to WMLsAllow DAG updates to propogate to WMLs (probably better to just always backport - consolidating would be a mess)Add tests for different airflow versionVersion lock travis tox and poetry version?Other featuresValidate on backend or front end or both?DocoAdd permission restrictions for valid tagsOnly include dist folder from app in poetry buildDev User GuideTo run as a dev:Clone from gitRunpoetry install -E airflowRunnpm install from windmill/http/appRunwindmill-dev start-frontendRunwindmill-dev start-backendOpen127.0.0.1:1234Run Python TestsExpects Tox and Poetry to be available on pathpyenv install 3.6.5 3.7.7
pyenv local 3.6.5 3.7.7
toxFuture Usage PatternsAuto-sync for windmill project to gitDeploymentDeployment to PyPi is managed using Travis and should be done in the following steps:Runpoetry version {patch|minor|major}Increment the version number inwindmill/__init__.pyCommit and merge code into the master branchEnsure that the travis build is greenCreate a git tag for the new buildPush the tag to origin