Package: airflow-e2e
airflow-e2eThis packages aims to set up the scripts to run Airflow DAGs E2E tests.Installationpipinstallairflow-e2eUsagePre-requisitesBefore generating and running the E2E test scripts, the following folders and files are
required to be present in your repository:A folder that contains the Airflow DAGs under testA folder that contains the E2E test suite(s)Optionally, we can have arequirements.txtfile at the root of your repository, which
contains all Python packages required by your Airflow scheduler and workers to perform
the tasks under tests.In addition, we can optionally have arequirements-dev.txtfile at the root of your
repository, which contains all the Python packages required by the test runner to run
your E2E test suites.Generating the test scriptsTo generate the Airflow E2E test scripts, run the following command at the root
of your repository:Generating Airflow E2E test scripts withoutrequirements.txtandrequirements-dev.txt:airflow-e2e--dagsdags/--teststests/e2eIf you have packages to be installed in the Airflow services:airflow-e2e--dagsdags/--teststests/e2e--with-custom-airflow-packagesIf you have packages to be installed in the test runner service:airflow-e2e--dagsdags/--teststests/e2e--with-custom-test-packagesIf you would like to have a MongoDB service to be set up together:airflow-e2e--dagsdags/--teststests/e2e--with-mongoThis will generate adockerfolder at the root of your repository, and it will
contain the following files:<root_of_repository>
 |- docker/
     |- airflow_connections_and_variables_seeder/
     |   |- connections.yml
     |   |- variables.json
     |- .envrc
     |- docker-compose.yml
     |- docker-compose-dev.yml
     |- docker-compose-extras.yml
     |- docker-compose-manual-testing.yml
     |- docker-compose-tests.ymlIn addition, for your convenience, the followingmakecommands are printed on
the console, should you be interested to use them:clean:source./docker/.envrc&&\docker-compose\-f./docker/docker-compose.yml\-f./docker/docker-compose-dev.yml\-f./docker/docker-compose-tests.yml\-f./docker/docker-compose-extras.yml\down--remove-orphans--volumesdev:cleansource./docker/.envrc&&\docker-compose\-f./docker/docker-compose.yml\-f./docker/docker-compose-dev.yml\-f./docker/docker-compose-extras.yml\up-dwait_for_airflow_web_to_be_healthy:until[$$(dockerinspect-f'{{.State.Health.Status}}'airflow-web)="healthy"];do\sleep1;\doneseeded_dev:devwait_for_airflow_web_to_be_healthydockerexecairflow-schedulersh-c\"airflow connections import /tmp/seed/connections.yaml && airflow variables import /tmp/seed/variables.json"e2e:source./docker/.envrc&&\docker-compose\-f./docker/docker-compose.yml\-f./docker/docker-compose-tests.yml\-f./docker/docker-compose-extras.yml\up--exit-code-fromtest-runnerSetting up the E2E testsA.envrcfile is generated in thedocker/folder as well. Replace the values of
the fields with the placeholder<SECRET_STRING_TO_BE_FILLED_IN>with actual values
of your choice. Please remember to add the following to your source code versioning
tool ignore file (.gitignorefor Git, for example):.envrc*Even though we may be using dummy credentials for our tests, we should still be
vigilant when it comes to committing secrets.Running the E2E testsTo run the E2E tests, you can run the following command:source./docker/.envrc&&\docker-compose\-f./docker/docker-compose.yml\-f./docker/docker-compose-tests.yml\up--exit-code-fromtest-runnerOr, if you have copied the convenientmakecommand from before, you can run:makee2eLicenseGNU GENERAL PUBLIC LICENSE v3TestingTo run the tests, run the following command at the root of the repository:maketestChangelogRefer toCHANGELOG.md