Package: airflow-python-sdk
airflow-python-sdkOverviewTo facilitate management, Apache Airflow supports a range of REST API endpoints across its
objects.
This section provides an overview of the API design, methods, and supported use cases.Most of the endpoints acceptJSONas input and returnJSONresponses.
This means that you must usually add the following headers to your request:Content-type: application/json
Accept: application/jsonResourcesThe termresourcerefers to a single type of object in the Airflow metadata. An API is broken up by its
endpoint's corresponding resource.
The name of a resource is typically plural and expressed in camelCase. Example:dagRuns.Resource names are used as part of endpoint URLs, as well as in API parameters and responses.CRUD OperationsThe platform supportsCreate,Read,Update, andDelete operations on most resources.
You can review the standards for these operations and their standard parameters below.Some endpoints have special behavior as exceptions.CreateTo create a resource, you typically submit an HTTPPOSTrequest with the resource's required metadata
in the request body.
The response returns a201 Createdresponse code upon success with the resource's metadata, including
its internalid, in the response body.ReadThe HTTPGETrequest can be used to read a resource or to list a number of resources.A resource'sidcan be submitted in the request parameters to read a specific resource.
The response usually returns a200 OKresponse code upon success, with the resource's metadata in
the response body.If aGETrequest does not include a specific resourceid, it is treated as a list request.
The response usually returns a200 OKresponse code upon success, with an object containing a list
of resources' metadata in the response body.When reading resources, some common query parameters are usually available. e.g.:v1/connections?limit=25&offset=25Query ParameterTypeDescriptionlimitintegerMaximum number of objects to fetch. Usually 25 by defaultoffsetintegerOffset after which to start returning objects. For use with limit query parameter.UpdateUpdating a resource requires the resourceid, and is typically done using an HTTPPATCHrequest,
with the fields to modify in the request body.
The response usually returns a200 OKresponse code upon success, with information about the modified
resource in the response body.DeleteDeleting a resource requires the resourceidand is typically executing via an HTTPDELETErequest.
The response usually returns a204 No Contentresponse code upon success.ConventionsResource names are plural and expressed in camelCase.Names are consistent between URL parameter name and field name.Field names are in snake_case.{\"name\": \"string\",\"slots\": 0,\"occupied_slots\": 0,\"used_slots\": 0,\"queued_slots\": 0,\"open_slots\": 0}Update MaskUpdate mask is available as a query parameter in patch endpoints. It is used to notify the
API which fields you want to update. Usingupdate_maskmakes it easier to update objects
by helping the server know which fields to update in an object instead of updating all fields.
The update request ignores any fields that aren't specified in the field mask, leaving them with
their current values.Example:resource = request.get('/resource/my-id').json()
  resource['my_field'] = 'new-value'
  request.patch('/resource/my-id?update_mask=my_field', data=json.dumps(resource))Versioning and Endpoint LifecycleAPI versioning is not synchronized to specific releases of the Apache Airflow.APIs are designed to be backward compatible.Any changes to the API will first go through a deprecation phase.Summary of ChangesAirflow versionDescriptionv2.0Initial releaseTrying the APIYou can use a third party client, such ascurl,HTTPie,Postmanorthe Insomnia rest clientto test
the Apache Airflow API.Note that you will need to pass credentials data.For e.g., here is how to pause a DAG withcurl, when basic authorization is used:curl-XPOST'https://example.com/api/v1/dags/{dag_id}?update_mask=is_paused'\\-H'Content-Type: application/json'\\--user\"username:password\"\\-d'{\"is_paused\": true}'Using a graphical tool such asPostmanorInsomnia,
it is possible to import the API specifications directly:Download the API specification by clicking theDownloadbutton at top of this documentImport the JSON specification in the graphical tool of your choice.InPostman, you can click theimportbutton at the topWithInsomnia, you can just drag-and-drop the file on the UINote that withPostman, you can also generate code snippets by selecting a request and clicking on
theCodebutton.AuthenticationTo be able to meet the requirements of many organizations, Airflow supports many authentication methods,
and it is even possible to add your own method.If you want to check which auth backend is currently set, you can useairflow config get-value api auth_backendcommand as in the example below.$airflowconfigget-valueapiauth_backend
airflow.api.auth.backend.basic_authThe default is to deny all requests.For details on configuring the authentication, seeAPI Authorization.ErrorsWe follow the error response format proposed inRFC 7807also known as Problem Details for HTTP APIs. As with our normal API responses,
your client must be prepared to gracefully handle additional members of the response.UnauthenticatedThis indicates that the request has not been applied because it lacks valid authentication
credentials for the target resource. Please check that you have valid credentials.PermissionDeniedThis response means that the server understood the request but refuses to authorize
it because it lacks sufficient rights to the resource. It happens when you do not have the
necessary permission to execute the action you performed. You need to get the appropriate
permissions in other to resolve this error.BadRequestThis response means that the server cannot or will not process the request due to something
that is perceived to be a client error (e.g., malformed request syntax, invalid request message
framing, or deceptive request routing). To resolve this, please ensure that your syntax is correct.NotFoundThis client error response indicates that the server cannot find the requested resource.MethodNotAllowedIndicates that the request method is known by the server but is not supported by the target resource.NotAcceptableThe target resource does not have a current representation that would be acceptable to the user
agent, according to the proactive negotiation header fields received in the request, and the
server is unwilling to supply a default representation.AlreadyExistsThe request could not be completed due to a conflict with the current state of the target
resource, meaning that the resource already existsUnknownThis means that the server encountered an unexpected condition that prevented it from
fulfilling the request.This Python package is automatically generated by theOpenAPI Generatorproject based onthe specs:API version: 1.0.0Package version: 1.0.1Build package: org.openapitools.codegen.languages.PythonClientCodegen
For more information, please visithttps://github.com/zachliuRequirements.Python >= 3.6Installation & Usagepip installThe python package is hosted on PyPI, you can install directly using:pipinstallairflow-python-sdkThen import the package:importairflow_python_sdkSetuptoolsInstall viaSetuptools.pythonsetup.pyinstall--user(orsudo python setup.py installto install the package for all users)Then import the package:importairflow_python_sdkGetting StartedPlease follow theinstallation procedureand then run the following:importtimeimportairflow_python_sdkfrompprintimportpprintfromairflow_python_sdk.apiimportconfig_apifromairflow_python_sdk.model.configimportConfigfromairflow_python_sdk.model.errorimportError# Defining the host is optional and defaults to http://localhost/api/v1# See configuration.py for a list of all supported configuration parameters.configuration=airflow_python_sdk.Configuration(host="http://localhost/api/v1")# The client must configure the authentication and authorization parameters# in accordance with the API server security policy.# Examples for each auth method are provided below, use the example that# satisfies your auth use case.# Configure HTTP basic authorization: Basicconfiguration=airflow_python_sdk.Configuration(host="https://<your-airflow-2.0.0>/api/v1",username='YOUR_USERNAME',password='YOUR_PASSWORD')# Enter a context with an instance of the API clientwithairflow_python_sdk.ApiClient(configuration)asapi_client:# Create an instance of the API classapi_instance=config_api.ConfigApi(api_client)try:# Get current configurationapi_response=api_instance.get_config()pprint(api_response)exceptairflow_python_sdk.ApiExceptionase:print("Exception when calling ConfigApi->get_config:%s\n"%e)Documentation for API EndpointsAll URIs are relative tohttp://localhost/api/v1ClassMethodHTTP requestDescriptionConfigApiget_configGET/configGet current configurationConnectionApidelete_connectionDELETE/connections/{connection_id}Delete a connectionConnectionApiget_connectionGET/connections/{connection_id}Get a connectionConnectionApiget_connectionsGET/connectionsList connectionsConnectionApipatch_connectionPATCH/connections/{connection_id}Update a connectionConnectionApipost_connectionPOST/connectionsCreate a connectionDAGApiget_dagGET/dags/{dag_id}Get basic information about a DAGDAGApiget_dag_detailsGET/dags/{dag_id}/detailsGet a simplified representation of DAGDAGApiget_dag_sourceGET/dagSources/{file_token}Get a source codeDAGApiget_dagsGET/dagsList DAGsDAGApiget_taskGET/dags/{dag_id}/tasks/{task_id}Get simplified representation of a taskDAGApiget_tasksGET/dags/{dag_id}/tasksGet tasks for DAGDAGApipatch_dagPATCH/dags/{dag_id}Update a DAGDAGApipost_clear_task_instancesPOST/dags/{dag_id}/clearTaskInstancesClear a set of task instancesDAGApipost_set_task_instances_statePOST/dags/{dag_id}/updateTaskInstancesStateSet a state of task instancesDAGRunApidelete_dag_runDELETE/dags/{dag_id}/dagRuns/{dag_run_id}Delete a DAG runDAGRunApiget_dag_runGET/dags/{dag_id}/dagRuns/{dag_run_id}Get a DAG runDAGRunApiget_dag_runsGET/dags/{dag_id}/dagRunsList DAG runsDAGRunApiget_dag_runs_batchPOST/dags/~/dagRuns/listList DAG runs (batch)DAGRunApipost_dag_runPOST/dags/{dag_id}/dagRunsTrigger a new DAG runEventLogApiget_event_logGET/eventLogs/{event_log_id}Get a log entryEventLogApiget_event_logsGET/eventLogsList log entriesImportErrorApiget_import_errorGET/importErrors/{import_error_id}Get an import errorImportErrorApiget_import_errorsGET/importErrorsList import errorsMonitoringApiget_healthGET/healthGet instance statusMonitoringApiget_versionGET/versionGet version informationPoolApidelete_poolDELETE/pools/{pool_name}Delete a poolPoolApiget_poolGET/pools/{pool_name}Get a poolPoolApiget_poolsGET/poolsList poolsPoolApipatch_poolPATCH/pools/{pool_name}Update a poolPoolApipost_poolPOST/poolsCreate a poolTaskInstanceApiget_extra_linksGET/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/linksList extra linksTaskInstanceApiget_logGET/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/logs/{task_try_number}Get logsTaskInstanceApiget_task_instanceGET/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}Get a task instanceTaskInstanceApiget_task_instancesGET/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstancesList task instancesTaskInstanceApiget_task_instances_batchPOST/dags//dagRuns//taskInstances/listList task instances (batch)VariableApidelete_variableDELETE/variables/{variable_key}Delete a variableVariableApiget_variableGET/variables/{variable_key}Get a variableVariableApiget_variablesGET/variablesList variablesVariableApipatch_variablePATCH/variables/{variable_key}Update a variableVariableApipost_variablesPOST/variablesCreate a variableXComApiget_xcom_entriesGET/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/xcomEntriesList XCom entriesXComApiget_xcom_entryGET/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/xcomEntries/{xcom_key}Get an XCom entryDocumentation For ModelsClassReferenceClearTaskInstanceCollectionInfoColorConfigConfigOptionConfigSectionConnectionConnectionAllOfConnectionCollectionConnectionCollectionAllOfConnectionCollectionItemCronExpressionDAGDAGCollectionDAGCollectionAllOfDAGDetailDAGDetailAllOfDAGRunDAGRunCollectionDAGRunCollectionAllOfDagStateErrorEventLogEventLogCollectionEventLogCollectionAllOfExtraLinkExtraLinkCollectionHealthInfoHealthStatusImportErrorImportErrorCollectionImportErrorCollectionAllOfInlineResponse200InlineResponse2001ListDagRunsFormListTaskInstanceFormMetadatabaseStatusPoolPoolCollectionPoolCollectionAllOfRelativeDeltaSLAMissScheduleIntervalSchedulerStatusTagTaskTaskCollectionTaskCollectionAllOfTaskExtraLinksTaskInstanceTaskInstanceCollectionTaskInstanceCollectionAllOfTaskInstanceReferenceTaskInstanceReferenceCollectionTaskStateTimeDeltaTriggerRuleUpdateTaskInstancesStateVariableVariableAllOfVariableCollectionVariableCollectionAllOfVariableCollectionItemVersionInfoWeightRuleXComXComAllOfXComCollectionXComCollectionAllOfXComCollectionItemDocumentation For AuthorizationBasicType: HTTP basic authenticationKerberosAuthorzach.z.liu@gmail.comNotes for Large OpenAPI documentsIf the OpenAPI document is large, imports in airflow_python_sdk.apis and airflow_python_sdk.models may fail with a
RecursionError indicating the maximum recursion limit has been exceeded. In that case, there are a couple of solutions:Solution 1:
Use specific imports for apis and models like:from airflow_python_sdk.api.default_api import DefaultApifrom airflow_python_sdk.model.pet import PetSolution 1:
Before importing the package, adjust the maximum recursion limit as shown below:import sys
sys.setrecursionlimit(1500)
import airflow_python_sdk
from airflow_python_sdk.apis import *
from airflow_python_sdk.models import *