Package: airflow-provider-census
Census Provider for Apache AirflowThis package allows you to trigger syncs forCensus.InstallationInstall theairflow-provider-censuspackage from PyPI using your preferred way of installing python packages.ConfigurationThere are 2 ways to configure a Census connection depending on whether you are using Airflow 1.10 or Airflow 2.TheCensusHookandCensusOperatoruse thecensus_defaultconnection id as a default, although this is configurable if you would like to use your own connection id.Finding the secret-tokenGo to any sync athttps://app.getcensus.com/syncsClick on the Sync Configuration tab.Next to API TRIGGER, click "Click to show"The url will be of the formathttps://bearer:secret-token:arandomstring@app.getcensus.com/api/v1/syncs/0/triggerthe secret token will be of the format "secret-token:arandomstring" in the url above, including the "secret-token:" part. Do not include the "@".Configuration in Airflow 1.10In the Airflow Connections UI, create a new connection:Conn ID: census_defaultConn Type: HTTPPassword: secret-tokenConfiguration in Airflow 2In the Airflow Connections UI, create a new connection:Conn Id: census_defaultConn Type: CensusCensus Secret Token: secret-tokenHooksCensusHookCensusHookis a class that inherits fromHttpHookand can be used to run http requests for Census.
You will most likely interact with the operator rather than the hook.The hook can be imported by the following code:fromairflow_provider_census.hooks.censusimportCensusHookOperatorsCensusOperatorCensusOperatortriggers a sync job in Census. The operator takes the following parameters:sync_id : Navigate to the sync and check the url for the sync id. For examplehttps://app.getcensus.com/syncs/0/overviewhere, the sync_id would be 0.census_conn_id : The connection id to use. This is optional and defaults to 'census_default'.The operator can be imported by the following code:fromairflow_provider_census.operators.censusimportCensusOperatorSensorsCensusSensorCensusSensorpolls a sync run in Census. The sensor takes the following parameters:sync_run_id : The sync run id you get back from the CensusOperator which triggers a new sync.census_conn_id : The connection id to use. This is optional and defaults to 'census_default'.The sensor can be imported by the following code:fromairflow_provider_census.sensors.censusimportCensusSensorExampleThe following example will run a Census sync once a day:fromairflow_provider_census.operators.censusimportCensusOperatorfromairflowimportDAGfromairflow.utils.datesimportdays_agofromdatetimeimporttimedeltadefault_args={"owner":"airflow","start_date":days_ago(1)}dag=DAG('census',default_args=default_args)sync=CensusOperator(sync_id=27,dag=dag,task_id='sync')sensor=CensusSensor(sync_run_id="{{ ti.xcom_pull(task_ids = 'sync') }}",dag=dag,task_id='sensor')sync>>sensorFeedbackSource code available on Github. Feedback and pull requests are greatly appreciated. Let us know if we can improve this.From:wave: The folks atCensusoriginally put this together. Have data? We'll sync your data warehouse with your CRM and the customer success apps critical to your team.Need help setting this up?You can always contact us viasupport@getcensus.comorin-appvia the live chat in the bottom right corner.