Saravanan666/pypi-org huggingface dataset:
Package: apache-airflow-providers-msgraph
Airflow Provider Microsoft Graph APIAirflow provider package for Microsoft Graph API.How to develop a Providers package correctly:https://airflow.apache.org/docs/apache-airflow-providers/Astronomer Providers registry:https://registry.astronomer.io/providersMaking async API calls with Airflow:https://betterprogramming.pub/making-async-api-calls-with-airflow-dynamic-task-mapping-d0cbd3066ebbDocumentationInstallingpipinstallapache-airflow-providers-msgraphConfigrationExamplesGetting users:fromairflow.providers.microsoft.msgraph.operators.msgraphimportMSGraphSDKAsyncOperatorusers_task=MSGraphSDKAsyncOperator(task_id="users_delta",conn_id="msgraph_api",expression="users.get()",)Getting users delta:fromairflow.providers.microsoft.msgraph.operators.msgraphimportMSGraphSDKAsyncOperatorusers_delta_task=MSGraphSDKAsyncOperator(task_id="users_delta",conn_id="msgraph_api",expression="users.delta.get()",)Getting a site from it's relative path and then get pages related to that site:fromairflow.providers.microsoft.msgraph.operators.msgraphimportMSGraphSDKAsyncOperatorsite_task=MSGraphSDKAsyncOperator(task_id="wgive_site",conn_id="msgraph_api",expression="sites.by_site_id('850v1v.sharepoint.com:/sites/wgive').get()",)site_pages_task=MSGraphSDKAsyncOperator(task_id="news_site_pages",conn_id="msgraph_api",expression=("sites.by_site_id('%s').pages.get()"%"{{ ti.xcom_pull(task_ids='wgive_site')['id'] }}"),)site_task>>site_pages_taskGetting users delta paged results and passing those to another DAG which processes them:importloggingfromairflowimportDAGfromairflow.decoratorsimporttaskfromairflow.providers.microsoft.msgraph.operators.msgraphimportMSGraphSDKAsyncOperatorfrompendulumimportdatetimewithDAG("process_users_delta",start_date=datetime(2022,12,20),default_args={"owner":"dabla",},schedule_interval=None,catchup=False,)asdag:@task(dag=dag)defget_response(**context):logging.info("context:%s",context)params=context.get("params")logging.info("params:%s",params)response=params.get("response")logging.info("response:%s",response)logging.info("response_type:%s",type(response))returnresponseget_response()withDAG("test_msgraph_sdk",start_date=datetime(2022,12,20),default_args={"owner":"dabla",},schedule="@daily",catchup=False,)asdag:users_delta_task=MSGraphSDKAsyncOperator(task_id="users_delta",conn_id="msgraph_api",expression="users.delta.get()",trigger_dag_id="process_users_delta",)site_task=MSGraphSDKAsyncOperator(task_id="wget_site",conn_id="msgraph_api",expression="sites.by_site_id('850v1v.sharepoint.com:/sites/wget').get()",)site_pages_task=MSGraphSDKAsyncOperator(task_id="wget_site_pages",conn_id="msgraph_api",expression=("sites.by_site_id('%s').pages.get()"%"{{ ti.xcom_pull(task_ids='wget_site')['id'] }}"),)@task(dag=dag)defshow(response):logging.info("response:%s",response)users_delta_task>>site_task>>site_pages_task>>show(site_pages_task.output)
Package: apache-airflow-providers-mysql
Packageapache-airflow-providers-mysqlRelease:5.5.3MySQLProvider packageThis is a provider package formysqlprovider. All classes for this provider package
are inairflow.providers.mysqlpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-mysqlThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0apache-airflow-providers-common-sql>=1.3.1mysqlclient>=1.3.6mysql-connector-python>=8.0.29Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-mysql[amazon]Dependent packageExtraapache-airflow-providers-amazonamazonapache-airflow-providers-common-sqlcommon.sqlapache-airflow-providers-openlineageopenlineageapache-airflow-providers-prestoprestoapache-airflow-providers-trinotrinoapache-airflow-providers-verticaverticaThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-neo4j
Packageapache-airflow-providers-neo4jRelease:3.5.0Neo4jProvider packageThis is a provider package forneo4jprovider. All classes for this provider package
are inairflow.providers.neo4jpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-neo4jThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0neo4j>=4.2.1The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-odbc
Packageapache-airflow-providers-odbcRelease:4.4.1ODBCProvider packageThis is a provider package forodbcprovider. All classes for this provider package
are inairflow.providers.odbcpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-odbcThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0apache-airflow-providers-common-sql>=1.10.0pyodbcCross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-odbc[common.sql]Dependent packageExtraapache-airflow-providers-common-sqlcommon.sqlThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-openai
Packageapache-airflow-providers-openaiRelease:1.1.0OpenAIProvider packageThis is a provider package foropenaiprovider. All classes for this provider package
are inairflow.providers.openaipython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-openaiThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0openai[datalib]>=1.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-openfaas
Packageapache-airflow-providers-openfaasRelease:3.4.0OpenFaaSProvider packageThis is a provider package foropenfaasprovider. All classes for this provider package
are inairflow.providers.openfaaspython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-openfaasThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-openlineage
Packageapache-airflow-providers-openlineageRelease:1.5.0OpenLineageProvider packageThis is a provider package foropenlineageprovider. All classes for this provider package
are inairflow.providers.openlineagepython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-openlineageThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.7.0apache-airflow-providers-common-sql>=1.6.0attrs>=22.2openlineage-integration-common>=0.28.0openlineage-python>=0.28.0Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-openlineage[common.sql]Dependent packageExtraapache-airflow-providers-common-sqlcommon.sqlThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-opensearch
Packageapache-airflow-providers-opensearchRelease:1.1.1OpenSearchProvider packageThis is a provider package foropensearchprovider. All classes for this provider package
are inairflow.providers.opensearchpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-opensearchThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0opensearch-py>=2.2.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-openstack
Packageapache-airflow-providers-openstackRelease:1.0.4Provider packageThis is a provider package foropenstackprovider. All classes for this provider package
are inairflow.providers.openstackpython package.InstallationYou can install this package on top of an existing airflow 2.1+ installation viapip installapache-airflow-providers-openstackThe package supports the following python versions: 3.6,3.7,3.8,3.91.0.0Initial version of the provider.
Package: apache-airflow-providers-opsgenie
Packageapache-airflow-providers-opsgenieRelease:5.5.0OpsgenieProvider packageThis is a provider package foropsgenieprovider. All classes for this provider package
are inairflow.providers.opsgeniepython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-opsgenieThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0opsgenie-sdk>=2.1.5The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-oracle
Packageapache-airflow-providers-oracleRelease:3.9.2OracleProvider packageThis is a provider package fororacleprovider. All classes for this provider package
are inairflow.providers.oraclepython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-oracleThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0apache-airflow-providers-common-sql>=1.3.1oracledb>=1.0.0Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-oracle[common.sql]Dependent packageExtraapache-airflow-providers-common-sqlcommon.sqlThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-pagerduty
Packageapache-airflow-providers-pagerdutyRelease:3.6.2PagerdutyProvider packageThis is a provider package forpagerdutyprovider. All classes for this provider package
are inairflow.providers.pagerdutypython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-pagerdutyThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0pdpyras>=4.1.2The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-papermill
Packageapache-airflow-providers-papermillRelease:3.6.1PapermillProvider packageThis is a provider package forpapermillprovider. All classes for this provider package
are inairflow.providers.papermillpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-papermillThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0papermill[all]>=2.4.0scrapbook[all]ipykernelThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-pgvector
Packageapache-airflow-providers-pgvectorRelease:1.1.0pgvectorProvider packageThis is a provider package forpgvectorprovider. All classes for this provider package
are inairflow.providers.pgvectorpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-pgvectorThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0apache-airflow-providers-postgres>=5.7.1pgvector>=0.2.3Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-pgvector[common.sql]Dependent packageExtraapache-airflow-providers-common-sqlcommon.sqlapache-airflow-providers-postgrespostgresThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-pinecone
Packageapache-airflow-providers-pineconeRelease:1.1.2PineconeProvider packageThis is a provider package forpineconeprovider. All classes for this provider package
are inairflow.providers.pineconepython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-pineconeThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0pinecone-client>=2.2.4,<3.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-plexus
Packageapache-airflow-providers-plexusRelease:3.4.1PlexusProvider packageThis is a provider package forplexusprovider. All classes for this provider package
are inairflow.providers.plexuspython package.You can find package information and changelog for the provider
in thedocumentation.WarningThis provider is not maintained anymore by the community. It has been removed and is not going to be
updated anymore. The removal was done according to the process described inRemoving community providersFeel free to contact Airflow Development Mailing List if you have any questions.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-plexusThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0arrow>=0.16.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-postgres
Packageapache-airflow-providers-postgresRelease:5.10.1PostgreSQLProvider packageThis is a provider package forpostgresprovider. All classes for this provider package
are inairflow.providers.postgrespython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-postgresThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0apache-airflow-providers-common-sql>=1.3.1psycopg2-binary>=2.8.0Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-postgres[amazon]Dependent packageExtraapache-airflow-providers-amazonamazonapache-airflow-providers-common-sqlcommon.sqlapache-airflow-providers-openlineageopenlineageThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-presto
Packageapache-airflow-providers-prestoRelease:5.4.1PrestoProvider packageThis is a provider package forprestoprovider. All classes for this provider package
are inairflow.providers.prestopython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-prestoThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0apache-airflow-providers-common-sql>=1.3.1presto-python-client>=0.8.4pandas>=1.2.5Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-presto[common.sql]Dependent packageExtraapache-airflow-providers-common-sqlcommon.sqlapache-airflow-providers-googlegoogleThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-qdrant
Packageapache-airflow-providers-qdrantRelease:1.0.0QdrantProvider packageThis is a provider package forqdrantprovider. All classes for this provider package
are inairflow.providers.qdrantpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-qdrantThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredqdrant_client>=1.7.0apache-airflow>=2.7.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-qubole
Packageapache-airflow-providers-quboleRelease:3.4.3QuboleProvider packageThis is a provider package forquboleprovider. All classes for this provider package
are inairflow.providers.qubolepython package.You can find package information and changelog for the provider
in thedocumentation.WarningThis provider is not maintained anymore by the community. It has been removed and is not going to be
updated anymore. The removal was done according to the process described inRemoving community providersFeel free to contact Airflow Development Mailing List if you have any questions.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-quboleThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.5.0apache-airflow-providers-common-sql>=1.3.1qds-sdk>=1.10.4The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-redis
Packageapache-airflow-providers-redisRelease:3.6.0RedisProvider packageThis is a provider package forredisprovider. All classes for this provider package
are inairflow.providers.redispython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-redisThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0redis>=4.5.2,!=4.5.5,<5.0.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-salesforce
Packageapache-airflow-providers-salesforceRelease:5.6.2SalesforceProvider packageThis is a provider package forsalesforceprovider. All classes for this provider package
are inairflow.providers.salesforcepython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-salesforceThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0simple-salesforce>=1.0.0pandas>=1.2.5The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-samba
Packageapache-airflow-providers-sambaRelease:4.5.0SambaProvider packageThis is a provider package forsambaprovider. All classes for this provider package
are inairflow.providers.sambapython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-sambaThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0smbprotocol>=1.5.0Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-samba[google]Dependent packageExtraapache-airflow-providers-googlegoogleThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-segment
Packageapache-airflow-providers-segmentRelease:3.4.0SegmentProvider packageThis is a provider package forsegmentprovider. All classes for this provider package
are inairflow.providers.segmentpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-segmentThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0analytics-python>=1.2.9The changelog for the provider package can be found in thechangelog.