Saravanan666/pypi-org huggingface dataset:
Package: apache-airflow-providers-exasol
Packageapache-airflow-providers-exasolRelease:4.4.2ExasolProvider packageThis is a provider package forexasolprovider. All classes for this provider package
are inairflow.providers.exasolpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-exasolThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0apache-airflow-providers-common-sql>=1.10.0pyexasol>=0.5.1pandas>=1.2.5Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-exasol[common.sql]Dependent packageExtraapache-airflow-providers-common-sqlcommon.sqlThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-fab
Packageapache-airflow-providers-fabRelease:1.0.1Flask App BuilderProvider packageThis is a provider package forfabprovider. All classes for this provider package
are inairflow.providers.fabpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-fabThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.9.0.dev0flask>=2.2,<2.3flask-appbuilder==4.3.11flask-login>=0.6.2google-re2>=1.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-facebook
Packageapache-airflow-providers-facebookRelease:3.4.0Facebook AdsProvider packageThis is a provider package forfacebookprovider. All classes for this provider package
are inairflow.providers.facebookpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-facebookThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0facebook-business>=6.0.2The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-fastetl
FastETL framework, modern, versatile, does almost everything.Este texto também está disponível em português: 🇧🇷LEIAME.md.FastETLis a plugins package for Airflow for building data pipelines
for a number of common scenarios.Main features:Full or incrementalreplicationof tables in SQL Server, Postgres
and MySQL databasesLoad data fromGSheetsand from spreadsheets onSamba/WindowsnetworksExtractingCSVfrom SQLClean data using custom data patching tasks (e.g. for messy
geographical coordinates, mapping canonical values for columns, etc.)Using aOpen Street Routing Machineservice to calculate route distancesUsingCKANor
dados.gov.br's API to update dataset metadataUsing FrictionlessTabular Data Packagesto write OpenDocument Text format data dictionariesThis framework is maintained by a network of developers from many teams
at the Ministry of Management and Innovation in Public Services and is
the cumulative result of usingApache Airflow, a free and open source
tool, starting in 2019.For government:FastETL is widely used for replication of data queried
via Quartzo (DaaS) from Serpro.Installation in AirflowFastETL implements the standards for Airflow plugins. To install it,
simply add theapache-airflow-providers-fastetlpackage to your
Python dependencies in your Airflow environment.Or install it withpipinstallapache-airflow-providers-fastetlTo see an example of an Apache Airflow container that uses FastETL,
check out theairflow2-dockerrepository.To ensure appropriate results, please make sure to install themsodbcsql17andunixodbc-devlibraries on your Apache Airflow workers.TestsThe test suite uses Docker containers to simulate a complete use
environment, including Airflow and the databases. For that reason, to
execute the tests, you first need to install Docker and docker-compose.For people using Ubuntu 20.04, you can just type on the terminal:snapinstalldockerFor other versions and operating systems, see theofficial Docker documentation.To build the containers:makesetupTo run the tests, use:makesetup&&maketestsTo shutdown the environment, use:makedownUsage examplesThe main FastETL feature is theDbToDbOperatoroperator. It copies data
betweenpostgresandmssqldatabases. MySQL is also supported as a
source.Here goes an example:fromdatetimeimportdatetimefromairflowimportDAGfromfastetl.operators.db_to_db_operatorimportDbToDbOperatordefault_args={"start_date":datetime(2023,4,1),}dag=DAG("copy_db_to_db_example",default_args=default_args,schedule_interval=None,)t0=DbToDbOperator(task_id="copy_data",source={"conn_id":airflow_source_conn_id,"schema":source_schema,"table":table_name,},destination={"conn_id":airflow_dest_conn_id,"schema":dest_schema,"table":table_name,},destination_truncate=True,copy_table_comments=True,chunksize=10000,dag=dag,)More detail about the parameters and the workings ofDbToDbOperatorcan bee seen on the following files:fast_etl.pydb_to_db_operator.pyHow to contributeTo be written on theCONTRIBUTING.mddocument (issue#4).
Package: apache-airflow-providers-ftp
Packageapache-airflow-providers-ftpRelease:3.7.0File Transfer Protocol (FTP)Provider packageThis is a provider package forftpprovider. All classes for this provider package
are inairflow.providers.ftppython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-ftpThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-ftp[openlineage]Dependent packageExtraapache-airflow-providers-openlineageopenlineageThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-github
Packageapache-airflow-providers-githubRelease:2.5.1GitHubProvider packageThis is a provider package forgithubprovider. All classes for this provider package
are inairflow.providers.githubpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-githubThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0PyGithub!=1.58The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-grpc
Packageapache-airflow-providers-grpcRelease:3.4.1gRPCProvider packageThis is a provider package forgrpcprovider. All classes for this provider package
are inairflow.providers.grpcpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-grpcThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0google-auth>=1.0.0,<3.0.0google-auth-httplib2>=0.0.1grpcio>=1.15.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-hashicorp
Packageapache-airflow-providers-hashicorpRelease:3.6.3Hashicorp includingHashicorp VaultProvider packageThis is a provider package forhashicorpprovider. All classes for this provider package
are inairflow.providers.hashicorppython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-hashicorpThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0hvac>=1.1.0Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-hashicorp[google]Dependent packageExtraapache-airflow-providers-googlegoogleThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-http
Packageapache-airflow-providers-httpRelease:4.9.1Hypertext Transfer Protocol (HTTP)Provider packageThis is a provider package forhttpprovider. All classes for this provider package
are inairflow.providers.httppython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-httpThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0requests>=2.27.0,<3requests_toolbeltaiohttp>=3.9.2asgirefThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-imap
Packageapache-airflow-providers-imapRelease:3.5.0Internet Message Access Protocol (IMAP)Provider packageThis is a provider package forimapprovider. All classes for this provider package
are inairflow.providers.imappython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-imapThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-influxdb
Packageapache-airflow-providers-influxdbRelease:2.4.1InfluxDBProvider packageThis is a provider package forinfluxdbprovider. All classes for this provider package
are inairflow.providers.influxdbpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-influxdbThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0influxdb-client>=1.19.0requests>=2.27.0,<3The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-jdbc
Packageapache-airflow-providers-jdbcRelease:4.2.2Java Database Connectivity (JDBC)Provider packageThis is a provider package forjdbcprovider. All classes for this provider package
are inairflow.providers.jdbcpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-jdbcThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0apache-airflow-providers-common-sql>=1.3.1jaydebeapi>=1.1.1Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-jdbc[common.sql]Dependent packageExtraapache-airflow-providers-common-sqlcommon.sqlThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-jenkins
Packageapache-airflow-providers-jenkinsRelease:3.5.1JenkinsProvider packageThis is a provider package forjenkinsprovider. All classes for this provider package
are inairflow.providers.jenkinspython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-jenkinsThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0python-jenkins>=1.0.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-jira
Packageapache-airflow-providers-jiraRelease:3.1.0Atlassian JiraProvider packageThis is a provider package forjiraprovider. All classes for this provider package
are inairflow.providers.jirapython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-jiraThe package supports the following python versions: 3.7,3.8,3.9,3.10RequirementsPIP packageVersion requiredapache-airflow>=2.2.0apache-airflow-providers-atlassian-jiraCross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-jira[atlassian.jira]Dependent packageExtraapache-airflow-providers-atlassian-jiraatlassian.jiraChangelog3.1.0WarningThis is the last release for jira provider. The provider is discontinued.
Please switch to atlassian.jira providerDeprecate jira provider in favor of atlassian.jira provider  (#25930)3.0.1Bug FixesFix the error caused by passing unused context in JiraSensor.poke (#23352)3.0.0Breaking changesThis release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow
providers support policyhttps://github.com/apache/airflow/blob/main/README.md#support-for-providers2.0.5Bug FixesUse JiraHook instead of JiraOperator for JiraSensor2.0.4Bug FixesFix mistakenly added install_requires for all providers (#22382)2.0.3MiscAdd Trove classifiers in PyPI (Framework :: Apache Airflow :: Provider)2.0.2MiscSupport for Python 3.102.0.1MiscOptimise connection importing for Airflow 2.2.02.0.0Breaking changesAuto-applyapply_default decorator (#15667)WarningDue to apply_default decorator removal, this version of the provider requires Airflow 2.1.0+.
If your Airflow version is < 2.1.0, and you want to install this provider version, first upgrade
Airflow to at least version 2.1.0. Otherwise your Airflow package version will be upgraded
automatically and you will have to manually runairflow upgrade dbto complete the migration.1.0.2Fix 'logging.exception' redundancy (#14823)1.0.1Updated documentation and readme files.1.0.0Initial version of the provider.
Package: apache-airflow-providers-microsoft-azure
Packageapache-airflow-providers-microsoft-azureRelease:9.0.0Microsoft AzureProvider packageThis is a provider package formicrosoft.azureprovider. All classes for this provider package
are inairflow.providers.microsoft.azurepython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-microsoft-azureThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0adlfs>=2023.10.0azure-batch>=8.0.0azure-cosmos>=4.0.0azure-mgmt-cosmosdbazure-datalake-store>=0.0.45azure-identity>=1.3.1azure-keyvault-secrets>=4.1.0azure-mgmt-datalake-store>=0.5.0azure-mgmt-resource>=2.2.0azure-storage-blob>=12.14.0azure-mgmt-storage>=16.0.0azure-storage-file-shareazure-servicebus>=7.6.1azure-synapse-sparkazure-synapse-artifacts>=0.17.0adal>=1.2.7azure-storage-file-datalake>=12.9.1azure-kusto-data>=4.1.0azure-mgmt-datafactory>=2.0.0azure-mgmt-containerregistry>=8.0.0azure-mgmt-containerinstance>=9.0.0Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-microsoft-azure[google]Dependent packageExtraapache-airflow-providers-googlegoogleapache-airflow-providers-oracleoracleapache-airflow-providers-sftpsftpThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-microsoft-mssql
Packageapache-airflow-providers-microsoft-mssqlRelease:3.6.1Microsoft SQL Server (MSSQL)Provider packageThis is a provider package formicrosoft.mssqlprovider. All classes for this provider package
are inairflow.providers.microsoft.mssqlpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-microsoft-mssqlThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0apache-airflow-providers-common-sql>=1.3.1pymssql>=2.1.8Cross provider package dependenciesThose are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.You can install such cross-provider dependencies when installing from PyPI. For example:pipinstallapache-airflow-providers-microsoft-mssql[common.sql]Dependent packageExtraapache-airflow-providers-common-sqlcommon.sqlThe changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-microsoft-psrp
Packageapache-airflow-providers-microsoft-psrpRelease:2.5.0This package provides remote execution capabilities via thePowerShell Remoting Protocol (PSRP).Provider packageThis is a provider package formicrosoft.psrpprovider. All classes for this provider package
are inairflow.providers.microsoft.psrppython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-microsoft-psrpThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0pypsrp>=0.8.0The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-microsoft-winrm
Packageapache-airflow-providers-microsoft-winrmRelease:3.4.0Windows Remote Management (WinRM)Provider packageThis is a provider package formicrosoft.winrmprovider. All classes for this provider package
are inairflow.providers.microsoft.winrmpython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-microsoft-winrmThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0pywinrm>=0.4The changelog for the provider package can be found in thechangelog.
Package: apache-airflow-providers-mnb
MNB Provider for Apache AirflowThis package contains all the necessary operators, hooks and sensors necessary to access the daily exchange rates published by MNB (Central Bank of Hungary) in Apache Airflow.InstallationYou can install it with any package manager compatible with PyPI:pip install apache-airflow-provider-mnbComponentsComponentNotesMnbHookImplements low-level functions to interact with MNB's APIMnbExchangeRateOperatorProvides access to the exchange rates published on a specific dayMnbExchangeRateSensorSenses whether the rates were already published for a specific dayExampleThis following example demonstrates a typical use case:Start running at 8:00AM every dayCheck if the exchange rates were already published (repeat every 10 minutes)Permanently fail after 3 hours if the rates were never published (there are no rates available on the weekends and public holidays)Generate the SQL commandsRun them against a PostgreSQL databasefromdatetimeimportdateimportjsonimportpendulumfromairflow.decoratorsimportdag,taskfromairflow.providers.postgres.operators.postgresimportPostgresOperatorfromairflow.providers.mnb.sensors.mnbimportMnbExchangeRateSensorfromairflow.providers.mnb.operators.mnbimportMnbExchangeRateOperator@dag(dag_id="refresh_currency_rates",description="Refresh currency exchange rates",schedule="0 8 * * *",start_date=pendulum.datetime(2023,1,16,tz="Europe/Budapest"),catchup=False)defmnb():is_exchange_rate_available=MnbExchangeRateSensor(task_id="is_exchange_rate_available",timeout=10800,poke_interval=600,date="{{ ds }}")exchange_rates=MnbExchangeRateOperator(task_id="get_exchange_rates",date="{{ ds }}")@taskdefgenerate_queries(exchange_rates:str):rates=json.loads(exchange_rates)queries=""mnb_date=rates["date"]forrateinrates["rates"]:currency_id=rate["currency"]mnb_rate=rate["rate"]queries+=f"UPDATE finance.currency SET mnb_rate = '{mnb_rate}', mnb_date = '{mnb_date}' WHERE currency_id = '{currency_id}';\n"returnqueriesis_exchange_rate_available>>exchange_ratesqueries=generate_queries(exchange_rates.output)PostgresOperator(task_id="update_exchange_rates",postgres_conn_id="postgres_default",database="erp",sql=queries)mnb()
Package: apache-airflow-providers-mongo
Packageapache-airflow-providers-mongoRelease:4.0.0MongoDBProvider packageThis is a provider package formongoprovider. All classes for this provider package
are inairflow.providers.mongopython package.You can find package information and changelog for the provider
in thedocumentation.InstallationYou can install this package on top of an existing Airflow 2 installation (seeRequirementsbelow
for the minimum Airflow version supported) viapip installapache-airflow-providers-mongoThe package supports the following python versions: 3.8,3.9,3.10,3.11RequirementsPIP packageVersion requiredapache-airflow>=2.6.0dnspython>=1.13.0pymongo>=3.6.0The changelog for the provider package can be found in thechangelog.