Package: apache-airflow-backport-providers-apache-spark
Package apache-airflow-backport-providers-apache-sparkRelease: 2021.3.3Table of contentsBackport packageInstallationPIP requirementsProvider class summaryOperatorsMoved operatorsHooksMoved hooksReleasesRelease 2021.3.3Release 2020.10.29Release 2020.10.5Release 2020.6.24Backport packageThis is a backport providers package forapache.sparkprovider. All classes for this provider package
are inairflow.providers.apache.sparkpython package.Only Python 3.6+ is supported for this backport package.While Airflow 1.10.* continues to support Python 2.7+ - you need to upgrade python to 3.6+ if you
want to use this backport package.Release 2021.3.3Bug fixesUse apache.spark provider without kubernetes (#14187)InstallationYou can install this package on top of an existing airflow 1.10.* installation viapip install apache-airflow-backport-providers-apache-sparkPIP requirementsPIP packageVersion requiredpysparkProvider classes summaryIn Airflow 2.0, all operators, transfers, hooks, sensors, secrets for theapache.sparkprovider
are in theairflow.providers.apache.sparkpackage. You can read more about the naming conventions used
inNaming conventions for provider packagesOperatorsMoved operatorsAirflow 2.0 operators:airflow.providers.apache.sparkpackageAirflow 1.10.* previous location (usuallyairflow.contrib)operators.spark_jdbc.SparkJDBCOperatorcontrib.operators.spark_jdbc_operator.SparkJDBCOperatoroperators.spark_sql.SparkSqlOperatorcontrib.operators.spark_sql_operator.SparkSqlOperatoroperators.spark_submit.SparkSubmitOperatorcontrib.operators.spark_submit_operator.SparkSubmitOperatorHooksMoved hooksAirflow 2.0 hooks:airflow.providers.apache.sparkpackageAirflow 1.10.* previous location (usuallyairflow.contrib)hooks.spark_jdbc.SparkJDBCHookcontrib.hooks.spark_jdbc_hook.SparkJDBCHookhooks.spark_sql.SparkSqlHookcontrib.hooks.spark_sql_hook.SparkSqlHookhooks.spark_submit.SparkSubmitHookcontrib.hooks.spark_submit_hook.SparkSubmitHookReleasesRelease 2021.3.3CommitCommittedSubjectf9c9e9c382021-02-11Use apache.spark provider without kubernetes (#14187)88bdcfa0d2021-02-04Prepare to release a new wave of providers. (#14013)ac2f72c982021-02-01Implement provider versioning tools (#13767)a9ac2b0402021-01-23Switch to f-strings using flynt. (#13732)295d66f912020-12-30Fix Grammar in PIP warning (#13380)6cf76d7ac2020-12-18Fix typo in pip upgrade command :( (#13148)5090fb0c82020-12-15Add script to generate integrations.json (#13073)32971a1a22020-12-09Updates providers versions to 1.0.0 (#12955)b40dffa082020-12-08Rename remaing modules to match AIP-21 (#12917)9b39f24782020-12-08Add support for dynamic connection form fields per provider (#12558)bd90136aa2020-11-30Move operator guides to provider documentation packages (#12681)c34ef853c2020-11-20Separate out documentation building per provider  (#12444)0080354502020-11-18Update provider READMEs for 1.0.0b2 batch release (#12449)ae7cb4a1e2020-11-17Update wrong commit hash in backport provider changes (#12390)6889a333c2020-11-15Improvements for operators and hooks ref docs (#12366)7825e8f592020-11-13Docs installation improvements (#12304)85a18e13d2020-11-09Point at pypi project pages for cross-dependency of provider packages (#12212)59eb5de782020-11-09Update provider READMEs for up-coming 1.0.0beta1 releases (#12206)b2a28d1592020-11-09Moves provider packages scripts to dev (#12082)4e8f9cc8d2020-11-03Enable Black - Python Auto Formmatter (#9550)8c42cf1b02020-11-03Use PyUpgrade to use Python 3.6 features (#11447)5a439e84e2020-10-26Prepare providers release 0.0.2a1 (#11855)Release 2020.10.29CommitCommittedSubjectb680bbc0b2020-10-24Generated backport providers readmes/setup for 2020.10.29349b0811c2020-10-20Add D200 pydocstyle check (#11688)16e7129712020-10-13Added support for provider packages for Airflow 2.0 (#11487)d305876be2020-10-12Remove redundant None provided as default to dict.get() (#11448)0a0e1af802020-10-03Fix Broken Markdown links in Providers README TOC (#11249)Release 2020.10.5CommitCommittedSubjectca4238eb42020-10-02Fixed month in backport packages to October (#11242)5220e4c382020-10-02Prepare Backport release 2020.09.07 (#11238)f3e87c5032020-09-22Add D202 pydocstyle check (#11032)fdd9b6f652020-08-25Enable Black on Providers Packages (#10543)d760265452020-08-25PyDocStyle: No whitespaces allowed surrounding docstring text (#10533)d1bce91bb2020-08-25PyDocStyle: Enable D403: Capitalized first word of docstring (#10530)3696c34c22020-08-24Fix typo in the word "release" (#10528)ee7ca128a2020-08-22Fix broken Markdown refernces in Providers README (#10483)7c206a82a2020-08-22Replace assigment with Augmented assignment (#10468)3b3287d7a2020-08-05Enforce keyword only arguments on apache operators (#10170)7d24b088c2020-07-25Stop using start_date in default_args in example_dags (2) (#9985)33f0cd2652020-07-22apply_default keeps the function signature for mypy (#9784)1427e4acb2020-07-22Update Spark submit operator for Spark 3 support (#8730)4d74ac2112020-07-19Increase typing for Apache and http provider package (#9729)0873070e02020-07-11Mask other forms of password arguments in SparkSubmitOperator (#9615)13a827d802020-07-09Ensure Kerberos token is valid in SparkSubmitOperator before runningyarn kill(#9044)067806d592020-06-29Add tests for spark_jdbc_script (#9491)d0e7db4022020-06-19Fixed release number for fresh release (#9408)Release 2020.6.24CommitCommittedSubject12af6a0802020-06-19Final cleanup for 2020.6.23rc1 release preparation (#9404)c7e5bce572020-06-19Prepare backport release candidate for 2020.6.23rc1 (#9370)40bf8f28f2020-06-18Detect automatically the lack of reference to the guide in the operator descriptions (#9290)f6bd817a32020-06-16Introduce 'transfers' packages (#9320)0b0e4f7a42020-05-26Preparing for RC3 release of backports (#9026)00642a46d2020-05-26Fixed name of 20 remaining wrongly named operators. (#8994)375d1ca222020-05-19Release candidate 2 for backport packages 2020.05.20 (#8898)12c5e5d8a2020-05-17Prepare release candidate for backport packages (#8891)f3521fb0e2020-05-16Regenerate readme files for backport package release (#8886)92585ca4c2020-05-15Added automated release notes generation for backport operators (#8807)7506c73f12020-05-10Add defaultconfparameter to Spark JDBC Hook (#8787)487b5cc502020-05-06Add guide for Apache Spark operators (#8305)87969a3502020-04-09[AIRFLOW-6515] Change Log Levels from Info/Warn to Error (#8170)be1451b0e2020-04-02[AIRFLOW-7026] Improve SparkSqlHook's error message (#7749)4bde99f132020-03-23Make airflow/providers pylint compatible (#7802)7e6372a682020-03-23Add call to Super call in apache providers (#7820)2327aa5a22020-03-12[AIRFLOW-7025] Fix SparkSqlHook.run_query to handle its parameter properly (#7677)024b4bf962020-03-10[AIRFLOW-7024] Add the verbose parameter support to SparkSqlOperator (#7676)b59042b5a2020-02-28[AIRFLOW-6949] Respect explicitspark.kubernetes.namespaceconf to SparkSubmitOperator (#7575)97a429f9d2020-02-02[AIRFLOW-6714] Remove magic comments about UTF-8 (#7338)0481b9a952020-01-12[AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package (#7142)