Issue 1:
Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

Airflow Version: 2.4.2

In my organization, I need to run an SSHOperator to execute a script that triggers a spark-submit task. I encountered a situation where I had to rerun a specific execution, so I cleared the state of the task. However, I noticed that there were two distinct applications running on YARN when there should have been only one. This occurred three times in total.

From the log, I observed that each line is being logged twice. I'm curious to know if there is any connection or reason behind this duplication. I have remote_logging to be true.

What you think should happen instead

No response

How to reproduce

Clear the state of Task.

But I can't produce it several hours later.

Operating System

Debian GNU/Linux 10 (buster)

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

Issue 2:
Body

I have a kind request for all the contributors to the latest provider packages release.

Could you please help us to test the RC versions of the providers?

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider alibaba: 2.1.0rc1

Auto tail file logs in Web UI (#26169): @jens-scheffler-bosch

Provider amazon: 6.0.0rc1

Remove Amazon S3 Connection Type (#25980): @Taragolis

Add RdsDbSensor to amazon provider package (#26003): @hankehly

Set template_fields on RDS operators (#26005): @hankehly

Auto tail file logs in Web UI (#26169): @jens-scheffler-bosch

Fix SageMakerEndpointConfigOperator's return value (#26541): @ferruzzi

EMR Serverless Fix for Jobs marked as success even on failure (#26218): @syedahsn

Fix AWS Connection warn condition for invalid profile_name argument (#26464): @Taragolis

Athena and EMR operator max_retries mix-up fix (#25971): @syedahsn

Fixes SageMaker operator return values (#23628): @ferruzzi

Remove redundant catch exceptions in Amazon Log Task Handlers (#26442): @Taragolis

Remove duplicated connection-type within the provider (#26628): @Taragolis

Provider apache.drill: 2.2.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider apache.druid: 3.2.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider apache.hive: 4.0.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider apache.pinot: 3.2.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Fix PinotDB dependencies (#26705): @potiuk

Provider databricks: 3.3.0rc1

DatabricksSubmitRunOperator dbt task support (#25623): @gaborratky-db

Add common-sql lower bound for common-sql (#25789): @potiuk

Remove duplicated connection-type within the provider (#26628): @Taragolis

Databricks: fix provider name in the User-Agent string (#25873): @alexott

Provider dbt.cloud: 2.2.0rc1

Add DbtCloudListJobsOperator (#26475): @phanikumv

Provider docker: 3.2.0rc1

Add logging options to docker operator (#26653): @nshetty15

Add pre-commit hook for custom_operator_name (#25786): @uranusjr

Implement ExternalPythonOperator (#25780): @potiuk

Provider elasticsearch: 4.2.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider exasol: 4.0.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider google: 8.4.0rc1

Add BigQuery Column and Table Check Operators (#26368): @denimalpaca

Add deferrable big query operators and sensors (#26156): @phanikumv

Add output property to MappedOperator (#25604): @josh-fell

Added append_job_name parameter to DataflowTemplatedJobStartOperator (#25746): @Voldurk

Adding a parameter for exclusion of trashed files in GoogleDriveHook (#25675): @alexkruc

Cloud Data Loss Prevention Operators assets (AIP-47) (#26618): @bkossakowska

Cloud Storage Transfer Operators assets & system tests migration (AIP-47) (#26072): @bkossakowska

Merge deferrable BigQuery operators to exisitng one (#26433): @lwyszomi

specifying project id when calling wait_for_operation in delete/create cluster (#26418): @SPTKL

Auto tail file logs in Web UI (#26169): @jens-scheffler-bosch

Cloud Functions Operators assets & system tests migration (AIP-47) (#26073): @bkossakowska

GCSToBigQueryOperator Resolve max_id_key job retrieval and xcom return (#26285): @patricker

Allow for the overriding of stringify_dict for json export format on BaseSQLToGCSOperator (#26277): @patricker

Use common url base for Google links (#26057): @bhirsz

Cloud Video Intelligence Operators assets & system tests migration (AIP-47) (#26132): @bkossakowska

Life Science assets & system tests migration (AIP-47) (#25548): @bkossakowska

GCSToBigQueryOperator allow for schema_object in alternate GCS Bucket (#26190): @patricker

Use AsyncClient for Composer Operators in deferrable mode (#25951): @lwyszomi

Fix BigQueryCreateEmptyDatasetOperator to use the provided project_id (#25984): @phanikumv

Cloud Build assets & system tests migration (AIP-47) (#25895): @bkossakowska

Dataproc submit job operator async (#25302): @bjankie1

Support project_id argument in BigQueryGetDataOperator (#25782): @sudohainguyen

Fix JSONDecodeError in Datafusion operators (#26202): @VladaZakharova

Fixed never ending loop in CreateWorkflowInvocation (#25737): @lwyszomi

Fix error message on gcs.py (#26570): @Corea

Don't throw an exception when a BQ cursor job has no schema (#26096): @patricker

Google Cloud Tasks Sensor for queue being empty (#25622): @vinitpayal

SQL to GCS operators update docs for parquet format (#25878): @pierrejeambrun

Limit Google Protobuf for compatibility with biggtable client (#25886): @potiuk

Make GoogleBaseHook credentials functions public (#25785): @feluelle

Consolidate to one schedule param (#25410): @dstandish

Provider jdbc: 3.2.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider jira: 3.1.0rc1

Deprecate jira provider in favor of atlassian.jira provider (#25930): @eladkal

Provider microsoft.azure: 4.3.0rc1

Add DataFlow operations to Azure DataFactory hook (#26345): @phanikumv

Add network_profile param in AzureContainerInstancesOperator (#26117): @pankajastro

Implement Azure synapse operator (#26038): @phanikumv

Auto tail file logs in Web UI (#26169): @jens-scheffler-bosch

Implement Azure Service Bus Topic (Create, Delete) Operators (#25436): @bharanidharan14

Fix AzureBatchOperator false negative status (#25844): @pankajastro

Provider microsoft.mssql: 3.2.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider mysql: 3.2.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider odbc: 3.1.2rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider oracle: 3.4.0rc1

Add oracledb thick mode support for oracle provider (#26576): @pauldalewilliams

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider postgres: 5.2.2rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

Rename schema to database in PostgresHook (#26436): @feluelle

Revert "Rename schema to database in PostgresHook (#26436)" (#26734): @jedcunningham

Provider qubole: 3.2.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

D400 first line should end with period batch02 (#25268): @edithturn

Provider sftp: 4.1.0rc1

SFTPOperator - add support for list of file paths (#26666): @pauldalewilliams

Provider slack: 6.0.0rc1

Refactor SlackWebhookOperator: Get rid of mandatory http-provider dependency (#26648): @Taragolis

Refactor SlackWebhookHook in order to use slack_sdk instead of HttpHook methods (#26452): @Taragolis

Move send_file method into SlackHook (#26118): @Taragolis

Refactor Slack API Hook and add new Connection type (#25852): @Taragolis

Remove unsafe imports in Slack API Connection (#26459): @Taragolis

Add common-sql lower bound for common-sql (#25789): @potiuk

Provider snowflake: 3.3.0rc1

Add custom handler param in SnowflakeOperator (#25983): @pankajastro

Fix wrong deprecation warning for S3ToSnowflakeOperator (#26047): @eladkal

Provider ssh: 3.2.0rc1

SSHHook: Saving new host keys as described in docs (#25979): @doiken

Provider trino: 4.1.0rc1

trino: Support CertificateAuthentication in the trino hook (#26246): @c2zwdjnlcg

Provider vertica: 3.2.1rc1

Add common-sql lower bound for common-sql (#25789): @potiuk

The guidelines on how to test providers can be found in

Verify providers by contributors

Committer

I acknowledge that I am a maintainer/committer of the Apache Airflow project.