Issue 1:
Body

We have a kind request for all the contributors to the latest Apache Airflow RC 2.3.3rc3.

Could you please help us to test the RC versions of Airflow?

Please let us know in the comment if the issue is addressed in the latest RC.

Switch Markdown engine to markdown-it-py (#19702): @uranusjr @torbjornvatn

Linked issues:

Switch Markdown engine to markdown-it-py (#16435)

Fix StatD timing metric units (#21106): @viktorvia

Linked issues:

Some timing metrics are in seconds but reported as milliseconds (#20804)

Parse error for task added to multiple groups (#23071): @uranusjr @wolfier

Linked issues:

Using TaskGroup without context manager (Graph view visual bug) (#14864)

Check bag DAG schedule_interval match tiemtable (#23113): @uranusjr @sdebruyn

Linked issues:

Show DAG parse error if 'timetable' and 'schedule_interval' are set together (#23087)

Allow for LOGGING_LEVEL=DEBUG (#23360): @GApfel

Fix closing connection dbapi.get_pandas_df (#23452): @bauerfranz @hubert-pietron

Linked issues:

closing connection chunks in DbApiHook.get_pandas_df (#22947)

Fix permission issue for dag that has dot in name (#23510): @ephraimbuddy

ExternalTaskSensor respects soft_fail if the external task enters a failed_state (#23647): @argibbs @alexbegg

Linked issues:

ExternalTaskSensor should skip if soft_fail=True and external task in one of the failed_states (#19754)

Fix mapped task immutability after clear (#23667): @ephraimbuddy

Maintain grid view selection on filtering upstream (#23779): @bbovenzi

Add TaskInstance State 'REMOVED' to finished states and success states (#23797): @ephraimbuddy

Do not fail requeued TIs (#23846): @malthe @tanelk

Linked issues:

Race condition between Triggerer and Scheduler (#23824)

Fix race condition between triggerer and scheduler (#21316)

Remove special serde logic for mapped op_kwargs (#23860): @uranusjr @erdos2n

Linked issues:

Dynamic Task Mapping not working with op_kwargs in PythonOperator (#23833)

Grid fix details button truncated and small UI tweaks (#23934): @pierrejeambrun

Replace use_task_execution_date with use_task_logical_date (#23983): @eladkal

Drop Python 3.6 compatibility objects/modules (#24048): @Taragolis

Removing magic numbers from api_connecxion (#24050): @bernardocouto

Rename Permissions to Permission Pairs. (#24065): @zachliu @gbonazzoli @jhtimmins

Linked issues:

Random "duplicate key value violates unique constraint" errors when initializing the postgres database (#23512)

Airflow 2.3.1 - gunicorn keeps removing and adding Permission menu access on Permissions to role Admin (#23926)

Handle occasional deadlocks in trigger with retries (#24071): @humit0 @potiuk

Linked issues:

Triggerer process die with DB Deadlock (#23639)

Reduce grid view API calls (#24083): @bbovenzi

Unify return_code interface for task runner (#24093): @pingzh

Do not support MSSQL less than v2017 in code (#24095): @ephraimbuddy

Refactor DagRun.verify_integrity (#24114): @ephraimbuddy

Don't crash scheduler if exec config has old k8s objects (#24117): @patryk126p @dstandish

Linked issues:

Airflow 2.3 scheduler error: 'V1Container' object has no attribute '_startup_probe' (#23727)

Fix useTasks crash on error (#24152): @bbovenzi

Apply per-run log templates to log handlers (#24153): @uranusjr

Prevent dag dependencies page from crashing (#24166): @bbovenzi

Update 'rich' to latest version across the board. (#24186): @potiuk

ScheduleInterval nullable true added in openapi spec (#24253): @omarsmak @Bowrna

Linked issues:

ScheduleInterval schema in OpenAPI specs should have "nullable: true" otherwise generated OpenAPI client will throw an error in case of nullable "schedule_interval" (#22358)

Use get_hostname instead of socket.getfqdn (#24260): @jedcunningham

Optimize calendar view for cron scheduled DAGs (#24262): @jedcunningham @potiuk

Linked issues:

Calendar view is taking too long to load in airflow 2.3.0 (#23602)

Speed up grid_data endpoint by 10x (#24284): @ashb

Fix bugs in URI constructor for MySQL connection (#24320): @MaksYermak

Check for run_id for grid task groups (#24327): @jedcunningham @bbovenzi

Linked issues:

Fix and speed up grid view (#23947)

Grid data: do not load all mapped instances (#23813)

TI.log_url fix for map_index (#24335): @fritz-astronomer

Linked issues:

TI.log_url is incorrect with mapped tasks (#24328)

Add typescript (#24337): @bbovenzi

DebugExecutor use ti.run() instead of ti._run_raw_task (#24357): @sarvothaman @o-nikolas

Linked issues:

Backfill crashes with "KeyError: TaskInstanceKey" when task has retries (#13322)

Mask secrets in stdout for CLI command (#24362): @uranusjr @alex-astronomer @marclamberti

Linked issues:

Mask secrets in stdout for airflow tasks test ... CLI command (#17476) (#21281)

Sensitive variables don't get masked when rendered with airflow tasks test (#17476)

Add tests for the grid_data endpoint (#24375): @ashb

Fix links to sources for examples (#24386): @potiuk

Upgrade FAB to 4.1.1 (#24399): @potiuk

Fix flaky order of returned dag runs (#24405): @potiuk

Improve grid rendering performance with a custom tooltip (#24417): @bbovenzi

Upgrade to react 18 and chakra 2 (#24430): @bbovenzi

Remove framer-motion from custom tooltip (#24449): @bbovenzi

Add missing types to FSHook (#24470): @lmarvaud

Return empty dict if Pod JSON encoding fails (#24478): @dstandish

Upgrade to webpack 5 (#24485): @bbovenzi

Add indexes for CASCADE deletes for task_instance (#24488): @wolfier @dstandish

Linked issues:

airflow db clean task_instance takes a long time (#24484)

Remove internet explorer support (#24495): @bbovenzi

Add cache_ok flag to sqlalchemy TypeDecorators. (#24499): @jkinkead @NK74

Linked issues:

SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the cache_ok attribute is not set to True (#22647)

Fix toast messages (#24505): @bbovenzi

Fix deprecated log_id_template value (#24506): @jedcunningham

Migrate jsx files that affect run/task selection to tsx (#24509): @bbovenzi

Seed log_template table (#24511): @jedcunningham

Update flask-appbuilder authlib/oauth dependency (#24516): @potiuk

Get rid of TimedJSONWebSignatureSerializer (#24519): @potiuk

Linked issues:

Upgrade to FlaskAppBuilder 4.0.* (#22397)

Do not calculate grid root instances (#24528): @bbovenzi

Patch sql_alchemy_conn if old Postgres schemes used (#24569): @slyapustin @dstandish

Linked issues:

Patch sql_alchemy_conn if old postgres scheme used (#22333)

Fix migration 0080_2_0_2 - Replace null values before setting column not null (#24585): @ephraimbuddy @tm-drtina

Linked issues:

Support generating SQL script for upgrades (#20962)

Migration changes column to NOT NULL without updating NULL data first (#24566)

Move fallible ti.task.dag assignment back inside try/except block (#24533) (#24592): @ejk43

Linked issues:

es_task_handler.py crashes accessing "View Logs in Elasticsearch" (#24533)

Upgrade FAB to 4.1.2 (#24619): @potiuk

We now need at least Flask-WTF 0.15 (#24621): @ashb @potiuk

Linked issues:

Upgrade FAB to 4.1.1 (#24399)

Don't render child rows for closed groups (#24637): @bbovenzi

Fix timestamp defaults for sensorinstance (#24638): @dstandish

Fix Grid vertical scrolling (#24684): @bbovenzi

Debounce status highlighting in Grid view (#24710): @bbovenzi

Address all yarn test warnings (#24722): @bbovenzi

Fix grid date ticks (#24738): @bbovenzi

Linked issues:

Fix Grid vertical scrolling (#24684)

Remove upper-binding for SQLAlchemy (#24819): @potiuk

Fix cycle bug with attaching label to task group (#24847): @ashb @rustikk

Linked issues:

Applying labels to task groups shows a cycle in the graph view for the dag (#19785)

improve grid date tick spacing (#24849): @bbovenzi

Fix exception in mini task scheduler. (#24865): @ashb

Thanks to all who contributed to the release (probably not a complete list!):

@Bowrna @GApfel @uranusjr @alexbegg @sdebruyn @NK74 @argibbs @slyapustin @tanelk @rustikk @omarsmak @hubert-pietron @ashb @marclamberti @eladkal @wolfier @jhtimmins @torbjornvatn @bernardocouto @gbonazzoli @erdos2n @sarvothaman @bbovenzi @dstandish @potiuk @jedcunningham @o-nikolas @MaksYermak @humit0 @tm-drtina @Taragolis @ejk43 @ephraimbuddy @patryk126p @malthe @viktorvia @fritz-astronomer @jkinkead @alex-astronomer @pierrejeambrun @pingzh @lmarvaud @zachliu @bauerfranz

Committer

I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Issue 2:
Apache Airflow version: 2.0.0a2

As of #9363, the tasks run command attempts to restore the root logger handlers to the previous state after running the task with the handlers all replaced by those from the airflow.tasks logger.

However, because no actual copy is created of the root handlers list, what you effectively end up with is either an empty list of handlers and nothing is restored. To make matters worse, the block ends with a logging.shutdown() call, which completely closes and releases all handlers.

So, either the code to 'restore' can just be removed, or a proper copy needs to be made and the logging.shutdown() call needs to be removed from the command. logging.shutdown is going to be called when the interpreter exits, anyway.

In detail, this is the code section:

airflow/airflow/cli/commands/task_command.py

Lines 194 to 222
in
bec9f3b

# Get all the Handlers from 'airflow.task' logger

# Add these handlers to the root logger so that we can get logs from

# any custom loggers defined in the DAG

airflow_logger_handlers = logging.getLogger('airflow.task').handlers

root_logger = logging.getLogger()

root_logger_handlers = root_logger.handlers

# Remove all handlers from Root Logger to avoid duplicate logs

for handler in root_logger_handlers:

root_logger.removeHandler(handler)

for handler in airflow_logger_handlers:

root_logger.addHandler(handler)

root_logger.setLevel(logging.getLogger('airflow.task').level)

with redirect_stdout(StreamLogWriter(ti.log, logging.INFO)), redirect_stderr(

StreamLogWriter(ti.log, logging.WARN)

):

_run_task_by_selected_method(args, dag, ti)

# We need to restore the handlers to the loggers as celery worker process

# can call this command multiple times,

# so if we don't reset this then logs from next task would go to the wrong place

for handler in airflow_logger_handlers:

root_logger.removeHandler(handler)

for handler in root_logger_handlers:

root_logger.addHandler(handler)

logging.shutdown()

The first problem here is that root_logger_handlers = root_logger.handlers merely creates a reference to the logger handlers list. The root_logger.removeHandler(handler) in a loop further on remove the handlers from that same list as you iterate, and this causes it to only remove every second handler. Luckily, this was never a problem because in the standard configuration there is only a single handler on the root. If there are more, there is a bigger problem further down.

Continuing on, the next loop, adding handlers from airflow_logger_handlers, causes those same handlers to show up in the root_logger_handlers reference, but those very same handlers are removed again in a second loop over the same airflow_logger_handlers list.

So when you hit the final for handler in root_logger_handlers: root_logger.addHandler(handler) loop, either root_logger_handlers is an empty list (no handlers added), or, if you started with more than one root handler and every second handler was left in place, you now are adding handlers to the list that are already in the list. The Logger.addHandler() method uses self.handler.append(), appending to a list you are iterating over, and so you have an infinite loop on your hands.

Either just remove all handlers from the root logger (taking care to loop over the list in reverse, e.g. for h in reversed(root_logger.handlers): root_logger.removeHandler(h)), or create proper copy of the list with root_logger_handlers = root_logger.handlers[:]. The logging.shutdown() call has to be removed then too.