CVE-2026-53122 btrfs: fix deadlock between reflink and transaction commit when using flushoncommit
In the Linux kernel, the following vulnerability has been resolved:

btrfs: fix deadlock between reflink and transaction commit when using flushoncommit

When using the flushoncommit mount option, we can have a deadlock between
a transaction commit and a reflink operation that copied an inline extent
to an offset beyond the current i_size of the destination node.

The deadlock happens like this:

1) Task A clones an inline extent from inode X to an offset of inode Y
   that is beyond Y's current i_size. This means we copied the inline
   extent's data to a folio of inode Y that is beyond its EOF, using a
   call to copy_inline_to_page();

2) Task B starts a transaction commit and calls
   btrfs_start_delalloc_flush() to flush delalloc;

3) The delalloc flushing sees the new dirty folio of inode Y and when it
   attempts to flush it, it ends up at extent_writepage() and sees that
   the offset of the folio is beyond the i_size of inode Y, so it attempts
   to invalidate the folio by calling folio_invalidate(), which ends up at
   btrfs' folio invalidate callback - btrfs_invalidate_folio(). There it
   tries to lock the folio's range in inode Y's extent io tree, but it
   blocks since it's currently locked by task A - during a reflink we lock
   the inodes and the source and destination ranges after flushing all
   delalloc and waiting for ordered extent completion - after that we
   don't expect to have dirty folios in the ranges, the exception is if
   we have to copy an inline extent's data (because the destination offset
   is not zero);

4) Task A then attempts to start a transaction to update the inode item,
   and then it's blocked since the current transaction is in the
   TRANS_STATE_COMMIT_START state. Therefore task A has to wait for the
   current transaction to become unblocked (its state >=
   TRANS_STATE_UNBLOCKED).

   So task A is waiting for the transaction commit done by task B, and
   the later waiting on the extent lock of inode Y that is currently
   held by task A.

Syzbot recently reported this with the following stack traces:

  INFO: task kworker/u8:7:1053 blocked for more than 143 seconds.
        Not tainted syzkaller #0
  "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  task:kworker/u8:7    state:D stack:23520 pid:1053  tgid:1053  ppid:2      task_flags:0x4208060 flags:0x00080000
  Workqueue: writeback wb_workfn (flush-btrfs-46)
  Call Trace:
   <TASK>
   context_switch kernel/sched/core.c:5298 [inline]
   __schedule+0x1553/0x5240 kernel/sched/core.c:6911
   __schedule_loop kernel/sched/core.c:6993 [inline]
   schedule+0x164/0x360 kernel/sched/core.c:7008
   wait_extent_bit fs/btrfs/extent-io-tree.c:811 [inline]
   btrfs_lock_extent_bits+0x59c/0x700 fs/btrfs/extent-io-tree.c:1914
   btrfs_lock_extent fs/btrfs/extent-io-tree.h:152 [inline]
   btrfs_invalidate_folio+0x43d/0xc40 fs/btrfs/inode.c:7704
   extent_writepage fs/btrfs/extent_io.c:1852 [inline]
   extent_write_cache_pages fs/btrfs/extent_io.c:2580 [inline]
   btrfs_writepages+0x12ff/0x2440 fs/btrfs/extent_io.c:2713
   do_writepages+0x32e/0x550 mm/page-writeback.c:2554
   __writeback_single_inode+0x133/0x11a0 fs/fs-writeback.c:1750
   writeback_sb_inodes+0x995/0x19d0 fs/fs-writeback.c:2042
   wb_writeback+0x456/0xb70 fs/fs-writeback.c:2227
   wb_do_writeback fs/fs-writeback.c:2374 [inline]
   wb_workfn+0x41a/0xf60 fs/fs-writeback.c:2414
   process_one_work kernel/workqueue.c:3276 [inline]
   process_scheduled_works+0xb6e/0x18c0 kernel/workqueue.c:3359
   worker_thread+0xa53/0xfc0 kernel/workqueue.c:3440
   kthread+0x388/0x470 kernel/kthread.c:436
   ret_from_fork+0x51e/0xb90 arch/x86/kernel/process.c:158
   ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
   </TASK>
  INFO: task syz.4.64:6910 blocked for more than 143 seconds.
        Not tainted syzkaller #0
  "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  task:syz.4.64        state:D stack:22752 pid:6910  tgid:
---truncated---
CVSS v3.1 score: 5.5