In the Linux kernel, the following vulnerability has been resolved:

btrfs: fix deadlock in wait_current_trans() due to ignored transaction type

When wait_current_trans() is called during start_transaction(), it currently waits for a blocked transaction without considering whether the given transaction type actually needs to wait for that particular transaction state. The btrfs_blocked_trans_types[] array already defines which transaction types should wait for which transaction states, but this check was missing in wait_current_trans().

This can lead to a deadlock scenario involving two transactions and pending ordered extents:

1. Transaction A is in TRANS_STATE_COMMIT_DOING state

2. A worker processing an ordered extent calls start_transaction() with TRANS_JOIN

3. join_transaction() returns -EBUSY because Transaction A is in TRANS_STATE_COMMIT_DOING

4. Transaction A moves to TRANS_STATE_UNBLOCKED and completes

5. A new Transaction B is created (TRANS_STATE_RUNNING)

6. The ordered extent from step 2 is added to Transaction Bs pending ordered extents

7. Transaction B immediately starts commit by another task and enters TRANS_STATE_COMMIT_START

8. The worker finally reaches wait_current_trans(), sees Transaction B in TRANS_STATE_COMMIT_START (a blocked state), and waits unconditionally

9. However, TRANS_JOIN should NOT wait for TRANS_STATE_COMMIT_START according to btrfs_blocked_trans_types[]

10. Transaction B is waiting for pending ordered extents to complete

11. Deadlock: Transaction B waits for ordered extent, ordered extent waits for Transaction B

This can be illustrated by the following call stacks: CPU0 CPU1 btrfs_finish_ordered_io() start_transaction(TRANS_JOIN) join_transaction() # -EBUSY (Transaction A is # TRANS_STATE_COMMIT_DOING)

Transaction A completes

Transaction B created

ordered extent added to

Transaction Bs pending list

btrfs_commit_transaction() # Transaction B enters # TRANS_STATE_COMMIT_START # waiting for pending ordered # extents wait_current_trans() # waits for Transaction B # (should not wait!)

Task bstore_kv_sync in btrfs_commit_transaction waiting for ordered extents:

__schedule+0x2e7/0x8a0 schedule+0x64/0xe0 btrfs_commit_transaction+0xbf7/0xda0 [btrfs] btrfs_sync_file+0x342/0x4d0 [btrfs] __x64_sys_fdatasync+0x4b/0x80 do_syscall_64+0x33/0x40 entry_SYSCALL_64_after_hwframe+0x44/0xa9

Task kworker in wait_current_trans waiting for transaction commit:

Workqueue: btrfs-syno_nocow btrfs_work_helper [btrfs] __schedule+0x2e7/0x8a0 schedule+0x64/0xe0 wait_current_trans+0xb0/0x110 [btrfs] start_transaction+0x346/0x5b0 [btrfs] btrfs_finish_ordered_io.isra.0+0x49b/0x9c0 [btrfs] btrfs_work_helper+0xe8/0x350 [btrfs] process_one_work+0x1d3/0x3c0 worker_thread+0x4d/0x3e0 kthread+0x12d/0x150 ret_from_fork+0x1f/0x30

Fix this by passing the transaction type to wait_current_trans() and checking btrfs_blocked_trans_types[cur_trans->state] against the given type before deciding to wait. This ensures that transaction types which are allowed to join during certain blocked states will not unnecessarily wait and cause deadlocks.

Affected Software

References

• https://git.kernel.org/stable/c/5037b342825df7094a4906d1e2a9674baab50cb2
• https://git.kernel.org/stable/c/8b0bb145d3bc264360f525c9717653be3522e528
• https://git.kernel.org/stable/c/99da896614d17e8a84aeb2b2d464ac046cc8633d
• https://git.kernel.org/stable/c/9ac63333d600732a56b35ee1fa46836da671eb50
• https://git.kernel.org/stable/c/d7b04b40ac8e6d814e35202a0e1568809b818295
• https://git.kernel.org/stable/c/dc84036c173cff6a432d9ab926298850b1d2a659
• https://git.kernel.org/stable/c/e563f59395981fcd69d130761290929806e728d6