In the Linux kernel, the following vulnerability has been resolved:
rcu/nocb: Fix missed RCU barrier on deoffloading
Currently, running rcutorture test with torture_type=rcu fwd_progress=8 n_barrier_cbs=8 nocbs_nthreads=8 nocbs_toggle=100 onoff_interval=60 test_boost=2, will trigger the following warning:
WARNING: CPU: 19 PID: 100 at kernel/rcu/tree_nocb.h:1061 rcu_nocb_rdp_deoffload+0x292/0x2a0
RIP: 0010:rcu_nocb_rdp_deoffload+0x292/0x2a0
Call Trace:
<TASK>
? __warn+0x7e/0x120
? rcu_nocb_rdp_deoffload+0x292/0x2a0
? report_bug+0x18e/0x1a0
? handle_bug+0x3d/0x70
? exc_invalid_op+0x18/0x70
? asm_exc_invalid_op+0x1a/0x20
? rcu_nocb_rdp_deoffload+0x292/0x2a0
rcu_nocb_cpu_deoffload+0x70/0xa0
rcu_nocb_toggle+0x136/0x1c0
? __pfx_rcu_nocb_toggle+0x10/0x10
kthread+0xd1/0x100
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2f/0x50
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>
CPU0 CPU2 CPU3 //rcu_nocb_toggle //nocb_cb_wait //rcutorture
// deoffload CPU1 // process CPU1s rdp rcu_barrier() rcu_segcblist_entrain() rcu_segcblist_add_len(1); // len == 2 // enqueue barrier // callback to CPU1s // rdp->cblist rcu_do_batch() // invoke CPU1s rdp->cblist // callback rcu_barrier_callback() rcu_barrier() mutex_lock(&rcu_state.barrier_mutex); // still see len == 2 // enqueue barrier callback // to CPU1s rdp->cblist rcu_segcblist_entrain() rcu_segcblist_add_len(1); // len == 3 // decrement len rcu_segcblist_add_len(-2); kthread_parkme()
// CPU1s rdp->cblist len == 1 // Warn because there is // still a pending barrier // trigger warning WARN_ON_ONCE(rcu_segcblist_n_cbs(&rdp->cblist)); cpus_read_unlock();
// wait CPU1 to comes online and
// invoke barrier callback on
// CPU1 rdps->cblist
wait_for_completion(&rcu_state.barrier_completion);
// deoffload CPU4 cpus_read_lock() rcu_barrier() mutex_lock(&rcu_state.barrier_mutex); // block on barrier_mutex // wait rcu_barrier() on // CPU3 to unlock barrier_mutex // but CPU3 unlock barrier_mutex // need to wait CPU1 comes online // when CPU1 going online will block on cpus_write_lock
The above scenario will not only trigger a WARN_ON_ONCE(), but also trigger a deadlock.
Thanks to nocb locking, a second racing rcu_barrier() on an offline CPU will either observe the decremented callback counter down to 0 and spare the callback enqueue, or rcuo will observe the new callback and keep rdp->nocb_cb_sleep to false.
Therefore check rdp->nocb_cb_sleep before parking to make sure no further rcu_barrier() is waiting on the rdp.
Name | Vendor | Start Version | End Version |
---|---|---|---|
Linux-allwinner-5.19 | Ubuntu | jammy | * |
Linux-allwinner-5.19 | Ubuntu | upstream | * |
Linux-aws-5.0 | Ubuntu | esm-infra/bionic | * |
Linux-aws-5.0 | Ubuntu | upstream | * |
Linux-aws-5.11 | Ubuntu | focal | * |
Linux-aws-5.11 | Ubuntu | upstream | * |
Linux-aws-5.13 | Ubuntu | focal | * |
Linux-aws-5.13 | Ubuntu | upstream | * |
Linux-aws-5.19 | Ubuntu | jammy | * |
Linux-aws-5.19 | Ubuntu | upstream | * |
Linux-aws-5.3 | Ubuntu | esm-infra/bionic | * |
Linux-aws-5.3 | Ubuntu | upstream | * |
Linux-aws-5.8 | Ubuntu | focal | * |
Linux-aws-5.8 | Ubuntu | upstream | * |
Linux-aws-6.2 | Ubuntu | jammy | * |
Linux-aws-6.2 | Ubuntu | upstream | * |
Linux-aws-6.5 | Ubuntu | jammy | * |
Linux-aws-6.5 | Ubuntu | upstream | * |
Linux-azure | Ubuntu | esm-infra/bionic | * |
Linux-azure-5.11 | Ubuntu | focal | * |
Linux-azure-5.11 | Ubuntu | upstream | * |
Linux-azure-5.13 | Ubuntu | focal | * |
Linux-azure-5.13 | Ubuntu | upstream | * |
Linux-azure-5.19 | Ubuntu | jammy | * |
Linux-azure-5.19 | Ubuntu | upstream | * |
Linux-azure-5.3 | Ubuntu | esm-infra/bionic | * |
Linux-azure-5.3 | Ubuntu | upstream | * |
Linux-azure-5.8 | Ubuntu | focal | * |
Linux-azure-5.8 | Ubuntu | upstream | * |
Linux-azure-6.2 | Ubuntu | jammy | * |
Linux-azure-6.2 | Ubuntu | upstream | * |
Linux-azure-6.5 | Ubuntu | jammy | * |
Linux-azure-6.5 | Ubuntu | upstream | * |
Linux-azure-edge | Ubuntu | esm-infra/bionic | * |
Linux-azure-edge | Ubuntu | upstream | * |
Linux-azure-fde | Ubuntu | focal | * |
Linux-azure-fde-5.19 | Ubuntu | jammy | * |
Linux-azure-fde-5.19 | Ubuntu | upstream | * |
Linux-azure-fde-6.2 | Ubuntu | jammy | * |
Linux-azure-fde-6.2 | Ubuntu | upstream | * |
Linux-gcp | Ubuntu | esm-infra/bionic | * |
Linux-gcp-5.11 | Ubuntu | focal | * |
Linux-gcp-5.11 | Ubuntu | upstream | * |
Linux-gcp-5.13 | Ubuntu | focal | * |
Linux-gcp-5.13 | Ubuntu | upstream | * |
Linux-gcp-5.19 | Ubuntu | jammy | * |
Linux-gcp-5.19 | Ubuntu | upstream | * |
Linux-gcp-5.3 | Ubuntu | esm-infra/bionic | * |
Linux-gcp-5.3 | Ubuntu | upstream | * |
Linux-gcp-5.8 | Ubuntu | focal | * |
Linux-gcp-5.8 | Ubuntu | upstream | * |
Linux-gcp-6.2 | Ubuntu | jammy | * |
Linux-gcp-6.2 | Ubuntu | upstream | * |
Linux-gcp-6.5 | Ubuntu | jammy | * |
Linux-gcp-6.5 | Ubuntu | upstream | * |
Linux-gke | Ubuntu | focal | * |
Linux-gke-4.15 | Ubuntu | esm-infra/bionic | * |
Linux-gke-4.15 | Ubuntu | upstream | * |
Linux-gke-5.15 | Ubuntu | focal | * |
Linux-gke-5.15 | Ubuntu | upstream | * |
Linux-gke-5.4 | Ubuntu | esm-infra/bionic | * |
Linux-gke-5.4 | Ubuntu | upstream | * |
Linux-gkeop-5.4 | Ubuntu | esm-infra/bionic | * |
Linux-gkeop-5.4 | Ubuntu | upstream | * |
Linux-hwe | Ubuntu | esm-infra/bionic | * |
Linux-hwe-5.11 | Ubuntu | focal | * |
Linux-hwe-5.11 | Ubuntu | upstream | * |
Linux-hwe-5.13 | Ubuntu | focal | * |
Linux-hwe-5.13 | Ubuntu | upstream | * |
Linux-hwe-5.19 | Ubuntu | jammy | * |
Linux-hwe-5.19 | Ubuntu | upstream | * |
Linux-hwe-5.8 | Ubuntu | focal | * |
Linux-hwe-5.8 | Ubuntu | upstream | * |
Linux-hwe-6.2 | Ubuntu | jammy | * |
Linux-hwe-6.2 | Ubuntu | upstream | * |
Linux-hwe-6.5 | Ubuntu | jammy | * |
Linux-hwe-6.5 | Ubuntu | upstream | * |
Linux-hwe-edge | Ubuntu | esm-infra/bionic | * |
Linux-hwe-edge | Ubuntu | esm-infra/xenial | * |
Linux-hwe-edge | Ubuntu | upstream | * |
Linux-intel-5.13 | Ubuntu | focal | * |
Linux-intel-5.13 | Ubuntu | upstream | * |
Linux-lowlatency-hwe-5.19 | Ubuntu | jammy | * |
Linux-lowlatency-hwe-5.19 | Ubuntu | upstream | * |
Linux-lowlatency-hwe-6.2 | Ubuntu | jammy | * |
Linux-lowlatency-hwe-6.2 | Ubuntu | upstream | * |
Linux-lowlatency-hwe-6.5 | Ubuntu | jammy | * |
Linux-lowlatency-hwe-6.5 | Ubuntu | upstream | * |
Linux-nvidia-6.2 | Ubuntu | jammy | * |
Linux-nvidia-6.2 | Ubuntu | upstream | * |
Linux-nvidia-6.5 | Ubuntu | jammy | * |
Linux-nvidia-6.5 | Ubuntu | upstream | * |
Linux-oem | Ubuntu | esm-infra/bionic | * |
Linux-oem | Ubuntu | upstream | * |
Linux-oem-5.10 | Ubuntu | focal | * |
Linux-oem-5.10 | Ubuntu | upstream | * |
Linux-oem-5.13 | Ubuntu | focal | * |
Linux-oem-5.13 | Ubuntu | upstream | * |
Linux-oem-5.14 | Ubuntu | focal | * |
Linux-oem-5.14 | Ubuntu | upstream | * |
Linux-oem-5.17 | Ubuntu | jammy | * |
Linux-oem-5.17 | Ubuntu | upstream | * |
Linux-oem-5.6 | Ubuntu | focal | * |
Linux-oem-5.6 | Ubuntu | upstream | * |
Linux-oem-6.0 | Ubuntu | jammy | * |
Linux-oem-6.0 | Ubuntu | upstream | * |
Linux-oem-6.1 | Ubuntu | jammy | * |
Linux-oem-6.1 | Ubuntu | upstream | * |
Linux-oem-6.5 | Ubuntu | jammy | * |
Linux-oem-6.5 | Ubuntu | upstream | * |
Linux-oracle-5.0 | Ubuntu | esm-infra/bionic | * |
Linux-oracle-5.0 | Ubuntu | upstream | * |
Linux-oracle-5.11 | Ubuntu | focal | * |
Linux-oracle-5.11 | Ubuntu | upstream | * |
Linux-oracle-5.13 | Ubuntu | focal | * |
Linux-oracle-5.13 | Ubuntu | upstream | * |
Linux-oracle-5.3 | Ubuntu | esm-infra/bionic | * |
Linux-oracle-5.3 | Ubuntu | upstream | * |
Linux-oracle-5.8 | Ubuntu | focal | * |
Linux-oracle-5.8 | Ubuntu | upstream | * |
Linux-oracle-6.5 | Ubuntu | jammy | * |
Linux-oracle-6.5 | Ubuntu | upstream | * |
Linux-raspi2 | Ubuntu | focal | * |
Linux-raspi2 | Ubuntu | upstream | * |
Linux-realtime | Ubuntu | jammy | * |
Linux-riscv | Ubuntu | focal | * |
Linux-riscv | Ubuntu | jammy | * |
Linux-riscv-5.11 | Ubuntu | focal | * |
Linux-riscv-5.11 | Ubuntu | upstream | * |
Linux-riscv-5.19 | Ubuntu | jammy | * |
Linux-riscv-5.19 | Ubuntu | upstream | * |
Linux-riscv-5.8 | Ubuntu | focal | * |
Linux-riscv-5.8 | Ubuntu | upstream | * |
Linux-riscv-6.5 | Ubuntu | jammy | * |
Linux-riscv-6.5 | Ubuntu | upstream | * |
Linux-starfive-5.19 | Ubuntu | jammy | * |
Linux-starfive-5.19 | Ubuntu | upstream | * |
Linux-starfive-6.2 | Ubuntu | jammy | * |
Linux-starfive-6.2 | Ubuntu | upstream | * |
Linux-starfive-6.5 | Ubuntu | jammy | * |
Linux-starfive-6.5 | Ubuntu | upstream | * |