CVE Vulnerabilities

CVE-2024-53169

Published: Dec 27, 2024 | Modified: Dec 27, 2024
CVSS 3.x
N/A
Source:
NVD
CVSS 2.x
RedHat/V2
RedHat/V3
5.8 MODERATE
CVSS:3.1/AV:L/AC:H/PR:L/UI:N/S:U/C:L/I:L/A:H
Ubuntu
MEDIUM

In the Linux kernel, the following vulnerability has been resolved:

nvme-fabrics: fix kernel crash while shutting down controller

The nvme keep-alive operation, which executes at a periodic interval, could potentially sneak in while shutting down a fabric controller. This may lead to a race between the fabric controller admin queue destroy code path (invoked while shutting down controller) and hw/hctx queue dispatcher called from the nvme keep-alive async request queuing operation. This race could lead to the kernel crash shown below:

Call Trace: autoremove_wake_function+0x0/0xbc (unreliable) __blk_mq_sched_dispatch_requests+0x114/0x24c blk_mq_sched_dispatch_requests+0x44/0x84 blk_mq_run_hw_queue+0x140/0x220 nvme_keep_alive_work+0xc8/0x19c [nvme_core] process_one_work+0x200/0x4e0 worker_thread+0x340/0x504 kthread+0x138/0x140 start_kernel_thread+0x14/0x18

While shutting down fabric controller, if nvme keep-alive request sneaks in then it would be flushed off. The nvme_keep_alive_end_io function is then invoked to handle the end of the keep-alive operation which decrements the admin->q_usage_counter and assuming this is the last/only request in the admin queue then the admin->q_usage_counter becomes zero. If that happens then blk-mq destroy queue operation (blk_mq_destroy_ queue()) which could be potentially running simultaneously on another cpu (as this is the controller shutdown code path) would forward progress and deletes the admin queue. So, now from this point onward we are not supposed to access the admin queue resources. However the issue heres that the nvme keep-alive thread running hw/hctx queue dispatch operation hasnt yet finished its work and so it could still potentially access the admin queue resource while the admin queue had been already deleted and that causes the above crash.

The above kernel crash is regression caused due to changes implemented in commit a54a93d0e359 (nvme: move stopping keep-alive into nvme_uninit_ctrl()). Ideally we should stop keep-alive before destroyin g the admin queue and freeing the admin tagset so that it wouldnt sneak in during the shutdown operation. However we removed the keep alive stop operation from the beginning of the controller shutdown code path in commit a54a93d0e359 (nvme: move stopping keep-alive into nvme_uninit_ctrl()) and added it under nvme_uninit_ctrl() which executes very late in the shutdown code path after the admin queue is destroyed and its tagset is removed. So this change created the possibility of keep-alive sneaking in and interfering with the shutdown operation and causing observed kernel crash.

To fix the observed crash, we decided to move nvme_stop_keep_alive() from nvme_uninit_ctrl() to nvme_remove_admin_tag_set(). This change would ensure that we dont forward progress and delete the admin queue until the keep- alive operation is finished (if its in-flight) or cancelled and that would help contain the race condition explained above and hence avoid the crash.

Moving nvme_stop_keep_alive() to nvme_remove_admin_tag_set() instead of adding nvme_stop_keep_alive() to the beginning of the controller shutdown code path in nvme_stop_ctrl(), as was the case earlier before commit a54a93d0e359 (nvme: move stopping keep-alive into nvme_uninit_ctrl()), would help save one callsite of nvme_stop_keep_alive().

Affected Software

Name Vendor Start Version End Version
Linux Ubuntu noble *
Linux Ubuntu oracular *
Linux Ubuntu upstream *
Linux-allwinner-5.19 Ubuntu jammy *
Linux-allwinner-5.19 Ubuntu upstream *
Linux-aws Ubuntu noble *
Linux-aws Ubuntu oracular *
Linux-aws Ubuntu upstream *
Linux-aws-5.0 Ubuntu bionic *
Linux-aws-5.0 Ubuntu esm-infra/bionic *
Linux-aws-5.0 Ubuntu upstream *
Linux-aws-5.11 Ubuntu focal *
Linux-aws-5.11 Ubuntu upstream *
Linux-aws-5.13 Ubuntu focal *
Linux-aws-5.13 Ubuntu upstream *
Linux-aws-5.15 Ubuntu upstream *
Linux-aws-5.19 Ubuntu jammy *
Linux-aws-5.19 Ubuntu upstream *
Linux-aws-5.3 Ubuntu bionic *
Linux-aws-5.3 Ubuntu esm-infra/bionic *
Linux-aws-5.3 Ubuntu upstream *
Linux-aws-5.4 Ubuntu upstream *
Linux-aws-5.8 Ubuntu focal *
Linux-aws-5.8 Ubuntu upstream *
Linux-aws-6.2 Ubuntu jammy *
Linux-aws-6.2 Ubuntu upstream *
Linux-aws-6.5 Ubuntu jammy *
Linux-aws-6.5 Ubuntu upstream *
Linux-aws-6.8 Ubuntu jammy *
Linux-aws-6.8 Ubuntu upstream *
Linux-aws-fips Ubuntu trusty *
Linux-aws-fips Ubuntu upstream *
Linux-aws-fips Ubuntu xenial *
Linux-aws-hwe Ubuntu upstream *
Linux-azure Ubuntu bionic *
Linux-azure Ubuntu esm-infra/bionic *
Linux-azure Ubuntu noble *
Linux-azure Ubuntu oracular *
Linux-azure Ubuntu upstream *
Linux-azure-4.15 Ubuntu upstream *
Linux-azure-5.11 Ubuntu focal *
Linux-azure-5.11 Ubuntu upstream *
Linux-azure-5.13 Ubuntu focal *
Linux-azure-5.13 Ubuntu upstream *
Linux-azure-5.15 Ubuntu upstream *
Linux-azure-5.19 Ubuntu jammy *
Linux-azure-5.19 Ubuntu upstream *
Linux-azure-5.3 Ubuntu bionic *
Linux-azure-5.3 Ubuntu esm-infra/bionic *
Linux-azure-5.3 Ubuntu upstream *
Linux-azure-5.4 Ubuntu upstream *
Linux-azure-5.8 Ubuntu focal *
Linux-azure-5.8 Ubuntu upstream *
Linux-azure-6.2 Ubuntu jammy *
Linux-azure-6.2 Ubuntu upstream *
Linux-azure-6.5 Ubuntu jammy *
Linux-azure-6.5 Ubuntu upstream *
Linux-azure-6.8 Ubuntu jammy *
Linux-azure-6.8 Ubuntu upstream *
Linux-azure-edge Ubuntu bionic *
Linux-azure-edge Ubuntu esm-infra/bionic *
Linux-azure-edge Ubuntu upstream *
Linux-azure-fde Ubuntu focal *
Linux-azure-fde Ubuntu upstream *
Linux-azure-fde-5.15 Ubuntu upstream *
Linux-azure-fde-5.19 Ubuntu jammy *
Linux-azure-fde-5.19 Ubuntu upstream *
Linux-azure-fde-6.2 Ubuntu jammy *
Linux-azure-fde-6.2 Ubuntu upstream *
Linux-azure-fips Ubuntu trusty *
Linux-azure-fips Ubuntu upstream *
Linux-azure-fips Ubuntu xenial *
Linux-bluefield Ubuntu upstream *
Linux-fips Ubuntu upstream *
Linux-gcp Ubuntu bionic *
Linux-gcp Ubuntu esm-infra/bionic *
Linux-gcp Ubuntu noble *
Linux-gcp Ubuntu oracular *
Linux-gcp Ubuntu upstream *
Linux-gcp-4.15 Ubuntu upstream *
Linux-gcp-5.11 Ubuntu focal *
Linux-gcp-5.11 Ubuntu upstream *
Linux-gcp-5.13 Ubuntu focal *
Linux-gcp-5.13 Ubuntu upstream *
Linux-gcp-5.15 Ubuntu upstream *
Linux-gcp-5.19 Ubuntu jammy *
Linux-gcp-5.19 Ubuntu upstream *
Linux-gcp-5.3 Ubuntu bionic *
Linux-gcp-5.3 Ubuntu esm-infra/bionic *
Linux-gcp-5.3 Ubuntu upstream *
Linux-gcp-5.4 Ubuntu upstream *
Linux-gcp-5.8 Ubuntu focal *
Linux-gcp-5.8 Ubuntu upstream *
Linux-gcp-6.2 Ubuntu jammy *
Linux-gcp-6.2 Ubuntu upstream *
Linux-gcp-6.5 Ubuntu jammy *
Linux-gcp-6.5 Ubuntu upstream *
Linux-gcp-6.8 Ubuntu jammy *
Linux-gcp-6.8 Ubuntu upstream *
Linux-gcp-fips Ubuntu trusty *
Linux-gcp-fips Ubuntu upstream *
Linux-gcp-fips Ubuntu xenial *
Linux-gke Ubuntu focal *
Linux-gke Ubuntu noble *
Linux-gke Ubuntu upstream *
Linux-gke Ubuntu xenial *
Linux-gke-4.15 Ubuntu bionic *
Linux-gke-4.15 Ubuntu esm-infra/bionic *
Linux-gke-4.15 Ubuntu upstream *
Linux-gke-5.15 Ubuntu focal *
Linux-gke-5.15 Ubuntu upstream *
Linux-gke-5.4 Ubuntu bionic *
Linux-gke-5.4 Ubuntu esm-infra/bionic *
Linux-gke-5.4 Ubuntu upstream *
Linux-gkeop Ubuntu focal *
Linux-gkeop Ubuntu noble *
Linux-gkeop Ubuntu upstream *
Linux-gkeop-5.15 Ubuntu focal *
Linux-gkeop-5.15 Ubuntu upstream *
Linux-gkeop-5.4 Ubuntu bionic *
Linux-gkeop-5.4 Ubuntu esm-infra/bionic *
Linux-gkeop-5.4 Ubuntu upstream *
Linux-hwe Ubuntu bionic *
Linux-hwe Ubuntu esm-infra/bionic *
Linux-hwe Ubuntu upstream *
Linux-hwe-5.11 Ubuntu focal *
Linux-hwe-5.11 Ubuntu upstream *
Linux-hwe-5.13 Ubuntu focal *
Linux-hwe-5.13 Ubuntu upstream *
Linux-hwe-5.15 Ubuntu upstream *
Linux-hwe-5.19 Ubuntu jammy *
Linux-hwe-5.19 Ubuntu upstream *
Linux-hwe-5.4 Ubuntu upstream *
Linux-hwe-5.8 Ubuntu focal *
Linux-hwe-5.8 Ubuntu upstream *
Linux-hwe-6.11 Ubuntu upstream *
Linux-hwe-6.2 Ubuntu jammy *
Linux-hwe-6.2 Ubuntu upstream *
Linux-hwe-6.5 Ubuntu jammy *
Linux-hwe-6.5 Ubuntu upstream *
Linux-hwe-6.8 Ubuntu jammy *
Linux-hwe-6.8 Ubuntu upstream *
Linux-hwe-edge Ubuntu bionic *
Linux-hwe-edge Ubuntu esm-infra/bionic *
Linux-hwe-edge Ubuntu esm-infra/xenial *
Linux-hwe-edge Ubuntu upstream *
Linux-hwe-edge Ubuntu xenial *
Linux-ibm Ubuntu noble *
Linux-ibm Ubuntu upstream *
Linux-ibm-5.15 Ubuntu upstream *
Linux-ibm-5.4 Ubuntu upstream *
Linux-intel Ubuntu noble *
Linux-intel Ubuntu upstream *
Linux-intel-5.13 Ubuntu focal *
Linux-intel-5.13 Ubuntu upstream *
Linux-intel-iot-realtime Ubuntu upstream *
Linux-intel-iotg Ubuntu upstream *
Linux-intel-iotg-5.15 Ubuntu upstream *
Linux-iot Ubuntu upstream *
Linux-kvm Ubuntu upstream *
Linux-lowlatency Ubuntu noble *
Linux-lowlatency Ubuntu oracular *
Linux-lowlatency Ubuntu upstream *
Linux-lowlatency-hwe-5.15 Ubuntu upstream *
Linux-lowlatency-hwe-5.19 Ubuntu jammy *
Linux-lowlatency-hwe-5.19 Ubuntu upstream *
Linux-lowlatency-hwe-6.11 Ubuntu noble *
Linux-lowlatency-hwe-6.11 Ubuntu upstream *
Linux-lowlatency-hwe-6.2 Ubuntu jammy *
Linux-lowlatency-hwe-6.2 Ubuntu upstream *
Linux-lowlatency-hwe-6.5 Ubuntu jammy *
Linux-lowlatency-hwe-6.5 Ubuntu upstream *
Linux-lowlatency-hwe-6.8 Ubuntu jammy *
Linux-lowlatency-hwe-6.8 Ubuntu upstream *
Linux-lts-xenial Ubuntu upstream *
Linux-nvidia Ubuntu noble *
Linux-nvidia Ubuntu upstream *
Linux-nvidia-6.2 Ubuntu jammy *
Linux-nvidia-6.2 Ubuntu upstream *
Linux-nvidia-6.5 Ubuntu jammy *
Linux-nvidia-6.5 Ubuntu upstream *
Linux-nvidia-6.8 Ubuntu jammy *
Linux-nvidia-6.8 Ubuntu upstream *
Linux-nvidia-lowlatency Ubuntu noble *
Linux-nvidia-lowlatency Ubuntu upstream *
Linux-nvidia-tegra Ubuntu noble *
Linux-nvidia-tegra Ubuntu upstream *
Linux-nvidia-tegra-igx Ubuntu upstream *
Linux-oem Ubuntu bionic *
Linux-oem Ubuntu esm-infra/bionic *
Linux-oem Ubuntu upstream *
Linux-oem Ubuntu xenial *
Linux-oem-5.10 Ubuntu focal *
Linux-oem-5.10 Ubuntu upstream *
Linux-oem-5.13 Ubuntu focal *
Linux-oem-5.13 Ubuntu upstream *
Linux-oem-5.14 Ubuntu focal *
Linux-oem-5.14 Ubuntu upstream *
Linux-oem-5.17 Ubuntu jammy *
Linux-oem-5.17 Ubuntu upstream *
Linux-oem-5.6 Ubuntu focal *
Linux-oem-5.6 Ubuntu upstream *
Linux-oem-6.0 Ubuntu jammy *
Linux-oem-6.0 Ubuntu upstream *
Linux-oem-6.1 Ubuntu jammy *
Linux-oem-6.1 Ubuntu upstream *
Linux-oem-6.11 Ubuntu noble *
Linux-oem-6.11 Ubuntu upstream *
Linux-oem-6.5 Ubuntu jammy *
Linux-oem-6.5 Ubuntu upstream *
Linux-oem-6.8 Ubuntu noble *
Linux-oem-6.8 Ubuntu upstream *
Linux-oracle Ubuntu noble *
Linux-oracle Ubuntu oracular *
Linux-oracle Ubuntu upstream *
Linux-oracle-5.0 Ubuntu bionic *
Linux-oracle-5.0 Ubuntu esm-infra/bionic *
Linux-oracle-5.0 Ubuntu upstream *
Linux-oracle-5.11 Ubuntu focal *
Linux-oracle-5.11 Ubuntu upstream *
Linux-oracle-5.13 Ubuntu focal *
Linux-oracle-5.13 Ubuntu upstream *
Linux-oracle-5.15 Ubuntu upstream *
Linux-oracle-5.3 Ubuntu bionic *
Linux-oracle-5.3 Ubuntu esm-infra/bionic *
Linux-oracle-5.3 Ubuntu upstream *
Linux-oracle-5.4 Ubuntu upstream *
Linux-oracle-5.8 Ubuntu focal *
Linux-oracle-5.8 Ubuntu upstream *
Linux-oracle-6.5 Ubuntu jammy *
Linux-oracle-6.5 Ubuntu upstream *
Linux-oracle-6.8 Ubuntu jammy *
Linux-oracle-6.8 Ubuntu upstream *
Linux-raspi Ubuntu noble *
Linux-raspi Ubuntu oracular *
Linux-raspi Ubuntu upstream *
Linux-raspi-5.4 Ubuntu upstream *
Linux-raspi-realtime Ubuntu realtime/noble *
Linux-raspi-realtime Ubuntu upstream *
Linux-raspi2 Ubuntu bionic *
Linux-raspi2 Ubuntu focal *
Linux-raspi2 Ubuntu upstream *
Linux-raspi2 Ubuntu xenial *
Linux-realtime Ubuntu jammy *
Linux-realtime Ubuntu noble *
Linux-realtime Ubuntu oracular *
Linux-realtime Ubuntu realtime/noble *
Linux-realtime Ubuntu upstream *
Linux-riscv Ubuntu focal *
Linux-riscv Ubuntu jammy *
Linux-riscv Ubuntu noble *
Linux-riscv Ubuntu oracular *
Linux-riscv Ubuntu upstream *
Linux-riscv-5.11 Ubuntu focal *
Linux-riscv-5.11 Ubuntu upstream *
Linux-riscv-5.15 Ubuntu upstream *
Linux-riscv-5.19 Ubuntu jammy *
Linux-riscv-5.19 Ubuntu upstream *
Linux-riscv-5.8 Ubuntu focal *
Linux-riscv-5.8 Ubuntu upstream *
Linux-riscv-6.5 Ubuntu jammy *
Linux-riscv-6.5 Ubuntu upstream *
Linux-riscv-6.8 Ubuntu jammy *
Linux-riscv-6.8 Ubuntu upstream *
Linux-starfive-5.19 Ubuntu jammy *
Linux-starfive-5.19 Ubuntu upstream *
Linux-starfive-6.2 Ubuntu jammy *
Linux-starfive-6.2 Ubuntu upstream *
Linux-starfive-6.5 Ubuntu jammy *
Linux-starfive-6.5 Ubuntu upstream *
Linux-xilinx-zynqmp Ubuntu upstream *

References