vLLM is an inference and serving engine for large language models. Prior to 0.28.0, request bodies for Chat Completions and Responses can set media_io_kwargs.video.video_backend to pynvvideocodec, and MediaConnector.fetch_video forwards that choice to VideoMediaIO even when startup configuration selected a software decoder. The engines _reserve_mm_ipc_gpu_memory logic budgets decoder memory only from static configuration, so the request-selected VIDEO_LOADER_REGISTRY backend can create a CUDA context, decoder surfaces, and decoded-frame allocations that were not removed from the engines KV-cache budget. An attacker able to submit video requests to a video-capable GPU deployment with PyNvVideoCodec installed can exhaust shared GPU memory, causing request failures, worker crashes, or denial of service. The first release containing the fix is version 0.28.0.
Weakness
The product does not properly control the allocation and maintenance of a limited resource.
Potential Mitigations
Mitigation of resource exhaustion attacks requires that the target system either:
The first of these solutions is an issue in itself though, since it may allow attackers to prevent the use of the system by a particular valid user. If the attacker impersonates the valid user, they may be able to prevent the user from accessing the server in question.
The second solution is simply difficult to effectively institute – and even when properly done, it does not provide a full solution. It simply makes the attack require more resources on the part of the attacker.
Related Attack Patterns
- https://cwe.mitre.org/data/definitions/147.html
- https://cwe.mitre.org/data/definitions/227.html
- https://cwe.mitre.org/data/definitions/492.html
References
- https://github.com/vllm-project/vllm/commit/283893c72292ede38d277e3cd2b9b64c3e4f1dda
- https://github.com/vllm-project/vllm/commit/ba22152096b2484faa3579624a253d54804d876d
- https://github.com/vllm-project/vllm/pull/47259
- https://github.com/vllm-project/vllm/security/advisories/GHSA-8pw2-6jv3-mj5j
- https://github.com/vllm-project/vllm/security/advisories/GHSA-8pw2-6jv3-mj5j