vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.23.1rc0, ll temperature validation gates use comparison operators (<, >), which silently evaluate to False for NaN and for positive Infinity in Pythons IEEE 754 float semantics. Both values pass every guard and propagate to GPU sampling kernels, where they produce undefined behavior or CUDA errors that can crash the inference worker. This vulnerability is fixed in 0.23.1rc0.
Weakness
The product receives input that is expected to be of a certain type, but it does not validate or incorrectly validates that the input is actually of the expected type.
Affected Software
| Name | Vendor | Start Version | End Version |
|---|---|---|---|
| Vllm | Vllm | * | 0.23.1 (excluding) |
| Red Hat AI Inference Server 3.2 | RedHat | rhaiis/vllm-cuda-rhel9:1782951012 | * |
| Red Hat AI Inference Server 3.2 | RedHat | rhaiis/vllm-rocm-rhel9:1782951244 | * |
Extended Description
When input does not comply with the expected type, attackers could trigger unexpected errors, cause incorrect actions to take place, or exploit latent vulnerabilities that would not be possible if the input conformed with the expected type. This weakness can appear in type-unsafe programming languages, or in programming languages that support casting or conversion of an input to another type.
Potential Mitigations
- Assume all input is malicious. Use an “accept known good” input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does.
- When performing input validation, consider all potentially relevant properties, including length, type of input, the full range of acceptable values, missing or extra inputs, syntax, consistency across related fields, and conformance to business rules. As an example of business rule logic, “boat” may be syntactically valid because it only contains alphanumeric characters, but it is not valid if the input is only expected to contain colors such as “red” or “blue.”
- Do not rely exclusively on looking for malicious or malformed inputs. This is likely to miss at least one undesirable input, especially if the code’s environment changes. This can give attackers enough room to bypass the intended validation. However, denylists can be useful for detecting potential attacks or determining which inputs are so malformed that they should be rejected outright.