vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs. Prior to 0.24.0, a frontend-legal multi-request speculative decoding workload can cause the rejection sampler to produce a recovered token equal to the model vocabulary size boundary value, which is then converted to negative one when the engine selects the next live token for a request and is written back into the drafters input ids; that out-of-vocabulary value is later consumed by the models embedding and attention path and crashes the engine worker with a GPU device-side assertion. The same triggering request sequence is reachable through the public gRPC Generate and Abort endpoints, so a remote client that can send generation requests can crash the shared engine worker, aborting concurrent requests and causing a service-wide denial of service for other clients of the deployment until the worker is restarted. This issue is fixed in version 0.24.0.
The product receives input or data, but it does not validate or incorrectly validates that the input has the properties that are required to process the data safely and correctly.
| Name | Vendor | Start Version | End Version |
|---|---|---|---|
| Vllm | Vllm | * | 0.24.0 (excluding) |
| Red Hat AI Inference Server 3.2 | RedHat | rhaiis/vllm-cuda-rhel9:1787860580 | * |
| Red Hat AI Inference Server 3.2 | RedHat | rhaiis/vllm-rocm-rhel9:1787884873 | * |
| Red Hat AI Inference Server 3.3 | RedHat | rhaiis/vllm-spyre-rhel9:1787161776 | * |
| Red Hat AI Inference Server 3.4 | RedHat | rhaii/vllm-spyre-rhel9:1789681201 | * |
| Red Hat AI Inference Server 3.4 | RedHat | rhaii/vllm-cpu-rhel9:1789681128 | * |
| Red Hat AI Inference Server 3.4 | RedHat | rhaii/vllm-cuda-rhel9:1789681126 | * |
| Red Hat AI Inference Server 3.4 | RedHat | rhaii/vllm-rocm-rhel9:1789681126 | * |
| Red Hat AI Inference Server 3.4 | RedHat | rhaii/vllm-cpu-rhel9:1790075793 | * |
| Red Hat AI Inference Server 3.4 | RedHat | rhaii/vllm-spyre-rhel9:1790076141 | * |
| Red Hat AI Inference Server 3.4 | RedHat | rhaii/vllm-cuda-rhel9:1790090131 | * |
| Red Hat AI Inference Server 3.4 | RedHat | rhaii/vllm-rocm-rhel9:1790109620 | * |
| Red Hat Enterprise Linux AI 3.4 | RedHat | rhelai3/disk-image-cuda-rhel9:1790957417 | * |
| Red Hat Enterprise Linux AI 3.4 | RedHat | rhelai3/bootc-aws-cuda-rhel9:1790881122 | * |
| Red Hat Enterprise Linux AI 3.4 | RedHat | rhelai3/bootc-azure-cuda-rhel9:1790881120 | * |
| Red Hat Enterprise Linux AI 3.4 | RedHat | rhelai3/bootc-azure-rocm-rhel9:1790707552 | * |
| Red Hat Enterprise Linux AI 3.4 | RedHat | rhelai3/bootc-cuda-rhel9:1790868254 | * |
| Red Hat Enterprise Linux AI 3.4 | RedHat | rhelai3/bootc-gcp-cuda-rhel9:1790881123 | * |
| Red Hat Enterprise Linux AI 3.4 | RedHat | rhelai3/bootc-rocm-rhel9:1790614447 | * |
Input validation is a frequently-used technique for checking potentially dangerous inputs in order to ensure that the inputs are safe for processing within the code, or when communicating with other components. Input can consist of:
Data can be simple or structured. Structured data can be composed of many nested layers, composed of combinations of metadata and raw data, with other simple or structured data. Many properties of raw data or metadata may need to be validated upon entry into the code, such as:
Implied or derived properties of data must often be calculated or inferred by the code itself. Errors in deriving properties may be considered a contributing factor to improper input validation.