vLLM is an inference and serving engine for large language models. From 0.19.0 until 0.32.0, authenticated callers can supply a request chat template when no server-side –chat-template is configured, or a malicious model repository can provide a template that is compiled and rendered by tokenizer.apply_chat_template() without a wall-clock timeout, output-size limit, or iteration bound. Rendering runs in a ThreadPoolExecutor with one worker by default, so a nonterminating render can block /v1/responses and other chat-template users after the request ends, while an output-expanding render can exhaust server memory. Harmony models are not affected because they do not use this Jinja path. This issue is fixed in version 0.32.0.
Weakness
The product does not properly control the allocation and maintenance of a limited resource.
Affected Software
| Name | Vendor | Start Version | End Version |
|---|---|---|---|
| Vllm | Vllm | 0.19.0 (including) | 0.32.0 (excluding) |
Potential Mitigations
Mitigation of resource exhaustion attacks requires that the target system either:
The first of these solutions is an issue in itself though, since it may allow attackers to prevent the use of the system by a particular valid user. If the attacker impersonates the valid user, they may be able to prevent the user from accessing the server in question.
The second solution is simply difficult to effectively institute – and even when properly done, it does not provide a full solution. It simply makes the attack require more resources on the part of the attacker.
Related Attack Patterns
- https://cwe.mitre.org/data/definitions/147.html
- https://cwe.mitre.org/data/definitions/227.html
- https://cwe.mitre.org/data/definitions/492.html