vllm-project vllm version 0.5.2.2 is vulnerable to Denial of Service attacks. The issue occurs in the POST /v1/completions and POST /v1/embeddings endpoints. For POST /v1/completions, enabling use_beam_search and setting best_of to a high value causes the HTTP connection to time out, with vllm ceasing effective work and the request remaining in a pending state, blocking new completion requests. For POST /v1/embeddings, supplying invalid inputs to the JSON object causes an issue in the background loop, resulting in all further completion requests returning a 500 HTTP error code (Internal Server Error) until vllm is restarted.
The product does not properly control the allocation and maintenance of a limited resource, thereby enabling an actor to influence the amount of resources consumed, eventually leading to the exhaustion of available resources.
Limited resources include memory, file system storage, database connection pool entries, and CPU. If an attacker can trigger the allocation of these limited resources, but the number or size of the resources is not controlled, then the attacker could cause a denial of service that consumes all available resources. This would prevent valid users from accessing the product, and it could potentially have an impact on the surrounding environment. For example, a memory exhaustion attack against an application could slow down the application as well as its host operating system. There are at least three distinct scenarios which can commonly lead to resource exhaustion:
Resource exhaustion problems are often result due to an incorrect implementation of the following situations:
Mitigation of resource exhaustion attacks requires that the target system either:
The first of these solutions is an issue in itself though, since it may allow attackers to prevent the use of the system by a particular valid user. If the attacker impersonates the valid user, they may be able to prevent the user from accessing the server in question.
The second solution is simply difficult to effectively institute – and even when properly done, it does not provide a full solution. It simply makes the attack require more resources on the part of the attacker.