GHSA-33cg-gxv8-3p8g vLLM denial of service via prompt embeds on M-RoPE models
Summary_Short summary of the problem. Make the impact and severity as clear as possible. For example: An unsafe deserialization vulnerability allows any unauthenticated user to execute arbitrary code on the server._Sending a pure prompt embeds payload in a `/v1/completions` request with a model using M-RoPE causes the EngineCore to fail an assertion and fatally crash, shutting down the entire server application.Any remote user who is authorized to make a `/v1/completions` endpoint can trivially make such a request and induce a crash.Details_Give all details on the vulnerability. Pointing to the incriminated source code is very helpful for the maintainer._In commit [56669c1](https://github.com/vllm-project/vllm/commit/56669c1f293d5c53b6a19ddf2f78802fa9fff2c2), a simple assert intended to be a type-narrowing assert was added to the `_init_mrope_positions` method in `GPUModelRunner` (the offending line on main at the time of writing: https://github.com/vllm-project/vllm/blob/2d481f8a946ee0521872af0f098674a8ee01ce4a/vllm/v1/worker/gpu_model_runner.py#L1588-L1607).This type narrowing assert is to prevent mypy errors later in the function because `None` is not a valid type for `mrope_model.get_mrope_input_positions`. Unfortunately, this assertion is not always true. `/v1/completions` requests that specify `prompt=None` and `prompt_embeds=<not none>` will indeed create a CachedRequestState where `prompt_token_ids` is `None`. This triggers the assertion, which in turn crashes the EngineCore and the Server application.All requests using the `/v1/chat/completions` endpoints will have text/`prompt_token_ids` parts (corresponding to the chat template), and `prompt_embeds` parts are handled as mm_features. This method (rightly) filters out those `prompt_embeds` content parts as they are treated as text positions.A sufficient solution to type narrowing here without raising a fatal assertion is to instead replace the assertion with a using dummy token ids:Technically, in isolation, this method still crashes in the case where `req_state.prompt_token_ids is None and req_state.mm_features`, so the solution above still leaves that potential vector open. As far as can be determined, however, such a `req_state` is impossible in the first place in online mode, because it would require a `/v1/completions` request with `prompt_embeds` AND multimodal features, but the `/v1/completions` request schema does not expose multimodal inputs in any discernible way. Today, those are the only two endpoints with `prompt_embeds` support.When in offline mode, it *is technically* possible to directly create an `EngineCoreRequest` that has `prompt_embeds and not prompt_token_ids and mm_features`, and pass that to `LLM.generate`. That would trigger this same assertion, and no validation would prevent that combination. It is strongly suspected, though, that this combination would be undefined in any model that support M-RoPE, because it would not be possible to determine which token positions correspond to `mm_features`. The proposed solution above would end up not setting `req_state.mrope_positions` and `req_state.mrope_position_delta` in this scenario, which could result in undefined behavior.`prompt_embeds` is far more familiar here than M-RoPE, and it is understood that each model that supports it is responsible for defining its own `get_mrope_input_positions` which have varying implementations. There is insufficient knowledge to be prescriptive in how the two features should interact in the offline case, other than possibly raising a validation error earlier on preventing that combination (which would emulate the current assertion behavior). Regardless, in offline mode, the chances of a remote user being able to exploit this are slim-to-nil compared to the online case which is incredibly straightforward.Impact_What kind of vulnerability is it? Who is impacted?_Denial of Service caused by an incorrect assertion inside of the `GPUModelRunner` which causes a fatal EngineCore exceptionAny configuration with `--enable-prompt-embeds` and M-RoPE-supported model is vulnerableThe attack is extremely easy from the remote attacker's perspective (copying the official `prompt_embeds` online mode docs examples almost-verbatim, accounting for model-name and connection details, of course, will induce a guaranteed shutdown)
CVSS v4.0 score: 7.1