Repository: All-Hands-AI/OpenHands

Issue #7167: [Bug]: Tool descriptions too long for OpenAI's models
Created: 2025-03-10T06:08:47Z
Is there an existing issue for the same bug?

 I have checked the existing issues.

Describe the bug and reproduction steps
After the prompt changes in #7018, it seems the tool descriptions are now exceeding the max length limit for OpenAI's models. I got the following error when using o3-mini and gpt-4o:
litellm.exceptions.ContextWindowExceededError: litellm.ContextWindowExceededError: litellm.BadRequestError: ContextWindowExceededError: OpenAIException - Error code: 400 - {'error': {'message': 'litellm.ContextWindowExceededError: litellm.BadRequestError: ContextWindowExceededError: OpenAIException - Error code: 400 - {\'error\': {\'message\': "Invalid \'tools[0].function.description\': string too long. Expected a string with maximum length 1024, but got a string with length 1602 instead.", \'type\': \'invalid_request_error\', \'param\': \'tools[0].function.description\', \'code\': \'string_above_max_length\'}}. Received Model Group=o3-mini\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.ContextWindowExceededError: litellm.BadRequestError: ContextWindowExceededError: OpenAIException - Error code: 400 - {\'error\': {\'message\': "Invalid \'tools[0].function.description\': string too long. Expected a string with maximum length 1024, but got a string with length 1602 instead.", \'type\': \'invalid_request_error\', \'param\': \'tools[0].function.description\', \'code\': \'string_above_max_length\'}}', 'type': None, 'param': None, 'code': '400'}}
OpenHands Installation
Development workflow
OpenHands Version
main
Operating System
Linux
Logs, Errors, Screenshots, and Additional Context
No response

Duplicate issue #7259: [Bug]: The new system prompt is almost 6k, too large for GPT-4
Created: 2025-03-14T17:40:39Z
Is there an existing issue for the same bug?

 I have checked the existing issues.

Describe the bug and reproduction steps
GPT-4 is unusable after 0.28.1, the app fails immediately with a ContextWindowExceeededError.
The underlying error is:
("litellm.BadRequestError: ContextWindowExceededError: OpenAIException - This model's maximum context length is 8192 tokens. 
However, you requested 9836 tokens (5740 in the messages, 4096 in the completion). 
Please reduce the length of the messages or completion.",)

Ugh.


5740 input tokens at the first request is due in part to the system prompt change. The old prompt was 3-4k.


I think the error itself is something that openai and anthropic have done since reasoning models: they enforce now a context limit beforehand, at input_tokens + max_tokens.


The old behavior used to be different: this case was allowed by the API, and the API would simply cut off the response if it became too large for the context window. Now we can't send 4096 max. 🤔
OpenHands Installation
Docker command in README
OpenHands Version
No response
Operating System
None
Logs, Errors, Screenshots, and Additional Context
No response

Closing comment (2025-03-14T17:43:14Z): While the error is different (no tool use), this is related to:

#7167

It might be fixed or alleviated by

#7258

Maybe with a tweak to apply to GPT-4. cc: @ryanhoangt