Repository: Significant-Gravitas/AutoGPT (Python)
Date: 2023-06-11 09:10:14+00:00
Task type: Bug Fix

Issue:
 COMMAND = list_files - openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens
### ⚠️ Search for existing issues first ⚠️

- [X] I have searched the existing issues, and there is no existing issue for my problem

### Which Operating System are you using?

Docker

### Which version of Auto-GPT are you using?

Master (branch)

### GPT-3 or GPT-4?

GPT-3.5

### Steps to reproduce 🕹

listing the auto_gpt_workspace folder errors out. maybe this is an erroneous bug, not really sure, but why is it calling openai when it's merely listing the files in the folder?

openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 4819 tokens. Please reduce the length of the messages.

### Current behavior 😯

listing the folder contents errors out and kills the program if there's too many files in there.

### Expected behavior 🤔

not ... error out :D

### Your prompt 📝

```yaml
# Paste your prompt here
```


### Your Logs 📒

```log
NEXT ACTION:  COMMAND = list_files ARGUMENTS = {'directory': '/home/atlas/autogpt/auto_gpt_workspace/atlas_repo'}
SYSTEM:  Command list_files returned: ['atlas_repo/docker-compose.yml', 'atlas_repo/mkdocs.yml', 'atlas_repo/run.bat', 'atlas_repo/run_continuous.bat', 'atlas_repo/requirements.txt', 'atlas_repo/tests.py', 'atlas_repo/CODE_OF_CONDUCT.md', 'atlas_repo/main.py', 'atlas_repo/plugin.png', 'atlas_repo/codecov.yml', 'atlas_repo/CONTRIBUTING.md', 'atlas_repo/BULLETIN.md', 'atlas_repo/run_continuous.sh', 'atlas_repo/LICENSE', 'atlas_repo/pyproject.toml', 'atlas_repo/azure.yaml.template', 'atlas_repo/README.md', 'atlas_repo/data_ingestion.py', 'atlas_repo/run.sh', 'atlas_repo/Dockerfile', 'atlas_repo/scripts/install_plugin_deps.py', 'atlas_repo/scripts/check_requirements.py', 'atlas_repo/scripts/__init__.py', 'atlas_repo/.git/packed-refs', 'atlas_repo/.git/config', 'atlas_repo/.git/index', 'atlas_repo/.git/description', 'atlas_repo/.git/HEAD', 'atlas_repo/.git/hooks/pre-applypatch.sample', 'atlas_repo/.git/hooks/pre-rebase.sample', 'atlas_repo/.git/hooks/pre-merge-commit.sample', 'atlas_repo/.git/hooks/post-update.sample', 'atlas_repo/.git/hooks/pre-push.sample', 'atlas_repo/.git/hooks/pre-receive.sample', 'atlas_repo/.git/hooks/push-to-checkout.sample', 'atlas_repo/.git/hooks/fsmonitor-watchman.sample', 'atlas_repo/.git/hooks/prepare-commit-msg.sample', 'atlas_repo/.git/hooks/commit-msg.sample', 'atlas_repo/.git/hooks/applypatch-msg.sample', 'atlas_repo/.git/hooks/pre-commit.sample', 'atlas_repo/.git/hooks/update.sample', 'atlas_repo/.git/logs/HEAD', 'atlas_repo/.git/logs/refs/heads/master', 'atlas_repo/.git/logs/refs/remotes/origin/HEAD', 'atlas_repo/.git/info/exclude', 'atlas_repo/.git/refs/heads/master', 'atlas_repo/.git/refs/remotes/origin/HEAD', 'atlas_repo/.git/objects/pack/pack-f6a3dd32fbb51bd88bf8f6872667b2c80c8833ee.pack', 'atlas_repo/.git/objects/pack/pack-f6a3dd32fbb51bd88bf8f6872667b2c80c8833ee.idx', 'atlas_repo/plugins/__PUT_PLUGIN_ZIPS_HERE__', 'atlas_repo/benchmark/benchmark_entrepreneur_gpt_with_difficult_user.py', 'atlas_repo/benchmark/__init__.py', 'atlas_repo/tests/test_agent.py', 'atlas_repo/tests/test_image_gen.py', 'atlas_repo/tests/context.py', 'atlas_repo/tests/test_ai_config.py', 'atlas_repo/tests/test_logs.py', 'atlas_repo/tests/test_config.py', 'atlas_repo/tests/test_commands.py', 'atlas_repo/tests/test_agent_manager.py', 'atlas_repo/tests/test_utils.py', 'atlas_repo/tests/milvus_memory_test.py', 'atlas_repo/tests/test_token_counter.py', 'atlas_repo/tests/utils.py', 'atlas_repo/tests/conftest.py', 'atlas_repo/tests/test_prompt_generator.py', 'atlas_repo/tests/test_workspace.py', 'atlas_repo/tests/test_api_manager.py', 'atlas_repo/tests/__init__.py', 'atlas_repo/tests/integration/agent_factory.py', 'atlas_repo/tests/integration/test_memory_management.py', 'atlas_repo/tests/integration/milvus_memory_tests.py', 'atlas_repo/tests/integration/test_git_commands.py', 'atlas_repo/tests/integration/memory_tests.py', 'atlas_repo/tests/integration/test_execute_code.py', 'atlas_repo/tests/integration/test_setup.py', 'atlas_repo/tests/integration/agent_utils.py', 'atlas_repo/tests/integration/weaviate_memory_tests.py', 'atlas_repo/tests/integration/test_local_cache.py', 'atlas_repo/tests/integration/conftest.py', 'atlas_repo/tests/integration/test_llm_utils.py', 'atlas_repo/tests/integration/__init__.py', 'atlas_repo/tests/integration/cassettes/test_memory_management/test_save_memory_trimmed_from_context_window.yaml', 'atlas_repo/tests/integration/cassettes/test_setup/test_generate_aiconfig_automatic_default.yaml', 'atlas_repo/tests/integration/cassettes/test_setup/test_generate_aiconfig_automatic_typical.yaml', 'atlas_repo/tests/integration/cassettes/test_setup/test_generate_aiconfig_automatic_fallback.yaml', 'atlas_repo/tests/integration/cassettes/test_llm_utils/test_get_ada_embedding_large_context.yaml', 'atlas_repo/tests/integration/cassettes/test_llm_utils/test_get_ada_embedding.yaml', 'atlas_repo/tests/integration/cassettes/test_local_cache/test_get_relevant.yaml', 'atlas_repo/tests/integration/challenges/utils.py', 'atlas_repo/tests/integration/challenges/conftest.py', 'atlas_repo/tests/integration/challenges/__init__.py', 'atlas_repo/tests/integration/challenges/memory/test_memory_challenge_b.py', 'atlas_repo/tests/integration/challenges/memory/test_memory_challenge_a.py', 'atlas_repo/tests/integration/challenges/memory/__init__.py', 'atlas_repo/tests/integration/challenges/memory/cassettes/test_memory_challenge_a/test_memory_challenge_a.yaml', 'atlas_repo/tests/integration/challenges/memory/cassettes/test_memory_challenge_b/test_memory_challenge_b.yaml', 'atlas_repo/tests/integration/goal_oriented/goal_oriented_tasks.md', 'atlas_repo/tests/integration/goal_oriented/test_write_file.py', 'atlas_repo/tests/integration/goal_oriented/test_browse_website.py', 'atlas_repo/tests/integration/goal_oriented/__init__.py', 'atlas_repo/tests/integration/goal_oriented/cassettes/test_browse_website/test_browse_website.yaml', 'atlas_repo/tests/integration/goal_oriented/cassettes/test_write_file/test_write_file.yaml', 'atlas_repo/tests/unit/test_get_self_feedback.py', 'atlas_repo/tests/unit/test_plugins.py', 'atlas_repo/tests/unit/test_browse_scrape_links.py', 'atlas_repo/tests/unit/test_chat.py', 'atlas_repo/tests/unit/test_browse_scrape_text.py', 'atlas_repo/tests/unit/test_web_selenium.py', 'atlas_repo/tests/unit/test_commands.py', 'atlas_repo/tests/unit/test_file_operations.py', 'atlas_repo/tests/unit/test_spinner.py', 'atlas_repo/tests/unit/test_json_parser.py', 'atlas_repo/tests/unit/test_json_utils_llm.py', 'atlas_repo/tests/unit/test_url_validation.py', 'atlas_repo/tests/unit/_test_json_parser.py', 'atlas_repo/tests/unit/test_llm_utils.py', 'atlas_repo/tests/unit/__init__.py', 'atlas_repo/tests/unit/data/test_plugins/Auto-GPT-Plugin-Test-master.zip', 'atlas_repo/tests/unit/models/test_base_open_api_plugin.py', 'atlas_repo/tests/mocks/mock_commands.py', 'atlas_repo/tests/mocks/__init__.py', 'atlas_repo/tests/vcr/openai_filter.py', 'atlas_repo/tests/vcr/__init__.py', 'atlas_repo/.github/FUNDING.yml', 'atlas_repo/.github/PULL_REQUEST_TEMPLATE.md', 'atlas_repo/.github/workflows/docker-release.yml', 'atlas_repo/.github/workflows/docker-cache-clean.yml', 'atlas_repo/.github/workflows/ci.yml', 'atlas_repo/.github/workflows/sponsors_readme.yml', 'atlas_repo/.github/workflows/docker-ci.yml', 'atlas_repo/.github/workflows/benchmarks.yml', 'atlas_repo/.github/workflows/documentation-release.yml', 'atlas_repo/.github/workflows/pr-label.yml', 'atlas_repo/.github/workflows/scripts/docker-ci-summary.sh', 'atlas_repo/.github/workflows/scripts/docker-release-summary.sh', 'atlas_repo/.github/ISSUE_TEMPLATE/1.bug.yml', 'atlas_repo/.github/ISSUE_TEMPLATE/2.feature.yml', 'atlas_repo/autogpt/app.py', 'atlas_repo/autogpt/configurator.py', 'atlas_repo/autogpt/main.py', 'atlas_repo/autogpt/singleton.py', 'atlas_repo/autogpt/logs.py', 'atlas_repo/autogpt/utils.py', 'atlas_repo/autogpt/cli.py', 'atlas_repo/autogpt/plugins.py', 'atlas_repo/autogpt/setup.py', 'atlas_repo/autogpt/__main__.py', 'atlas_repo/autogpt/__init__.py', 'atlas_repo/autogpt/spinner.py', 'atlas_repo/autogpt/memory_management/store_memory.py', 'atlas_repo/autogpt/memory_management/summary_memory.py', 'atlas_repo/autogpt/json_utils/llm_response_format_1.json', 'atlas_repo/autogpt/json_utils/json_fix_llm.py', 'atlas_repo/autogpt/json_utils/json_fix_general.py', 'atlas_repo/autogpt/json_utils/__init__.py', 'atlas_repo/autogpt/json_utils/utilities.py', 'atlas_repo/autogpt/processing/text.py', 'atlas_repo/autogpt/processing/html.py', 'atlas_repo/autogpt/processing/__init__.py', 'atlas_repo/autogpt/memory/local.py', 'atlas_repo/autogpt/memory/pinecone.py', 'atlas_repo/autogpt/memory/no_memory.py', 'atlas_repo/autogpt/memory/weaviate.py', 'atlas_repo/autogpt/memory/milvus.py', 'atlas_repo/autogpt/memory/base.py', 'atlas_repo/autogpt/memory/redismem.py', 'atlas_repo/autogpt/memory/__init__.py', 'atlas_repo/autogpt/commands/write_tests.py', 'atlas_repo/autogpt/commands/web_playwright.py', 'atlas_repo/autogpt/commands/improve_code.py', 'atlas_repo/autogpt/commands/google_search.py', 'atlas_repo/autogpt/commands/audio_text.py', 'atlas_repo/autogpt/commands/web_selenium.py', 'atlas_repo/autogpt/commands/image_gen.py', 'atlas_repo/autogpt/commands/web_requests.py', 'atlas_repo/autogpt/commands/command.py', 'atlas_repo/autogpt/commands/times.py', 'atlas_repo/autogpt/commands/file_operations.py', 'atlas_repo/autogpt/commands/git_operations.py', 'atlas_repo/autogpt/commands/twitter.py', 'atlas_repo/autogpt/commands/analyze_code.py', 'atlas_repo/autogpt/commands/execute_code.py', 'atlas_repo/autogpt/commands/__init__.py', 'atlas_repo/autogpt/config/ai_config.py', 'atlas_repo/autogpt/config/config.py', 'atlas_repo/autogpt/config/__init__.py', 'atlas_repo/autogpt/prompts/prompt.py', 'atlas_repo/autogpt/prompts/generator.py', 'atlas_repo/autogpt/prompts/__init__.py', 'atlas_repo/autogpt/url_utils/__init__.py', 'atlas_repo/autogpt/url_utils/validators.py', 'atlas_repo/autogpt/workspace/workspace.py', 'atlas_repo/autogpt/workspace/__init__.py', 'atlas_repo/autogpt/llm/modelsinfo.py', 'atlas_repo/autogpt/llm/api_manager.py', 'atlas_repo/autogpt/llm/chat.py', 'atlas_repo/autogpt/llm/llm_utils.py', 'atlas_repo/autogpt/llm/token_counter.py', 'atlas_repo/autogpt/llm/base.py', 'atlas_repo/autogpt/llm/__init__.py', 'atlas_repo/autogpt/llm/providers/openai.py', 'atlas_repo/autogpt/llm/providers/__init__.py', 'atlas_repo/autogpt/agent/agent_manager.py', 'atlas_repo/autogpt/agent/agent.py', 'atlas_repo/autogpt/agent/__init__.py', 'atlas_repo/autogpt/models/base_open_ai_plugin.py', 'atlas_repo/autogpt/speech/brian.py', 'atlas_repo/autogpt/speech/eleven_labs.py', 'atlas_repo/autogpt/speech/gtts.py', 'atlas_repo/autogpt/speech/say.py', 'atlas_repo/autogpt/speech/base.py', 'atlas_repo/autogpt/speech/macos_tts.py', 'atlas_repo/autogpt/speech/__init__.py', 'atlas_repo/autogpt/js/overlay.js', 'atlas_repo/docs/usage.md', 'atlas_repo/docs/plugins.md', 'atlas_repo/docs/testing.md', 'atlas_repo/docs/index.md', 'atlas_repo/docs/code-of-conduct.md', 'atlas_repo/docs/setup.md', 'atlas_repo/docs/contributing.md', 'atlas_repo/docs/imgs/openai-api-key-billing-paid-account.png', 'atlas_repo/docs/configuration/search.md', 'atlas_repo/docs/configuration/voice.md', 'atlas_repo/docs/configuration/imagegen.md', 'atlas_repo/docs/configuration/memory.md', 'atlas_repo/.devcontainer/docker-compose.yml', 'atlas_repo/.devcontainer/Dockerfile', 'atlas_repo/.devcontainer/devcontainer.json']
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/atlas/autogpt/__main__.py", line 5, in <module>
    autogpt.cli.main()
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1635, in invoke
    rv = super().invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/atlas/autogpt/cli.py", line 90, in main
    run_auto_gpt(
  File "/home/atlas/autogpt/main.py", line 157, in run_auto_gpt
    agent.start_interaction_loop()
  File "/home/atlas/autogpt/agent/agent.py", line 94, in start_interaction_loop
    assistant_reply = chat_with_ai(
  File "/home/atlas/autogpt/llm/chat.py", line 166, in chat_with_ai
    agent.summary_memory = update_running_summary(
  File "/home/atlas/autogpt/memory_management/summary_memory.py", line 114, in update_running_summary
    current_memory = create_chat_completion(messages, cfg.fast_llm_model)
  File "/home/atlas/autogpt/llm/llm_utils.py", line 166, in create_chat_completion
    response = api_manager.create_chat_completion(
  File "/home/atlas/autogpt/llm/api_manager.py", line 55, in create_chat_completion
    response = openai.ChatCompletion.create(
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 226, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 619, in _interpret_response
    self._interpret_response_line(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 682, in _interpret_response_line
    raise self.handle_error_response(
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 4819 tokens. Please reduce the length of the messages.

```



Issue discussion:
I also encountered the same problem and couldn't continue the project
It's coming from updating memory summary. That appears to be a global behaviour. Your are constrained by 4096 tokens context window given the model you are using - likely gpt 3.5 - if you used gpt-4, you would not error out here. I can think of adding chunking for a certain class of commands?
> It's coming from updating memory summary. That appears to be a global behaviour. Your are constrained by 4096 tokens context window given the model you are using - likely gpt 3.5 - if you used gpt-4, you would not error out here. I can think of adding chunking for a certain class of commands?

i've already set the token limit to 4000 since i am on GPT3.5, but it's not working, so idk.

```
### LLM MODEL SETTINGS
## FAST_TOKEN_LIMIT - Fast token limit for OpenAI (Default: 4000)
## SMART_TOKEN_LIMIT - Smart token limit for OpenAI (Default: 8000)
## When using --gpt3only this needs to be set to 4000.
# FAST_TOKEN_LIMIT=4000
SMART_TOKEN_LIMIT=4000

```
```
ghly targeted prospect lists. Bulks. Search or verify contact lists in minutes with bulk tasks. Enrichment." } ]
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/app/autogpt/__main__.py", line 5, in <module>
    autogpt.cli.main()
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1635, in invoke
    rv = super().invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/app/autogpt/cli.py", line 90, in main
    run_auto_gpt(
  File "/app/autogpt/main.py", line 171, in run_auto_gpt
    agent.start_interaction_loop()
  File "/app/autogpt/agent/agent.py", line 112, in start_interaction_loop
    assistant_reply = chat_with_ai(
  File "/app/autogpt/llm/chat.py", line 165, in chat_with_ai
    agent.summary_memory = update_running_summary(
  File "/app/autogpt/memory_management/summary_memory.py", line 123, in update_running_summary
    current_memory = create_chat_completion(messages, cfg.fast_llm_model)
  File "/app/autogpt/llm/llm_utils.py", line 166, in create_chat_completion
    response = api_manager.create_chat_completion(
  File "/app/autogpt/llm/api_manager.py", line 55, in create_chat_completion
    response = openai.ChatCompletion.create(
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 226, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 619, in _interpret_response
    self._interpret_response_line(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 682, in _interpret_response_line
    raise self.handle_error_response(
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 7009 tokens. Please reduce the length of the messages.
my@my-Mac-mini auto-gpt % 
```

****
> > It's coming from updating memory summary. That appears to be a global behaviour. Your are constrained by 4096 tokens context window given the model you are using - likely gpt 3.5 - if you used gpt-4, you would not error out here. I can think of adding chunking for a certain class of commands?
> 
> i've already set the token limit to 4000 since i am on GPT3.5, but it's not working, so idk.
> 
> ```
> ### LLM MODEL SETTINGS
> ## FAST_TOKEN_LIMIT - Fast token limit for OpenAI (Default: 4000)
> ## SMART_TOKEN_LIMIT - Smart token limit for OpenAI (Default: 8000)
> ## When using --gpt3only this needs to be set to 4000.
> # FAST_TOKEN_LIMIT=4000
> SMART_TOKEN_LIMIT=4000
> ```

```
### LLM MODEL SETTINGS
## FAST_TOKEN_LIMIT - Fast token limit for OpenAI (Default: 4000)
## SMART_TOKEN_LIMIT - Smart token limit for OpenAI (Default: 8000)
## When using --gpt3only this needs to be set to 4000.
 FAST_TOKEN_LIMIT=3000
 SMART_TOKEN_LIMIT=3000
 
 ### EMBEDDINGS
## EMBEDDING_MODEL       - Model to use for creating embeddings
## EMBEDDING_TOKENIZER   - Tokenizer to use for chunking large inputs
## EMBEDDING_TOKEN_LIMIT - Chunk size limit for large inputs
 EMBEDDING_MODEL=text-embedding-ada-002
 EMBEDDING_TOKENIZER=cl100k_base
 EMBEDDING_TOKEN_LIMIT=8191
```
 
 same, not sure if I was running GPT3 only tough
I am experiencing the same behavior since i updated to version 3.0
I got this error also in the latest stable branch v0.3.0 
Same here on the latest version can't move forward building 
Same here
Same question
I am new to this i think i have the exact same issue as its the last request i will post all of it here just in case im missing something. Thanks everyone 

  File "c:\Autogpt\Auto-GPT\autogpt\__main__.py", line 5, in <module>
    autogpt.cli.main()
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\click\core.py", line 1635, in invoke
    rv = super().invoke(ctx)
         ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\click\decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Autogpt\Auto-GPT\autogpt\cli.py", line 90, in main
    run_auto_gpt(
  File "c:\Autogpt\Auto-GPT\autogpt\main.py", line 186, in run_auto_gpt
    agent.start_interaction_loop()
  File "c:\Autogpt\Auto-GPT\autogpt\agent\agent.py", line 112, in start_interaction_loop
    assistant_reply = chat_with_ai(
                      ^^^^^^^^^^^^^
  File "c:\Autogpt\Auto-GPT\autogpt\llm\chat.py", line 244, in chat_with_ai
    assistant_reply = create_chat_completion(
                      ^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Autogpt\Auto-GPT\autogpt\llm\llm_utils.py", line 166, in create_chat_completion
    response = api_manager.create_chat_completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Autogpt\Auto-GPT\autogpt\llm\api_manager.py", line 55, in create_chat_completion
    response = openai.ChatCompletion.create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\openai\api_resources\chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
                           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\openai\api_requestor.py", line 226, in request
    resp, got_stream = self._interpret_response(result, stream)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\openai\api_requestor.py", line 619, in _interpret_response
    self._interpret_response_line(
  File "C:\Users\ROOT\AppData\Roaming\Python\Python311\site-packages\openai\api_requestor.py", line 682, in _interpret_response_line
    raise self.handle_error_response(
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, you requested 4289 tokens (2521 in the messages, 1768 in the completion). Please reduce the length of the messages or completion.
Same problem with any branch ( Master or Stable 0.3.0/0.2.2 )
I cant move project with this... Same problem
Thanks"
I am currently working on a possible fix for this, as in theory I think this is caused by total tokens in request for gpt3 model. There is a 'send_token_limit' variable that is currently subtracting 1000 to retain for the response request. I am testing out 1500 for this to see if it still errors. I am shooting in the dark here, but I will let you all know if this resolves the issue or not.
Hi Guys, have the same issue. the number of tokens can be significantly higher. workin for hours on a solution....unfortunately without success so far. 

openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 25424 tokens. Please reduce the length of the messages.


Same problem here since I upgrade to 0.3.0... why is the agent sending messages longer than 4000?
It's a hard limit imposed by openai
Same issue here
Same issue when i update to 0.3.0
+1
i have same problem, 
when i use langchain DB_chain to query mysql database.

 This model's maximum context length is 4097 tokens, however you requested 4582 tokens (4326 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.
+1
+1
The same problem, but I have a slightly different error message, as: `openai.error.InvalidRequestError: This model's maximum context length is 8191 tokens, however you requested 10549 tokens (10549 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.`
I tried to catch the exception. Works on some occasions but it seems to cause other issues or the program terminates since the response is none. In general, this issue is one of the biggest issues in autogpt currently. You basically can't use it since it breaks down every 2 seconds depending on the task you have given it.
+1
HI All, so I searched a bit and basically what has to be done is this, but of course adapted to autoGPT: (see also here https://blog.devgenius.io/how-to-get-around-openai-gpt-3-token-limits-b11583691b32) 

def break_up_file(tokens, chunk_size, overlap_size):
    if len(tokens) <= chunk_size:
        yield tokens
    else:
        chunk = tokens[:chunk_size]
        yield chunk
        yield from break_up_file(tokens[chunk_size-overlap_size:], chunk_size, overlap_size)

def break_up_file_to_chunks(filename, chunk_size=2000, overlap_size=100):
    with open(filename, 'r') as f:
        text = f.read()
    tokens = word_tokenize(text)
    return list(break_up_file(tokens, chunk_size, overlap_size))


def convert_to_detokenized_text(tokenized_text):
    prompt_text = " ".join(tokenized_text)
    prompt_text = prompt_text.replace(" 's", "'s")
    return detokenized_text


filename = "/content/drive/MyDrive/Colab Notebooks/minutes/data/Round_22_Online_Kickoff_Meeting.txt"

prompt_response = []
chunks = break_up_file_to_chunks(filename)

for i, chunk in enumerate(chunks):
    prompt_request = "Summarize this meeting transcript: " + convert_to_detokenized_text(chunks[i])
    response = openai.Completion.create(
            model="text-davinci-003",
            prompt=prompt_request,
            temperature=.5,
            max_tokens=500,
            top_p=1,
            frequency_penalty=0,
            presence_penalty=0
    )
    
    prompt_response.append(response["choices"][0]["text"].strip())
found some interesting solutions for the issue...if you look outside autogpt the issue is also well known. 

1) https://medium.com/@shweta-lodha/how-to-deal-with-openai-token-limit-issue-part-1-d0157c9e4d4e
2) https://www.youtube.com/watch?v=_vetq4G0Gsc
3) https://www.youtube.com/watch?v=Oj1GUJnJrWs
4) https://www.youtube.com/watch?v=xkCzP4-YoNA
+1
+1
+1
+1
+1
+1
+1
Should this part of the [text.py](autogpt/processing/text.py) prevent this?
`        if expected_token_usage <= max_length:
            current_chunk.append(sentence)
        else:
            yield " ".join(current_chunk)
            current_chunk = [sentence]
            message_this_sentence_only = [
                create_message(" ".join(current_chunk), question)
            ]
            expected_token_usage = (
                count_message_tokens(messages=message_this_sentence_only, model=model)
                + 1
            )
            if expected_token_usage > max_length:
                raise ValueError(
                    f"Sentence is too long in webpage: {expected_token_usage} tokens."
                )`
I have been consistently getting this and the JSON error. 

I thought changing (i.e. de-commenting) the below in the .env file appears to have resolved the token length issue. 
**UPDATED: it did not resolve the error.**

running on Docker, gpt3only 

EMBEDDING_MODEL=text-embedding-ada-002
EMBEDDING_TOKENIZER=cl100k_base
EMBEDDING_TOKEN_LIMIT=8191

> I have been consistently getting this and the JSON error.
> 
> I thought changing (i.e. de-commenting) the below in the .env file appears to have resolved the token length issue. **UPDATED: it did not resolve the error.**
> 
> running on Docker, gpt3only
> 
> EMBEDDING_MODEL=text-embedding-ada-002 EMBEDDING_TOKENIZER=cl100k_base EMBEDDING_TOKEN_LIMIT=8191

That doesn't work. You will run into issues eventually 
Playing around with some experimental code that was commented out in chat.py will also try setting subtraction amount to 2000 but that's not ideal. my chat.py code below

import time
from random import shuffle

from openai.error import RateLimitError

from autogpt.config import Config
from autogpt.llm.api_manager import ApiManager
from autogpt.llm.base import Message
from autogpt.llm.llm_utils import create_chat_completion
from autogpt.llm.token_counter import count_message_tokens
from autogpt.logs import logger
from autogpt.memory_management.store_memory import (
    save_memory_trimmed_from_context_window,
)
from autogpt.memory_management.summary_memory import (
    get_newly_trimmed_messages,
    update_running_summary,
)

cfg = Config()


def create_chat_message(role, content) -> Message:
    """
    Create a chat message with the given role and content.

    Args:
    role (str): The role of the message sender, e.g., "system", "user", or "assistant".
    content (str): The content of the message.

    Returns:
    dict: A dictionary containing the role and content of the message.
    """
    return {"role": role, "content": content}


def generate_context(prompt, relevant_memory, full_message_history, model):
    current_context = [
        create_chat_message("system", prompt),
        create_chat_message(
            "system", f"The current time and date is {time.strftime('%c')}"
        ),
        create_chat_message(
             "system",
             f"This reminds you of these events from your past:\n{relevant_memory}\n\n",
         ),
    ]

    # Add messages from the full message history until we reach the token limit
    next_message_to_add_index = len(full_message_history) - 1
    insertion_index = len(current_context)
    # Count the currently used tokens
    current_tokens_used = count_message_tokens(current_context, model)
    return (
        next_message_to_add_index,
        current_tokens_used,
        insertion_index,
        current_context,
    )


# TODO: Change debug from hardcode to argument
def chat_with_ai(
    agent, prompt, user_input, full_message_history, permanent_memory, token_limit
):
    """Interact with the OpenAI API, sending the prompt, user input, message history,
    and permanent memory."""
    while True:
        try:
            """
            Interact with the OpenAI API, sending the prompt, user input,
                message history, and permanent memory.

            Args:
                prompt (str): The prompt explaining the rules to the AI.
                user_input (str): The input from the user.
                full_message_history (list): The list of all messages sent between the
                    user and the AI.
                permanent_memory (Obj): The memory object containing the permanent
                  memory.
                token_limit (int): The maximum number of tokens allowed in the API call.

            Returns:
            str: The AI's response.
            """
            model = cfg.fast_llm_model  # TODO: Change model from hardcode to argument
            # Reserve 1000 tokens for the response
            logger.debug(f"Token limit: {token_limit}")
            send_token_limit = token_limit - 1000

            if len(full_message_history) == 0:
                 relevant_memory = ""
            else:
                 recent_history = full_message_history[-5:]
                 shuffle(recent_history)
                 relevant_memories = permanent_memory.get_relevant(
                     str(recent_history), 5
                 )
                 if relevant_memories:
                     shuffle(relevant_memories)
                 relevant_memory = str(relevant_memories)
            relevant_memory = ""
            logger.debug(f"Memory Stats: {permanent_memory.get_stats()}")

            (
                next_message_to_add_index,
                current_tokens_used,
                insertion_index,
                current_context,
            ) = generate_context(prompt, relevant_memory, full_message_history, model)

            while current_tokens_used > 2500:
                 # remove memories until we are under 2500 tokens
                 relevant_memory = relevant_memory[:-1]
                 (
                     next_message_to_add_index,
                     current_tokens_used,
                     insertion_index,
                     current_context,
                 ) = generate_context(
                     prompt, relevant_memory, full_message_history, model
                 )

            current_tokens_used += count_message_tokens(
                [create_chat_message("user", user_input)], model
            )  # Account for user input (appended later)

            current_tokens_used += 500  # Account for memory (appended later) TODO: The final memory may be less than 500 tokens

            # Add Messages until the token limit is reached or there are no more messages to add.
            while next_message_to_add_index >= 0:
                # print (f"CURRENT TOKENS USED: {current_tokens_used}")
                message_to_add = full_message_history[next_message_to_add_index]

                tokens_to_add = count_message_tokens([message_to_add], model)
                if current_tokens_used + tokens_to_add > send_token_limit:
                    save_memory_trimmed_from_context_window(
                        full_message_history,
                        next_message_to_add_index,
                         permanent_memory,
                     )
                    break

                # Add the most recent message to the start of the current context,
                #  after the two system prompts.
                current_context.insert(
                    insertion_index, full_message_history[next_message_to_add_index]
                )

                # Count the currently used tokens
                current_tokens_used += tokens_to_add

                # Move to the next most recent message in the full message history
                next_message_to_add_index -= 1

            # Insert Memories
            if len(full_message_history) > 0:
                (
                    newly_trimmed_messages,
                    agent.last_memory_index,
                ) = get_newly_trimmed_messages(
                    full_message_history=full_message_history,
                    current_context=current_context,
                    last_memory_index=agent.last_memory_index,
                )
                agent.summary_memory = update_running_summary(
                    current_memory=agent.summary_memory,
                    new_events=newly_trimmed_messages,
                )
                current_context.insert(insertion_index, agent.summary_memory)

            api_manager = ApiManager()
            # inform the AI about its remaining budget (if it has one)
            if api_manager.get_total_budget() > 0.0:
                remaining_budget = (
                    api_manager.get_total_budget() - api_manager.get_total_cost()
                )
                if remaining_budget < 0:
                    remaining_budget = 0
                system_message = (
                    f"Your remaining API budget is ${remaining_budget:.3f}"
                    + (
                        " BUDGET EXCEEDED! SHUT DOWN!\n\n"
                        if remaining_budget == 0
                        else " Budget very nearly exceeded! Shut down gracefully!\n\n"
                        if remaining_budget < 0.005
                        else " Budget nearly exceeded. Finish up.\n\n"
                        if remaining_budget < 0.01
                        else "\n\n"
                    )
                )
                logger.debug(system_message)
                current_context.append(create_chat_message("system", system_message))

            # Append user input, the length of this is accounted for above
            current_context.extend([create_chat_message("user", user_input)])

            plugin_count = len(cfg.plugins)
            for i, plugin in enumerate(cfg.plugins):
                if not plugin.can_handle_on_planning():
                    continue
                plugin_response = plugin.on_planning(
                    agent.prompt_generator, current_context
                )
                if not plugin_response or plugin_response == "":
                    continue
                tokens_to_add = count_message_tokens(
                    [create_chat_message("system", plugin_response)], model
                )
                if current_tokens_used + tokens_to_add > send_token_limit:
                    logger.debug("Plugin response too long, skipping:", plugin_response)
                    logger.debug("Plugins remaining at stop:", plugin_count - i)
                    break
                current_context.append(create_chat_message("system", plugin_response))

            # Calculate remaining tokens
            tokens_remaining = token_limit - current_tokens_used
            assert tokens_remaining >= 0, "Tokens remaining is negative"
            # This should never happen, please submit a bug report at
            #  https://www.github.com/Torantulino/Auto-GPT"

            # Debug print the current context
            logger.debug(f"Token limit: {token_limit}")
            logger.debug(f"Send Token Count: {current_tokens_used}")
            logger.debug(f"Tokens remaining for response: {tokens_remaining}")
            logger.debug("------------ CONTEXT SENT TO AI ---------------")
            for message in current_context:
                # Skip printing the prompt
                if message["role"] == "system" and message["content"] == prompt:
                    continue
                logger.debug(f"{message['role'].capitalize()}: {message['content']}")
                logger.debug("")
            logger.debug("----------- END OF CONTEXT ----------------")

            # TODO: use a model defined elsewhere, so that model can contain
            # temperature and other settings we care about
            assistant_reply = create_chat_completion(
                model=model,
                messages=current_context,
                max_tokens=tokens_remaining,
            )

            # Update full message history
            full_message_history.append(create_chat_message("user", user_input))
            full_message_history.append(
                create_chat_message("assistant", assistant_reply)
            )

            return assistant_reply
        except RateLimitError:
            # TODO: When we switch to langchain, this is built in
            logger.warn("Error: ", "API Rate Limit Reached. Waiting 10 seconds...")
            time.sleep(10)

Still having this problem in 0.3.1
This problem crashed the entire flow, maybe we just prevent crashing it and keep it continuing? 

same problem with long html 
+1 same error, nothing has worked as a workaround. 
+1 Same error
+1 same error
+1
**UPDATE: My experiment ultimately did not work as expected, and the dev team should consider using chunks.**

I'm running locally with automatic coding disabled (not in Docker).

Here's my commit reference: commit 3d494f1032f77884f348ba0e89cfe0fd5022f9f4 (HEAD -> stable, tag: v0.3.1, origin/stable)

In my case, the error is caused by the function `create_chat_completion` on line 55 of `Auto-GPT\autogpt\llm\api_manager.py`. I believe the message list exceeds Open API's expected input. I added some hard-coded message limits to see if it would fix the issue. I will let you know if this works or not.

> **UPDATE: Currently testing the changes.**
> 
> I'm running locally with automatic coding disabled (not in Docker).
> 
> Here's my commit reference: commit [3d494f1](https://github.com/Significant-Gravitas/Auto-GPT/commit/3d494f1032f77884f348ba0e89cfe0fd5022f9f4) (HEAD -> stable, tag: v0.3.1, origin/stable)
> 
> In my case, the error is caused by the function `create_chat_completion` on line 55 of `Auto-GPT\autogpt\llm\api_manager.py`. I believe the message list exceeds Open API's expected input. I added some hard-coded message limits to see if it would fix the issue. I will let you know if this works or not.
> 
> Here's what I'm experimenting with:
> 
> api_manger.py
> llm_utils.py



thank you for working on this, let us know if your solution works out.
HI

I am brand new to autogpt and only set it up yesterday.

I have this issue!  Does anyone yet have a fix?
Same here for exceeding 4097 tokens.  None of my agents will finish a task.  They all blow up with this error at some point and then I see what I can salvage from the files created.  
### This has been reported numerous times across multiple issues, and the core contributors are already aware of and working on it.

That said...

Through trial and error, and as previously mentioned, I also believe the optimal solution lies in segmenting the requests into "chunks," akin to the method employed by the Superpower ChatGPT plugin. I will explain.

With ChatGPT 3.5, a token budget of 4097 is allocated, which can be utilized for either input, output or a combination of both.

The issue arises when Auto-GPT transmits a considerable volume of data, consuming all the allocated tokens, and leaving none for the response. Alternatively, truncating the data sent to ChatGPT results in errors during the response creation and handling.

Therefore, the proposed fix involves identifying the total token count using a tokenizer on the input text, dividing the request into segments or 'chunks,' appending the pre and post-sections, and progressively submitting them until the quota is exhausted. The submission would be divided into 'X' parts, where 'X' is a factor of (4000 - pre/post section token length).

For instance, here's how Superpower ChatGPT effectively implements this strategy:

```text
Act like a document/text loader until you load and remember the content of the next text/s or document/s.
There might be multiple files, each file is marked by name in the format ### DOCUMENT NAME.
I will send them to you in chunks. Each chunk starts will be noted as [START CHUNK x/TOTAL], and the end of this chunk will be noted as [END CHUNK x/TOTAL], where x is the number of current chunks, and TOTAL is the number of all chunks I will send you.
I will split the message in chunks, and send them to you one by one. For each message follow the instructions at the end of the message.
Let's begin:

[START CHUNK 1/2]

... THE CHUNK CONTENT GOES HERE ...

[END CHUNK 1/2]
Reply with OK: [CHUNK x/TOTAL]
Don't reply with anything else!
```

Superpower ChatGPT on the Google Chrome webstore: https://chrome.google.com/webstore/detail/superpower-chatgpt/amhmeenmapldpjdedekalnfifgnpfnkc

See also: https://github.com/saeedezzati/superpower-chatgpt
See also: https://medium.com/@josediazmoreno/break-the-limits-send-large-text-blocks-to-chatgpt-with-ease-6824b86d3270
If anyone is working on a patch, I'd definitely give it a whirl.  Not at a point right now (commitment and time wise) to work on one...even with Copilot and ChatGPT as my pair programming buddy!
-- Feature Leech
just leaving a +1 here 

needs core devs to work on a way to chunk - oh and thanks to them for helping a bunch of us - this is a challenging one as it stops workflow of agents (ie no recovery)

------

openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 5394 tokens. Please reduce the length of the messages.
Same issue :(
The tool became totally unusable 
Any solution?
Same here. Now trying 0.4, but still get the fatal error "This model's maximum context length is 4097 tokens" each time I try diferent --manual goals or automatic prompt
> Same here. Now trying 0.4, but still get the fatal error "This model's maximum context length is 4097 tokens" each time I try diferent --manual goals or automatic prompt

what are the goals that you guys usually give?
I think the issue is that the devs have not integrated tiktoken into the platform, this is why this is happening.

TikToken will basically count the tokens needed to send your request and then we can automatically adjust the max tokens we send openAI so that it does not try to send back a response that would exceed the max token count for your model. Also there should be some left unused to accommodate the small margin of error tik token can produce.

We have developed an AutoGPT UI that we are about to release opensource and we are debating on integrating tiktoken and filing a pull request to bring it into the platform but we dont want to duplicate the effort if the core dev team is already working this.
BRUTAL fix : either substring messages to 4000 lengh, or use OpenAI to do summarize.

for summarizing here is the code which i made in function create_chat_completion in file api_manager.py

```
def summarise(self, conversation) -> str:
        """
        Summarises the conversation history.
        :param conversation: The conversation history
        :return: The summary
        """
        messages = [
            { "role": "assistant", "content": "Summarize this conversation in 2000 characters or less" },
            { "role": "user", "content": str(conversation) }
        ]
        response = openai.ChatCompletion.create(
            model=self.config['model'],
            messages=messages,
            temperature=0.1
        )
        return response.choices[0]['message']['content']

```

and in create_chat_completion i made this:

        `#fix length
        sumlen=0
        strmess=""
        for mess in messages:
            sumlen=sumlen+len(mess.content)
            strmess = strmess + " "+mess.content
        if sumlen>=4000:
            #summarize
            summary = self.summarise(strmess)
            response = openai.ChatCompletion.create(
                deployment_id=deployment_id,
                model=model,
                messages=summary,
                temperature=temperature,
                max_tokens=max_tokens,
                api_key=cfg.openai_api_key,
            )
            return response`
HI

I couldn't get this to work, could you paste the full file of your api_manager.py so i can copy/paste?

Solution patch:
diff --git a/autogpt/memory/message_history.py b/autogpt/memory/message_history.py
--- a/autogpt/memory/message_history.py
+++ b/autogpt/memory/message_history.py
@@ -14,7 +14,8 @@
     is_string_valid_json,
 )
 from autogpt.llm.base import ChatSequence, Message, MessageRole, MessageType
-from autogpt.llm.utils import create_chat_completion
+from autogpt.llm.providers.openai import OPEN_AI_CHAT_MODELS
+from autogpt.llm.utils import count_string_tokens, create_chat_completion
 from autogpt.log_cycle.log_cycle import PROMPT_SUMMARY_FILE_NAME, SUMMARY_FILE_NAME
 from autogpt.logs import logger
 
@@ -167,20 +168,49 @@ def update_running_summary(self, new_events: list[Message]) -> Message:
             elif event.role == "user":
                 new_events.remove(event)
 
+        # Summarize events and current summary in batch to a new running summary
+
+        # Assume an upper bound length for the summary prompt template, i.e. Your task is to create a concise running summary...., in summarize_batch func
+        # TODO make this default dynamic
+        prompt_template_length = 100
+        max_tokens = OPEN_AI_CHAT_MODELS.get(cfg.fast_llm_model).max_tokens
+        batch = []
+        batch_tlength = 0
+
+        # TODO Can put a cap on length of total new events and drop some previous events to save API cost, but need to think thru more how to do it without losing the context
+        for event in new_events:
+            event_tlength = count_string_tokens(str(event), cfg.fast_llm_model)
+
+            if batch_tlength + event_tlength > max_tokens - prompt_template_length:
+                # The batch is full. Summarize it and start a new one.
+                self.summarize_batch(batch, cfg)
+                batch = [event]
+                batch_tlength = event_tlength
+            else:
+                batch.append(event)
+                batch_tlength += event_tlength
+
+        if batch:
+            # There's an unprocessed batch. Summarize it.
+            self.summarize_batch(batch, cfg)
+
+        return self.summary_message()
+
+    def summarize_batch(self, new_events_batch, cfg):
         prompt = f'''Your task is to create a concise running summary of actions and information results in the provided text, focusing on key and potentially important information to remember.
 
-You will receive the current summary and the your latest actions. Combine them, adding relevant key information from the latest development in 1st person past tense and keeping the summary concise.
+        You will receive the current summary and your latest actions. Combine them, adding relevant key information from the latest development in 1st person past tense and keeping the summary concise.
 
-Summary So Far:
-"""
-{self.summary}
-"""
+        Summary So Far:
+        """
+        {self.summary}
+        """
 
-Latest Development:
-"""
-{new_events or "Nothing new happened."}
-"""
-'''
+        Latest Development:
+        """
+        {new_events_batch or "Nothing new happened."}
+        """
+        '''
 
         prompt = ChatSequence.for_model(cfg.fast_llm_model, [Message("user", prompt)])
         self.agent.log_cycle_handler.log_cycle(
@@ -200,5 +230,3 @@ def update_running_summary(self, new_events: list[Message]) -> Message:
             self.summary,
             SUMMARY_FILE_NAME,
         )
-
-        return self.summary_message()


Test patch:
diff --git a/tests/unit/test_message_history.py b/tests/unit/test_message_history.py
new file mode 100644
--- /dev/null
+++ b/tests/unit/test_message_history.py
@@ -0,0 +1,145 @@
+import math
+import time
+from unittest.mock import MagicMock
+
+import pytest
+
+from autogpt.agent import Agent
+from autogpt.config import AIConfig
+from autogpt.config.config import Config
+from autogpt.llm.base import ChatSequence, Message
+from autogpt.llm.providers.openai import OPEN_AI_CHAT_MODELS
+from autogpt.llm.utils import count_string_tokens
+from autogpt.memory.message_history import MessageHistory
+
+
+@pytest.fixture
+def agent(config: Config):
+    ai_name = "Test AI"
+    memory = MagicMock()
+    next_action_count = 0
+    command_registry = MagicMock()
+    ai_config = AIConfig(ai_name=ai_name)
+    system_prompt = "System prompt"
+    triggering_prompt = "Triggering prompt"
+    workspace_directory = "workspace_directory"
+
+    agent = Agent(
+        ai_name=ai_name,
+        memory=memory,
+        next_action_count=next_action_count,
+        command_registry=command_registry,
+        ai_config=ai_config,
+        config=config,
+        system_prompt=system_prompt,
+        triggering_prompt=triggering_prompt,
+        workspace_directory=workspace_directory,
+    )
+    return agent
+
+
+def test_message_history_batch_summary(mocker, agent):
+    config = Config()
+    history = MessageHistory(agent)
+    model = config.fast_llm_model
+    message_tlength = 0
+    message_count = 0
+
+    # Setting the mock output and inputs
+    mock_summary_text = "I executed browse_website command for each of the websites returned from Google search, but none of them have any job openings."
+    mock_summary = mocker.patch(
+        "autogpt.memory.message_history.create_chat_completion",
+        return_value=mock_summary_text,
+    )
+
+    system_prompt = 'You are AIJobSearcher, an AI designed to search for job openings for software engineer role\nYour decisions must always be made independently without seeking user assistance. Play to your strengths as an LLM and pursue simple strategies with no legal complications.\n\nGOALS:\n\n1. Find any job openings for software engineers online\n2. Go through each of the websites and job openings to summarize their requirements and URL, and skip that if you already visit the website\n\nIt takes money to let you run. Your API budget is $5.000\n\nConstraints:\n1. ~4000 word limit for short term memory. Your short term memory is short, so immediately save important information to files.\n2. If you are unsure how you previously did something or want to recall past events, thinking about similar events will help you remember.\n3. No user assistance\n4. Exclusively use the commands listed in double quotes e.g. "command name"\n\nCommands:\n1. google_search: Google Search, args: "query": "<query>"\n2. browse_website: Browse Website, args: "url": "<url>", "question": "<what_you_want_to_find_on_website>"\n3. task_complete: Task Complete (Shutdown), args: "reason": "<reason>"\n\nResources:\n1. Internet access for searches and information gathering.\n2. Long Term memory management.\n3. GPT-3.5 powered Agents for delegation of simple tasks.\n4. File output.\n\nPerformance Evaluation:\n1. Continuously review and analyze your actions to ensure you are performing to the best of your abilities.\n2. Constructively self-criticize your big-picture behavior constantly.\n3. Reflect on past decisions and strategies to refine your approach.\n4. Every command has a cost, so be smart and efficient. Aim to complete tasks in the least number of steps.\n5. Write all code to a file.\n\nYou should only respond in JSON format as described below \nResponse Format: \n{\n    "thoughts": {\n        "text": "thought",\n        "reasoning": "reasoning",\n        "plan": "- short bulleted\\n- list that conveys\\n- long-term plan",\n        "criticism": "constructive self-criticism",\n        "speak": "thoughts summary to say to user"\n    },\n    "command": {\n        "name": "command name",\n        "args": {\n            "arg name": "value"\n        }\n    }\n} \nEnsure the response can be parsed by Python json.loads'
+    message_sequence = ChatSequence.for_model(
+        model,
+        [
+            Message("system", system_prompt),
+            Message("system", f"The current time and date is {time.strftime('%c')}"),
+        ],
+    )
+    insertion_index = len(message_sequence)
+
+    user_input = "Determine which next command to use, and respond using the format specified above:'"
+    user_input_msg = Message("user", user_input)
+    history.append(user_input_msg)
+
+    # mock a reponse from AI
+    assistant_reply = '{\n    "thoughts": {\n        "text": "I will use the \'google_search\' command to find more websites with job openings for software engineering manager role.",\n        "reasoning": "Since the previous website did not provide any relevant information, I will use the \'google_search\' command to find more websites with job openings for software engineer role.",\n        "plan": "- Use \'google_search\' command to find more websites with job openings for software engineer role",\n        "criticism": "I need to ensure that I am able to extract the relevant information from each website and job opening.",\n        "speak": "I will now use the \'google_search\' command to find more websites with job openings for software engineer role."\n    },\n    "command": {\n        "name": "google_search",\n        "args": {\n            "query": "software engineer job openings"\n        }\n    }\n}'
+    msg = Message("assistant", assistant_reply, "ai_response")
+    history.append(msg)
+    message_tlength += count_string_tokens(str(msg), config.fast_llm_model)
+    message_count += 1
+
+    # mock some websites returned from google search command in the past
+    result = "Command google_search returned: ["
+    for i in range(50):
+        result += "http://www.job" + str(i) + ".com,"
+    result += "]"
+    msg = Message("system", result, "action_result")
+    history.append(msg)
+    message_tlength += count_string_tokens(str(msg), config.fast_llm_model)
+    message_count += 1
+
+    user_input = "Determine which next command to use, and respond using the format specified above:'"
+    user_input_msg = Message("user", user_input)
+    history.append(user_input_msg)
+
+    # mock numbers of AI response and action results from browse_website commands in the past, doesn't need the thoughts part, as the summarization code discard them anyway
+    for i in range(50):
+        assistant_reply = (
+            '{\n    "command": {\n        "name": "browse_website",\n        "args": {\n            "url": "https://www.job'
+            + str(i)
+            + '.com",\n            "question": "software engineer"\n        }\n    }\n}'
+        )
+        msg = Message("assistant", assistant_reply, "ai_response")
+        history.append(msg)
+        message_tlength += count_string_tokens(str(msg), config.fast_llm_model)
+        message_count += 1
+
+        result = (
+            "Command browse_website returned: Answer gathered from website: The text in job"
+            + str(i)
+            + " does not provide information on specific job requirements or a job URL.]",
+        )
+        msg = Message("system", result, "action_result")
+        history.append(msg)
+        message_tlength += count_string_tokens(str(msg), config.fast_llm_model)
+        message_count += 1
+
+        user_input = "Determine which next command to use, and respond using the format specified above:'"
+        user_input_msg = Message("user", user_input)
+        history.append(user_input_msg)
+
+    # only take the last cycle of the message history,  trim the rest of previous messages, and generate a summary for them
+    for cycle in reversed(list(history.per_cycle())):
+        messages_to_add = [msg for msg in cycle if msg is not None]
+        message_sequence.insert(insertion_index, *messages_to_add)
+        break
+
+    # count the expected token length of the trimmed message by reducing the token length of messages in the last cycle
+    for message in messages_to_add:
+        if message.role != "user":
+            message_tlength -= count_string_tokens(str(message), config.fast_llm_model)
+            message_count -= 1
+
+    # test the main trim_message function
+    new_summary_message, trimmed_messages = history.trim_messages(
+        current_message_chain=list(message_sequence),
+    )
+
+    expected_call_count = math.ceil(
+        message_tlength / (OPEN_AI_CHAT_MODELS.get(config.fast_llm_model).max_tokens)
+    )
+    # Expecting 2 batches because of over max token
+    assert mock_summary.call_count == expected_call_count  # 2 at the time of writing
+    # Expecting 100 messages because 50 pairs of ai_response and action_result, based on the range set above
+    assert len(trimmed_messages) == message_count  # 100 at the time of writing
+    assert new_summary_message == Message(
+        role="system",
+        content="This reminds you of these events from your past: \n"
+        + mock_summary_text,
+        type=None,
+    )