Repository: Significant-Gravitas/AutoGPT
Pull request: Fix #942: Respect --gpt4only, --gpt3only cli args
### Background
Addressing #942:

AutoGPT is hard coded to use fast_llm_model for everything except the write_tests, evaluate_code, and improve_code commands. (see https://github.com/Torantulino/Auto-GPT/blob/master/scripts/chat.py#L66 for example)

The result is that even if I use --gpt4only, we still see GPT 3.5 being used.

**Steps to reproduce 🕹**
Run python scripts/main.py with OpenAI API key that supports gp4 usage in env and pass --gpt4only as cli arg

**Current behavior 😯**
From #942: AutoGPT is only making use of gpt-3.5-turbo-0301 and text-embedding-ada-002-v2

**Expected behavior 🤔**
AutoGPT should use gpt-4 model if --gpt4only is passed in as cli arg

### Changes
Fix [#942](https://github.com/Significant-Gravitas/Auto-GPT/issues/942): Respect --gpt4only, --gpt3only cli args

- Add `cfg.fast_llm_model` to `chat_with_ai` method parameters.
- Remove the hardcoded model in `chat_with_ai` method and `Agent` class
- Add `GPT_4_MODEL` and `GPT_3_MODEL` constants to `configurator.py`.
- Change `--gpt3only` to use `GPT_3_MODEL` not `CFG.fast_llm_model`.
- Change `--gpt4only` to use `GPT_4_MODEL` not `CFG.smart_llm_model`.
- Add tests for `--gpt4only` and `--gpt3only` CLI args

### Documentation
No documentation changes. The docs will finally do what they say.

### Test Plan
- Added test_cli tests to test the behavior

### PR Quality Checklist
- [x] My pull request is atomic and focuses on a single change.
- [x] I have thoroughly tested my changes with multiple different prompts.
- [x] I have considered potential risks and mitigations for my changes.
- [x] I have documented my changes clearly and comprehensively.
- [x] I have not snuck in any "extra" small tweaks changes 


Problem statement:
Only gpt-3.5 being used, without using --gpt3only modifier
### Duplicates

- [X] I have searched the existing issues

### Steps to reproduce 🕹

Run python scripts/main.py with OpenAI api key that supports gp4 usage in env 

### Current behavior 😯

Judging by my USAGE page on Open AI, AutoGPT is only making use of gpt-3.5-turbo-0301 and text-embedding-ada-002-v2

### Expected behavior 🤔

AutoGPT should use gpt-4 model 

### Your prompt 📝

```yaml
# Paste your prompt here
```

Hints from issue discussion:
I have also seen this, I'm not sure on a cause, yet.
from what I can tell. the script defaults to 3.5. you must use the gpt4only flag.
It's hardcoded to use `fast_llm_model` for everything except the `write_tests`, `evaluate_code` and `improve_code` commands. (see https://github.com/Torantulino/Auto-GPT/blob/master/scripts/chat.py#L66 for example)

So you could change that to gpt-4 if you want to wrack up a bill 😉 
Maybe GPT 4 (or ` smart_llm_model`) should be used by default when choosing the command to use next, which should not use many tokens but is the core of AutoGPT. Oftentimes GPT 3.5 fails to select the appropriate command given the agent's thoughts.
definitely want more input on this #2621 and if it's a good direction need to add tests
See #3144 proposed fix.

Commit message: Merge branch 'master' into bug/942-only-gpt35-being-used-fix

Patch to review:
diff --git a/autogpt/agent/agent.py b/autogpt/agent/agent.py
index 9b48d540dc83..f26783fd3ddc 100644
--- a/autogpt/agent/agent.py
+++ b/autogpt/agent/agent.py
@@ -98,7 +98,8 @@ def start_interaction_loop(self):
                     self.full_message_history,
                     self.memory,
                     cfg.fast_token_limit,
-                )  # TODO: This hardcodes the model to use GPT3.5. Make this an argument
+                    cfg.fast_llm_model,
+                ) 
 
             assistant_reply_json = fix_json_using_multiple_techniques(assistant_reply)
             for plugin in cfg.plugins:
diff --git a/autogpt/chat.py b/autogpt/chat.py
index 469ec9bd38df..b5f1898fd138 100644
--- a/autogpt/chat.py
+++ b/autogpt/chat.py
@@ -54,7 +54,7 @@ def generate_context(prompt, relevant_memory, full_message_history, model):
 
 # TODO: Change debug from hardcode to argument
 def chat_with_ai(
-    agent, prompt, user_input, full_message_history, permanent_memory, token_limit
+    agent, prompt, user_input, full_message_history, permanent_memory, token_limit, model=cfg.fast_llm_model
 ):
     """Interact with the OpenAI API, sending the prompt, user input, message history,
     and permanent memory."""
@@ -76,9 +76,7 @@ def chat_with_ai(
             Returns:
             str: The AI's response.
             """
-            model = cfg.fast_llm_model  # TODO: Change model from hardcode to argument
             # Reserve 1000 tokens for the response
-
             logger.debug(f"Token limit: {token_limit}")
             send_token_limit = token_limit - 1000
             if len(full_message_history) == 0:
diff --git a/autogpt/configurator.py b/autogpt/configurator.py
index 84000e576558..89434e373d2d 100644
--- a/autogpt/configurator.py
+++ b/autogpt/configurator.py
@@ -9,6 +9,8 @@
 
 CFG = Config()
 
+GPT_4_MODEL = "gpt-4"
+GPT_3_MODEL = "gpt-3.5-turbo"
 
 def create_config(
     continuous: bool,
@@ -75,11 +77,15 @@ def create_config(
 
     if gpt3only:
         logger.typewriter_log("GPT3.5 Only Mode: ", Fore.GREEN, "ENABLED")
-        CFG.set_smart_llm_model(CFG.fast_llm_model)
+        # --gpt3only should always use gpt-3.4-turbo, despite user's FAST_LLM_MODEL config
+        CFG.set_fast_llm_model(GPT_3_MODEL)
+        CFG.set_smart_llm_model(GPT_3_MODEL)
 
     if gpt4only:
         logger.typewriter_log("GPT4 Only Mode: ", Fore.GREEN, "ENABLED")
-        CFG.set_fast_llm_model(CFG.smart_llm_model)
+        # --gpt4only should always use gpt-4, despite user's SMART_LLM_MODEL config
+        CFG.set_fast_llm_model(GPT_4_MODEL)
+        CFG.set_smart_llm_model(GPT_4_MODEL)
 
     if memory_type:
         supported_memory = get_supported_memory_backends()
diff --git a/tests/test_cli.py b/tests/test_cli.py
new file mode 100644
index 000000000000..21d6f6dc6978
--- /dev/null
+++ b/tests/test_cli.py
@@ -0,0 +1,20 @@
+from click.testing import CliRunner
+from autogpt import cli
+from autogpt.config.config import Config
+from autogpt.configurator import GPT_3_MODEL, GPT_4_MODEL
+
+def test_gpt4only_cli_arg():
+    runner = CliRunner()
+    runner.invoke(cli.main, ['--gpt4only'])
+    
+    config = Config()
+    assert config.smart_llm_model == GPT_4_MODEL
+    assert config.fast_llm_model == GPT_4_MODEL
+
+def test_gpt3only_cli_arg():
+    runner = CliRunner()
+    runner.invoke(cli.main, ['--gpt3only'])
+    
+    config = Config()
+    assert config.smart_llm_model == GPT_3_MODEL
+    assert config.fast_llm_model == GPT_3_MODEL
\ No newline at end of file


Code review comments:
diff_hunk: @@ -0,0 +1,20 @@
+from click.testing import CliRunner
+from autogpt import cli
+from autogpt.config.config import Config
+from autogpt.configurator import GPT_3_MODEL, GPT_4_MODEL
+
+def test_gpt4only_cli_arg():
+    runner = CliRunner()
+    runner.invoke(cli.main, ['--gpt4only'])
path: tests/test_cli.py
text: @user1:
This influences the `Config` singleton, causing the test suite to fail.