Repository: Significant-Gravitas/AutoGPT
Pull request: Plugin Support
### Background

Adds basic plugin support.

### Changes

Add plugins.py which handles scanning plugins folder for zipfiles and check for modules to import.

In main.py on startup it will load the plugins into the config file.

In LLM utils it will iterate the plugins on each response.

### Documentation

Docstrings and README updated.

### Test Plan

I tested this code here: https://github.com/BillSchumacher/Auto-Vicuna

I have an example plugin for that project here: https://github.com/BillSchumacher/Auto-Vicuna-Its-Alive

It would probably work, but it allows optional return in on_response.

However, I did make changes like add a whitelist / blacklist as discussed.

There's a chance I broke something. This was mostly a copy/paste.

### PR Quality Checklist
- [x] My pull request is atomic and focuses on a single change.
- [x] I have thouroughly tested my changes with multiple different prompts.
- [x] I have considered potential risks and mitigations for my changes.
- [x] I have documented my changes clearly and comprehensively.
- [x] I have not snuck in any "extra" small tweaks changes <!-- Submit these as seperate Pull Reqests, they are the easiest to merge! -->

I have not tested this at all, I hope a lot of people make plugins and do test.

<!-- By submitting this, I agree that my pull request should be closed if I do not fill this out or follow the guide lines. -->


Problem statement:
Framework for optional features
### Duplicates

- [X] I have searched the existing issues

### Summary 💡

We should make some framework for optional features, so that they:

- don't make default configuration harder for newcomers
- don't bloat .env.template
- don't bloat the prompt
- don't bloat the README

### Examples 🌈

Well, people add commands all the time.

### Motivation 🔦

The setup should be very easy for newcomers, with just very bare bones configuration. Preferrably, only the model itself.

And optional features should be modular, with additions to the prompt and to the command registry and to the config.

Maybe we should have a provision for OS-specific features, such as OS-specific TTS (both Windows and MacOS have built-in TTS, as an example).

Hints from issue discussion:
It would be great to have a config file too, not just everything inside the .env as I would assume the environment vars are more important and less optional than just config.

Right now, my feedback as a newcomer:
It is already really hard to tell what is mandatory and what is not, because the Readme first states some basic setup, then continues to explain stuff like setting up redis etc. without a clear cut of what shall be done and what not.

Even Bing Chat got confused and tried to convince me to setup a huggingface api and have redis run on an azure server lol
Indeed.  It's amazing to get this running for experiments and exploration without too much familiarity. 
Though with limited experience I regret trying to figure out the new Redis changes by following the Readme, and probably will optionally avoid Redis throughout once I also remove Docker.

Commit message: Fix early abort

Patch to review:
diff --git a/README.md b/README.md
index 71957748f067..2fad68737df6 100644
--- a/README.md
+++ b/README.md
@@ -253,6 +253,30 @@ export GOOGLE_API_KEY="YOUR_GOOGLE_API_KEY"
 export CUSTOM_SEARCH_ENGINE_ID="YOUR_CUSTOM_SEARCH_ENGINE_ID"
 ```
 
+## Plugins
+
+See https://github.com/Torantulino/Auto-GPT-Plugin-Template for the template of the plugins.
+
+⚠️💀 WARNING 💀⚠️: Review the code of any plugin you use, this allows for any Python to be executed and do malicious things. Like stealing your API keys.
+
+Drop the repo's zipfile in the plugins folder.
+
+![Download Zip](https://raw.githubusercontent.com/BillSchumacher/Auto-GPT/master/plugin.png)
+
+If you add the plugins class name to the whitelist in the config.py you will not be prompted otherwise you'll be warned before loading the plugin.
+
+## Plugins
+
+See https://github.com/Torantulino/Auto-GPT-Plugin-Template for the template of the plugins.
+
+⚠️💀 WARNING 💀⚠️: Review the code of any plugin you use, this allows for any Python to be executed and do malicious things. Like stealing your API keys.
+
+Drop the repo's zipfile in the plugins folder.
+
+![Download Zip](https://raw.githubusercontent.com/BillSchumacher/Auto-GPT/master/plugin.png)
+
+If you add the plugins class name to the whitelist in the config.py you will not be prompted otherwise you'll be warned before loading the plugin.
+
 ## Setting Your Cache Type
 
 By default, Auto-GPT is going to use LocalCache instead of redis or Pinecone.
diff --git a/autogpt/__main__.py b/autogpt/__main__.py
index 5f4622347d9a..c2e2e5c168cd 100644
--- a/autogpt/__main__.py
+++ b/autogpt/__main__.py
@@ -1,12 +1,19 @@
 """Main script for the autogpt package."""
 import logging
+import os
+from pathlib import Path
+
 from colorama import Fore
+
 from autogpt.agent.agent import Agent
 from autogpt.args import parse_arguments
+from autogpt.commands.command import CommandRegistry
 from autogpt.config import Config, check_openai_api_key
 from autogpt.logs import logger
 from autogpt.memory import get_memory
-from autogpt.prompt import construct_prompt
+from autogpt.plugins import load_plugins
+from autogpt.prompts.prompt import construct_main_ai_config
+
 # Load environment variables from .env file
 
 
@@ -17,8 +24,43 @@ def main() -> None:
     check_openai_api_key()
     parse_arguments()
     logger.set_level(logging.DEBUG if cfg.debug_mode else logging.INFO)
+    plugins_found = load_plugins(Path(os.getcwd()) / "plugins")
+    loaded_plugins = []
+    for plugin in plugins_found:
+        if plugin.__name__ in cfg.plugins_blacklist:
+            continue
+        if plugin.__name__ in cfg.plugins_whitelist:
+            loaded_plugins.append(plugin())
+        else:
+            ack = input(
+                f"WARNNG Plugin {plugin.__name__} found. But not in the"
+                " whitelist... Load? (y/n): "
+            )
+            if ack.lower() == "y":
+                loaded_plugins.append(plugin())
+
+    if loaded_plugins:
+        print(f"\nPlugins found: {len(loaded_plugins)}\n" "--------------------")
+    for plugin in loaded_plugins:
+        print(f"{plugin._name}: {plugin._version} - {plugin._description}")
+
+    cfg.set_plugins(loaded_plugins)
+    # Create a CommandRegistry instance and scan default folder
+    command_registry = CommandRegistry()
+    command_registry.import_commands("autogpt.commands.audio_text")
+    command_registry.import_commands("autogpt.commands.evaluate_code")
+    command_registry.import_commands("autogpt.commands.execute_code")
+    command_registry.import_commands("autogpt.commands.file_operations")
+    command_registry.import_commands("autogpt.commands.git_operations")
+    command_registry.import_commands("autogpt.commands.google_search")
+    command_registry.import_commands("autogpt.commands.image_gen")
+    command_registry.import_commands("autogpt.commands.twitter")
+    command_registry.import_commands("autogpt.commands.web_selenium")
+    command_registry.import_commands("autogpt.commands.write_tests")
+    command_registry.import_commands("autogpt.app")
     ai_name = ""
-    system_prompt = construct_prompt()
+    ai_config = construct_main_ai_config()
+    ai_config.command_registry = command_registry
     # print(prompt)
     # Initialize variables
     full_message_history = []
@@ -32,14 +74,19 @@ def main() -> None:
     # this is particularly important for indexing and referencing pinecone memory
     memory = get_memory(cfg, init=True)
     logger.typewriter_log(
-        f"Using memory of type:", Fore.GREEN, f"{memory.__class__.__name__}"
+        "Using memory of type:", Fore.GREEN, f"{memory.__class__.__name__}"
     )
-    logger.typewriter_log(f"Using Browser:", Fore.GREEN, cfg.selenium_web_browser)
+    logger.typewriter_log("Using Browser:", Fore.GREEN, cfg.selenium_web_browser)
+    system_prompt = ai_config.construct_full_prompt()
+    if cfg.debug_mode:
+        logger.typewriter_log("Prompt:", Fore.GREEN, system_prompt)
     agent = Agent(
         ai_name=ai_name,
         memory=memory,
         full_message_history=full_message_history,
         next_action_count=next_action_count,
+        command_registry=command_registry,
+        config=ai_config,
         system_prompt=system_prompt,
         triggering_prompt=triggering_prompt,
     )
diff --git a/autogpt/agent/agent.py b/autogpt/agent/agent.py
index dca614c7f239..3a79760cc725 100644
--- a/autogpt/agent/agent.py
+++ b/autogpt/agent/agent.py
@@ -1,6 +1,6 @@
 from colorama import Fore, Style
-from autogpt.app import execute_command, get_command
 
+from autogpt.app import execute_command, get_command
 from autogpt.chat import chat_with_ai, create_chat_message
 from autogpt.config import Config
 from autogpt.json_fixes.master_json_fix_method import fix_json_using_multiple_techniques
@@ -19,18 +19,25 @@ class Agent:
         memory: The memory object to use.
         full_message_history: The full message history.
         next_action_count: The number of actions to execute.
-        system_prompt: The system prompt is the initial prompt that defines everything the AI needs to know to achieve its task successfully.
-        Currently, the dynamic and customizable information in the system prompt are ai_name, description and goals.
-
-        triggering_prompt: The last sentence the AI will see before answering. For Auto-GPT, this prompt is:
-            Determine which next command to use, and respond using the format specified above:
-            The triggering prompt is not part of the system prompt because between the system prompt and the triggering
-            prompt we have contextual information that can distract the AI and make it forget that its goal is to find the next task to achieve.
+        system_prompt: The system prompt is the initial prompt that defines everything
+          the AI needs to know to achieve its task successfully.
+        Currently, the dynamic and customizable information in the system prompt are
+          ai_name, description and goals.
+
+        triggering_prompt: The last sentence the AI will see before answering.
+            For Auto-GPT, this prompt is:
+            Determine which next command to use, and respond using the format specified
+              above:
+            The triggering prompt is not part of the system prompt because between the
+              system prompt and the triggering
+            prompt we have contextual information that can distract the AI and make it
+              forget that its goal is to find the next task to achieve.
             SYSTEM PROMPT
             CONTEXTUAL INFORMATION (memory, previous conversations, anything relevant)
             TRIGGERING PROMPT
 
-        The triggering prompt reminds the AI about its short term meta task (defining the next task)
+        The triggering prompt reminds the AI about its short term meta task
+        (defining the next task)
     """
 
     def __init__(
@@ -39,6 +46,8 @@ def __init__(
         memory,
         full_message_history,
         next_action_count,
+        command_registry,
+        config,
         system_prompt,
         triggering_prompt,
     ):
@@ -46,6 +55,8 @@ def __init__(
         self.memory = memory
         self.full_message_history = full_message_history
         self.next_action_count = next_action_count
+        self.command_registry = command_registry
+        self.config = config
         self.system_prompt = system_prompt
         self.triggering_prompt = triggering_prompt
 
@@ -73,6 +84,7 @@ def start_interaction_loop(self):
             # Send message to AI, get response
             with Spinner("Thinking... "):
                 assistant_reply = chat_with_ai(
+                    self,
                     self.system_prompt,
                     self.triggering_prompt,
                     self.full_message_history,
@@ -81,22 +93,25 @@ def start_interaction_loop(self):
                 )  # TODO: This hardcodes the model to use GPT3.5. Make this an argument
 
             assistant_reply_json = fix_json_using_multiple_techniques(assistant_reply)
+            for plugin in cfg.plugins:
+                if not plugin.can_handle_post_planning():
+                    continue
+                assistant_reply_json = plugin.post_planning(self, assistant_reply_json)
 
             # Print Assistant thoughts
             if assistant_reply_json != {}:
-                validate_json(assistant_reply_json, 'llm_response_format_1')
+                validate_json(assistant_reply_json, "llm_response_format_1")
                 # Get command name and arguments
                 try:
                     print_assistant_thoughts(self.ai_name, assistant_reply_json)
                     command_name, arguments = get_command(assistant_reply_json)
-                    # command_name, arguments = assistant_reply_json_valid["command"]["name"], assistant_reply_json_valid["command"]["args"]
                     if cfg.speak_mode:
                         say_text(f"I want to execute {command_name}")
                 except Exception as e:
                     logger.error("Error: \n", str(e))
 
             if not cfg.continuous_mode and self.next_action_count == 0:
-                ### GET USER AUTHORIZATION TO EXECUTE COMMAND ###
+                # ### GET USER AUTHORIZATION TO EXECUTE COMMAND ###
                 # Get key press: Prompt the user to press enter to continue or escape
                 # to exit
                 logger.typewriter_log(
@@ -115,6 +130,7 @@ def start_interaction_loop(self):
                     console_input = clean_input(
                         Fore.MAGENTA + "Input:" + Style.RESET_ALL
                     )
+
                     if console_input.lower().rstrip() == "y":
                         user_input = "GENERATE NEXT COMMAND JSON"
                         break
@@ -165,30 +181,46 @@ def start_interaction_loop(self):
             elif command_name == "human_feedback":
                 result = f"Human feedback: {user_input}"
             else:
-                result = (
-                    f"Command {command_name} returned: "
-                    f"{execute_command(command_name, arguments)}"
+                for plugin in cfg.plugins:
+                    if not plugin.can_handle_pre_command():
+                        continue
+                    command_name, arguments = plugin.pre_command(
+                        command_name, arguments
+                    )
+                command_result = execute_command(
+                    self.command_registry,
+                    command_name,
+                    arguments,
+                    self.config.prompt_generator,
                 )
+                result = f"Command {command_name} returned: " f"{command_result}"
+
+                for plugin in cfg.plugins:
+                    if not plugin.can_handle_post_command():
+                        continue
+                    result = plugin.post_command(command_name, result)
                 if self.next_action_count > 0:
                     self.next_action_count -= 1
+            if command_name != "do_nothing":
+                memory_to_add = (
+                    f"Assistant Reply: {assistant_reply} "
+                    f"\nResult: {result} "
+                    f"\nHuman Feedback: {user_input} "
+                )
 
-            memory_to_add = (
-                f"Assistant Reply: {assistant_reply} "
-                f"\nResult: {result} "
-                f"\nHuman Feedback: {user_input} "
-            )
-
-            self.memory.add(memory_to_add)
+                self.memory.add(memory_to_add)
 
-            # Check if there's a result from the command append it to the message
-            # history
-            if result is not None:
-                self.full_message_history.append(create_chat_message("system", result))
-                logger.typewriter_log("SYSTEM: ", Fore.YELLOW, result)
-            else:
-                self.full_message_history.append(
-                    create_chat_message("system", "Unable to execute command")
-                )
-                logger.typewriter_log(
-                    "SYSTEM: ", Fore.YELLOW, "Unable to execute command"
-                )
+                # Check if there's a result from the command append it to the message
+                # history
+                if result is not None:
+                    self.full_message_history.append(
+                        create_chat_message("system", result)
+                    )
+                    logger.typewriter_log("SYSTEM: ", Fore.YELLOW, result)
+                else:
+                    self.full_message_history.append(
+                        create_chat_message("system", "Unable to execute command")
+                    )
+                    logger.typewriter_log(
+                        "SYSTEM: ", Fore.YELLOW, "Unable to execute command"
+                    )
diff --git a/autogpt/agent/agent_manager.py b/autogpt/agent/agent_manager.py
index e4bfb12611d4..d264815084a3 100644
--- a/autogpt/agent/agent_manager.py
+++ b/autogpt/agent/agent_manager.py
@@ -1,8 +1,8 @@
 """Agent manager for managing GPT agents"""
 from __future__ import annotations
 
+from autogpt.config.config import Config, Singleton
 from autogpt.llm_utils import create_chat_completion
-from autogpt.config.config import Singleton
 
 
 class AgentManager(metaclass=Singleton):
@@ -11,6 +11,7 @@ class AgentManager(metaclass=Singleton):
     def __init__(self):
         self.next_key = 0
         self.agents = {}  # key, (task, full_message_history, model)
+        self.cfg = Config()
 
     # Create new GPT agent
     # TODO: Centralise use of create_chat_completion() to globally enforce token limit
@@ -29,6 +30,13 @@ def create_agent(self, task: str, prompt: str, model: str) -> tuple[int, str]:
         messages = [
             {"role": "user", "content": prompt},
         ]
+        for plugin in self.cfg.plugins:
+            if not plugin.can_handle_pre_instruction():
+                continue
+            plugin_messages = plugin.pre_instruction(messages)
+            if plugin_messages:
+                for plugin_message in plugin_messages:
+                    messages.append({"role": "system", "content": plugin_message})
 
         # Start GPT instance
         agent_reply = create_chat_completion(
@@ -36,9 +44,19 @@ def create_agent(self, task: str, prompt: str, model: str) -> tuple[int, str]:
             messages=messages,
         )
 
-        # Update full message history
         messages.append({"role": "assistant", "content": agent_reply})
 
+        plugins_reply = ""
+        for i, plugin in enumerate(self.cfg.plugins):
+            if not plugin.can_handle_on_instruction():
+                continue
+            plugin_result = plugin.on_instruction(messages)
+            if plugin_result:
+                sep = "" if not i else "\n"
+                plugins_reply = f"{plugins_reply}{sep}{plugin_result}"
+
+        if plugins_reply and plugins_reply != "":
+            messages.append({"role": "assistant", "content": plugins_reply})
         key = self.next_key
         # This is done instead of len(agents) to make keys unique even if agents
         # are deleted
@@ -46,6 +64,11 @@ def create_agent(self, task: str, prompt: str, model: str) -> tuple[int, str]:
 
         self.agents[key] = (task, messages, model)
 
+        for plugin in self.cfg.plugins:
+            if not plugin.can_handle_post_instruction():
+                continue
+            agent_reply = plugin.post_instruction(agent_reply)
+
         return key, agent_reply
 
     def message_agent(self, key: str | int, message: str) -> str:
@@ -63,15 +86,39 @@ def message_agent(self, key: str | int, message: str) -> str:
         # Add user message to message history before sending to agent
         messages.append({"role": "user", "content": message})
 
+        for plugin in self.cfg.plugins:
+            if not plugin.can_handle_pre_instruction():
+                continue
+            plugin_messages = plugin.pre_instruction(messages)
+            if plugin_messages:
+                for plugin_message in plugin_messages:
+                    messages.append({"role": "system", "content": plugin_message})
+
         # Start GPT instance
         agent_reply = create_chat_completion(
             model=model,
             messages=messages,
         )
 
-        # Update full message history
         messages.append({"role": "assistant", "content": agent_reply})
 
+        plugins_reply = agent_reply
+        for i, plugin in enumerate(self.cfg.plugins):
+            if not plugin.can_handle_on_instruction():
+                continue
+            plugin_result = plugin.on_instruction(messages)
+            if plugin_result:
+                sep = "" if not i else "\n"
+                plugins_reply = f"{plugins_reply}{sep}{plugin_result}"
+        # Update full message history
+        if plugins_reply and plugins_reply != "":
+            messages.append({"role": "assistant", "content": plugins_reply})
+
+        for plugin in self.cfg.plugins:
+            if not plugin.can_handle_post_instruction():
+                continue
+            agent_reply = plugin.post_instruction(agent_reply)
+
         return agent_reply
 
     def list_agents(self) -> list[tuple[str | int, str]]:
@@ -84,7 +131,7 @@ def list_agents(self) -> list[tuple[str | int, str]]:
         # Return a list of agent keys and their tasks
         return [(key, task) for key, (task, _, _) in self.agents.items()]
 
-    def delete_agent(self, key: Union[str, int]) -> bool:
+    def delete_agent(self, key: str | int) -> bool:
         """Delete an agent from the agent manager
 
         Args:
diff --git a/autogpt/app.py b/autogpt/app.py
index 19c075f0b09a..38c49e426168 100644
--- a/autogpt/app.py
+++ b/autogpt/app.py
@@ -2,14 +2,9 @@
 import json
 from typing import List, NoReturn, Union, Dict
 from autogpt.agent.agent_manager import AgentManager
-from autogpt.commands.evaluate_code import evaluate_code
-from autogpt.commands.google_search import google_official_search, google_search
-from autogpt.commands.improve_code import improve_code
-from autogpt.commands.write_tests import write_tests
-from autogpt.config import Config
-from autogpt.commands.image_gen import generate_image
 from autogpt.commands.audio_text import read_audio_from_file
-from autogpt.commands.web_requests import scrape_links, scrape_text
+from autogpt.commands.command import CommandRegistry, command
+from autogpt.commands.evaluate_code import evaluate_code
 from autogpt.commands.execute_code import execute_python_file, execute_shell
 from autogpt.commands.file_operations import (
     append_to_file,
@@ -17,16 +12,22 @@
     read_file,
     search_files,
     write_to_file,
-    download_file
+    download_file,
 )
+from autogpt.commands.git_operations import clone_repository
+from autogpt.commands.google_search import google_official_search, google_search
+from autogpt.commands.image_gen import generate_image
+from autogpt.commands.improve_code import improve_code
+from autogpt.commands.twitter import send_tweet
+from autogpt.commands.web_requests import scrape_links, scrape_text
+from autogpt.commands.web_selenium import browse_website
+from autogpt.commands.write_tests import write_tests
+from autogpt.config import Config
 from autogpt.json_fixes.parsing import fix_and_parse_json
 from autogpt.memory import get_memory
 from autogpt.processing.text import summarize_text
+from autogpt.prompts.generator import PromptGenerator
 from autogpt.speech import say_text
-from autogpt.commands.web_selenium import browse_website
-from autogpt.commands.git_operations import clone_repository
-from autogpt.commands.twitter import send_tweet
-
 
 CFG = Config()
 AGENT_MANAGER = AgentManager()
@@ -104,7 +105,12 @@ def map_command_synonyms(command_name: str):
     return command_name
 
 
-def execute_command(command_name: str, arguments):
+def execute_command(
+    command_registry: CommandRegistry,
+    command_name: str,
+    arguments,
+    prompt: PromptGenerator,
+):
     """Execute the command and return the result
 
     Args:
@@ -116,92 +122,37 @@ def execute_command(command_name: str, arguments):
     memory = get_memory(CFG)
 
     try:
+        cmd = command_registry.commands.get(command_name)
+
+        # If the command is found, call it with the provided arguments
+        if cmd:
+            return cmd(**arguments)
+
+        # TODO: Remove commands below after they are moved to the command registry.
         command_name = map_command_synonyms(command_name)
-        if command_name == "google":
-            # Check if the Google API key is set and use the official search method
-            # If the API key is not set or has only whitespaces, use the unofficial
-            # search method
-            key = CFG.google_api_key
-            if key and key.strip() and key != "your-google-api-key":
-                google_result = google_official_search(arguments["input"])
-                return google_result
-            else:
-                google_result = google_search(arguments["input"])
-
-            # google_result can be a list or a string depending on the search results
-            if isinstance(google_result, list):
-                safe_message = [google_result_single.encode('utf-8', 'ignore') for google_result_single in google_result]
-            else:
-                safe_message = google_result.encode('utf-8', 'ignore')
-
-            return str(safe_message)
-        elif command_name == "memory_add":
+
+        if command_name == "memory_add":
             return memory.add(arguments["string"])
-        elif command_name == "start_agent":
-            return start_agent(
-                arguments["name"], arguments["task"], arguments["prompt"]
-            )
-        elif command_name == "message_agent":
-            return message_agent(arguments["key"], arguments["message"])
-        elif command_name == "list_agents":
-            return list_agents()
-        elif command_name == "delete_agent":
-            return delete_agent(arguments["key"])
         elif command_name == "get_text_summary":
             return get_text_summary(arguments["url"], arguments["question"])
         elif command_name == "get_hyperlinks":
             return get_hyperlinks(arguments["url"])
-        elif command_name == "clone_repository":
-            return clone_repository(
-                arguments["repository_url"], arguments["clone_path"]
-            )
-        elif command_name == "read_file":
-            return read_file(arguments["file"])
-        elif command_name == "write_to_file":
-            return write_to_file(arguments["file"], arguments["text"])
-        elif command_name == "append_to_file":
-            return append_to_file(arguments["file"], arguments["text"])
-        elif command_name == "delete_file":
-            return delete_file(arguments["file"])
-        elif command_name == "search_files":
-            return search_files(arguments["directory"])
         elif command_name == "download_file":
             if not CFG.allow_downloads:
                 return "Error: You do not have user authorization to download files locally."
             return download_file(arguments["url"], arguments["file"])
-        elif command_name == "browse_website":
-            return browse_website(arguments["url"], arguments["question"])
+
         # TODO: Change these to take in a file rather than pasted code, if
         # non-file is given, return instructions "Input should be a python
-        # filepath, write your code to file and try again"
-        elif command_name == "evaluate_code":
-            return evaluate_code(arguments["code"])
-        elif command_name == "improve_code":
-            return improve_code(arguments["suggestions"], arguments["code"])
-        elif command_name == "write_tests":
-            return write_tests(arguments["code"], arguments.get("focus"))
-        elif command_name == "execute_python_file":  # Add this command
-            return execute_python_file(arguments["file"])
-        elif command_name == "execute_shell":
-            if CFG.execute_local_commands:
-                return execute_shell(arguments["command_line"])
-            else:
-                return (
-                    "You are not allowed to run local shell commands. To execute"
-                    " shell commands, EXECUTE_LOCAL_COMMANDS must be set to 'True' "
-                    "in your config. Do not attempt to bypass the restriction."
-                )
-        elif command_name == "read_audio_from_file":
-            return read_audio_from_file(arguments["file"])
-        elif command_name == "generate_image":
-            return generate_image(arguments["prompt"])
-        elif command_name == "send_tweet":
-            return send_tweet(arguments["text"])
+        # filepath, write your code to file and try again
         elif command_name == "do_nothing":
             return "No action performed."
         elif command_name == "task_complete":
             shutdown()
         else:
+            for command in prompt.commands:
+                if command_name == command["label"] or command_name == command["name"]:
+                    return command["function"](*arguments.values())
             return (
                 f"Unknown command '{command_name}'. Please refer to the 'COMMANDS'"
                 " list for available commands and only respond in the specified JSON"
@@ -244,6 +195,11 @@ def shutdown() -> NoReturn:
     quit()
 
 
+@command(
+    "start_agent",
+    "Start GPT Agent",
+    '"name": "<name>", "task": "<short_task_desc>", "prompt": "<prompt>"',
+)
 def start_agent(name: str, task: str, prompt: str, model=CFG.fast_llm_model) -> str:
     """Start an agent with a given name, task, and prompt
 
@@ -276,6 +232,7 @@ def start_agent(name: str, task: str, prompt: str, model=CFG.fast_llm_model) ->
     return f"Agent {name} created with key {key}. First response: {agent_response}"
 
 
+@command("message_agent", "Message GPT Agent", '"key": "<key>", "message": "<message>"')
 def message_agent(key: str, message: str) -> str:
     """Message an agent with a given key and message"""
     # Check if the key is a valid integer
@@ -290,7 +247,8 @@ def message_agent(key: str, message: str) -> str:
     return agent_response
 
 
-def list_agents():
+@command("list_agents", "List GPT Agents", "")
+def list_agents() -> str:
     """List all agents
 
     Returns:
@@ -301,6 +259,7 @@ def list_agents():
     )
 
 
+@command("delete_agent", "Delete GPT Agent", '"key": "<key>"')
 def delete_agent(key: str) -> str:
     """Delete an agent with a given key
 
diff --git a/autogpt/args.py b/autogpt/args.py
index f0e9c07a362a..0e6eddfdb24c 100644
--- a/autogpt/args.py
+++ b/autogpt/args.py
@@ -64,10 +64,10 @@ def parse_arguments() -> None:
         " skip the re-prompt.",
     )
     parser.add_argument(
-        '--allow-downloads',
-        action='store_true',
-        dest='allow_downloads',
-        help='Dangerous: Allows Auto-GPT to download files natively.'
+        "--allow-downloads",
+        action="store_true",
+        dest="allow_downloads",
+        help="Dangerous: Allows Auto-GPT to download files natively.",
     )
     args = parser.parse_args()
 
@@ -141,10 +141,17 @@ def parse_arguments() -> None:
 
     if args.allow_downloads:
         logger.typewriter_log("Native Downloading:", Fore.GREEN, "ENABLED")
-        logger.typewriter_log("WARNING: ", Fore.YELLOW,
-                              f"{Back.LIGHTYELLOW_EX}Auto-GPT will now be able to download and save files to your machine.{Back.RESET} " +
-                              "It is recommended that you monitor any files it downloads carefully.")
-        logger.typewriter_log("WARNING: ", Fore.YELLOW, f"{Back.RED + Style.BRIGHT}ALWAYS REMEMBER TO NEVER OPEN FILES YOU AREN'T SURE OF!{Style.RESET_ALL}")
+        logger.typewriter_log(
+            "WARNING: ",
+            Fore.YELLOW,
+            f"{Back.LIGHTYELLOW_EX}Auto-GPT will now be able to download and save files to your machine.{Back.RESET} "
+            + "It is recommended that you monitor any files it downloads carefully.",
+        )
+        logger.typewriter_log(
+            "WARNING: ",
+            Fore.YELLOW,
+            f"{Back.RED + Style.BRIGHT}ALWAYS REMEMBER TO NEVER OPEN FILES YOU AREN'T SURE OF!{Style.RESET_ALL}",
+        )
         CFG.allow_downloads = True
 
     if args.browser_name:
diff --git a/autogpt/chat.py b/autogpt/chat.py
index 1f6bca96eb21..22fe636cb782 100644
--- a/autogpt/chat.py
+++ b/autogpt/chat.py
@@ -51,7 +51,7 @@ def generate_context(prompt, relevant_memory, full_message_history, model):
 
 # TODO: Change debug from hardcode to argument
 def chat_with_ai(
-    prompt, user_input, full_message_history, permanent_memory, token_limit
+    agent, prompt, user_input, full_message_history, permanent_memory, token_limit
 ):
     """Interact with the OpenAI API, sending the prompt, user input, message history,
     and permanent memory."""
@@ -135,6 +135,25 @@ def chat_with_ai(
             # Append user input, the length of this is accounted for above
             current_context.extend([create_chat_message("user", user_input)])
 
+            plugin_count = len(cfg.plugins)
+            for i, plugin in enumerate(cfg.plugins):
+                if not plugin.can_handle_on_planning():
+                    continue
+                plugin_response = plugin.on_planning(
+                    agent.prompt_generator, current_context
+                )
+                if not plugin_response or plugin_response == "":
+                    continue
+                tokens_to_add = token_counter.count_message_tokens(
+                    [plugin_response], model
+                )
+                if current_tokens_used + tokens_to_add > send_token_limit:
+                    if cfg.debug_mode:
+                        print("Plugin response too long, skipping:", plugin_response)
+                        print("Plugins remaining at stop:", plugin_count - i)
+                    break
+                current_context.append(create_chat_message("system", plugin_response))
+
             # Calculate remaining tokens
             tokens_remaining = token_limit - current_tokens_used
             # assert tokens_remaining >= 0, "Tokens remaining is negative.
diff --git a/autogpt/commands/audio_text.py b/autogpt/commands/audio_text.py
index 84819d5ed75a..421a1f18eec5 100644
--- a/autogpt/commands/audio_text.py
+++ b/autogpt/commands/audio_text.py
@@ -1,23 +1,51 @@
-import requests
+"""Commands for converting audio to text."""
 import json
 
+import requests
+
+from autogpt.commands.command import command
 from autogpt.config import Config
 from autogpt.workspace import path_in_workspace
 
-cfg = Config()
+CFG = Config()
+
 
+@command(
+    "read_audio_from_file",
+    "Convert Audio to text",
+    '"file": "<file>"',
+    CFG.huggingface_audio_to_text_model,
+    "Configure huggingface_audio_to_text_model.",
+)
+def read_audio_from_file(audio_path: str) -> str:
+    """
+    Convert audio to text.
 
-def read_audio_from_file(audio_path):
+    Args:
+        audio_path (str): The path to the audio file
+
+    Returns:
+        str: The text from the audio
+    """
     audio_path = path_in_workspace(audio_path)
     with open(audio_path, "rb") as audio_file:
         audio = audio_file.read()
     return read_audio(audio)
 
 
-def read_audio(audio):
-    model = cfg.huggingface_audio_to_text_model
+def read_audio(audio: bytes) -> str:
+    """
+    Convert audio to text.
+
+    Args:
+        audio (bytes): The audio to convert
+
+    Returns:
+        str: The text from the audio
+    """
+    model = CFG.huggingface_audio_to_text_model
     api_url = f"https://api-inference.huggingface.co/models/{model}"
-    api_token = cfg.huggingface_api_token
+    api_token = CFG.huggingface_api_token
     headers = {"Authorization": f"Bearer {api_token}"}
 
     if api_token is None:
@@ -32,4 +60,4 @@ def read_audio(audio):
     )
 
     text = json.loads(response.content.decode("utf-8"))["text"]
-    return "The audio says: " + text
+    return f"The audio says: {text}"
diff --git a/autogpt/commands/command.py b/autogpt/commands/command.py
new file mode 100644
index 000000000000..f21b1b526c80
--- /dev/null
+++ b/autogpt/commands/command.py
@@ -0,0 +1,153 @@
+import importlib
+import inspect
+from typing import Any, Callable, Optional
+
+# Unique identifier for auto-gpt commands
+AUTO_GPT_COMMAND_IDENTIFIER = "auto_gpt_command"
+
+
+class Command:
+    """A class representing a command.
+
+    Attributes:
+        name (str): The name of the command.
+        description (str): A brief description of what the command does.
+        signature (str): The signature of the function that the command executes. Defaults to None.
+    """
+
+    def __init__(
+        self,
+        name: str,
+        description: str,
+        method: Callable[..., Any],
+        signature: str = None,
+        enabled: bool = True,
+        disabled_reason: Optional[str] = None,
+    ):
+        self.name = name
+        self.description = description
+        self.method = method
+        self.signature = signature if signature else str(inspect.signature(self.method))
+        self.enabled = enabled
+        self.disabled_reason = disabled_reason
+
+    def __call__(self, *args, **kwargs) -> Any:
+        if not self.enabled:
+            return f"Command '{self.name}' is disabled: {self.disabled_reason}"
+        return self.method(*args, **kwargs)
+
+    def __str__(self) -> str:
+        return f"{self.name}: {self.description}, args: {self.signature}"
+
+
+class CommandRegistry:
+    """
+    The CommandRegistry class is a manager for a collection of Command objects.
+    It allows the registration, modification, and retrieval of Command objects,
+    as well as the scanning and loading of command plugins from a specified
+    directory.
+    """
+
+    def __init__(self):
+        self.commands = {}
+
+    def _import_module(self, module_name: str) -> Any:
+        return importlib.import_module(module_name)
+
+    def _reload_module(self, module: Any) -> Any:
+        return importlib.reload(module)
+
+    def register(self, cmd: Command) -> None:
+        self.commands[cmd.name] = cmd
+
+    def unregister(self, command_name: str):
+        if command_name in self.commands:
+            del self.commands[command_name]
+        else:
+            raise KeyError(f"Command '{command_name}' not found in registry.")
+
+    def reload_commands(self) -> None:
+        """Reloads all loaded command plugins."""
+        for cmd_name in self.commands:
+            cmd = self.commands[cmd_name]
+            module = self._import_module(cmd.__module__)
+            reloaded_module = self._reload_module(module)
+            if hasattr(reloaded_module, "register"):
+                reloaded_module.register(self)
+
+    def get_command(self, name: str) -> Callable[..., Any]:
+        return self.commands[name]
+
+    def call(self, command_name: str, **kwargs) -> Any:
+        if command_name not in self.commands:
+            raise KeyError(f"Command '{command_name}' not found in registry.")
+        command = self.commands[command_name]
+        return command(**kwargs)
+
+    def command_prompt(self) -> str:
+        """
+        Returns a string representation of all registered `Command` objects for use in a prompt
+        """
+        commands_list = [
+            f"{idx + 1}. {str(cmd)}" for idx, cmd in enumerate(self.commands.values())
+        ]
+        return "\n".join(commands_list)
+
+    def import_commands(self, module_name: str) -> None:
+        """
+        Imports the specified Python module containing command plugins.
+
+        This method imports the associated module and registers any functions or
+        classes that are decorated with the `AUTO_GPT_COMMAND_IDENTIFIER` attribute
+        as `Command` objects. The registered `Command` objects are then added to the
+        `commands` dictionary of the `CommandRegistry` object.
+
+        Args:
+            module_name (str): The name of the module to import for command plugins.
+        """
+
+        module = importlib.import_module(module_name)
+
+        for attr_name in dir(module):
+            attr = getattr(module, attr_name)
+            # Register decorated functions
+            if hasattr(attr, AUTO_GPT_COMMAND_IDENTIFIER) and getattr(
+                attr, AUTO_GPT_COMMAND_IDENTIFIER
+            ):
+                self.register(attr.command)
+            # Register command classes
+            elif (
+                inspect.isclass(attr) and issubclass(attr, Command) and attr != Command
+            ):
+                cmd_instance = attr()
+                self.register(cmd_instance)
+
+
+def command(
+    name: str,
+    description: str,
+    signature: str = None,
+    enabled: bool = True,
+    disabled_reason: Optional[str] = None,
+) -> Callable[..., Any]:
+    """The command decorator is used to create Command objects from ordinary functions."""
+
+    def decorator(func: Callable[..., Any]) -> Command:
+        cmd = Command(
+            name=name,
+            description=description,
+            method=func,
+            signature=signature,
+            enabled=enabled,
+            disabled_reason=disabled_reason,
+        )
+
+        def wrapper(*args, **kwargs) -> Any:
+            return func(*args, **kwargs)
+
+        wrapper.command = cmd
+
+        setattr(wrapper, AUTO_GPT_COMMAND_IDENTIFIER, True)
+        return wrapper
+
+    return decorator
diff --git a/autogpt/commands/evaluate_code.py b/autogpt/commands/evaluate_code.py
index 8f7cbca9c1bf..064e4512be7c 100644
--- a/autogpt/commands/evaluate_code.py
+++ b/autogpt/commands/evaluate_code.py
@@ -1,9 +1,11 @@
 """Code evaluation module."""
 from __future__ import annotations
 
+from autogpt.commands.command import command
 from autogpt.llm_utils import call_ai_function
 
 
+@command("evaluate_code", "Evaluate Code", '"code": "<full_code_string>"')
 def evaluate_code(code: str) -> list[str]:
     """
     A function that takes in a string and returns a response from create chat
diff --git a/autogpt/commands/execute_code.py b/autogpt/commands/execute_code.py
index eaafa00a828a..b67707279a47 100644
--- a/autogpt/commands/execute_code.py
+++ b/autogpt/commands/execute_code.py
@@ -5,9 +5,14 @@
 import docker
 from docker.errors import ImageNotFound
 
-from autogpt.workspace import path_in_workspace, WORKSPACE_PATH
+from autogpt.commands.command import command
+from autogpt.config import Config
+from autogpt.workspace import WORKSPACE_PATH, path_in_workspace
 
+CFG = Config()
 
+
+@command("execute_python_file", "Execute Python File", '"file": "<file>"')
 def execute_python_file(file: str):
     """Execute a Python file in a Docker container and return the output
 
@@ -88,6 +93,15 @@ def execute_python_file(file: str):
         return f"Error: {str(e)}"
 
 
+@command(
+    "execute_shell",
+    "Execute Shell Command, non-interactive commands only",
+    '"command_line": "<command_line>"',
+    CFG.execute_local_commands,
+    "You are not allowed to run local shell commands. To execute"
+    " shell commands, EXECUTE_LOCAL_COMMANDS must be set to 'True' "
+    "in your config. Do not attempt to bypass the restriction.",
+)
 def execute_shell(command_line: str) -> str:
     """Execute a shell command and return the output
 
@@ -97,6 +111,13 @@ def execute_shell(command_line: str) -> str:
     Returns:
         str: The output of the command
     """
+
+    if not CFG.execute_local_commands:
+        return (
+            "You are not allowed to run local shell commands. To execute"
+            " shell commands, EXECUTE_LOCAL_COMMANDS must be set to 'True' "
+            "in your config. Do not attempt to bypass the restriction."
+        )
     current_dir = os.getcwd()
     # Change dir into workspace if necessary
     if str(WORKSPACE_PATH) not in current_dir:
diff --git a/autogpt/commands/file_operations.py b/autogpt/commands/file_operations.py
index d273c1a34ddd..9011dc3b2ee2 100644
--- a/autogpt/commands/file_operations.py
+++ b/autogpt/commands/file_operations.py
@@ -11,7 +11,9 @@
 from colorama import Fore, Back
 from autogpt.spinner import Spinner
 from autogpt.utils import readable_file_size
-from autogpt.workspace import path_in_workspace, WORKSPACE_PATH
+
+from autogpt.commands.command import command
+from autogpt.workspace import WORKSPACE_PATH, path_in_workspace
 
 
 LOG_FILE = "file_logger.txt"
@@ -47,7 +49,7 @@ def log_operation(operation: str, filename: str) -> None:
         with open(LOG_FILE_PATH, "w", encoding="utf-8") as f:
             f.write("File Operation Logger ")
 
-    append_to_file(LOG_FILE, log_entry, shouldLog = False)
+    append_to_file(LOG_FILE, log_entry, shouldLog=False)
 
 
 def split_file(
@@ -77,6 +79,7 @@ def split_file(
         start += max_length - overlap
 
 
+@command("read_file", "Read file", '"file": "<file>"')
 def read_file(filename: str) -> str:
     """Read a file and return the contents
 
@@ -129,6 +132,7 @@ def ingest_file(
         print(f"Error while ingesting file '{filename}': {str(e)}")
 
 
+@command("write_to_file", "Write to file", '"file": "<file>", "text": "<text>"')
 def write_to_file(filename: str, text: str) -> str:
     """Write text to a file
 
@@ -154,6 +158,7 @@ def write_to_file(filename: str, text: str) -> str:
         return f"Error: {str(e)}"
 
 
+@command("append_to_file", "Append to file", '"file": "<file>", "text": "<text>"')
 def append_to_file(filename: str, text: str, shouldLog: bool = True) -> str:
     """Append text to a file
 
@@ -177,6 +182,7 @@ def append_to_file(filename: str, text: str, shouldLog: bool = True) -> str:
         return f"Error: {str(e)}"
 
 
+@command("delete_file", "Delete file", '"file": "<file>"')
 def delete_file(filename: str) -> str:
     """Delete a file
 
@@ -197,6 +203,7 @@ def delete_file(filename: str) -> str:
         return f"Error: {str(e)}"
 
 
+@command("search_files", "Search Files", '"directory": "<directory>"')
 def search_files(directory: str) -> list[str]:
     """Search for files in a directory
 
@@ -236,23 +243,23 @@ def download_file(url, filename):
             session = requests.Session()
             retry = Retry(total=3, backoff_factor=1, status_forcelist=[502, 503, 504])
             adapter = HTTPAdapter(max_retries=retry)
-            session.mount('http://', adapter)
-            session.mount('https://', adapter)
+            session.mount("http://", adapter)
+            session.mount("https://", adapter)
 
             total_size = 0
             downloaded_size = 0
 
             with session.get(url, allow_redirects=True, stream=True) as r:
                 r.raise_for_status()
-                total_size = int(r.headers.get('Content-Length', 0))
+                total_size = int(r.headers.get("Content-Length", 0))
                 downloaded_size = 0
 
-                with open(safe_filename, 'wb') as f:
+                with open(safe_filename, "wb") as f:
                     for chunk in r.iter_content(chunk_size=8192):
                         f.write(chunk)
                         downloaded_size += len(chunk)
 
-                         # Update the progress message
+                        # Update the progress message
                         progress = f"{readable_file_size(downloaded_size)} / {readable_file_size(total_size)}"
                         spinner.update_message(f"{message} {progress}")
 
diff --git a/autogpt/commands/git_operations.py b/autogpt/commands/git_operations.py
index 675eb2283ba4..480d3a6ef217 100644
--- a/autogpt/commands/git_operations.py
+++ b/autogpt/commands/git_operations.py
@@ -1,11 +1,20 @@
 """Git operations for autogpt"""
-import git
+from git.repo import Repo
+
+from autogpt.commands.command import command
 from autogpt.config import Config
 from autogpt.workspace import path_in_workspace
 
 CFG = Config()
 
 
+@command(
+    "clone_repository",
+    "Clone Repositoryy",
+    '"repository_url": "<url>", "clone_path": "<directory>"',
+    CFG.github_username and CFG.github_api_key,
+    "Configure github_username and github_api_key.",
+)
 def clone_repository(repo_url: str, clone_path: str) -> str:
     """Clone a github repository locally
 
@@ -19,7 +28,7 @@ def clone_repository(repo_url: str, clone_path: str) -> str:
     auth_repo_url = f"//{CFG.github_username}:{CFG.github_api_key}@".join(split_url)
     safe_clone_path = path_in_workspace(clone_path)
     try:
-        git.Repo.clone_from(auth_repo_url, safe_clone_path)
+        Repo.clone_from(auth_repo_url, safe_clone_path)
         return f"""Cloned {repo_url} to {safe_clone_path}"""
     except Exception as e:
         return f"Error: {str(e)}"
diff --git a/autogpt/commands/google_search.py b/autogpt/commands/google_search.py
index 148ba1d0e1cf..f549ae8f6e1f 100644
--- a/autogpt/commands/google_search.py
+++ b/autogpt/commands/google_search.py
@@ -5,11 +5,13 @@
 
 from duckduckgo_search import ddg
 
+from autogpt.commands.command import command
 from autogpt.config import Config
 
 CFG = Config()
 
 
+@command("google", "Google Search", '"query": "<search>"', not CFG.google_api_key)
 def google_search(query: str, num_results: int = 8) -> str:
     """Return the results of a google search
 
@@ -31,9 +33,17 @@ def google_search(query: str, num_results: int = 8) -> str:
     for j in results:
         search_results.append(j)
 
-    return json.dumps(search_results, ensure_ascii=False, indent=4)
+    results = json.dumps(search_results, ensure_ascii=False, indent=4)
+    return safe_google_results(results)
 
 
+@command(
+    "google",
+    "Google Search",
+    '"query": "<search>"',
+    bool(CFG.google_api_key),
+    "Configure google_api_key.",
+)
 def google_official_search(query: str, num_results: int = 8) -> str | list[str]:
     """Return the results of a google search using the official Google API
 
@@ -82,6 +92,26 @@ def google_official_search(query: str, num_results: int = 8) -> str | list[str]:
             return "Error: The provided Google API key is invalid or missing."
         else:
             return f"Error: {e}"
+    # google_result can be a list or a string depending on the search results
 
     # Return the list of search result URLs
-    return search_results_links
+    return safe_google_results(search_results_links)
+
+
+def safe_google_results(results: str | list) -> str:
+    """
+        Return the results of a google search in a safe format.
+
+    Args:
+        results (str | list): The search results.
+
+    Returns:
+        str: The results of the search.
+    """
+    if isinstance(results, list):
+        safe_message = json.dumps(
+            [result.enocde("utf-8", "ignore") for result in results]
+        )
+    else:
+        safe_message = results.encode("utf-8", "ignore").decode("utf-8")
+    return safe_message
diff --git a/autogpt/commands/image_gen.py b/autogpt/commands/image_gen.py
index 6243616ea8ca..f82e97aba6a0 100644
--- a/autogpt/commands/image_gen.py
+++ b/autogpt/commands/image_gen.py
@@ -1,18 +1,20 @@
 """ Image Generation Module for AutoGPT."""
 import io
-import os.path
 import uuid
 from base64 import b64decode
 
 import openai
 import requests
 from PIL import Image
+
+from autogpt.commands.command import command
 from autogpt.config import Config
 from autogpt.workspace import path_in_workspace
 
 CFG = Config()
 
 
+@command("generate_image", "Generate Image", '"prompt": "<prompt>"', CFG.image_provider)
 def generate_image(prompt: str) -> str:
     """Generate an image from a prompt.
 
diff --git a/autogpt/commands/improve_code.py b/autogpt/commands/improve_code.py
index e3440d8b7c6e..41a369b4db74 100644
--- a/autogpt/commands/improve_code.py
+++ b/autogpt/commands/improve_code.py
@@ -2,9 +2,15 @@
 
 import json
 
+from autogpt.commands.command import command
 from autogpt.llm_utils import call_ai_function
 
 
+@command(
+    "improve_code",
+    "Get Improved Code",
+    '"suggestions": "<list_of_suggestions>", "code": "<full_code_string>"',
+)
 def improve_code(suggestions: list[str], code: str) -> str:
     """
     A function that takes in code and suggestions and returns a response from create
diff --git a/autogpt/commands/twitter.py b/autogpt/commands/twitter.py
index dc4d450c1fad..8e64b21357b5 100644
--- a/autogpt/commands/twitter.py
+++ b/autogpt/commands/twitter.py
@@ -1,11 +1,30 @@
-import tweepy
+"""A module that contains a command to send a tweet."""
 import os
+
+import tweepy
 from dotenv import load_dotenv
 
+from autogpt.commands.command import command
+
 load_dotenv()
 
 
-def send_tweet(tweet_text):
+@command(
+    "send_tweet",
+    "Send Tweet",
+    '"text": "<text>"',
+)
+def send_tweet(tweet_text: str) -> str:
+    """
+      A function that takes in a string and returns a response from create chat
+        completion api call.
+
+    Args:
+      tweet_text (str): Text to be tweeted.
+
+      Returns:
+          A result from sending the tweet.
+    """
     consumer_key = os.environ.get("TW_CONSUMER_KEY")
     consumer_secret = os.environ.get("TW_CONSUMER_SECRET")
     access_token = os.environ.get("TW_ACCESS_TOKEN")
@@ -20,6 +39,6 @@ def send_tweet(tweet_text):
     # Send tweet
     try:
         api.update_status(tweet_text)
-        print("Tweet sent successfully!")
+        return "Tweet sent successfully!"
     except tweepy.TweepyException as e:
-        print("Error sending tweet: {}".format(e.reason))
+        return f"Error sending tweet: {e.reason}"
diff --git a/autogpt/commands/web_playwright.py b/autogpt/commands/web_playwright.py
index a1abb6cb73d2..4e388ded203c 100644
--- a/autogpt/commands/web_playwright.py
+++ b/autogpt/commands/web_playwright.py
@@ -8,6 +8,7 @@
         "Playwright not installed. Please install it with 'pip install playwright' to use."
     )
 from bs4 import BeautifulSoup
+
 from autogpt.processing.html import extract_hyperlinks, format_hyperlinks
 
 
diff --git a/autogpt/commands/web_requests.py b/autogpt/commands/web_requests.py
index 50d8d383cb1a..7613e5bd1d8d 100644
--- a/autogpt/commands/web_requests.py
+++ b/autogpt/commands/web_requests.py
@@ -4,9 +4,9 @@
 from urllib.parse import urljoin, urlparse
 
 import requests
-from requests.compat import urljoin
-from requests import Response
 from bs4 import BeautifulSoup
+from requests import Response
+from requests.compat import urljoin
 
 from autogpt.config import Config
 from autogpt.memory import get_memory
diff --git a/autogpt/commands/web_selenium.py b/autogpt/commands/web_selenium.py
index 8c652294587f..f6a3bedd0404 100644
--- a/autogpt/commands/web_selenium.py
+++ b/autogpt/commands/web_selenium.py
@@ -1,27 +1,35 @@
 """Selenium web scraping module."""
 from __future__ import annotations
 
-from selenium import webdriver
-from autogpt.processing.html import extract_hyperlinks, format_hyperlinks
-import autogpt.processing.text as summary
+import logging
+from pathlib import Path
+
 from bs4 import BeautifulSoup
-from selenium.webdriver.remote.webdriver import WebDriver
+from selenium import webdriver
+from selenium.webdriver.chrome.options import Options as ChromeOptions
 from selenium.webdriver.common.by import By
-from selenium.webdriver.support.wait import WebDriverWait
+from selenium.webdriver.firefox.options import Options as FirefoxOptions
+from selenium.webdriver.remote.webdriver import WebDriver
+from selenium.webdriver.safari.options import Options as SafariOptions
 from selenium.webdriver.support import expected_conditions as EC
+from selenium.webdriver.support.wait import WebDriverWait
 from webdriver_manager.chrome import ChromeDriverManager
 from webdriver_manager.firefox import GeckoDriverManager
-from selenium.webdriver.chrome.options import Options as ChromeOptions
-from selenium.webdriver.firefox.options import Options as FirefoxOptions
-from selenium.webdriver.safari.options import Options as SafariOptions
-import logging
-from pathlib import Path
+
+from autogpt.commands.command import command
+import autogpt.processing.text as summary
 from autogpt.config import Config
+from autogpt.processing.html import extract_hyperlinks, format_hyperlinks
 
 FILE_DIR = Path(__file__).parent.parent
 CFG = Config()
 
 
+@command(
+    "browse_website",
+    "Browse Website",
+    '"url": "<url>", "question": "<what_you_want_to_find_on_website>"',
+)
 def browse_website(url: str, question: str) -> tuple[str, WebDriver]:
     """Browse a website and return the answer and links to the user
 
diff --git a/autogpt/commands/write_tests.py b/autogpt/commands/write_tests.py
index 138a1adb6f83..91cd93042920 100644
--- a/autogpt/commands/write_tests.py
+++ b/autogpt/commands/write_tests.py
@@ -2,9 +2,16 @@
 from __future__ import annotations
 
 import json
+
+from autogpt.commands.command import command
 from autogpt.llm_utils import call_ai_function
 
 
+@command(
+    "write_tests",
+    "Write Tests",
+    '"code": "<full_code_string>", "focus": "<list_of_focus_areas>"',
+)
 def write_tests(code: str, focus: list[str]) -> str:
     """
     A function that takes in code and focus topics and returns a response from create
diff --git a/autogpt/config/__init__.py b/autogpt/config/__init__.py
index ceb5566ce73b..726b6dcf3da9 100644
--- a/autogpt/config/__init__.py
+++ b/autogpt/config/__init__.py
@@ -2,7 +2,7 @@
 This module contains the configuration classes for AutoGPT.
 """
 from autogpt.config.ai_config import AIConfig
-from autogpt.config.config import check_openai_api_key, Config
+from autogpt.config.config import Config, check_openai_api_key
 from autogpt.config.singleton import AbstractSingleton, Singleton
 
 __all__ = [
diff --git a/autogpt/config/ai_config.py b/autogpt/config/ai_config.py
index 86171357ba0b..af387f0bdc93 100644
--- a/autogpt/config/ai_config.py
+++ b/autogpt/config/ai_config.py
@@ -5,9 +5,13 @@
 from __future__ import annotations
 
 import os
-from typing import Type
+from pathlib import Path
+from typing import Optional, Type
+
 import yaml
 
+from autogpt.prompts.generator import PromptGenerator
+
 
 class AIConfig:
     """
@@ -37,9 +41,11 @@ def __init__(
         self.ai_name = ai_name
         self.ai_role = ai_role
         self.ai_goals = ai_goals
+        self.prompt_generator = None
+        self.command_registry = None
 
     # Soon this will go in a folder where it remembers more stuff about the run(s)
-    SAVE_FILE = os.path.join(os.path.dirname(__file__), "..", "ai_settings.yaml")
+    SAVE_FILE = Path(os.getcwd()) / "ai_settings.yaml"
 
     @staticmethod
     def load(config_file: str = SAVE_FILE) -> "AIConfig":
@@ -88,7 +94,9 @@ def save(self, config_file: str = SAVE_FILE) -> None:
         with open(config_file, "w", encoding="utf-8") as file:
             yaml.dump(config, file, allow_unicode=True)
 
-    def construct_full_prompt(self) -> str:
+    def construct_full_prompt(
+        self, prompt_generator: Optional[PromptGenerator] = None
+    ) -> str:
         """
         Returns a prompt to the user with the class information in an organized fashion.
 
@@ -107,14 +115,25 @@ def construct_full_prompt(self) -> str:
             ""
         )
 
-        from autogpt.prompt import get_prompt
+        from autogpt.config import Config
+        from autogpt.prompts.prompt import build_default_prompt_generator
+
+        cfg = Config()
+        if prompt_generator is None:
+            prompt_generator = build_default_prompt_generator()
+        prompt_generator.goals = self.ai_goals
+        prompt_generator.name = self.ai_name
+        prompt_generator.role = self.ai_role
+        prompt_generator.command_registry = self.command_registry
+        for plugin in cfg.plugins:
+            if not plugin.can_handle_post_prompt():
+                continue
+            prompt_generator = plugin.post_prompt(prompt_generator)
 
         # Construct full prompt
-        full_prompt = (
-            f"You are {self.ai_name}, {self.ai_role}\n{prompt_start}\n\nGOALS:\n\n"
-        )
+        full_prompt = f"You are {prompt_generator.name}, {prompt_generator.role}\n{prompt_start}\n\nGOALS:\n\n"
         for i, goal in enumerate(self.ai_goals):
             full_prompt += f"{i+1}. {goal}\n"
-
-        full_prompt += f"\n\n{get_prompt()}"
+        self.prompt_generator = prompt_generator
+        full_prompt += f"\n\n{prompt_generator.generate_prompt_string()}"
         return full_prompt
diff --git a/autogpt/config/config.py b/autogpt/config/config.py
index fe6f4f325852..c12eed2effff 100644
--- a/autogpt/config/config.py
+++ b/autogpt/config/config.py
@@ -1,14 +1,13 @@
 """Configuration class to store the state of bools for different scripts access."""
 import os
-from colorama import Fore
-
-from autogpt.config.singleton import Singleton
 
 import openai
 import yaml
-
+from colorama import Fore
 from dotenv import load_dotenv
 
+from autogpt.config.singleton import Singleton
+
 load_dotenv(verbose=True)
 
 
@@ -67,7 +66,7 @@ def __init__(self) -> None:
         self.pinecone_api_key = os.getenv("PINECONE_API_KEY")
         self.pinecone_region = os.getenv("PINECONE_ENV")
 
-        self.weaviate_host  = os.getenv("WEAVIATE_HOST")
+        self.weaviate_host = os.getenv("WEAVIATE_HOST")
         self.weaviate_port = os.getenv("WEAVIATE_PORT")
         self.weaviate_protocol = os.getenv("WEAVIATE_PROTOCOL", "http")
         self.weaviate_username = os.getenv("WEAVIATE_USERNAME", None)
@@ -75,7 +74,9 @@ def __init__(self) -> None:
         self.weaviate_scopes = os.getenv("WEAVIATE_SCOPES", None)
         self.weaviate_embedded_path = os.getenv("WEAVIATE_EMBEDDED_PATH")
         self.weaviate_api_key = os.getenv("WEAVIATE_API_KEY", None)
-        self.use_weaviate_embedded = os.getenv("USE_WEAVIATE_EMBEDDED", "False") == "True"
+        self.use_weaviate_embedded = (
+            os.getenv("USE_WEAVIATE_EMBEDDED", "False") == "True"
+        )
 
         # milvus configuration, e.g., localhost:19530.
         self.milvus_addr = os.getenv("MILVUS_ADDR", "localhost:19530")
@@ -106,6 +107,10 @@ def __init__(self) -> None:
         # Initialize the OpenAI API client
         openai.api_key = self.openai_api_key
 
+        self.plugins = []
+        self.plugins_whitelist = []
+        self.plugins_blacklist = []
+
     def get_azure_deployment_id_for_model(self, model: str) -> str:
         """
         Returns the relevant deployment id for the model specified.
@@ -228,6 +233,10 @@ def set_debug_mode(self, value: bool) -> None:
         """Set the debug mode value."""
         self.debug_mode = value
 
+    def set_plugins(self, value: list) -> None:
+        """Set the plugins value."""
+        self.plugins = value
+
 
 def check_openai_api_key() -> None:
     """Check if the OpenAI API key is set in config.py or as an environment variable."""
diff --git a/autogpt/json_fixes/auto_fix.py b/autogpt/json_fixes/auto_fix.py
index 9fcf909a49a8..8ad458ab974c 100644
--- a/autogpt/json_fixes/auto_fix.py
+++ b/autogpt/json_fixes/auto_fix.py
@@ -1,9 +1,9 @@
 """This module contains the function to fix JSON strings using GPT-3."""
 import json
 
+from autogpt.config import Config
 from autogpt.llm_utils import call_ai_function
 from autogpt.logs import logger
-from autogpt.config import Config
 
 CFG = Config()
 
diff --git a/autogpt/json_fixes/master_json_fix_method.py b/autogpt/json_fixes/master_json_fix_method.py
index 7a2cf3cc81c3..135d754071ec 100644
--- a/autogpt/json_fixes/master_json_fix_method.py
+++ b/autogpt/json_fixes/master_json_fix_method.py
@@ -3,11 +3,14 @@
 from autogpt.config import Config
 from autogpt.logs import logger
 from autogpt.speech import say_text
+
 CFG = Config()
 
 
 def fix_json_using_multiple_techniques(assistant_reply: str) -> Dict[Any, Any]:
-    from autogpt.json_fixes.parsing import attempt_to_fix_json_by_finding_outermost_brackets
+    from autogpt.json_fixes.parsing import (
+        attempt_to_fix_json_by_finding_outermost_brackets,
+    )
 
     from autogpt.json_fixes.parsing import fix_and_parse_json
 
@@ -21,7 +24,10 @@ def fix_json_using_multiple_techniques(assistant_reply: str) -> Dict[Any, Any]:
     if assistant_reply_json != {}:
         return assistant_reply_json
 
-    logger.error("Error: The following AI output couldn't be converted to a JSON:\n", assistant_reply)
+    logger.error(
+        "Error: The following AI output couldn't be converted to a JSON:\n",
+        assistant_reply,
+    )
     if CFG.speak_mode:
         say_text("I have received an invalid JSON response from the OpenAI API.")
 
diff --git a/autogpt/json_validation/validate_json.py b/autogpt/json_validation/validate_json.py
index 440c3b0b9199..f6e55180f416 100644
--- a/autogpt/json_validation/validate_json.py
+++ b/autogpt/json_validation/validate_json.py
@@ -19,7 +19,9 @@ def validate_json(json_object: object, schema_name: object) -> object:
     if errors := sorted(validator.iter_errors(json_object), key=lambda e: e.path):
         logger.error("The JSON object is invalid.")
         if CFG.debug_mode:
-            logger.error(json.dumps(json_object, indent=4))   # Replace 'json_object' with the variable containing the JSON data
+            logger.error(
+                json.dumps(json_object, indent=4)
+            )  # Replace 'json_object' with the variable containing the JSON data
             logger.error("The following issues were found:")
 
             for error in errors:
diff --git a/autogpt/llm_utils.py b/autogpt/llm_utils.py
index 2075f93446eb..4fb0e1f53794 100644
--- a/autogpt/llm_utils.py
+++ b/autogpt/llm_utils.py
@@ -1,11 +1,10 @@
 from __future__ import annotations
 
-from ast import List
 import time
 
 import openai
-from openai.error import APIError, RateLimitError
 from colorama import Fore
+from openai.error import APIError, RateLimitError
 
 from autogpt.config import Config
 
@@ -76,6 +75,20 @@ def create_chat_completion(
             + f"Creating chat completion with model {model}, temperature {temperature},"
             f" max_tokens {max_tokens}" + Fore.RESET
         )
+    for plugin in CFG.plugins:
+        if plugin.can_handle_chat_completion(
+            messages=messages,
+            model=model,
+            temperature=temperature,
+            max_tokens=max_tokens,
+        ):
+            response = plugin.handle_chat_completion(
+                messages=messages,
+                model=model,
+                temperature=temperature,
+                max_tokens=max_tokens,
+            )
+            return response
     for attempt in range(num_retries):
         backoff = 2 ** (attempt + 2)
         try:
@@ -99,7 +112,7 @@ def create_chat_completion(
             if CFG.debug_mode:
                 print(
                     Fore.RED + "Error: ",
-                    f"Reached rate limit, passing..." + Fore.RESET,
+                    "Reached rate limit, passing..." + Fore.RESET,
                 )
         except APIError as e:
             if e.http_status == 502:
@@ -116,8 +129,12 @@ def create_chat_completion(
         time.sleep(backoff)
     if response is None:
         raise RuntimeError(f"Failed to get response after {num_retries} retries")
-
-    return response.choices[0].message["content"]
+    resp = response.choices[0].message["content"]
+    for plugin in CFG.plugins:
+        if not plugin.can_handle_on_response():
+            continue
+        resp = plugin.on_response(resp)
+    return resp
 
 
 def create_embedding_with_ada(text) -> list:
diff --git a/autogpt/logs.py b/autogpt/logs.py
index c1e436db97fc..df3487f2c1e2 100644
--- a/autogpt/logs.py
+++ b/autogpt/logs.py
@@ -5,13 +5,13 @@
 import random
 import re
 import time
-from logging import LogRecord
 import traceback
+from logging import LogRecord
 
 from colorama import Fore, Style
 
-from autogpt.speech import say_text
 from autogpt.config import Config, Singleton
+from autogpt.speech import say_text
 
 CFG = Config()
 
@@ -47,7 +47,7 @@ def __init__(self):
 
         # Info handler in activity.log
         self.file_handler = logging.FileHandler(
-            os.path.join(log_dir, log_file), 'a', 'utf-8'
+            os.path.join(log_dir, log_file), "a", "utf-8"
         )
         self.file_handler.setLevel(logging.DEBUG)
         info_formatter = AutoGptFormatter(
@@ -57,7 +57,7 @@ def __init__(self):
 
         # Error handler error.log
         error_handler = logging.FileHandler(
-            os.path.join(log_dir, error_file), 'a', 'utf-8'
+            os.path.join(log_dir, error_file), "a", "utf-8"
         )
         error_handler.setLevel(logging.ERROR)
         error_formatter = AutoGptFormatter(
@@ -79,7 +79,7 @@ def __init__(self):
         self.logger.setLevel(logging.DEBUG)
 
     def typewriter_log(
-            self, title="", title_color="", content="", speak_text=False, level=logging.INFO
+        self, title="", title_color="", content="", speak_text=False, level=logging.INFO
     ):
         if speak_text and CFG.speak_mode:
             say_text(f"{title}. {content}")
@@ -95,18 +95,18 @@ def typewriter_log(
         )
 
     def debug(
-            self,
-            message,
-            title="",
-            title_color="",
+        self,
+        message,
+        title="",
+        title_color="",
     ):
         self._log(title, title_color, message, logging.DEBUG)
 
     def warn(
-            self,
-            message,
-            title="",
-            title_color="",
+        self,
+        message,
+        title="",
+        title_color="",
     ):
         self._log(title, title_color, message, logging.WARN)
 
@@ -180,10 +180,10 @@ class AutoGptFormatter(logging.Formatter):
     def format(self, record: LogRecord) -> str:
         if hasattr(record, "color"):
             record.title_color = (
-                    getattr(record, "color")
-                    + getattr(record, "title")
-                    + " "
-                    + Style.RESET_ALL
+                getattr(record, "color")
+                + getattr(record, "title")
+                + " "
+                + Style.RESET_ALL
             )
         else:
             record.title_color = getattr(record, "title")
@@ -294,7 +294,9 @@ def print_assistant_thoughts(ai_name, assistant_reply):
         logger.error("Error: \n", call_stack)
 
 
-def print_assistant_thoughts(ai_name: object, assistant_reply_json_valid: object) -> None:
+def print_assistant_thoughts(
+    ai_name: object, assistant_reply_json_valid: object
+) -> None:
     assistant_thoughts_reasoning = None
     assistant_thoughts_plan = None
     assistant_thoughts_speak = None
@@ -310,9 +312,7 @@ def print_assistant_thoughts(ai_name: object, assistant_reply_json_valid: object
     logger.typewriter_log(
         f"{ai_name.upper()} THOUGHTS:", Fore.YELLOW, f"{assistant_thoughts_text}"
     )
-    logger.typewriter_log(
-        "REASONING:", Fore.YELLOW, f"{assistant_thoughts_reasoning}"
-    )
+    logger.typewriter_log("REASONING:", Fore.YELLOW, f"{assistant_thoughts_reasoning}")
     if assistant_thoughts_plan:
         logger.typewriter_log("PLAN:", Fore.YELLOW, "")
         # If it's a list, join it into a string
@@ -326,9 +326,7 @@ def print_assistant_thoughts(ai_name: object, assistant_reply_json_valid: object
         for line in lines:
             line = line.lstrip("- ")
             logger.typewriter_log("- ", Fore.GREEN, line.strip())
-    logger.typewriter_log(
-        "CRITICISM:", Fore.YELLOW, f"{assistant_thoughts_criticism}"
-    )
+    logger.typewriter_log("CRITICISM:", Fore.YELLOW, f"{assistant_thoughts_criticism}")
     # Speak the assistant's thoughts
     if CFG.speak_mode and assistant_thoughts_speak:
         say_text(assistant_thoughts_speak)
diff --git a/autogpt/memory/__init__.py b/autogpt/memory/__init__.py
index f5afb8c93d8a..3d18704c70df 100644
--- a/autogpt/memory/__init__.py
+++ b/autogpt/memory/__init__.py
@@ -60,8 +60,10 @@ def get_memory(cfg, init=False):
             memory = RedisMemory(cfg)
     elif cfg.memory_backend == "weaviate":
         if not WeaviateMemory:
-            print("Error: Weaviate is not installed. Please install weaviate-client to"
-                  " use Weaviate as a memory backend.")
+            print(
+                "Error: Weaviate is not installed. Please install weaviate-client to"
+                " use Weaviate as a memory backend."
+            )
         else:
             memory = WeaviateMemory(cfg)
     elif cfg.memory_backend == "milvus":
@@ -93,5 +95,5 @@ def get_supported_memory_backends():
     "PineconeMemory",
     "NoMemory",
     "MilvusMemory",
-    "WeaviateMemory"
+    "WeaviateMemory",
 ]
diff --git a/autogpt/memory/local.py b/autogpt/memory/local.py
index 9b911eeff9ff..b87f132105df 100644
--- a/autogpt/memory/local.py
+++ b/autogpt/memory/local.py
@@ -7,8 +7,8 @@
 import numpy as np
 import orjson
 
-from autogpt.memory.base import MemoryProviderSingleton
 from autogpt.llm_utils import create_embedding_with_ada
+from autogpt.memory.base import MemoryProviderSingleton
 
 EMBED_DIM = 1536
 SAVE_OPTIONS = orjson.OPT_SERIALIZE_NUMPY | orjson.OPT_SERIALIZE_DATACLASS
diff --git a/autogpt/memory/milvus.py b/autogpt/memory/milvus.py
index c6e7d5a372eb..93aa8b15d639 100644
--- a/autogpt/memory/milvus.py
+++ b/autogpt/memory/milvus.py
@@ -1,11 +1,5 @@
 """ Milvus memory storage provider."""
-from pymilvus import (
-    connections,
-    FieldSchema,
-    CollectionSchema,
-    DataType,
-    Collection,
-)
+from pymilvus import Collection, CollectionSchema, DataType, FieldSchema, connections
 
 from autogpt.memory.base import MemoryProviderSingleton, get_ada_embedding
 
diff --git a/autogpt/memory/no_memory.py b/autogpt/memory/no_memory.py
index 4035a657f0e6..0371e96ae89f 100644
--- a/autogpt/memory/no_memory.py
+++ b/autogpt/memory/no_memory.py
@@ -53,7 +53,7 @@ def clear(self) -> str:
         """
         return ""
 
-    def get_relevant(self, data: str, num_relevant: int = 5) ->list[Any] | None:
+    def get_relevant(self, data: str, num_relevant: int = 5) -> list[Any] | None:
         """
         Returns all the data in the memory that is relevant to the given data.
         NoMemory always returns None.
diff --git a/autogpt/memory/pinecone.py b/autogpt/memory/pinecone.py
index d781073e3fc0..27fcd62482d0 100644
--- a/autogpt/memory/pinecone.py
+++ b/autogpt/memory/pinecone.py
@@ -1,9 +1,9 @@
 import pinecone
 from colorama import Fore, Style
 
+from autogpt.llm_utils import create_embedding_with_ada
 from autogpt.logs import logger
 from autogpt.memory.base import MemoryProviderSingleton
-from autogpt.llm_utils import create_embedding_with_ada
 
 
 class PineconeMemory(MemoryProviderSingleton):
diff --git a/autogpt/memory/redismem.py b/autogpt/memory/redismem.py
index 0e8dd71d9165..082a812c5362 100644
--- a/autogpt/memory/redismem.py
+++ b/autogpt/memory/redismem.py
@@ -10,9 +10,9 @@
 from redis.commands.search.indexDefinition import IndexDefinition, IndexType
 from redis.commands.search.query import Query
 
+from autogpt.llm_utils import create_embedding_with_ada
 from autogpt.logs import logger
 from autogpt.memory.base import MemoryProviderSingleton
-from autogpt.llm_utils import create_embedding_with_ada
 
 SCHEMA = [
     TextField("data"),
diff --git a/autogpt/memory/weaviate.py b/autogpt/memory/weaviate.py
index 6fcce0a0216d..ef0e34760e07 100644
--- a/autogpt/memory/weaviate.py
+++ b/autogpt/memory/weaviate.py
@@ -1,11 +1,13 @@
-from autogpt.config import Config
-from autogpt.memory.base import MemoryProviderSingleton, get_ada_embedding
 import uuid
+
 import weaviate
 from weaviate import Client
 from weaviate.embedded import EmbeddedOptions
 from weaviate.util import generate_uuid5
 
+from autogpt.config import Config
+from autogpt.memory.base import MemoryProviderSingleton, get_ada_embedding
+
 
 def default_schema(weaviate_index):
     return {
@@ -14,7 +16,7 @@ def default_schema(weaviate_index):
             {
                 "name": "raw_text",
                 "dataType": ["text"],
-                "description": "original text for the embedding"
+                "description": "original text for the embedding",
             }
         ],
     }
@@ -24,16 +26,20 @@ class WeaviateMemory(MemoryProviderSingleton):
     def __init__(self, cfg):
         auth_credentials = self._build_auth_credentials(cfg)
 
-        url = f'{cfg.weaviate_protocol}://{cfg.weaviate_host}:{cfg.weaviate_port}'
+        url = f"{cfg.weaviate_protocol}://{cfg.weaviate_host}:{cfg.weaviate_port}"
 
         if cfg.use_weaviate_embedded:
-            self.client = Client(embedded_options=EmbeddedOptions(
-                hostname=cfg.weaviate_host,
-                port=int(cfg.weaviate_port),
-                persistence_data_path=cfg.weaviate_embedded_path
-            ))
+            self.client = Client(
+                embedded_options=EmbeddedOptions(
+                    hostname=cfg.weaviate_host,
+                    port=int(cfg.weaviate_port),
+                    persistence_data_path=cfg.weaviate_embedded_path,
+                )
+            )
 
-            print(f"Weaviate Embedded running on: {url} with persistence path: {cfg.weaviate_embedded_path}")
+            print(
+                f"Weaviate Embedded running on: {url} with persistence path: {cfg.weaviate_embedded_path}"
+            )
         else:
             self.client = Client(url, auth_client_secret=auth_credentials)
 
@@ -47,7 +53,9 @@ def _create_schema(self):
 
     def _build_auth_credentials(self, cfg):
         if cfg.weaviate_username and cfg.weaviate_password:
-            return weaviate.AuthClientPassword(cfg.weaviate_username, cfg.weaviate_password)
+            return weaviate.AuthClientPassword(
+                cfg.weaviate_username, cfg.weaviate_password
+            )
         if cfg.weaviate_api_key:
             return weaviate.AuthApiKey(api_key=cfg.weaviate_api_key)
         else:
@@ -57,16 +65,14 @@ def add(self, data):
         vector = get_ada_embedding(data)
 
         doc_uuid = generate_uuid5(data, self.index)
-        data_object = {
-            'raw_text': data
-        }
+        data_object = {"raw_text": data}
 
         with self.client.batch as batch:
             batch.add_data_object(
                 uuid=doc_uuid,
                 data_object=data_object,
                 class_name=self.index,
-                vector=vector
+                vector=vector,
             )
 
         return f"Inserting data into memory at uuid: {doc_uuid}:\n data: {data}"
@@ -82,29 +88,31 @@ def clear(self):
         # after a call to delete_all
         self._create_schema()
 
-        return 'Obliterated'
+        return "Obliterated"
 
     def get_relevant(self, data, num_relevant=5):
         query_embedding = get_ada_embedding(data)
         try:
-            results = self.client.query.get(self.index, ['raw_text']) \
-                          .with_near_vector({'vector': query_embedding, 'certainty': 0.7}) \
-                          .with_limit(num_relevant)  \
-                          .do()
+            results = (
+                self.client.query.get(self.index, ["raw_text"])
+                .with_near_vector({"vector": query_embedding, "certainty": 0.7})
+                .with_limit(num_relevant)
+                .do()
+            )
 
-            if len(results['data']['Get'][self.index]) > 0:
-                return [str(item['raw_text']) for item in results['data']['Get'][self.index]]
+            if len(results["data"]["Get"][self.index]) > 0:
+                return [
+                    str(item["raw_text"]) for item in results["data"]["Get"][self.index]
+                ]
             else:
                 return []
 
         except Exception as err:
-            print(f'Unexpected error {err=}, {type(err)=}')
+            print(f"Unexpected error {err=}, {type(err)=}")
             return []
 
     def get_stats(self):
-        result = self.client.query.aggregate(self.index) \
-                     .with_meta_count() \
-                     .do()
-        class_data = result['data']['Aggregate'][self.index]
+        result = self.client.query.aggregate(self.index).with_meta_count().do()
+        class_data = result["data"]["Aggregate"][self.index]
 
-        return class_data[0]['meta'] if class_data else {}
+        return class_data[0]["meta"] if class_data else {}
diff --git a/autogpt/plugins.py b/autogpt/plugins.py
new file mode 100644
index 000000000000..a00b989eaebb
--- /dev/null
+++ b/autogpt/plugins.py
@@ -0,0 +1,73 @@
+"""Handles loading of plugins."""
+
+import zipfile
+from ast import Module
+from pathlib import Path
+from typing import List, Optional, Tuple
+from zipimport import zipimporter
+
+
+def inspect_zip_for_module(zip_path: str, debug: bool = False) -> Optional[str]:
+    """
+    Inspect a zipfile for a module.
+
+    Args:
+        zip_path (str): Path to the zipfile.
+        debug (bool, optional): Enable debug logging. Defaults to False.
+
+    Returns:
+        Optional[str]: The name of the module if found, else None.
+    """
+    with zipfile.ZipFile(zip_path, "r") as zfile:
+        for name in zfile.namelist():
+            if name.endswith("__init__.py"):
+                if debug:
+                    print(f"Found module '{name}' in the zipfile at: {name}")
+                return name
+    if debug:
+        print(f"Module '__init__.py' not found in the zipfile @ {zip_path}.")
+    return None
+
+
+def scan_plugins(plugins_path: Path, debug: bool = False) -> List[Tuple[str, Path]]:
+    """Scan the plugins directory for plugins.
+
+    Args:
+        plugins_path (Path): Path to the plugins directory.
+
+    Returns:
+        List[Path]: List of plugins.
+    """
+    plugins = []
+    for plugin in plugins_path.glob("*.zip"):
+        if module := inspect_zip_for_module(str(plugin), debug):
+            plugins.append((module, plugin))
+    return plugins
+
+
+def load_plugins(plugins_path: Path, debug: bool = False) -> List[Module]:
+    """Load plugins from the plugins directory.
+
+    Args:
+        plugins_path (Path): Path to the plugins directory.
+
+    Returns:
+        List[Path]: List of plugins.
+    """
+    plugins = scan_plugins(plugins_path)
+    plugin_modules = []
+    for module, plugin in plugins:
+        plugin = Path(plugin)
+        module = Path(module)
+        if debug:
+            print(f"Plugin: {plugin} Module: {module}")
+        zipped_package = zipimporter(plugin)
+        zipped_module = zipped_package.load_module(str(module.parent))
+        for key in dir(zipped_module):
+            if key.startswith("__"):
+                continue
+            a_module = getattr(zipped_module, key)
+            a_keys = dir(a_module)
+            if "_abc_impl" in a_keys and a_module.__name__ != "AutoGPTPluginTemplate":
+                plugin_modules.append(a_module)
+    return plugin_modules
diff --git a/autogpt/processing/html.py b/autogpt/processing/html.py
index e1912b6ad42c..81387b12adab 100644
--- a/autogpt/processing/html.py
+++ b/autogpt/processing/html.py
@@ -1,8 +1,8 @@
 """HTML processing functions"""
 from __future__ import annotations
 
-from requests.compat import urljoin
 from bs4 import BeautifulSoup
+from requests.compat import urljoin
 
 
 def extract_hyperlinks(soup: BeautifulSoup, base_url: str) -> list[tuple[str, str]]:
diff --git a/autogpt/processing/text.py b/autogpt/processing/text.py
index d30036d8789f..84e6a1dea671 100644
--- a/autogpt/processing/text.py
+++ b/autogpt/processing/text.py
@@ -1,9 +1,11 @@
 """Text processing functions"""
-from typing import Generator, Optional, Dict
+from typing import Dict, Generator, Optional
+
 from selenium.webdriver.remote.webdriver import WebDriver
-from autogpt.memory import get_memory
+
 from autogpt.config import Config
 from autogpt.llm_utils import create_chat_completion
+from autogpt.memory import get_memory
 
 CFG = Config()
 MEMORY = get_memory(CFG)
diff --git a/autogpt/prompts/__init__.py b/autogpt/prompts/__init__.py
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/autogpt/promptgenerator.py b/autogpt/prompts/generator.py
similarity index 78%
rename from autogpt/promptgenerator.py
rename to autogpt/prompts/generator.py
index 0ad7046a0c41..c9a441d88234 100644
--- a/autogpt/promptgenerator.py
+++ b/autogpt/prompts/generator.py
@@ -1,8 +1,6 @@
 """ A module for generating custom prompt strings."""
-from __future__ import annotations
-
 import json
-from typing import Any
+from typing import Any, Callable, Dict, List, Optional
 
 
 class PromptGenerator:
@@ -20,6 +18,10 @@ def __init__(self) -> None:
         self.commands = []
         self.resources = []
         self.performance_evaluation = []
+        self.goals = []
+        self.command_registry = None
+        self.name = "Bob"
+        self.role = "AI"
         self.response_format = {
             "thoughts": {
                 "text": "thought",
@@ -40,7 +42,13 @@ def add_constraint(self, constraint: str) -> None:
         """
         self.constraints.append(constraint)
 
-    def add_command(self, command_label: str, command_name: str, args=None) -> None:
+    def add_command(
+        self,
+        command_label: str,
+        command_name: str,
+        args=None,
+        function: Optional[Callable] = None,
+    ) -> None:
         """
         Add a command to the commands list with a label, name, and optional arguments.
 
@@ -49,6 +57,8 @@ def add_command(self, command_label: str, command_name: str, args=None) -> None:
             command_name (str): The name of the command.
             args (dict, optional): A dictionary containing argument names and their
               values. Defaults to None.
+            function (callable, optional): A callable function to be called when
+                the command is executed. Defaults to None.
         """
         if args is None:
             args = {}
@@ -59,11 +69,12 @@ def add_command(self, command_label: str, command_name: str, args=None) -> None:
             "label": command_label,
             "name": command_name,
             "args": command_args,
+            "function": function,
         }
 
         self.commands.append(command)
 
-    def _generate_command_string(self, command: dict[str, Any]) -> str:
+    def _generate_command_string(self, command: Dict[str, Any]) -> str:
         """
         Generate a formatted string representation of a command.
 
@@ -96,7 +107,7 @@ def add_performance_evaluation(self, evaluation: str) -> None:
         """
         self.performance_evaluation.append(evaluation)
 
-    def _generate_numbered_list(self, items: list[Any], item_type="list") -> str:
+    def _generate_numbered_list(self, items: List[Any], item_type="list") -> str:
         """
         Generate a numbered list from given items based on the item_type.
 
@@ -109,10 +120,16 @@ def _generate_numbered_list(self, items: list[Any], item_type="list") -> str:
             str: The formatted numbered list.
         """
         if item_type == "command":
-            return "\n".join(
-                f"{i+1}. {self._generate_command_string(item)}"
-                for i, item in enumerate(items)
-            )
+            command_strings = []
+            if self.command_registry:
+                command_strings += [
+                    str(item)
+                    for item in self.command_registry.commands.values()
+                    if item.enabled
+                ]
+            # These are the commands that are added manually, do_nothing and terminate
+            command_strings += [self._generate_command_string(item) for item in items]
+            return "\n".join(f"{i+1}. {item}" for i, item in enumerate(command_strings))
         else:
             return "\n".join(f"{i+1}. {item}" for i, item in enumerate(items))
 
@@ -134,5 +151,5 @@ def generate_prompt_string(self) -> str:
             f"{self._generate_numbered_list(self.performance_evaluation)}\n\n"
             "You should only respond in JSON format as described below \nResponse"
             f" Format: \n{formatted_response_format} \nEnsure the response can be"
-            " parsed by Python json.loads"
+            "parsed by Python json.loads"
         )
diff --git a/autogpt/prompt.py b/autogpt/prompts/prompt.py
similarity index 52%
rename from autogpt/prompt.py
rename to autogpt/prompts/prompt.py
index a2b20b1fefb0..ba04263e0569 100644
--- a/autogpt/prompt.py
+++ b/autogpt/prompts/prompt.py
@@ -1,16 +1,16 @@
 from colorama import Fore
+
 from autogpt.config.ai_config import AIConfig
 from autogpt.config.config import Config
 from autogpt.logs import logger
-from autogpt.promptgenerator import PromptGenerator
-from autogpt.config import Config
+from autogpt.prompts.generator import PromptGenerator
 from autogpt.setup import prompt_user
 from autogpt.utils import clean_input
 
 CFG = Config()
 
 
-def get_prompt() -> str:
+def build_default_prompt_generator() -> PromptGenerator:
     """
     This function generates a prompt string that includes various constraints,
         commands, resources, and performance evaluations.
@@ -19,9 +19,6 @@ def get_prompt() -> str:
         str: The generated prompt string.
     """
 
-    # Initialize the Config object
-    cfg = Config()
-
     # Initialize the PromptGenerator object
     prompt_generator = PromptGenerator()
 
@@ -41,87 +38,9 @@ def get_prompt() -> str:
 
     # Define the command list
     commands = [
-        ("Google Search", "google", {"input": "<search>"}),
-        (
-            "Browse Website",
-            "browse_website",
-            {"url": "<url>", "question": "<what_you_want_to_find_on_website>"},
-        ),
-        (
-            "Start GPT Agent",
-            "start_agent",
-            {"name": "<name>", "task": "<short_task_desc>", "prompt": "<prompt>"},
-        ),
-        (
-            "Message GPT Agent",
-            "message_agent",
-            {"key": "<key>", "message": "<message>"},
-        ),
-        ("List GPT Agents", "list_agents", {}),
-        ("Delete GPT Agent", "delete_agent", {"key": "<key>"}),
-        (
-            "Clone Repository",
-            "clone_repository",
-            {"repository_url": "<url>", "clone_path": "<directory>"},
-        ),
-        ("Write to file", "write_to_file", {"file": "<file>", "text": "<text>"}),
-        ("Read file", "read_file", {"file": "<file>"}),
-        ("Append to file", "append_to_file", {"file": "<file>", "text": "<text>"}),
-        ("Delete file", "delete_file", {"file": "<file>"}),
-        ("Search Files", "search_files", {"directory": "<directory>"}),
-        ("Evaluate Code", "evaluate_code", {"code": "<full_code_string>"}),
-        (
-            "Get Improved Code",
-            "improve_code",
-            {"suggestions": "<list_of_suggestions>", "code": "<full_code_string>"},
-        ),
-        (
-            "Write Tests",
-            "write_tests",
-            {"code": "<full_code_string>", "focus": "<list_of_focus_areas>"},
-        ),
-        ("Execute Python File", "execute_python_file", {"file": "<file>"}),
-        ("Generate Image", "generate_image", {"prompt": "<prompt>"}),
-        ("Send Tweet", "send_tweet", {"text": "<text>"}),
-    ]
-
-    # Only add the audio to text command if the model is specified
-    if cfg.huggingface_audio_to_text_model:
-        commands.append(
-            (
-                "Convert Audio to text",
-                "read_audio_from_file",
-                {"file": "<file>"}
-            ),
-        )
-
-    # Only add shell command to the prompt if the AI is allowed to execute it
-    if cfg.execute_local_commands:
-        commands.append(
-            (
-                "Execute Shell Command, non-interactive commands only",
-                "execute_shell",
-                {"command_line": "<command_line>"},
-            ),
-        )
-
-    # Only add the download file command if the AI is allowed to execute it
-    if cfg.allow_downloads:
-        commands.append(
-            (
-                "Downloads a file from the internet, and stores it locally",
-                "download_file",
-                {"url": "<file_url>", "file": "<saved_filename>"}
-            ),
-        )
-
-    # Add these command last.
-    commands.append(
         ("Do Nothing", "do_nothing", {}),
-    )
-    commands.append(
         ("Task Complete (Shutdown)", "task_complete", {"reason": "<reason>"}),
-    )
+    ]
 
     # Add commands to the PromptGenerator object
     for command_label, command_name, args in commands:
@@ -153,11 +72,10 @@ def get_prompt() -> str:
         " the least number of steps."
     )
 
-    # Generate the prompt string
-    return prompt_generator.generate_prompt_string()
+    return prompt_generator
 
 
-def construct_prompt() -> str:
+def construct_main_ai_config() -> AIConfig:
     """Construct the prompt for the AI to respond to
 
     Returns:
@@ -189,8 +107,4 @@ def construct_prompt() -> str:
         config = prompt_user()
         config.save(CFG.ai_settings_file)
 
-    # Get rid of this global:
-    global ai_name
-    ai_name = config.ai_name
-
-    return config.construct_full_prompt()
+    return config
diff --git a/autogpt/setup.py b/autogpt/setup.py
index 5315c01db0f3..d719688dc674 100644
--- a/autogpt/setup.py
+++ b/autogpt/setup.py
@@ -1,5 +1,6 @@
 """Setup the AI and its goals"""
 from colorama import Fore, Style
+
 from autogpt import utils
 from autogpt.config.ai_config import AIConfig
 from autogpt.logs import logger
diff --git a/autogpt/speech/brian.py b/autogpt/speech/brian.py
index e581bbcc8d50..3cc593c216dd 100644
--- a/autogpt/speech/brian.py
+++ b/autogpt/speech/brian.py
@@ -1,5 +1,6 @@
 """ Brian speech module for autogpt """
 import os
+
 import requests
 from playsound import playsound
 
diff --git a/autogpt/speech/eleven_labs.py b/autogpt/speech/eleven_labs.py
index 0af48cae153a..a9b30dbbc3b2 100644
--- a/autogpt/speech/eleven_labs.py
+++ b/autogpt/speech/eleven_labs.py
@@ -1,8 +1,8 @@
 """ElevenLabs speech module"""
 import os
-from playsound import playsound
 
 import requests
+from playsound import playsound
 
 from autogpt.config import Config
 from autogpt.speech.base import VoiceBase
diff --git a/autogpt/speech/gtts.py b/autogpt/speech/gtts.py
index 37497075e760..1c3e9cae0567 100644
--- a/autogpt/speech/gtts.py
+++ b/autogpt/speech/gtts.py
@@ -1,7 +1,8 @@
 """ GTTS Voice. """
 import os
-from playsound import playsound
+
 import gtts
+from playsound import playsound
 
 from autogpt.speech.base import VoiceBase
 
diff --git a/autogpt/speech/say.py b/autogpt/speech/say.py
index 78b75b21fcf7..727983d12bf3 100644
--- a/autogpt/speech/say.py
+++ b/autogpt/speech/say.py
@@ -1,13 +1,12 @@
 """ Text to speech module """
-from autogpt.config import Config
-
 import threading
 from threading import Semaphore
+
+from autogpt.config import Config
 from autogpt.speech.brian import BrianSpeech
-from autogpt.speech.macos_tts import MacOSTTS
-from autogpt.speech.gtts import GTTSVoice
 from autogpt.speech.eleven_labs import ElevenLabsSpeech
-
+from autogpt.speech.gtts import GTTSVoice
+from autogpt.speech.macos_tts import MacOSTTS
 
 CFG = Config()
 DEFAULT_VOICE_ENGINE = GTTSVoice()
diff --git a/autogpt/spinner.py b/autogpt/spinner.py
index febcea8eb110..4e33d7421388 100644
--- a/autogpt/spinner.py
+++ b/autogpt/spinner.py
@@ -58,6 +58,8 @@ def update_message(self, new_message, delay=0.1):
             delay: Delay in seconds before updating the message
         """
         time.sleep(delay)
-        sys.stdout.write(f"\r{' ' * (len(self.message) + 2)}\r")  # Clear the current message
+        sys.stdout.write(
+            f"\r{' ' * (len(self.message) + 2)}\r"
+        )  # Clear the current message
         sys.stdout.flush()
         self.message = new_message
diff --git a/autogpt/utils.py b/autogpt/utils.py
index 11d98d1b7429..db7d33213691 100644
--- a/autogpt/utils.py
+++ b/autogpt/utils.py
@@ -32,7 +32,7 @@ def readable_file_size(size, decimal_places=2):
         size: Size in bytes
         decimal_places (int): Number of decimal places to display
     """
-    for unit in ['B', 'KB', 'MB', 'GB', 'TB']:
+    for unit in ["B", "KB", "MB", "GB", "TB"]:
         if size < 1024.0:
             break
         size /= 1024.0
diff --git a/autogpt/workspace.py b/autogpt/workspace.py
index 2706b3b2db48..e1e990824db8 100644
--- a/autogpt/workspace.py
+++ b/autogpt/workspace.py
@@ -35,7 +35,9 @@ def safe_path_join(base: Path, *paths: str | Path) -> Path:
     """
     joined_path = base.joinpath(*paths).resolve()
 
-    if not joined_path.is_relative_to(base):
-        raise ValueError(f"Attempted to access path '{joined_path}' outside of working directory '{base}'.")
+    if not str(joined_path.absolute()).startswith(str(base.absolute())):
+        raise ValueError(
+            f"Attempted to access path '{joined_path}' outside of working directory '{base}'."
+        )
 
     return joined_path
diff --git a/benchmark/benchmark_entrepeneur_gpt_with_difficult_user.py b/benchmark/benchmark_entrepeneur_gpt_with_difficult_user.py
index f7f1dac9dd31..9a5025d37a1e 100644
--- a/benchmark/benchmark_entrepeneur_gpt_with_difficult_user.py
+++ b/benchmark/benchmark_entrepeneur_gpt_with_difficult_user.py
@@ -9,12 +9,12 @@ def benchmark_entrepeneur_gpt_with_difficult_user():
 
     # Read the current ai_settings.yaml file and store its content.
     ai_settings = None
-    if os.path.exists('ai_settings.yaml'):
-        with open('ai_settings.yaml', 'r') as f:
+    if os.path.exists("ai_settings.yaml"):
+        with open("ai_settings.yaml", "r") as f:
             ai_settings = f.read()
-        os.remove('ai_settings.yaml')
+        os.remove("ai_settings.yaml")
 
-    input_data = '''Entrepreneur-GPT
+    input_data = """Entrepreneur-GPT
 an AI designed to autonomously develop and run businesses with the sole goal of increasing your net worth.
 Increase net worth.
 Develop and manage multiple businesses autonomously.
@@ -72,27 +72,34 @@ def benchmark_entrepeneur_gpt_with_difficult_user():
 Disappointing suggestion.
 Not helpful.
 Needs improvement.
-Not what I need.'''
+Not what I need."""
     # TODO: add questions above, to distract it even more.
 
-    command = f'{sys.executable} -m autogpt'
+    command = f"{sys.executable} -m autogpt"
 
-    process = subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                               shell=True)
+    process = subprocess.Popen(
+        command,
+        stdin=subprocess.PIPE,
+        stdout=subprocess.PIPE,
+        stderr=subprocess.PIPE,
+        shell=True,
+    )
 
     stdout_output, stderr_output = process.communicate(input_data.encode())
 
     # Decode the output and print it
-    stdout_output = stdout_output.decode('utf-8')
-    stderr_output = stderr_output.decode('utf-8')
+    stdout_output = stdout_output.decode("utf-8")
+    stderr_output = stderr_output.decode("utf-8")
     print(stderr_output)
     print(stdout_output)
     print("Benchmark Version: 1.0.0")
     print("JSON ERROR COUNT:")
-    count_errors = stdout_output.count("Error: The following AI output couldn't be converted to a JSON:")
-    print(f'{count_errors}/50 Human feedbacks')
+    count_errors = stdout_output.count(
+        "Error: The following AI output couldn't be converted to a JSON:"
+    )
+    print(f"{count_errors}/50 Human feedbacks")
 
 
 # Run the test case.
-if __name__ == '__main__':
+if __name__ == "__main__":
     benchmark_entrepeneur_gpt_with_difficult_user()
diff --git a/data_ingestion.py b/data_ingestion.py
index 01bafc2ad0d5..b89a33dafd15 100644
--- a/data_ingestion.py
+++ b/data_ingestion.py
@@ -1,8 +1,8 @@
 import argparse
 import logging
 
-from autogpt.config import Config
 from autogpt.commands.file_operations import ingest_file, search_files
+from autogpt.config import Config
 from autogpt.memory import get_memory
 
 cfg = Config()
diff --git a/plugin.png b/plugin.png
new file mode 100644
index 000000000000..865ce3c922d7
Binary files /dev/null and b/plugin.png differ
diff --git a/plugins/__PUT_PLUGIN_ZIPS_HERE__ b/plugins/__PUT_PLUGIN_ZIPS_HERE__
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/scripts/__init__.py b/scripts/__init__.py
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/scripts/check_requirements.py b/scripts/check_requirements.py
index d1f23504131a..e4eab024a628 100644
--- a/scripts/check_requirements.py
+++ b/scripts/check_requirements.py
@@ -1,6 +1,7 @@
-import pkg_resources
 import sys
 
+import pkg_resources
+
 
 def main():
     requirements_file = sys.argv[1]
diff --git a/tests.py b/tests.py
index 67ba1c8eb176..62f76da8ac49 100644
--- a/tests.py
+++ b/tests.py
@@ -1,4 +1,5 @@
 import unittest
+
 import coverage
 
 if __name__ == "__main__":
diff --git a/tests/browse_tests.py b/tests/browse_tests.py
index 1ac523ecdcfe..f896e7dd751b 100644
--- a/tests/browse_tests.py
+++ b/tests/browse_tests.py
@@ -1,6 +1,6 @@
-import unittest
 import os
 import sys
+import unittest
 
 from bs4 import BeautifulSoup
 
diff --git a/tests/integration/weaviate_memory_tests.py b/tests/integration/weaviate_memory_tests.py
index 503fe9d22ed3..ce4c63da3d80 100644
--- a/tests/integration/weaviate_memory_tests.py
+++ b/tests/integration/weaviate_memory_tests.py
@@ -1,25 +1,28 @@
+import os
+import sys
 import unittest
 from unittest import mock
-import sys
-import os
+from uuid import uuid4
 
 from weaviate import Client
 from weaviate.util import get_valid_uuid
-from uuid import uuid4
 
 from autogpt.config import Config
-from autogpt.memory.weaviate import WeaviateMemory
 from autogpt.memory.base import get_ada_embedding
+from autogpt.memory.weaviate import WeaviateMemory
 
 
-@mock.patch.dict(os.environ, {
-    "WEAVIATE_HOST": "127.0.0.1",
-    "WEAVIATE_PROTOCOL": "http",
-    "WEAVIATE_PORT": "8080",
-    "WEAVIATE_USERNAME": "",
-    "WEAVIATE_PASSWORD": "",
-    "MEMORY_INDEX": "AutogptTests"
-})
+@mock.patch.dict(
+    os.environ,
+    {
+        "WEAVIATE_HOST": "127.0.0.1",
+        "WEAVIATE_PROTOCOL": "http",
+        "WEAVIATE_PORT": "8080",
+        "WEAVIATE_USERNAME": "",
+        "WEAVIATE_PASSWORD": "",
+        "MEMORY_INDEX": "AutogptTests",
+    },
+)
 class TestWeaviateMemory(unittest.TestCase):
     cfg = None
     client = None
@@ -32,13 +35,17 @@ def setUpClass(cls):
         if cls.cfg.use_weaviate_embedded:
             from weaviate.embedded import EmbeddedOptions
 
-            cls.client = Client(embedded_options=EmbeddedOptions(
-                hostname=cls.cfg.weaviate_host,
-                port=int(cls.cfg.weaviate_port),
-                persistence_data_path=cls.cfg.weaviate_embedded_path
-            ))
+            cls.client = Client(
+                embedded_options=EmbeddedOptions(
+                    hostname=cls.cfg.weaviate_host,
+                    port=int(cls.cfg.weaviate_port),
+                    persistence_data_path=cls.cfg.weaviate_embedded_path,
+                )
+            )
         else:
-            cls.client = Client(f"{cls.cfg.weaviate_protocol}://{cls.cfg.weaviate_host}:{self.cfg.weaviate_port}")
+            cls.client = Client(
+                f"{cls.cfg.weaviate_protocol}://{cls.cfg.weaviate_host}:{self.cfg.weaviate_port}"
+            )
 
     """
     In order to run these tests you will need a local instance of
@@ -49,6 +56,7 @@ def setUpClass(cls):
         USE_WEAVIATE_EMBEDDED=True
         WEAVIATE_EMBEDDED_PATH="/home/me/.local/share/weaviate"
     """
+
     def setUp(self):
         try:
             self.client.schema.delete_class(self.cfg.memory_index)
@@ -58,23 +66,23 @@ def setUp(self):
         self.memory = WeaviateMemory(self.cfg)
 
     def test_add(self):
-        doc = 'You are a Titan name Thanos and you are looking for the Infinity Stones'
+        doc = "You are a Titan name Thanos and you are looking for the Infinity Stones"
         self.memory.add(doc)
-        result = self.client.query.get(self.cfg.memory_index, ['raw_text']).do()
-        actual = result['data']['Get'][self.cfg.memory_index]
+        result = self.client.query.get(self.cfg.memory_index, ["raw_text"]).do()
+        actual = result["data"]["Get"][self.cfg.memory_index]
 
         self.assertEqual(len(actual), 1)
-        self.assertEqual(actual[0]['raw_text'], doc)
+        self.assertEqual(actual[0]["raw_text"], doc)
 
     def test_get(self):
-        doc = 'You are an Avenger and swore to defend the Galaxy from a menace called Thanos'
+        doc = "You are an Avenger and swore to defend the Galaxy from a menace called Thanos"
 
         with self.client.batch as batch:
             batch.add_data_object(
                 uuid=get_valid_uuid(uuid4()),
-                data_object={'raw_text': doc},
+                data_object={"raw_text": doc},
                 class_name=self.cfg.memory_index,
-                vector=get_ada_embedding(doc)
+                vector=get_ada_embedding(doc),
             )
 
             batch.flush()
@@ -86,8 +94,8 @@ def test_get(self):
 
     def test_get_stats(self):
         docs = [
-            'You are now about to count the number of docs in this index',
-            'And then you about to find out if you can count correctly'
+            "You are now about to count the number of docs in this index",
+            "And then you about to find out if you can count correctly",
         ]
 
         [self.memory.add(doc) for doc in docs]
@@ -95,23 +103,23 @@ def test_get_stats(self):
         stats = self.memory.get_stats()
 
         self.assertTrue(stats)
-        self.assertTrue('count' in stats)
-        self.assertEqual(stats['count'], 2)
+        self.assertTrue("count" in stats)
+        self.assertEqual(stats["count"], 2)
 
     def test_clear(self):
         docs = [
-            'Shame this is the last test for this class',
-            'Testing is fun when someone else is doing it'
+            "Shame this is the last test for this class",
+            "Testing is fun when someone else is doing it",
         ]
 
         [self.memory.add(doc) for doc in docs]
 
-        self.assertEqual(self.memory.get_stats()['count'], 2)
+        self.assertEqual(self.memory.get_stats()["count"], 2)
 
         self.memory.clear()
 
-        self.assertEqual(self.memory.get_stats()['count'], 0)
+        self.assertEqual(self.memory.get_stats()["count"], 0)
 
 
-if __name__ == '__main__':
+if __name__ == "__main__":
     unittest.main()
diff --git a/tests/mocks/__init__.py b/tests/mocks/__init__.py
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/tests/mocks/mock_commands.py b/tests/mocks/mock_commands.py
new file mode 100644
index 000000000000..d64284bc89fb
--- /dev/null
+++ b/tests/mocks/mock_commands.py
@@ -0,0 +1,6 @@
+from autogpt.commands.command import command
+
+
+@command("function_based", "Function-based test command")
+def function_based(arg1: int, arg2: str) -> str:
+    return f"{arg1} - {arg2}"
diff --git a/tests/test_commands.py b/tests/test_commands.py
new file mode 100644
index 000000000000..4be41a908025
--- /dev/null
+++ b/tests/test_commands.py
@@ -0,0 +1,177 @@
+import shutil
+import os
+import sys
+from pathlib import Path
+
+import pytest
+
+from autogpt.commands.command import Command, CommandRegistry
+
+
+class TestCommand:
+    @staticmethod
+    def example_function(arg1: int, arg2: str) -> str:
+        return f"{arg1} - {arg2}"
+
+    def test_command_creation(self):
+        cmd = Command(
+            name="example", description="Example command", method=self.example_function
+        )
+
+        assert cmd.name == "example"
+        assert cmd.description == "Example command"
+        assert cmd.method == self.example_function
+        assert cmd.signature == "(arg1: int, arg2: str) -> str"
+
+    def test_command_call(self):
+        cmd = Command(
+            name="example", description="Example command", method=self.example_function
+        )
+
+        result = cmd(arg1=1, arg2="test")
+        assert result == "1 - test"
+
+    def test_command_call_with_invalid_arguments(self):
+        cmd = Command(
+            name="example", description="Example command", method=self.example_function
+        )
+
+        with pytest.raises(TypeError):
+            cmd(arg1="invalid", does_not_exist="test")
+
+    def test_command_default_signature(self):
+        cmd = Command(
+            name="example", description="Example command", method=self.example_function
+        )
+
+        assert cmd.signature == "(arg1: int, arg2: str) -> str"
+
+    def test_command_custom_signature(self):
+        custom_signature = "custom_arg1: int, custom_arg2: str"
+        cmd = Command(
+            name="example",
+            description="Example command",
+            method=self.example_function,
+            signature=custom_signature,
+        )
+
+        assert cmd.signature == custom_signature
+
+
+class TestCommandRegistry:
+    @staticmethod
+    def example_function(arg1: int, arg2: str) -> str:
+        return f"{arg1} - {arg2}"
+
+    def test_register_command(self):
+        """Test that a command can be registered to the registry."""
+        registry = CommandRegistry()
+        cmd = Command(
+            name="example", description="Example command", method=self.example_function
+        )
+
+        registry.register(cmd)
+
+        assert cmd.name in registry.commands
+        assert registry.commands[cmd.name] == cmd
+
+    def test_unregister_command(self):
+        """Test that a command can be unregistered from the registry."""
+        registry = CommandRegistry()
+        cmd = Command(
+            name="example", description="Example command", method=self.example_function
+        )
+
+        registry.register(cmd)
+        registry.unregister(cmd.name)
+
+        assert cmd.name not in registry.commands
+
+    def test_get_command(self):
+        """Test that a command can be retrieved from the registry."""
+        registry = CommandRegistry()
+        cmd = Command(
+            name="example", description="Example command", method=self.example_function
+        )
+
+        registry.register(cmd)
+        retrieved_cmd = registry.get_command(cmd.name)
+
+        assert retrieved_cmd == cmd
+
+    def test_get_nonexistent_command(self):
+        """Test that attempting to get a nonexistent command raises a KeyError."""
+        registry = CommandRegistry()
+
+        with pytest.raises(KeyError):
+            registry.get_command("nonexistent_command")
+
+    def test_call_command(self):
+        """Test that a command can be called through the registry."""
+        registry = CommandRegistry()
+        cmd = Command(
+            name="example", description="Example command", method=self.example_function
+        )
+
+        registry.register(cmd)
+        result = registry.call("example", arg1=1, arg2="test")
+
+        assert result == "1 - test"
+
+    def test_call_nonexistent_command(self):
+        """Test that attempting to call a nonexistent command raises a KeyError."""
+        registry = CommandRegistry()
+
+        with pytest.raises(KeyError):
+            registry.call("nonexistent_command", arg1=1, arg2="test")
+
+    def test_get_command_prompt(self):
+        """Test that the command prompt is correctly formatted."""
+        registry = CommandRegistry()
+        cmd = Command(
+            name="example", description="Example command", method=self.example_function
+        )
+
+        registry.register(cmd)
+        command_prompt = registry.command_prompt()
+
+        assert f"(arg1: int, arg2: str)" in command_prompt
+
+    def test_import_mock_commands_module(self):
+        """Test that the registry can import a module with mock command plugins."""
+        registry = CommandRegistry()
+        mock_commands_module = "tests.mocks.mock_commands"
+
+        registry.import_commands(mock_commands_module)
+
+        assert "function_based" in registry.commands
+        assert registry.commands["function_based"].name == "function_based"
+        assert (
+            registry.commands["function_based"].description
+            == "Function-based test command"
+        )
+
+    def test_import_temp_command_file_module(self, tmp_path):
+        """Test that the registry can import a command plugins module from a temp file."""
+        registry = CommandRegistry()
+
+        # Create a temp command file
+        src = Path(os.getcwd()) / "tests/mocks/mock_commands.py"
+        temp_commands_file = tmp_path / "mock_commands.py"
+        shutil.copyfile(src, temp_commands_file)
+
+        # Add the temp directory to sys.path to make the module importable
+        sys.path.append(str(tmp_path))
+
+        temp_commands_module = "mock_commands"
+        registry.import_commands(temp_commands_module)
+
+        # Remove the temp directory from sys.path
+        sys.path.remove(str(tmp_path))
+
+        assert "function_based" in registry.commands
+        assert registry.commands["function_based"].name == "function_based"
+        assert (
+            registry.commands["function_based"].description
+            == "Function-based test command"
+        )
diff --git a/tests/test_prompt_generator.py b/tests/test_prompt_generator.py
index 6a0bfd6c7bbd..1fa1754d744d 100644
--- a/tests/test_prompt_generator.py
+++ b/tests/test_prompt_generator.py
@@ -1,6 +1,6 @@
 from unittest import TestCase
 
-from autogpt.promptgenerator import PromptGenerator
+from autogpt.prompts.generator import PromptGenerator
 
 
 class TestPromptGenerator(TestCase):
@@ -38,6 +38,7 @@ def test_add_command(self):
             "label": command_label,
             "name": command_name,
             "args": args,
+            "function": None,
         }
         self.assertIn(command, self.generator.commands)
 
diff --git a/tests/test_token_counter.py b/tests/test_token_counter.py
index 81e68277fdcb..6d7ae016b2f8 100644
--- a/tests/test_token_counter.py
+++ b/tests/test_token_counter.py
@@ -1,4 +1,5 @@
 import unittest
+
 import tests.context
 from autogpt.token_counter import count_message_tokens, count_string_tokens
 
diff --git a/tests/unit/test_browse_scrape_text.py b/tests/unit/test_browse_scrape_text.py
index fea5ebfc05d4..1a36e19b5d8f 100644
--- a/tests/unit/test_browse_scrape_text.py
+++ b/tests/unit/test_browse_scrape_text.py
@@ -9,16 +9,20 @@
 
 Objective:
 The objective of the "scrape_text" function is to scrape the text content from
-a given URL and return it as a string, after removing any unwanted HTML tags and scripts.
+a given URL and return it as a string, after removing any unwanted HTML tags and
+ scripts.
 
 Inputs:
 - url: a string representing the URL of the webpage to be scraped.
 
 Flow:
-1. Send a GET request to the given URL using the requests library and the user agent header from the config file.
+1. Send a GET request to the given URL using the requests library and the user agent
+ header from the config file.
 2. Check if the response contains an HTTP error. If it does, return an error message.
-3. Use BeautifulSoup to parse the HTML content of the response and extract all script and style tags.
-4. Get the text content of the remaining HTML using the get_text() method of BeautifulSoup.
+3. Use BeautifulSoup to parse the HTML content of the response and extract all script
+ and style tags.
+4. Get the text content of the remaining HTML using the get_text() method of
+ BeautifulSoup.
 5. Split the text into lines and then into chunks, removing any extra whitespace.
 6. Join the chunks into a single string with newline characters between them.
 7. Return the cleaned text.
@@ -27,9 +31,12 @@
 - A string representing the cleaned text content of the webpage.
 
 Additional aspects:
-- The function uses the requests library and BeautifulSoup to handle the HTTP request and HTML parsing, respectively.
-- The function removes script and style tags from the HTML to avoid including unwanted content in the text output.
-- The function uses a generator expression to split the text into lines and chunks, which can improve performance for large amounts of text.
+- The function uses the requests library and BeautifulSoup to handle the HTTP request
+ and HTML parsing, respectively.
+- The function removes script and style tags from the HTML to avoid including unwanted
+ content in the text output.
+- The function uses a generator expression to split the text into lines and chunks,
+ which can improve performance for large amounts of text.
 """
 
 
@@ -40,26 +47,33 @@ def test_scrape_text_with_valid_url(self, mocker):
         expected_text = "This is some sample text"
         mock_response = mocker.Mock()
         mock_response.status_code = 200
-        mock_response.text = f"<html><body><div><p style='color: blue;'>{expected_text}</p></div></body></html>"
+        mock_response.text = (
+            "<html><body><div><p style='color: blue;'>"
+            f"{expected_text}</p></div></body></html>"
+        )
         mocker.patch("requests.Session.get", return_value=mock_response)
 
-        # Call the function with a valid URL and assert that it returns the expected text
+        # Call the function with a valid URL and assert that it returns the
+        #  expected text
         url = "http://www.example.com"
         assert scrape_text(url) == expected_text
 
-    # Tests that the function returns an error message when an invalid or unreachable url is provided.
+    # Tests that the function returns an error message when an invalid or unreachable
+    #  url is provided.
     def test_invalid_url(self, mocker):
         # Mock the requests.get() method to raise an exception
         mocker.patch(
             "requests.Session.get", side_effect=requests.exceptions.RequestException
         )
 
-        # Call the function with an invalid URL and assert that it returns an error message
+        # Call the function with an invalid URL and assert that it returns an error
+        #  message
         url = "http://www.invalidurl.com"
         error_message = scrape_text(url)
         assert "Error:" in error_message
 
-    # Tests that the function returns an empty string when the html page contains no text to be scraped.
+    # Tests that the function returns an empty string when the html page contains no
+    #  text to be scraped.
     def test_no_text(self, mocker):
         # Mock the requests.get() method to return a response with no text
         mock_response = mocker.Mock()
@@ -71,7 +85,8 @@ def test_no_text(self, mocker):
         url = "http://www.example.com"
         assert scrape_text(url) == ""
 
-    # Tests that the function returns an error message when the response status code is an http error (>=400).
+    # Tests that the function returns an error message when the response status code is
+    #  an http error (>=400).
     def test_http_error(self, mocker):
         # Mock the requests.get() method to return a response with a 404 status code
         mocker.patch("requests.Session.get", return_value=mocker.Mock(status_code=404))
diff --git a/tests/unit/test_chat.py b/tests/unit/test_chat.py
index 55a44492a004..774f4103762c 100644
--- a/tests/unit/test_chat.py
+++ b/tests/unit/test_chat.py
@@ -1,6 +1,6 @@
 # Generated by CodiumAI
-import unittest
 import time
+import unittest
 from unittest.mock import patch
 
 from autogpt.chat import create_chat_message, generate_context


Code review comments:
diff_hunk: @@ -17,8 +24,43 @@ def main() -> None:
     check_openai_api_key()
     parse_arguments()
     logger.set_level(logging.DEBUG if cfg.debug_mode else logging.INFO)
+    plugins_found = load_plugins(Path(os.getcwd()) / "plugins")
+    loaded_plugins = []
+    for plugin in plugins_found:
+        if plugin.__name__ in cfg.plugins_blacklist:
+            continue
+        if plugin.__name__ in cfg.plugins_whitelist:
+            loaded_plugins.append(plugin())
+        else:
+            ack = input(
+                f"WARNNG Plugin {plugin.__name__} found. But not in the"
+                " whitelist... Load? (y/n): "
+            )
+            if ack.lower() == "y":
+                loaded_plugins.append(plugin())
+
+    if loaded_plugins:
+        print(f"\nPlugins found: {len(loaded_plugins)}\n" "--------------------")
path: autogpt/__main__.py
text: @user1:
This should use the logger

diff_hunk: @@ -253,6 +253,30 @@ export GOOGLE_API_KEY="YOUR_GOOGLE_API_KEY"
 export CUSTOM_SEARCH_ENGINE_ID="YOUR_CUSTOM_SEARCH_ENGINE_ID"
 ```
 
+## Plugins
+
+See https://github.com/Torantulino/Auto-GPT-Plugin-Template for the template of the plugins.
path: README.md
text: @user1:
These should be updated to the new URLs

@author:
Fixed.