Repository: agiresearch/AIOS (language: python)

Issue:
fix: waiting time and turnaround time calculation
fix the calculation with following two points
1. accumulation_time should be accumulation_time += (agent_turnaround_time. - agent_waiting_time)
2. In sequential execution, only the first request of the agent will wait much longer, as an agent's remaining request will be executed once they have been put into the queue, so only the first element of request_waiting_times and request_turnaround_times should add the accumulated time


Discussion / hints:

Fix patch:
diff --git a/.env.example b/.env.example
new file mode 100644
index 00000000..42855dba
--- /dev/null
+++ b/.env.example
@@ -0,0 +1,4 @@
+OPENAI_API_KEY='sk-...'
+GEMINI_API_KEY=''
+HUGGING_FACE_HUB_TOKEN=''
+HF_HOME=''
diff --git a/.gitignore b/.gitignore
index c5225705..a5bdb75b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,6 @@
+logs/
+.idea/
+.DS_Store
 .ipynb_checkpoints
 __pycache__
 src/storage/*.json
diff --git a/README.md b/README.md
index 3b9d53c8..d8b9a27c 100644
--- a/README.md
+++ b/README.md
@@ -8,33 +8,44 @@
 AIOS, a Large Language Model (LLM) Agent operating system, embeds large language model into Operating Systems (OS) as the brain of the OS, enabling an operating system "with soul" -- an important step towards AGI. AIOS is designed to optimize resource allocation, facilitate context switch across agents, enable concurrent execution of agents, provide tool service for agents, maintain access control for agents, and provide a rich set of toolkits for LLM Agent developers.
 
 
-## 🏠 Architecture of AIOS
+## 🏠 1. Architecture of AIOS
 <p align="center">
 <img src="images/AIOS-Architecture.png">
 </p>
 
 
-## 📰 News
-- **[2024-04-14]** 🚀 AIOS currently supports generation interrupt (for open-sourced llms) and customized console loggers. Feel free to try!
+## 📰 2. News
+- **[2024-04-14]** 🚀 AIOS currently supports generation interrupt (for open-sourced llms from huggingface) and customized console loggers. Feel free to try!
 - **[2024-04-05]** 🛠️ AIOS codebase has been updated to add shell simulator, rapid API calls, and pre-commit test cases. Please see [CONTRIBUTE](https://github.com/agiresearch/AIOS/blob/main/CONTRIBUTE.md) for how to test your contributions and create pull requests.
-- **[2024-04-02]** 🌟 AIOS [Discord Community](https://discord.gg/B2HFxEgTJX) is up. Welcome to join the community for discussions, brainstorming, development, or just random chats!
+- **[2024-04-02]** 🤝 AIOS [Discord Community](https://discord.gg/B2HFxEgTJX) is up. Welcome to join the community for discussions, brainstorming, development, or just random chats!
 - **[2024-03-25]** ✈️ Our paper [AIOS: LLM Agent Operating System](https://arxiv.org/abs/2403.16971) is released and AIOS repository is officially launched!
 - **[2023-12-06]** 📋 After several months of working, our perspective paper [LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystem](https://arxiv.org/abs/2312.03815) is officially released.
 
 
-## ✈️ Getting Started
+## ✈️ 3. Getting Started
 
-### Installation
+### 3.1 Installation
+To run AIOS, you will need to install our agent creation package, OpenAGI.
+**Git clone AIOS and OpenAGI**
 ```bash
 git clone https://github.com/agiresearch/AIOS.git
+git clone git@github.com:agiresearch/OpenAGI.git
 ```
-**Make sure you have Python >= 3.9 and <= 3.11**
+**Make sure you have Python = 3.11**
 Install the required packages using pip
 ```bash
+conda create -n AIOS python=3.11
+source activate AIOS
+cd AIOS
 pip install -r requirements.txt
 ```
+**Allow your code to be able to see 'openagi'**
+```
+cd ../OpenAGI
+pip install -e .
+```
 
-### Usage
+### 3.2 Usage
 If you use open-sourced models from huggingface, you need to setup your [Hugging Face token](https://huggingface.co/settings/tokens) and cache directory
 ```bash
 export HUGGING_FACE_HUB_TOKEN=<YOUR READ TOKEN>
@@ -46,54 +57,81 @@ export OPENAI_API_KEY=<YOUR OPENAI API KEY>
 export GEMINI_API_KEY=<YOUR GEMINI API KEY>
 ```
 
-Here we provide two modes to run the AIOS: interactive mode and deployment mode
-#### Interactive Mode
-In the interactive mode, you can interact with AIOS to see the output of each step in running multiple agents
+You can also create .env file from the .env.example file, and then use dotenv to load the environment variables using .env file into your application's environment at runtime. 
+
+```bash
+cp .env.example .env
+```
+
+Here we provide two modes to run the AIOS: interactive mode
+#### (1) Demonstration Mode
+In the demonstration mode, we provide a toy example: we hardcode three agents and allow you to change the parameters. Then you can see the output of each step in running multiple agents
 For open-sourced LLMs, you need to setup the name of the LLM you would like to use the max gpu memory, the evaluation device and the maximum length of generated new tokens.
-```python
+```bash
 # For open-sourced LLMs
 python main.py --llm_name <llm_name> --max_gpu_memory <max_gpu_memory> --eval_device <eval_device> --max_new_tokens <max_new_tokens>
-## Use Gemma-2b-it for example
-python main.py --llm_name gemma-2b-it --max_gpu_memory '{"0": "24GB"}' --eval_device "cuda:0" --max_new_tokens 256
+## Use google/gemma-1.1-2b-it for example
+python main.py --llm_name google/gemma-1.1-2b-it --max_gpu_memory '{"0": "24GB"}' --eval_device "cuda:0" --max_new_tokens 256
 ```
 For close-sourced LLMs, you just need to setup the name of the LLM.
-```python
+```bash
 # For close-sourced LLMs
 python main.py --llm_name <llm_name>
 ## Use gpt-4 for example
 python main.py --llm_name gpt-4
 ```
-
-#### Deployment Mode
+You can use bash script to start the agent execution like this
+```bash
+bash scripts/run/gpt4.sh
+````
+#### (2) Interactive Mode
 In the deployment mode, the outputs of running agents are stored in files. And in this mode, you are provided with multiple commands to run agents and see resource usage of agents (e.g., run \<xxxAgent\>: \<YOUR TASK\>, print agent).
 Different from the interactive mode, you need to set all the default loggers as file loggers.
-```python
+```bash
 # For open-sourced LLMs
 python simulator.py --llm_name <llm_name> --max_gpu_memory <max_gpu_memory> --eval_device <eval_device> --max_new_tokens <max_new_tokens> --scheduler_log_mode file --agent_log_mode file --llm_kernel_log_mode file
-## Use Gemma-2b-it for example
-python simulator.py --llm_name gemma-2b-it --max_gpu_memory '{"0": "24GB"}' --eval_device "cuda:0" --max_new_tokens 256 --scheduler_log_mode file --agent_log_mode file --llm_kernel_log_mode file
+## Use google/gemma-1.1-2b-it for example
+python simulator.py --llm_name google/gemma-1.1-2b-it --max_gpu_memory '{"0": "24GB"}' --eval_device "cuda:0" --max_new_tokens 256 --scheduler_log_mode file --agent_log_mode file --llm_kernel_log_mode file
 ```
-```python
+```bash
 # For close-sourced LLMs
 python simulator.py --llm_name <llm_name> --scheduler_log_mode file --agent_log_mode file --llm_kernel_log_mode file
 ## Use gpt-4 for example
 python simulator.py --llm_name gpt-4 --scheduler_log_mode file --agent_log_mode file --llm_kernel_log_mode file
 ```
-#### Supported LLM backbones
-| LLM Name | Open-sourced                                     | Corporation |
-|-------------|--------------------------------------------------|----------|
-| gemma-2b-it      | Yes                                 | Google |
-| gemma-1.1-7b-it       | Yes                                         | Google |
-| Mistral-7B-Instruct-v0.2     | Yes | Mistral AI
-| Mixtral-8x7B-Instruct-v0.1     | Yes | Mistral AI |
-| Mixtral-8x22B-Instruct-v0.1      | Yes                              | Mistral AI |
-| Meta-Llama-3-8B-Instruct  | Yes | Meta |
-| Meta-Llama-3-70B-Instruct      | Yes                              | Meta |
-| gemini-pro      | No | Google |
-| gpt-3.5-turbo     | No   | OpenAI |
-| gpt-4    | No                   | OpenAI |
-
-## 🖋️ References
+You can use bash script to start the interactive simulation session like this
+```bash
+bash scripts/interactive/gpt4.sh
+````
+sample command
+```bash
+run NarrativeAgent: Craft a tale about a valiant warrior on a quest to uncover priceless treasures hidden within a mystical island.
+print agent
+run agent
+print agent
+```
+
+#### (3) Evaluation Mode
+In the evaluation mode, we allow you to configure different types of predefined agents (MathAgent, NarrativeAgent, RecAgent) with a configurable number of agents for each type. Additionally, you can evaluate the acceleration performance with or without AIOS by comparing the waiting time and turnaround time.
+```bash
+python eval.py --llm_name gpt-4 --agents MathAgent:1,NarrativeAgent:1,RecAgent:1
+```
+You can use bash script to start the agent execution like this
+```bash
+bash scripts/eval/gpt4.sh
+````
+If you want to obtain metrics for either concurrent execution (with AIOS) or sequential execution (without AIOS), you can specify the mode parameter when running the eval.py file."
+```bash
+python eval.py --llm_name gpt-4 --agents MathAgent:1,NarrativeAgent:1,RecAgent:1 --mode concurrent-only
+python eval.py --llm_name gpt-4 --agents MathAgent:1,NarrativeAgent:1,RecAgent:1 --mode sequential-only
+```
+
+### 3.3 Supported LLM backbones
+- gpt-3.5-turbo, gpt-4
+- gemini-pro
+- Open-sourced LLM from Huggingface
+
+## 🖋️ 4. References
 ```
 @article{mei2024aios,
   title={AIOS: LLM Agent Operating System},
@@ -109,17 +147,22 @@ python simulator.py --llm_name gpt-4 --scheduler_log_mode file --agent_log_mode
 }
 ```
 
-## 🚀 Contributions
+## 🚀 5. Contributions
 AIOS is dedicated to facilitating LLM agents' development and deployment in a systematic way, collaborators and contributions are always welcome to foster a cohesive, effective and efficient AIOS-Agent ecosystem!
 
 For detailed information on how to contribute, see [CONTRIBUTE](https://github.com/agiresearch/AIOS/blob/main/CONTRIBUTE.md). If you would like to contribute to the codebase, [issues](https://github.com/agiresearch/AIOS/issues) or [pull requests](https://github.com/agiresearch/AIOS/pulls) are always welcome!
 
-## 🌟 Discord Channel
+## 🌍 6. AIOS Contributors
+[![AIOS contributors](https://contrib.rocks/image?repo=agiresearch/AIOS&max=300)](https://github.com/agiresearch/AIOS/graphs/contributors)
+
+
+## 🤝 7. Discord Channel
 If you would like to join the community, ask questions, chat with fellows, learn about or propose new features, and participate in future developments, join our [Discord Community](https://discord.gg/B2HFxEgTJX)!
 
-## 📪 Contact
+## 📪 8. Contact
 
 For issues related to AIOS development, we encourage submitting [issues](https://github.com/agiresearch/AIOS/issues), [pull requests](https://github.com/agiresearch/AIOS/pulls), or initiating discussions in the AIOS [Discord Channel](https://discord.gg/B2HFxEgTJX). For other issues please feel free to contact Kai Mei (marknju2018@gmail.com) and Yongfeng Zhang (yongfeng@email.com).
 
-## 🌍 AIOS Contributors
-[![AIOS contributors](https://contrib.rocks/image?repo=agiresearch/AIOS&max=300)](https://github.com/agiresearch/AIOS/graphs/contributors)
+## 🌟 9. Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=agiresearch/AIOS&type=Date)](https://star-history.com/#agiresearch/AIOS&Date)
diff --git a/agent_configs/MathAgent_task.txt b/agent_configs/MathAgent_task.txt
new file mode 100644
index 00000000..1a714a6e
--- /dev/null
+++ b/agent_configs/MathAgent_task.txt
@@ -0,0 +1,30 @@
+Mark has 4 bags of marbles, each with 25 marbles. He gives 3 marbles to each of his 5 friends. How many marbles does he have left?
+Sara bakes 60 cookies and divides them evenly among her 4 friends. How many cookies does each friend receive?
+A farmer has 5 fields with 120 apple trees each. If each tree produces 30 apples, how many apples does the farmer have in total?
+Jenny reads 150 pages of a book each day. If she continues at this rate, how many pages will she read in 10 days?
+Alex buys 3 packs of pens, with each pack containing 12 pens, and gives 5 pens away. How many pens does Alex have now?
+A school orders 200 notebooks, each containing 50 pages. If students take 3 pages each day from their notebooks, how many days will the pages last?
+A baker makes 10 cakes, each cut into 8 slices. If he sells 45 slices, how many full cakes does he have left?
+There are 6 boxes with 18 chocolates each. If 4 chocolates are eaten from each box, how many chocolates are left in total?
+A library has 8 shelves, each with 35 books. If 15 books are checked out from each shelf, how many books remain in the library?
+Paul has 150 stamps and decides to distribute them equally among his 7 friends. How many stamps does each friend receive, and how many are left over?
+A train travels 450 miles in 5 hours. If it maintains the same speed, how far will it travel in 8 hours?
+A class of 30 students needs 5 pencils each for an exam. If each box contains 50 pencils, how many boxes are needed?
+A gardener plants 200 seeds. If 85% of the seeds germinate, how many plants does he expect to grow?
+An office has 9 computers, each running 4 applications simultaneously. How many applications are running in total?
+Emma saves $75 each month. After how many months will she have saved enough to buy a bike for $450?
+A marathon runner completes 3/5 of a race in 2 hours. At the same pace, how long will it take to finish the race?
+A box contains 250 screws. If one piece of furniture requires 15 screws, how many pieces of furniture can be assembled with the screws in the box?
+A movie theater has 15 rows with 20 seats each. If 70% of the seats are occupied, how many people are in the theater?
+If a car travels at 60 miles per hour and increases its speed by 5 miles per hour each hour, what is its speed after 4 hours?
+A wall needs 120 bricks to be built completely. If a worker can install 15 bricks per hour, how many hours will it take to build the wall?
+A recipe calls for 250 grams of flour, which is 25% of the total ingredients' weight. What is the total weight of all ingredients?
+During a sale, a shirt’s price is reduced by 15%, making the new price $85. What was the original price?
+A parking lot has 100 spaces and is 75% full. How many spaces are currently occupied, and how many are available?
+A water tank holds 500 liters and is currently 80% full. How many more liters can it hold?
+A runner increases his distance by 10% each day, starting with 5 km on day one. How far will he run on the fifth day?
+A cube's volume is 125 cubic inches. What is the length of each side of the cube?
+A factory produces 400 units of a product each day. If each unit sells for $25, what is the total daily revenue?
+A bus travels 300 km with 10 stops along the way. If each stop lasts 5 minutes, how long is the total stop time?
+A book has 20 chapters, each with 15 pages. How many pages does the book have?
+A store has 300 apples and sells 85% of them. How many apples are sold, and how many are left?
diff --git a/agent_configs/NarrativeAgent_task.txt b/agent_configs/NarrativeAgent_task.txt
new file mode 100644
index 00000000..5046e393
--- /dev/null
+++ b/agent_configs/NarrativeAgent_task.txt
@@ -0,0 +1,1 @@
+Craft a tale about a valiant warrior on a quest to uncover priceless treasures hidden within a mystical island.
diff --git a/agent_configs/RecAgent_task.txt b/agent_configs/RecAgent_task.txt
new file mode 100644
index 00000000..7012b9b5
--- /dev/null
+++ b/agent_configs/RecAgent_task.txt
@@ -0,0 +1,1 @@
+I want to take a tour to New York during the spring break, recommend some restaurants around for me.
diff --git a/eval.py b/eval.py
new file mode 100644
index 00000000..2ba7bbfe
--- /dev/null
+++ b/eval.py
@@ -0,0 +1,129 @@
+import os
+import sys
+import json
+
+from src.scheduler.fifo_scheduler import FIFOScheduler
+
+from src.scheduler.rr_scheduler import RRScheduler
+
+from openagi.src.agents.agent_factory import AgentFactory
+
+from openagi.src.agents.agent_process import AgentProcessFactory
+
+import warnings
+
+from src.llm_kernel import llms
+
+from concurrent.futures import ThreadPoolExecutor, as_completed
+
+from multiprocessing import Process
+
+from src.utils.utils import delete_directories
+from src.utils.calculator import get_numbers_concurrent, get_numbers_sequential, comparison
+
+import argparse
+
+import random
+
+import numpy as np
+from dotenv import find_dotenv, load_dotenv
+
+
+def parse_global_args():
+    parser = argparse.ArgumentParser(description="Parse global parameters")
+    parser.add_argument('--llm_name', type=str, default="gemma-2b-it", help="Specify the LLM name of AIOS")
+    parser.add_argument('--max_gpu_memory', type=json.loads, help="Max gpu memory allocated for the LLM")
+    parser.add_argument('--eval_device', type=str, help="Evaluation device")
+    parser.add_argument('--max_new_tokens', type=int, default=256,
+                        help="The maximum number of new tokens for generation")
+    parser.add_argument("--scheduler_log_mode", type=str, default="console", choices=["console", "file"])
+    parser.add_argument("--agent_log_mode", type=str, default="console", choices=["console", "file"])
+    parser.add_argument("--mode", type=str, default="compare", choices=["compare", "concurrent-only", "sequential-only"])
+    parser.add_argument("--llm_kernel_log_mode", type=str, default="console", choices=["console", "file"])
+    parser.add_argument("--agents", type=str, required=True,
+                        help="following the format of <agent1>:<agent1_num>,<agent2>:<agent2_num>")
+
+    return parser
+
+
+def clean_cache(root_directory):
+    targets = {'.ipynb_checkpoints', '__pycache__', ".pytest_cache", "context_restoration"}
+    delete_directories(root_directory, targets)
+
+
+def main():
+    warnings.filterwarnings("ignore")
+    parser = parse_global_args()
+    args = parser.parse_args()
+
+    llm_name = args.llm_name
+    max_gpu_memory = args.max_gpu_memory
+    eval_device = args.eval_device
+    max_new_tokens = args.max_new_tokens
+    scheduler_log_mode = args.scheduler_log_mode
+    agent_log_mode = args.agent_log_mode
+    llm_kernel_log_mode = args.llm_kernel_log_mode
+    load_dotenv()
+
+    llm = llms.LLMKernel(
+        llm_name=llm_name,
+        max_gpu_memory=max_gpu_memory,
+        eval_device=eval_device,
+        max_new_tokens=max_new_tokens,
+        log_mode=llm_kernel_log_mode
+    )
+
+    scheduler = RRScheduler(
+        llm=llm,
+        log_mode=scheduler_log_mode
+    )
+
+    agent_process_factory = AgentProcessFactory()
+
+    agent_factory = AgentFactory(
+        llm=llm,
+        agent_process_queue=scheduler.agent_process_queue,
+        agent_process_factory=agent_process_factory,
+        agent_log_mode=agent_log_mode
+    )
+
+    agent_thread_pool = ThreadPoolExecutor(max_workers=2000)
+
+    scheduler.start()
+
+    agents = args.agents
+    agent_list = []
+    for agent in agents.split(","):
+        agent = agent.split(":")
+        agent_name = agent[0]
+        agent_num = int(agent[1])
+        agent_list.append((agent_name, agent_num))
+
+    def execute_mode(mode, agent_list, agent_factory, agent_thread_pool=None):
+        print(f"**** {mode} Execution Statistics Starts ****\n")
+        if mode == "concurrent":
+            metrics = get_numbers_concurrent(agent_list, agent_factory, agent_thread_pool)
+        else:
+            metrics = get_numbers_sequential(agent_list, agent_factory)
+        print(f"{mode.capitalize()} Metrics:", metrics)
+        print(f"**** {mode} Execution Statistics Ends ****\n")
+        return metrics
+
+    if args.mode == "compare":
+        concurrent_metrics = execute_mode("concurrent", agent_list, agent_factory, agent_thread_pool)
+        sequential_metrics = execute_mode("sequential", agent_list, agent_factory)
+        comparison(concurrent_metrics, sequential_metrics)
+    elif args.mode == "concurrent-only":
+        execute_mode("concurrent", agent_list, agent_factory, agent_thread_pool)
+    elif args.mode == "sequential-only":
+        execute_mode("sequential", agent_list, agent_factory)
+    else:
+        print("Error: Invalid mode")
+
+    clean_cache(root_directory="./")
+
+    scheduler.stop()
+
+
+if __name__ == "__main__":
+    main()
diff --git a/main.py b/main.py
index 1935871b..59c6a29d 100644
--- a/main.py
+++ b/main.py
@@ -10,9 +10,9 @@
     parse_global_args,
 )
 
-from src.agents.agent_factory import AgentFactory
+from openagi.src.agents.agent_factory import AgentFactory
 
-from src.agents.agent_process import AgentProcessFactory
+from openagi.src.agents.agent_process import AgentProcessFactory
 
 import warnings
 
@@ -23,6 +23,7 @@
 from multiprocessing import Process
 
 from src.utils.utils import delete_directories
+from dotenv import find_dotenv, load_dotenv
 
 def clean_cache(root_directory):
     targets = {'.ipynb_checkpoints', '__pycache__', ".pytest_cache", "context_restoration"}
@@ -40,6 +41,7 @@ def main():
     scheduler_log_mode = args.scheduler_log_mode
     agent_log_mode = args.agent_log_mode
     llm_kernel_log_mode = args.llm_kernel_log_mode
+    load_dotenv()
 
     llm = llms.LLMKernel(
         llm_name = llm_name,
@@ -49,12 +51,6 @@ def main():
         log_mode = llm_kernel_log_mode
     )
 
-    # start the scheduler
-    # scheduler = FIFOScheduler(
-    #     llm = llm,
-    #     log_mode = scheduler_log_mode
-    # )
-
     scheduler = RRScheduler(
         llm = llm,
         log_mode = scheduler_log_mode
@@ -92,19 +88,9 @@ def main():
     )
 
     agent_tasks = [math_agent, narrative_agent, rec_agent]
-    # agent_tasks = [math_agent]
 
     for r in as_completed(agent_tasks):
         res = r.result()
-    # math_agent = Process(
-    #     target=agent_factory.run_agent,
-    #     args=("MathAgent",
-    #           "Solve the problem that Albert is wondering how much pizza he can eat in one day. He buys 2 large pizzas and 2 small pizzas. A large pizza has 16 slices and a small pizza has 8 slices. If he eats it all, how many pieces does he eat that day?")
-    # )
-
-    # math_agent.start()
-
-    # math_agent.join()
 
     scheduler.stop()
 
diff --git a/requirements.txt b/requirements.txt
index 9c9926f1..27933640 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -15,3 +15,4 @@ transformers==4.38.1
 wolframalpha==5.0.0
 pre-commit==3.7.0
 click==8.1.7
+python-dotenv==1.0.0
diff --git a/scripts/eval/gemini.sh b/scripts/eval/gemini.sh
new file mode 100644
index 00000000..d7a5a74d
--- /dev/null
+++ b/scripts/eval/gemini.sh
@@ -0,0 +1,1 @@
+python eval.py --llm_name gemini-pro --agents MathAgent:1,NarrativeAgent:1,RecAgent:1
\ No newline at end of file
diff --git a/scripts/eval/gpt4.sh b/scripts/eval/gpt4.sh
new file mode 100644
index 00000000..723bb7e2
--- /dev/null
+++ b/scripts/eval/gpt4.sh
@@ -0,0 +1,1 @@
+python eval.py --llm_name gpt-4 --agents MathAgent:1,NarrativeAgent:1,RecAgent:1
\ No newline at end of file
diff --git a/scripts/interactive/gemini.sh b/scripts/interactive/gemini.sh
new file mode 100644
index 00000000..8bd32b04
--- /dev/null
+++ b/scripts/interactive/gemini.sh
@@ -0,0 +1,1 @@
+python simulator.py --llm_name gemini-pro --scheduler_log_mode file --agent_log_mode file --llm_kernel_log_mode file
\ No newline at end of file
diff --git a/scripts/interactive/gpt4.sh b/scripts/interactive/gpt4.sh
new file mode 100644
index 00000000..c6462ca9
--- /dev/null
+++ b/scripts/interactive/gpt4.sh
@@ -0,0 +1,1 @@
+python simulator.py --llm_name gpt-4 --scheduler_log_mode file --agent_log_mode file --llm_kernel_log_mode file
\ No newline at end of file
diff --git a/scripts/run/gemini.sh b/scripts/run/gemini.sh
new file mode 100644
index 00000000..90e92448
--- /dev/null
+++ b/scripts/run/gemini.sh
@@ -0,0 +1,1 @@
+python main.py --llm_name gemini-pro
\ No newline at end of file
diff --git a/scripts/run/gpt4.sh b/scripts/run/gpt4.sh
new file mode 100644
index 00000000..5828ee87
--- /dev/null
+++ b/scripts/run/gpt4.sh
@@ -0,0 +1,1 @@
+python main.py --llm_name gpt-4
\ No newline at end of file
diff --git a/simulator.py b/simulator.py
index c32405cb..aead2302 100644
--- a/simulator.py
+++ b/simulator.py
@@ -19,9 +19,9 @@
     parse_global_args,
 )
 
-from src.agents.agent_factory import AgentFactory
+from openagi.src.agents.agent_factory import AgentFactory
 
-from src.agents.agent_process import AgentProcessFactory
+from openagi.src.agents.agent_process import AgentProcessFactory
 
 import warnings
 
@@ -30,6 +30,7 @@
 import threading
 
 from src.utils.utils import delete_directories
+from dotenv import find_dotenv, load_dotenv
 
 def clean_cache(root_directory):
     targets = {'.ipynb_checkpoints', '__pycache__', ".pytest_cache", "context_restoration"}
@@ -47,6 +48,7 @@ def main():
     scheduler_log_mode = args.scheduler_log_mode
     agent_log_mode = args.agent_log_mode
     llm_kernel_log_mode = args.llm_kernel_log_mode
+    load_dotenv()
 
     llm = llms.LLMKernel(
         llm_name = llm_name,
@@ -56,12 +58,6 @@ def main():
         log_mode = llm_kernel_log_mode
     )
 
-    # start the scheduler
-    # scheduler = FIFOScheduler(
-    #     llm = llm,
-    #     log_mode = scheduler_log_mode
-    # )
-
     scheduler = RRScheduler(
         llm = llm,
         log_mode = scheduler_log_mode
diff --git a/src/agents/__init__.py b/src/agents/__init__.py
deleted file mode 100644
index e69de29b..00000000
diff --git a/src/agents/agent_config/MathAgent.json b/src/agents/agent_config/MathAgent.json
deleted file mode 100644
index 12e1b786..00000000
--- a/src/agents/agent_config/MathAgent.json
+++ /dev/null
@@ -1,13 +0,0 @@
-{
-    "name": "MathAgent",
-    "description": [
-        "You are an expert who is good at solving mathematical problems. ",
-        "Given a mathematical problem, if the problem is complex, you need to break down this problem into smaller sub-problems and solve."
-    ],
-    "flow": [
-
-    ],
-    "tool_info": [
-
-    ]
-}
diff --git a/src/agents/agent_config/NarrativeAgent.json b/src/agents/agent_config/NarrativeAgent.json
deleted file mode 100644
index 83d67267..00000000
--- a/src/agents/agent_config/NarrativeAgent.json
+++ /dev/null
@@ -1,9 +0,0 @@
-{
-    "name": "NarrativeAgent",
-    "description": [
-        "You are an expert who is good at writing novels. ",
-        "Given a theme or background, you need to write a short story with a well-developed plot and characters, develop different sections of the story, such as introduction, rising action, climax, and conclusion."
-    ],
-    "flow": [],
-    "tool_info": []
-}
diff --git a/src/agents/agent_config/RecAgent.json b/src/agents/agent_config/RecAgent.json
deleted file mode 100644
index 9a65d26b..00000000
--- a/src/agents/agent_config/RecAgent.json
+++ /dev/null
@@ -1,9 +0,0 @@
-{
-    "name": "RecAgent",
-    "description": [
-        "You are an expert who is good at recommending restaurants or hotels for users.",
-        "Given a request, you need to first determine the right recommendation direction and then provide the recommendation lists."
-    ],
-    "flow": [],
-    "tool_info": []
-}
diff --git a/src/agents/agent_factory.py b/src/agents/agent_factory.py
deleted file mode 100644
index 4faa9f50..00000000
--- a/src/agents/agent_factory.py
+++ /dev/null
@@ -1,119 +0,0 @@
-from datetime import datetime
-
-import heapq
-
-from .native_agents.math_agent.math_agent import MathAgent
-
-from .native_agents.narrative_agent.narrative_agent import NarrativeAgent
-
-from .native_agents.rec_agent.rec_agent import RecAgent
-
-from concurrent.futures import ThreadPoolExecutor, as_completed
-
-from threading import Thread, Lock, Event
-
-# from multiprocessing import Process, Lock, Event
-
-from pympler import asizeof
-
-class AgentFactory:
-    def __init__(self, llm, agent_process_queue, agent_process_factory, agent_log_mode):
-        self.max_aid = 256
-        self.llm = llm
-        self.aid_pool = [i for i in range(self.max_aid)]
-        heapq.heapify(self.aid_pool)
-        self.agent_process_queue = agent_process_queue
-        self.agent_process_factory = agent_process_factory
-
-        self.agent_table = {
-            "MathAgent": MathAgent,
-            "NarrativeAgent": NarrativeAgent,
-            "RecAgent": RecAgent,
-            # "TravelAgent": TravelAgent
-        }
-
-        self.current_agents = {}
-
-        self.current_agents_lock = Lock()
-
-        # self.thread = Thread(target=self.deactivate_agent)
-
-        self.terminate_signal = Event()
-
-        self.agent_log_mode = agent_log_mode
-
-    def activate_agent(self, agent_name, task_input):
-        # print(task_input)
-        agent = self.agent_table[agent_name](
-            agent_name = agent_name,
-            task_input = task_input,
-            llm = self.llm,
-            agent_process_queue = self.agent_process_queue,
-            agent_process_factory = self.agent_process_factory,
-            log_mode = self.agent_log_mode
-        )
-        aid = heapq.heappop(self.aid_pool)
-
-        agent.set_aid(aid)
-        time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
-
-        agent.set_status("active")
-        agent.set_created_time(time)
-        if not self.terminate_signal.is_set():
-            with self.current_agents_lock:
-                self.current_agents[aid] = agent
-
-        return agent
-
-    def run_agent(self, agent_name, task_input):
-        agent = self.activate_agent(
-            agent_name=agent_name,
-            task_input=task_input
-        )
-        # print(task_input)
-        agent.run()
-        self.deactivate_agent(agent.get_aid())
-
-    def print_agent(self):
-        headers = ["Agent ID", "Agent Name", "Created Time", "Status", "Memory Usage"]
-        data = []
-        for id, agent in self.current_agents.items():
-            agent_name = agent.agent_name
-            created_time = agent.created_time
-            status = agent.status
-            memory_usage = f"{asizeof.asizeof(agent)} bytes"
-            data.append(
-                [id, agent_name, created_time, status, memory_usage]
-            )
-        self.print(headers=headers, data=data)
-
-
-    def print(self, headers, data):
-        # align output
-        column_widths = [
-            max(len(str(row[i])) for row in [headers] + data) for i in range(len(headers))
-        ]
-        print("+" + "-" * (sum(column_widths) + len(headers) * 3 - 3 ) + "+")
-        print(self.format_row(headers, column_widths))
-        print("=" * (sum(column_widths) + len(headers) * 3 - 1))
-        for i, row in enumerate(data):
-            print(self.format_row(row, column_widths))
-            if i < len(data):
-                print("-" * (sum(column_widths) + len(headers) * 3 - 1))
-        print("+" + "-" * (sum(column_widths) + len(headers) * 3 - 3 ) + "+")
-
-
-    def format_row(self, row, widths, align="<"):
-        row_str = " | ".join(f"{str(item):{align}{widths[i]}}" for i, item in enumerate(row))
-        return row_str
-
-    def deactivate_agent(self, aid):
-        self.current_agents.pop(aid)
-        heapq.heappush(self.aid_pool, aid)
-
-    def start(self):
-        """start the factory to check inactive agent"""
-        self.thread.start()
-
-    def stop(self):
-        self.thread.join()
diff --git a/src/agents/agent_process.py b/src/agents/agent_process.py
deleted file mode 100644
index 515fe7e2..00000000
--- a/src/agents/agent_process.py
+++ /dev/null
@@ -1,168 +0,0 @@
-import os
-
-import time
-
-from queue import Queue
-
-from datetime import datetime
-
-import heapq
-
-from concurrent.futures import ThreadPoolExecutor, as_completed
-
-from threading import Thread, Lock, Event
-
-from pympler import asizeof
-
-class AgentProcess:
-    def __init__(self, agent_name, prompt):
-        self.agent_name = agent_name
-        self.prompt = prompt
-        self.pid: int = None
-        self.status = None
-        self.response = None
-        self.time_limit = None
-        self.created_time = None
-        self.start_time = None
-        self.end_time = None
-
-    def set_created_time(self, time):
-        self.created_time = time
-
-    def get_created_time(self):
-        return self.created_time
-
-    def set_start_time(self, time):
-        self.start_time = time
-
-    def get_start_time(self):
-        return self.start_time
-
-    def set_end_time(self, time):
-        self.end_time = time
-
-    def get_end_time(self):
-        return self.end_time
-
-    def set_priority(self, priority):
-        self.priority = priority
-
-    def get_priority(self):
-        return self.priority
-
-    def set_status(self, status):
-        self.status = status
-
-    def get_status(self):
-        return self.status
-
-    def set_pid(self, pid):
-        self.pid = pid
-
-    def get_pid(self):
-        return self.pid
-
-    def set_prompt(self, prompt):
-        self.prompt = prompt
-
-    def get_prompt(self):
-        return self.prompt
-
-    def get_response(self):
-        return self.response
-
-    def set_response(self, response):
-        self.response = response
-
-
-    def get_time_limit(self):
-        return self.time_limit
-
-    def set_time_limit(self, time_limit):
-        self.time_limit = time_limit
-
-class AgentProcessFactory:
-    def __init__(self, agent_process_log_mode = None):
-        self.max_pid = 1024
-        self.pid_pool = [i for i in range(self.max_pid)]
-        heapq.heapify(self.pid_pool)
-
-        self.thread = Thread(target=self.deactivate_agent_process)
-
-        self.current_agent_processes = dict()
-
-        self.current_agent_processes_lock = Lock()
-
-        self.terminate_signal = Event()
-
-        self.agent_process_log_mode = agent_process_log_mode
-
-    def activate_agent_process(self, agent_name, prompt):
-        if not self.terminate_signal.is_set():
-            with self.current_agent_processes_lock:
-                agent_process = AgentProcess(
-                    agent_name = agent_name,
-                    prompt = prompt,
-                )
-                pid = heapq.heappop(self.pid_pool)
-                agent_process.set_pid(pid)
-                agent_process.set_status("active")
-                self.current_agent_processes[pid] = agent_process
-                return agent_process
-
-    def print_agent_process(self):
-        headers = ["Agent Process ID", "Agent Name", "Created Time", "Status"]
-        data = []
-        for id, agent_process in self.current_agent_processes.items():
-            agent_name = agent_process.agent_name
-            created_time = agent_process.created_time
-            status = agent_process.status
-            # memory_usage = f"{asizeof.asizeof(agent)} bytes"
-            data.append(
-                [id, agent_name, created_time, status]
-            )
-        self.print(headers=headers, data=data)
-
-
-    def print(self, headers, data):
-        # align output
-        column_widths = [
-            max(len(str(row[i])) for row in [headers] + data) for i in range(len(headers))
-        ]
-        print("+" + "-" * (sum(column_widths) + len(headers) * 3 - 3 ) + "+")
-        print(self.format_row(headers, column_widths))
-        print("=" * (sum(column_widths) + len(headers) * 3 - 1))
-        for i, row in enumerate(data):
-            print(self.format_row(row, column_widths))
-            if i < len(data):
-                print("-" * (sum(column_widths) + len(headers) * 3 - 1))
-        print("+" + "-" * (sum(column_widths) + len(headers) * 3 - 3 ) + "+")
-
-
-    def format_row(self, row, widths, align="<"):
-        row_str = " | ".join(f"{str(item):{align}{widths[i]}}" for i, item in enumerate(row))
-        return row_str
-
-    def deactivate_agent_process(self, pid):
-        # import time
-        # while not self.terminate_signal.is_set():
-        #     with self.current_agent_processes_lock:
-        #         invalid_pids = []
-        #         items = self.current_agent_processes.items()
-        #         for pid, agent_process in items:
-        #             if agent_process.get_status() == "done":
-        #                 # agent.set_status("inactive")
-        #                 time.sleep(5)
-        #                 invalid_pids.append(pid)
-        #         for pid in invalid_pids:
-        #             self.current_agent_processes.pop(pid)
-        #             heapq.heappush(self.pid_pool, pid)
-        self.current_agent_processes.pop(pid)
-        heapq.heappush(self.pid_pool, pid)
-
-    def start(self):
-        """start the factory to check inactive agent"""
-        self.thread.start()
-
-    def stop(self):
-        self.thread.join()
diff --git a/src/agents/base.py b/src/agents/base.py
deleted file mode 100644
index ccd51d9d..00000000
--- a/src/agents/base.py
+++ /dev/null
@@ -1,189 +0,0 @@
-import os
-
-import json
-
-from src.agents.agent_process import (
-    AgentProcess,
-    # AgentProcessQueue
-)
-
-# from multiprocessing import Process
-
-import logging
-
-import time
-
-from threading import Thread
-
-from datetime import datetime
-
-import numpy as np
-
-from src.utils.logger import AgentLogger
-class CustomizedThread(Thread):
-    def __init__(self, target, args=()):
-        super().__init__()
-        self.target = target
-        self.args = args
-        self.result = None
-
-    def run(self):
-        self.result = self.target(*self.args)
-
-    def join(self):
-        super().join()
-        return self.result
-
-# class CustomizedProcess(Process):
-#     def __init__(self, target, args=()):
-#         super().__init__()
-#         self.target = target
-#         self.args = args
-#         self.result = None
-
-#     def run(self):
-#         self.result = self.target(*self.args)
-
-#     def join(self):
-#         super().join()
-#         return self.result
-
-class BaseAgent:
-    def __init__(self,
-                 agent_name,
-                 task_input,
-                 llm,
-                 agent_process_queue,
-                 agent_process_factory,
-                 log_mode: str
-        ):
-        self.agent_name = agent_name
-        self.config = self.load_config()
-        self.prefix = " ".join(self.config["description"])
-        self.task_input = task_input
-        self.llm = llm
-        self.agent_process_queue = agent_process_queue
-        self.agent_process_factory = agent_process_factory
-
-        self.log_mode = log_mode
-        self.logger = self.setup_logger()
-        self.logger.log(f"Initialized. \n", level="info")
-
-        self.set_status("active")
-        self.set_created_time(time)
-
-    def run(self):
-        '''Execute each step to finish the task.'''
-        pass
-
-    def setup_logger(self):
-        logger = AgentLogger(self.agent_name, self.log_mode)
-        return logger
-
-    def load_config(self):
-        config_file = os.path.join(os.getcwd(), "src", "agents", "agent_config/{}.json".format(self.agent_name))
-        with open(config_file, "r") as f:
-            config = json.load(f)
-            return config
-
-    def get_response(self, prompt, temperature=0.0):
-        thread = CustomizedThread(target=self.query_loop, args=(prompt, ))
-        thread.start()
-        return thread.join()
-        # process = CustomizedProcess(target=self.query_loop, args=(prompt, ))
-        # process.start()
-        # return process.join()
-        # return self.query_loop(prompt)
-
-    def query_loop(self, prompt):
-        agent_process = self.create_agent_request(prompt)
-
-        # print(f"Loop Prompt: {prompt}")
-        completed_response, waiting_times, turnaround_times = "", [], []
-
-        # print("Already put into the queue")
-        while agent_process.get_status() != "done":
-            # print(agent_process.get_status())
-            thread = Thread(target=self.listen, args=(agent_process, ))
-            # process = Process(target=self.listen, args=(agent_process, ))
-            current_time = time.time()
-
-            # reinitialize agent status
-            agent_process.set_created_time(current_time)
-            agent_process.set_response(None)
-            self.agent_process_queue.put(agent_process)
-
-            thread.start()
-            thread.join()
-            # process.start()
-            # process.join()
-
-            completed_response = agent_process.get_response()
-            if agent_process.get_status() != "done":
-                self.logger.log(
-                    f"Suspended due to the reach of time limit ({agent_process.get_time_limit()}s). Current result is: {completed_response}\n",
-                    level="suspending"
-                )
-
-            waiting_time = agent_process.get_start_time() - current_time
-            turnaround_time = agent_process.get_end_time() - current_time
-
-            waiting_times.append(waiting_time)
-            turnaround_times.append(turnaround_time)
-            # Re-start the thread if not done
-
-        self.agent_process_factory.deactivate_agent_process(agent_process.get_pid())
-
-        completed_response = completed_response.replace("\n", "")
-        return completed_response, np.mean(np.array(waiting_times)), np.mean(np.array(turnaround_times))
-
-    def create_agent_request(self, prompt):
-        agent_process = self.agent_process_factory.activate_agent_process(
-            agent_name = self.agent_name,
-            prompt = prompt
-        )
-        agent_process.set_created_time(time.time())
-        # print("Already put into the queue")
-        return agent_process
-
-    def listen(self, agent_process):
-        """Response Listener for agent
-
-        Args:
-            agent_process (AgentProcess): Listened AgentProcess
-
-        Returns:
-            str: LLM response of Agent Process
-        """
-        while agent_process.get_response() is None:
-            time.sleep(0.2)
-
-        return agent_process.get_response()
-
-    def set_aid(self, aid):
-        self.aid = aid
-
-    def get_aid(self):
-        return self.aid
-
-    def get_agent_name(self):
-        return self.agent_name
-
-    def set_status(self, status):
-
-        """
-        Status type: Waiting, Running, Done, Inactive
-        """
-        self.status = status
-
-    def get_status(self):
-        return self.status
-
-    def set_created_time(self, time):
-        self.created_time = time
-
-    def get_created_time(self):
-        return self.created_time
-
-    def parse_result(self, prompt):
-        pass
diff --git a/src/agents/native_agents/math_agent/math_agent.py b/src/agents/native_agents/math_agent/math_agent.py
deleted file mode 100644
index 2623d23f..00000000
--- a/src/agents/native_agents/math_agent/math_agent.py
+++ /dev/null
@@ -1,105 +0,0 @@
-from ...base import BaseAgent
-
-import os
-
-import sys
-
-from ...agent_process import (
-    AgentProcess
-)
-
-import argparse
-
-from concurrent.futures import as_completed
-
-import numpy as np
-
-from src.tools.online.currency_converter import CurrencyConverterAPI
-
-from src.tools.online.wolfram_alpha import WolframAlpha
-
-import re
-class MathAgent(BaseAgent):
-    def __init__(self,
-                 agent_name,
-                 task_input,
-                 llm, agent_process_queue,
-                 agent_process_factory,
-                 log_mode: str
-        ):
-        BaseAgent.__init__(self, agent_name, task_input, llm, agent_process_queue, agent_process_factory, log_mode)
-        self.tool_list = {
-            # "wolfram_alpha": WolframAlpha(),
-            # "currenct_converter": CurrencyConverterAPI()
-        }
-        self.tool_check_max_fail_times = 10
-        self.tool_select_max_fail_times = 10
-        self.tool_calling_max_fail_times = 10
-        self.tool_info = "".join(self.config["tool_info"])
-
-    def load_flow(self):
-        return
-
-    def run(self):
-        prompt = ""
-        prefix = self.prefix
-        task_input = self.task_input
-        prompt += prefix
-        waiting_times = []
-        turnaround_times = []
-        task_input = "The task you need to solve is: " + task_input
-        prompt += task_input
-
-        self.logger.log(f"{task_input}\n", level="info")
-
-        # predefined steps
-        steps = [
-            "identify and outline the sub-problems that need to be solved as stepping stones toward the solution. ",
-            "solve each sub-problem. ",
-            "integrate the solutions to these sub-problems in the previous step to get the final solution. "
-        ]
-        for i, step in enumerate(steps):
-            prompt += f"\nIn step {i+1}, you need to {step}. Output should focus on current step and don't be verbose!"
-            # print(f"current prompt: {prompt}")
-
-            self.logger.log(f"Step {i+1}: {step}\n", level="info")
-            response, waiting_time, turnaround_time = self.get_response(prompt)
-
-            # print(f"Current step: {i+1}")
-            waiting_times.append(waiting_time)
-            turnaround_times.append(turnaround_time)
-            prompt += f"The solution to step {i+1} is: {response}\n"
-            self.logger.log(f"The solution to step {i+1}: {response}\n", level="info")
-
-        prompt += f"Given the interaction history: '{prompt}', integrate solutions in all steps to give a final answer, don't be verbose!"
-
-        final_result, waiting_time, turnaround_time = self.get_response(prompt)
-        waiting_times.append(waiting_time)
-        turnaround_times.append(turnaround_time)
-
-        self.set_status("done")
-
-        self.logger.log(
-            f"Done. Average waiting time: {np.mean(np.array(waiting_times))} seconds. Average turnaround time: {np.mean(np.array(turnaround_times))} seconds. \n",
-            level = "done"
-        )
-
-        self.logger.log(
-            f"{task_input} Final result is: {final_result}\n",
-            level="info"
-        )
-
-        return final_result
-
-
-if __name__ == "__main__":
-    parser = argparse.ArgumentParser(description='Run MathAgent')
-    parser.add_argument("--agent_name")
-    parser.add_argument("--task_input")
-
-    args = parser.parse_args()
-    agent = MathAgent(args.agent_name, args.task_input)
-
-    agent.run()
-    # thread_pool.submit(agent.run)
-    # agent.run()
diff --git a/src/agents/native_agents/narrative_agent/narrative_agent.py b/src/agents/native_agents/narrative_agent/narrative_agent.py
deleted file mode 100644
index dc88ed57..00000000
--- a/src/agents/native_agents/narrative_agent/narrative_agent.py
+++ /dev/null
@@ -1,87 +0,0 @@
-from ...base import BaseAgent
-
-import time
-
-from ...agent_process import (
-    AgentProcess
-)
-
-import numpy as np
-
-import argparse
-
-from concurrent.futures import as_completed
-
-class NarrativeAgent(BaseAgent):
-    def __init__(self,
-                 agent_name,
-                 task_input,
-                 llm,
-                 agent_process_queue,
-                 agent_process_factory,
-                 log_mode: str
-        ):
-        BaseAgent.__init__(self, agent_name, task_input, llm, agent_process_queue, agent_process_factory, log_mode)
-        # print(self.log_mode)
-
-    def run(self):
-        waiting_times = []
-        turnaround_times = []
-        prompt = ""
-        prefix = self.prefix
-        prompt += prefix
-        task_input = self.task_input
-        task_input = "The task you need to solve is: " + task_input
-        self.logger.log(f"{task_input}\n", level="info")
-        # print(f"[{self.agent_name}] {task_input}\n")
-
-        prompt += task_input
-
-        steps = [
-            "develop the story's setting and characters, establish a background and introduce the main characters.",
-            "given the background and characters, create situations that lead to the rising action, develop the climax with a significant turning point, and then move towards the resolution.",
-            "conclude the story and reflect on the narrative. This could involve tying up loose ends, resolving any conflicts, and providing a satisfactory conclusion for the characters."
-        ]
-
-        for i, step in enumerate(steps):
-            prompt += f"\nIn step {i+1}, you need to {step}. Output should focus on current step and don't be verbose!"
-
-            self.logger.log(f"Step {i+1}: {step}\n", level="info")
-
-            response, waiting_time, turnaround_time = self.get_response(prompt)
-            waiting_times.append(waiting_time)
-            turnaround_times.append(turnaround_time)
-
-            prompt += f"The solution to step {i+1} is: {response}\n"
-
-            self.logger.log(f"The solution to step {i+1}: {response}\n", level="info")
-
-            prompt += response
-
-        prompt += f"Given the interaction history: '{prompt}', integrate content in each step to give a full story, don't be verbose!"
-
-        final_result, waiting_time, turnaround_time = self.get_response(prompt)
-        waiting_times.append(waiting_time)
-        turnaround_times.append(turnaround_time)
-        # return res
-        # print(f"Average waiting time: {np.mean(np.array(waiting_times))}")
-        self.logger.log(
-            f"Done. Average waiting time: {np.mean(np.array(waiting_times))} seconds. Average turnaround time: {np.mean(np.array(turnaround_times))} seconds\n",
-            level="done"
-        )
-        # time.sleep(10)
-        self.set_status("done")
-
-        self.logger.log(f"{task_input} Final result is: {final_result}\n", level="info")
-
-        return final_result
-
-if __name__ == "__main__":
-    parser = argparse.ArgumentParser(description='Run NarrativeAgent')
-    parser.add_argument("--agent_name")
-    parser.add_argument("--task_input")
-
-    args = parser.parse_args()
-    agent = NarrativeAgent(args.agent_name, args.task_input)
-
-    agent.run()
diff --git a/src/agents/native_agents/rec_agent/rec_agent.py b/src/agents/native_agents/rec_agent/rec_agent.py
deleted file mode 100644
index 22da3c5e..00000000
--- a/src/agents/native_agents/rec_agent/rec_agent.py
+++ /dev/null
@@ -1,81 +0,0 @@
-from ...base import BaseAgent
-
-import time
-
-from ...agent_process import (
-    AgentProcess
-)
-
-import argparse
-
-from concurrent.futures import as_completed
-
-import numpy as np
-class RecAgent(BaseAgent):
-    def __init__(self,
-                 agent_name,
-                 task_input,
-                 llm,
-                 agent_process_queue,
-                 agent_process_factory,
-                 log_mode
-        ):
-        BaseAgent.__init__(self, agent_name, task_input, llm, agent_process_queue, agent_process_factory, log_mode)
-
-    def run(self):
-        prompt = ""
-        prefix = self.prefix
-        prompt += prefix
-        task_input = self.task_input
-        task_input = "The task you need to solve is: " + task_input
-        self.logger.log(f"{task_input}\n", level="info")
-        prompt += task_input
-        waiting_times = []
-        turnaround_times = []
-        steps = [
-            "give a general recommendation direction for users.",
-            "based on the above recommendation direction, give a recommendation list."
-        ]
-
-        for i, step in enumerate(steps):
-            prompt += f"\nIn step {i+1}, you need to {step}. Output should focus on current step and don't be verbose!"
-
-            self.logger.log(f"Step {i+1}: {step}\n", level="info")
-
-            response, waiting_time, turnaround_time = self.get_response(prompt)
-            waiting_times.append(waiting_time)
-            turnaround_times.append(turnaround_time)
-
-            prompt += f"The solution to step {i+1} is: {response}\n"
-
-            self.logger.log(f"The solution to step {i+1}: {response}\n", level="info")
-
-        prompt += f"Given the interaction history: '{prompt}', give a final recommendation list and explanations, don't be verbose!"
-
-        final_result, waiting_time, turnaround_time = self.get_response(prompt)
-        # time.sleep(10)
-        self.set_status("done")
-        # print(f"Average waiting time: {np.mean(np.array(waiting_times))}")
-        self.logger.log(
-            f"Done. Average waiting time: {np.mean(np.array(waiting_times))} seconds. Average turnaround time: {np.mean(np.array(turnaround_times))} seconds\n",
-            level="done"
-        )
-
-        self.logger.log(
-            f"{task_input} Final result is: {final_result}\n",
-            level="info"
-        )
-
-        return final_result
-
-    def parse_result(self, prompt):
-        return prompt
-
-if __name__ == "__main__":
-    parser = argparse.ArgumentParser(description='Run NarrativeAgent')
-    parser.add_argument("--agent_name")
-    parser.add_argument("--task_input")
-
-    args = parser.parse_args()
-    agent = RecAgent(args.agent_name, args.task_input)
-    agent.run()
diff --git a/src/command_executor.py b/src/command_executor.py
index 3eb03a8e..61d34fe2 100644
--- a/src/command_executor.py
+++ b/src/command_executor.py
@@ -2,7 +2,7 @@
 
 import sys
 
-from .agents.agent_factory import AgentFactory
+from openagi.src.agents.agent_factory import AgentFactory
 
 import subprocess
 
diff --git a/src/command_parser.py b/src/command_parser.py
index 2335cef3..39e63908 100644
--- a/src/command_parser.py
+++ b/src/command_parser.py
@@ -20,10 +20,10 @@ def parse(self, instruction):
         3) command_type command_name command_body
         """
         if ": " in instruction:
-            splited_command = instruction.split(": ")
+            splitted_command = instruction.split(": ")
 
-            command_head = splited_command[0].split(" ")
-            command_body = splited_command[-1]
+            command_head = splitted_command[0].split(" ")
+            command_body = splitted_command[-1]
 
             command_type = command_head[0]
             command_name = command_head[1]
diff --git a/src/llm_kernel/llm_classes/base_llm.py b/src/llm_kernel/llm_classes/base_llm.py
index f6c6b2b6..b0b9e9b9 100644
--- a/src/llm_kernel/llm_classes/base_llm.py
+++ b/src/llm_kernel/llm_classes/base_llm.py
@@ -1,4 +1,3 @@
-from openai import OpenAI
 import os
 import json
 import re
@@ -16,19 +15,22 @@ def __init__(self,
                  max_gpu_memory: dict = None,
                  eval_device: str = None,
                  max_new_tokens: int = 256,
-                 log_mode: str = "console"
+                 log_mode: str = "console",
+                 use_vllm: bool = False
         ):
         self.max_gpu_memory = max_gpu_memory
         self.eval_device = eval_device
         self.MAX_NEW_TOKENS = max_new_tokens
 
         self.log_mode = log_mode
-        # print(self.log_mode)
-        self.config = self.load_config(llm_name)
+
+        self.model_name = llm_name
+
+        self.use_vllm = use_vllm
+
         self.context_manager = SimpleContextManager()
-        self.open_sourced = self.config["open_sourced"]
-        self.model_type = self.config["model_type"]
-        self.model_name = self.config["model_name"]
+        self.open_sourced = self.check_opensourced(self.model_name)
+        self.model_type = self.check_model_type(self.model_name)
 
         self.load_llm_and_tokenizer()
         self.logger = self.setup_logger()
@@ -37,7 +39,6 @@ def __init__(self,
             "AIOS LLM successfully loaded.\n",
             level = "info"
         )
-        # print("AIOS LLM successfully loaded. \n", flush=True)
 
     def convert_map(self, map: dict) -> dict:
         new_map = {}
@@ -46,14 +47,20 @@ def convert_map(self, map: dict) -> dict:
         return new_map
 
     def load_config(self, llm_name):
-        # print(os.getcwd())
         config_file = os.path.join(os.getcwd(), "src", "llm_kernel", "llm_config/{}.json".format(llm_name))
         with open(config_file, "r") as f:
             config = json.load(f)
             return config
 
+    def check_opensourced(self, model_name):
+        pattern = r'(?i)\bgpt\b|\bclaude\b|\bgemini\b'
+        return re.search(pattern, model_name) is not None
+
+    def check_model_type(self, model_name):
+        # TODO add more model types
+        return "causal_lm"
+
     def setup_logger(self):
-        # print(self.log_mode)
         logger = LLMKernelLogger(self.model_name, self.log_mode)
         return logger
 
diff --git a/src/llm_kernel/llm_classes/gemini_llm.py b/src/llm_kernel/llm_classes/gemini_llm.py
index 4c6acae0..9f6548e2 100644
--- a/src/llm_kernel/llm_classes/gemini_llm.py
+++ b/src/llm_kernel/llm_classes/gemini_llm.py
@@ -1,13 +1,8 @@
 import re
 
-import torch
-from .constant import MODEL_CLASS
 from .base_llm import BaseLLMKernel
 import time
-from transformers import AutoTokenizer
 from ...utils.utils import get_from_env
-
-
 class GeminiLLM(BaseLLMKernel):
     def __init__(self, llm_name: str,
                  max_gpu_memory: dict = None,
@@ -45,7 +40,6 @@ def process(self,
             f"{agent_process.agent_name} is switched to executing.\n",
             level = "executing"
         )
-        # print(f"Prompt: {prompt}")
         outputs = self.model.generate_content(
             prompt
         )
diff --git a/src/llm_kernel/llm_classes/gpt_llm.py b/src/llm_kernel/llm_classes/gpt_llm.py
index c395806f..868adf02 100644
--- a/src/llm_kernel/llm_classes/gpt_llm.py
+++ b/src/llm_kernel/llm_classes/gpt_llm.py
@@ -28,7 +28,6 @@ def process(self,
         agent_process.set_status("executing")
         agent_process.set_start_time(time.time())
         prompt = agent_process.prompt
-        # print(f"Prompt: {prompt}")
         self.logger.log(
             f"{agent_process.agent_name} is switched to executing.\n",
             level = "executing"
@@ -39,7 +38,6 @@ def process(self,
                 {"role": "user", "content": prompt}
             ]
         )
-        # time.sleep(2) # set according to your request per minite
         agent_process.set_response(response.choices[0].message.content)
         agent_process.set_status("done")
         agent_process.set_end_time(time.time())
diff --git a/src/llm_kernel/llm_classes/open_llm.py b/src/llm_kernel/llm_classes/open_llm.py
index c00297e5..944bea8d 100644
--- a/src/llm_kernel/llm_classes/open_llm.py
+++ b/src/llm_kernel/llm_classes/open_llm.py
@@ -4,117 +4,121 @@
 import time
 from transformers import AutoTokenizer
 
-from click import progressbar
+from vllm import LLM
 
+import os
 class OpenLLM(BaseLLMKernel):
 
     def load_llm_and_tokenizer(self) -> None:
-        self.model_type = self.config["model_type"]
-        self.model_name = self.config["model_name"]
-        self.max_gpu_memory = self.convert_map(self.max_gpu_memory)
-        hf_token = self.config["hf_token"] if "hf_token" in self.config.keys() else None
-        cache_dir = self.config["cache_dir"] if "cache_dir" in self.config.keys() else None
-        self.model = MODEL_CLASS[self.model_type].from_pretrained(
-            self.model_name,
-            use_auth_token=hf_token,
-            cache_dir=cache_dir,
-            # torch_dtype=torch.float16,
-            # load_in_8bit = True,
-            device_map="auto",
-            max_memory=self.max_gpu_memory
-        )
-        # self.model = self.model.to(self.eval_device)
-        self.tokenizer = AutoTokenizer.from_pretrained(
-            self.model_name,
-            use_auth_token=hf_token,
-            cache_dir=cache_dir
-        )
-        # print(f"EOS token id: {self.model.config.eos_token_id}")
-        self.tokenizer.pad_token_id = self.tokenizer.eos_token_id
+        if self.use_vllm:
+            cuda_devices = self.max_gpu_memory.keys()
+            # cuda_devices = ",".join(self.max_gpu_memory.keys())
+            os.environ["CUDA_VISIBLE_DEVICES"] = ",".join(cuda_devices)
+            self.model = LLM(
+                model=self.model_name,
+                tensor_parallel_size=len(cuda_devices)
+            )
+            self.tokenizer = None
+
+        else:
+            self.max_gpu_memory = self.convert_map(self.max_gpu_memory)
+
+            self.model = MODEL_CLASS[self.model_type].from_pretrained(
+                self.model_name,
+                device_map="auto",
+                max_memory=self.max_gpu_memory
+            )
+            self.tokenizer = AutoTokenizer.from_pretrained(
+                self.model_name,
+            )
+            self.tokenizer.pad_token_id = self.tokenizer.eos_token_id
 
     def process(self,
                 agent_process,
                 temperature=0.0) -> None:
         agent_process.set_status("executing")
+        agent_process.set_start_time(time.time())
         self.logger.log(
             f"{agent_process.agent_name} is switched to executing.\n",
             level = "executing"
         )
 
-        if self.context_manager.check_restoration(agent_process.get_pid()):
-            restored_context = self.context_manager.gen_recover(
-                agent_process.get_pid()
-            )
-            start_idx = restored_context["start_idx"]
-            beams = restored_context["beams"]
-            beam_scores = restored_context["beam_scores"]
-            beam_attention_masks = restored_context["beam_attention_masks"]
-
-            outputs = self.generate(
-                search_mode = "beam_search",
-                beam_size = 1,
-                beams = beams,
-                beam_scores = beam_scores,
-                beam_attention_masks = beam_attention_masks,
-                max_new_tokens = self.MAX_NEW_TOKENS,
-                start_idx = start_idx,
-                timestamp = agent_process.get_time_limit()
-            )
-        else:
+        if self.use_vllm:
             prompt = agent_process.prompt
-            input_ids = self.tokenizer.encode(prompt, return_tensors="pt")
-            attention_masks = input_ids != self.tokenizer.pad_token_id
-            input_ids = input_ids.to(self.eval_device)
-            attention_masks = attention_masks.to(self.eval_device)
-
-            outputs = self.generate(
-                input_ids = input_ids,
-                attention_masks = attention_masks,
-                search_mode = "beam_search",
-                beam_size = 1,
-                max_new_tokens=self.MAX_NEW_TOKENS,
-                start_idx = 0,
-                timestamp = agent_process.get_time_limit()
-            )
-
-        output_ids = outputs["result"]
-
-        # print(f"Output ID: {output_ids}")
-        prompt = agent_process.prompt
-        result = self.tokenizer.decode(output_ids, skip_special_tokens=True)
-        # print(result)
-        result = result[len(prompt)+1: ]
-
-        if outputs["finished_flag"]: # finished flag is set as True
-
-            # print("Finished")
-            if self.context_manager.check_restoration(
-                agent_process.get_pid()):
-                self.context_manager.clear_restoration(
-                    agent_process.get_pid()
-                )
-            # print(f"{agent_process.agent_name} done: {result}")
+            outputs = self.model.generate(prompt)
+            result = outputs[0].outputs[0].text
             agent_process.set_response(result)
             agent_process.set_status("done")
 
         else:
-            # print(f"{agent_process.agent_name} suspended: {result}")
-            self.logger.log(
-                f"{agent_process.agent_name} is switched to suspending due to the reach of time limit ({agent_process.get_time_limit()}s).\n",
-                level = "suspending"
-            )
-            # print(f'{agent_process.get_pid()}: {outputs["start_idx"]}')
-            self.context_manager.gen_snapshot(
-                agent_process.get_pid(),
-                context = {
-                    "start_idx": outputs["start_idx"],
-                    "beams": outputs["beams"],
-                    "beam_scores": outputs["beam_scores"],
-                    "beam_attention_masks": outputs["beam_attention_masks"]
-                }
-            )
-            agent_process.set_response(result)
-            agent_process.set_status("suspending")
+            if self.context_manager.check_restoration(agent_process.get_pid()):
+                restored_context = self.context_manager.gen_recover(
+                    agent_process.get_pid()
+                )
+                start_idx = restored_context["start_idx"]
+                beams = restored_context["beams"]
+                beam_scores = restored_context["beam_scores"]
+                beam_attention_masks = restored_context["beam_attention_masks"]
+
+                outputs = self.generate(
+                    search_mode = "beam_search",
+                    beam_size = 1,
+                    beams = beams,
+                    beam_scores = beam_scores,
+                    beam_attention_masks = beam_attention_masks,
+                    max_new_tokens = self.MAX_NEW_TOKENS,
+                    start_idx = start_idx,
+                    timestamp = agent_process.get_time_limit()
+                )
+            else:
+                prompt = agent_process.prompt
+                input_ids = self.tokenizer.encode(prompt, return_tensors="pt")
+                attention_masks = input_ids != self.tokenizer.pad_token_id
+                input_ids = input_ids.to(self.eval_device)
+                attention_masks = attention_masks.to(self.eval_device)
+
+                outputs = self.generate(
+                    input_ids = input_ids,
+                    attention_masks = attention_masks,
+                    search_mode = "beam_search",
+                    beam_size = 1,
+                    max_new_tokens=self.MAX_NEW_TOKENS,
+                    start_idx = 0,
+                    timestamp = agent_process.get_time_limit()
+                )
+
+            output_ids = outputs["result"]
+
+            prompt = agent_process.prompt
+            result = self.tokenizer.decode(output_ids, skip_special_tokens=True)
+            result = result[len(prompt)+1: ]
+
+            if outputs["finished_flag"]: # finished flag is set as True
+
+                if self.context_manager.check_restoration(
+                    agent_process.get_pid()):
+                    self.context_manager.clear_restoration(
+                        agent_process.get_pid()
+                    )
+                agent_process.set_response(result)
+                agent_process.set_status("done")
+
+            else:
+                self.logger.log(
+                    f"{agent_process.agent_name} is switched to suspending due to the reach of time limit ({agent_process.get_time_limit()}s).\n",
+                    level = "suspending"
+                )
+                self.context_manager.gen_snapshot(
+                    agent_process.get_pid(),
+                    context = {
+                        "start_idx": outputs["start_idx"],
+                        "beams": outputs["beams"],
+                        "beam_scores": outputs["beam_scores"],
+                        "beam_attention_masks": outputs["beam_attention_masks"]
+                    }
+                )
+                agent_process.set_response(result)
+                agent_process.set_status("suspending")
 
         agent_process.set_end_time(time.time())
 
@@ -146,6 +150,34 @@ def generate(self,
         else:
             return NotImplementedError
 
+    def generate(self,
+                 input_ids: torch.Tensor = None,
+                 attention_masks: torch.Tensor = None,
+                 beams: torch.Tensor = None,
+                 beam_scores: torch.Tensor = None,
+                 beam_attention_masks: torch.Tensor = None,
+                 beam_size: int = None,
+                 max_new_tokens: int = None,
+                 search_mode: str = None,
+                 start_idx: int = 0,
+                 timestamp: int = None
+                 ):
+        if search_mode == "beam_search":
+            output_ids = self.beam_search(
+                input_ids = input_ids,
+                attention_masks = attention_masks,
+                beam_size = beam_size,
+                beams = beams,
+                beam_scores = beam_scores,
+                beam_attention_masks = beam_attention_masks,
+                max_new_tokens = max_new_tokens,
+                start_idx = start_idx,
+                timestamp = timestamp
+            )
+            return output_ids
+        else:
+            return NotImplementedError
+
     def beam_search(self,
                     input_ids: torch.Tensor = None,
                     attention_masks: torch.Tensor = None,
diff --git a/src/llm_kernel/llm_config/Llama-2-13b-chat-hf.json b/src/llm_kernel/llm_config/Llama-2-13b-chat-hf.json
deleted file mode 100644
index 2cde47b7..00000000
--- a/src/llm_kernel/llm_config/Llama-2-13b-chat-hf.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": true,
-    "model_name": "meta-llama/Llama-2-13b-chat-hf"
-}
diff --git a/src/llm_kernel/llm_config/Meta-Llama-3-70B-Instruct.json b/src/llm_kernel/llm_config/Meta-Llama-3-70B-Instruct.json
deleted file mode 100644
index f5cb9c35..00000000
--- a/src/llm_kernel/llm_config/Meta-Llama-3-70B-Instruct.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": true,
-    "model_name": "meta-llama/Meta-Llama-3-70B-Instruct"
-}
diff --git a/src/llm_kernel/llm_config/Meta-Llama-3-8B-Instruct.json b/src/llm_kernel/llm_config/Meta-Llama-3-8B-Instruct.json
deleted file mode 100644
index 8a96e2eb..00000000
--- a/src/llm_kernel/llm_config/Meta-Llama-3-8B-Instruct.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": true,
-    "model_name": "meta-llama/Meta-Llama-3-8B-Instruct"
-}
diff --git a/src/llm_kernel/llm_config/Mistral-7B-Instruct-v0.2.json b/src/llm_kernel/llm_config/Mistral-7B-Instruct-v0.2.json
deleted file mode 100644
index 083dd698..00000000
--- a/src/llm_kernel/llm_config/Mistral-7B-Instruct-v0.2.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": true,
-    "model_name": "mistralai/Mistral-7B-Instruct-v0.2"
-}
diff --git a/src/llm_kernel/llm_config/Mixtral-8x22B-Instruct-v0.1.json b/src/llm_kernel/llm_config/Mixtral-8x22B-Instruct-v0.1.json
deleted file mode 100644
index 6f062b66..00000000
--- a/src/llm_kernel/llm_config/Mixtral-8x22B-Instruct-v0.1.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": true,
-    "model_name": "mistralai/Mixtral-8x22B-Instruct-v0.1"
-}
diff --git a/src/llm_kernel/llm_config/Mixtral-8x7B-Instruct-v0.1.json b/src/llm_kernel/llm_config/Mixtral-8x7B-Instruct-v0.1.json
deleted file mode 100644
index b085b265..00000000
--- a/src/llm_kernel/llm_config/Mixtral-8x7B-Instruct-v0.1.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": true,
-    "model_name": "mistralai/Mixtral-8x7B-Instruct-v0.1"
-}
diff --git a/src/llm_kernel/llm_config/bedrock-anthropic-claude3-haiku.json b/src/llm_kernel/llm_config/bedrock-anthropic-claude3-haiku.json
deleted file mode 100644
index 47482635..00000000
--- a/src/llm_kernel/llm_config/bedrock-anthropic-claude3-haiku.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": false,
-    "model_name": "bedrock/anthropic.claude-3-haiku-20240307-v1:0"
-}
diff --git a/src/llm_kernel/llm_config/bedrock-anthropic-claude3-sonnet.json b/src/llm_kernel/llm_config/bedrock-anthropic-claude3-sonnet.json
deleted file mode 100644
index 46eb0227..00000000
--- a/src/llm_kernel/llm_config/bedrock-anthropic-claude3-sonnet.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": false,
-    "model_name": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0"
-}
diff --git a/src/llm_kernel/llm_config/gemini-pro.json b/src/llm_kernel/llm_config/gemini-pro.json
deleted file mode 100644
index 3e634e64..00000000
--- a/src/llm_kernel/llm_config/gemini-pro.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": false,
-    "model_name": "gemini-pro"
-}
diff --git a/src/llm_kernel/llm_config/gemma-1.1-7b-it.json b/src/llm_kernel/llm_config/gemma-1.1-7b-it.json
deleted file mode 100644
index cda9233b..00000000
--- a/src/llm_kernel/llm_config/gemma-1.1-7b-it.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": true,
-    "model_name": "google/gemma-1.1-7b-it"
-}
diff --git a/src/llm_kernel/llm_config/gemma-2b-it.json b/src/llm_kernel/llm_config/gemma-2b-it.json
deleted file mode 100644
index c58b13ca..00000000
--- a/src/llm_kernel/llm_config/gemma-2b-it.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": true,
-    "model_name": "google/gemma-2b-it"
-}
diff --git a/src/llm_kernel/llm_config/gpt-3.5-turbo.json b/src/llm_kernel/llm_config/gpt-3.5-turbo.json
deleted file mode 100644
index 542ad69b..00000000
--- a/src/llm_kernel/llm_config/gpt-3.5-turbo.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": false,
-    "model_name": "gpt-3.5-turbo"
-}
diff --git a/src/llm_kernel/llm_config/gpt-4.json b/src/llm_kernel/llm_config/gpt-4.json
deleted file mode 100644
index d7159e46..00000000
--- a/src/llm_kernel/llm_config/gpt-4.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
-    "model_type": "causal_lm",
-    "open_sourced": false,
-    "model_name": "gpt-4"
-}
diff --git a/src/llm_kernel/llms.py b/src/llm_kernel/llms.py
index 59431044..191c059d 100644
--- a/src/llm_kernel/llms.py
+++ b/src/llm_kernel/llms.py
@@ -7,7 +7,8 @@ def __init__(self,
                  max_gpu_memory: dict = None,
                  eval_device: str = None,
                  max_new_tokens: int = 256,
-                 log_mode: str = "console"
+                 log_mode: str = "console",
+                 use_vllm: bool = False
         ):
         # print(log_mode)
         if llm_name in MODEL_REGISTRY.keys():
@@ -21,7 +22,8 @@ def __init__(self,
                                  max_gpu_memory=max_gpu_memory,
                                  eval_device=eval_device,
                                  max_new_tokens=max_new_tokens,
-                                 log_mode=log_mode)
+                                 log_mode=log_mode,
+                                 use_vllm = use_vllm)
 
     def address_request(self,
                         agent_process,
diff --git a/src/scheduler/rr_scheduler.py b/src/scheduler/rr_scheduler.py
index 0a793f24..e26310e0 100644
--- a/src/scheduler/rr_scheduler.py
+++ b/src/scheduler/rr_scheduler.py
@@ -19,18 +19,12 @@ def __init__(self, llm, log_mode):
     def run(self):
         while self.active:
             try:
-                agent_process = self.agent_process_queue.get(block=True, timeout=0.2)
+                agent_process = self.agent_process_queue.get(block=True, timeout=0.05)
                 agent_process.set_time_limit(self.time_limit)
 
                 agent_process.set_status("executing")
                 # self.logger.log(f"{agent_process.agent_name} is switched to executing.\n", level="execute")
-                agent_process.set_start_time(time.time())
                 self.execute_request(agent_process)
-                # if agent_process.get_status() != "done":
-                #     self.logger.log(
-                #         f"{agent_process.agent_name} is switched to suspending due to the reach of time limit ({self.time_limit}s). \n",
-                #         level="suspend"
-                #     )
 
             except Empty:
                 pass
diff --git a/src/tools/__init__.py b/src/tools/__init__.py
deleted file mode 100644
index e69de29b..00000000
diff --git a/src/tools/base.py b/src/tools/base.py
deleted file mode 100644
index e25b8651..00000000
--- a/src/tools/base.py
+++ /dev/null
@@ -1,29 +0,0 @@
-from src.utils.utils import get_from_env
-import json
-
-class BaseTool:
-    """Base class for calling tool
-    """
-    def __init__(self):
-        pass
-
-    def run(self):
-        pass
-class BaseRapidAPITool(BaseTool):
-    """Base class for calling tool from RapidAPI hub: https://rapidapi.com/hub
-
-    Args:
-        BaseTool (_type_): _description_
-    """
-    def __init__(self):
-        super().__init__()
-        self.url: str = None
-        self.host_name: str = None
-        self.api_key: str = None
-
-
-    def run(self, prompt):
-        pass
-
-    def parse_result(self, response: json):
-        pass
diff --git a/src/tools/online/arxiv.py b/src/tools/online/arxiv.py
deleted file mode 100644
index 12218c53..00000000
--- a/src/tools/online/arxiv.py
+++ /dev/null
@@ -1,114 +0,0 @@
-import logging
-import os
-import re
-from typing import Any, Dict, Iterator, List, Optional
-
-from src.tools.base import BaseTool
-
-from pydantic import root_validator
-
-from langchain_core.documents import Document
-
-class Arxiv(BaseTool):
-    """Arxiv Tool, refactored from Langchain.
-
-    To use, you should have the ``arxiv`` python package installed.
-    https://lukasschwab.me/arxiv.py/index.html
-    This wrapper will use the Arxiv API to conduct searches and
-    fetch document summaries. By default, it will return the document summaries
-    of the top-k results.
-    If the query is in the form of arxiv identifier
-    (see https://info.arxiv.org/help/find/index.html), it will return the paper
-    corresponding to the arxiv identifier.
-    It limits the Document content by doc_content_chars_max.
-    Set doc_content_chars_max=None if you don't want to limit the content size.
-
-    Attributes:
-        top_k_results: number of the top-scored document used for the arxiv tool
-        ARXIV_MAX_QUERY_LENGTH: the cut limit on the query used for the arxiv tool.
-        load_max_docs: a limit to the number of loaded documents
-        load_all_available_meta:
-            if True: the `metadata` of the loaded Documents contains all available
-            meta info (see https://lukasschwab.me/arxiv.py/index.html#Result),
-            if False: the `metadata` contains only the published date, title,
-            authors and summary.
-        doc_content_chars_max: an optional cut limit for the length of a document's
-            content
-    """
-
-    arxiv_search: Any  #: :meta private:
-    arxiv_exceptions: Any  # :meta private:
-    top_k_results: int = 3
-    ARXIV_MAX_QUERY_LENGTH: int = 300
-    load_max_docs: int = 100
-    load_all_available_meta: bool = False
-    doc_content_chars_max: Optional[int] = 4000
-
-    def is_arxiv_identifier(self, query: str) -> bool:
-        """Check if a query is an arxiv identifier."""
-        arxiv_identifier_pattern = r"\d{2}(0[1-9]|1[0-2])\.\d{4,5}(v\d+|)|\d{7}.*"
-        for query_item in query[: self.ARXIV_MAX_QUERY_LENGTH].split():
-            match_result = re.match(arxiv_identifier_pattern, query_item)
-            if not match_result:
-                return False
-            assert match_result is not None
-            if not match_result.group(0) == query_item:
-                return False
-        return True
-
-    @root_validator(pre=True)
-    def validate_environment(cls, values: Dict) -> Dict:
-        """Validate that the python package exists in environment."""
-        try:
-            import tools.online.arxiv as arxiv
-
-            values["arxiv_search"] = arxiv.Search
-            values["arxiv_exceptions"] = (
-                arxiv.ArxivError,
-                arxiv.UnexpectedEmptyPageError,
-                arxiv.HTTPError,
-            )
-            values["arxiv_result"] = arxiv.Result
-        except ImportError:
-            raise ImportError(
-                "Could not import arxiv python package. "
-                "Please install it with `pip install arxiv`."
-            )
-        return values
-
-    def run(self, query: str) -> str:
-        """
-        Performs an arxiv search and A single string
-        with the publish date, title, authors, and summary
-        for each article separated by two newlines.
-
-        If an error occurs or no documents found, error text
-        is returned instead. Wrapper for
-        https://lukasschwab.me/arxiv.py/index.html#Search
-
-        Args:
-            query: a plaintext search query
-        """  # noqa: E501
-        try:
-            if self.is_arxiv_identifier(query):
-                results = self.arxiv_search(
-                    id_list=query.split(),
-                    max_results=self.top_k_results,
-                ).results()
-            else:
-                results = self.arxiv_search(  # type: ignore
-                    query[: self.ARXIV_MAX_QUERY_LENGTH], max_results=self.top_k_results
-                ).results()
-        except self.arxiv_exceptions as ex:
-            return f"Arxiv exception: {ex}"
-        docs = [
-            f"Published: {result.updated.date()}\n"
-            f"Title: {result.title}\n"
-            f"Authors: {', '.join(a.name for a in result.authors)}\n"
-            f"Summary: {result.summary}"
-            for result in results
-        ]
-        if docs:
-            return "\n\n".join(docs)[: self.doc_content_chars_max]
-        else:
-            return "No good Arxiv Result was found"
diff --git a/src/tools/online/bing_search.py b/src/tools/online/bing_search.py
deleted file mode 100644
index 00be8249..00000000
--- a/src/tools/online/bing_search.py
+++ /dev/null
@@ -1,56 +0,0 @@
-from typing import Dict, List
-
-import requests
-
-from ..base import BaseTool
-
-# from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-
-class BingSearch(BaseTool):
-    """Bing Search Tool, refactored from langchain.
-    In order to set this up, follow instructions at:
-    https://levelup.gitconnected.com/api-tutorial-how-to-use-bing-web-search-api-in-python-4165d5592a7e
-    """
-    def __init__(self):
-        super().__init__()
-        self.url = "https://api.bing.microsoft.com/v7.0/search" # temporarily
-        self.bing_subscription_key = get_from_env("BING_SUBSCRIPTION_KEY")
-        self.k: int = 10 # topk searched results
-        # search_kwargs: dict
-
-    def _bing_search_results(self, search_term: str, count: int) -> List[dict]:
-        headers = {"Ocp-Apim-Subscription-Key": self.bing_subscription_key}
-        params = {
-            "q": search_term,
-            "count": count,
-            "textDecorations": True,
-            "textFormat": "HTML",
-            # **self.search_kwargs,
-        }
-        response = requests.get(
-            self.bing_search_url,
-            headers=headers,
-            params=params,  # type: ignore
-        )
-        response.raise_for_status()
-        search_results = response.json()
-        if "webPages" in search_results:
-            return search_results["webPages"]["value"]
-        return []
-
-    def run(self, query: str) -> str:
-        """Run query through BingSearch and parse result."""
-        response = self._bing_search_results(query, count=self.k)
-        result = self.parse_result(response)
-        return result
-
-    def parse_result(self, response):
-        snippets = []
-        if len(response) == 0:
-            return "No good Bing Search Result was found"
-        for result in response:
-            snippets.append(result["snippet"])
-
-        return " ".join(snippets)
diff --git a/src/tools/online/currency_converter.py b/src/tools/online/currency_converter.py
deleted file mode 100644
index 7e71fbbe..00000000
--- a/src/tools/online/currency_converter.py
+++ /dev/null
@@ -1,57 +0,0 @@
-from ..base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-# from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-
-import requests
-
-import os
-
-class CurrencyConverterAPI(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://currency-converter5.p.rapidapi.com/currency/convert"
-        self.host_name = "currency-converter5.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params: dict):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        try:
-            self.query_string = {
-                "from": params["from"],
-                "to": params["to"],
-                "amount": params["amount"]
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for currency converter api. "
-                "Please make sure it contains two keys: 'from' and 'to' and 'amount"
-            )
-
-        # print(self.query_string)
-
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-
-        result = self.parse_result(response)
-        return result
-
-    def parse_result(self, response) -> str:
-        results = []
-        amount = str(response["amount"])
-        base = response["base_currency_name"]
-        rates = response["rates"]
-
-        for key, value in rates.items():
-            converted = value["currency_name"]
-            converted_rate = value["rate"]
-            converted_amount = value["rate_for_amount"]
-            results.append("Currency from " + base + " to " + converted + "is " + converted_rate + ".", )
-            results.append(amount + " " + base + "can be converted to " + converted_amount + " " + converted + ".")
-
-        return " ".join(results)
diff --git a/src/tools/online/google_place.py b/src/tools/online/google_place.py
deleted file mode 100644
index 03aca9b3..00000000
--- a/src/tools/online/google_place.py
+++ /dev/null
@@ -1,102 +0,0 @@
-import logging
-from typing import Any, Dict, Optional
-
-from ..base import BaseTool
-
-from src.utils.utils import get_from_env
-
-
-class GooglePlacesAPI(BaseTool):
-    """Google Places API, refactored from langchain.
-
-    To use, you should have the ``googlemaps`` python package installed,
-     **an API key for the google maps platform**,
-     and the environment variable ''GPLACES_API_KEY''
-     set with your API key , or pass 'gplaces_api_key'
-     as a named parameter to the constructor.
-
-    By default, this will return the all the results on the input query.
-     You can use the top_k_results argument to limit the number of results.
-
-    Example:
-        .. code-block:: python
-
-
-            from langchain_community.utilities import GooglePlacesAPIWrapper
-            gplaceapi = GooglePlacesAPIWrapper()
-    """
-    def __init__(self):
-        super().__init__()
-        self.gplaces_api_key = get_from_env("GPLACES_API_KEY")
-        self.google_map_client: Any  #: :meta private:
-        self.top_k_results: Optional[int] = None
-
-    def build_client(self):
-        """Validate that api key is in your environment variable."""
-        try:
-            import googlemaps
-
-            client = googlemaps.Client(self.gplaces_api_key)
-        except ImportError:
-            raise ImportError(
-                "Could not import googlemaps python package. "
-                "Please install it with `pip install googlemaps`."
-            )
-        return client
-
-    def run(self, query: str) -> str:
-        """Run Places search and get k number of places that exists that match."""
-        search_results = self.google_map_client.places(query)["results"]
-        num_to_return = len(search_results)
-
-        places = []
-
-        if num_to_return == 0:
-            return "Google Places did not find any places that match the description"
-
-        num_to_return = (
-            num_to_return
-            if self.top_k_results is None
-            else min(num_to_return, self.top_k_results)
-        )
-
-        for i in range(num_to_return):
-            result = search_results[i]
-            details = self.fetch_place_details(result["place_id"])
-
-            if details is not None:
-                places.append(details)
-
-        return "\n".join([f"{i+1}. {item}" for i, item in enumerate(places)])
-
-    def fetch_place_details(self, place_id: str) -> Optional[str]:
-        try:
-            place_details = self.google_map_client.place(place_id)
-            place_details["place_id"] = place_id
-            formatted_details = self.format_place_details(place_details)
-            return formatted_details
-        except Exception as e:
-            logging.error(f"An Error occurred while fetching place details: {e}")
-            return None
-
-    def format_place_details(self, place_details: Dict[str, Any]) -> Optional[str]:
-        try:
-            name = place_details.get("result", {}).get("name", "Unknown")
-            address = place_details.get("result", {}).get(
-                "formatted_address", "Unknown"
-            )
-            phone_number = place_details.get("result", {}).get(
-                "formatted_phone_number", "Unknown"
-            )
-            website = place_details.get("result", {}).get("website", "Unknown")
-            place_id = place_details.get("result", {}).get("place_id", "Unknown")
-
-            formatted_details = (
-                f"{name}\nAddress: {address}\n"
-                f"Google place ID: {place_id}\n"
-                f"Phone: {phone_number}\nWebsite: {website}\n\n"
-            )
-            return formatted_details
-        except Exception as e:
-            logging.error(f"An error occurred while formatting place details: {e}")
-            return None
diff --git a/src/tools/online/google_search.py b/src/tools/online/google_search.py
deleted file mode 100644
index 7dd468b1..00000000
--- a/src/tools/online/google_search.py
+++ /dev/null
@@ -1,89 +0,0 @@
-from typing import Any, Dict, List, Optional
-
-from ..base import BaseTool
-
-from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-
-class GoogleSearch(BaseTool):
-    """Google Search Tool, refactored from langchain.
-
-    Adapted from: Instructions adapted from https://stackoverflow.com/questions/
-    37083058/
-    programmatically-searching-google-in-python-using-custom-search
-
-    1. Install google-api-python-client
-    - If you don't already have a Google account, sign up.
-    - If you have never created a Google APIs Console project,
-    read the Managing Projects page and create a project in the Google API Console.
-    - Install the library using pip install google-api-python-client
-
-    2. Enable the Custom Search API
-    - Navigate to the APIs & Services→Dashboard panel in Cloud Console.
-    - Click Enable APIs and Services.
-    - Search for Custom Search API and click on it.
-    - Click Enable.
-    URL for it: https://console.cloud.google.com/apis/library/customsearch.googleapis
-    .com
-
-    3. To create an API key:
-    - Navigate to the APIs & Services → Credentials panel in Cloud Console.
-    - Select Create credentials, then select API key from the drop-down menu.
-    - The API key created dialog box displays your newly created key.
-    - You now have an API_KEY
-
-    Alternatively, you can just generate an API key here:
-    https://developers.google.com/custom-search/docs/paid_element#api_key
-
-    4. Setup Custom Search Engine so you can search the entire web
-    - Create a custom search engine here: https://programmablesearchengine.google.com/.
-    - In `What to search` to search, pick the `Search the entire Web` option.
-    After search engine is created, you can click on it and find `Search engine ID`
-      on the Overview page.
-
-    """
-    def __init__(self):
-        super().__init__()
-        self.google_api_key = get_from_env("GOOGLE_API_KEY")
-        self.google_cse_id = get_from_env("GOOGLE_CSE_ID")
-        self.k: int = 10 # topk searched results
-        self.search_engine = self.build_engine()
-        self.siterestrict: bool = False
-
-    def build_engine(self):
-        try:
-            from googleapiclient.discovery import build
-
-        except ImportError:
-            raise ImportError(
-                "google-api-python-client is not installed. "
-                "Please install it with `pip install google-api-python-client"
-                ">=2.100.0`"
-            )
-        engine = build("customsearch", "v1", developerKey=self.google_api_key)
-        return engine
-
-    def _google_search_results(self, search_term: str, **kwargs: Any) -> List[dict]:
-        cse = self.search_engine.cse()
-        if self.siterestrict:
-            cse = cse.siterestrict() # TODO add siterestrict verification
-        res = cse.list(q=search_term, cx=self.google_cse_id, **kwargs).execute()
-        return res.get("items", [])
-
-
-    def run(self, query: str) -> str:
-        """Run query through GoogleSearch and parse result."""
-        response = self._google_search_results(query, num=self.k)
-        result = self.parse_result(response)
-        return result
-
-    def parse_result(self, response):
-        snippets = []
-        if len(response) == 0:
-            return "No good Google Search Result was found"
-        for result in response:
-            if "snippet" in result:
-                snippets.append(result["snippet"])
-
-        return " ".join(snippets)
diff --git a/src/tools/online/hotels/search_hotel.py b/src/tools/online/hotels/search_hotel.py
deleted file mode 100644
index 5e9cd089..00000000
--- a/src/tools/online/hotels/search_hotel.py
+++ /dev/null
@@ -1,41 +0,0 @@
-from ...base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-# from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-
-import requests
-
-class HotelSearchAPI(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://hotels4.p.rapidapi.com/locations/v3/search"
-        self.host_name = "hotels4.p.rapidapi.com"
-
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        try:
-            self.query_string = {
-                "q": params["q"] # city
-                # "locale": params["locale"]
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for currency converter api. "
-                "Please make sure it contains the key 'q'"
-            )
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-
-        result = self.parse_result(response)
-        return result
-
-
-    def parse_result(self, response) -> str:
-        return "Completion hints are: " + ",".join(response["hints"].values())
diff --git a/src/tools/online/imdb/top_movie.py b/src/tools/online/imdb/top_movie.py
deleted file mode 100644
index 78ac477e..00000000
--- a/src/tools/online/imdb/top_movie.py
+++ /dev/null
@@ -1,34 +0,0 @@
-from src.tools.base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-# from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-
-import requests
-
-class TopMovieAPI(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://imdb-top-100-movies.p.rapidapi.com/"
-        self.host_name = "imdb-top-100-movies.p.rapidapi.com"
-
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, k):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        response = requests.get(self.url, headers=headers)
-        result = self.parse_result(response, k)
-        return result
-
-
-    def parse_result(self, response, k) -> str:
-        result = []
-        for i in range(k):
-            result.append(response[str(i)]["title"])
-
-        return f"Top {k} movies ranked by IMDB are: " + ",".join(result)
diff --git a/src/tools/online/imdb/top_series.py b/src/tools/online/imdb/top_series.py
deleted file mode 100644
index fa6b7f68..00000000
--- a/src/tools/online/imdb/top_series.py
+++ /dev/null
@@ -1,34 +0,0 @@
-from src.tools.base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-# from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-
-import requests
-
-class TopSeriesAPI(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://imdb-top-100-movies.p.rapidapi.com/series/"
-        self.host_name = "imdb-top-100-movies.p.rapidapi.com"
-
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, k):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        response = requests.get(self.url, headers=headers)
-        result = self.parse_result(response, k)
-        return result
-
-
-    def parse_result(self, response, k) -> str:
-        result = []
-        for i in range(k):
-            result.append(response[str(i)]["title"])
-
-        return f"Top {k} series ranked by IMDB are: " + ",".join(result)
diff --git a/src/tools/online/meteosource_weather/find_place.py b/src/tools/online/meteosource_weather/find_place.py
deleted file mode 100644
index 91136f34..00000000
--- a/src/tools/online/meteosource_weather/find_place.py
+++ /dev/null
@@ -1,48 +0,0 @@
-from ...base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-# from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-
-import requests
-
-class SongAutocompleteAPI(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://ai-weather-by-meteosource.p.rapidapi.com/find_places"
-        self.host_name = "ai-weather-by-meteosource.p.rapidapi.com"
-
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        try:
-            self.query_string = {
-                "text": params["text"],
-                "language": params["language"]
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for weather find place api. "
-                "Please make sure it contains two keys: 'text' and 'language'"
-            )
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-
-        result = self.parse_result(response)
-        return result
-
-
-    def parse_result(self, response) -> str:
-        location = [
-            response["radm_area1"],
-            response["adm_area2"],
-            response["country"],
-            response["lat"],
-            response["lon"]
-        ]
-        return f"Found place of {response["name"]}: " + ",".join(location)
diff --git a/src/tools/online/moon_phase_search.py b/src/tools/online/moon_phase_search.py
deleted file mode 100644
index 2df9aa00..00000000
--- a/src/tools/online/moon_phase_search.py
+++ /dev/null
@@ -1,31 +0,0 @@
-from ..base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-# from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-
-import requests
-
-import os
-
-class MoonPhaseSearch(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://moon-phase.p.rapidapi.com/basic"
-        self.host_name = "moon-phase.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        response = requests.get(self.url, headers=headers)
-
-        result = self.parse_result(response)
-        return result
-
-    def parse_result(self, response) -> str:
-        return f'Current moon phase is {response["phase_name"]}. It has {response["days_until_next_full_moon"]} until next full moon. It has {response["days_until_next_new_moon"]} until the next new moon.'
diff --git a/src/tools/online/shazam/song_auto_complete.py b/src/tools/online/shazam/song_auto_complete.py
deleted file mode 100644
index 6a1a5dae..00000000
--- a/src/tools/online/shazam/song_auto_complete.py
+++ /dev/null
@@ -1,41 +0,0 @@
-from ...base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-# from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-
-import requests
-
-class SongAutocompleteAPI(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://shazam.p.rapidapi.com/auto-complete"
-        self.host_name = "shazam.p.rapidapi.com"
-
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        try:
-            self.query_string = {
-                "term": params["term"],
-                "locale": params["locale"]
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for currency converter api. "
-                "Please make sure it contains two keys: 'term' and 'locale'"
-            )
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-
-        result = self.parse_result(response)
-        return result
-
-
-    def parse_result(self, response) -> str:
-        return "Completion hints are: " + ",".join(response["hints"].values())
diff --git a/src/tools/online/trip_advisor/flights.py b/src/tools/online/trip_advisor/flights.py
deleted file mode 100644
index a140b73d..00000000
--- a/src/tools/online/trip_advisor/flights.py
+++ /dev/null
@@ -1,89 +0,0 @@
-from ...base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-from src.utils.utils import get_from_env
-
-import requests
-
-import os
-
-import json
-
-class AirportSearch(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://tripadvisor16.p.rapidapi.com/api/v1/flights/searchAirport"
-        self.host_name = "tripadvisor16.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params: dict):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        try:
-            self.query_string = {
-                "query": params["query"],
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for tripadvisor search airport api. "
-                "Please make sure it contains the key: 'query'"
-            )
-
-        # print(self.query_string)
-
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-        return json.dumps(response)
-
-        # result = self.parse_result(response)
-        # return result
-
-    # def parse_result(self, response) -> str:
-    #     airports = []
-    #     for d in response["data"]:
-    #         print(d)
-    #         airports.append(f"{d["name"]}, airport code is {d["airportCode"]}")
-
-    #     return "Available airports are " + ";".join(airports)
-
-class FlightSearch(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://tripadvisor16.p.rapidapi.com/api/v1/flights/searchFlights"
-        self.host_name = "tripadvisor16.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params: dict):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-
-        try:
-            self.query_string = {
-                "sourceAirportCode": params["sourceAirportCode"],
-                "date": params["date"],
-                "destinationAirportCode": params["destinationAirportCode"],
-                "itineraryType": params["itineraryType"],
-                "sortOrder": params["sortOrder"],
-                "classOfService": params["classOfService"]
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for tripadvisor search flight api. "
-                "Please make sure it contains following required keys: "
-                "sourceAirportCode",
-                "destinationAirport",
-                "itineraryType",
-                "sortOrder",
-                "classOfServices"
-            )
-
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-        return json.dumps(response)
-
-
-    def parse_result(self, response) -> str:
-        raise NotImplementedError
diff --git a/src/tools/online/trip_advisor/hotels.py b/src/tools/online/trip_advisor/hotels.py
deleted file mode 100644
index ac59b9b1..00000000
--- a/src/tools/online/trip_advisor/hotels.py
+++ /dev/null
@@ -1,112 +0,0 @@
-from ...base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-from src.utils.utils import get_from_env
-
-import requests
-
-import os
-
-import json
-
-class HotelLocationSearch(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://tripadvisor16.p.rapidapi.com/api/v1/hotels/searchLocation"
-        self.host_name = "tripadvisor16.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params: dict):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        try:
-            self.query_string = {
-                "query": params["query"],
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for tripadvisor search hotel location api. "
-                "Please make sure it contains the key: 'query'"
-            )
-
-        # print(self.query_string)
-
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-
-        return json.dumps(response)
-
-    def parse_result(self, response) -> str:
-        raise NotImplementedError
-
-class HotelSearch(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://tripadvisor16.p.rapidapi.com/api/v1/hotels/searchHotels"
-        self.host_name = "tripadvisor16.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params: dict):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-
-        try:
-            self.query_string = {
-                "geoID": params["geoID"],
-                "checkIn": params["checkIn"],
-                "checkOut": params["checkOut"],
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for tripadvisor search hotel api. "
-                "Please make sure it contains following required keys: "
-                "geoID",
-                "checkIn",
-                "checkOut",
-            )
-
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-        return json.dumps(response)
-
-
-    def parse_result(self, response) -> str:
-        raise NotImplementedError
-
-class GetHotelDetails(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://tripadvisor16.p.rapidapi.com/api/v1/hotels/getHotelDetails"
-        self.host_name = "tripadvisor16.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params: dict):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-
-        try:
-            self.query_string = {
-                "id": params["id"],
-                "checkIn": params["checkIn"],
-                "checkOut": params["checkOut"],
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for tripadvisor get hotel details api. "
-                "Please make sure it contains following required keys: "
-                "id",
-                "checkIn",
-                "checkOut",
-            )
-
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-        return json.dumps(response)
-
-
-    def parse_result(self, response) -> str:
-        raise NotImplementedError
diff --git a/src/tools/online/trip_advisor/restaurant.py b/src/tools/online/trip_advisor/restaurant.py
deleted file mode 100644
index af35afa2..00000000
--- a/src/tools/online/trip_advisor/restaurant.py
+++ /dev/null
@@ -1,105 +0,0 @@
-from ...base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-from src.utils.utils import get_from_env
-
-import requests
-
-import os
-
-import json
-
-class RestaurantLocationSearch(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://tripadvisor16.p.rapidapi.com/api/v1/restaurant/searchLocation"
-        self.host_name = "tripadvisor16.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params: dict):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        try:
-            self.query_string = {
-                "query": params["query"],
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for tripadvisor search restaurant location api. "
-                "Please make sure it contains the key: 'query'"
-            )
-
-        # print(self.query_string)
-
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-
-        return json.dumps(response)
-
-    def parse_result(self, response) -> str:
-        raise NotImplementedError
-
-
-class RestaurantSearch(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://tripadvisor16.p.rapidapi.com/api/v1/restaurant/searchRestaurants"
-        self.host_name = "tripadvisor16.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params: dict):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-
-        try:
-            self.query_string = {
-                "locationId": params["locationID"]
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for tripadvisor search restaurant api. "
-                "Please make sure it contains following required keys: "
-                "locationID",
-            )
-
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-        return json.dumps(response)
-
-
-    def parse_result(self, response) -> str:
-        raise NotImplementedError
-
-
-class GetRestaurantDetails(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://tripadvisor16.p.rapidapi.com/api/v1/restaurant/getRestaurantDetails"
-        self.host_name = "tripadvisor16.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params: dict):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-
-        try:
-            self.query_string = {
-                "restaurantsID": params["restaurantsID"],
-            }
-        except ValueError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for tripadvisor get restaurant details api. "
-                "Please make sure it contains the key: 'restaurantsID'"
-            )
-
-        response = requests.get(self.url, headers=headers, params=self.query_string).json()
-        return json.dumps(response)
-
-
-    def parse_result(self, response) -> str:
-        raise NotImplementedError
diff --git a/src/tools/online/wikipedia.py b/src/tools/online/wikipedia.py
deleted file mode 100644
index c8409e43..00000000
--- a/src/tools/online/wikipedia.py
+++ /dev/null
@@ -1,118 +0,0 @@
-import logging
-from typing import Any, Dict, Iterator, List, Optional
-
-from ..base import BaseTool
-from langchain_core.documents import Document
-
-class Wikipedia(BaseTool):
-    """Wikipedia tool, refactored from langchain.
-
-    To use, you should have the ``wikipedia`` python package installed.
-    This wrapper will use the Wikipedia API to conduct searches and
-    fetch page summaries. By default, it will return the page summaries
-    of the top-k results.
-    It limits the Document content by doc_content_chars_max.
-    """
-    def __init__(self):
-        super().__init__()
-        self.WIKIPEDIA_MAX_QUERY_LENGTH = 300
-        self.top_k_results = 3
-        self.lang = "en"
-        self.load_all_available_meta: bool = False
-        self.doc_content_chars_max: int = 4000
-        self.wiki_client = self.build_client()
-
-    def build_client(self):
-        try:
-            import wikipedia
-
-            wikipedia.set_lang(self.lang)
-
-        except ImportError:
-            raise ImportError(
-                "Could not import wikipedia python package. "
-                "Please install it with `pip install wikipedia`."
-            )
-        return wikipedia
-
-    def run(self, query: str) -> str:
-        """Run Wikipedia search and get page summaries."""
-        page_titles = self.wiki_client.search(
-            query[:self.WIKIPEDIA_MAX_QUERY_LENGTH], results=self.top_k_results
-        )
-        summaries = []
-        for page_title in page_titles[: self.top_k_results]:
-            if wiki_page := self._fetch_page(page_title):
-                if summary := self._formatted_page_summary(page_title, wiki_page):
-                    summaries.append(summary)
-        if not summaries:
-            return "No good Wikipedia Search Result was found"
-        return "\n\n".join(summaries)[: self.doc_content_chars_max]
-
-    @staticmethod
-    def _formatted_page_summary(page_title: str, wiki_page: Any) -> Optional[str]:
-        return f"Page: {page_title}\nSummary: {wiki_page.summary}"
-
-    def _page_to_document(self, page_title: str, wiki_page: Any) -> Document:
-        main_meta = {
-            "title": page_title,
-            "summary": wiki_page.summary,
-            "source": wiki_page.url,
-        }
-        add_meta = (
-            {
-                "categories": wiki_page.categories,
-                "page_url": wiki_page.url,
-                "image_urls": wiki_page.images,
-                "related_titles": wiki_page.links,
-                "parent_id": wiki_page.parent_id,
-                "references": wiki_page.references,
-                "revision_id": wiki_page.revision_id,
-                "sections": wiki_page.sections,
-            }
-            if self.load_all_available_meta
-            else {}
-        )
-        doc = Document(
-            page_content=wiki_page.content[: self.doc_content_chars_max],
-            metadata={
-                **main_meta,
-                **add_meta,
-            },
-        )
-        return doc
-
-    def _fetch_page(self, page: str) -> Optional[str]:
-        try:
-            return self.wiki_client.page(title=page, auto_suggest=False)
-        except (
-            self.wiki_client.exceptions.PageError,
-            self.wiki_client.exceptions.DisambiguationError,
-        ):
-            return None
-
-    def load(self, query: str) -> List[Document]:
-        """
-        Run Wikipedia search and get the article text plus the meta information.
-        See
-
-        Returns: a list of documents.
-
-        """
-        return list(self.lazy_load(query))
-
-    def lazy_load(self, query: str) -> Iterator[Document]:
-        """
-        Run Wikipedia search and get the article text plus the meta information.
-        See
-
-        Returns: a list of documents.
-
-        """
-        page_titles = self.wiki_client.search(
-            query[:self.WIKIPEDIA_MAX_QUERY_LENGTH], results=self.top_k_results
-        )
-        for page_title in page_titles[: self.top_k_results]:
-            if wiki_page := self._fetch_page(page_title):
-                if doc := self._page_to_document(page_title, wiki_page):
-                    yield doc
diff --git a/src/tools/online/wolfram_alpha.py b/src/tools/online/wolfram_alpha.py
deleted file mode 100644
index 1d859c94..00000000
--- a/src/tools/online/wolfram_alpha.py
+++ /dev/null
@@ -1,53 +0,0 @@
-from typing import Any, Dict, Optional
-
-from ..base import BaseTool
-
-# import wolfram_alpha
-
-from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-class WolframAlpha(BaseTool):
-    """Wolfram Alpha Tool, refactored from langchain.
-
-    Docs for using:
-
-    1. Go to wolfram alpha and sign up for a developer account
-    2. Create an app and get your APP ID
-    3. Save your APP ID into WOLFRAM_ALPHA_APPID env variable
-    4. pip install wolframalpha
-
-    """
-    def __init__(self):
-        super().__init__()
-        self.wolfram_alpha_appid = get_from_env("WOLFRAM_ALPHA_APPID")
-        self.wolfram_client = self.build_client()
-
-
-    def build_client(self):
-        try:
-            import wolframalpha
-
-        except ImportError:
-            raise ImportError(
-                "wolframalpha is not installed. "
-                "Please install it with `pip install wolframalpha`"
-            )
-        client = wolframalpha.Client(self.wolfram_alpha_appid)
-        return client
-
-    def run(self, query: str) -> str:
-        """Run query through WolframAlpha and parse result."""
-        res = self.wolfram_client.query(query)
-
-        try:
-            assumption = next(res.pods).text
-            answer = next(res.results).text
-        except StopIteration:
-            return "Wolfram Alpha wasn't able to answer it"
-
-        if answer is None or answer == "":
-            # We don't want to return the assumption alone if answer is empty
-            return "No good Wolfram Alpha Result was found"
-        else:
-            return f"Assumption: {assumption} \nAnswer: {answer}"
diff --git a/src/tools/online/words_api.py b/src/tools/online/words_api.py
deleted file mode 100644
index 51d05b0b..00000000
--- a/src/tools/online/words_api.py
+++ /dev/null
@@ -1,40 +0,0 @@
-from ..base import BaseRapidAPITool
-
-from typing import Any, Dict, List, Optional
-
-# from pydantic import root_validator
-
-from src.utils.utils import get_from_env
-
-import requests
-
-import os
-
-class WordsAPI(BaseRapidAPITool):
-    def __init__(self):
-        super().__init__()
-        self.url = "https://wordsapiv1.p.rapidapi.com/words/"
-        self.host_name = "wordsapiv1.p.rapidapi.com"
-        self.api_key = get_from_env("RAPID_API_KEY")
-
-    def run(self, params):
-        headers = {
-            "X-RapidAPI-Key": self.api_key,
-            "X-RapidAPI-Host": self.host_name
-        }
-        try:
-            self.word = params["word"]
-            self.api_name = params["api_name"]
-        except KeyError:
-            raise KeyError(
-                "The keys in params do not match the excepted keys in params for words api. "
-                "Please make sure it contains two keys: 'words' and 'api_name'"
-            )
-        self.url = os.path.join(self.url, self.word, self.api_name)
-
-        response = requests.get(self.url, headers=headers).json()
-        result = self.parse_result(response)
-        return result
-
-    def parse_result(self, response) -> str:
-        return response["word"] + " " + self.api_name + " [" + ",".join(response[self.api_name]) + "]"
diff --git a/src/utils/calculator.py b/src/utils/calculator.py
new file mode 100644
index 00000000..d29b49dc
--- /dev/null
+++ b/src/utils/calculator.py
@@ -0,0 +1,132 @@
+from concurrent.futures import ThreadPoolExecutor, as_completed
+import numpy as np
+import os
+
+
+def load_agent_tasks(agent_name):
+    file_path = os.path.join(os.getcwd(), "agent_configs", f"{agent_name}_task.txt")
+    with open(file_path) as f:
+        task_inputs = f.readlines()
+        return task_inputs
+
+
+def get_numbers_concurrent(agent_list, agent_factory, agent_thread_pool):
+    agent_tasks = []
+    for agent_name, agent_num in agent_list:
+        task_input = load_agent_tasks(agent_name=agent_name)[0]
+        for i in range(agent_num):
+            agent_task = agent_thread_pool.submit(
+                agent_factory.run_agent,
+                agent_name,
+                task_input
+            )
+            agent_tasks.append(agent_task)
+    stats = {
+        'turnaround_times': [],
+        'waiting_times': [],
+        'request_waiting_times': [],
+        'request_turnaround_times': []
+    }
+
+    # Collect data
+    for result in as_completed(agent_tasks):
+        output = result.result()
+        stats['turnaround_times'].append(output["agent_turnaround_time"])
+        stats['waiting_times'].append(output["agent_waiting_time"])
+        stats['request_waiting_times'].extend(output["request_waiting_times"])
+        stats['request_turnaround_times'].extend(output["request_turnaround_times"])
+
+    # Compute averages and percentiles
+    def compute_metrics(data):
+        return {
+            'avg': np.mean(data),
+            'p90': np.percentile(data, 90),
+            'p99': np.percentile(data, 99)
+        }
+
+    metrics = {
+        'agent_turnaround_time': compute_metrics(stats['turnaround_times']),
+        'agent_waiting_time': compute_metrics(stats['waiting_times']),
+        'request_waiting_time': compute_metrics(stats['request_waiting_times']),
+        'request_turnaround_time': compute_metrics(stats['request_turnaround_times'])
+    }
+
+    return metrics
+
+
+def get_numbers_sequential(agent_list, agent_factory):
+    stats = {
+        'turnaround_times': [],
+        'waiting_times': [],
+        'request_waiting_times': [],
+        'request_turnaround_times': []
+    }
+
+    accumulated_time = 0
+    for agent_name, agent_num in agent_list:
+        task_input = load_agent_tasks(agent_name=agent_name)[0]  # Assuming first task relevant
+        for _ in range(agent_num):
+            output = agent_factory.run_agent(agent_name=agent_name, task_input=task_input)
+
+            agent_turnaround_time = output["agent_turnaround_time"] + accumulated_time
+            agent_waiting_time = output["agent_waiting_time"] + accumulated_time
+            rounds = output["rounds"]
+
+            # Adjust times by the accumulated time
+            request_waiting_times = output["request_waiting_times"]
+            request_turnaround_times = output["request_turnaround_times"]
+            request_waiting_times[0] += accumulated_time
+            request_turnaround_times[0] += accumulated_time
+
+            # Append to lists
+            stats['turnaround_times'].append(agent_turnaround_time)
+            stats['waiting_times'].append(agent_waiting_time)
+            stats['request_waiting_times'].extend(request_waiting_times)
+            stats['request_turnaround_times'].extend(request_turnaround_times)
+
+            accumulated_time += (agent_turnaround_time - agent_waiting_time)
+
+    # Compute metrics for each category
+    def compute_metrics(data):
+        return {
+            'avg': np.mean(data),
+            'p90': np.percentile(data, 90),
+            'p99': np.percentile(data, 99)
+        }
+
+    metrics = {
+        'agent_turnaround_time': compute_metrics(stats['turnaround_times']),
+        'agent_waiting_time': compute_metrics(stats['waiting_times']),
+        'request_waiting_time': compute_metrics(stats['request_waiting_times']),
+        'request_turnaround_time': compute_metrics(stats['request_turnaround_times'])
+    }
+
+    return metrics
+
+
+def calculate_improvement(sequential, concurrent):
+    return (sequential - concurrent) / sequential
+
+
+def comparison(concurrent_metrics, sequential_metrics):
+    # Print analysis
+    print("**** Improvement Analysis Starts ****")
+
+    # Calculate and print improvements for averages and percentiles
+    improvements = {}
+    for key in concurrent_metrics.keys():
+        concurrent_avg = concurrent_metrics[key]['avg']
+        sequential_avg = sequential_metrics[key]['avg']
+        improvements[key + '_avg_improv'] = calculate_improvement(sequential_avg, concurrent_avg) * 100
+
+        concurrent_p90 = concurrent_metrics[key]['p90']
+        sequential_p90 = sequential_metrics[key]['p90']
+        improvements[key + '_p90_improv'] = calculate_improvement(sequential_p90, concurrent_p90) * 100
+
+        concurrent_p99 = concurrent_metrics[key]['p99']
+        sequential_p99 = sequential_metrics[key]['p99']
+        improvements[key + '_p99_improv'] = calculate_improvement(sequential_p99, concurrent_p99) * 100
+
+    # Print improvements
+    for improv_key, improv_value in improvements.items():
+        print(f"Improvement of {improv_key}: {improv_value:.2f}%")


Test patch:
diff --git a/tests/test_llms.py b/tests/test_llms.py
index c064ae25..44f2629d 100644
--- a/tests/test_llms.py
+++ b/tests/test_llms.py
@@ -4,7 +4,7 @@
 
 from src.llm_kernel.llms import LLMKernel
 
-from src.agents.agent_process import AgentProcess
+from openagi.src.agents.agent_process import AgentProcess
 
 from src.context.simple_context import SimpleContextManager