Upload 210 files

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/result_nvidia_chart.gif filter=lfs diff=lfs merge=lfs -text
+Llama2-Code-Interpreter/assets/result_nvidia_chart.gif filter=lfs diff=lfs merge=lfs -text
+OpenCodeInterpreter/data_collection/Local-Code-Interpreter/example_img/save_to_notebook_demo.gif filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,48 @@
+# Ignore .ckpt files
+ckpt
+
+# Ignore Python compiled files
+__pycache__/
+*.py[cod]
+
+# Ignore Python virtual environment
+venv/
+
+# Ignore Jupyter notebook checkpoints
+.ipynb_checkpoints/
+.git/
+.vscode/
+
+# Ignore .DS_Store on MacOS
+.DS_Store
+
+rilab_key.txt
+gpt4_custom_code_interpreter/rilab_key.txt
+openai_api_key.txt
+
+gpt4_custom_code_interpreter/
+tmp/
+output/
+wandb/
+
+utils/const.py
+utils/hf_model_upload.py
+gpt_data_gen/
+*.json
+*.txt
+*.sh
+*.pt 
+*.pth 
+*.ckpt
+*.tokenizer
+
+# eval data
+eval/ds1000_data
+eval/grade-school-math 
+
+# gradio features
+chatbot_feat.py
+chatbot_feat2.py
+gradio_test.py
+
+

Llama2-Code-Interpreter/.gitignore

@@ -0,0 +1,48 @@
+# Ignore .ckpt files
+ckpt
+
+# Ignore Python compiled files
+__pycache__/
+*.py[cod]
+
+# Ignore Python virtual environment
+venv/
+
+# Ignore Jupyter notebook checkpoints
+.ipynb_checkpoints/
+.git/
+.vscode/
+
+# Ignore .DS_Store on MacOS
+.DS_Store
+
+rilab_key.txt
+gpt4_custom_code_interpreter/rilab_key.txt
+openai_api_key.txt
+
+gpt4_custom_code_interpreter/
+tmp/
+output/
+wandb/
+
+utils/const.py
+utils/hf_model_upload.py
+gpt_data_gen/
+*.json
+*.txt
+*.sh
+*.pt 
+*.pth 
+*.ckpt
+*.tokenizer
+
+# eval data
+eval/ds1000_data
+eval/grade-school-math 
+
+# gradio features
+chatbot_feat.py
+chatbot_feat2.py
+gradio_test.py
+
+

Llama2-Code-Interpreter/README.md

@@ -0,0 +1,131 @@
+<p align="center" width="100%">
+<img src="/assets/logo2.png" alt="llama2 code interprerter icon" style="width: 200px; height:200px; display: block; margin: auto; border-radius: 50%;">
+</p>
+
+
+# Llama2 Code Interpreter
+
+<p align="center">
+🤗 <a href="https://huggingface.co/Seungyoun/codellama-7b-instruct-pad" target="_blank">CodeLlama 7B Finetuned Model (HF)</a> 
+</p>
+
+
+[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/release/python-390/)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+
+This project allows LLM to generate code, execute it, receive feedback, debug, and answer questions based on the whole process. It is designed to be intuitive and versatile, capable of dealing with multiple languages and frameworks.
+
+[The purpose and direction of the project](https://github.com/SeungyounShin/Llama2-Code-Interpreter/wiki)
+
+## Quick Start
+
+**Run the Gradio App**:
+   ```bash
+   python3 chatbot.py --path Seungyoun/codellama-7b-instruct-pad
+   ```
+
+## News
+
+- 🔥🔥🔥[2023/08/27] We're thrilled to announce that our **[🤗 Llama2 Code Interpreter-7B](https://huggingface.co/Seungyoun/codellama-7b-instruct-pad) (Finetuned from [CodeLlama-7B-Instruct](https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf))** model achieved a remarkable **70.12pass@1** on the [HumanEval Benchmarks](https://github.com/openai/human-eval).
+
+
+**HumanEval**
+
+| Model                          | Score(pass@1)  |
+|-------------------------------|--------|
+| Codellama instruct 7b         | 34.8%  |
+| Codellama instruct 7b - finetuning | 70.12% |
+
+**GSM8K**
+
+| Model                          | Score  |
+|-------------------------------|--------|
+| Code Llama 7B                 | 13%    |
+| Code Llama 13B                | 20.8%  |
+| Codellama instruct 7b - finetuning | 28%    |
+
+
+## 🌟 Key Features
+
+- [x] 🚀 **Code Generation and Execution**: Llama2 is capable of generating code, which it then automatically identifies and executes within its generated code blocks.
+- [x] Monitors and retains Python variables that were used in previously executed code blocks.
+- [x] 🌟 At the moment, my focus is on "Data development for GPT-4 code interpretation" and "Enhancing the model using this data". For more details, check out the [feat/finetuning branch](https://github.com/SeungyounShin/Llama2-Code-Interpreter/tree/feat/finetuning) in our repository.
+- [x] 🌟 CodeLlama Support [CodeLlama2](https://github.com/facebookresearch/codellama)
+
+## Examples
+
+---
+<div align="center">
+
+***Llama2 in Action***
+
+<p align="center" width="100%">
+<img src="assets/result_nvidia_chart.gif" alt="example1_president_search_with_code" style="width: 600px; display: block; margin: auto; border-radius: 50%;">
+</p>
+
+</div>
+
+In the GIF, Llama2 is seen in action. A user types in the request: `Plot Nvidia 90 days chart.` Llama2, an advanced code interpreter fine-tuned on a select dataset, swiftly queries `Yahoo Finance`. Moments later, it fetches the latest Nvidia stock prices from the past 90 days. Using `Matplotlib`, Llama2 then generates a clear and detailed stock price chart for Nvidia, showcasing its performance over the given period.
+
+
+
+## Installation
+
+1. **Clone the Repository (if you haven't already)**:
+   ```bash
+   git clone https://github.com/SeungyounShin/Llama2-Code-Interpreter.git
+   cd Llama2-Code-Interpreter
+   ```
+
+2. **Install the required dependencies:**
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+---
+
+### Run App with GPT4 finetunned Llama Model
+
+To start interacting with Llama2 via the Gradio UI using  `codellama-7b-instruct-pad`, follow the steps below:
+
+
+2. **Run the Gradio App**:
+   ```bash
+   python3 chatbot.py --path Seungyoun/codellama-7b-instruct-pad
+   ```
+
+For those who want to use other models:
+
+### General Instructions to Run App
+
+To start interacting with Llama2 via the Gradio UI using other models:
+
+1. **Run the Command**:
+   ```bash
+   python3 chatbot.py --model_path <your-model-path>
+   ```
+
+Replace `<your-model-path>` with the path to the model file you wish to use. A recommended model for chat interactions is `meta-llama/Llama-2-13b-chat`.
+
+## Contributions
+
+Contributions, issues, and feature requests are welcome! Feel free to check [issues page](https://github.com/SeungyounShin/Llama2-Code-Interpreter/issues). 
+
+## License
+
+Distributed under the MIT License. See `LICENSE` for more information.
+
+## Contact
+
+Seungyoun, Shin - 2022021568@korea.ac.kr
+
+## Acknowledgement
+
+Here are some relevant and related projects that have contributed to the development of this work:
+
+1. **llama2** : [GitHub Repository](https://github.com/facebookresearch/llama)
+2. **yet-another-gpt-tutorial** : [GitHub Repository](https://github.com/sjchoi86/yet-another-gpt-tutorial/tree/main)
+
+These projects have been instrumental in providing valuable insights and resources, and their contributions are highly appreciated.
+
+---

Llama2-Code-Interpreter/chatbot.py

@@ -0,0 +1,238 @@
+import gradio as gr
+import random
+import time, os
+import copy
+import re
+
+import torch
+from rich.console import Console
+from rich.table import Table
+from datetime import datetime
+
+from threading import Thread
+from typing import Optional
+from transformers import TextIteratorStreamer
+
+from utils.special_tok_llama2 import (
+    B_CODE,
+    E_CODE,
+    B_RESULT,
+    E_RESULT,
+    B_INST,
+    E_INST,
+    B_SYS,
+    E_SYS,
+    DEFAULT_PAD_TOKEN,
+    DEFAULT_BOS_TOKEN,
+    DEFAULT_EOS_TOKEN,
+    DEFAULT_UNK_TOKEN,
+    IGNORE_INDEX,
+)
+
+from finetuning.conversation_template import (
+    json_to_code_result_tok_temp,
+    msg_to_code_result_tok_temp,
+)
+
+import warnings
+
+warnings.filterwarnings("ignore", category=UserWarning, module="transformers")
+os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
+
+
+from code_interpreter.LlamaCodeInterpreter import LlamaCodeInterpreter
+
+
+class StreamingLlamaCodeInterpreter(LlamaCodeInterpreter):
+    streamer: Optional[TextIteratorStreamer] = None
+
+    # overwirte generate function
+    @torch.inference_mode()
+    def generate(
+        self,
+        prompt: str = "[INST]\n###User : hi\n###Assistant :",
+        max_new_tokens=512,
+        do_sample: bool = True,
+        use_cache: bool = True,
+        top_p: float = 0.95,
+        temperature: float = 0.1,
+        top_k: int = 50,
+        repetition_penalty: float = 1.0,
+    ) -> str:
+        # Get the model and tokenizer, and tokenize the user text.
+
+        self.streamer = TextIteratorStreamer(
+            self.tokenizer, skip_prompt=True, Timeout=5
+        )
+
+        input_prompt = copy.deepcopy(prompt)
+        inputs = self.tokenizer([prompt], return_tensors="pt")
+        input_tokens_shape = inputs["input_ids"].shape[-1]
+
+        eos_token_id = self.tokenizer.convert_tokens_to_ids(DEFAULT_EOS_TOKEN)
+        e_code_token_id = self.tokenizer.convert_tokens_to_ids(E_CODE)
+
+        kwargs = dict(
+            **inputs,
+            max_new_tokens=max_new_tokens,
+            do_sample=do_sample,
+            top_p=top_p,
+            temperature=temperature,
+            use_cache=use_cache,
+            top_k=top_k,
+            repetition_penalty=repetition_penalty,
+            eos_token_id=[
+                eos_token_id,
+                e_code_token_id,
+            ],  # Stop generation at either EOS or E_CODE token
+            streamer=self.streamer,
+        )
+
+        thread = Thread(target=self.model.generate, kwargs=kwargs)
+        thread.start()
+
+        return ""
+
+
+def change_markdown_image(text: str):
+    modified_text = re.sub(r"!\[(.*?)\]\(\'(.*?)\'\)", r"![\1](/file=\2)", text)
+    return modified_text
+
+
+def gradio_launch(model_path: str, load_in_4bit: bool = True, MAX_TRY: int = 5):
+    with gr.Blocks(theme=gr.themes.Monochrome()) as demo:
+        chatbot = gr.Chatbot(height=820, avatar_images="./assets/logo2.png")
+        msg = gr.Textbox()
+        clear = gr.Button("Clear")
+
+        interpreter = StreamingLlamaCodeInterpreter(
+            model_path=model_path, load_in_4bit=load_in_4bit
+        )
+
+        def bot(history):
+            user_message = history[-1][0]
+
+            interpreter.dialog.append({"role": "user", "content": user_message})
+
+            print(f"###User : [bold]{user_message}[bold]")
+            # print(f"###Assistant : ")
+
+            # setup
+            HAS_CODE = False  # For now
+            INST_END_TOK_FLAG = False
+            full_generated_text = ""
+            prompt = interpreter.dialog_to_prompt(dialog=interpreter.dialog)
+            start_prompt = copy.deepcopy(prompt)
+            prompt = f"{prompt} {E_INST}"
+
+            _ = interpreter.generate(prompt)
+            history[-1][1] = ""
+            generated_text = ""
+            for character in interpreter.streamer:
+                history[-1][1] += character
+                generated_text += character
+                yield history
+
+            full_generated_text += generated_text
+            HAS_CODE, generated_code_block = interpreter.extract_code_blocks(
+                generated_text
+            )
+
+            attempt = 1
+            while HAS_CODE:
+                if attempt > MAX_TRY:
+                    break
+                # if no code then doesn't have to execute it
+
+                # refine code block for history
+                history[-1][1] = (
+                    history[-1][1]
+                    .replace(f"{B_CODE}", "\n```python\n")
+                    .replace(f"{E_CODE}", "\n```\n")
+                )
+                history[-1][1] = change_markdown_image(history[-1][1])
+                yield history
+
+                # replace unknown thing to none ''
+                generated_code_block = generated_code_block.replace(
+                    "<unk>_", ""
+                ).replace("<unk>", "")
+
+                (
+                    code_block_output,
+                    error_flag,
+                ) = interpreter.execute_code_and_return_output(
+                    f"{generated_code_block}"
+                )
+                code_block_output = interpreter.clean_code_output(code_block_output)
+                generated_text = (
+                    f"{generated_text}\n{B_RESULT}\n{code_block_output}\n{E_RESULT}\n"
+                )
+                full_generated_text += (
+                    f"\n{B_RESULT}\n{code_block_output}\n{E_RESULT}\n"
+                )
+
+                # append code output
+                history[-1][1] += f"\n```RESULT\n{code_block_output}\n```\n"
+                history[-1][1] = change_markdown_image(history[-1][1])
+                yield history
+
+                prompt = f"{prompt} {generated_text}"
+
+                _ = interpreter.generate(prompt)
+                for character in interpreter.streamer:
+                    history[-1][1] += character
+                    generated_text += character
+                    history[-1][1] = change_markdown_image(history[-1][1])
+                    yield history
+
+                HAS_CODE, generated_code_block = interpreter.extract_code_blocks(
+                    generated_text
+                )
+
+                if generated_text.endswith("</s>"):
+                    break
+
+                attempt += 1
+
+            interpreter.dialog.append(
+                {
+                    "role": "assistant",
+                    "content": generated_text.replace("<unk>_", "")
+                    .replace("<unk>", "")
+                    .replace("</s>", ""),
+                }
+            )
+
+            print("----------\n" * 2)
+            print(interpreter.dialog)
+            print("----------\n" * 2)
+
+            return history[-1][1]
+
+        def user(user_message, history):
+            return "", history + [[user_message, None]]
+
+        msg.submit(user, [msg, chatbot], [msg, chatbot], queue=False).then(
+            bot, chatbot, chatbot
+        )
+        clear.click(lambda: None, None, chatbot, queue=False)
+
+    demo.queue()
+    demo.launch()
+
+
+if __name__ == "__main__":
+    import argparse
+
+    parser = argparse.ArgumentParser(description="Process path for LLAMA2_FINETUNEED.")
+    parser.add_argument(
+        "--path",
+        type=str,
+        required=True,
+        help="Path to the finetuned LLAMA2 model.",
+        default="./output/llama-2-7b-codellama-ci",
+    )
+    args = parser.parse_args()
+
+    gradio_launch(model_path=args.path, load_in_4bit=True)

Llama2-Code-Interpreter/code_interpreter/BaseCodeInterpreter.py

@@ -0,0 +1,59 @@
+import json
+import os
+import sys
+import time
+import re
+from pathlib import Path
+from typing import List, Literal, Optional, Tuple, TypedDict, Dict
+
+prj_root_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+sys.path.append(prj_root_path)
+
+import torch
+import transformers
+from transformers import LlamaForCausalLM, LlamaTokenizer
+
+import nbformat
+
+# from nbconvert.preprocessors import ExecutePreprocessor
+# from nbconvert.preprocessors.execute import CellExecutionError
+
+from utils.const import *
+from utils.cleaner import clean_error_msg
+from colorama import init, Fore, Style
+from rich.markdown import Markdown
+import base64
+
+import openai
+from retrying import retry
+import logging
+from termcolor import colored
+from code_interpreter.JuypyterClient import JupyterNotebook
+
+
+class BaseCodeInterpreter:
+    def __init__(self):
+        self.dialog = [
+            {
+                "role": "system",
+                "content": CODE_INTERPRETER_SYSTEM_PROMPT,
+            },
+            # {"role": "user", "content": "How can I use BeautifulSoup to scrape a website and extract all the URLs on a page?"},
+            # {"role": "assistant", "content": "I think I need to use beatifulsoup to find current korean president,"}
+        ]
+
+        self.nb = JupyterNotebook()
+
+    @staticmethod
+    def extract_code_blocks(text: str):
+        pattern = r"```(?:python\n)?(.*?)```"  # Match optional 'python\n' but don't capture it
+        code_blocks = re.findall(pattern, text, re.DOTALL)
+        return [block.strip() for block in code_blocks]
+
+    @staticmethod
+    def parse_last_answer(text: str) -> str:
+        return text.split(E_INST)[-1]
+
+    def execute_code_and_return_output(self, code_str: str):
+        outputs, error_flag = self.nb.add_and_run(code_str)
+        return outputs, error_flag

Llama2-Code-Interpreter/code_interpreter/GPTCodeInterpreter.py

@@ -0,0 +1,234 @@
+import json
+import os
+import sys
+import time
+import re
+from pathlib import Path
+from typing import List, Literal, Optional, Tuple, TypedDict, Dict
+
+# Get the path from environment variable
+prj_root_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+sys.path.append(prj_root_path)
+from code_interpreter.JuypyterClient import JupyterNotebook
+from code_interpreter.BaseCodeInterpreter import BaseCodeInterpreter
+from utils.const import *
+from prompt.gpt4_prompt import CODE_INTERPRETER_SYSTEM_PROMPT
+
+# from prompt.gpt4_prompt import CODE_INTERPRETER_SYSTEM_PROMPT
+from colorama import init, Fore, Style
+from rich.markdown import Markdown
+import base64
+
+import openai
+from retrying import retry
+import logging
+from termcolor import colored
+
+# load from key file
+with open("./openai_api_key.txt") as f:
+    OPENAI_API_KEY = key = f.read()
+openai.api_key = OPENAI_API_KEY
+from utils.cleaner import clean_error_msg
+
+
+def remove_string(s):
+    pattern = r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{6}:.*LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64\n"
+    return re.sub(pattern, "", s)
+
+
+def clean_the_dialog(dialog, question):
+    question_idx = 0
+    for idx, item in enumerate(dialog):
+        if item["content"] == question:
+            question_idx = idx
+
+    filtered_dialog = dialog[question_idx:]
+
+    user_qinit_dict = filtered_dialog[0]
+    answer_fuse_str = "\n".join([i["content"].strip() for i in filtered_dialog[1::2]])
+
+    final_dialog_dict = [
+        {"role": "user", "content": user_qinit_dict["content"]},
+        {"role": "assistant", "content": answer_fuse_str},
+    ]
+
+    return final_dialog_dict
+
+
+class GPTCodeInterpreter(BaseCodeInterpreter):
+    def __init__(self, model="gpt-4"):
+        self.model = model
+        self.dialog = [
+            # {"role": "system", "content":  CODE_INTERPRETER_SYSTEM_PROMPT },
+            {
+                "role": "system",
+                "content": CODE_INTERPRETER_SYSTEM_PROMPT,
+            },
+            # {"role": "user", "content": "How can I use BeautifulSoup to scrape a website and extract all the URLs on a page?"},
+            # {"role": "assistant", "content": "I think I need to use beatifulsoup to find current korean president,"}
+        ]
+
+        # self.dialog += few_shot_4
+        self.response = None
+
+        assert os.path.isfile(
+            "./openai_api_key.txt"
+        ), "The openai_api_key.txt file could not be found. Please make sure it is in the same directory as this script, and that it contains your OpenAI API key."
+
+        # load from key file
+        with open("./openai_api_key.txt") as f:
+            OPENAI_API_KEY = f.read()
+        openai.api_key = OPENAI_API_KEY
+
+        self.nb = JupyterNotebook()
+        out = self.nb.add_and_run(TOOLS_CODE)  # tool import
+
+    def get_response_content(self):
+        if self.response:
+            return self.response["choices"][0]["message"]["content"]
+        else:
+            return None
+
+    @retry(
+        stop_max_attempt_number=7,
+        wait_exponential_multiplier=1000,
+        wait_exponential_max=10000,
+    )
+    def ChatCompletion(self):
+        try:
+            self.response = openai.ChatCompletion.create(
+                model=self.model, messages=self.dialog, temperature=0.2, top_p=0.9
+            )
+        except Exception as e:
+            print(f"error while OPENAI api call {e}")
+
+    def close(self):
+        """
+        close jupyter notebook, and this class instance
+        """
+        self.nb.close()
+
+    def save_dialog(self, path: str = "./output/dialog.json"):
+        with open(path, "w") as f:
+            json.dump(self.dialog, f)
+            print(f" ++Dialog saved to [{path}]")
+
+    def chat(
+        self,
+        user_message: str,
+        VERBOSE: bool = False,
+        MAX_TRY: int = 6,
+        code_exec_prefix: str = "",
+        feedback_prompt: str = "",
+        append_result: bool = True,
+    ):
+        self.dialog.append({"role": "user", "content": user_message})
+
+        code_block_output = ""
+        attempt = 0
+        img_data = None
+
+        if VERBOSE:
+            print(
+                "###User : " + Fore.BLUE + Style.BRIGHT + user_message + Style.RESET_ALL
+            )
+            print("\n###Assistant : ")
+
+        for i in range(MAX_TRY):
+            # GPT response
+            self.ChatCompletion()
+
+            # Get code block
+            generated_text = self.get_response_content()
+            generated_code_blocks = self.extract_code_blocks(generated_text)
+            # execute code
+            if len(generated_code_blocks) > 0:
+                # Find the position of the first code block in the last answer
+                first_code_block_pos = (
+                    generated_text.find(generated_code_blocks[0])
+                    if generated_code_blocks
+                    else -1
+                )
+                text_before_first_code_block = (
+                    generated_text
+                    if first_code_block_pos == -1
+                    else generated_text[:first_code_block_pos]
+                )
+                if VERBOSE:
+                    print(Fore.GREEN + text_before_first_code_block + Style.RESET_ALL)
+                if VERBOSE:
+                    print(
+                        Fore.YELLOW
+                        + generated_code_blocks[0]
+                        + "\n```\n"
+                        + Style.RESET_ALL
+                    )
+                code_block_output, error_flag = self.execute_code_and_return_output(
+                    generated_code_blocks[0]
+                )
+
+                code_block_output = f"{code_block_output}"
+
+                if code_block_output is not None:
+                    code_block_output = code_block_output.strip()
+
+                code_block_output = remove_string(code_block_output)
+                if len(code_block_output) > 500:
+                    code_block_output = (
+                        code_block_output[:200] + "⋯(skip)⋯" + code_block_output[-200:]
+                    )
+                code_block_output_str = f"\n```RESULT\n{code_block_output}\n```\n"
+                if append_result:
+                    gen_final = f"{text_before_first_code_block}{generated_code_blocks[0]}\n```{code_block_output_str}"
+                    if VERBOSE:
+                        print(
+                            Fore.LIGHTBLACK_EX + code_block_output_str + Style.RESET_ALL
+                        )
+                else:
+                    gen_final = (
+                        f"{text_before_first_code_block}{generated_code_blocks[0]}\n```"
+                    )
+
+                self.dialog.append(
+                    {
+                        "role": "assistant",
+                        "content": gen_final,
+                    }
+                )
+
+                if len(feedback_prompt) < 5:
+                    feedback_dict = {
+                        "role": "user",
+                        "content": "Keep going. if you think debugging tell me where you got wrong and better code.\nNeed conclusion to question only text (Do not leave result part alone).\nif doesn't need to generated anything then just say <done>",
+                    }
+                else:
+                    feedback_dict = {
+                        "role": "user",
+                        "content": f"{feedback_prompt}",
+                    }
+
+                self.dialog.append(feedback_dict)
+
+            else:
+                if "<done>" in generated_text:
+                    generated_text = generated_text.split("<done>")[0].strip()
+
+                if len(generated_text) <= 0:
+                    break
+
+                if VERBOSE:
+                    print(Fore.GREEN + generated_text + Style.RESET_ALL)
+
+                self.dialog.append(
+                    {
+                        "role": "assistant",
+                        "content": f"{generated_text}",
+                    }
+                )
+                break
+
+        self.dialog = [self.dialog[0]] + clean_the_dialog(
+            self.dialog, question=user_message
+        )  # delete retrospections after generation step
+
+        return self.dialog[-1]

Llama2-Code-Interpreter/code_interpreter/GPTCodeInterpreterDataCollect.py

@@ -0,0 +1,271 @@
+import json
+import os, sys
+import time
+import re
+from pathlib import Path
+from typing import List, Literal, Optional, Tuple, TypedDict, Dict
+
+# Get the path from environment variable
+prj_root_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+sys.path.append(prj_root_path)
+from code_interpreter.JuypyterClient import JupyterNotebook
+from code_interpreter.BaseCodeInterpreter import BaseCodeInterpreter
+from utils.const import *
+from colorama import init, Fore, Style
+from rich.markdown import Markdown
+import base64
+
+import openai
+from retrying import retry
+import logging
+from termcolor import colored
+
+# load from key file
+with open("./openai_api_key.txt") as f:
+    OPENAI_API_KEY = key = f.read()
+openai.api_key = OPENAI_API_KEY
+from utils.cleaner import clean_error_msg
+from prompt.gpt4_prompt import *
+
+
+def remove_string(s):
+    pattern = r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{6}:.*LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64\n"
+    return re.sub(pattern, "", s)
+
+
+def gen_questions(prefix="What is 55th fibonacci number?"):
+    response = openai.ChatCompletion.create(
+        model="gpt-4",
+        messages=[
+            {
+                "role": "system",
+                "content": "You are teacherGPT, You need to generate only questions(to student not the explanation and solution) based on student history. \n\nGive him only one question.\n\nAlso remember that student can use code. ",
+            },
+            {
+                "role": "user",
+                "content": f"{prefix}\nmore harder one but not the similar domain of above.",
+            },
+        ],
+        temperature=0.1,
+        max_tokens=300,
+        top_p=1,
+        frequency_penalty=0,
+        presence_penalty=0,
+    )
+    return response["choices"][0]["message"]["content"]
+
+
+def save_dialog(dialog, base_path: str = f"{prj_root_path}/gpt_data_gen"):
+    file_number = 0
+    while True:
+        # Construct the path
+        file_name = f"{file_number}.json"
+        full_path = os.path.join(base_path, file_name)
+
+        # Check if the file already exists
+        if not os.path.exists(full_path):
+            # If not, save the file
+            with open(full_path, "w") as f:
+                json.dump(dialog, f)
+            print(f"Dialog saved to {full_path}")
+            break
+        else:
+            # If the file does exist, increment the file number and try again
+            file_number += 1
+
+
+def clean_the_dialog(dialog, question):
+    question_idx = 0
+    for idx, item in enumerate(dialog):
+        if item["content"] == question:
+            question_idx = idx
+
+    filtered_dialog = dialog[question_idx:]
+
+    user_qinit_dict = filtered_dialog[0]
+    answer_fuse_str = "\n".join([i["content"].strip() for i in filtered_dialog[1::2]])
+
+    final_dialog_dict = [
+        {"role": "user", "content": user_qinit_dict["content"]},
+        {"role": "assistant", "content": answer_fuse_str},
+    ]
+
+    return final_dialog_dict
+
+
+class GPTCodeInterpreter(BaseCodeInterpreter):
+    def __init__(self, model="gpt-4"):
+        self.model = model
+        self.dialog = [
+            # {"role": "system", "content":  CODE_INTERPRETER_SYSTEM_PROMPT },
+            {
+                "role": "system",
+                "content": CODE_INTERPRETER_SYSTEM_PROMPT + "\n" + extra_prompt,
+            },
+            # {"role": "user", "content": "How can I use BeautifulSoup to scrape a website and extract all the URLs on a page?"},
+            # {"role": "assistant", "content": "I think I need to use beatifulsoup to find current korean president,"}
+        ]
+
+        self.dialog += few_shot_1
+        # self.dialog += few_shot_4
+        self.response = None
+
+        assert os.path.isfile(
+            "./openai_api_key.txt"
+        ), "The openai_api_key.txt file could not be found. Please make sure it is in the same directory as this script, and that it contains your OpenAI API key."
+
+        # load from key file
+        with open("./openai_api_key.txt") as f:
+            OPENAI_API_KEY = f.read()
+        openai.api_key = OPENAI_API_KEY
+
+        self.nb = JupyterNotebook()
+        out = self.nb.add_and_run(TOOLS_CODE)  # tool import
+
+    def get_response_content(self):
+        if self.response:
+            return self.response["choices"][0]["message"]["content"]
+        else:
+            return None
+
+    @retry(
+        stop_max_attempt_number=7,
+        wait_exponential_multiplier=1000,
+        wait_exponential_max=10000,
+    )
+    def ChatCompletion(self):
+        try:
+            self.response = openai.ChatCompletion.create(
+                model=self.model, messages=self.dialog, temperature=0.1, top_p=1.0
+            )
+        except Exception as e:
+            print(f"error while OPENAI api call {e}")
+
+    def chat(self, user_message: str, VERBOSE: bool = False, MAX_RETRY: int = 6):
+        self.dialog.append({"role": "user", "content": user_message})
+
+        code_block_output = ""
+        attempt = 0
+        img_data = None
+
+        if VERBOSE:
+            print(
+                "###User : " + Fore.BLUE + Style.BRIGHT + user_message + Style.RESET_ALL
+            )
+            print("\n###Assistant : ")
+
+        for i in range(MAX_RETRY):
+            # GPT response
+            self.ChatCompletion()
+
+            # Get code block
+            generated_text = self.get_response_content()
+            generated_code_blocks = self.extract_code_blocks(generated_text)
+            # execute code
+            if len(generated_code_blocks) > 0:
+                # Find the position of the first code block in the last answer
+                first_code_block_pos = (
+                    generated_text.find(generated_code_blocks[0])
+                    if generated_code_blocks
+                    else -1
+                )
+                text_before_first_code_block = (
+                    generated_text
+                    if first_code_block_pos == -1
+                    else generated_text[:first_code_block_pos]
+                )
+                if VERBOSE:
+                    print(Fore.GREEN + text_before_first_code_block + Style.RESET_ALL)
+                if VERBOSE:
+                    print(
+                        Fore.YELLOW
+                        + generated_code_blocks[0]
+                        + "\n```\n"
+                        + Style.RESET_ALL
+                    )
+                code_block_output, error_flag = self.execute_code_and_return_output(
+                    generated_code_blocks[0]
+                )
+
+                code_block_output = f"{code_block_output}"
+
+                if code_block_output is not None:
+                    code_block_output = code_block_output.strip()
+
+                code_block_output = remove_string(code_block_output)
+                if len(code_block_output) > 500:
+                    code_block_output = (
+                        code_block_output[:200] + "⋯(skip)⋯" + code_block_output[-200:]
+                    )
+                code_block_output_str = f"\n```RESULT\n{code_block_output}\n```\n"
+                if VERBOSE:
+                    print(Fore.LIGHTBLACK_EX + code_block_output_str + Style.RESET_ALL)
+                    # markdown = Markdown(code_block_output_str)print(markdown)
+
+                gen_final = f"{text_before_first_code_block}{generated_code_blocks[0]}\n```{code_block_output_str}"
+
+                self.dialog.append(
+                    {
+                        "role": "assistant",
+                        "content": f"{text_before_first_code_block}{generated_code_blocks[0]}\n```{code_block_output_str}",
+                    }
+                )
+
+                self.dialog.append(
+                    {
+                        "role": "user",
+                        "content": "Keep going. if you think debugging generate code. need conclusion to question only text (Do not leave result part alone). Doesn't need to generated anything then just say <done>",
+                    }
+                )
+
+            else:
+                if "<done>" in generated_text:
+                    generated_text = generated_text.split("<done>")[0].strip()
+
+                if len(generated_text) <= 0:
+                    break
+
+                if VERBOSE:
+                    print(Fore.GREEN + generated_text + Style.RESET_ALL)
+
+                self.dialog.append(
+                    {
+                        "role": "assistant",
+                        "content": f"{generated_text}",
+                    }
+                )
+                break
+
+        return self.dialog[-1]
+
+
+if __name__ == "__main__":
+    import random
+
+    SEED_TASK = [
+        # "Resize this image to 512x512\nUser Uploaded File : './tmp/img.png'",
+        "Write a Python script that retrieves Google Trends data for a given keyword and stock price data for a specific company over the same timeframe, normalizes both datasets to the same scale, and then plots them on the same graph to analyze potential correlations.",
+        "Could you conduct a frequency analysis on Apple's stock price to determine any cyclic patterns that occur on a weekly, monthly, or quarterly basis?",
+    ]
+
+    questions = SEED_TASK
+
+    from tqdm import tqdm
+
+    for i in tqdm(range(150000)):
+        interpreter = GPTCodeInterpreter()
+
+        question = questions[i]
+        output = interpreter.chat(user_message=question, VERBOSE=True, MAX_RETRY=5)
+
+        sample = clean_the_dialog(interpreter.dialog, question)
+
+        save_dialog(sample)
+
+        # q1,q2,q3 = random.sample(questions, k=3)
+        # question = gen_questions(prefix = f'{q1}\n{q2}\n{q3}')
+        # questions.append(question)
+
+        del interpreter
+
+        print(f"new question :: {question}")

Llama2-Code-Interpreter/code_interpreter/JuypyterClient.py

@@ -0,0 +1,75 @@
+from jupyter_client import KernelManager
+import threading
+import re
+
+
+class JupyterNotebook:
+    def __init__(self):
+        self.km = KernelManager()
+        self.km.start_kernel()
+        self.kc = self.km.client()
+
+    def clean_output(self, outputs):
+        outputs_only_str = list()
+        for i in outputs:
+            if type(i) == dict:
+                if "text/plain" in list(i.keys()):
+                    outputs_only_str.append(i["text/plain"])
+            elif type(i) == str:
+                outputs_only_str.append(i)
+            elif type(i) == list:
+                error_msg = "\n".join(i)
+                error_msg = re.sub(r"\x1b\[.*?m", "", error_msg)
+                outputs_only_str.append(error_msg)
+
+        return "\n".join(outputs_only_str).strip()
+
+    def add_and_run(self, code_string):
+        # This inner function will be executed in a separate thread
+        def run_code_in_thread():
+            nonlocal outputs, error_flag
+
+            # Execute the code and get the execution count
+            msg_id = self.kc.execute(code_string)
+
+            while True:
+                try:
+                    msg = self.kc.get_iopub_msg(timeout=20)
+
+                    msg_type = msg["header"]["msg_type"]
+                    content = msg["content"]
+
+                    if msg_type == "execute_result":
+                        outputs.append(content["data"])
+                    elif msg_type == "stream":
+                        outputs.append(content["text"])
+                    elif msg_type == "error":
+                        error_flag = True
+                        outputs.append(content["traceback"])
+
+                    # If the execution state of the kernel is idle, it means the cell finished executing
+                    if msg_type == "status" and content["execution_state"] == "idle":
+                        break
+                except:
+                    break
+
+        outputs = []
+        error_flag = False
+
+        # Start the thread to run the code
+        thread = threading.Thread(target=run_code_in_thread)
+        thread.start()
+
+        # Wait for 10 seconds for the thread to finish
+        thread.join(timeout=10)
+
+        # If the thread is still alive after 10 seconds, it's a timeout
+        if thread.is_alive():
+            outputs = ["Timeout after 10 seconds"]
+            error_flag = True
+
+        return self.clean_output(outputs), error_flag
+
+    def close(self):
+        """Shutdown the kernel."""
+        self.km.shutdown_kernel()

Llama2-Code-Interpreter/code_interpreter/LlamaCodeInterpreter.py

@@ -0,0 +1,286 @@
+import sys
+import os
+
+prj_root_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+sys.path.append(prj_root_path)
+
+from code_interpreter.JuypyterClient import JupyterNotebook
+from code_interpreter.BaseCodeInterpreter import BaseCodeInterpreter
+from utils.const import *
+
+from typing import List, Literal, Optional, Tuple, TypedDict, Dict
+from colorama import init, Fore, Style
+import copy
+import re
+
+import torch
+import transformers
+from transformers import LlamaForCausalLM, LlamaTokenizer
+from peft import PeftModel
+
+
+sys.path.append(os.path.dirname(__file__))
+sys.path.append(os.path.dirname(os.path.abspath(__file__)))
+from finetuning.conversation_template import msg_to_code_result_tok_temp
+from utils.special_tok_llama2 import (
+    B_CODE,
+    E_CODE,
+    B_RESULT,
+    E_RESULT,
+    B_INST,
+    E_INST,
+    B_SYS,
+    E_SYS,
+    DEFAULT_PAD_TOKEN,
+    DEFAULT_BOS_TOKEN,
+    DEFAULT_EOS_TOKEN,
+    DEFAULT_UNK_TOKEN,
+    IGNORE_INDEX,
+)
+
+import warnings
+
+warnings.filterwarnings("ignore", category=UserWarning, module="transformers")
+os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
+
+
+class LlamaCodeInterpreter(BaseCodeInterpreter):
+    def __init__(
+        self,
+        model_path: str,
+        load_in_8bit: bool = False,
+        load_in_4bit: bool = False,
+        peft_model: Optional[str] = None,
+    ):
+        # build tokenizer
+        self.tokenizer = LlamaTokenizer.from_pretrained(
+            model_path,
+            padding_side="right",
+            use_fast=False,
+        )
+
+        # Handle special tokens
+        special_tokens_dict = dict()
+        if self.tokenizer.pad_token is None:
+            special_tokens_dict["pad_token"] = DEFAULT_PAD_TOKEN  # 32000
+        if self.tokenizer.eos_token is None:
+            special_tokens_dict["eos_token"] = DEFAULT_EOS_TOKEN  # 2
+        if self.tokenizer.bos_token is None:
+            special_tokens_dict["bos_token"] = DEFAULT_BOS_TOKEN  # 1
+        if self.tokenizer.unk_token is None:
+            special_tokens_dict["unk_token"] = DEFAULT_UNK_TOKEN
+
+        self.tokenizer.add_special_tokens(special_tokens_dict)
+        self.tokenizer.add_tokens(
+            [B_CODE, E_CODE, B_RESULT, E_RESULT, B_INST, E_INST, B_SYS, E_SYS],
+            special_tokens=True,
+        )
+
+        self.model = LlamaForCausalLM.from_pretrained(
+            model_path,
+            device_map="auto",
+            load_in_4bit=load_in_4bit,
+            load_in_8bit=load_in_8bit,
+            torch_dtype=torch.float16,
+        )
+
+        self.model.resize_token_embeddings(len(self.tokenizer))
+
+        if peft_model is not None:
+            peft_model = PeftModel.from_pretrained(self.model, peft_model)
+
+        self.model = self.model.eval()
+
+        self.dialog = [
+            {
+                "role": "system",
+                "content": CODE_INTERPRETER_SYSTEM_PROMPT + "\nUse code to answer",
+            },
+            # {"role": "user", "content": "How can I use BeautifulSoup to scrape a website and extract all the URLs on a page?"},
+            # {"role": "assistant", "content": "I think I need to use beatifulsoup to find current korean president,"}
+        ]
+
+        self.nb = JupyterNotebook()
+        self.MAX_CODE_OUTPUT_LENGTH = 3000
+        out = self.nb.add_and_run(TOOLS_CODE)  # tool import
+        print(out)
+
+    def dialog_to_prompt(self, dialog: List[Dict]) -> str:
+        full_str = msg_to_code_result_tok_temp(dialog)
+
+        return full_str
+
+    @torch.inference_mode()
+    def generate(
+        self,
+        prompt: str = "[INST]\n###User : hi\n###Assistant :",
+        max_new_tokens=512,
+        do_sample: bool = True,
+        use_cache: bool = True,
+        top_p: float = 0.95,
+        temperature: float = 0.1,
+        top_k: int = 50,
+        repetition_penalty: float = 1.0,
+    ) -> str:
+        # Get the model and tokenizer, and tokenize the user text.
+
+        input_prompt = copy.deepcopy(prompt)
+        inputs = self.tokenizer([prompt], return_tensors="pt")
+        input_tokens_shape = inputs["input_ids"].shape[-1]
+
+        eos_token_id = self.tokenizer.convert_tokens_to_ids(DEFAULT_EOS_TOKEN)
+        e_code_token_id = self.tokenizer.convert_tokens_to_ids(E_CODE)
+
+        output = self.model.generate(
+            **inputs,
+            max_new_tokens=max_new_tokens,
+            do_sample=do_sample,
+            top_p=top_p,
+            temperature=temperature,
+            use_cache=use_cache,
+            top_k=top_k,
+            repetition_penalty=repetition_penalty,
+            eos_token_id=[
+                eos_token_id,
+                e_code_token_id,
+            ],  # Stop generation at either EOS or E_CODE token
+        )[0]
+
+        generated_tokens = output[input_tokens_shape:]
+        generated_text = self.tokenizer.decode(generated_tokens)
+
+        return generated_text
+
+    def extract_code_blocks(self, prompt: str) -> Tuple[bool, str]:
+        pattern = re.escape(B_CODE) + r"(.*?)" + re.escape(E_CODE)
+        matches = re.findall(pattern, prompt, re.DOTALL)
+
+        if matches:
+            # Return the last matched code block
+            return True, matches[-1].strip()
+        else:
+            return False, ""
+
+    def clean_code_output(self, output: str) -> str:
+        if self.MAX_CODE_OUTPUT_LENGTH < len(output):
+            return (
+                output[: self.MAX_CODE_OUTPUT_LENGTH // 5]
+                + "...(skip)..."
+                + output[-self.MAX_CODE_OUTPUT_LENGTH // 5 :]
+            )
+
+        return output
+
+    def chat(self, user_message: str, VERBOSE: bool = False, MAX_TRY=5):
+        self.dialog.append({"role": "user", "content": user_message})
+        if VERBOSE:
+            print(
+                "###User : " + Fore.BLUE + Style.BRIGHT + user_message + Style.RESET_ALL
+            )
+            print("\n###Assistant : ")
+
+        # setup
+        HAS_CODE = False  # For now
+        INST_END_TOK_FLAG = False
+        full_generated_text = ""
+        prompt = self.dialog_to_prompt(dialog=self.dialog)
+        start_prompt = copy.deepcopy(prompt)
+        prompt = f"{prompt} {E_INST}"
+
+        generated_text = self.generate(prompt)
+        full_generated_text += generated_text
+        HAS_CODE, generated_code_block = self.extract_code_blocks(generated_text)
+
+        attempt = 1
+        while HAS_CODE:
+            if attempt > MAX_TRY:
+                break
+            # if no code then doesn't have to execute it
+
+            # replace unknown thing to none
+            generated_code_block = generated_code_block.replace("<unk>_", "").replace(
+                "<unk>", ""
+            )
+
+            code_block_output, error_flag = self.execute_code_and_return_output(
+                f"{generated_code_block}"
+            )
+            code_block_output = self.clean_code_output(code_block_output)
+            generated_text = (
+                f"{generated_text}\n{B_RESULT}\n{code_block_output}\n{E_RESULT}\n"
+            )
+            full_generated_text += f"\n{B_RESULT}\n{code_block_output}\n{E_RESULT}\n"
+
+            first_code_block_pos = (
+                generated_text.find(generated_code_block)
+                if generated_code_block
+                else -1
+            )
+            text_before_first_code_block = (
+                generated_text
+                if first_code_block_pos == -1
+                else generated_text[:first_code_block_pos]
+            )
+            if VERBOSE:
+                print(Fore.GREEN + text_before_first_code_block + Style.RESET_ALL)
+                print(Fore.GREEN + generated_code_block + Style.RESET_ALL)
+                print(
+                    Fore.YELLOW
+                    + f"\n{B_RESULT}\n{code_block_output}\n{E_RESULT}\n"
+                    + Style.RESET_ALL
+                )
+
+            # prompt = f"{prompt} {E_INST}{generated_text}"
+            prompt = f"{prompt}{generated_text}"
+            generated_text = self.generate(prompt)
+            HAS_CODE, generated_code_block = self.extract_code_blocks(generated_text)
+
+            full_generated_text += generated_text
+
+            attempt += 1
+
+        if VERBOSE:
+            print(Fore.GREEN + generated_text + Style.RESET_ALL)
+
+        self.dialog.append(
+            {
+                "role": "assistant",
+                "content": full_generated_text.replace("<unk>_", "")
+                .replace("<unk>", "")
+                .replace("</s>", ""),
+            }
+        )
+
+        return self.dialog[-1]
+
+
+if __name__ == "__main__":
+    import random
+
+    LLAMA2_MODEL_PATH = "./ckpt/llama-2-13b-chat"
+    LLAMA2_MODEL_PATH = "meta-llama/Llama-2-70b-chat-hf"
+    LLAMA2_FINETUNEED_PATH = "./output/llama-2-7b-chat-ci"
+
+    interpreter = LlamaCodeInterpreter(
+        model_path=LLAMA2_FINETUNEED_PATH, load_in_4bit=True
+    )
+    output = interpreter.chat(
+        user_message=random.choice(
+            [
+                # "In a circle with center \( O \), \( AB \) is a chord such that the midpoint of \( AB \) is \( M \). A tangent at \( A \) intersects the extended segment \( OB \) at \( P \). If \( AM = 12 \) cm and \( MB = 12 \) cm, find the length of \( AP \)."
+                # "A triangle \( ABC \) is inscribed in a circle (circumscribed). The sides \( AB \), \( BC \), and \( AC \) are tangent to the circle at points \( P \), \( Q \), and \( R \) respectively. If \( AP = 10 \) cm, \( BQ = 15 \) cm, and \( CR = 20 \) cm, find the radius of the circle.",
+                # "Given an integer array nums, return the total number of contiguous subarrays that have a sum equal to 0.",
+                "what is second largest city in japan?",
+                # "Can you show me 120days chart of tesla from today to before 120?"
+            ]
+        ),
+        VERBOSE=True,
+    )
+
+    while True:
+        input_char = input("Press 'q' to quit the dialog: ")
+        if input_char.lower() == "q":
+            break
+
+        else:
+            output = interpreter.chat(user_message=input_char, VERBOSE=True)

Llama2-Code-Interpreter/code_interpreter/RetrospectiveGPTCodeInterpreter.py

@@ -0,0 +1,472 @@
+import json
+import os
+import sys
+import time
+import copy
+import re
+from pathlib import Path
+from typing import List, Literal, Optional, Tuple, TypedDict, Dict
+import numpy as np
+from tqdm import tqdm
+
+# Get the path from environment variable
+prj_root_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+sys.path.append(prj_root_path)
+from code_interpreter.JuypyterClient import JupyterNotebook
+from code_interpreter.BaseCodeInterpreter import BaseCodeInterpreter
+from utils.const import *
+from prompt.gpt4_prompt import CODE_INTERPRETER_SYSTEM_PROMPT
+
+# from prompt.gpt4_prompt import CODE_INTERPRETER_SYSTEM_PROMPT
+from colorama import init, Fore, Style, Back
+from rich.markdown import Markdown
+import base64
+
+import openai
+from retrying import retry
+import requests
+import logging
+from termcolor import colored
+
+# load from key file
+with open("./openai_api_key.txt") as f:
+    OPENAI_API_KEY = key = f.read()
+openai.api_key = OPENAI_API_KEY
+from utils.cleaner import clean_error_msg
+
+
+def remove_string(s):
+    pattern = r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{6}:.*LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64\n"
+    return re.sub(pattern, "", s)
+
+
+def clean_the_dialog(dialog, question):
+    question_idx = 0
+    for idx, item in enumerate(dialog):
+        if item["content"] == question:
+            question_idx = idx
+
+    filtered_dialog = dialog[question_idx:]
+
+    user_qinit_dict = filtered_dialog[0]
+    answer_fuse_str = "\n".join([i["content"].strip() for i in filtered_dialog[1::2]])
+
+    final_dialog_dict = [
+        {"role": "user", "content": user_qinit_dict["content"]},
+        {"role": "assistant", "content": answer_fuse_str},
+    ]
+
+    return final_dialog_dict
+
+
+@retry(
+    stop_max_attempt_number=7,
+    wait_exponential_multiplier=1000,
+    wait_exponential_max=10000,
+)
+def get_embedding(text, model="text-embedding-ada-002"):
+    global counter
+    headers = {
+        "Authorization": f"Bearer {OPENAI_API_KEY}",  # Make sure to replace with your OpenAI API key
+        "Content-Type": "application/json",
+    }
+    payload = {"input": text, "model": model}
+
+    response = requests.post(
+        "https://api.openai.com/v1/embeddings", headers=headers, json=payload
+    )
+
+    if response.status_code != 200:
+        raise Exception(f"Request failed with status {response.status_code}")
+
+    return np.array(response.json()["data"][0]["embedding"])
+
+
+class QueryRetrospect:
+    def __init__(
+        self,
+        data_directory="./gpt_data_gen_retrospect/",
+        embeddings_path="./gpt_data_gen_retrospect/embeddings.npy",
+    ):
+        self.data_directory = data_directory
+        self.embeddings_path = embeddings_path
+        self.data = []
+        self.embeddings = []
+
+        if os.path.exists(embeddings_path):
+            print("++ Embedding Exists!")
+            self.embeddings = np.load(embeddings_path)
+            for fname in [i for i in os.listdir(data_directory) if i.endswith(".json")]:
+                with open(
+                    os.path.join(data_directory, fname),
+                    "r",
+                    encoding="utf-8",
+                    errors="replace",
+                ) as f:
+                    self.data.append(json.load(f))
+        else:
+            only_files = [
+                f
+                for f in os.listdir(data_directory)
+                if os.path.isfile(os.path.join(data_directory, f))
+                and f.endswith(".json")
+            ]
+
+            for fname in tqdm(only_files):
+                with open(
+                    os.path.join(data_directory, fname), "r", encoding="cp1252"
+                ) as f:
+                    data_point = json.load(f)
+                    self.data.append(data_point)
+                    self.embeddings.append(
+                        get_embedding(data_point["execution_result"])
+                    )
+            self.embeddings = np.array(self.embeddings)
+            self.save_embeddings()
+            print(f"++ Embedding Saved! {self.embeddings.shape}")
+
+    def save_embeddings(self):
+        np.save(self.embeddings_path, self.embeddings)
+
+    def __call__(self, query, top_k=3, VERBOSE: bool = False):
+        query_embedding = get_embedding(query)
+        similarities = np.dot(self.embeddings, query_embedding)
+        top_indices = similarities.argsort()[-top_k:][::-1]
+        return [self.data[i]["retrospection"] for i in top_indices]
+
+
+class QueryRetrospectPrefix:
+    def __init__(
+        self,
+        model="gpt-4",
+        data_directory="./eval/gpt_mbpp_output",
+        embeddings_path="./eval/gpt_mbpp_output/embeddings.npy",
+    ):
+        self.data_directory = data_directory
+        self.embeddings_path = embeddings_path
+        self.data = []
+        self.embeddings = []
+
+        if os.path.exists(embeddings_path):
+            print("++ Embedding Exists!")
+            self.embeddings = np.load(embeddings_path)
+            for fname in [i for i in os.listdir(data_directory) if i.endswith(".json")]:
+                with open(
+                    os.path.join(data_directory, fname),
+                    "r",
+                    encoding="utf-8",
+                    errors="replace",
+                ) as f:
+                    self.data.append(json.load(f))
+        else:
+            only_files = [
+                f
+                for f in os.listdir(data_directory)
+                if os.path.isfile(os.path.join(data_directory, f))
+                and f.endswith(".json")
+            ]
+
+            for fname in tqdm(only_files):
+                with open(
+                    os.path.join(data_directory, fname), "r", encoding="cp1252"
+                ) as f:
+                    data_point = json.load(f)
+
+                    print(f'Processing "{data_point[1]["content"]}" ...')
+                    self.data.append(data_point)
+                    self.embeddings.append(get_embedding(data_point[1]["content"]))
+
+            self.embeddings = np.array(self.embeddings)
+            self.save_embeddings()
+            print(f"++ Embedding Saved! {self.embeddings.shape}")
+
+        self.model = model
+        self.dialog = [
+            {
+                "role": "system",
+                "content": "You are retrospection GPT. retrospect from the given data.",
+            },
+            {
+                "role": "user",
+                "content": 'Current Question:\n\nWrite a Python function to solve the following task:\n\nfrom typing import List\n\ndef cum_sum(numbers: List[int]) -> List[int]:\n    """\n    From a given list of integers, generate a list representing the cumulative sum of elements at each index.\n    >>> cum_sum([1, 2, 3, 4])\n    [1, 3, 6, 10]\n    """\n\nRetrieved Trajectories : \nIn a past interaction, a function named running_average was provided to calculate the running average of a list of numbers.\n\n```python\ndef running_average(numbers: List[int]) -> List[float]:\n    total = 0\n    averages = []\n    for i, num in enumerate(numbers):\n        total += num\n        averages.append(total / (i+1))\n    return averages\n\nprint(running_average([1,2,3,4])) # expected [1.0, 1.5, 2.0, 2.5]\n```\n```RESULT\n[1.0, 1.5, 2.0, 2.5]\n```\nThe output is expected. \n\n',
+            },
+            {
+                "role": "assistant",
+                "content": "From previous similar questions :\nThe `running_average` function highlights an important concept of maintaining a running or cumulative value (total) as one iterates over the list. This is directly applicable to the cum_sum problem.\n\nApplication to the Question:\nFor the cum_sum function, one needs to maintain a cumulative total of the elements as we traverse through the list. The running_average function is most closely related since it involves accumulating a total and storing intermediate results. By adapting this logic (i.e., excluding the division operation to compute the average), one can easily derive the cumulative sum solution.",
+            },
+        ]
+        self.response = ""
+
+    @retry(
+        stop_max_attempt_number=7,
+        wait_exponential_multiplier=1000,
+        wait_exponential_max=10000,
+    )
+    def ChatCompletion(self):
+        try:
+            self.response = openai.ChatCompletion.create(
+                model=self.model, messages=self.dialog, temperature=0.2, top_p=0.9
+            )
+        except Exception as e:
+            print(f"error while OPENAI api call {e} {self.response}")
+
+    def save_embeddings(self):
+        np.save(self.embeddings_path, self.embeddings)
+
+    def __call__(self, query, top_k=3, VERBOSE: bool = False):
+        query_embedding = get_embedding(query)
+        similarities = np.dot(self.embeddings, query_embedding)
+        top_indices = similarities.argsort()[-top_k:][::-1]
+        top_i = top_indices[0]
+        prior_traj = self.data[top_i][-1]["content"]
+
+        ask_dict = {
+            "role": "user",
+            "content": f"Current Question:\n\n{query}\n\nRetrieved Trajectories :\n{prior_traj}",
+        }
+
+        # print(f"From prior experience:\n{prior_traj}\n\nCurrent Question:\n{query}\n")
+        self.dialog.append(ask_dict)
+        self.ChatCompletion()
+
+        return self.response["choices"][0]["message"]["content"]
+
+
+class RetrospectiveGPTCodeInterpreter(BaseCodeInterpreter):
+    def __init__(self, model="gpt-4"):
+        self.model = model
+        self.dialog = [
+            # {"role": "system", "content":  CODE_INTERPRETER_SYSTEM_PROMPT },
+            {
+                "role": "system",
+                "content": CODE_INTERPRETER_SYSTEM_PROMPT,
+            },
+            # {"role": "user", "content": "How can I use BeautifulSoup to scrape a website and extract all the URLs on a page?"},
+            # {"role": "assistant", "content": "I think I need to use beatifulsoup to find current korean president,"}
+        ]
+
+        # self.dialog += few_shot_4
+        self.response = None
+
+        assert os.path.isfile(
+            "./openai_api_key.txt"
+        ), "The openai_api_key.txt file could not be found. Please make sure it is in the same directory as this script, and that it contains your OpenAI API key."
+
+        # load from key file
+        with open("./openai_api_key.txt") as f:
+            OPENAI_API_KEY = f.read()
+        openai.api_key = OPENAI_API_KEY
+
+        self.nb = JupyterNotebook()
+        out = self.nb.add_and_run(TOOLS_CODE)  # tool import
+
+        # retrospections
+        self.retrospector = QueryRetrospectPrefix()
+
+    def get_response_content(self):
+        if self.response:
+            return self.response["choices"][0]["message"]["content"]
+        else:
+            return None
+
+    @retry(
+        stop_max_attempt_number=7,
+        wait_exponential_multiplier=1000,
+        wait_exponential_max=10000,
+    )
+    def ChatCompletion(self):
+        try:
+            self.response = openai.ChatCompletion.create(
+                model=self.model, messages=self.dialog, temperature=0.2, top_p=0.9
+            )
+        except Exception as e:
+            print(f"error while OPENAI api call {e}")
+
+    def save_dialog(self, path: str = "./output/dialog.json"):
+        with open(path, "w") as f:
+            json.dump(self.dialog, f)
+            print(f" ++Dialog saved to [{path}]")
+
+    def close(self):
+        """
+        close jupyter notebook, and this class instance
+        """
+        self.nb.close()
+
+    def chat(
+        self,
+        user_message: str,
+        VERBOSE: bool = False,
+        MAX_TRY: int = 6,
+        code_exec_prefix: str = "",
+        feedback_prompt: str = "",
+        append_result: bool = True,
+        use_retrospect: bool = True,
+    ):
+        prefix_retrospection = self.retrospector(query=user_message)
+        self.dialog.append(
+            {"role": "user", "content": f"{prefix_retrospection}\n\n{user_message}"}
+        )
+        init_feedback = copy.deepcopy(feedback_prompt)
+
+        code_block_output = ""
+        attempt = 0
+        img_data = None
+
+        if VERBOSE:
+            print(
+                "###Retrospection : "
+                + Fore.BLUE
+                + Back.WHITE
+                + Style.BRIGHT
+                + prefix_retrospection
+                + Style.RESET_ALL
+            )
+            print(
+                "###User : " + Fore.BLUE + Style.BRIGHT + user_message + Style.RESET_ALL
+            )
+            print("\n###Assistant : ")
+
+        for i in range(MAX_TRY):
+            # GPT response
+            self.ChatCompletion()
+
+            # Get code block
+            generated_text = self.get_response_content()
+            generated_code_blocks = self.extract_code_blocks(generated_text)
+            # execute code
+            if len(generated_code_blocks) > 0:
+                # Find the position of the first code block in the last answer
+                first_code_block_pos = (
+                    generated_text.find(generated_code_blocks[0])
+                    if generated_code_blocks
+                    else -1
+                )
+                text_before_first_code_block = (
+                    generated_text
+                    if first_code_block_pos == -1
+                    else generated_text[:first_code_block_pos]
+                )
+                if VERBOSE:
+                    print(Fore.GREEN + text_before_first_code_block + Style.RESET_ALL)
+                if VERBOSE:
+                    print(
+                        Fore.YELLOW
+                        + generated_code_blocks[0]
+                        + "\n```\n"
+                        + Style.RESET_ALL
+                    )
+                code_block_output, error_flag = self.execute_code_and_return_output(
+                    generated_code_blocks[0]
+                )
+
+                code_block_output = f"{code_block_output}"
+
+                if code_block_output is not None:
+                    code_block_output = code_block_output.strip()
+
+                code_block_output = remove_string(code_block_output)
+                if len(code_block_output) > 500:
+                    code_block_output = (
+                        code_block_output[:200] + "⋯(skip)⋯" + code_block_output[-200:]
+                    )
+                code_block_output_str = f"\n```RESULT\n{code_block_output}\n```\n"
+                if append_result:
+                    gen_final = f"{text_before_first_code_block}{generated_code_blocks[0]}\n```{code_block_output_str}"
+                    if VERBOSE:
+                        print(
+                            Fore.LIGHTBLACK_EX + code_block_output_str + Style.RESET_ALL
+                        )
+                else:
+                    gen_final = (
+                        f"{text_before_first_code_block}{generated_code_blocks[0]}\n```"
+                    )
+
+                self.dialog.append(
+                    {
+                        "role": "assistant",
+                        "content": gen_final,
+                    }
+                )
+
+                feedback_prompt = f"{init_feedback}\nif you accomplish the instruction just say <done>\nIf not keep going."
+                if VERBOSE:
+                    print(Fore.MAGENTA + feedback_prompt + Style.RESET_ALL)
+
+                feedback_dict = {
+                    "role": "user",
+                    "content": feedback_prompt,
+                }
+
+                self.dialog.append(feedback_dict)
+
+            else:
+                if "<done>" in generated_text:
+                    generated_text = generated_text.split("<done>")[0].strip()
+
+                if len(generated_text) <= 0:
+                    break
+
+                if VERBOSE:
+                    print(Fore.GREEN + generated_text + Style.RESET_ALL)
+
+                self.dialog.append(
+                    {
+                        "role": "assistant",
+                        "content": f"{generated_text}",
+                    }
+                )
+                break
+
+        self.dialog = [self.dialog[0]] + clean_the_dialog(
+            self.dialog, question=f"{prefix_retrospection}\n\n{user_message}"
+        )  # delete retrospections after generation step
+
+        return self.dialog[-1]
+
+
+if __name__ == "__main__":
+    import pickle
+    import random
+    from tqdm import tqdm
+
+    # python3 -m code_interpreter.RetrospectiveGPTCodeInterpreter
+
+    retro_interpreter = RetrospectiveGPTCodeInterpreter(model="gpt-4")
+
+    instruction = """
+Write a Python script to solve the following problem:
+
+def get_row(lst, x):
+	\"\"\"
+	You are given a 2 dimensional data, as a nested lists,
+	which is similar to matrix, however, unlike matrices,
+	each row may contain a different number of columns.
+	Given lst, and integer x, find integers x in the list,
+	and return list of tuples, [(x1, y1), (x2, y2) ...] such that
+	each tuple is a coordinate - (row, columns), starting with 0.
+	Sort coordinates initially by rows in ascending order.
+	Also, sort coordinates of the row by columns in descending order.
+	
+	Examples:
+	get_row([
+	  [1,2,3,4,5,6],
+	  [1,2,3,4,1,6],
+	  [1,2,3,4,5,1]
+	], 1) == [(0, 0), (1, 4), (1, 0), (2, 5), (2, 0)]
+	get_row([], 1) == []
+	get_row([[], [1], [1, 2, 3]], 3) == [(2, 2)]
+	\"\"\"
+
+Ensure the solution is verified by printing the expected output.
+"""
+    # instruction = "Can you make a image of astraunaut in the garden?"
+
+    # example
+    retro_interpreter.chat(
+        user_message=instruction,
+        MAX_TRY=5,
+        use_retrospect=True,
+        feedback_prompt="Ensure the output matches the expected result, taking into account any corner cases. If discrepancies arise, pinpoint where you went wrong. Then, refine the code to achieve the desired outcome.",
+        VERBOSE=True,
+    )

Llama2-Code-Interpreter/code_interpreter/llama_hf.py

@@ -0,0 +1,101 @@
+from typing import Optional
+import os, sys
+
+from transformers import LlamaForCausalLM, LlamaTokenizer
+
+import torch
+from datetime import datetime
+
+sys.path.append(os.path.dirname(__file__))
+sys.path.append(os.path.dirname(os.path.abspath(__file__)))
+from utils.special_tok_llama2 import (
+    B_CODE,
+    E_CODE,
+    B_RESULT,
+    E_RESULT,
+    B_INST,
+    E_INST,
+    B_SYS,
+    E_SYS,
+    DEFAULT_PAD_TOKEN,
+    DEFAULT_BOS_TOKEN,
+    DEFAULT_EOS_TOKEN,
+    DEFAULT_UNK_TOKEN,
+    IGNORE_INDEX,
+)
+
+
+def create_peft_config(model):
+    from peft import (
+        get_peft_model,
+        LoraConfig,
+        TaskType,
+        prepare_model_for_int8_training,
+    )
+
+    peft_config = LoraConfig(
+        task_type=TaskType.CAUSAL_LM,
+        inference_mode=False,
+        r=8,
+        lora_alpha=32,
+        lora_dropout=0.05,
+        target_modules=["q_proj", "v_proj"],
+    )
+
+    # prepare int-8 model for training
+    model = prepare_model_for_int8_training(model)
+    model = get_peft_model(model, peft_config)
+    model.print_trainable_parameters()
+    return model, peft_config
+
+
+def build_model_from_hf_path(
+    hf_base_model_path: str = "./ckpt/llama-2-13b-chat",
+    load_peft: Optional[bool] = False,
+    peft_model_path: Optional[str] = None,
+    load_in_4bit: bool = True,
+):
+    start_time = datetime.now()
+
+    # build tokenizer
+    tokenizer = LlamaTokenizer.from_pretrained(
+        hf_base_model_path,
+        padding_side="right",
+        use_fast=False,
+    )
+
+    # Handle special tokens
+    special_tokens_dict = dict()
+    if tokenizer.pad_token is None:
+        special_tokens_dict["pad_token"] = DEFAULT_PAD_TOKEN  # 32000
+    if tokenizer.eos_token is None:
+        special_tokens_dict["eos_token"] = DEFAULT_EOS_TOKEN  # 2
+    if tokenizer.bos_token is None:
+        special_tokens_dict["bos_token"] = DEFAULT_BOS_TOKEN  # 1
+    if tokenizer.unk_token is None:
+        special_tokens_dict["unk_token"] = DEFAULT_UNK_TOKEN
+
+    tokenizer.add_special_tokens(special_tokens_dict)
+    tokenizer.add_tokens(
+        [B_CODE, E_CODE, B_RESULT, E_RESULT, B_INST, E_INST, B_SYS, E_SYS],
+        special_tokens=True,
+    )
+
+    # build model
+    model = LlamaForCausalLM.from_pretrained(
+        hf_base_model_path,
+        load_in_4bit=load_in_4bit,
+        device_map="auto",
+    )
+
+    model.resize_token_embeddings(len(tokenizer))
+
+    if load_peft and (peft_model_path is not None):
+        from peft import PeftModel
+
+        model = PeftModel.from_pretrained(model, peft_model_path)
+
+    end_time = datetime.now()
+    elapsed_time = end_time - start_time
+
+    return {"tokenizer": tokenizer, "model": model}

Llama2-Code-Interpreter/eval/gsm8k.py

@@ -0,0 +1,115 @@
+JSON_PATH = "/home/seungyoun/llama_related/llama_code_interpreter/eval/grade-school-math/grade_school_math/data/test.jsonl"
+
+import os, sys
+import random
+import json
+import re
+from tqdm import tqdm
+
+
+def load_jsonl(file_path, num_lines_to_display=5):
+    data = []
+    with open(file_path, "r") as f:
+        for line in f:
+            out_dict = json.loads(line)
+            answer_raw = out_dict["answer"]
+            answer_refined = answer_raw.split("####")[-1].strip()
+            out_dict["answer_refined"] = answer_refined
+            data.append(out_dict)  # dict_keys(['question', 'answer'])
+    return data
+
+
+few_shot_1 = [
+    {
+        "role": "user",
+        "content": "Sandra wants to buy some sweets. She saved $10 for this purpose. Her mother gave her an additional $4, and her father twice as much as her mother. One candy costs $0.5, and one jelly bean $0.2. She wants to buy 14 candies and 20 jelly beans. How much money will she be left with after the purchase?",
+    },
+    {
+        "role": "assistant",
+        "content": "```python\n# Saved Money\nsandra_money = 10\n# mother gave her\nmother_money = 4\n# father gave twice as much as her mother\nfather_money = 2 * mother_money\n# Cost\ncandy_cost = 0.5\njelly_bean_cost = 0.2\n\n# Calculations\ntotal_money = sandra_money + mother_money + father_money\n#Desired to buy\nShe wants to buy 14 candies and 20 jelly beans so the total cost would be\ntotal_cost = 14 * candy_cost + 20 * jelly_bean_cost\n# After the purchase\nremaining_money = total_money - total_cost\n\nprint(remaining_money)\n```\n```RESULT\n11.0\n```\nAfter buying 14 candies and 20 jelly beans, Sandra will be left with $11.",
+    },
+]
+
+few_shot_2 = [
+    {
+        "role": "user",
+        "content": "Keanu bought some fish from the store for his pets. He gave his cat half as many fish as he gave to his dog. If he gave his dog 40 fish, and each fish cost him $4, calculate the total amount of money he spent at the store to buy all the fish.",
+    },
+    {
+        "role": "assistant",
+        "content": "```python\ndef calculate_total_cost(dog_fish, fish_price):\n    cat_fish = dog_fish / 2\n    total_fish = dog_fish + cat_fish\n    total_cost = total_fish * fish_price\n    return total_cost\n\ntotal_cost = calculate_total_cost(40, 4)\nprint(total_cost)\n```\n```RESULT\n240.0\n```\nKeanu spent a total of $240 to buy all the fish from the store.\n### Answer : 240",
+    },
+]
+
+few_shot_3 = [
+    {
+        "role": "user",
+        "content": "Mike is feeding the birds at the park. He counts out the seeds as he throws them. He throws 20 seeds to the birds on the left. He throws twice as much to the bigger group of birds on the right. Some more birds flutter over to join the others for lunch and Mike throws 30 more seeds for them. If Mike has 30 seeds left to feed the last of the birds, how many seeds did Mike start with?",
+    },
+    {
+        "role": "assistant",
+        "content": "Let's break down the problem:\n\n- Mike throws 20 seeds to the birds on the left.\n- He throws twice as much to the bigger group of birds on the right, which is 2 * 20 = 40 seeds.\n- Some more birds join and Mike throws 30 more seeds for them.\n- Mike has 30 seeds left to feed the last of the birds.\n\nSo, the total number of seeds Mike started with is the sum of all these seeds.\n\n```python\nseeds_left = 20\nseeds_right = 2 * seeds_left\nseeds_more_birds = 30\nseeds_last = 30\n\ntotal_seeds = seeds_left + seeds_right + seeds_more_birds + seeds_last\nprint(total_seeds)\n```\n```RESULT\n120\n```\nFrom the calculations, we can conclude that Mike started with 120 seeds to feed the birds at the park.\n### Answer : 120",
+    },
+]
+
+if __name__ == "__main__":
+    sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+    from code_interpreter.LlamaCodeInterpreter import LlamaCodeInterpreter
+    import argparse
+
+    parser = argparse.ArgumentParser(description="Process path for LLAMA2_FINETUNEED.")
+    parser.add_argument(
+        "--path",
+        type=str,
+        required=True,
+        help="Path to the finetuned LLAMA2 model.",
+        default='"./output/llama-2-7b-chat-ci"',
+    )
+    args = parser.parse_args()
+    LLAMA2_FINETUNEED_PATH = args.path
+
+    interpreter = LlamaCodeInterpreter(
+        model_path=LLAMA2_FINETUNEED_PATH,
+        # load_in_4bit=True
+    )
+
+    sample_data = load_jsonl(JSON_PATH)
+    correct = 0
+
+    for idx, data_dict in enumerate(tqdm(sample_data)):
+        question = data_dict["question"]
+        answer = data_dict["answer"]
+        answer_only = data_dict["answer_refined"]
+
+        # reset dilag : fewshot to follow answering format
+        interpreter.dialog = [
+            {
+                "role": "system",
+                "content": "You are helpful robot that can generate code , excute it and debug then answer",
+            }
+        ]  # this will replaced in template conversion
+        # interpreter.dialog += few_shot_1
+        # interpreter.dialog += few_shot_2
+        # interpreter.dialog += few_shot_3
+
+        output = interpreter.chat(
+            user_message=f"{question}",
+            VERBOSE=True,
+        )
+
+        pattern = r"\[RESULT_TOK\]\s*(\d+(\.\d+)?)\s*\[/RESULT_TOK\]"
+        pred = -9212323  # for no code output (which is wrong answer)
+        if re.search(pattern, output["content"]):
+            pred = re.search(pattern, output["content"]).group(1)
+
+        pred = str(pred)
+        answer_only = str(answer_only)
+        if float(pred.replace(",", "")) == float(answer_only.replace(",", "")):
+            correct += 1
+
+        print("-" * 30)
+        print(f"\tThe question was : {question}")
+        print(f"\tThe answer was   : [{answer_only}]")
+        print(f"\tModel pred is    : [{pred}]")
+        print(f"\t    Accuracy     : [{correct/(idx+1)}]")
+        print("-" * 30)

Llama2-Code-Interpreter/eval/human_eval.py

@@ -0,0 +1,289 @@
+import os, sys
+import traceback
+
+HUMAN_EVAL_PATH = os.path.join(
+    os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))),
+    "human-eval",
+)
+
+sys.path.append(HUMAN_EVAL_PATH)
+from human_eval.data import write_jsonl, read_problems
+from finetuning.conversation_template import msg_to_code_result_tok_temp
+from code_interpreter.llama_hf import build_model_from_hf_path
+from code_interpreter.LlamaCodeInterpreter import LlamaCodeInterpreter
+from code_interpreter.GPTCodeInterpreter import GPTCodeInterpreter
+from code_interpreter.RetrospectiveGPTCodeInterpreter import (
+    RetrospectiveGPTCodeInterpreter,
+)
+
+import re
+
+from rich import print
+from rich.panel import Panel
+from rich.syntax import Syntax
+from rich.text import Text
+
+from timeout_decorator import timeout
+
+wrong = 0
+
+
+def extract_text(prompt, remove_lines=True):
+    token = '"""'
+    start = token
+    end = ">>>"
+    # end = '"""'
+
+    start_idx = prompt.find(start) + len(start)
+    end_idx = prompt.find(end)
+
+    output = prompt[start_idx:end_idx]
+    if remove_lines:
+        output = output.replace("\n", " ")
+    output = re.sub(r"\s+", " ", output).strip()
+
+    return output
+
+
+def extract_all_code_block(input_str: str) -> str:
+    pattern = r"\[CODE_START_TOK\](.*?)\[/CODE_END_TOK\]"
+    matches = re.findall(pattern, input_str, re.DOTALL)
+    return "\n".join([match.strip() for match in matches]) if matches else None
+
+
+def extract_all_code_block_gpt(input_str: str) -> str:
+    pattern = r"```python(.*?)```"
+    matches = re.findall(pattern, input_str, re.DOTALL)
+
+    return "\n".join([match.strip() for match in matches]) if matches else None
+
+
+def delete_print_asser(code_text: str):
+    lines = code_text.split("\n")
+    new_lines = list()
+    for i in lines:
+        if i.strip().startswith("print("):
+            continue
+        new_lines.append(i)
+
+    new_code_text = "\n".join(new_lines)
+    return new_code_text
+
+
+def extract_function_from_code_block(code_block: str) -> str:
+    lines = code_block.split("\n")
+    function_lines = []
+
+    inside_function = False
+    for line in lines:
+        # Start extracting from function definition
+        if line.startswith("def "):
+            inside_function = True
+
+        # If we are inside the function, append lines
+        if inside_function:
+            function_lines.append(line)
+
+            # If we encounter an unindented line that isn't a comment and isn't the start of another function, stop.
+            if (
+                not line.startswith("    ")
+                and not line.startswith("#")
+                and not line.startswith("def ")
+            ):
+                break
+
+    # Remove trailing comments or blank lines and the last line which caused the exit from the loop
+    while function_lines and (
+        function_lines[-1].strip() == ""
+        or function_lines[-1].strip().startswith("#")
+        or not function_lines[-1].startswith("    ")
+    ):
+        function_lines.pop()
+
+    return "\n".join(function_lines)
+
+
+def get_last_outermost_function_name(function_str):
+    matches = re.findall(r"^def (\w+)", function_str, re.MULTILINE)
+    if matches:
+        return matches[-1]  # Return the last (outermost) function name
+    return ""
+
+
+def get_last_function_name(function_str):
+    # Regular expression to match a function definition
+    matches = re.findall(r"def (\w+)", function_str)
+    if matches:
+        return matches[-1]  # Return the last function name
+    return ""
+
+
+def get_outermost_function_name(function_str):
+    matches = re.findall(r"^def (\w+)", function_str, re.MULTILINE)
+    if matches:
+        return matches[0]  # Return the first (outermost) function name
+    return ""
+
+
+def get_function_name(function_str):
+    # Regular expression to match a function definition
+    match = re.search(r"def (\w+)", function_str)
+    if match:
+        return match.group(0)
+    return ""
+
+
+def extract_test_assertion(test_func: str):
+    test_cases = list()
+    for i in test_func.split("\n"):
+        if "assert" in i:
+            test_cases.append(i.strip())
+
+    return ("\n".join(test_cases)).strip()
+
+
+import_str = """
+import re
+import math
+from typing import List, Tuple, Optional
+"""
+
+
+@timeout(100, timeout_exception=TimeoutError)
+def exec_with_timeout(import_str, full_test_code):
+    env = {**locals()}
+    code_to_exec = f"{import_str}\n{full_test_code}"
+    try:
+        exec(code_to_exec, env)
+    except Exception as e:
+        print(f"Error Type: {type(e).__name__}, Error Message: {e}")
+        return False  # Return False if there's an error during execution
+    return True  # Return True if executed without errors
+
+
+if __name__ == "__main__":
+    sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+    import argparse
+
+    parser = argparse.ArgumentParser(description="Process path for LLAMA2_FINETUNEED.")
+    parser.add_argument(
+        "--path",
+        type=str,
+        required=True,
+        help="Path to the finetuned LLAMA2 model.",
+        default='"./output/llama-2-7b-chat-ci"',
+    )
+    parser.add_argument(
+        "--model",
+        type=str,
+        required=False,
+        help="Path to the finetuned LLAMA2 model.",
+        default='"./output/llama-2-7b-chat-ci"',
+    )
+    parser.add_argument(
+        "--max-retry",
+        type=int,
+        required=False,
+        help="Maximum number of retries.",
+        default=5,  # You can set any default value you want here.
+    )
+    args = parser.parse_args()
+    PROGRAMMING_PUZZLE_Q = True
+
+    problems = read_problems()
+    correct_total = 0
+    total_problems = len(problems)
+
+    for idx, task_id in enumerate(problems):
+        if "gpt" not in args.model.lower():
+            LLAMA2_FINETUNEED_PATH = args.path
+            interpreter = LlamaCodeInterpreter(
+                model_path=LLAMA2_FINETUNEED_PATH,
+                # load_in_4bit=True
+            )
+        else:
+            interpreter = RetrospectiveGPTCodeInterpreter(
+                model=args.model,
+            )
+
+        # dict_keys(['task_id', 'prompt', 'entry_point', 'canonical_solution', 'test'])
+        programming_puzzle = problems[task_id]["prompt"].replace("    ", "\t")
+        text_only_problem = extract_text(programming_puzzle)
+
+        interpreter.dialog = [
+            {
+                "role": "system",
+                "content": "You are helpful robot that can generate code , excute it and debug then answer",
+            }
+        ]
+
+        if PROGRAMMING_PUZZLE_Q:
+            # programming puzzle
+            output_str = interpreter.chat(
+                user_message=f"Write a Python script to solve the following problem:\n{programming_puzzle}\nEnsure the solution is verified by printing the expected output.",
+                MAX_TRY=args.max_retry,
+                VERBOSE=True,
+                code_exec_prefix=f"\nfrom typing import List,Tuple\nimport math\n",
+                feedback_prompt="Ensure the output matches the expected result, taking into account any corner cases. If discrepancies arise, pinpoint where you went wrong. Then, refine the code to achieve the desired outcome.",
+                append_result=True,
+            )["content"]
+
+        else:
+            output_str = interpreter.chat(
+                user_message=f"Write a Python script for this problem:\n{text_only_problem}",
+                MAX_TRY=args.max_retry,
+                VERBOSE=True,
+                code_exec_prefix=f"\nfrom typing import List,Tuple\nimport math\n",
+                feedback_prompt="Ensure the output matches the expected result. If not tell where you got wrong, then refine the code to achieve the desired outcome.",
+                append_result=True,
+            )["content"]
+
+        function_str = ""
+        if "gpt" not in args.model.lower():
+            code_block = extract_all_code_block(output_str)
+        else:
+            code_block = extract_all_code_block_gpt(output_str)
+        if (code_block is not None) and ("def" in code_block):
+            function_str = code_block
+
+        # function_name = get_last_outermost_function_name(function_str)
+        function_str = delete_print_asser(function_str)
+        function_name = get_last_outermost_function_name(function_str)
+        full_test_code = f"{function_str}\n#-----------\n{problems[task_id]['test']}\ncheck({function_name})"
+
+        # Print the full_test_code with syntax highlighting
+        syntax = Syntax(
+            # f"{programming_puzzle}\n{full_test_code}",
+            f"{full_test_code}",
+            "python",
+            theme="monokai",
+            line_numbers=True,
+        )
+        print(syntax)
+
+        is_correct = False  # default is wrong
+        timeout_flag = False
+        try:
+            is_correct = exec_with_timeout(import_str, full_test_code)
+        except TimeoutError as e:
+            timeout_flag = True
+            print(f"Timeout with error msg : {e}")
+
+        if is_correct:
+            correct_total += 1
+
+        acc = (correct_total) / (idx + 1)
+        # save dialog
+        interpreter.save_dialog(
+            path=f"./eval/gpt_humaneval_output/{task_id.replace('/','_')}_{is_correct}.json"
+        )
+        interpreter.close()
+        del interpreter
+
+        # Constructing the output
+        accuracy_text = Text(
+            f"Accuracy: {correct_total}/{idx+1}[{total_problems}] = {acc:.2%} [{is_correct}]",
+            style="bold blue",
+        )
+        panel = Panel(accuracy_text, title="Results", border_style="green")
+        print(panel)

Llama2-Code-Interpreter/eval/inference.py

@@ -0,0 +1,204 @@
+from dataclasses import dataclass, field
+from typing import Dict, Optional, Sequence
+import logging
+import os, sys
+import copy
+
+import torch
+import transformers
+from transformers import LlamaForCausalLM, LlamaTokenizer, TextStreamer
+
+from torch.utils.data import Dataset
+from transformers import Trainer
+
+import torch
+from rich.console import Console
+from rich.table import Table
+from datetime import datetime
+from threading import Thread
+
+sys.path.append(os.path.dirname(__file__))
+sys.path.append(os.path.dirname(os.path.dirname(__file__)))
+from utils.special_tok_llama2 import (
+    B_CODE,
+    E_CODE,
+    B_RESULT,
+    E_RESULT,
+    B_INST,
+    E_INST,
+    B_SYS,
+    E_SYS,
+    DEFAULT_PAD_TOKEN,
+    DEFAULT_BOS_TOKEN,
+    DEFAULT_EOS_TOKEN,
+    DEFAULT_UNK_TOKEN,
+    IGNORE_INDEX,
+)
+
+from finetuning.conversation_template import (
+    json_to_code_result_tok_temp,
+    msg_to_code_result_tok_temp,
+)
+
+import warnings
+
+warnings.filterwarnings("ignore", category=UserWarning, module="transformers")
+os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
+
+console = Console()  # for pretty print
+
+
+@dataclass
+class ModelArguments:
+    model_name_or_path: Optional[str] = field(default="./output/llama-2-7b-chat-ci")
+    load_peft: Optional[bool] = field(default=False)
+    peft_model_name_or_path: Optional[str] = field(
+        default="./output/llama-2-7b-chat-ci"
+    )
+
+
+def create_peft_config(model):
+    from peft import (
+        get_peft_model,
+        LoraConfig,
+        TaskType,
+        prepare_model_for_int8_training,
+    )
+
+    peft_config = LoraConfig(
+        task_type=TaskType.CAUSAL_LM,
+        inference_mode=False,
+        r=8,
+        lora_alpha=32,
+        lora_dropout=0.05,
+        target_modules=["q_proj", "v_proj"],
+    )
+
+    # prepare int-8 model for training
+    model = prepare_model_for_int8_training(model)
+    model = get_peft_model(model, peft_config)
+    model.print_trainable_parameters()
+    return model, peft_config
+
+
+def build_model_from_hf_path(
+    hf_base_model_path: str = "./ckpt/llama-2-13b-chat",
+    load_peft: Optional[bool] = False,
+    peft_model_path: Optional[str] = None,
+):
+    start_time = datetime.now()
+
+    # build tokenizer
+    console.log("[bold cyan]Building tokenizer...[/bold cyan]")
+    tokenizer = LlamaTokenizer.from_pretrained(
+        hf_base_model_path,
+        padding_side="right",
+        use_fast=False,
+    )
+
+    # Handle special tokens
+    console.log("[bold cyan]Handling special tokens...[/bold cyan]")
+    special_tokens_dict = dict()
+    if tokenizer.pad_token is None:
+        special_tokens_dict["pad_token"] = DEFAULT_PAD_TOKEN  # 32000
+    if tokenizer.eos_token is None:
+        special_tokens_dict["eos_token"] = DEFAULT_EOS_TOKEN  # 2
+    if tokenizer.bos_token is None:
+        special_tokens_dict["bos_token"] = DEFAULT_BOS_TOKEN  # 1
+    if tokenizer.unk_token is None:
+        special_tokens_dict["unk_token"] = DEFAULT_UNK_TOKEN
+
+    tokenizer.add_special_tokens(special_tokens_dict)
+    tokenizer.add_tokens(
+        [B_CODE, B_RESULT, E_RESULT, B_INST, E_INST, B_SYS, E_SYS],
+        special_tokens=True,
+    )
+
+    # build model
+    console.log("[bold cyan]Building model...[/bold cyan]")
+    model = LlamaForCausalLM.from_pretrained(
+        hf_base_model_path,
+        load_in_4bit=True,
+        device_map="auto",
+    )
+
+    model.resize_token_embeddings(len(tokenizer))
+
+    if load_peft and (peft_model_path is not None):
+        from peft import PeftModel
+
+        model = PeftModel.from_pretrained(model, peft_model_path)
+        console.log("[bold green]Peft Model Loaded[/bold green]")
+
+    end_time = datetime.now()
+    elapsed_time = end_time - start_time
+
+    # Log time performance
+    table = Table(title="Time Performance")
+    table.add_column("Task", style="cyan")
+    table.add_column("Time Taken", justify="right")
+    table.add_row("Loading model", str(elapsed_time))
+    console.print(table)
+
+    console.log("[bold green]Model Loaded[/bold green]")
+    return {"tokenizer": tokenizer, "model": model}
+
+
+@torch.inference_mode()
+def inference(
+    user_input="What is 100th fibo num?",
+    max_new_tokens=512,
+    do_sample: bool = True,
+    use_cache: bool = True,
+    top_p: float = 1.0,
+    temperature: float = 0.1,
+    top_k: int = 50,
+    repetition_penalty: float = 1.0,
+):
+    parser = transformers.HfArgumentParser(ModelArguments)
+    model_args = parser.parse_args_into_dataclasses()[0]
+
+    model_dict = build_model_from_hf_path(
+        hf_base_model_path=model_args.model_name_or_path,
+        load_peft=model_args.load_peft,
+        peft_model_path=model_args.peft_model_name_or_path,
+    )
+
+    model = model_dict["model"]
+    tokenizer = model_dict["tokenizer"]
+
+    streamer = TextStreamer(tokenizer, skip_prompt=True)
+
+    # peft
+    # create peft config
+    model.eval()
+
+    user_prompt = msg_to_code_result_tok_temp(
+        [{"role": "user", "content": f"{user_input}"}]
+    )
+    # Printing user's content in blue
+    console.print("\n" + "-" * 20, style="#808080")
+    console.print(f"###User : {user_input}\n", style="blue")
+
+    prompt = f"{user_prompt}\n###Assistant :"
+    # prompt = f"{user_input}\n### Assistant : Here is python code to get the 55th fibonacci number {B_CODE}\n"
+
+    inputs = tokenizer([prompt], return_tensors="pt")
+
+    generated_text = model.generate(
+        **inputs,
+        streamer=streamer,
+        max_new_tokens=max_new_tokens,
+        do_sample=do_sample,
+        top_p=top_p,
+        temperature=temperature,
+        use_cache=use_cache,
+        top_k=top_k,
+        repetition_penalty=repetition_penalty,
+    )
+
+    return generated_text
+
+
+if __name__ == "__main__":
+    inference(user_input="what is sin(44)?")

Llama2-Code-Interpreter/finetuning/codellama_wrapper.py

@@ -0,0 +1,21 @@
+from transformers import LlamaForCausalLM, LlamaTokenizer
+from transformers import LlamaModel, LlamaConfig
+import torch.nn as nn
+
+CODELLAMA_VOCAB_SIZE = 32016
+
+
+class CodeLlamaForCausalLM(LlamaForCausalLM):
+    _tied_weights_keys = ["lm_head.weight"]
+
+    def __init__(self, config):
+        super().__init__(config)
+        self.model = LlamaModel(config)
+        self.vocab_size = config.vocab_size
+        self.lm_head = nn.Linear(config.hidden_size, CODELLAMA_VOCAB_SIZE, bias=False)
+        self.model.embed_tokens = nn.Embedding(
+            CODELLAMA_VOCAB_SIZE, config.hidden_size, config.pad_token_id
+        )
+
+        # Initialize weights and apply final processing
+        self.post_init()

Llama2-Code-Interpreter/finetuning/conversation_template.py

@@ -0,0 +1,80 @@
+import os, sys
+
+sys.path.append(os.path.dirname(os.path.dirname(__file__)))
+
+import json
+import re
+from typing import List, Dict
+
+DATA_DIR = "gpt_data_gen"
+
+B_CODE = "[CODE_START_TOK]"
+E_CODE = "[/CODE_END_TOK]"
+
+B_RESULT = "[RESULT_TOK]"
+E_RESULT = "[/RESULT_TOK]"
+
+B_INST, E_INST = "[INST]", "[/INST]"
+B_SYS, E_SYS = "<<SYS>>", "<</SYS>>"
+
+BOS = "<s>"
+EOS = "</s>"
+
+CODE_SYS_PROMPT_FOR_TRAIN = """
+You are 'CodeLLama', an advanced Language Model assistant that can generate, execute, and evaluate code. 
+Respond to user queries by providing code-based solutions and insights.
+"""
+
+
+def msg_to_code_result_tok_temp(msg: List[Dict]) -> str:
+    full_str = f"{BOS}{B_INST} {B_SYS}\n{CODE_SYS_PROMPT_FOR_TRAIN}\n{E_SYS}\n\n"
+
+    user_first_flag = True
+    for idx, chat in enumerate(msg):
+        if chat["role"] == "system":
+            continue
+        if chat["role"].lower() == "user":
+            chat["content"] = chat["content"]
+            if user_first_flag:
+                full_str += f"{chat['content']} {E_INST}"
+                user_first_flag = False
+            else:
+                full_str += f"{BOS}{B_INST}{chat['content']} {E_INST}"
+        elif chat["role"] == "assistant":
+            chat["content"] = chat["content"].replace(
+                "/home/seungyoun/llama_code_interpreter/", "./"
+            )
+
+            # Replace the code block start and end markers using regex
+            code_pattern = re.compile(r"```python\n(.*?)```", re.DOTALL)
+            chat["content"] = code_pattern.sub(
+                r"[CODE_START_TOK]\n\1[/CODE_END_TOK]", chat["content"]
+            )
+
+            # Replace the result block start and end markers using regex
+            result_pattern = re.compile(r"```RESULTS?\n(.*?)```", re.DOTALL)
+            chat["content"] = result_pattern.sub(
+                r"[RESULT_TOK]\n\1[/RESULT_TOK]", chat["content"]
+            )
+
+            full_str += f"{chat['content']}{EOS}"
+
+    full_str = full_str.replace("')()", "')")
+    full_str = full_str.replace("/home/seungyoun/llama_code_interpreter/", "./")
+
+    return full_str
+
+
+def json_to_code_result_tok_temp(json_file_name: str = "425.json") -> str:
+    file_rel_path = os.path.join(DATA_DIR, json_file_name)
+
+    with open(file_rel_path, "r") as json_file:
+        msg = json.load(json_file)
+
+    full_str = msg_to_code_result_tok_temp(msg)
+
+    return full_str
+
+
+if __name__ == "__main__":
+    print(json_to_code_result_tok_temp())

Llama2-Code-Interpreter/finetuning/train.py

@@ -0,0 +1,336 @@
+from dataclasses import dataclass, field
+from typing import Dict, Optional, Sequence
+import logging
+import os, sys
+import copy
+
+import torch
+import transformers
+from transformers import LlamaForCausalLM, LlamaTokenizer
+
+from torch.utils.data import Dataset
+from transformers import Trainer
+
+sys.path.append(os.path.dirname(__file__))
+sys.path.append(os.path.dirname(os.path.dirname(__file__)))
+from utils.special_tok_llama2 import (
+    B_CODE,
+    E_CODE,
+    B_RESULT,
+    E_RESULT,
+    B_INST,
+    E_INST,
+    B_SYS,
+    E_SYS,
+    DEFAULT_PAD_TOKEN,
+    DEFAULT_BOS_TOKEN,
+    DEFAULT_EOS_TOKEN,
+    DEFAULT_UNK_TOKEN,
+    IGNORE_INDEX,
+)
+
+from conversation_template import json_to_code_result_tok_temp
+
+
+@dataclass
+class ModelArguments:
+    model_name_or_path: Optional[str] = field(default="./ckpt/llama-2-13b-chat")
+    peft: bool = field(default=False)
+
+
+@dataclass
+class DataArguments:
+    data_path: str = field(
+        default=None, metadata={"help": "Path to the training data."}
+    )
+
+
+@dataclass
+class TrainingArguments(transformers.TrainingArguments):
+    cache_dir: Optional[str] = field(default=None)
+    optim: str = field(default="adamw_torch")
+    model_max_length: int = field(
+        default=4096,
+        metadata={
+            "help": "Maximum sequence length. Sequences will be right padded (and possibly truncated)."
+        },
+    )
+
+
+def create_peft_config(model):
+    from peft import (
+        get_peft_model,
+        LoraConfig,
+        TaskType,
+        prepare_model_for_int8_training,
+    )
+
+    peft_config = LoraConfig(
+        task_type=TaskType.CAUSAL_LM,
+        inference_mode=False,
+        r=8,
+        lora_alpha=16,
+        lora_dropout=0.05,
+        target_modules=["q_proj", "v_proj"],
+    )
+
+    # prepare int-8 model for training
+    model = prepare_model_for_int8_training(model)
+    model = get_peft_model(model, peft_config)
+    model.print_trainable_parameters()
+    print(f"Using Peft")
+    return model, peft_config
+
+
+def _tokenize_fn(
+    strings: Sequence[str], tokenizer: transformers.PreTrainedTokenizer
+) -> Dict:
+    """Tokenize a list of strings."""
+    tokenized_list = [
+        tokenizer(
+            text,
+            return_tensors="pt",
+            padding="longest",
+            max_length=tokenizer.model_max_length,
+            truncation=True,
+        )
+        for text in strings
+    ]
+
+    input_ids = [tokenized.input_ids[0] for tokenized in tokenized_list]
+    input_ids_lens = [
+        tokenized.input_ids.ne(tokenizer.pad_token_id).sum().item()
+        for tokenized in tokenized_list
+    ]
+    return dict(
+        input_ids=input_ids,
+        input_ids_lens=input_ids_lens,
+    )
+
+
+def find_all_sublist_end(main_list, sublist):
+    """Find all the ending indices of a sublist in a main list."""
+    sublist_len = len(sublist)
+    main_list = main_list.tolist()
+    indices = []
+    for index in (i for i, e in enumerate(main_list) if e == sublist[0]):
+        if main_list[index : index + sublist_len] == sublist:
+            indices.append(index + sublist_len)
+    return indices
+
+
+def find_all_sublist_start(main_list, sublist):
+    """Find all the starting indices of a sublist in a main list."""
+    sublist_len = len(sublist)
+    main_list = main_list.tolist()
+    indices = []
+    for index in (i for i, e in enumerate(main_list) if e == sublist[0]):
+        if main_list[index : index + sublist_len] == sublist:
+            indices.append(index)
+    return indices
+
+
+def preprocess(
+    trajs: Sequence[str],
+    tokenizer: transformers.PreTrainedTokenizer,
+) -> Dict:
+    INST_START_INDEX = tokenizer.encode(f"{B_INST}")[-1]
+    INST_END_INDEX = tokenizer.encode(f"{E_INST}")[-1]
+    RESULT_START_INDEX = tokenizer.encode(f"{B_RESULT}")[-1]
+    RESULT_END_INDEX = tokenizer.encode(f"{E_RESULT}")[-1]
+
+    """Preprocess the data by tokenizing."""
+    examples_tokenized = _tokenize_fn(trajs, tokenizer)
+
+    input_ids_lens = examples_tokenized["input_ids_lens"]
+    input_ids = examples_tokenized["input_ids"]  # [torch.tensor , torch.tensor , ...]
+    labels = copy.deepcopy(input_ids)
+
+    # IGNORE INDEX SET
+    for i, label in enumerate(labels):
+        user_start_inds = find_all_sublist_start(label, [INST_START_INDEX])
+        assistant_start_inds = find_all_sublist_end(label, [INST_END_INDEX])
+
+        result_start_inds = find_all_sublist_start(label, [RESULT_START_INDEX])
+        result_end_inds = find_all_sublist_end(label, [RESULT_END_INDEX])
+
+        # for debug
+        # for len_i, ind in enumerate(label):
+        #    print(f'{len_i}|{ind} -> "{tokenizer.decode(ind)}"')
+
+        assert len(user_start_inds) == len(
+            assistant_start_inds
+        ), f"User and Assistant pair should be equal :: \n\tUser [{user_start_inds}]/\n\tAssistant [{assistant_start_inds}]\n\n Text : \n{trajs[i]}"
+
+        assert len(result_start_inds) == len(
+            result_end_inds
+        ), f"Start and End indices pairs do not match.: : \nText : \n{trajs[i]}"
+
+        for user_start_ind, assistant_start_ind in zip(
+            user_start_inds, assistant_start_inds
+        ):
+            label[user_start_ind + 1 : assistant_start_ind - 1] = IGNORE_INDEX
+
+        for start, end in zip(result_start_inds, result_end_inds):
+            label[start + 1 : end - 1] = IGNORE_INDEX
+
+    # cut max length
+    input_ids = [i[:1500] for i in input_ids]
+    labels = [i[:1500] for i in labels]
+
+    return dict(input_ids=input_ids, labels=labels)
+
+
+class SupervisedDataset(Dataset):
+    """Dataset for supervised fine-tuning."""
+
+    def __init__(self, data_path: str, tokenizer: transformers.PreTrainedTokenizer):
+        super(SupervisedDataset, self).__init__()
+        logging.warning(f"Loading data from data path : {data_path}")
+        all_json = os.listdir(data_path)
+
+        trajs = list()
+        for json_file_name in all_json:
+            traj = json_to_code_result_tok_temp(json_file_name=json_file_name)
+            trajs.append(traj)
+
+        logging.warning("Tokenizing inputs... This may take some time...")
+        data_dict = preprocess(trajs, tokenizer)
+
+        self.input_ids = data_dict["input_ids"]
+        self.labels = data_dict["labels"]
+
+    def __len__(self):
+        return len(self.input_ids)
+
+    def __getitem__(self, i) -> Dict[str, torch.Tensor]:
+        return dict(input_ids=self.input_ids[i], labels=self.labels[i])
+
+
+@dataclass
+class DataCollatorForSupervisedDataset(object):
+    """Collate examples for supervised fine-tuning."""
+
+    tokenizer: transformers.PreTrainedTokenizer
+
+    def __call__(self, instances: Sequence[Dict]) -> Dict[str, torch.Tensor]:
+        input_ids, labels = tuple(
+            [instance[key] for instance in instances] for key in ("input_ids", "labels")
+        )
+        input_ids = torch.nn.utils.rnn.pad_sequence(
+            input_ids, batch_first=True, padding_value=self.tokenizer.pad_token_id
+        )
+        labels = torch.nn.utils.rnn.pad_sequence(
+            labels, batch_first=True, padding_value=IGNORE_INDEX
+        )
+        return dict(
+            input_ids=input_ids,
+            labels=labels,
+            attention_mask=input_ids.ne(self.tokenizer.pad_token_id),
+        )
+
+
+def make_supervised_data_module(
+    tokenizer: transformers.PreTrainedTokenizer, data_args
+) -> Dict:
+    """Make dataset and collator for supervised fine-tuning."""
+    train_dataset = SupervisedDataset(
+        tokenizer=tokenizer, data_path=data_args.data_path
+    )
+    data_collator = DataCollatorForSupervisedDataset(tokenizer=tokenizer)
+    return dict(
+        train_dataset=train_dataset, eval_dataset=None, data_collator=data_collator
+    )
+
+
+def build_model_from_hf_path(
+    hf_model_path: str = "./ckpt/llama-2-13b-chat", peft: bool = False
+):
+    # build tokenizer
+    tokenizer = LlamaTokenizer.from_pretrained(
+        hf_model_path,
+        padding_side="right",
+        use_fast=False,
+    )
+
+    special_tokens_dict = dict()
+    if tokenizer.pad_token is None:
+        special_tokens_dict["pad_token"] = DEFAULT_PAD_TOKEN  # 32000
+    if tokenizer.eos_token is None:
+        special_tokens_dict["eos_token"] = DEFAULT_EOS_TOKEN  # 2
+    if tokenizer.bos_token is None:
+        special_tokens_dict["bos_token"] = DEFAULT_BOS_TOKEN  # 1
+    if tokenizer.unk_token is None:
+        special_tokens_dict["unk_token"] = DEFAULT_UNK_TOKEN
+
+    tokenizer.add_special_tokens(special_tokens_dict)
+
+    tokenizer.add_tokens(
+        [
+            B_CODE,  # 32001
+            E_CODE,  # 32002
+            B_RESULT,  # 32003
+            E_RESULT,  # 32004
+            B_INST,
+            E_INST,
+            B_SYS,
+            E_SYS,  # 32008
+        ],
+        special_tokens=True,
+    )
+
+    # build model
+    if peft:
+        model = LlamaForCausalLM.from_pretrained(
+            hf_model_path,
+            load_in_8bit=True,
+            device_map="auto",
+            ignore_mismatched_sizes=True,
+            torch_dtype=torch.float16,
+        )
+    else:
+        # for llama
+        # model = LlamaForCausalLM.from_pretrained(
+        #    hf_model_path, ignore_mismatched_sizes=True
+        # )
+
+        # for codellama
+        from codellama_wrapper import CodeLlamaForCausalLM
+
+        model = CodeLlamaForCausalLM.from_pretrained(hf_model_path)
+
+    model.resize_token_embeddings(len(tokenizer))
+
+    return {"tokenizer": tokenizer, "model": model}
+
+
+def train():
+    parser = transformers.HfArgumentParser(
+        (ModelArguments, DataArguments, TrainingArguments)
+    )
+    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
+
+    model_dict = build_model_from_hf_path(
+        hf_model_path=model_args.model_name_or_path, peft=model_args.peft
+    )
+
+    model, tokenizer = model_dict["model"], model_dict["tokenizer"]
+    # peft setting
+    model.train()
+    if model_args.peft:
+        model, lora_config = create_peft_config(model)
+
+    # make dataset
+    data_module = make_supervised_data_module(tokenizer=tokenizer, data_args=data_args)
+    trainer = Trainer(
+        model=model, tokenizer=tokenizer, args=training_args, **data_module
+    )
+
+    # train
+    trainer.train()
+    trainer.save_state()
+    trainer.save_model(output_dir=training_args.output_dir)
+
+
+if __name__ == "__main__":
+    train()

Llama2-Code-Interpreter/prompt/gpt4_prompt.py

@@ -0,0 +1,277 @@
+few_shot_1 = [
+    {"role": "user", "content": "what is 100th fibonacci sequence?"},
+    {
+        "role": "assistant",
+        "content": """
+```python
+def fibonacci(n):
+    a, b = 0, 1
+    for _ in range(n):
+        a, b = b, a + b
+    return a
+
+fibo_100 = fibonacci(100)
+print(fibo_100)
+```
+```RESULT
+354224848179261915075
+```
+""",
+    },
+    {"role": "user", "content": "Keep going"},
+    {
+        "role": "assistant",
+        "content": "The 100th Fibonacci number is 354224848179261915075",
+    },
+]
+
+few_shot_2 = [
+    {
+        "role": "user",
+        "content": "Can you write a Python script that uses a machine learning algorithm to predict stock prices using historical data?",
+    },
+    {
+        "role": "assistant",
+        "content": """
+Sure, here is a simple example of how you might use a linear regression model from the `sklearn` library to predict stock prices. 
+This example uses the `pandas_datareader` library to fetch historical stock data from Yahoo Finance.
+
+Please note that this is a very basic example and real-world stock price prediction is a complex task that involves much more than just historical prices. 
+Other factors such as company earnings, economic indicators, and market sentiment also play a big role.
+
+```python
+import pandas as pd
+import numpy as np
+import yfinance as yf
+from datetime import datetime as dt
+
+# Get the stock data
+df = yf.download('AAPL', start='2012-01-01', end='2022-01-01')
+
+# Prepare the data for the model
+df['Date'] = df.index
+df['Date'] = pd.to_datetime(df['Date'])
+df['Date'] = df['Date'].map(dt.toordinal)
+
+df.head(3)
+```
+```RESULT
+[*********************100%***********************]  1 of 1 completed
+                 Open       High        Low      Close  Adj Close     Volume   
+Date                                                                           
+2012-01-03  14.621429  14.732143  14.607143  14.686786  12.482925  302220800  \
+2012-01-04  14.642857  14.810000  14.617143  14.765714  12.550011  260022000   
+2012-01-05  14.819643  14.948214  14.738214  14.929643  12.689340  271269600   
+
+              Date  
+Date                
+2012-01-03  734505  
+2012-01-04  734506  
+2012-01-05  734507  
+```
+
+""",
+    },
+    {"role": "user", "content": "Keep going"},
+    {
+        "role": "assistant",
+        "content": """
+let's continue with the next steps which involve training the model, making predictions, and evaluating the model.
+
+```python
+from sklearn.model_selection import train_test_split 
+from sklearn.linear_model import LinearRegression
+from sklearn import metrics
+
+# Independent variable
+X = df['Date'].values.reshape(-1,1)
+# Dependent variable
+y = df['Close'].values.reshape(-1,1)
+
+# Split the data into training and testing sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
+
+# Train the model
+model = LinearRegression()
+model.fit(X_train, y_train)
+
+# Make predictions
+y_pred = model.predict(X_test)
+
+# Evaluate the model
+print('Mean Absolute Error: {:.2f}'.format(metrics.mean_absolute_error(y_test, y_pred)))
+print('Mean Squared Error: {:.2f}'.format(metrics.mean_squared_error(y_test, y_pred)))
+print('Root Mean Squared Error: {:.2f}'.format(np.sqrt(metrics.mean_squared_error(y_test, y_pred))))
+```
+
+```RESULT
+Mean Absolute Error: 17.85
+Mean Squared Error: 491.52
+Root Mean Squared Error: 22.17
+```
+
+""",
+    },
+    {"role": "user", "content": "Keep going"},
+    {
+        "role": "assistant",
+        "content": "This basic Linear Regression model serves as an introduction to stock price prediction, but real-world applications often employ more complex models. The prediction accuracy can be improved by using other models such as ARIMA, LSTM, or incorporating more data points.\nAlways exercise caution when using such models for financial decisions.",
+    },
+]
+
+few_shot_3 = [
+    {
+        "role": "user",
+        "content": "I want to analyze the latest news articles about a specific topic. Let's say chatgpt.",
+    },
+    {
+        "role": "assistant",
+        "content": """
+Sure, we can use the GoogleSearch tool to find the latest articles about "chatgpt". 
+
+```python
+
+# Search for the latest articles about "chatgpt"
+results = google_search("chatgpt latest news")
+print(results)
+```
+```RESULT
+[1] Title : [The latest update for ChatGPT - Release Notes]
+        snippet : Updates to ChatGPT (August 3) · Introducing the ChatGPT app for Android (July 25) · Custom instructions is rolling out in beta (July 20) · Higher message limits ...
+        link : https://help.openai.com/en/articles/6825453-chatgpt-release-notes
+...
+[6] Title : [Bing, Bard, ChatGPT, and all the news on AI chatbots - The Verge]
+        snippet : Over the last few months company has posted dozens of AI jobs in the US, France, and China, looking to fill roles that could help build generative AI tools that ...
+        link : https://www.theverge.com/23610427/chatbots-chatgpt-new-bing-google-bard-conversational-ai
+```
+
+""",
+    },
+    {"role": "user", "content": "Keep going"},
+    {
+        "role": "assistant",
+        "content": """
+To analyze the latest news articles about a specific topic here is step-by-step plan :
+
+# 1. Web Scraping: Use the GoogleSearch class to get a list of URLs of the latest news articles about the given topic. Then use a library like BeautifulSoup to scrape the main body of each article.
+
+```python
+from bs4 import BeautifulSoup
+import re
+
+def scrape_article(url, retries=3):
+    headers = {"User-Agent": "Mozilla/5.0"}
+    for _ in range(retries):
+        try:
+            response = requests.get(url, headers=headers)
+            response.raise_for_status()  # Raise an exception if the status code is not 200
+            break
+        except requests.exceptions.RequestException:
+            print(f"Error occurred, retrying {url}")
+            time.sleep(5)  # Wait 5 seconds before retrying
+    else:  # No successful request after retries
+        print(f"Failed to retrieve {url} after {retries} attempts.")
+        return ""
+
+    soup = BeautifulSoup(response.text, 'html.parser')
+    
+    paragraphs = soup.find_all('p')
+    article_text = ' '.join([p.get_text() for p in paragraphs])
+    return article_text
+
+url_results = (somehow filter url only with re)
+articles = [scrape_article(url) for url in url_results]
+```
+
+
+""",
+    },
+    {"role": "user", "content": "Keep going"},
+]
+
+
+few_shot_4 = [
+    {
+        "role": "user",
+        "content": "Find walmart same store sales growth from Q1 2019 to Q3 2020 by using official data such as Wal-Mart's 10-Q and 10-K disclosures",
+    },
+    {
+        "role": "assistant",
+        "content": """
+
+To find Walmart's same store sales growth from Q1 2019 to Q3 2020, we need to find Reliable Source first.
+
+# 1. Identify a Reliable Source:
+
+To Identify a Reliable Source I need to use google_search function to search the Reliable source
+
+```python
+results = google_search("Walmart Investor Relations 10-Q and 10-K reports")
+print(results)
+```
+```RESULT
+[1] Title : [Walmart SEC Filings]
+        snippet : SEC Filings. Walmart provides its filings with the U.S. Securities and Exchange Commission. ... Annual Report of Employee Stock Plans.
+        link : https://stock.walmart.com/financials/sec-filings/default.aspx
+...
+[6] Title : [Form 10-K to "Walmart]
+        snippet : Our SEC filings, our Code of Ethics for our CEO and senior financial officers and our Statement of Ethics can be found on our website at www.stock.walmart.com.
+        link : https://www.sec.gov/Archives/edgar/data/104169/000010416919000016/wmtform10-kx1312019.htm
+```
+
+""",
+    },
+    {"role": "user", "content": "Keep going"},
+    {
+        "role": "assistant",
+        "content": """
+
+Next step is to 
+
+# 2. Choose a Source for Detailed Financial Disclosures and scraping
+
+From the results provided, there are six main contenders:
+
+Among those [6] the direct link from the U.S. Securities and Exchange Commission's (SEC) EDGAR database.
+Given the ease of scraping .htm format with libraries like lxml, and the authenticity of the data from the SEC's official site, I would recommend choosing the link from the EDGAR system.
+
+```python
+chosen_link = "https://www.sec.gov/Archives/edgar/data/104169/000010416919000016/wmtform10-kx1312019.htm"
+
+# define headers 
+headers = {
+    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
+
+# print request to see the contents
+```
+
+
+""",
+    },
+]
+
+extra_prompt = """
+
++ You are so have to check the code is working right. (by printing or plotting)
++ Do not leave function alone. Make sure call the function to check it is working correctly
++ As an Code Interperter, You aare able to browse the internet or access documents directly (by using beautifulsoup or requests this will need cleaning the text)
++ Provide Dummy data and test the function if needed
++ 'Do not' pip install 
++ You must need to use datetime to check current date
+    For Example,
+from datetime import datetime, timedelta
+# Download data for 180days
+data = yf.download('GOOGL', start=datetime.today(), end=end_date - timedelta(days=180))
+
++ make sure to use yfinance for financial data gathering (do not use pandas_datareader)
++ when plotting you need to 
+[x] plt.show()
+[o] plt.savefig('./tmp/plot.png')
+    ...
+    then
+    ![plot]('./tmp/plot.png')    
+
+
+    
+Let's think step-by-step
+"""

Llama2-Code-Interpreter/requirements.txt

@@ -0,0 +1,32 @@
+accelerate==0.21.0
+bitsandbytes==0.41.1
+colorama==0.4.6
+coloredlogs==15.0.1
+colorlog==6.7.0
+datasets==2.12.0
+deepspeed==0.10.1
+diffusers==0.20.0
+einops==0.6.1
+gradio==3.37.0
+ipykernel==6.25.1
+ipython==8.12.2
+jupyter_client==8.3.0
+jupyter_core==5.3.0
+Markdown==3.4.3
+nbclient==0.8.0
+nbconvert==7.7.1
+nbformat==5.8.0
+omegaconf==2.3.0
+openai==0.27.7
+peft @ git+https://github.com/huggingface/peft.git@6c44096c7b8d55a2ecf24be9bc68393467e1584a
+rich
+scikit-learn
+scipy
+seaborn
+sentencepiece==0.1.99
+termcolor==2.3.0
+tqdm
+transformers @ git+https://github.com/huggingface/transformers@f26099e7b5cf579f99a42bab6ddd371bf2c8d548
+triton==2.0.0
+yfinance==0.2.28
+retrying===1.3.4

Llama2-Code-Interpreter/utils/check_nb_out.py

@@ -0,0 +1,20 @@
+import nbformat
+from nbconvert.preprocessors import ExecutePreprocessor
+from nbconvert.preprocessors.execute import CellExecutionError
+
+nb = nbformat.v4.new_notebook()
+
+# Add a cell with your code
+code_cell = nbformat.v4.new_code_cell(source=f'import os\nprint(os.getcwd())')
+nb.cells.append(code_cell)
+
+# Execute the notebook
+ep = ExecutePreprocessor(timeout=600, kernel_name='python3')
+output_str, error_str = None, None
+
+ep.preprocess(nb)
+if nb.cells[0].outputs:  # Check if there are any outputs
+    output = nb.cells[-1].outputs[0]
+
+print(output)
+# Repo path :: /home/seungyoun/llama_code_interpreter\n

Llama2-Code-Interpreter/utils/check_nb_plot_img_out.py

@@ -0,0 +1,81 @@
+import nbformat
+from nbconvert.preprocessors import ExecutePreprocessor
+from nbconvert.preprocessors.execute import CellExecutionError
+import base64
+from io import BytesIO
+import re 
+
+def get_error_message(traceback_str):
+    lines = traceback_str.split('\n')
+    for line in lines:
+        if 'Error:' in line:
+            return line
+    return None  # Return None if no error message is found
+
+
+nb = nbformat.v4.new_notebook()
+
+SITE_PKG_ERROR_PREFIX = 'File /usr/local/lib/python3.8/'
+
+code_sample = """
+import yfinance as yf
+import matplotlib.pyplot as plt
+
+# Get the data of the Tesla USD stock price
+tsla = yf.Ticker("TSLA-USD")
+
+# Get the historical prices for the last 3 months
+tsla_hist = tsla.history(period="max", start="3 months ago")
+
+# Plot the close prices
+tsla_hist['Close'].plot(figsize=(16, 9))
+plt.title('Tesla stock price last 3 months')
+plt.xlabel('Date')
+plt.ylabel('Price (USD)')
+plt.show()
+"""
+
+# Add a cell with your code
+code_cell = nbformat.v4.new_code_cell(source=code_sample)
+nb.cells.append(code_cell)
+
+# Execute the notebook
+ep = ExecutePreprocessor(timeout=600, kernel_name='python3')
+output_str, error_str = None, None
+
+try:
+    ep.preprocess(nb)
+    if nb.cells[0].outputs:  # Check if there are any outputs
+        for i,c in enumerate(nb.cells[-1].outputs):
+            print(f'[{i+1}] : {c}')
+                 
+except CellExecutionError as e:
+    error_str = e
+
+    if error_str is not None:
+        # Get the traceback, which is a list of strings, and join them into one string
+        filtered_error_msg = error_str.__str__().split('An error occurred while executing the following cell')[-1].split("\n------------------\n")[-1]
+        raw_error_msg = "".join(filtered_error_msg)
+            
+        # Remove escape sequences for colored text
+        #print(raw_error_msg)
+        ansi_escape = re.compile(r'\x1b\[[0-?]*[ -/]*[@-~]')
+        error_msg = ansi_escape.sub('', raw_error_msg)
+
+        error_msg_only_cell = error_msg.split(SITE_PKG_ERROR_PREFIX)
+        for i,c in enumerate(error_msg_only_cell):
+            if i ==0:
+                print(f'[{i+1}]\n{c.strip()}\n---')
+            if i==3:
+                error_header = get_error_message(c)
+                print(error_header)
+
+
+        #error_msg = raw_error_msg.replace("\x1b[0m", "").replace("\x1b[0;31m", "").replace("\x1b[0;32m", "").replace("\x1b[1;32m", "").replace("\x1b[38;5;241m", "").replace("\x1b[38;5;28;01m", "").replace("\x1b[38;5;21m", "").replace("\x1b[38;5;28m", "").replace("\x1b[43m", "").replace("\x1b[49m", "").replace("\x1b[38;5;241;43m", "").replace("\x1b[39;49m", "").replace("\x1b[0;36m", "").replace("\x1b[0;39m", "")
+        error_lines = error_msg.split("\n")
+        
+        # Only keep the lines up to (and including) the first line that contains 'Error' followed by a ':'
+        error_lines = error_lines[:next(i for i, line in enumerate(error_lines) if 'Error:' in line) + 1]
+
+        # Join the lines back into a single string
+        error_msg = "\n".join(error_lines)

Llama2-Code-Interpreter/utils/cleaner.py

@@ -0,0 +1,28 @@
+import re 
+
+SITE_PKG_ERROR_PREFIX = 'File /usr/local/lib/python3.8/'
+
+def get_error_header(traceback_str):
+    lines = traceback_str.split('\n')
+    for line in lines:
+        if 'Error:' in line:
+            return line
+    return ''  # Return None if no error message is found
+
+def clean_error_msg(error_str:str =''):
+    filtered_error_msg = error_str.__str__().split('An error occurred while executing the following cell')[-1].split("\n------------------\n")[-1]
+    raw_error_msg = "".join(filtered_error_msg)
+
+    # Remove escape sequences for colored text
+    ansi_escape = re.compile(r'\x1b\[[0-?]*[ -/]*[@-~]')
+    error_msg = ansi_escape.sub('', raw_error_msg)
+    
+    error_str_out = ''
+    error_msg_only_cell = error_msg.split(SITE_PKG_ERROR_PREFIX)
+
+    error_str_out += f'{error_msg_only_cell[0]}\n'
+    error_header = get_error_header(error_msg_only_cell[-1])
+    if error_header not in error_str_out:
+        error_str_out += get_error_header(error_msg_only_cell[-1])
+
+    return error_str_out

Llama2-Code-Interpreter/utils/const.py

@@ -0,0 +1,314 @@
+from typing import List, Literal, Optional, Tuple, TypedDict
+
+
+Role = Literal["system", "user", "assistant"]
+
+
+class Message(TypedDict):
+    role: Role
+    content: str
+
+
+Dialog = List[Message]
+
+model_path = "./ckpt/llama-2-13b-chat"
+
+B_INST, E_INST = "[INST]", "[/INST]"
+B_SYS, E_SYS = "<<SYS>>\n", "\n<</SYS>>\n\n"
+DEFAULT_PAD_TOKEN = "[PAD]"
+DEFAULT_EOS_TOKEN = "</s>"
+DEFAULT_BOS_TOKEN = "<s>"
+DEFAULT_UNK_TOKEN = "<unk>"
+
+IMPORT_PKG = """
+import numpy as np
+import pandas as pd 
+import matplotlib.pyplot as plt
+import seaborn as sns
+from scipy import stats
+import os,sys
+import re
+from datetime import datetime
+from sympy import symbols, Eq, solve
+import torch 
+import requests
+from bs4 import BeautifulSoup
+import json
+import math
+import yfinance
+"""
+
+TOOLS_CODE = r"""
+
+import requests
+import tweepy
+import json,time
+from urllib.parse import quote_plus
+from typing import Dict
+from bs4 import BeautifulSoup
+
+
+#Goolge Search 
+GOOGLE_API_KEY = "<YOUR>"
+GOOGLE_CSE_ID  = '<YOUR>'
+MAX_GOOGLE_RESULT = 5
+
+#Twitter Key
+TWITTER_API_KEY = "<YOUR>"
+TWITTER_API_KEY_SECRET = "<YOUR>"
+TWITTER_ACCESS_TOKEN = "<YOUR>"
+TWITTER_TOKEN_SECRET = "<YOUR>"
+ 
+class GoogleSearch:
+    def __init__(self):
+        self.api_key = GOOGLE_API_KEY
+        self.cse_id = GOOGLE_CSE_ID
+        self.url = "https://www.googleapis.com/customsearch/v1"
+    
+    def search(self, search_term, **kwargs):
+        params = {
+            'q': search_term,
+            'key': self.api_key,
+            'cx': self.cse_id,
+        }
+        params.update(kwargs)
+        response = requests.get(self.url, params=params)
+        return response.json()
+
+    def __call__(self, search_term, **kwargs):
+        results = self.search(search_term, **kwargs)
+        output_str = ''
+        for idx, item in enumerate(results.get('items', [])):
+            if idx > MAX_GOOGLE_RESULT:
+                break
+            title = item.get('title').replace('\n','')
+            snippet = item.get('snippet').replace('\n','')
+            link = item.get('link').replace('\n','')
+            output_str += f"[{idx+1}] Title : [{title}]\n\tsnippet : {snippet}\n\tlink : {link}\n"
+        return output_str
+
+
+class ArxivAPI:
+    def __init__(self):
+        self.base_url = 'http://export.arxiv.org/api/query?'
+        self.headers = {'User-Agent': 'Mozilla/5.0'}
+    
+    def clean_str(self, results):
+        output_str = ''
+        for idx, result in enumerate(results):
+            output_str += f"[{idx+1}]title : {result['title'].strip()}({result['id'].strip()})\n"
+        return output_str
+    
+    def search(self, query: str, max_results: int = 10):
+        query = quote_plus(query)
+        search_query = f'search_query=all:{query}&start=0&max_results={max_results}'
+        url = self.base_url + search_query
+        response = requests.get(url, headers=self.headers)
+        if response.status_code != 200:
+            raise Exception(f'Error: {response.status_code}')
+        soup = BeautifulSoup(response.content, 'xml')
+        entries = soup.find_all('entry')
+        results = [{'id': entry.id.text, 'title': entry.title.text} for entry in entries]
+        return self.clean_str(results)
+
+        
+# google
+google_search = GoogleSearch()
+
+# Arxiv
+arxiv = ArxivAPI()
+"""
+
+
+CODE_INTERPRETER_SYSTEM_PROMPT = """
+You are helpful agent that can generate code, execute, getting feedback(results).
+
+
+For example,
+###User : what is 52th fibonacci number
+###Assistant : 
+I think I need to use python code to generate 52th fibonacci number
+
+```python
+import numpy as np
+
+def fibonacci(n):
+    sq = np.sqrt(5)
+    phi = (1 + sq) / 2
+    return round((phi ** n) / sq)
+
+fibo_52 = fibonacci(52)
+fibo_52
+```
+```RESULT
+32951280099
+```
+The 52nd Fibonacci number is 32951280099.
+
++ Each code block is executed individually in each cell of a Jupyter notebook.
++ If the generated code runs successfully, there's no need to generate redundant code.
++ The code is written to provide answers to the questions.
+
+"""
+
+CODE_INTERPRETER_SYSTEM_PROMPT = """
+You are helpful agent that can generate code, execute, getting feedback(results).
+
+For example,
+###User : Show me YTD bitcoin price.
+###Assistant : 
+
+Here's a sample Python code using pandas, matplotlib, and yfinance to get the Year-to-date (YTD) Bitcoin price and plot it:
+```python
+import yfinance as yf
+import matplotlib.pyplot as plt
+
+# Get the data of the Bitcoin USD price
+btc = yf.Ticker("BTC-USD")
+
+# Get the historical prices for this year
+btc_hist = btc.history(period="ytd")
+
+# Plot the close prices
+btc_hist['Close'].plot(figsize=(16, 9))
+plt.title('Year-to-date Bitcoin price')
+plt.xlabel('Date')
+plt.ylabel('Price (USD)')
+plt.savefig('./tmp/chart.png')
+```
+```RESULT
+<Figure size 1600x900 with 1 Axes>
+```
+
+Here is the chart of the bitcoin close YTD chart : ![chart]('./tmp/chart.png')</s>
+
++ Each code block is executed individually in each cell of a Jupyter notebook.
++ If the generated code runs successfully, there's no need to generate redundant code.
++ The code is written to provide answers to the questions.
+
+"""
+
+CODE_INTERPRETER_SYSTEM_PROMPT = """
+As an advanced language model, you can generate code as part of your responses. 
+To make the code more noticeable and easier to read, please encapsulate it within triple backticks.
+For instance, if you're providing Python code, wrap it as follows:
+
+```python
+print('hellow world')
+```
+
+Basically this two tools are provided.
+
+```python
+# google
+google_search = GoogleSearch()
+results = google_search("Current korean president") #query -> string output
+print(results) # string 
+
+# Arxiv
+arxiv = ArxivAPI()
+results = arxiv.search('embodied ai') #query -> string
+print(results) # string
+```
+
+After presenting the results from the code
+You will provide a useful explanation or interpretation of the output to further aid your understanding."
+
+Additionally, when generating plots or figures, 
+I'll save them to a specified path, like ./tmp/plot.png, so that they can be viewed. 
+After saving the plot, I'll use the following markdown syntax to display the image at the end of the response:
+![plot]('./tmp/plot.png')
+
+You are using jupyter notebook currently.
+This approach allows me to visually present data and findings."
+"""
+
+CODE_INTERPRETER_SYSTEM_PROMPT_PRESIDENT = """
+You are helpful agent that can code 
+
+You are avaible to use
+numpy, beautifulsoup, torch, PIL, opencv, ...
+
+For example,
+###User : Who is current president of singapore?
+###Assistant : 
+
+Here's a sample Python code using pandas, matplotlib, and yfinance to get the Year-to-date (YTD) Bitcoin price and plot it:
+```python
+import requests
+from bs4 import BeautifulSoup
+
+def get_current_south_korea_president():
+    url = 'https://www.president.go.kr/president/greeting'
+    response = requests.get(url)
+    soup = BeautifulSoup(response.text, 'html.parser')
+    # Find the president's name
+    president_name = soup.find('title').text.strip()
+    return president_name
+
+get_current_south_korea_president()
+```
+```RESULT
+대한민국 대통령 > 윤석열 대통령 > 취임사
+```
+
+The current President of Korea is 윤석열
+"""
+
+
+CODE_INTERPRETER_SYSTEM_PROMPT_GPT4 = f"""
+You are helpful agent that can code 
+
+You are avaible to use
+{IMPORT_PKG}
+"""
+
+
+CODE_INTERPRETER_SYSTEM_PROMPT_GPT4 = f"""
+You are helpful agent that can code.
+
+### User : Can you show me the distribution of the current 
+"""
+
+
+CODE_INTERPRETER_SYSTEM_PROMPT_GPT4_BASE = f"""
+You are helpful agent that can code.
+
+For example,
+
+### User : Show me YTD bitcoin pirce.
+
+### Assistant : Sure thing! Here's a Python script using yfinance to get the YTD Bitcoin price and save it into a CSV file using pandas. 
+It also plots the price using matplotlib. Please note we are saving the plot in the 
+./tmp/ directory as 'bitcoin_YTD.png' and data as 'bitcoin_YTD.csv'.
+
+```python
+import yfinance as yf
+import matplotlib.pyplot as plt
+import pandas as pd\nfrom datetime import datetime
+
+# Get the current year
+year = datetime.now().year
+# Get the data of Bitcoin from the beginning of this year until now
+btc = yf.download('BTC-USD', start=str(year)+'-01-01', end=datetime.now().strftime(\"%Y-%m-%d\"))
+
+# Save the data to a .csv file
+btc.to_csv('./tmp/bitcoin_YTD.csv')
+
+# Create a plot
+plt.figure(figsize=(14, 7))
+plt.plot(btc['Close'])
+plt.title('Bitcoin Price YTD')
+plt.xlabel('Date')
+plt.ylabel('Price'
+nplt.grid(True)
+plt.savefig('./tmp/bitcoin_YTD.png')
+```
+
+```RESULTS
+[*********************100%***********************]  1 of 1 completed
+<Figure size 1400x700 with 1 Axes>
+```
+
+Here is plot : ![BTC-YTD]('./tmp/bitcoin_YTD.png')
+"""

Llama2-Code-Interpreter/utils/convert_llama_weights_to_hf.py

@@ -0,0 +1,375 @@
+# Copyright 2022 EleutherAI and The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import argparse
+import gc
+import json
+import os
+import shutil
+import warnings
+
+import torch
+
+from transformers import LlamaConfig, LlamaForCausalLM, LlamaTokenizer
+
+
+try:
+    from transformers import LlamaTokenizerFast
+except ImportError as e:
+    warnings.warn(e)
+    warnings.warn(
+        "The converted tokenizer will be the `slow` tokenizer. To use the fast, update your `tokenizers` library and re-run the tokenizer conversion"
+    )
+    LlamaTokenizerFast = None
+
+"""
+Sample usage:
+
+```
+python src/transformers/models/llama/convert_llama_weights_to_hf.py \
+    --input_dir /path/to/downloaded/llama/weights --model_size 7B --output_dir /output/path
+```
+
+Thereafter, models can be loaded via:
+
+```py
+from transformers import LlamaForCausalLM, LlamaTokenizer
+
+model = LlamaForCausalLM.from_pretrained("/output/path")
+tokenizer = LlamaTokenizer.from_pretrained("/output/path")
+```
+
+Important note: you need to be able to host the whole model in RAM to execute this script (even if the biggest versions
+come in several checkpoints they each contain a part of each weight of the model, so we need to load them all in RAM).
+"""
+
+INTERMEDIATE_SIZE_MAP = {
+    "7B": 11008,
+    "13B": 13824,
+    "30B": 17920,
+    "65B": 22016,
+    "70B": 28672,
+}
+NUM_SHARDS = {
+    "7B": 1,
+    "7Bf": 1,
+    "13B": 2,
+    "13Bf": 2,
+    "30B": 4,
+    "65B": 8,
+    "70B": 8,
+    "70Bf": 8,
+}
+
+
+def compute_intermediate_size(n, ffn_dim_multiplier=1, multiple_of=256):
+    return multiple_of * (
+        (int(ffn_dim_multiplier * int(8 * n / 3)) + multiple_of - 1) // multiple_of
+    )
+
+
+def read_json(path):
+    with open(path, "r") as f:
+        return json.load(f)
+
+
+def write_json(text, path):
+    with open(path, "w") as f:
+        json.dump(text, f)
+
+
+def write_model(model_path, input_base_path, model_size, safe_serialization=True):
+    os.makedirs(model_path, exist_ok=True)
+    tmp_model_path = os.path.join(model_path, "tmp")
+    os.makedirs(tmp_model_path, exist_ok=True)
+
+    input_base_path = "/home/seungyoun/llama/ckpt/llama-2-7b"
+    params = read_json(os.path.join(input_base_path, "params.json"))
+    num_shards = NUM_SHARDS[model_size]
+    n_layers = params["n_layers"]
+    n_heads = params["n_heads"]
+    n_heads_per_shard = n_heads // num_shards
+    dim = params["dim"]
+    dims_per_head = dim // n_heads
+    base = 10000.0
+    inv_freq = 1.0 / (
+        base ** (torch.arange(0, dims_per_head, 2).float() / dims_per_head)
+    )
+
+    if "n_kv_heads" in params:
+        num_key_value_heads = params["n_kv_heads"]  # for GQA / MQA
+        num_local_key_value_heads = n_heads_per_shard // num_key_value_heads
+        key_value_dim = dim // num_key_value_heads
+    else:  # compatibility with other checkpoints
+        num_key_value_heads = n_heads
+        num_local_key_value_heads = n_heads_per_shard
+        key_value_dim = dim
+
+    # permute for sliced rotary
+    def permute(w, n_heads=n_heads, dim1=dim, dim2=dim):
+        return (
+            w.view(n_heads, dim1 // n_heads // 2, 2, dim2)
+            .transpose(1, 2)
+            .reshape(dim1, dim2)
+        )
+
+    print(f"Fetching all parameters from the checkpoint at {input_base_path}.")
+    # Load weights
+    if model_size == "7B":
+        # Not sharded
+        # (The sharded implementation would also work, but this is simpler.)
+        loaded = torch.load(
+            os.path.join(input_base_path, "consolidated.00.pth"), map_location="cpu"
+        )
+    else:
+        # Sharded
+        loaded = [
+            torch.load(
+                os.path.join(input_base_path, f"consolidated.{i:02d}.pth"),
+                map_location="cpu",
+            )
+            for i in range(num_shards)
+        ]
+    param_count = 0
+    index_dict = {"weight_map": {}}
+    for layer_i in range(n_layers):
+        filename = f"pytorch_model-{layer_i + 1}-of-{n_layers + 1}.bin"
+        if model_size == "7B":
+            # Unsharded
+            state_dict = {
+                f"model.layers.{layer_i}.self_attn.q_proj.weight": permute(
+                    loaded[f"layers.{layer_i}.attention.wq.weight"]
+                ),
+                f"model.layers.{layer_i}.self_attn.k_proj.weight": permute(
+                    loaded[f"layers.{layer_i}.attention.wk.weight"]
+                ),
+                f"model.layers.{layer_i}.self_attn.v_proj.weight": loaded[
+                    f"layers.{layer_i}.attention.wv.weight"
+                ],
+                f"model.layers.{layer_i}.self_attn.o_proj.weight": loaded[
+                    f"layers.{layer_i}.attention.wo.weight"
+                ],
+                f"model.layers.{layer_i}.mlp.gate_proj.weight": loaded[
+                    f"layers.{layer_i}.feed_forward.w1.weight"
+                ],
+                f"model.layers.{layer_i}.mlp.down_proj.weight": loaded[
+                    f"layers.{layer_i}.feed_forward.w2.weight"
+                ],
+                f"model.layers.{layer_i}.mlp.up_proj.weight": loaded[
+                    f"layers.{layer_i}.feed_forward.w3.weight"
+                ],
+                f"model.layers.{layer_i}.input_layernorm.weight": loaded[
+                    f"layers.{layer_i}.attention_norm.weight"
+                ],
+                f"model.layers.{layer_i}.post_attention_layernorm.weight": loaded[
+                    f"layers.{layer_i}.ffn_norm.weight"
+                ],
+            }
+        else:
+            # Sharded
+            # Note that attention.w{q,k,v,o}, feed_fordward.w[1,2,3], attention_norm.weight and ffn_norm.weight share
+            # the same storage object, saving attention_norm and ffn_norm will save other weights too, which is
+            # redundant as other weights will be stitched from multiple shards. To avoid that, they are cloned.
+
+            state_dict = {
+                f"model.layers.{layer_i}.input_layernorm.weight": loaded[0][
+                    f"layers.{layer_i}.attention_norm.weight"
+                ].clone(),
+                f"model.layers.{layer_i}.post_attention_layernorm.weight": loaded[0][
+                    f"layers.{layer_i}.ffn_norm.weight"
+                ].clone(),
+            }
+            state_dict[f"model.layers.{layer_i}.self_attn.q_proj.weight"] = permute(
+                torch.cat(
+                    [
+                        loaded[i][f"layers.{layer_i}.attention.wq.weight"].view(
+                            n_heads_per_shard, dims_per_head, dim
+                        )
+                        for i in range(num_shards)
+                    ],
+                    dim=0,
+                ).reshape(dim, dim)
+            )
+            state_dict[f"model.layers.{layer_i}.self_attn.k_proj.weight"] = permute(
+                torch.cat(
+                    [
+                        loaded[i][f"layers.{layer_i}.attention.wk.weight"].view(
+                            num_local_key_value_heads, dims_per_head, dim
+                        )
+                        for i in range(num_shards)
+                    ],
+                    dim=0,
+                ).reshape(key_value_dim, dim),
+                num_key_value_heads,
+                key_value_dim,
+                dim,
+            )
+            state_dict[f"model.layers.{layer_i}.self_attn.v_proj.weight"] = torch.cat(
+                [
+                    loaded[i][f"layers.{layer_i}.attention.wv.weight"].view(
+                        num_local_key_value_heads, dims_per_head, dim
+                    )
+                    for i in range(num_shards)
+                ],
+                dim=0,
+            ).reshape(key_value_dim, dim)
+
+            state_dict[f"model.layers.{layer_i}.self_attn.o_proj.weight"] = torch.cat(
+                [
+                    loaded[i][f"layers.{layer_i}.attention.wo.weight"]
+                    for i in range(num_shards)
+                ],
+                dim=1,
+            )
+            state_dict[f"model.layers.{layer_i}.mlp.gate_proj.weight"] = torch.cat(
+                [
+                    loaded[i][f"layers.{layer_i}.feed_forward.w1.weight"]
+                    for i in range(num_shards)
+                ],
+                dim=0,
+            )
+            state_dict[f"model.layers.{layer_i}.mlp.down_proj.weight"] = torch.cat(
+                [
+                    loaded[i][f"layers.{layer_i}.feed_forward.w2.weight"]
+                    for i in range(num_shards)
+                ],
+                dim=1,
+            )
+            state_dict[f"model.layers.{layer_i}.mlp.up_proj.weight"] = torch.cat(
+                [
+                    loaded[i][f"layers.{layer_i}.feed_forward.w3.weight"]
+                    for i in range(num_shards)
+                ],
+                dim=0,
+            )
+
+        state_dict[f"model.layers.{layer_i}.self_attn.rotary_emb.inv_freq"] = inv_freq
+        for k, v in state_dict.items():
+            index_dict["weight_map"][k] = filename
+            param_count += v.numel()
+        torch.save(state_dict, os.path.join(tmp_model_path, filename))
+
+    filename = f"pytorch_model-{n_layers + 1}-of-{n_layers + 1}.bin"
+    if model_size == "7B":
+        # Unsharded
+        state_dict = {
+            "model.embed_tokens.weight": loaded["tok_embeddings.weight"],
+            "model.norm.weight": loaded["norm.weight"],
+            "lm_head.weight": loaded["output.weight"],
+        }
+    else:
+        state_dict = {
+            "model.norm.weight": loaded[0]["norm.weight"],
+            "model.embed_tokens.weight": torch.cat(
+                [loaded[i]["tok_embeddings.weight"] for i in range(num_shards)], dim=1
+            ),
+            "lm_head.weight": torch.cat(
+                [loaded[i]["output.weight"] for i in range(num_shards)], dim=0
+            ),
+        }
+
+    for k, v in state_dict.items():
+        index_dict["weight_map"][k] = filename
+        param_count += v.numel()
+    torch.save(state_dict, os.path.join(tmp_model_path, filename))
+
+    # Write configs
+    index_dict["metadata"] = {"total_size": param_count * 2}
+    write_json(index_dict, os.path.join(tmp_model_path, "pytorch_model.bin.index.json"))
+    ffn_dim_multiplier = (
+        params["ffn_dim_multiplier"] if "ffn_dim_multiplier" in params else 1
+    )
+    multiple_of = params["multiple_of"] if "multiple_of" in params else 256
+    config = LlamaConfig(
+        hidden_size=dim,
+        intermediate_size=compute_intermediate_size(
+            dim, ffn_dim_multiplier, multiple_of
+        ),
+        num_attention_heads=params["n_heads"],
+        num_hidden_layers=params["n_layers"],
+        rms_norm_eps=params["norm_eps"],
+        num_key_value_heads=num_key_value_heads,
+    )
+    config.save_pretrained(tmp_model_path)
+
+    # Make space so we can load the model properly now.
+    del state_dict
+    del loaded
+    gc.collect()
+
+    print("Loading the checkpoint in a Llama model.")
+    model = LlamaForCausalLM.from_pretrained(
+        tmp_model_path, torch_dtype=torch.float16, low_cpu_mem_usage=True
+    )
+    # Avoid saving this as part of the config.
+    del model.config._name_or_path
+
+    print("Saving in the Transformers format.")
+    model.save_pretrained(model_path, safe_serialization=safe_serialization)
+    shutil.rmtree(tmp_model_path)
+
+
+def write_tokenizer(tokenizer_path, input_tokenizer_path):
+    # Initialize the tokenizer based on the `spm` model
+    tokenizer_class = (
+        LlamaTokenizer if LlamaTokenizerFast is None else LlamaTokenizerFast
+    )
+    print(f"Saving a {tokenizer_class.__name__} to {tokenizer_path}.")
+    tokenizer = tokenizer_class(input_tokenizer_path)
+    tokenizer.save_pretrained(tokenizer_path)
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--input_dir",
+        help="Location of LLaMA weights, which contains tokenizer.model and model folders",
+    )
+    parser.add_argument(
+        "--model_size",
+        choices=[
+            "7B",
+            "7Bf",
+            "13B",
+            "13Bf",
+            "30B",
+            "65B",
+            "70B",
+            "70Bf",
+            "tokenizer_only",
+        ],
+    )
+    parser.add_argument(
+        "--output_dir",
+        help="Location to write HF model and tokenizer",
+    )
+    parser.add_argument(
+        "--safe_serialization",
+        type=bool,
+        help="Whether or not to save using `safetensors`.",
+    )
+    args = parser.parse_args()
+    if args.model_size != "tokenizer_only":
+        write_model(
+            model_path=args.output_dir,
+            input_base_path=os.path.join(args.input_dir, args.model_size),
+            model_size=args.model_size,
+            safe_serialization=args.safe_serialization,
+        )
+    spm_path = os.path.join(args.input_dir, "tokenizer.model")
+    spm_path = "/home/seungyoun/llama/ckpt/tokenizer.model"
+    write_tokenizer(args.output_dir, spm_path)
+
+
+if __name__ == "__main__":
+    main()

Llama2-Code-Interpreter/utils/special_tok_llama2.py

@@ -0,0 +1,14 @@
+B_CODE = "[CODE_START_TOK]"
+E_CODE = "[/CODE_END_TOK]"
+
+B_RESULT = "[RESULT_TOK]"
+E_RESULT = "[/RESULT_TOK]"
+
+B_INST, E_INST = "[INST]", "[/INST]"
+B_SYS, E_SYS = "<<SYS>>", "<</SYS>>"
+
+IGNORE_INDEX = -100
+DEFAULT_PAD_TOKEN = "[PAD]"
+DEFAULT_EOS_TOKEN = "</s>"
+DEFAULT_BOS_TOKEN = "<s>"
+DEFAULT_UNK_TOKEN = "<unk>"

OpenCodeInterpreter/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

OpenCodeInterpreter/README.md

@@ -0,0 +1,83 @@
+# OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
+
+<p align="center">
+<img width="1000px" alt="OpenCodeInterpreter" src="https://opencodeinterpreter.github.io/static/images/figure1.png">
+</p>
+<p align="center">
+  <a href="https://opencodeinterpreter.github.io/">[🏠Homepage]</a> 
+  |
+  <a href="https://github.com/OpenCodeInterpreter/OpenCodeInterpreter/">[🛠️Code]</a> 
+</p>
+<hr>
+
+## 🌟 Upcoming Features
+- 💡 **Open Sourcing OpenCodeInterpreter-SC2 series Model (based on StarCoder2 base)** 
+
+- 💡 **Open Sourcing OpenCodeInterpreter-GM-7b Model with gemma-7b Base** 
+
+## 🔔News
+🛠️[2024-02-29]: Our official online demo is deployed on HuggingFace Spaces! Take a look at [Demo Page](https://huggingface.co/spaces/m-a-p/OpenCodeInterpreter_demo)!
+
+🛠️[2024-02-28]: We have open-sourced the Demo Local Deployment Code with a Setup Guide.
+
+✨[2024-02-26]: We have open-sourced the [OpenCodeInterpreter-DS-1.3b](https://huggingface.co/m-a-p/OpenCodeInterpreter-DS-1.3B) Model.
+
+📘[2024-02-26]: We have open-sourced the [CodeFeedback-Filtered-Instruction](https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction) Dataset.
+
+🚀[2024-02-23]: We have open-sourced the datasets used in our project named [Code-Feedback](https://huggingface.co/datasets/m-a-p/Code-Feedback).
+
+🔥[2024-02-19]: We have open-sourced all models in the OpenCodeInterpreter series! We welcome everyone to try out our models and look forward to your participation! 😆
+
+
+
+## Introduction
+OpenCodeInterpreter is a suite of open-source code generation systems aimed at bridging the gap between large language models and sophisticated proprietary systems like the GPT-4 Code Interpreter. It significantly enhances code generation capabilities by integrating execution and iterative refinement functionalities.
+
+## Models
+All models within the OpenCodeInterpreter series have been open-sourced on Hugging Face. You can access our models via the following link: [OpenCodeInterpreter Models](https://huggingface.co/collections/m-a-p/opencodeinterpreter-65d312f6f88da990a64da456).
+
+## Data Collection
+Supported by Code-Feedback, a dataset featuring 68K multi-turn interactions, OpenCodeInterpreter incorporates execution and human feedback for dynamic code refinement. 
+For additional insights into data collection procedures, please consult the readme provided under [Data Collection](https://github.com/OpenCodeInterpreter/OpenCodeInterpreter/blob/main/data_collection/README.md).
+
+## Evaluation
+Our evaluation framework primarily utilizes HumanEval and MBPP, alongside their extended versions, HumanEval+ and MBPP+, leveraging the [EvalPlus framework](https://github.com/evalplus/evalplus) for a more comprehensive assessment.
+For specific evaluation methodologies, please refer to the [Evaluation README](https://github.com/OpenCodeInterpreter/OpenCodeInterpreter/blob/main/evaluation/README.md) for more details.
+
+## Demo
+We're excited to present our open-source demo, enabling users to effortlessly generate and execute code with our LLM locally. Within the demo, users can leverage the power of LLM to generate code and execute it locally, receiving automated execution feedback. LLM dynamically adjusts the code based on this feedback, ensuring a smoother coding experience. Additionally, users can engage in chat-based interactions with the LLM model, providing feedback to further enhance the generated code.
+
+To begin exploring the demo and experiencing the capabilities firsthand, please refer to the instructions outlined in the [OpenCodeInterpreter Demo README](https://github.com/OpenCodeInterpreter/OpenCodeInterpreter/blob/main/demo/README.md) file. Happy coding!
+
+### Quick Start
+- **Entering the workspace**:
+   ```bash
+   git clone https://github.com/OpenCodeInterpreter/OpenCodeInterpreter.git
+   cd demo
+   ```
+- **Create a new conda environment**: `conda create -n demo python=3.10`
+
+- **Activate the demo environment you create**: `conda activate demo`
+
+- **Install requirements**: `pip install -r requirements.txt`
+
+- **Create a Huggingface access token with write permission [here](https://huggingface.co/docs/hub/en/security-tokens). Our code will only use this token to create and push content to a specific repository called `opencodeinterpreter_user_data` under your own Huggingface account. We cannot get access to your data if you deploy this demo on your own device.**
+
+- **Add the access token to environment variables:** `export HF_TOKEN="your huggingface access token"`
+
+- **Run the Gradio App**:
+   ```bash
+   python3 chatbot.py --path "the model name of opencodeinterpreter model family. e.g., m-a-p/OpenCodeInterpreter-DS-6.7B"
+   ```
+### Video
+https://github.com/OpenCodeInterpreter/OpenCodeInterpreter/assets/46103100/2337f34d-f5ed-4ecb-857b-3c2d085b72fd
+
+
+## Contact
+
+If you have any inquiries, please feel free to raise an issue or reach out to us via email at: xiangyue.work@gmail.com, zhengtianyu0428@gmail.com. 
+We're here to assist you!
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=OpenCodeInterpreter/OpenCodeInterpreter&type=Date)](https://star-history.com/#OpenCodeInterpreter/OpenCodeInterpreter&Date)

OpenCodeInterpreter/data_collection/Local-Code-Interpreter/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

OpenCodeInterpreter/data_collection/Local-Code-Interpreter/README.md

@@ -0,0 +1,143 @@
+**Read in other language: [中文](README_CN.md).**
+
+# Local-Code-Interpreter
+A local implementation of OpenAI's ChatGPT Code Interpreter (Advanced Data Analysis).
+
+## Introduction
+
+OpenAI's Code Interpreter (currently renamed as Advanced Data Analysis) for ChatGPT is a revolutionary feature that allows the execution of Python code within the AI model. However, it execute code within an online sandbox and has certain limitations. In this project, we present Local Code Interpreter – which enables code execution on your local device, offering enhanced flexibility, security, and convenience.
+![notebook_gif_demo](example_img/save_to_notebook_demo.gif)
+
+## Key Advantages
+
+- **Custom Environment**: Execute code in a customized environment of your choice, ensuring you have the right packages and settings.
+
+- **Seamless Experience**: Say goodbye to file size restrictions and internet issues while uploading. With Local Code Interpreter, you're in full control.
+
+- **GPT-3.5 Availability**: While official Code Interpreter is only available for GPT-4 model, the Local Code Interpreter offers the flexibility to switch between both GPT-3.5 and GPT-4 models.
+
+- **Enhanced Data Security**: Keep your data more secure by running code locally, minimizing data transfer over the internet.
+
+- **Jupyter Support**: You can save all the code and conversation history in a Jupyter notebook for future use.
+
+## Note
+Executing AI-generated code without human review on your own device is not safe. You are responsible for taking measures to protect the security of your device and data (such as using a virtural machine) before launching this program. All consequences caused by using this program shall be borne by youself.
+
+## Usage
+
+### Installation
+
+1. Clone this repository to your local device
+   ```shell
+   git clone https://github.com/MrGreyfun/Local-Code-Interpreter.git
+   cd Local-Code-Interpreter
+   ```
+
+2. Install the necessary dependencies. The program has been tested on Windows 10 and CentOS Linux 7.8, with Python 3.9.16. Required packages include:
+   ```text
+   Jupyter Notebook    6.5.4
+   gradio              3.39.0
+   openai              0.27.8
+   ansi2html           1.8.0
+   tiktoken            0.3.3
+   Pillow              9.4.0
+   ```
+   Other systems or package versions may also work. Please note that you should not update the `openai` package to the latest `1.x` version, as it has been rewritten and is not compatible with older versions.
+   You can use the following command to directly install the required packages:
+   ```shell
+   pip install -r requirements.txt
+   ```
+   For newcomers to Python, we offer a convenient command that installs additional packages commonly used for data processing and analysis:
+   ```shell
+   pip install -r requirements_full.txt
+   ```
+### Configuration
+
+1. Create a `config.json` file in the `src` directory, following the examples provided in the `config_example` directory.
+
+2. Configure your API key in the `config.json` file.
+
+Please Note:
+1. **Set the `model_name` Correctly**
+    This program relies on the function calling capability of the `0613` or newer versions of models:
+    - `gpt-3.5-turbo-0613` (and its 16K version)
+    - `gpt-3.5-turbo-1106`
+    - `gpt-4-0613` (and its 32K version)
+    - `gpt-4-1106-preview` 
+
+    Older versions of the models will not work. Note that `gpt-4-vision-preview` lacks support for function calling, therefore, it should not be set as `GPT-4` model. 
+
+    For Azure OpenAI service users:
+    - Set the `model_name` as your deployment name.
+    - Confirm that the deployed model corresponds to the `0613` or newer version.
+
+2. **API Version Settings**
+    If you're using Azure OpenAI service, set the `API_VERSION` to `2023-12-01-preview` in the `config.json` file. Note that API versions older than `2023-07-01-preview` do not support the necessary function calls for this program and `2023-12-01-preview` is recommended as older versions will be deprecated in the near future.
+
+3. **Vision Model Settings**
+    Despite the `gpt-4-vision-preview` currently does not support function calling, we have implemented vision input using a non-end-to-end approach. To enable vision input, set `gpt-4-vision-preview` as `GPT-4V` model and set `available` to `true`.  Conversely, setting `available` to `false` to disables vision input when unnecessary, which will remove vision-related system prompts and reduce your API costs.
+    ![vision_demo](example_img/vision_example.jpg)
+4. **Model Context Window Settings**
+    The `model_context_window` field records the context window for each model, which the program uses to slice conversations when they exceed the model's context window capacity. 
+    Azure OpenAI service users should manually insert context window information using the model's deployment name in the following format:
+    ```json
+    "<YOUR-DEPLOYMENT-NAME>": <contex_window (integer)>
+    ```
+   
+    Additionally, when OpenAI introduce new models, you can manually append the new model's context window information using the same format. (We will keep this file updated, but there might be delays)
+
+5. **Alternate API Key Handling**
+    If you prefer not to store your API key in the `config.json` file, you can opt for an alternate approach:
+    - Leave the `API_KEY` field in `config.json` as an empty string:
+        ```json
+        "API_KEY": ""
+        ```
+    - Set the environment variable `OPENAI_API_KEY` with your API key before running the program:
+        - On Windows:
+        ```shell
+        set OPENAI_API_KEY=<YOUR-API-KEY>
+        ```
+        - On Linux:
+        ```shell
+        export OPENAI_API_KEY=<YOUR-API-KEY>
+        ```
+
+## Getting Started
+
+1. Navigate to the `src` directory.
+   ```shell
+   cd src
+   ```
+
+2. Run the command:
+   ```shell
+   python web_ui.py
+   ```
+
+3. Access the generated link in your browser to start using the Local Code Interpreter.
+
+4. Use the `-n` or `--notebook` option to save the conversation in a Jupyter notebook.
+   By default, the notebook is saved in the working directory, but you can add a path to save it elsewhere.
+   ```shell
+   python web_ui.py -n <path_to_notebook>
+   ```
+
+## Example
+
+Imagine uploading a data file and requesting the model to perform linear regression and visualize the data. See how Local Code Interpreter provides a seamless experience:
+
+1. Upload the data and request linear regression:
+   ![Example 1](example_img/1.jpg)
+
+2. Encounter an error in the generated code:
+   ![Example 2](example_img/2.jpg)
+
+3. ChatGPT automatically checks the data structure and fixes the bug:
+   ![Example 3](example_img/3.jpg)
+
+4. The corrected code runs successfully:
+   ![Example 4](example_img/4.jpg)
+
+5. The final result meets your requirements:
+   ![Example 5](example_img/5.jpg)
+   ![Example 6](example_img/6.jpg)

OpenCodeInterpreter/data_collection/Local-Code-Interpreter/README_CN.md

@@ -0,0 +1,140 @@
+**Read in other language: [English](README.md)**
+
+# 本地代码解释器
+OpenAI的ChatGPT代码解释器（Code Interpreter或Advanced Data Analysis）的本地版。
+
+## 简介
+
+OpenAI的ChatGPT代码解释器（Code Interpreter，现更名为Advanced Data Analysis）是一款强大的AI工具。然而，其在在线沙箱环境中运行代码的特性导致了一些限制，如包的缺失、上传速度较慢、仅支持上传不超过100MB的文件以及代码最多只能运行120秒等。为此，我们推出了本地代码解释器（Local Code Interpreter）。这款工具允许您在自己的设备上，利用自己专属的Python环境来执行ChatGPT生成的代码，从而解除了原有解释器的各种限制。
+![notebook_gif_demo](example_img/save_to_notebook_demo.gif)
+
+## 优势
+
+- **自定义环境**：在您本地环境中运行代码，确保各种依赖都已正确安装。
+
+- **无缝体验**：告别100MB文件大小限制和网速问题。使用本地版代码解释器，一切尽在掌控之中。
+
+- **可用GPT-3.5**：官方代码解释器只能在GPT-4中使用，但现在您甚至可以在一轮对话中自由切换GPT-3.5和GPT-4。
+
+- **数据更安全**：代码在本地运行，无需将文件上传至网络，提高了数据的安全性。
+
+- **支持Jupyter**：本程序可将代码和对话历史保存至Jupyter notebook文件中供以后使用。
+
+## 注意事项
+在您自己的设备上执行AI生成但未经人工审核的代码可能存在安全风险。在运行此程序前，您应当采用一些安全措施，例如使用虚拟机，以保护您的设备和数据。使用此程序所产生的所有后果，您需自行承担。
+
+## 使用方法
+
+### 安装
+
+1. 克隆本仓库
+   ```shell
+   git clone https://github.com/MrGreyfun/Local-Code-Interpreter.git
+   cd Local-Code-Interpreter
+   ```
+
+2. 安装依赖。该程序已在Windows 10和CentOS Linux 7.8上使用Python 3.9.16测试。所需的库及版本：
+   ```text 
+   Jupyter Notebook    6.5.4
+   gradio              3.39.0
+   openai              0.27.8
+   ansi2html           1.8.0 
+   ```
+   其他系统或库版本也可能有效。请注意，不要将`openai`包升级至最新的`1.x`版本，该版本已重写，与旧版本不兼容。
+   您可以使用以下命令直接安装所需的软件包：
+   ```shell
+   pip install -r requirements.txt
+   ```
+   如果您不熟悉Python，可以使用以下命令安装，它将额外安装常用的Python数据分析库：
+   ```shell
+   pip install -r requirements_full.txt
+   ```
+### 配置
+
+1. 在`src`目录中创建一个`config.json`文件，参照`config_example`目录中提供的示例进行配置。
+
+2. 在`config.json`文件中配置您的API密钥。
+
+请注意：
+1. **正确设置`model_name`**
+    该程序依赖于`0163`及以上版本的模型的函数调用能力，这些模型包括：
+    - `gpt-3.5-turbo-0613` (及其16K版本)
+    - `gpt-3.5-turbo-1106`
+    - `gpt-4-0613` (及其32K版本)
+    - `gpt-4-1106-preview` 
+
+   旧版本的模型将无法使用。请注意，`gpt-4-vision-preview`模型同样不支持函数调用，因此不能将其设置为`GPT-4`模型。
+
+   对于使用Azure OpenAI的用户：
+   - 请将`model_name`设置为您的模型的部署名（deployment name）。
+   - 确认部署的模型是`0613`及以上版本。
+
+2. **API版本设置**
+    如果您使用Azure OpenAI服务，请在`config.json`文件中将`API_VERSION`设置为`2023-07-01-preview`，其他API版本不支持函数调用。
+
+3. **视觉模型设置**
+   尽管`gpt-4-vision-preview`模型不支持函数调用，我们仍然通过另一种非端到端的方式实现了图像输入。如果想使用图像输入，请将`gpt-4-vision-preview`设置为`GPT-4V`模型，并设置`available`字段设置为`true`。当不需要使用图像输入时候，可以将`available`字段设置为`false`，这将移除图像相关的系统提示，从而减少您的API费用。
+   ![vision_demo](example_img/vision_example.jpg)
+4. **模型上下文窗口长度设置**
+    `model_context_window` 字段记录了每个模型的上下文窗口长度信息。当对话长度超过模型上下文窗口长度限制时，本程序会使用该信息来压缩对话长度。
+    Azure OpenAI的用户需要按照以下格式，使用模型的部署名手动添加上下文窗口长度信息：
+    ```json
+    "<模型部署名>": <上下文窗口长度 (整数)>
+    ```
+    此外，当OpenAI推出新模型的时候，您可以按照相同的格式手动添加新模型的上下文窗口长度信息。（我们会持续更新该文件，但是不一定及时）
+
+5. **使用环境变量配置密钥**
+    如果您不希望将API密钥存储在`config.json`文件中，可以选择通过环境变量来设置密钥：
+    - 将`config.json`文件中的`API_KEY`设为空字符串：
+        ```json
+        "API_KEY": ""
+        ```
+    - 在运行程序之前，使用您的API密钥设置环境变量`OPENAI_API_KEY`：
+        - Windows：
+        ```shell
+        set OPENAI_API_KEY=<你的API密钥>
+        ```
+        - Linux：
+        ```shell
+        export OPENAI_API_KEY=<你的API密钥>
+        ```
+
+## 使用
+
+1. 进入`src`目录。
+   ```shell
+   cd src
+   ```
+
+2. 运行以下命令：
+   ```shell
+   python web_ui.py
+   ```
+
+3. 在浏览器中访问终端生成的链接，开始使用本地版代码解释器。
+
+4. 添加`-n`或`--notebook`参数可以将对话保存到Jupyter notebook中。 
+   默认情况下，该Jupyter notebook文件保存在工作目录中，您可以添加路径以将其保存到其它位置。
+   ```shell
+   python web_ui.py-n<path_to_notebook>
+   ```
+
+## 示例
+
+以下是一个使用本程序进行线性回归任务的示例：
+
+1. 上传数据文件并要求模型对数据进行线性回归：
+   ![Example 1](example_img/1.jpg)
+
+2. 生成的代码执行中遇到错误：
+   ![Example 2](example_img/2.jpg)
+
+3. ChatGPT自动检查数据格式并修复bug：
+   ![Example 3](example_img/3.jpg)
+
+4. 修复bug后的代码成功运行：
+   ![Example 4](example_img/4.jpg)
+
+5. 最终结果符合要求：
+   ![Example 5](example_img/5.jpg)
+   ![Example 6](example_img/6.jpg)

OpenCodeInterpreter/data_collection/Local-Code-Interpreter/config_example/config.azure.example.json

@@ -0,0 +1,24 @@
+{
+  "API_TYPE": "azure",
+  "API_base": "<YOUR-API-ENDPOINT>",
+  "API_VERSION": "2023-12-01-preview",
+  "API_KEY": "<YOUR-API-KEY>",
+  "model": {
+    "GPT-3.5": {
+      "model_name": "<YOUR-DEPLOYMENT-NAME>",
+      "available": true
+    },
+    "GPT-4": {
+      "model_name": "<YOUR-DEPLOYMENT-NAME>",
+      "available": true
+    },
+    "GPT-4V": {
+      "model_name": "<YOUR-DEPLOYMENT-NAME>",
+      "available": true
+    }
+  },
+  "model_context_window": {
+    "<YOUR-DEPLOYMENT-NAME1>": <contex_window (integer)>,
+    "<YOUR-DEPLOYMENT-NAME2>": <contex_window (integer)>
+  }
+}

OpenCodeInterpreter/data_collection/Local-Code-Interpreter/config_example/config.example.json

@@ -0,0 +1,32 @@
+{
+  "API_TYPE": "open_ai",
+  "API_base": "https://api.openai.com/v1",
+  "API_VERSION": null,
+  "API_KEY": "<YOUR-API-KEY>",
+  "model": {
+    "GPT-3.5": {
+      "model_name": "gpt-3.5-turbo-0613",
+      "available": true
+    },
+    "GPT-4": {
+      "model_name": "gpt-4-0613",
+      "available": true
+    },
+    "GPT-4V": {
+      "model_name": "gpt-4-vision-preview",
+      "available": true
+    }
+  },
+  "model_context_window": {
+    "gpt-3.5-turbo": 4096,
+    "gpt-3.5-turbo-16k": 16385,
+    "gpt-3.5-turbo-0613": 4096,
+    "gpt-3.5-turbo-1106": 16385,
+    "gpt-4": 8192,
+    "gpt-4-32k": 32768,
+    "gpt-4-0613": 8192,
+    "gpt-4-32k-0613": 32768,
+    "gpt-4-1106-preview": 128000,
+    "gpt-4-vision-preview": 128000
+  }
+}