Update README.md

README.md

@@ -1,79 +1,201 @@
 ---
 license: apache-2.0
+language:
+- en
+library_name: transformers
 pipeline_tag: text-generation
-tags:
-- llm-foundry
-- docsgpt
 ---
 
-DocsGPT-7B is a decoder-style transformer that is fine-tuned specifically for providing answers based on documentation given in context. It is built on to of the MosaicPretrainedTransformer (MPT), being fine-tuned from the MPT-7B model developed by MosaicML. The model inherits the powerful language understanding capabilities of MPT-7B and has been specialized for the purpose of documentation-oriented question answering.
 
+This model is fine tuned on top of llama-2-7b
 
+DocsGPT is optimized for Documentation: Specifically fine-tuned for providing answers that are based on documentation provided in context, making it particularly useful for developers and technical support teams.
 
-## Model Description
+We used 50k high quality examples to finetune it over 1.5 days on A10G GPU.
+We used lora fine tuning process.
 
-Architecture: Decoder-style Transformer
+Its an apache-2.0 license so you can use it for commercial purposes too.
 
-Training data: Fine-tuned on approximately 1000 high-quality examples of documentation answering workflows.
 
-Base model: Fine-tuned version of [MPT-7B](https://huggingface.co/mosaicml/mpt-7b), which is pretrained from scratch on 1T tokens of English text and code.
+# How to run it
+```
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import transformers
+import torch
+
+model = "Arc53/docsgpt-7b"
+
+tokenizer = AutoTokenizer.from_pretrained(model)
+pipeline = transformers.pipeline(
+    "text-generation",
+    model=model,
+    tokenizer=tokenizer,
+    torch_dtype=torch.bfloat16,
+    trust_remote_code=True,
+    device_map="auto",
+)
+sequences = pipeline(
+   "Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:",
+    max_length=200,
+    do_sample=True,
+    top_k=10,
+    num_return_sequences=1,
+    eos_token_id=tokenizer.eos_token_id,
+)
+for seq in sequences:
+    print(f"Result: {seq['generated_text']}")
+```
 
-License: Apache 2.0
 
-## Features
+Benchmarks are still WIP
 
-* Attention with Linear Biases (ALiBi): Inherited from the MPT family, this feature eliminates the context length limits by replacing positional embeddings, allowing for efficient and effective processing of lengthy documents. In future we are planning to finish training on our larger dataset and to increase amount of tokens for context.
-* Optimized for Documentation: Specifically fine-tuned for providing answers that are based on documentation provided in context, making it particularly useful for developers and technical support teams.
-* Easy to Serve: Can be efficiently served using standard HuggingFace pipelines or NVIDIA's FasterTransformer.
 
 
-## How to Use
+To prepare your prompts make sure you keep this format:
 
-```python
-import transformers
-model = transformers.AutoModelForCausalLM.from_pretrained(
-  'Arc53/DocsGPT-7B',
-  trust_remote_code=True
-)
+```
+ ### Instruction
+(where the question goes)
+### Context
+(your document retrieval + system instructions)
+### Answer
 ```
 
 
-This model was uses [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
+Here is an example comparing it to meta-llama/Llama-2-7b
 
-```python
-from transformers import AutoTokenizer
-tokenizer = AutoTokenizer.from_pretrained('EleutherAI/gpt-neox-20b')
+Prompt:
 ```
+### Instruction
+Create a mock request to /api/answer in python
+
+### Context
+You are a DocsGPT, friendly and helpful AI assistant by Arc53 that provides help with documents. You give thorough answers with code examples if possible.
+Use the following pieces of context to help answer the users question. If its not relevant to the question, provide friendly responses.
+You have access to chat history, and can use it to help answer the question.
+When using code examples, use the following format:
+`` ` `` (language)
+(code)
+`` ` ``
+
+----------------
+
+
+/api/answer
+Its a POST request that sends a JSON in body with 4 values. Here is a JavaScript fetch example
+It will recieve an answer for a user provided question
+
+`` ` ``
+// answer (POST http://127.0.0.1:5000/api/answer)
+fetch("http://127.0.0.1:5000/api/answer", {
+      "method": "POST",
+      "headers": {
+            "Content-Type": "application/json; charset=utf-8"
+      },
+      "body": JSON.stringify({"question":"Hi","history":null,"api_key":"OPENAI_API_KEY","embeddings_key":"OPENAI_API_KEY",
+      "active_docs": "javascript/.project/ES2015/openai_text-embedding-ada-002/"})
+})
+.then((res) => res.text())
+.then(console.log.bind(console))
+`` ` ``
+
+In response you will get a json document like this one:
+
+`` ` ``
+{
+  "answer": " Hi there! How can I help you?\\n",
+  "query": "Hi",
+  "result": " Hi there! How can I help you?\\nSOURCES:"
+}
+`` ` ``
 
-## Warning 
+ 
 
-This is a early version, fine tuning with 1k examples is just proof of concept we plan to fine tune on at least 100k more examples
+/api/docs_check
+It will make sure documentation is loaded on a server (just run it everytime user is switching between libraries (documentations)
+Its a POST request that sends a JSON in body with 1 value. Here is a JavaScript fetch example
+
+`` ` ``
+// answer (POST http://127.0.0.1:5000/api/docs_check)
+fetch("http://127.0.0.1:5000/api/docs_check", {
+      "method": "POST",
+      "headers": {
+            "Content-Type": "application/json; charset=utf-8"
+      },
+      "body": JSON.stringify({"docs":"javascript/.project/ES2015/openai_text-embedding-ada-002/"})
+})
+.then((res) => res.text())
+.then(console.log.bind(console))
+`` ` ``
+
+In response you will get a json document like this one:
+`` ` ``
+{
+  "status": "exists"
+}
+`` ` ``
 
-## Documentation
 
-* [Base model documentation](https://github.com/mosaicml/llm-foundry/)
-* Our community [Discord](https://discord.gg/n5BX8dh8rU)
-* [DocsGPT](https://github.com/arc53/DocsGPT) project
- 
 
-## Training Configuration
-It took 3 hours on 4xA100 GPU's on Google Cloud
 
-## Training data
-Its base on all feedback that we have recieved from [Here](https://docsgpt.arc53.com/), There is a thumbs up or a down button next to each response.
-In this version we used 1k responses.
+Issues and Pull requests
 
-## Disclaimer
+We value contributions to our issues in form of discussion or suggestion, we recommend that you check out existing issues and our Roadmap
 
-The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please cosult an attorney before using this model for commercial purposes.
+If you want to contribute by writing code there are few things that you should know before doing it:
+We have frontend (React, Vite) and Backend (python)
 
-## Limitations
 
-Please be aware this is a relatively small llm and its prone to biases and hallucinations
+### Answer
+```
+
+
+docsgpt-7b:
+```
+To create a mock request to `/api/answer` in Python, you can use the `requests` library. Here's an example:
 
+`` ` ``python
+import requests
 
-Our live [demo](https://docsgpt.arc53.com/) that uses a mixture of models
+# Set up the URL and headers
+url = "http://127.0.0.1:5000/api/answer"
+headers = {
+    "Content-Type": "application/json; charset=utf-8"
+}
 
-## Model License
+# Set up the JSON data to send in the request body
+data = {
+    "question": "Hi",
+    "history": null,
+    "api_key": "OPENAI_API_KEY",
+    "embeddings_key": "OPENAI_API_KEY",
+    "active_docs": "javascript/.project/ES2015/openai_text-embedding-ada-002/"
+}
 
-Apache-2.0
+# Send the request and get the response
+response = requests.post(url, headers=headers, json=data)
+
+# Print the response
+print(response.text)
+`` ` ``
+
+This will send a POST request to `/api/answer` with the specified `question`, `history`, `api_key`, `embeddings_key`, and `active_docs` parameters. The response will be printed to the console.
+
+Note that you will need to replace `127.0.0.1:5000` with the actual URL and port of your server. Also, make sure you have the necessary dependencies installed for `requests`.
+```
+
+The original meta-llama/Llama-2-7b:
+```
+### DocsGPT
+
+
+### DocsGPT
+
+
+### DocsGPT
+
+
+### DocsGPT
+
+
+```