Update README.md

README.md

@@ -198,144 +198,6 @@ You are a helpful assistant that answers in JSON. Here's the json schema you mus
 
 Given the {schema} that you provide, it should follow the format of that json to create it's response, all you have to do is give a typical user prompt, and it will respond in JSON.
 
-
-# Benchmarks
-
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/suBbCUIxpcRvhCv6-DBDQ.png)
-
-## GPT4All:
-```
-
-|    Task     |Version| Metric |Value |   |Stderr|
-
-|-------------|------:|--------|-----:|---|-----:|
-
-|arc_challenge|      0|acc     |0.5529|±  |0.0145|
-
-|             |       |acc_norm|0.5870|±  |0.0144|
-
-|arc_easy     |      0|acc     |0.8371|±  |0.0076|
-
-|             |       |acc_norm|0.8144|±  |0.0080|
-
-|boolq        |      1|acc     |0.8599|±  |0.0061|
-
-|hellaswag    |      0|acc     |0.6133|±  |0.0049|
-
-|             |       |acc_norm|0.7989|±  |0.0040|
-
-|openbookqa   |      0|acc     |0.3940|±  |0.0219|
-
-|             |       |acc_norm|0.4680|±  |0.0223|
-
-|piqa         |      0|acc     |0.8063|±  |0.0092|
-
-|             |       |acc_norm|0.8156|±  |0.0090|
-
-|winogrande   |      0|acc     |0.7372|±  |0.0124|
-
-```
-
-Average: 72.59
-
-## AGIEval:
-```
-|             Task             |Version| Metric |Value |   |Stderr|
-|------------------------------|------:|--------|-----:|---|-----:|
-|agieval_aqua_rat              |      0|acc     |0.2441|±  |0.0270|
-|                              |       |acc_norm|0.2441|±  |0.0270|
-|agieval_logiqa_en             |      0|acc     |0.3687|±  |0.0189|
-|                              |       |acc_norm|0.3840|±  |0.0191|
-|agieval_lsat_ar               |      0|acc     |0.2304|±  |0.0278|
-|                              |       |acc_norm|0.2174|±  |0.0273|
-|agieval_lsat_lr               |      0|acc     |0.5471|±  |0.0221|
-|                              |       |acc_norm|0.5373|±  |0.0221|
-|agieval_lsat_rc               |      0|acc     |0.6617|±  |0.0289|
-|                              |       |acc_norm|0.6357|±  |0.0294|
-|agieval_sat_en                |      0|acc     |0.7670|±  |0.0295|
-|                              |       |acc_norm|0.7379|±  |0.0307|
-|agieval_sat_en_without_passage|      0|acc     |0.4417|±  |0.0347|
-|                              |       |acc_norm|0.4223|±  |0.0345|
-|agieval_sat_math              |      0|acc     |0.4000|±  |0.0331|
-|                              |       |acc_norm|0.3455|±  |0.0321|
-```
-
-Average: 44.05
-
-## BigBench:
-
-```
-
-|                      Task                      |Version|       Metric        |Value |   |Stderr|
-|------------------------------------------------|------:|---------------------|-----:|---|-----:|
-|bigbench_causal_judgement                       |      0|multiple_choice_grade|0.6000|±  |0.0356|
-|bigbench_date_understanding                     |      0|multiple_choice_grade|0.6585|±  |0.0247|
-|bigbench_disambiguation_qa                      |      0|multiple_choice_grade|0.3178|±  |0.0290|
-|bigbench_geometric_shapes                       |      0|multiple_choice_grade|0.2340|±  |0.0224|
-|                                                |       |exact_str_match      |0.0000|±  |0.0000|
-|bigbench_logical_deduction_five_objects         |      0|multiple_choice_grade|0.2980|±  |0.0205|
-|bigbench_logical_deduction_seven_objects        |      0|multiple_choice_grade|0.2057|±  |0.0153|
-|bigbench_logical_deduction_three_objects        |      0|multiple_choice_grade|0.5367|±  |0.0288|
-|bigbench_movie_recommendation                   |      0|multiple_choice_grade|0.4040|±  |0.0220|
-|bigbench_navigate                               |      0|multiple_choice_grade|0.4970|±  |0.0158|
-|bigbench_reasoning_about_colored_objects        |      0|multiple_choice_grade|0.7075|±  |0.0102|
-|bigbench_ruin_names                             |      0|multiple_choice_grade|0.4821|±  |0.0236|
-|bigbench_salient_translation_error_detection    |      0|multiple_choice_grade|0.2295|±  |0.0133|
-|bigbench_snarks                                 |      0|multiple_choice_grade|0.6906|±  |0.0345|
-|bigbench_sports_understanding                   |      0|multiple_choice_grade|0.5375|±  |0.0159|
-|bigbench_temporal_sequences                     |      0|multiple_choice_grade|0.6270|±  |0.0153|
-|bigbench_tracking_shuffled_objects_five_objects |      0|multiple_choice_grade|0.2216|±  |0.0118|
-|bigbench_tracking_shuffled_objects_seven_objects|      0|multiple_choice_grade|0.1594|±  |0.0088|
-|bigbench_tracking_shuffled_objects_three_objects|      0|multiple_choice_grade|0.5367|±  |0.0288|
-```
-
-Average: 44.13
-
-**IFEval**: 72.64
-
-**MT_Bench**: Turn 1 - 8.3875, Turn 2 - 8.00625, Average - 8.196875
-
-# Inference Code
-
-Here is example code using HuggingFace Transformers to inference the model (note: in 4bit, it will require around 5GB of VRAM)
-
-Note: To use function calling, you should see the github repo above.
-
-```python
-# Code to inference Hermes with HF Transformers
-# Requires pytorch, transformers, bitsandbytes, sentencepiece, protobuf, and flash-attn packages
-
-import torch
-from transformers import AutoTokenizer, AutoModelForCausalLM, LlamaForCausalLM
-import bitsandbytes, flash_attn
-
-tokenizer = AutoTokenizer.from_pretrained('NousResearch/Hermes-2-Theta-Llama-3-8B', trust_remote_code=True)
-model = LlamaForCausalLM.from_pretrained(
-    "NousResearch/Hermes-2-Theta-Llama-3-8B",
-    torch_dtype=torch.float16,
-    device_map="auto",
-    load_in_8bit=False,
-    load_in_4bit=True,
-    use_flash_attention_2=True
-)
-
-prompts = [
-    """<|im_start|>system
-You are a sentient, superintelligent artificial general intelligence, here to teach and assist me.<|im_end|>
-<|im_start|>user
-Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>
-<|im_start|>assistant""",
-    ]
-
-for chat in prompts:
-    print(chat)
-    input_ids = tokenizer(chat, return_tensors="pt").input_ids.to("cuda")
-    generated_ids = model.generate(input_ids, max_new_tokens=750, temperature=0.8, repetition_penalty=1.1, do_sample=True, eos_token_id=tokenizer.eos_token_id)
-    response = tokenizer.decode(generated_ids[0][input_ids.shape[-1]:], skip_special_tokens=True, clean_up_tokenization_space=True)
-    print(f"Response: {response}")
-```
-
-
 ## Inference Code for Function Calling:
 
 All code for utilizing, parsing, and building function calling templates is available on our github:
@@ -343,18 +205,6 @@ All code for utilizing, parsing, and building function calling templates is avai
 
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/oi4CiGh50xmoviUQnh8R3.png)
 
-# Chat Interfaces
-
-When quantized versions of the model are released, I recommend using LM Studio for chatting with Hermes 2 Pro. It does not support function calling - for that use our github repo. It is a GUI application that utilizes GGUF models with a llama.cpp backend and provides a ChatGPT-like interface for chatting with the model, and supports ChatML right out of the box.
-In LM-Studio, simply select the ChatML Prefix on the settings side pane:
-
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ls6WqV-GSxMw2RA3GuQiN.png)
-
-
-## Quantized Versions:
-
-GGUF Versions Available Here: https://huggingface.co/NousResearch/Hermes-2-Theta-Llama-3-8B-GGUF
-
 # How to cite:
 
 ```bibtext