Initial upload of GGML models.

Manticore-13B.ggmlv2.q4_0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ceb97efc6843ed07b9b56de89bfad248d3f6ae93e48853013325e1d99724183
+size 8136770688

Manticore-13B.ggmlv2.q4_1.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b7cb29c5366679bc7b42f07b428a958e0fc6a1743505e02f295aa16eb488d97
+size 9763701888

Manticore-13B.ggmlv2.q5_0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f205d0d02812a88e2bc6c0e7c132f41a1bb10815128786f0554af417d86eca8b
+size 8950236288

Manticore-13B.ggmlv2.q5_1.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6628f7cd71387fd8b2f3e55036d14dd7d83e4df72f89b8d3100be433b3cff88
+size 9763701888

Manticore-13B.ggmlv2.q8_0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1896b05065e5cf4f1c514424568d7bf4c7327287391276ea013660498b9e63e
+size 14644495488

README.md

@@ -0,0 +1,114 @@
+---
+datasets:
+- anon8231489123/ShareGPT_Vicuna_unfiltered
+- ehartford/wizard_vicuna_70k_unfiltered
+- ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered
+- QingyiSi/Alpaca-CoT
+- teknium/GPT4-LLM-Cleaned
+- teknium/GPTeacher-General-Instruct
+- metaeval/ScienceQA_text_only
+- hellaswag
+- tasksource/mmlu
+- openai/summarize_from_feedback
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+---
+
+# Manticore 13B GGML
+
+This is GGML format quantised 4bit and 5bit models of [OpenAccess AI Collective's Manticore 13B](https://huggingface.co/openaccess-ai-collective/manticore-13b).
+
+This repo is the result of quantising to 4-bit, 5-bit and 8-bit GGML for CPU (+CUDA) inference using [llama.cpp](https://github.com/ggerganov/llama.cpp).
+
+## Repositories available
+
+* [4-bit GPTQ models for GPU inference](https://huggingface.co/TheBloke/TheBloke/Manticore-13B-GPTQ).
+* [4-bit, 5-bit 8-bit GGML models for llama.cpp CPU (+CUDA) inference](https://huggingface.co/TheBloke/TheBloke/Manticore-13B-GGML).
+* [OpenAccess AI Collective's original float16 HF format repo for GPU inference and further conversions](https://huggingface.co/openaccess-ai-collective/manticore-13b).
+
+## THE FILES IN MAIN BRANCH REQUIRES LATEST LLAMA.CPP (May 12th 2023 - commit b9fd7ee)!
+
+llama.cpp recently made a breaking change to its quantisation methods.
+
+I have quantised the GGML files in this repo with the latest version. Therefore you will require llama.cpp compiled on May 12th or later (commit `b9fd7ee` or later) to use them.
+
+## Provided files
+| Name | Quant method | Bits | Size | RAM required | Use case |
+| ---- | ---- | ---- | ---- | ---- | ----- |
+`manticore-13B.ggmlv2.q4_0.bin` | q4_0 | 4bit | 8.14GB | 10.5GB | 4-bit. |
+`manticore-13B.ggmlv2.q4_1.bin` | q5_0 | 5bit | 8.95GB | 11.0GB | 5-bit. Higher accuracy, higher resource usage and slower inference. |
+`manticore-13B.ggmlv2.q5_0.bin` | q5_1 | 5bit | 9.76GB | 12.25GB | 5-bit. Even higher accuracy, and higher resource usage and slower inference. |
+`manticore-13B.ggmlv2.q8_0.bin` | q8_0 | 8bit | 14.6GB | 17GB | 8-bit. Almost indistinguishable from float16. Huge resource use and slow. Not recommended for normal use. |
+
+## How to run in `llama.cpp`
+
+I use the following command line; adjust for your tastes and needs:
+
+```
+./main -t 8 -m manticore-13B-.ggmlv2.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction: write a story about llamas ### Response:"
+```
+
+Change `-t 8` to the number of physical CPU cores you have.
+
+## How to run in `text-generation-webui`
+
+GGML models can be loaded into text-generation-webui by installing the llama.cpp module, then placing the ggml model file in a model folder as usual.
+
+Further instructions here: [text-generation-webui/docs/llama.cpp-models.md](https://github.com/oobabooga/text-generation-webui/blob/main/docs/llama.cpp-models.md).
+
+# Original Model Card: Manticore 13B - Preview Release (previously Wizard Mega)
+
+Manticore 13B is a Llama 13B model fine-tuned on the following datasets:
+- [ShareGPT](https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered) - based on a cleaned and de-suped subset
+- [WizardLM](https://huggingface.co/datasets/ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered)
+- [Wizard-Vicuna](https://huggingface.co/datasets/ehartford/wizard_vicuna_70k_unfiltered)
+- [subset of QingyiSi/Alpaca-CoT for roleplay and CoT](https://huggingface.co/QingyiSi/Alpaca-CoT)
+- [GPT4-LLM-Cleaned](https://huggingface.co/datasets/teknium/GPT4-LLM-Cleaned)
+- [GPTeacher-General-Instruct](https://huggingface.co/datasets/teknium/GPTeacher-General-Instruct)
+- ARC-Easy & ARC-Challenge - instruct augmented  for detailed responses
+- mmlu: instruct augmented for detailed responses subset including
+  - abstract_algebra
+  - conceptual_physics
+  - formal_logic
+  - high_school_physics
+  - logical_fallacies
+- [hellaswag](https://huggingface.co/datasets/hellaswag) - 5K row subset of instruct augmented for concise responses
+- [metaeval/ScienceQA_text_only](https://huggingface.co/datasets/metaeval/ScienceQA_text_only) - instruct for concise responses
+- [openai/summarize_from_feedback](https://huggingface.co/datasets/openai/summarize_from_feedback) - instruct augmented tl;dr summarization
+
+
+# Demo
+
+Try out the model in HF Spaces. The demo uses a quantized GGML version of the model to quickly return predictions on smaller GPUs (and even CPUs). Quantized GGML may have some minimal loss of model quality.
+- https://huggingface.co/spaces/openaccess-ai-collective/manticore-ggml
+
+## Release Notes
+
+- https://wandb.ai/wing-lian/manticore-13b/runs/nq3u3uoh/workspace
+
+## Build
+
+Manticore was built with [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) on 8xA100 80GB 
+ - Preview Release: 1 epoch taking 8 hours. 
+ - The configuration to duplicate this build is provided in this repo's [/config folder](https://huggingface.co/openaccess-ai-collective/manticore-13b/tree/main/configs).
+
+## Bias, Risks, and Limitations
+Manticore has not been aligned to human preferences with techniques like RLHF or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so).
+Manticore was fine-tuned from the base model LlaMa 13B, please refer to its model card's Limitations Section for relevant information.
+
+## Examples
+
+````
+### Instruction: write Python code that returns the first n numbers of the Fibonacci sequence using memoization.
+
+### Assistant: 
+````
+
+```
+### Instruction: Finish the joke, a mechanic and a car salesman walk into a bar...  
+
+### Assistant:
+```
+