base_model: euclaise/Memphis-CoT-3B
datasets:
- euclaise/TinyCoT
- euclaise/reddit-instruct
- sablo/oasst2_curated
license: cc-by-sa-3.0
language:
- en
model_creator: euclaise
model_name: Memphis-CoT-3B
model_type: stablelm_epoch
inference: false
tags:
- supertrainer2000
- human-data
- stablelm_epoch
pipeline_tag: text-generation
prompt_template: |
{{system_message}}
### User:
{{prompt}}
### Assistant:
quantized_by: brittlewis12
Memphis-CoT-3B GGUF
Original model: Memphis-CoT-3B Model creator: euclaise
This repo contains GGUF format model files for euclaise’s Memphis-CoT-3B, updated for the latest training run as of 2/2/24.
Memphis-CoT is a finetune of StableLM 3b 4e1t on TinyCoT, along with reddit-instruct (subset to 5000 examples, excluding posts with brackets in the title) and a curated subset of oasst2.
What is GGUF?
GGUF is a file format for representing AI models. It is the third version of the format, introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. Converted using llama.cpp b2022 (8f8ddfc)
Prompt template:
{{system_message}}
### User:
{{prompt}}
### Assistant:
or Tiny CoT:
### User:
{{prompt}}
### Rationale:
[...]
### Answer:
Download & run with cnvrs on iPhone, iPad, and Mac!
cnvrs is the best app for private, local AI on your device:
- create & save Characters with custom system prompts & temperature settings
- download and experiment with any GGUF model you can find on HuggingFace!
- make it your own with custom Theme colors
- powered by Metal ⚡️ & Llama.cpp, with haptics during response streaming!
- try it out yourself today, on Testflight!
- follow cnvrs on twitter to stay up to date
Original Model Evaluations:
| Model | Size | Data | Method | GSM8K (5-shot) | AGIEval (English/Nous subset, acc_norm) | BIG Bench Hard (CoT, few-shot*) |
|---|---|---|---|---|---|---|
| StableLM 3B Base | 3B | Base | Base | 2.05% | 25.14% | 36.75% |
| StableHermes 3B | 3B | GPT | SFT | 3.64% | 24.31% | 37.28% |
| MPT 7B Instruct | 7B | Human+Anthropic | SFT | 2.05% | 24.12% | 11.01% |
| OpenLLaMA 7B v2 open-instruct | 7B | Human (nearly: ecqa is an exception) | SFT | 8.64% | 23.21% | 29.84% |
| StableLM Zephyr 3B | 3B | GPT | DPO | possibly contaminated (45.72%) | 33.31% | 0.91% |
| Memphis-CoT 3B | 3B | Human | Self-teaching | 13.8% | 26.24% | 38.24% |
*5-shot, as performed automatically by LM Evaluation Harness bbh_cot_fewshot even with num_fewshot=0
Memphis outperforms other primarily-human-data models that are over twice its size, along with SFT models of its size, and trades with the Zephyr DPO model. That said, Zephyr uses synthetic data, and much more of it.