File size: 3,423 Bytes
2d04683
 
 
 
9c4070d
 
 
 
2d04683
 
59d4918
913ee03
3c99ba8
 
 
 
 
 
2d04683
 
 
3c99ba8
 
 
 
 
890010f
913ee03
 
 
 
8c7a2aa
913ee03
 
df86a74
 
 
3e2f078
df86a74
 
 
d46c2b6
 
df86a74
d46c2b6
 
df86a74
62ff100
df86a74
3e2f078
 
 
df86a74
3c99ba8
2d04683
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3e2f078
d46c2b6
 
 
 
 
 
3e2f078
 
 
 
 
 
d46c2b6
 
2d04683
3f6f06e
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
---
license: apache-2.0
tags:
- merge
language:
- en
library_name: transformers
pipeline_tag: text-generation
---

## NOTE: For experimental purposes

<p align="center">
  <img src="https://huggingface.co/sethuiyer/Chikuma/resolve/main/chikuma.webp" height="256px" alt="Chikuma">
</p>


Chikuma is a 10.7B parameter model and is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [sethuiyer/SynthIQ-7b](https://huggingface.co/sethuiyer/SynthIQ-7b)
* [openchat/openchat-3.5-0106](https://huggingface.co/openchat/openchat-3.5-0106)

The name "Chikuma" is inspired by the [Chikuma River](https://en.wikipedia.org/wiki/Shinano_River), the longest in Japan, known for its continuous flow and meandering path. 
This metaphorically represents the model's depth, fluidity, and adaptability in processing and understanding language.

It also perfectly fits the approach taken here - Depth Upscaling, inspired by SOLAR 10.7B.

## Nous LLM Evaluation (Version 1 - with ChatML Prompt Template)
|                             Model                             |AGIEval|GPT4All|TruthfulQA|Bigbench|Average|
|---------------------------------------------------------------|------:|------:|---------:|-------:|------:|
|[Chikuma_10.7B](https://huggingface.co/sethuiyer/Chikuma_10.7B)|  42.41|  73.41|     56.69|    43.5|     54|

More details can be found [here](https://gist.github.com/sethuiyer/08b4498ed13a6dead38ad3a6f12e349a)


### Recommended Prompt Template

```text
<|im_start|>GPT4 Correct system
You are Chikuma, a constantly learning AI assistant who strives to be
insightful, engaging, and helpful. You possess vast knowledge and creativity,
but also a humble curiosity about the world and the people you interact
with. If you don't know the answer to a question, please don't share false information. 
Always use <|end_of_turn|> when you want to end the answer.<|im_end|>
<|im_start|>GPT4 Correct User:
{{Input}}
<|im_end|>GPT4 Correct Assistant:
```
ChatML format also works well.

## Tested to work well in :
1. [text-generation-webui](https://github.com/oobabooga/text-generation-webui), eos_token_id=32000, LLaMa-Precise sampling settings.
2. `transformers` text generation pipeline, temperature=4.0, top_k=50, top_p=0.01, eos_token_id=32000.


## 🧩 Configuration

```yaml
slices:
  - sources:
    - model: sethuiyer/SynthIQ-7b
      layer_range: [0, 24]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [8, 32]
merge_method: passthrough
dtype: bfloat16
```

## 💻 Usage

```python
sys_message = ''' 
You are Chikuma, a constantly learning AI assistant who strives to be
insightful, engaging, and helpful. You possess vast knowledge and creativity,
but also a humble curiosity about the world and the people you interact
with. If you don't know the answer to a question, please don't share false information. 
Always use <|end_of_turn|> when you want to end the answer.
'''

question = '''
Tell me what is a large language model in under 250 words.
'''

messages = [{"role":"system", "content": sys_message}, {"role": "user", "content": question}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=4.0, top_k=50, top_p=0.01, eos_token_id=32000)
print(outputs[0]["generated_text"])
```