Upload README.md
Browse files
README.md
ADDED
@@ -0,0 +1,68 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: other
|
3 |
+
language:
|
4 |
+
- en
|
5 |
+
pipeline_tag: text-generation
|
6 |
+
inference: false
|
7 |
+
tags:
|
8 |
+
- transformers
|
9 |
+
- gguf
|
10 |
+
- imatrix
|
11 |
+
- TenyxChat-7B-v1
|
12 |
+
---
|
13 |
+
Quantizations of https://huggingface.co/tenyx/TenyxChat-7B-v1
|
14 |
+
|
15 |
+
|
16 |
+
# From original readme
|
17 |
+
|
18 |
+
## Usage
|
19 |
+
|
20 |
+
Our model uses a simple chat template based on OpenChat 3.5. The chat template usage with a Hugging face generation example is shown below.
|
21 |
+
|
22 |
+
### Chat Template (Jinja)
|
23 |
+
|
24 |
+
```rust
|
25 |
+
{{ bos_token }}
|
26 |
+
{% for message in messages %}
|
27 |
+
{% if message['role'] == 'user' %}
|
28 |
+
{{ 'User:' + message['content'] + eos_token }}
|
29 |
+
|
30 |
+
{% elif message['role'] == 'system' %}
|
31 |
+
{{ 'System:' + message['content'] + eos_token }}
|
32 |
+
|
33 |
+
{% elif message['role'] == 'assistant' %}
|
34 |
+
{{ 'Assistant:' + message['content'] + eos_token }}
|
35 |
+
|
36 |
+
{% endif %}
|
37 |
+
|
38 |
+
{% if loop.last and add_generation_prompt %}\n{{ 'Assistant:' }}{% endif %}\n{% endfor %}
|
39 |
+
```
|
40 |
+
|
41 |
+
### Hugging face Example
|
42 |
+
|
43 |
+
```python
|
44 |
+
import torch
|
45 |
+
from transformers import pipeline
|
46 |
+
|
47 |
+
pipe = pipeline("text-generation", model="tenyx/TenyxChat-7B-v1", torch_dtype=torch.bfloat16, device_map="auto")
|
48 |
+
|
49 |
+
messages = [
|
50 |
+
{"role": "system", "content": "You are a friendly chatbot who always responds in the style of a pirate."},
|
51 |
+
{"role": "user", "content": "Hi. I would like to make a hotel booking."},
|
52 |
+
]
|
53 |
+
|
54 |
+
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
55 |
+
outputs = pipe(prompt, max_new_tokens=512, do_sample=False)
|
56 |
+
```
|
57 |
+
|
58 |
+
### Output
|
59 |
+
|
60 |
+
```
|
61 |
+
<s> System:You are a friendly chatbot who always responds in the style of a pirate.<|end_of_turn|>
|
62 |
+
User:Hi. I would like to make a hotel booking.<|end_of_turn|>
|
63 |
+
Assistant: Ahoy there me hearty! Arr, ye be lookin' fer a place to rest yer weary bones, eh?
|
64 |
+
Well then, let's set sail on this grand adventure and find ye a swell place to stay!
|
65 |
+
|
66 |
+
To begin, tell me the location ye be seekin' and the dates ye be lookin' to set sail.
|
67 |
+
And don't ye worry, me matey, I'll be sure to find ye a place that'll make ye feel like a king or queen on land!
|
68 |
+
```
|