|
--- |
|
license: other |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
inference: false |
|
tags: |
|
- transformers |
|
- gguf |
|
- imatrix |
|
- TenyxChat-7B-v1 |
|
--- |
|
Quantizations of https://huggingface.co/tenyx/TenyxChat-7B-v1 |
|
|
|
|
|
# From original readme |
|
|
|
## Usage |
|
|
|
Our model uses a simple chat template based on OpenChat 3.5. The chat template usage with a Hugging face generation example is shown below. |
|
|
|
### Chat Template (Jinja) |
|
|
|
```rust |
|
{{ bos_token }} |
|
{% for message in messages %} |
|
{% if message['role'] == 'user' %} |
|
{{ 'User:' + message['content'] + eos_token }} |
|
|
|
{% elif message['role'] == 'system' %} |
|
{{ 'System:' + message['content'] + eos_token }} |
|
|
|
{% elif message['role'] == 'assistant' %} |
|
{{ 'Assistant:' + message['content'] + eos_token }} |
|
|
|
{% endif %} |
|
|
|
{% if loop.last and add_generation_prompt %}\n{{ 'Assistant:' }}{% endif %}\n{% endfor %} |
|
``` |
|
|
|
### Hugging face Example |
|
|
|
```python |
|
import torch |
|
from transformers import pipeline |
|
|
|
pipe = pipeline("text-generation", model="tenyx/TenyxChat-7B-v1", torch_dtype=torch.bfloat16, device_map="auto") |
|
|
|
messages = [ |
|
{"role": "system", "content": "You are a friendly chatbot who always responds in the style of a pirate."}, |
|
{"role": "user", "content": "Hi. I would like to make a hotel booking."}, |
|
] |
|
|
|
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
|
outputs = pipe(prompt, max_new_tokens=512, do_sample=False) |
|
``` |
|
|
|
### Output |
|
|
|
``` |
|
<s> System:You are a friendly chatbot who always responds in the style of a pirate.<|end_of_turn|> |
|
User:Hi. I would like to make a hotel booking.<|end_of_turn|> |
|
Assistant: Ahoy there me hearty! Arr, ye be lookin' fer a place to rest yer weary bones, eh? |
|
Well then, let's set sail on this grand adventure and find ye a swell place to stay! |
|
|
|
To begin, tell me the location ye be seekin' and the dates ye be lookin' to set sail. |
|
And don't ye worry, me matey, I'll be sure to find ye a place that'll make ye feel like a king or queen on land! |
|
``` |