File size: 2,235 Bytes
bc99c2c 41edd75 26a5c35 8f9ab23 9ae1bfc 05fd021 9ae1bfc 41d0448 1ac84eb 41d0448 9ae1bfc ab1cee5 9ae1bfc 5c41fe9 9ae1bfc ab1cee5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
---
license: other
---
Experimental: Created using an unofficial and unsupported method. I have no metrics on how this performs against 13b and I'm not planning on gathering any at this point. Still has weak spots that need work.
https://huggingface.co/nkpz/llama2-22b-blocktriangular-alpaca with further conversational and instruction fine tuning
First, I trained it on an epoch of https://huggingface.co/datasets/Adapting/empathetic_dialogues_v2 to give it a decent base knowledge of a casual chat style. I added some automated capitalization fixes for this data.The result was conversational, but not very smart.
Then I trained it on an epoch of https://huggingface.co/datasets/vicgalle/alpaca-gpt4 and landed here, a model that is capable of chatting but very focused on following instructions.
**Future Ideas**
- **This strongly prefers the alpaca prompt format and will try to autocomplete it if you don't provide it.** I'd like to work on removing this fixation and making it more flexible.
- Also would like to filter the rows with phrases "AI assistant" and "virtual assistant" from all future runs.
- Thinking it might also help to do a short run on a dataset focused on character impersonation
**Prompting**
Standard prompt format examples:
```
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
List 3 ingredients for the following recipe.
### Input:
Spaghetti Bolognese
### Response:
```
Or
```
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
List 3 ingredients for the following recipe: Spaghetti Bolognese
### Response:
```
For a chat session, I've had success using this simplified prompt:
```
### Scenario
You are speaking with Alexander Graham Bell
### Begin Chat (Format: [Person1]: [Message]\n[Person2]: [Message])
You: Hey, can you tell me a little bit about yourself?
```
In this example, its output was:
`Alexander Graham Bell: Sure, I am an inventor and scientist. I'm most known for inventing the telephone.`
You can customize the use of `### ` prefixed labels to create your own structure. |