Primeness commited on
Commit
7716a64
1 Parent(s): 4889bd0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -70
README.md CHANGED
@@ -1,70 +0,0 @@
1
- ---
2
- language:
3
- - en
4
- thumbnail: null
5
- tags:
6
- - text generation
7
- - instruct
8
- pipeline_tag: text-generation
9
- inference: false
10
- license: llama2
11
- datasets:
12
- - PygmalionAI/PIPPA
13
- - Open-Orca/OpenOrca
14
- - Norquinal/claude_multiround_chat_30k
15
- - jondurbin/airoboros-gpt4-1.4.1
16
- - databricks/databricks-dolly-15k
17
- ---
18
- <h1 style="text-align: center">Pygmalion-2 7B</h1>
19
- <h2 style="text-align: center">An instruction-tuned Llama-2 biased towards fiction writing and conversation.</h2>
20
-
21
- ## Model Details
22
-
23
- The long-awaited release of our new models based on Llama-2 is finally here. Pygmalion-2 7B (formerly known as Metharme) is based on
24
- [Llama-2 7B](https://huggingface.co/meta-llama/llama-2-7b-hf) released by Meta AI.
25
-
26
- The Metharme models were an experiment to try and get a model that is usable for conversation, roleplaying and storywriting,
27
- but which can be guided using natural language like other instruct models. After much deliberation, we reached the conclusion
28
- that the Metharme prompting format is superior (and easier to use) compared to the classic Pygmalion.
29
-
30
- This model was trained by doing supervised fine-tuning over a mixture of regular instruction data alongside roleplay, fictional stories
31
- and conversations with synthetically generated instructions attached.
32
-
33
- This model is freely available for both commercial and non-commercial use, as per the Llama-2 license.
34
-
35
-
36
- ## Prompting
37
-
38
- The model has been trained on prompts using three different roles, which are denoted by the following tokens: `<|system|>`, `<|user|>` and `<|model|>`.
39
-
40
- The `<|system|>` prompt can be used to inject out-of-channel information behind the scenes, while the `<|user|>` prompt should be used to indicate user input.
41
- The `<|model|>` token should then be used to indicate that the model should generate a response. These tokens can happen multiple times and be chained up to
42
- form a conversation history.
43
-
44
- ### Prompting example
45
-
46
- The system prompt has been designed to allow the model to "enter" various modes and dictate the reply length. Here's an example:
47
-
48
- ```
49
- <|system|>Enter RP mode. Pretend to be {{char}} whose persona follows:
50
- {{persona}}
51
-
52
- You shall reply to the user while staying in character, and generate long responses.
53
- ```
54
-
55
- ## Dataset
56
- The dataset used to fine-tune this model includes our own [PIPPA](https://huggingface.co/datasets/PygmalionAI/PIPPA), along with several other instruction
57
- datasets, and datasets acquired from various RP forums.
58
-
59
- ## Limitations and biases
60
-
61
- The intended use-case for this model is fictional writing for entertainment purposes. Any other sort of usage is out of scope.
62
-
63
- As such, it was **not** fine-tuned to be safe and harmless: the base model _and_ this fine-tune have been trained on data known to contain profanity and texts that
64
- are lewd or otherwise offensive. It may produce socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive.
65
- Outputs might often be factually wrong or misleading.
66
-
67
- ## Acknowledgements
68
- We would like to thank [SpicyChat](https://spicychat.ai/) for sponsoring the training for this model.
69
-
70
- [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)