Update README.md
Browse files
README.md
CHANGED
@@ -1,5 +1,60 @@
|
|
1 |
-
---
|
2 |
-
license: other
|
3 |
-
license_name: xt-aurora-license
|
4 |
-
license_link:
|
5 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: other
|
3 |
+
license_name: xt-aurora-license
|
4 |
+
license_link: LICENSE
|
5 |
+
language:
|
6 |
+
- en
|
7 |
+
tags:
|
8 |
+
- conversational
|
9 |
+
- chat
|
10 |
+
- rp
|
11 |
+
- roleplay
|
12 |
+
- friend
|
13 |
+
- slm
|
14 |
+
- small
|
15 |
+
- slim
|
16 |
+
- slender
|
17 |
+
- general
|
18 |
+
- creative
|
19 |
+
|
20 |
+
co2_eq_emissions:
|
21 |
+
emissions: 200
|
22 |
+
training_type: "fine-tuning"
|
23 |
+
hardware_used: "1 GTX1060-3GB, AMD Radeon(TM) Graphics & AMD Ryzen 5 5600G[4.4GHz OC]"
|
24 |
+
---
|
25 |
+
|
26 |
+
NOTE / ANNOUNCEMENT:
|
27 |
+
We've jumped from V0.5 to this version, V1.0, this is the last version of the series.
|
28 |
+
We're sad to announce the end of XT_AURORA, our first SLM series, due to no community activity.
|
29 |
+
We, XeTute, have put in a lot of effort and countless nights to improve our models, but given on how much time, passion and effort we've put in, we got nothing back from the community.
|
30 |
+
Thank you for so many downloads on this series of SLMs. We'll continue to update model cards and chat templates.
|
31 |
+
Thank you for being part of our journey.
|
32 |
+
|
33 |
+
About this model:
|
34 |
+
This model, XT_AURORA, is by us, XeTute. The model was finetuned ontop of the previos beta-verion[XT_AURORA-OpenBeta-V0.5-GGUF].
|
35 |
+
This version[V1.0] achieves better general performance, it outperforms every previos model[V0.1 - V0.5].
|
36 |
+
|
37 |
+
About XT_AURORA:
|
38 |
+
XT_AURORA is a series of SLMs[Slender Language Models], which all aim to provide a friendly, human-like conversation.
|
39 |
+
The serie is limited by its size[about 1.1B Params], but we still try to get the best possible output.
|
40 |
+
The context-length is very stable till 2048 tokens, after that limit, it will perform only slightly better than V0.5.
|
41 |
+
Tt can be upscaled using rope, with the cost being slightly less logic.
|
42 |
+
|
43 |
+
About this version[V1.0]:
|
44 |
+
* High quality output[sometimes outperforms 3B models in HumanEval], as long as the context size is under 2049 Tokens.
|
45 |
+
* We provide a system prompt[Files and Versions --> chat_template]. The SLM was partly trained using that template, so the output is better if you use the prompt at start.
|
46 |
+
* AURORA expects the chat template to be Vicuna[{{user}}: {some input}\nAURORA: {some output}\n{{user}}]. The model will only work correctly with this format.
|
47 |
+
* Recommended temperature is from 0.4 to 0.75.
|
48 |
+
* Improved chat quality in general emotional / unemotional chat, logical & illogical roleplaying, etc.
|
49 |
+
|
50 |
+
All in one, AURORA's aim is to provide a digital friend, which is also accessible to humans with low-end devices.
|
51 |
+
|
52 |
+
Using KoboldCPP, we got the model running[using termux] on a POCO X5 Pro 5G[CPU only, Octa Core].
|
53 |
+
We saw ~5 Tokens generation per second, ~15 Tokens processing per second. [In Energy Saver mode]
|
54 |
+
|
55 |
+
Support us:
|
56 |
+
X: <https://www.x.com/XeTute>
|
57 |
+
GitHub: <https://www.github.com/N0CTRON/>
|
58 |
+
Subdomain on Neocities: <https://xetute.neocities.org/>
|
59 |
+
|
60 |
+
We wish you a friendly chat with AURORA.
|