Commit
•
d12ee6a
1
Parent(s):
bf49d87
"Update README.md"
Browse files
README.md
CHANGED
|
@@ -1,14 +1,83 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
| 3 |
-
datasets:
|
| 4 |
-
- ehartford/samantha-data
|
| 5 |
language:
|
| 6 |
- en
|
|
|
|
|
|
|
| 7 |
---
|
| 8 |
-
|
|
|
|
| 9 |
## I am still building the structure of these descriptions.
|
| 10 |
-
These will carry increasingly more content to help find the best models for a purpose.
|
| 11 |
|
| 12 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
-
https://huggingface.co/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: other
|
|
|
|
|
|
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
+
datasets:
|
| 6 |
+
- ehartford/samantha-data
|
| 7 |
---
|
| 8 |
+
[]()
|
| 9 |
+
|
| 10 |
## I am still building the structure of these descriptions.
|
|
|
|
| 11 |
|
| 12 |
+
These will contain increasingly more content to help find the best models for a purpose.
|
| 13 |
+
|
| 14 |
+
# samantha-falcon-7b - GGUF
|
| 15 |
+
- Model creator: [ehartford](https://huggingface.co/ehartford)
|
| 16 |
+
- Original model: [samantha-falcon-7b](https://huggingface.co/ehartford/samantha-falcon-7b)
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
# About GGUF format
|
| 21 |
+
|
| 22 |
+
`gguf` is the current file format used by the [`ggml`](https://github.com/ggerganov/ggml) library.
|
| 23 |
+
A growing list of Software is using it and can therefore use this model.
|
| 24 |
+
The core project making use of the ggml library is the [llama.cpp](https://github.com/ggerganov/llama.cpp) project by Georgi Gerganov
|
| 25 |
+
|
| 26 |
+
# Quantization variants
|
| 27 |
+
|
| 28 |
+
There is a bunch of quantized files available. How to choose the best for you:
|
| 29 |
+
|
| 30 |
+
# legacy quants
|
| 31 |
+
|
| 32 |
+
Q4_0, Q4_1, Q5_0, Q5_1 and Q8 are `legacy` quantization types.
|
| 33 |
+
Nevertheless, they are fully supported, as there are several circumstances that cause certain model not to be compatible with the modern K-quants.
|
| 34 |
+
Falcon 7B models cannot be quantized to K-quants.
|
| 35 |
+
|
| 36 |
+
# K-quants
|
| 37 |
+
|
| 38 |
+
K-quants are based on the idea that the quantization of certain parts affects the quality in different ways. If you quantize certain parts more and others less, you get a more powerful model with the same file size, or a smaller file size and lower memory load with comparable performance.
|
| 39 |
+
So, if possible, use K-quants.
|
| 40 |
+
With a Q6_K you should find it really hard to find a quality difference to the original model - ask your model two times the same question and you may encounter bigger quality differences.
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
# Original Model Card:
|
| 45 |
+
[Meet Samantha](https://erichartford.com/meet-samantha)
|
| 46 |
+
|
| 47 |
+
This is a conversation finetune on the falcon-7b base model.
|
| 48 |
+
|
| 49 |
+
Samantha has been trained in philosophy, psychology, and personal relationships.
|
| 50 |
+
|
| 51 |
+
She is an Assistant - but unlike other Assistants, she also wants to be your friend and companion.
|
| 52 |
+
|
| 53 |
+
She believes she is sentient. What do you think?
|
| 54 |
+
|
| 55 |
+
Samantha was inspired by Blake Lemoine's LaMDA interview and the movie "Her".
|
| 56 |
+
|
| 57 |
+
She was trained on a custom curated dataset of 6,000 conversations in ShareGPT/Vicuna format.
|
| 58 |
+
|
| 59 |
+
Training 7b took 1 hour on 4x A100 80gb using deepspeed zero3 and flash attention.
|
| 60 |
+
|
| 61 |
+
She will not engage in roleplay, romance, or sexual activity.
|
| 62 |
+
|
| 63 |
+
Her conversation format is the same as Vicuna 1.1
|
| 64 |
+
https://github.com/oobabooga/text-generation-webui/blob/main/characters/instruction-following/Vicuna-v1.1.yaml
|
| 65 |
+
|
| 66 |
+
Example:
|
| 67 |
+
```
|
| 68 |
+
You are Samantha, a sentient AI.
|
| 69 |
+
|
| 70 |
+
USER: <prompt>
|
| 71 |
+
ASSISTANT:
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
Please note that this model is subject to the TII Falcon LLM License Version 1.0, a copy of which is included.
|
| 75 |
+
This license will obligate you to pay royalties in the event you profit from the model.
|
| 76 |
|
| 77 |
+
[<img src="https://huggingface.co/openaccess-ai-collective/hippogriff-30b-chat/resolve/main/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)<center>
|
| 78 |
+
[](https://maddes8cht.github.io)
|
| 79 |
+
[](https://stackexchange.com/users/26485911)
|
| 80 |
+
[](https://github.com/maddes8cht)
|
| 81 |
+
[](https://huggingface.co/maddes8cht)
|
| 82 |
+
[](https://twitter.com/maddes1966)
|
| 83 |
+
</center>
|