File size: 3,493 Bytes
b2f1ff7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 |
---
title: "WSB-GPT-7B Quantized in GGUF"
tags:
- GGUF
language: en
---
# Tsunemoto GGUF's of WSB-GPT-7B
This is a GGUF quantization of WSB-GPT-7B.
## Original Repo Link:
[Original Repository](https://huggingface.co/Sentdex/WSB-GPT-7B)
## Original Model Card:
---
# Model Card for WSB-GPT-7B
This is a Llama 2 7B Chat model fine-tuned with QLoRA on 2017-2018ish /r/wallstreetbets subreddit comments and responses, with the hopes of learning more about QLoRA and creating models with a little more character.
### Model Description
- **Developed by:** Sentdex
- **Shared by:** Sentdex
- **GPU Compute provided by:** [Lambda Labs](https://lambdalabs.com/service/gpu-cloud)
- **Model type:** Instruct/Chat
- **Language(s) (NLP):** Multilingual from Llama 2, but not sure what the fine-tune did to it, or if the fine-tuned behavior translates well to other languages. Let me know!
- **License:** Apache 2.0
- **Finetuned from Llama 2 7B Chat**
- **Demo [optional]:** [More Information Needed]
## Uses
This model's primary purpose is to be a fun chatbot and to learn more about QLoRA. It is not intended to be used for any other purpose and some people may find it abrasive/offensive.
## Bias, Risks, and Limitations
This model is prone to using at least 3 words that were popularly used in the WSB subreddit in that era that are much more frowned-upon. As time goes on, I may wind up pruning or find-replacing these words in the training data, or leaving it.
Just be advised this model can be offensive and is not intended for all audiences!
## How to Get Started with the Model
### Prompt Format:
```
### Comment:
[parent comment text]
### REPLY:
[bot's reply]
### END.
```
Use the code below to get started with the model.
```py
from transformers import pipeline
# Initialize the pipeline for text generation using the Sentdex/WSB-GPT-7B model
pipe = pipeline("text-generation", model="Sentdex/WSB-GPT-7B")
# Define your prompt
prompt = """### Comment:
How does the stock market actually work?
### REPLY:
"""
# Generate text based on the prompt
generated_text = pipe(prompt, max_length=128, num_return_sequences=1)
# Extract and print the generated text
print(generated_text[0]['generated_text'].split("### END.")[0])
```
Example continued generation from above:
```
### Comment:
How does the stock market actually work?
### REPLY:
You sell when you are up and buy when you are down.
```
Despite `</s>` being the typical Llama stop token, I was never able to get this token to be generated in training/testing so the model would just never stop generating. I wound up testing with ### END. and that worked, but obviously isn't ideal. Will fix this in the future maybe(tm).
#### Hardware
This QLoRA was trained on a Lambda Labs 1x H100 80GB GPU instance.
## Citation
- Llama 2 (Meta AI) for the base model.
- Farouk E / Far El: https://twitter.com/far__el for helping with all my silly questions about QLoRA
- Lambda Labs for the compute. The model itself only took a few hours to train, but it took me days to learn how to tie everything together.
- Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer for QLoRA + implementation on github: https://github.com/artidoro/qlora/
- @eugene-yh and @jinyongyoo on Github + @ChrisHayduk for the QLoRA merge: https://gist.github.com/ChrisHayduk/1a53463331f52dca205e55982baf9930
## Model Card Contact
harrison@pythonprogramming.net |