marianbasti
commited on
Commit
•
5243887
1
Parent(s):
94cd439
Update README.md
Browse files
README.md
CHANGED
@@ -13,7 +13,7 @@ This is a LoRA for Llama 2 13B trained on a translated [alpaca dataset](https://
|
|
13 |
Base model used was [The Bloke's Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16) trained in 4bit precision with an added padding token.
|
14 |
|
15 |
## Important INFO
|
16 |
-
The original Llama 2 model does not have a padding token, this came to be restrictive
|
17 |
```python
|
18 |
from transformers import LlamaTokenizer, LlamaForCausalLM
|
19 |
|
|
|
13 |
Base model used was [The Bloke's Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16) trained in 4bit precision with an added padding token.
|
14 |
|
15 |
## Important INFO
|
16 |
+
The original Llama 2 model does not have a padding token, this came to be restrictive when training. To address this, I added a padding token to the tokenizer associated with the model.
|
17 |
```python
|
18 |
from transformers import LlamaTokenizer, LlamaForCausalLM
|
19 |
|