Antony Kalloniatis commited on
Commit
daaf979
1 Parent(s): eda47fe

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -7
README.md CHANGED
@@ -9,23 +9,27 @@ pipeline_tag: text-classification
9
  ---
10
  #
11
  #
12
- # This model uses the Llama-3 model ("meta-llama/Meta-Llama-3-8B") fine-tuned with 4 bit quantization
13
- # Parameter Efficient Fine Tuning - PEFT training, using LoRA and QLoRA adaptations for the task
14
- # of Humor Recognition in Greek language.
15
  #
16
 
17
 
18
 
19
  ## Model Details
20
 
21
- The model was pre-trained on Greek Humorous Dataset
22
- #
 
 
 
 
23
  ## Pre-processing details
24
 
25
- The text needs to be pre-processed by removing all greek diacritics and punctuation
26
- and converting all letters to lowercase
 
27
 
28
  ## Load Pretrained Model
 
29
 
30
  ```python
31
  from transformers import AutoTokenizer, AutoModelForSequenceClassification
 
9
  ---
10
  #
11
  #
12
+ # This model uses the Llama-3 model ("meta-llama/Meta-Llama-3-8B") fine-tuned with 4 bit quantization Parameter Efficient Fine Tuning - PEFT training, using LoRA and QLoRA adaptations for the task of Humor Recognition in Greek language.
 
 
13
  #
14
 
15
 
16
 
17
  ## Model Details
18
 
19
+ The model was pre-trained on Greek Humorous Dataset
20
+
21
+ ## PEFT Configs
22
+ * Bits and bytes config for quantization - QLoRA
23
+ * LoRA config for LoRA adaptation
24
+
25
  ## Pre-processing details
26
 
27
+ The text needs to be pre-processed by:
28
+ * removing all greek diacritics and punctuations
29
+ * converting all letters to lowercase
30
 
31
  ## Load Pretrained Model
32
+ pad_token needs to be handle since Llama-3 pre-training doesn't have eos_token
33
 
34
  ```python
35
  from transformers import AutoTokenizer, AutoModelForSequenceClassification