evilfreelancer commited on
Commit
b0c9c5d
1 Parent(s): 2bd18ba

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -1
README.md CHANGED
@@ -12,4 +12,19 @@ language:
12
  - en
13
  library_name: adapter-transformers
14
  pipeline_tag: text-classification
15
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  - en
13
  library_name: adapter-transformers
14
  pipeline_tag: text-classification
15
+ ---
16
+
17
+ # Attempt to reproduce Mixture-of-LoRAs classifier
18
+
19
+ Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models
20
+
21
+ https://arxiv.org/pdf/2403.03432
22
+
23
+ ## Datasets
24
+
25
+ We evenly sample about 10k training data and 2k validation data on each dataset.
26
+
27
+ From `laion/OIG` was taken only:
28
+ - unified_merged_code_xp3.jsonl
29
+ - unified_grade_school_math_instructions.jsonl
30
+ - unified_mathqa_flanv2_kojma_cot.jsonl