gsgoncalves commited on
Commit
4d1c666
1 Parent(s): 0712d03

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +82 -0
README.md CHANGED
@@ -1,3 +1,85 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ datasets:
4
+ - race
5
+ language:
6
+ - en
7
+ tags:
8
+ - text classification
9
+ - multiple-choice
10
  ---
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+ This model was finetuned on RACE for multiple choice (text classification). The initial model used was distilbert-uncased-base https://huggingface.co/distilbert-uncased-base
16
+
17
+ The model was trained using the code from https://github.com/zphang/lrqa. Please refer to and cite the authors.
18
+
19
+ # Model Details
20
+
21
+ - **Initial model:** distilbert-uncased-base
22
+ - **LR:** 1e-5
23
+ - **Epochs:** 3
24
+ - **Warmup Ratio:** 0.1 (10%)
25
+ - **Batch Size:** 16
26
+ - **Max Seq Len:** 512
27
+
28
+ ## Model Description
29
+
30
+ <!-- Provide a longer summary of what this model is. -->
31
+ - **Model type:** [DistilBERT]
32
+ - **Language(s) (NLP):** [English]
33
+ - **License:** [Apache-2.0]
34
+ - **Finetuned from model [optional]:** [distilbert-uncased-base]
35
+
36
+ ## Model Sources [optional]
37
+
38
+ <!-- Provide the basic links for the model. -->
39
+
40
+ - **Repository:** [https://github.com/zphang/lrqa]
41
+ - **Dataset:** [https://huggingface.co/datasets/race]
42
+
43
+ # Bias, Risks, and Limitations
44
+
45
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
46
+
47
+ [More Information Needed]
48
+
49
+ ## How to Get Started with the Model
50
+
51
+ Use the code below to get started with the model.
52
+
53
+ [More Information Needed]
54
+
55
+ # Training Details
56
+
57
+ ## Training Data
58
+
59
+ <!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
60
+
61
+ [More Information Needed]
62
+
63
+
64
+ # Model Examination [optional]
65
+
66
+ <!-- Relevant interpretability work for the model goes here -->
67
+
68
+ [More Information Needed]
69
+
70
+ # Environmental Impact
71
+
72
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
73
+
74
+ - **Hardware Type:** A100 - 40GB
75
+ - **Hours used:** 4
76
+ - **Cloud Provider:** Private
77
+ - **Compute Region:** Portugal
78
+ - **Carbon Emitted:** 0.18 kgCO2
79
+
80
+ Experiments were conducted using a private infrastructure, which has a carbon efficiency of 0.178 kgCO$_2$eq/kWh. A cumulative of 4 hours of computation was performed on hardware of type A100 PCIe 40/80GB (TDP of 250W).
81
+ Total emissions are estimated to be 0.18 kgCO$_2$eq of which 0 percent were directly offset.
82
+ Estimations were conducted using the \href{https://mlco2.github.io/impact#compute}{MachineLearning Impact calculator} presented in \cite{lacoste2019quantifying}.
83
+
84
+
85
+