abritez commited on
Commit
1f983aa
·
verified ·
1 Parent(s): 4aebd0d

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +137 -0
README.md ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ datasets:
3
+ - piuba-bigdata/articles_and_comments
4
+ language:
5
+ - es
6
+ ---
7
+ # Model Card for Model ID
8
+
9
+ <!-- Provide a quick summary of what the model is/does. -->
10
+
11
+ This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).
12
+
13
+ ## Model Details
14
+
15
+ ### Model Description
16
+
17
+ - **Developed by:** A. Britez
18
+ - **Model type:** RoBERTa
19
+ - **Language(s) (NLP):** ES - Rioplatense Spanish
20
+
21
+ ### Model Sources [optional]
22
+
23
+ - **Repository:** https://github.com/abrtz/baby-lm
24
+
25
+ ## Uses
26
+
27
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
28
+
29
+ ### Direct Use
30
+
31
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
32
+
33
+ [More Information Needed]
34
+
35
+ ## Bias, Risks, and Limitations
36
+
37
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
38
+
39
+ [More Information Needed]
40
+
41
+ ### Recommendations
42
+
43
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
44
+
45
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
46
+
47
+ ## How to Get Started with the Model
48
+
49
+ Use the code below to get started with the model.
50
+
51
+ [More Information Needed]
52
+
53
+ ## Training Details
54
+
55
+ ### Training Data
56
+
57
+ datasets:
58
+ - piuba-bigdata/articles_and_comments
59
+
60
+ ### Training Procedure
61
+
62
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
63
+
64
+ #### Preprocessing [optional]
65
+
66
+ [More Information Needed]
67
+
68
+
69
+ #### Training Hyperparameters
70
+
71
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
72
+
73
+ #### Speeds, Sizes, Times [optional]
74
+
75
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
76
+
77
+ [More Information Needed]
78
+
79
+ ## Evaluation
80
+
81
+ <!-- This section describes the evaluation protocols and provides the results. -->
82
+
83
+ ### Testing Data, Factors & Metrics
84
+
85
+ #### Testing Data
86
+
87
+ <!-- This should link to a Dataset Card if possible. -->
88
+
89
+ [More Information Needed]
90
+
91
+ #### Factors
92
+
93
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
94
+
95
+ [More Information Needed]
96
+
97
+ #### Metrics
98
+
99
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ### Results
104
+
105
+ [More Information Needed]
106
+
107
+ #### Summary
108
+
109
+
110
+
111
+ ## Model Examination [optional]
112
+
113
+ <!-- Relevant interpretability work for the model goes here -->
114
+
115
+ [More Information Needed]
116
+
117
+ ## Environmental Impact
118
+
119
+ - **Hardware Type:** A100-SXM4-40GB
120
+ - **Hours used:** 1.2 hours
121
+ - **Cloud Provider:** Google Cloud Platform
122
+ - **Compute Region:** europe-west4
123
+ - **Carbon Emitted:** Total emissions are estimated to be 0.17 kgCO$_2$eq of which 100 percents were directly offset by the cloud provider.
124
+
125
+ Estimations were conducted using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
126
+
127
+ ## Technical Specifications [optional]
128
+
129
+ ### Model Architecture and Objective
130
+
131
+ RoBERTa for MLM
132
+
133
+
134
+ ## Glossary [optional]
135
+
136
+
137
+