Updates README.md so it does not have latex content incompatible with hugging face's model cards
Browse files
README.md
CHANGED
@@ -45,9 +45,13 @@ the **ASSIN (Avaliação de Similaridade Semântica e Inferência textual)** cor
|
|
45 |
### Direct Use
|
46 |
|
47 |
This fine-tuned version of [XLM-RoBERTa-base](https://huggingface.co/xlm-roberta-base) performs Natural
|
48 |
-
Language Inference (NLI), which is a text classification task.
|
49 |
|
50 |
-
|
|
|
|
|
|
|
|
|
51 |
|
52 |
**Definition 1.** Given a pair of sentences $(premise, hypothesis)$, let $\hat{f}^{(xlmr\_base)}$ be the fine-tuned models' inference function:
|
53 |
|
@@ -59,12 +63,12 @@ PARAPHRASE, & \text{if $premise$ entails $hypothesis$ and $hypothesis$ entails $
|
|
59 |
NONE & \text{otherwise}
|
60 |
\end{cases}
|
61 |
$$
|
62 |
-
</div>
|
|
|
|
|
63 |
|
64 |
|
65 |
-
The $(premise, hypothesis)$ entailment definition used is the same as the one found in Salvatore's paper [1].
|
66 |
|
67 |
-
Therefore, **this fine-tuned version of [XLM-RoBERTa-base](https://huggingface.co/xlm-roberta-base) classifies pairs of sentences into one of the following classes $ENTAILMENT, PARAPHRASE$ or $NONE$.** using [Definition 1](#assin_function).
|
68 |
|
69 |
<!-- ## Bias, Risks, and Limitations
|
70 |
|
@@ -135,15 +139,13 @@ The model's fine-tuning procedure can be summarized in three major subsequent ta
|
|
135 |
using the *cross-tests* approach described in the [this section](#evaluation), the models' performance were measured using different datasets and metrics.
|
136 |
</ol>
|
137 |
|
138 |
-
More information on the fine-tuning procedure can be found in [@tcc_paper].
|
139 |
-
|
140 |
|
141 |
<!-- ##### Column Renaming
|
142 |
The **Hugging Face**'s ```transformers``` module's ```DataCollator``` used by its ```Trainer``` requires that the ```class label``` column of the collated dataset to be called ```label```. [ASSIN](https://huggingface.co/datasets/assin)'s class label column for each hypothesis/premise pair is called ```entailment_judgement```. Therefore, as the first step of the data preprocessing pipeline the column ```entailment_judgement``` was renamed to ```label``` so that the **Hugging Face**'s ```transformers``` module's ```Trainer``` could be used. -->
|
143 |
|
144 |
#### Hyperparameter Tuning
|
145 |
|
146 |
-
The model's training hyperparameters were chosen according to the following definition:
|
147 |
|
148 |
<div id="hyperparameter_tuning">
|
149 |
|
@@ -152,13 +154,13 @@ The model's training hyperparameters were chosen according to the following defi
|
|
152 |
$$
|
153 |
Hyperparms = \argmax_{hyp}(eval\_acc(\hat{f}^{(xlmr\_base)}_{hyp}, assin\_validation))
|
154 |
$$
|
155 |
-
</div>
|
156 |
|
157 |
The following hyperparameters were tested in order to maximize the evaluation accuracy.
|
158 |
|
159 |
-
- **Number of Training Epochs:**
|
160 |
-
- **Per Device Train Batch Size:**
|
161 |
-
- **Learning Rate:**
|
162 |
|
163 |
|
164 |
The hyperaparemeter tuning experiments were run and tracked using the [Weights & Biases' API](https://docs.wandb.ai/ref/python/public-api/api) and can be found at this [link](https://wandb.ai/gio_projs/assin_xlm_roberta_v5?workspace=user-giogvn).
|
@@ -168,23 +170,22 @@ The hyperaparemeter tuning experiments were run and tracked using the [Weights &
|
|
168 |
|
169 |
The [hyperparameter tuning](#hyperparameter-tuning) performed yelded the following values:
|
170 |
|
171 |
-
- **Number of Training Epochs:**
|
172 |
-
- **Per Device Train Batch Size:**
|
173 |
-
- **Learning Rate:**
|
174 |
|
175 |
## Evaluation
|
176 |
|
177 |
### ASSIN
|
178 |
|
179 |
-
Testing this model in [ASSIN](https://huggingface.co/datasets/assin)'s test split is straightforward.
|
180 |
|
181 |
### ASSIN2
|
182 |
-
|
183 |
-
Given a pair of sentences $(premise, hypothesis)$, $\hat{f}^{(xlmr\_base)}(premise, hypothesis)$ can be equal to $PARAPHRASE, ENTAILMENT$ or $NONE$ as defined in [Definition 1](#assin_function).
|
184 |
|
185 |
[ASSIN2](https://huggingface.co/datasets/assin2)'s test split's class label's column has only two possible values: $ENTAILMENT$ and $NONE$. Therefore, in order to test this model in [ASSIN2](https://huggingface.co/datasets/assin2)'s test split some mapping must be done in order to make the [ASSIN2](https://huggingface.co/datasets/assin2)' class labels compatible with the model's inference function.
|
186 |
|
187 |
-
More information on how such mapping is performed
|
188 |
|
189 |
### Metrics
|
190 |
|
|
|
45 |
### Direct Use
|
46 |
|
47 |
This fine-tuned version of [XLM-RoBERTa-base](https://huggingface.co/xlm-roberta-base) performs Natural
|
48 |
+
Language Inference (NLI), which is a text classification task. Therefore, classifies pairs of sentences in the form *(premise, hypothesis)* into one of the following classes *ENTAILMENT, PARAPHRASE* or *NONE*. Salvatore's definition [1] for *ENTAILEMENT* is assumed to be the same as the one found in [ASSIN](https://huggingface.co/datasets/assin)'s labels.
|
49 |
|
50 |
+
*PARAPHRASE* and *NONE* are not defined in [1].Therefore, it is assumed that in [ASSIN](https://huggingface.co/datasets/assin), given a pair of sentences *(paraphase, hypothesis)*, *hypothesis* is a *PARAPHRASE* of *premise* if *premise* is an *ENTAILMENT* of *hypothesis* and vice-versa. If *(premise, hypothesis)* don't have an *ENTAILMENT* or *PARAPHARSE* relationship, *(premise, hypothesis)* is classified as *NONE* in [ASSIN](https://huggingface.co/datasets/assin).
|
51 |
+
|
52 |
+
|
53 |
+
|
54 |
+
<!-- <div id="assin_function">
|
55 |
|
56 |
**Definition 1.** Given a pair of sentences $(premise, hypothesis)$, let $\hat{f}^{(xlmr\_base)}$ be the fine-tuned models' inference function:
|
57 |
|
|
|
63 |
NONE & \text{otherwise}
|
64 |
\end{cases}
|
65 |
$$
|
66 |
+
</div>
|
67 |
+
|
68 |
+
The (premise, hypothesis)$ entailment definition used is the same as the one found in Salvatore's paper [1].-->
|
69 |
|
70 |
|
|
|
71 |
|
|
|
72 |
|
73 |
<!-- ## Bias, Risks, and Limitations
|
74 |
|
|
|
139 |
using the *cross-tests* approach described in the [this section](#evaluation), the models' performance were measured using different datasets and metrics.
|
140 |
</ol>
|
141 |
|
|
|
|
|
142 |
|
143 |
<!-- ##### Column Renaming
|
144 |
The **Hugging Face**'s ```transformers``` module's ```DataCollator``` used by its ```Trainer``` requires that the ```class label``` column of the collated dataset to be called ```label```. [ASSIN](https://huggingface.co/datasets/assin)'s class label column for each hypothesis/premise pair is called ```entailment_judgement```. Therefore, as the first step of the data preprocessing pipeline the column ```entailment_judgement``` was renamed to ```label``` so that the **Hugging Face**'s ```transformers``` module's ```Trainer``` could be used. -->
|
145 |
|
146 |
#### Hyperparameter Tuning
|
147 |
|
148 |
+
<!-- The model's training hyperparameters were chosen according to the following definition:
|
149 |
|
150 |
<div id="hyperparameter_tuning">
|
151 |
|
|
|
154 |
$$
|
155 |
Hyperparms = \argmax_{hyp}(eval\_acc(\hat{f}^{(xlmr\_base)}_{hyp}, assin\_validation))
|
156 |
$$
|
157 |
+
</div> -->
|
158 |
|
159 |
The following hyperparameters were tested in order to maximize the evaluation accuracy.
|
160 |
|
161 |
+
- **Number of Training Epochs:** (1,2,3)
|
162 |
+
- **Per Device Train Batch Size:** (16,32)
|
163 |
+
- **Learning Rate:** (1e-6, 2e-6,3e-6)
|
164 |
|
165 |
|
166 |
The hyperaparemeter tuning experiments were run and tracked using the [Weights & Biases' API](https://docs.wandb.ai/ref/python/public-api/api) and can be found at this [link](https://wandb.ai/gio_projs/assin_xlm_roberta_v5?workspace=user-giogvn).
|
|
|
170 |
|
171 |
The [hyperparameter tuning](#hyperparameter-tuning) performed yelded the following values:
|
172 |
|
173 |
+
- **Number of Training Epochs:** 3
|
174 |
+
- **Per Device Train Batch Size:** 16
|
175 |
+
- **Learning Rate:** 3e-6
|
176 |
|
177 |
## Evaluation
|
178 |
|
179 |
### ASSIN
|
180 |
|
181 |
+
Testing this model in [ASSIN](https://huggingface.co/datasets/assin)'s test split is straightforward because this model was tested using [ASSIN](https://huggingface.co/datasets/assin)'s training set and therefore can predict the same labels as the ones found in its test set.
|
182 |
|
183 |
### ASSIN2
|
184 |
+
<!-- Given a pair of sentences $(premise, hypothesis)$, $\hat{f}^{(xlmr\_base)}(premise, hypothesis)$ can be equal to $PARAPHRASE, ENTAILMENT$ or $NONE$ as defined in [Definition 1](#assin_function). -->
|
|
|
185 |
|
186 |
[ASSIN2](https://huggingface.co/datasets/assin2)'s test split's class label's column has only two possible values: $ENTAILMENT$ and $NONE$. Therefore, in order to test this model in [ASSIN2](https://huggingface.co/datasets/assin2)'s test split some mapping must be done in order to make the [ASSIN2](https://huggingface.co/datasets/assin2)' class labels compatible with the model's inference function.
|
187 |
|
188 |
+
More information on how such mapping is performed will be available in the [referred paper](#model-sources).
|
189 |
|
190 |
### Metrics
|
191 |
|