File size: 20,400 Bytes
95520dd 4d2a951 95520dd 8e7a912 95520dd 8e7a912 95520dd |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 |
---
license: apache-2.0
datasets:
- mbruton/spanish_srl
- CoNLL-2012
language:
- es
- en
metrics:
- seqeval
library_name: transformers
pipeline_tag: token-classification
---
# Model Card for SpaXLM-R-en for Semantic Role Labeling
This model is fine-tuned on a version of [XLM RoBERTa Base](https://huggingface.co/xlm-roberta-base) which is pre-trained on the SRL task for English, and is one of 24 models introduced as part of [this project](https://github.com/mbruton0426/GalicianSRL).
## Model Details
### Model Description
SpaXLM-R-en for Semantic Role Labeling (SRL) is a transformers model, leveraging XLM-R's extensive pretraining on 100 languages to achieve better SRL predictions for Spanish. This model is additionally pre-trained on the SRL task for English. It was fine-tuned on Spanish with the following objectives:
- Identify up to 16 verbal roots within a sentence.
- Identify available arguments and thematic roles for each verbal root.
Labels are formatted as: r#:tag, where r# links the token to a specific verbal root of index #, and tag identifies the token as the verbal root (root) or an individual argument (arg0/arg1/arg2/arg3/argM) and its thematic role (adv/agt/atr/ben/cau/cot/des/efi/ein/exp/ext/fin/ins/loc/mnr/ori/pat/src/tem/tmp)
- **Developed by:** [Micaella Bruton](mailto:micaellabruton@gmail.com)
- **Model type:** Transformers
- **Language(s) (NLP):** Spanish (es), English (en)
- **License:** Apache 2.0
- **Finetuned from model:** [English pre-trained XLM RoBERTa Base](https://huggingface.co/liaad/srl-en_xlmr-base)
### Model Sources
- **Repository:** [GalicianSRL](https://github.com/mbruton0426/GalicianSRL)
- **Paper:** To be updated
## Uses
This model is intended to be used to develop and improve natural language processing tools for Spanish.
## Bias, Risks, and Limitations
The Spanish training set lacked highly complex sentences and as such, performs much better on sentences of mid- to low-complexity.
## Training Details
### Training Data
This model was pre-trained on the [OntoNotes 5.0 English SRL corpus](http://catalog.ldc.upenn.edu/LDC2013T19).
This model was fine-tuned on the "train" portion of the [SpanishSRL Dataset](https://huggingface.co/datasets/mbruton/spanish_srl) produced as part of this same project.
#### Training Hyperparameters
- **Learning Rate:** 2e-5
- **Batch Size:** 16
- **Weight Decay:** 0.01
- **Early Stopping:** 10 epochs
## Evaluation
#### Testing Data
This model was tested on the "test" portion of the [SpanishSRL Dataset](https://huggingface.co/datasets/mbruton/spanish_srl) produced as part of this same project.
#### Metrics
[seqeval](https://huggingface.co/spaces/evaluate-metric/seqeval) is a Python framework for sequence labeling evaluation. It can evaluate the performance of chunking tasks such as named-entity recognition, part-of-speech tagging, and semantic role labeling.
It supplies scoring both overall and per label type.
Overall:
- `accuracy`: the average [accuracy](https://huggingface.co/metrics/accuracy), on a scale between 0.0 and 1.0.
- `precision`: the average [precision](https://huggingface.co/metrics/precision), on a scale between 0.0 and 1.0.
- `recall`: the average [recall](https://huggingface.co/metrics/recall), on a scale between 0.0 and 1.0.
- `f1`: the average [F1 score](https://huggingface.co/metrics/f1), which is the harmonic mean of the precision and recall. It also has a scale of 0.0 to 1.0.
Per label type:
- `precision`: the average [precision](https://huggingface.co/metrics/precision), on a scale between 0.0 and 1.0.
- `recall`: the average [recall](https://huggingface.co/metrics/recall), on a scale between 0.0 and 1.0.
- `f1`: the average [F1 score](https://huggingface.co/metrics/f1), on a scale between 0.0 and 1.0.
### Results
| Label | Precision | Recall | f1-score | Support |
| :----------: | :-------: | :----: | :------: | :-----: |
| 0:arg0:agt | 0.93 | 0.94 | 0.93 | 867 |
| 0:arg0:cau | 0.76 | 0.67 | 0.71 | 57 |
| 0:arg0:src | 0.00 | 0.00 | 0.00 | 1 |
| 0:arg1:ext | 0.00 | 0.00 | 0.00 | 3 |
| 0:arg1:pat | 0.88 | 0.90 | 0.89 | 536 |
| 0:arg1:tem | 0.88 | 0.89 | 0.89 | 589 |
| 0:arg2:atr | 0.85 | 0.90 | 0.87 | 278 |
| 0:arg2:ben | 0.81 | 0.90 | 0.85 | 78 |
| 0:arg2:efi | 0.75 | 0.43 | 0.55 | 7 |
| 0:arg2:exp | 0.60 | 0.50 | 0.55 | 6 |
| 0:arg2:ext | 0.56 | 0.60 | 0.58 | 15 |
| 0:arg2:loc | 0.66 | 0.40 | 0.50 | 57 |
| 0:arg3:ben | 0.50 | 0.20 | 0.29 | 5 |
| 0:arg3:ein | 0.33 | 1.00 | 0.50 | 1 |
| 0:arg3:fin | 0.67 | 1.00 | 0.80 | 2 |
| 0:arg3:ori | 0.50 | 0.50 | 0.50 | 10 |
| 0:arg4:des | 0.52 | 0.88 | 0.65 | 16 |
| 0:arg4:efi | 0.50 | 0.40 | 0.44 | 5 |
| 0:argM:adv | 0.65 | 0.59 | 0.62 | 268 |
| 0:argM:atr | 0.58 | 0.46 | 0.51 | 24 |
| 0:argM:cau | 0.70 | 0.63 | 0.67 | 41 |
| 0:argM:ext | 0.00 | 0.00 | 0.00 | 5 |
| 0:argM:fin | 0.80 | 0.80 | 0.80 | 46 |
| 0:argM:loc | 0.71 | 0.83 | 0.77 | 186 |
| 0:argM:mnr | 0.72 | 0.55 | 0.62 | 66 |
| 0:argM:tmp | 0.87 | 0.87 | 0.87 | 411 |
| 0:root | 0.99 | 0.99 | 0.99 | 1662 |
| 1:arg0:agt | 0.93 | 0.91 | 0.92 | 564 |
| 1:arg0:cau | 0.73 | 0.80 | 0.76 | 44 |
| 1:arg1:ext | 0.00 | 0.00 | 0.00 | 2 |
| 1:arg1:pat | 0.88 | 0.90 | 0.89 | 482 |
| 1:arg1:tem | 0.90 | 0.90 | 0.90 | 390 |
| 1:arg2:atr | 0.86 | 0.90 | 0.88 | 197 |
| 1:arg2:ben | 0.80 | 0.86 | 0.83 | 66 |
| 1:arg2:efi | 0.60 | 0.50 | 0.55 | 6 |
| 1:arg2:ext | 0.57 | 0.57 | 0.57 | 7 |
| 1:arg2:ins | 0.00 | 0.00 | 0.00 | 1 |
| 1:arg2:loc | 0.67 | 0.64 | 0.65 | 44 |
| 1:arg3:ben | 0.00 | 0.00 | 0.00 | 2 |
| 1:arg3:ein | 0.00 | 0.00 | 0.00 | 3 |
| 1:arg3:fin | 1.00 | 1.00 | 1.00 | 2 |
| 1:arg3:ori | 0.14 | 0.50 | 0.22 | 2 |
| 1:arg4:des | 0.50 | 0.90 | 0.64 | 10 |
| 1:arg4:efi | 1.00 | 0.50 | 0.67 | 2 |
| 1:argM:adv | 0.59 | 0.61 | 0.60 | 220 |
| 1:argM:atr | 0.63 | 0.63 | 0.63 | 19 |
| 1:argM:cau | 0.78 | 0.60 | 0.68 | 35 |
| 1:argM:ext | 0.00 | 0.00 | 0.00 | 7 |
| 1:argM:fin | 0.55 | 0.63 | 0.59 | 38 |
| 1:argM:loc | 0.74 | 0.79 | 0.76 | 156 |
| 1:argM:mnr | 0.60 | 0.27 | 0.37 | 44 |
| 1:argM:tmp | 0.81 | 0.82 | 0.82 | 247 |
| 1:root | 0.96 | 0.96 | 0.96 | 1323 |
| 2:arg0:agt | 0.88 | 0.89 | 0.88 | 336 |
| 2:arg0:cau | 0.76 | 0.80 | 0.78 | 35 |
| 2:arg0:exp | 0.00 | 0.00 | 0.00 | 1 |
| 2:arg0:src | 0.00 | 0.00 | 0.00 | 1 |
| 2:arg1:pat | 0.86 | 0.86 | 0.86 | 333 |
| 2:arg1:tem | 0.86 | 0.85 | 0.86 | 291 |
| 2:arg2:atr | 0.87 | 0.88 | 0.87 | 124 |
| 2:arg2:ben | 0.66 | 0.86 | 0.75 | 43 |
| 2:arg2:efi | 0.86 | 0.67 | 0.75 | 9 |
| 2:arg2:ext | 0.60 | 0.60 | 0.60 | 5 |
| 2:arg2:ins | 0.00 | 0.00 | 0.00 | 1 |
| 2:arg2:loc | 0.47 | 0.52 | 0.49 | 27 |
| 2:arg3:ben | 1.00 | 0.25 | 0.40 | 4 |
| 2:arg3:ein | 0.00 | 0.00 | 0.00 | 1 |
| 2:arg3:ori | 0.38 | 1.00 | 0.55 | 3 |
| 2:arg4:des | 0.56 | 0.88 | 0.68 | 16 |
| 2:arg4:efi | 0.00 | 0.00 | 0.00 | 6 |
| 2:argM:adv | 0.60 | 0.54 | 0.57 | 176 |
| 2:argM:atr | 0.56 | 0.33 | 0.42 | 15 |
| 2:argM:cau | 0.53 | 0.53 | 0.53 | 17 |
| 2:argM:ext | 0.00 | 0.00 | 0.00 | 4 |
| 2:argM:fin | 0.71 | 0.69 | 0.70 | 36 |
| 2:argM:ins | 0.00 | 0.00 | 0.00 | 1 |
| 2:argM:loc | 0.69 | 0.78 | 0.73 | 117 |
| 2:argM:mnr | 0.45 | 0.26 | 0.33 | 35 |
| 2:argM:tmp | 0.74 | 0.79 | 0.77 | 161 |
| 2:root | 0.93 | 0.94 | 0.94 | 913 |
| 3:arg0:agt | 0.85 | 0.89 | 0.87 | 227 |
| 3:arg0:cau | 0.69 | 0.64 | 0.67 | 14 |
| 3:arg1:pat | 0.84 | 0.90 | 0.87 | 199 |
| 3:arg1:tem | 0.82 | 0.79 | 0.81 | 160 |
| 3:arg2:atr | 0.80 | 0.81 | 0.81 | 79 |
| 3:arg2:ben | 0.81 | 0.93 | 0.86 | 27 |
| 3:arg2:efi | 0.00 | 0.00 | 0.00 | 1 |
| 3:arg2:ext | 0.00 | 0.00 | 0.00 | 3 |
| 3:arg2:loc | 0.53 | 0.43 | 0.47 | 21 |
| 3:arg3:ben | 0.00 | 0.00 | 0.00 | 3 |
| 3:arg3:ein | 0.00 | 0.00 | 0.00 | 2 |
| 3:arg3:ori | 0.25 | 0.33 | 0.29 | 3 |
| 3:arg4:des | 0.41 | 1.00 | 0.58 | 7 |
| 3:arg4:efi | 0.00 | 0.00 | 0.00 | 5 |
| 3:argM:adv | 0.54 | 0.47 | 0.50 | 98 |
| 3:argM:atr | 0.67 | 0.29 | 0.40 | 7 |
| 3:argM:cau | 0.50 | 0.46 | 0.48 | 13 |
| 3:argM:ext | 0.00 | 0.00 | 0.00 | 1 |
| 3:argM:fin | 0.60 | 0.40 | 0.48 | 15 |
| 3:argM:loc | 0.74 | 0.67 | 0.70 | 69 |
| 3:argM:mnr | 0.23 | 0.27 | 0.25 | 11 |
| 3:argM:tmp | 0.85 | 0.79 | 0.82 | 92 |
| 3:root | 0.92 | 0.93 | 0.93 | 569 |
| 4:arg0:agt | 0.88 | 0.86 | 0.87 | 119 |
| 4:arg0:cau | 1.00 | 0.67 | 0.80 | 6 |
| 4:arg1:pat | 0.77 | 0.78 | 0.78 | 87 |
| 4:arg1:tem | 0.85 | 0.75 | 0.80 | 109 |
| 4:arg2:atr | 0.74 | 0.70 | 0.72 | 53 |
| 4:arg2:ben | 0.64 | 0.82 | 0.72 | 11 |
| 4:arg2:ext | 0.00 | 0.00 | 0.00 | 1 |
| 4:arg2:loc | 1.00 | 0.64 | 0.78 | 11 |
| 4:arg3:ein | 0.00 | 0.00 | 0.00 | 1 |
| 4:arg3:ori | 0.00 | 0.00 | 0.00 | 1 |
| 4:arg4:des | 0.67 | 0.60 | 0.63 | 10 |
| 4:arg4:efi | 0.00 | 0.00 | 0.00 | 1 |
| 4:argM:adv | 0.48 | 0.50 | 0.49 | 50 |
| 4:argM:atr | 0.00 | 0.00 | 0.00 | 4 |
| 4:argM:cau | 0.00 | 0.00 | 0.00 | 3 |
| 4:argM:ext | 0.00 | 0.00 | 0.00 | 1 |
| 4:argM:fin | 0.67 | 0.73 | 0.70 | 11 |
| 4:argM:loc | 0.60 | 0.75 | 0.67 | 24 |
| 4:argM:mnr | 0.50 | 0.19 | 0.27 | 16 |
| 4:argM:tmp | 0.86 | 0.69 | 0.77 | 52 |
| 4:root | 0.90 | 0.87 | 0.88 | 322 |
| 5:arg0:agt | 0.77 | 0.69 | 0.73 | 72 |
| 5:arg0:cau | 1.00 | 0.20 | 0.33 | 5 |
| 5:arg1:pat | 0.65 | 0.69 | 0.67 | 71 |
| 5:arg1:tem | 0.66 | 0.51 | 0.58 | 41 |
| 5:arg2:atr | 0.39 | 0.57 | 0.46 | 21 |
| 5:arg2:ben | 0.45 | 0.83 | 0.59 | 6 |
| 5:arg2:efi | 0.00 | 0.00 | 0.00 | 1 |
| 5:arg2:ext | 0.00 | 0.00 | 0.00 | 1 |
| 5:arg2:loc | 0.00 | 0.00 | 0.00 | 1 |
| 5:arg3:ein | 0.00 | 0.00 | 0.00 | 1 |
| 5:arg4:des | 0.00 | 0.00 | 0.00 | 1 |
| 5:arg4:efi | 0.00 | 0.00 | 0.00 | 1 |
| 5:argM:adv | 0.41 | 0.35 | 0.38 | 26 |
| 5:argM:cau | 0.00 | 0.00 | 0.00 | 3 |
| 5:argM:fin | 0.33 | 0.60 | 0.43 | 5 |
| 5:argM:loc | 0.58 | 0.52 | 0.55 | 21 |
| 5:argM:mnr | 0.00 | 0.00 | 0.00 | 7 |
| 5:argM:tmp | 0.65 | 0.80 | 0.72 | 30 |
| 5:root | 0.77 | 0.66 | 0.71 | 173 |
| 6:arg0:agt | 0.56 | 0.65 | 0.60 | 34 |
| 6:arg0:cau | 0.00 | 0.00 | 0.00 | 1 |
| 6:arg1:loc | 0.00 | 0.00 | 0.00 | 1 |
| 6:arg1:pat | 0.45 | 0.79 | 0.57 | 28 |
| 6:arg1:tem | 0.27 | 0.25 | 0.26 | 16 |
| 6:arg2:atr | 0.46 | 0.46 | 0.46 | 13 |
| 6:arg2:ben | 0.33 | 0.60 | 0.43 | 5 |
| 6:arg2:loc | 0.00 | 0.00 | 0.00 | 1 |
| 6:arg3:ben | 0.00 | 0.00 | 0.00 | 1 |
| 6:argM:adv | 0.27 | 0.70 | 0.39 | 10 |
| 6:argM:atr | 0.00 | 0.00 | 0.00 | 2 |
| 6:argM:cau | 0.00 | 0.00 | 0.00 | 1 |
| 6:argM:fin | 0.00 | 0.00 | 0.00 | 2 |
| 6:argM:loc | 0.25 | 0.57 | 0.35 | 7 |
| 6:argM:mnr | 0.00 | 0.00 | 0.00 | 5 |
| 6:argM:tmp | 0.46 | 0.86 | 0.60 | 7 |
| 6:root | 0.46 | 0.55 | 0.50 | 82 |
| 7:arg0:agt | 0.33 | 0.88 | 0.48 | 17 |
| 7:arg1:pat | 0.41 | 0.71 | 0.52 | 17 |
| 7:arg1:tem | 0.50 | 0.80 | 0.62 | 15 |
| 7:arg2:atr | 0.50 | 0.60 | 0.55 | 15 |
| 7:arg2:ben | 0.00 | 0.00 | 0.00 | 7 |
| 7:arg2:loc | 0.00 | 0.00 | 0.00 | 1 |
| 7:arg3:ori | 0.00 | 0.00 | 0.00 | 1 |
| 7:arg4:des | 1.00 | 1.00 | 1.00 | 1 |
| 7:argM:adv | 0.17 | 0.40 | 0.24 | 5 |
| 7:argM:atr | 0.00 | 0.00 | 0.00 | 1 |
| 7:argM:fin | 0.00 | 0.00 | 0.00 | 1 |
| 7:argM:loc | 0.00 | 0.00 | 0.00 | 3 |
| 7:argM:tmp | 0.33 | 0.33 | 0.33 | 6 |
| 7:root | 0.43 | 0.78 | 0.55 | 45 |
| 8:arg0:agt | 0.00 | 0.00 | 0.00 | 8 |
| 8:arg0:cau | 0.00 | 0.00 | 0.00 | 1 |
| 8:arg1:pat | 0.00 | 0.00 | 0.00 | 4 |
| 8:arg1:tem | 0.24 | 0.44 | 0.31 | 9 |
| 8:arg2:atr | 0.10 | 0.25 | 0.14 | 4 |
| 8:arg2:ben | 0.00 | 0.00 | 0.00 | 0 |
| 8:arg2:ext | 0.00 | 0.00 | 0.00 | 1 |
| 8:arg2:loc | 0.00 | 0.00 | 0.00 | 2 |
| 8:arg3:ori | 0.00 | 0.00 | 0.00 | 1 |
| 8:argM:adv | 0.13 | 0.25 | 0.17 | 8 |
| 8:argM:ext | 0.00 | 0.00 | 0.00 | 1 |
| 8:argM:fin | 0.00 | 0.00 | 0.00 | 1 |
| 8:argM:loc | 0.00 | 0.00 | 0.00 | 4 |
| 8:argM:mnr | 0.00 | 0.00 | 0.00 | 1 |
| 8:argM:tmp | 0.00 | 0.00 | 0.00 | 1 |
| 8:root | 0.38 | 0.72 | 0.50 | 25 |
| 9:arg0:agt | 0.00 | 0.00 | 0.00 | 6 |
| 9:arg0:cau | 0.00 | 0.00 | 0.00 | 1 |
| 9:arg1:pat | 0.00 | 0.00 | 0.00 | 4 |
| 9:arg1:tem | 0.00 | 0.00 | 0.00 | 5 |
| 9:arg2:atr | 0.00 | 0.00 | 0.00 | 3 |
| 9:arg2:ben | 0.00 | 0.00 | 0.00 | 1 |
| 9:argM:adv | 0.00 | 0.00 | 0.00 | 6 |
| 9:argM:cau | 0.00 | 0.00 | 0.00 | 1 |
| 9:argM:fin | 0.00 | 0.00 | 0.00 | 2 |
| 9:argM:loc | 0.00 | 0.00 | 0.00 | 2 |
| 9:argM:tmp | 0.00 | 0.00 | 0.00 | 1 |
| 9:root | 0.09 | 0.18 | 0.12 | 17 |
| 10:arg0:agt | 0.00 | 0.00 | 0.00 | 3 |
| 10:arg1:pat | 0.00 | 0.00 | 0.00 | 5 |
| 10:arg1:tem | 0.00 | 0.00 | 0.00 | 3 |
| 10:arg2:atr | 0.00 | 0.00 | 0.00 | 1 |
| 10:arg2:ben | 0.00 | 0.00 | 0.00 | 2 |
| 10:argM:adv | 0.00 | 0.00 | 0.00 | 3 |
| 10:argM:fin | 0.00 | 0.00 | 0.00 | 1 |
| 10:argM:tmp | 0.00 | 0.00 | 0.00 | 1 |
| 10:root | 0.00 | 0.00 | 0.00 | 12 |
| 11:arg0:agt | 0.00 | 0.00 | 0.00 | 1 |
| 11:arg0:cau | 0.00 | 0.00 | 0.00 | 1 |
| 11:arg1:pat | 0.00 | 0.00 | 0.00 | 2 |
| 11:arg1:tem | 0.00 | 0.00 | 0.00 | 4 |
| 11:arg2:atr | 0.00 | 0.00 | 0.00 | 3 |
| 11:arg2:ben | 0.00 | 0.00 | 0.00 | 1 |
| 11:argM:adv | 0.00 | 0.00 | 0.00 | 4 |
| 11:argM:loc | 0.00 | 0.00 | 0.00 | 1 |
| 11:argM:tmp | 0.00 | 0.00 | 0.00 | 1 |
| 11:root | 0.00 | 0.00 | 0.00 | 9 |
| 12:arg0:agt | 0.00 | 0.00 | 0.00 | 3 |
| 12:arg1:pat | 0.00 | 0.00 | 0.00 | 1 |
| 12:arg1:tem | 0.00 | 0.00 | 0.00 | 2 |
| 12:arg2:atr | 0.00 | 0.00 | 0.00 | 2 |
| 12:argM:adv | 0.00 | 0.00 | 0.00 | 1 |
| 12:argM:cau | 0.00 | 0.00 | 0.00 | 1 |
| 12:argM:tmp | 0.00 | 0.00 | 0.00 | 3 |
| 12:root | 0.00 | 0.00 | 0.00 | 7 |
| 13:arg0:cau | 0.00 | 0.00 | 0.00 | 1 |
| 13:arg1:tem | 0.00 | 0.00 | 0.00 | 1 |
| 13:arg2:atr | 0.00 | 0.00 | 0.00 | 1 |
| 13:argM:adv | 0.00 | 0.00 | 0.00 | 1 |
| 13:argM:atr | 0.00 | 0.00 | 0.00 | 1 |
| 13:argM:loc | 0.00 | 0.00 | 0.00 | 1 |
| 13:root | 0.00 | 0.00 | 0.00 | 4 |
| 14:arg1:pat | 0.00 | 0.00 | 0.00 | 1 |
| 14:arg2:ben | 0.00 | 0.00 | 0.00 | 1 |
| 14:argM:mnr | 0.00 | 0.00 | 0.00 | 1 |
| 14:root | 0.00 | 0.00 | 0.00 | 2 |
| micro avg | 0.84 | 0.84 | 0.84 | 15436 |
| macro avg | 0.34 | 0.36 | 0.34 | 15436 |
| weighted avg | 0.83 | 0.84 | 0.83 | 15436 |
| tot root avg | 0.46 | 0.51 | 0.47 | 5165 |
| tot arg0:agt avg | 0.47 | 0.52 | 0.48 | 2257 |
| tot arg0:cau avg | 0.45 | 0.34 | 0.37 | 166 |
| tot arg0:exp avg | 0.00 | 0.00 | 0.00 | 1 |
| tot arg0:src avg | 0.00 | 0.00 | 0.00 | 2 |
| tot arg0 | 0.41 | 0.39 | 0.38 | 2426 |
| tot arg1:ext avg | 0.00 | 0.00 | 0.00 | 5 |
| tot arg1:loc avg | 0.00 | 0.00 | 0.00 | 1 |
| tot arg1:pat avg | 0.41 | 0.47 | 0.43 | 1770 |
| tot arg1:tem avg | 0.43 | 0.44 | 0.43 | 1635 |
| tot arg1 | 0.38 | 0.41 | 0.39 | 3411 |
| tot arg2:atr avg | 0.40 | 0.43 | 0.41 | 794 |
| tot arg2:ben avg | 0.35 | 0.45 | 0.39 | 248 |
| tot arg2:efi avg | 0.44 | 0.32 | 0.37 | 24 |
| tot arg2:exp avg | 0.60 | 0.50 | 0.55 | 6 |
| tot arg2:ext avg | 0.25 | 0.25 | 0.25 | 33 |
| tot arg2:ins avg | 0.00 | 0.00 | 0.00 | 2 |
| tot arg2:loc avg | 0.37 | 0.29 | 0.32 | 165 |
| tot arg2 | 0.35 | 0.36 | 0.35 | 1272 |
| tot arg3:ben avg | 0.30 | 0.09 | 0.14 | 15 |
| tot arg3:ein avg | 0.06 | 0.17 | 0.08 | 9 |
| tot arg3:fin avg | 0.84 | 1.00 | 0.90 | 4 |
| tot arg3:ori avg | 0.18 | 0.33 | 0.22 | 21 |
| tot arg3 | 0.24 | 0.29 | 0.23 | 49 |
| tot arg4:des avg | 0.52 | 0.75 | 0.60 | 61 |
| tot arg4:efi avg | 0.25 | 0.15 | 0.19 | 20 |
| tot arg4 | 0.40 | 0.47 | 0.41 | 81 |
| tot argM:adv avg | 0.27 | 0.32 | 0.28 | 876 |
| tot argM:atr avg | 0.31 | 0.21 | 0.25 | 73 |
| tot argM:cau avg | 0.28 | 0.25 | 0.26 | 115 |
| tot argM:ext avg | 0.00 | 0.00 | 0.00 | 19 |
| tot argM:fin avg | 0.33 | 0.35 | 0.34 | 158 |
| tot argM:ins avg | 0.00 | 0.00 | 0.00 | 1 |
| tot argM:loc avg | 0.36 | 0.41 | 0.38 | 591 |
| tot argM:mnr avg | 0.28 | 0.17 | 0.20 | 186 |
| tot argM:tmp avg | 0.43 | 0.46 | 0.44 | 1013 |
| tot argM | 0.30 | 0.30 | 0.29 | 3032 |
| tot r0 avg | 0.62 | 0.62 | 0.61 | 5242 |
| tot r1 avg | 0.59 | 0.58 | 0.57 | 3913 |
| tot r2 avg | 0.52 | 0.52 | 0.50 | 2711 |
| tot r3 avg | 0.48 | 0.48 | 0.47 | 1626 |
| tot r4 avg | 0.48 | 0.43 | 0.45 | 843 |
| tot r5 avg | 0.35 | 0.34 | 0.32 | 487 |
| tot r6 avg | 0.21 | 0.32 | 0.24 | 216 |
| tot r7 avg | 0.26 | 0.39 | 0.31 | 135 |
| tot r8 avg | 0.05 | 0.10 | 0.07 | 71 |
| tot r9 avg | 0.01 | 0.02 | 0.01 | 49 |
| tot r10 avg | 0.00 | 0.00 | 0.00 | 31 |
| tot r11 avg | 0.00 | 0.00 | 0.00 | 27 |
| tot r12 avg | 0.00 | 0.00 | 0.00 | 20 |
| tot r13 avg | 0.00 | 0.00 | 0.00 | 10 |
| tot r14 avg | 0.00 | 0.00 | 0.00 | 5 |
## Citation
**BibTeX:**
```
@inproceedings{bruton-beloucif-2023-bertie,
title = "{BERT}ie Bott{'}s Every Flavor Labels: A Tasty Introduction to Semantic Role Labeling for {G}alician",
author = "Bruton, Micaella and
Beloucif, Meriem",
editor = "Bouamor, Houda and
Pino, Juan and
Bali, Kalika",
booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
month = dec,
year = "2023",
address = "Singapore",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.emnlp-main.671",
doi = "10.18653/v1/2023.emnlp-main.671",
pages = "10892--10902",
abstract = "In this paper, we leverage existing corpora, WordNet, and dependency parsing to build the first Galician dataset for training semantic role labeling systems in an effort to expand available NLP resources. Additionally, we introduce verb indexing, a new pre-processing method, which helps increase the performance when semantically parsing highly-complex sentences. We use transfer-learning to test both the resource and the verb indexing method. Our results show that the effects of verb indexing were amplified in scenarios where the model was both pre-trained and fine-tuned on datasets utilizing the method, but improvements are also noticeable when only used during fine-tuning. The best-performing Galician SRL model achieved an f1 score of 0.74, introducing a baseline for future Galician SRL systems. We also tested our method on Spanish where we achieved an f1 score of 0.83, outperforming the baseline set by the 2009 CoNLL Shared Task by 0.025 showing the merits of our verb indexing method for pre-processing.",
}
``` |