5roop commited on
Commit
c877f97
1 Parent(s): 4c01bb2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -23,14 +23,14 @@ inference: false
23
 
24
  This model is based on [xlm-r-parla](https://huggingface.co/classla/xlm-r-parla), an XLM-R-large model additionally pre-trained on parliamentary proceedings. The model was fine-tuned on the [ParlaSent dataset](http://hdl.handle.net/11356/1868), a manually annotated selection of sentences of parliamentary proceedings from Bosnia and Herzegovina, Croatia, Czechia, Serbia, Slovakia, Slovenia, and the United Kingdom.
25
 
26
- Both the additionally pre-trained model, as the training dataset are results of the [ParlaMint project](https://www.clarin.eu/parlamint). The details on the models and the dataset are described in our [paper](https://arxiv.org/abs/2309.09783):
27
 
28
  ```latex
29
  @article{Mochtak_Rupnik_Ljubešić_2023, title={The ParlaSent multilingual training dataset for sentiment identification in parliamentary proceedings}, rights={All rights reserved}, url={http://arxiv.org/abs/2309.09783}, abstractNote={Sentiments inherently drive politics. How we receive and process information plays an essential role in political decision-making, shaping our judgment with strategic consequences both on the level of legislators and the masses. If sentiment plays such an important role in politics, how can we study and measure it systematically? The paper presents a new dataset of sentiment-annotated sentences, which are used in a series of experiments focused on training a robust sentiment classifier for parliamentary proceedings. The paper also introduces the first domain-specific LLM for political science applications additionally pre-trained on 1.72 billion domain-specific words from proceedings of 27 European parliaments. We present experiments demonstrating how the additional pre-training of LLM on parliamentary data can significantly improve the model downstream performance on the domain-specific tasks, in our case, sentiment detection in parliamentary proceedings. We further show that multilingual models perform very well on unseen languages and that additional data from other languages significantly improves the target parliament’s results. The paper makes an important contribution to multiple domains of social sciences and bridges them with computer science and computational linguistics. Lastly, it sets up a more robust approach to sentiment analysis of political texts in general, which allows scholars to study political sentiment from a comparative perspective using standardized tools and techniques.}, note={arXiv:2309.09783 [cs]}, number={arXiv:2309.09783}, publisher={arXiv}, author={Mochtak, Michal and Rupnik, Peter and Ljubešić, Nikola}, year={2023}, month={Sep}, language={en} }
30
  ```
31
  ## Annotation schema
32
 
33
- The discrete labels, present in the original dataset, were mapped to integers as follows:
34
 
35
  ```
36
  "Negative": 0.0,
@@ -44,7 +44,7 @@ The model was then fine-tuned on numeric labels and set up as a regressor.
44
 
45
  ## Finetuning procedure
46
 
47
- The fine-tuning procedure is described in the pending paper. Presumed optimal hyperparameters used are
48
  ```
49
  num_train_epochs=4,
50
  train_batch_size=32,
 
23
 
24
  This model is based on [xlm-r-parla](https://huggingface.co/classla/xlm-r-parla), an XLM-R-large model additionally pre-trained on parliamentary proceedings. The model was fine-tuned on the [ParlaSent dataset](http://hdl.handle.net/11356/1868), a manually annotated selection of sentences of parliamentary proceedings from Bosnia and Herzegovina, Croatia, Czechia, Serbia, Slovakia, Slovenia, and the United Kingdom.
25
 
26
+ Both the additionally pre-trained model, as well as [the training dataset](http://hdl.handle.net/11356/1868) are results of the [ParlaMint project](https://www.clarin.eu/parlamint). The details on the models and the dataset are described in our [paper](https://arxiv.org/abs/2309.09783):
27
 
28
  ```latex
29
  @article{Mochtak_Rupnik_Ljubešić_2023, title={The ParlaSent multilingual training dataset for sentiment identification in parliamentary proceedings}, rights={All rights reserved}, url={http://arxiv.org/abs/2309.09783}, abstractNote={Sentiments inherently drive politics. How we receive and process information plays an essential role in political decision-making, shaping our judgment with strategic consequences both on the level of legislators and the masses. If sentiment plays such an important role in politics, how can we study and measure it systematically? The paper presents a new dataset of sentiment-annotated sentences, which are used in a series of experiments focused on training a robust sentiment classifier for parliamentary proceedings. The paper also introduces the first domain-specific LLM for political science applications additionally pre-trained on 1.72 billion domain-specific words from proceedings of 27 European parliaments. We present experiments demonstrating how the additional pre-training of LLM on parliamentary data can significantly improve the model downstream performance on the domain-specific tasks, in our case, sentiment detection in parliamentary proceedings. We further show that multilingual models perform very well on unseen languages and that additional data from other languages significantly improves the target parliament’s results. The paper makes an important contribution to multiple domains of social sciences and bridges them with computer science and computational linguistics. Lastly, it sets up a more robust approach to sentiment analysis of political texts in general, which allows scholars to study political sentiment from a comparative perspective using standardized tools and techniques.}, note={arXiv:2309.09783 [cs]}, number={arXiv:2309.09783}, publisher={arXiv}, author={Mochtak, Michal and Rupnik, Peter and Ljubešić, Nikola}, year={2023}, month={Sep}, language={en} }
30
  ```
31
  ## Annotation schema
32
 
33
+ The discrete labels, present in the [original dataset](http://hdl.handle.net/11356/1868), were mapped to integers as follows:
34
 
35
  ```
36
  "Negative": 0.0,
 
44
 
45
  ## Finetuning procedure
46
 
47
+ The fine-tuning procedure is described in the paper, cited above. Presumed optimal hyperparameters used are
48
  ```
49
  num_train_epochs=4,
50
  train_batch_size=32,