feralvam commited on
Commit
22e8bfc
1 Parent(s): adf8db0

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +3 -1
app.py CHANGED
@@ -51,7 +51,9 @@ Models showcased in the demo are marked with (*) above. More details about how w
51
 
52
  ### Final Remarks
53
 
54
- - **Data.** One of the main challenges in the area of Readability Assessment is the availability of reliable data. For Spanish, in particular, the highest-quality existing dataset is Newsela. However, it has a restrictive license that prohibits publicly-sharing its texts. In addition, since its texts are translations from original English news, they can suffer from [translationese](https://en.wiktionary.org/wiki/translationese), deeming them less suitable for training models that will analyse texts produced directly in Spanish. Therefore, our first challenge was to find texts that were originally written in Spanish *and* that contained information about their readability level (i.e. the target gold label). Unfortunately, we could not find any other big publicly-available corpus, and decided to combine texts scraped from several webpages. This also prevented us for developing models that could estimate readability in more fine-grained levels (e.g. CEFR levels), which was our original goal. Future work will include contacting editorial groups that create texts for learners of Spanish as a second language, and establish collaborations that could result in creating new language resources for the readability research community.
 
 
55
 
56
  - **Models.** As explained before, our models are direct fine-tuned versions of [BERTIN](https://huggingface.co/bertin-project). In the future, we aim to compare our models to fine-tuned versions of [multilingual BERT](https://huggingface.co/bert-base-multilingual-cased), to analyse whether multilingual embeddings could offer additional benefits. In addition, our current setting treats Readability Assessment as a classification task. Future work includes studying models that treat the problem as a regression task or, as [recent work suggests](https://arxiv.org/abs/2203.07450), as a pair-wise ranking problem.
57
 
 
51
 
52
  ### Final Remarks
53
 
54
+ - **Limitations and Biases.** The readability of a document can be affected by its domain and target audience. For example, an article in a medical journal can be more difficult to understand than a news article. However, medical professionals may have less difficulty than lay readers. As such, it is important to take all characteristics of the documents into account when analysing the performance of our models. A deeper study of such type for our models is left as future work. The CAES dataset, in particular, offers benefits for that type of investigation, since its metadata includes information such as the domain of the document, the years of study of the person who wrote the text, etc. However, we did not use this dataset for our current models since its texts were produced *by* students and not *for* students, and due to the high variability of the characteristics of the writers and documents.
55
+
56
+ - **Data.** One of the main challenges in the area of Readability Assessment is the availability of reliable data. For Spanish, in particular, the highest-quality existing dataset is Newsela. However, it has a restrictive license that prohibits publicly-sharing its texts. In addition, since these texts are translations from original English news, they can suffer from [translationese](https://en.wiktionary.org/wiki/translationese), deeming them less suitable for training models that will analyse texts produced directly in Spanish. Therefore, our first challenge was to find texts that were originally-written in Spanish *and* that contained information about their readability level (i.e. the target gold label). Unfortunately, we could not find any other big publicly-available corpus with those characteristics, and decided to combine texts scraped from several webpages. This also prevented us from developing models that could estimate readability in more fine-grained levels (e.g. CEFR levels), which was our original goal. Future work will include contacting editorial groups that create texts for learners of Spanish as a second language, and establish collaborations that could result in creating new language resources for the readability research community.
57
 
58
  - **Models.** As explained before, our models are direct fine-tuned versions of [BERTIN](https://huggingface.co/bertin-project). In the future, we aim to compare our models to fine-tuned versions of [multilingual BERT](https://huggingface.co/bert-base-multilingual-cased), to analyse whether multilingual embeddings could offer additional benefits. In addition, our current setting treats Readability Assessment as a classification task. Future work includes studying models that treat the problem as a regression task or, as [recent work suggests](https://arxiv.org/abs/2203.07450), as a pair-wise ranking problem.
59