# High-Quality Data from Crowdsourcing towards the Creation of a Mexican Anti-Immigrant Speech Corpus
## 6 Conclusions and Future Work  
In conclusion, we describe in this paper the methodology used to create an ad-hoc anti-immigrant dataset to train automatic detectors of online anti-immigrant speech published in the Mexican context. We elaborate in detail on the methodology, which introduces elements not yet considered in the literature, and discuss the quality of the obtained data. The experiments conducted in this study provide valuable insights and answer the research questions.  
We have verified that the kappa coefficient value can be an efficient mechanism for quality annotations from non-experts. It also enhances the temporal efficiency by involving and utilizing contributions from non-expert annotators who meet the expert criteria, rather than relying solely on expert annotators (RQ1).  
We have confirmed that, given a sophisticated but concise guideline, non-expert annotators can detect almost all categories of anti-immigrant speech (RQ2), although some deeply rooted stereotypes remain undetected (RQ2A).  
We have verified that the data obtained from experts and non-experts represent a valid input for any learning algorithm whenever there is a quality control mechanism in the annotation process (RQ3).  
An analysis based on a white box model proved that the annotation process induces semantic aspects directly associated with our study object, which allows us to assert that our annotation approach is somewhat context-sensitive (RQ4).  
Overall, we have demonstrated in this paper that it is possible to build a good-quality, context-sensitive training dataset for machine learning purposes by combining experts' and non-experts' annotations supervised by a control mechanism. As we move towards the creation and analysis of the Mexican anti-immigrant speech corpus, we must conduct a formal external validation of our classifier to ensure its effectiveness when dealing with real new data. We will use the validated model to classify the rest of the data that we obtained through AGEI and INGEOTEC, several million tweets, thus creating the most complete database for the study of online Mexican anti-immigrant speech in a new context of intense migration and political turmoil. With all the necessary resources at hand, we will focus on geographic, temporal, semantic, and topological dimension analysis of online anti-immigrant speech in Mexico.  
Conceptualization, T.C., A.M.-V.; Methodology, A.M.-V., T.C.; Software, K.G.-H., A.M.-V.; Validation, A.M.-V., T.C., K.G.-H., E.A.-B.; Investigation, A.M.-V., T.C., K.G.-H., E.A.-B.; Resources, A.M.-V., E.A.-B.; Data curation, T.C., K.G.-H.; Writing--original draft preparation, T.C., A.M.-V., K.G.-H., E.A.-B.; Writing--review and editing, A.M.-V., T.C., K.G.-H., E.A.-B. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
Due to privacy and ethical restrictions, the anonymized datasets are shared only on demand for research purposes. Please visit [[https://github.com/Kgazch/Annotation-platform](https://github.com/Kgazch/Annotation-platform)]([https://github.com/Kgazch/Annotation-platform](https://github.com/Kgazch/Annotation-platform)) (accessed on 19 June 2023 ).  
**Acknowledgments:** The authors would like to express their gratitude to [PERSON] and [PERSON] for providing the raw data. Special thanks to [PERSON] and [PERSON] for their expert annotations, [PERSON] for English revision, and all the non-expert annotators, who made a significant contribution to the development of this research work.  
**Conflicts of Interest:** The authors declare no conflicts of interest.