Are Crescia and Piadina the Same? Towards Identifying Synonymy or Non-Synonymy between Italian Words to Enable Crowdsourcing from Language Learners
Authors: Lavinia Aparaschivei, Lionel Nicolas, Alberto Barrón-Cedeño (Università di Bologna, EURAC, 2021)
Topic: Lexical and Semantic Resources and Analysis

Abstract
We introduce a method to generate candidate pairs of related Italian words sharing (or not) synonymous relations from the ConceptNet knowledgebase. The pairs are intended to generate questions for a vocabulary trainer which combines exercises to enhance vocabulary skills with the implicit crowdsourcing of linguistic knowledge about the semantic relations between words. Our method relies on the idea that pairs of synonyms in a language tend to translate to pairs of synonyms in other languages. We generated 85k candidate pairs of Italian synonyms that can be used to produce questions for both teaching (3.8k pairs) and crowdsourcing purposes (80k pairs). Follow-up efforts are however needed in order to generate a complementary set of questions.

Introduction
Our efforts target the automatic generation of semantically-related candidate pairs of Italian words with a focus on synonymy. We address a cold start issue for a vocabulary trainer combining exercises to enhance vocabulary skills with the implicit crowdsourcing of linguistic knowledge about the semantic relations between words.While targeting a specific use case, our method contributes to a larger effort aimed at narrowing gaps on two fronts.On the NLP front, over the past few decades varied efforts have targeted the efficient creation, extension, and maintenance of resources, including crowdsourcing through platforms such as Amazon Mechanical Turk <cit.>. Still, the subject remains an open issue.On the computer-assisted language learning (CALL) front, the automatic generation of exercise content from NLP resources is almost non-existent, despite the fact that some of these datasets encode the knowledge that learners are often tested on (e.g., lexical knowledge). This absence is probably due to differences in expectations with respect to linguistic accuracy: learning materials are usually close to perfect, whereas NLP resources rarely are. Generating content from imperfect datasets poses a challenge in terms of its suitability for learning. We contribute to narrowing these gaps by producing data to tackle a cold start issue for a vocabulary trainer designed to both teach language and crowdsource linguistic knowledge from learners. We generate a collection of candidate pairs of Italian words tied to confidence scores, allowing to decide which pairs should be used for learning or for crowdsourcing purposes. Our method projects synonymy information in ConceptNet <cit.> from non-Italian onto Italian words.The obtained results show that we adequately tackle part of the cold start issue, while follow-up efforts are needed to address the remaining part. The rest of the paper is organised as follows: Section <ref> discusses the specific purpose of our method. Section <ref> summarises related work. Section <ref> and Section <ref> describe how the candidate pairs are generated and scored. Finally, Section <ref> discusses how suitable the pairs are for our specific use case and Section <ref> provides closing remarks.

Conclusion
We presented a method to generate candidate pairs of Italian words that are synonyms or non-synonyms of one another from ConceptNet. These pairs will be used to generate questions used by a vocabulary trainer designed to combine the crowdsourcing of NLP datasets with language learning. While overtime all questions will be used for both teaching and crowdsourcing purposes, part of the pairs generated will at first be used to teach learners while the other part will at first be used to crowdsource knowledge in order to enhance ConceptNet. The obtained pairs, known to be correct synonyms in advance, can be served to the learners to improve their vocabulary skills. Another subset, whose correctness is still to be confirmed, can be served to the learners for validation and to decide whether the synonym connection between them should be added to ConceptNet or not. Our results show that we can produce adequate data to generate part of the questions, while we are still unable to produce the data required to generate the complementary set of questions. In order to tackle the latter, we are devising a similar approach to identify candidate pairs of non-synonyms. We are adapting our overall procedure for the pairs of Italian words marked as translations of non-Italian words sharing any semantic relations (e.g. antonyms or hyponyms) instead of only considering the ones marked as translations of non-Italian words sharing a synonymy relation. We are also interested in exploring possibilities to perform a more informed selection of the binary classification algorithm and will explore metrics to quantify the correlation between confidence scores and the quality of the predicted labels (e.g. Pearson, Kendall). In the future, we aim at running a crowdsourcing experiment with students of Italian as a second language with the produced data.