UWB at A rabic Reverse Dictionary shared task: Computing the meaning of a gloss (2023)

Abstract: To extract the ‘meaning’ of a gloss phrase, we build a list of sense-IDs for each word in the phrase which is in our vocabulary. We choose one sense-ID from each list so as to maximise similarity of all the IDs in the chosen subset. We take the meaning of the phrase in semantic space to be the weighted sum of the embedding vectors of the IDs.

## introduction the ksaa reverse dictionary shared task is to find the embedding vector for a word, given a gloss, or short definition. the two sub-tasks use arabic and english glosses. the task is partly inspired by semeval-2022 task 1 @xcite , which provided training data for english, spanish, french, italian, and russion. for that task there was an additional subtask, generating a gloss from a word-embedding, which proved difficult to score; bleu scores, which are typically used to measure the success of machine translations, seemed to perform poorly to compare the adequacy of generated glosses. the individual records in the training data for the task provide: an id, which corresponds to a particular sense of a word; the word; a gloss or definition; and two embedding vectors, a sgns and an electra vector. the sgns vector is a word-based skipgram vector, so that each sense of the word has the same vector; the electra vector is a transformer context-based vector, and each id has a different vector. thus in all there are four subtasks: to find the sgns or electra semantic-vector from the arabic or english words of the gloss. for neither language are there sufficient training data to completely populate the vectors of the gloss. there are several possible reasons for this problem. considering just the problems for arabic gloss phrases: a correctly spelled arabic word can appear with complete vocalization, with partial vocalization, or with no vocalization at all. english has similar problems, but we spent more attention on arabic. an apparently obvious part of the solution might be to fine-tune a pre-trained transformer on the glosses, and then attempt to generate the word gloss by a transformation on the phrase embedding. this idea was used for baseline, and in the semeval-2020 task by for example, @xcite . however, in the current task, we are restricted to less than fifty thousand training examples. training a transformer on so little data seems problematic. using an externally pre-trained transformer means bringing in external data, not in the spirit of a closed task. we wanted to try a simple data-processing approach. we used the data files described in there are 35224 number of distinct tokens used for the english glosses, while the total english vocabulary of en.complete.with.id.json is only 26068 words. doing a set subtraction, we see that the larger set contains many capitalized and inflected words, but also a number of words like chat, majestic, dilemma, xanax, sio2, inactivate, that is, both technical terms, and relatively common words which happened not to be included in the dictionary at hand. doing the subtraction the other way, we see that 15212 of our vocabulary words do not occur in the glosses. we considered using a separate, larger english embedding, of which there are many available, with a cross-lingual transform, which could be easily prepared for the sgns vectors based on com-mon vocabulary. but it wasn't clear that 'open' as intended by the organizers included this option, and matching senses for the electra embedding seems to be exactly the problem on which we are already working. similarly extending the arabic vocabulary has the same problems, except that the organizers used the term 'closed' for subtask 1, which would clearly preclude doing it. ## system our system foot_0 does not use a neural network. it uses ar.train.json for its arabic vocabulary, and en.complete.with.it.json for its english vocabulary. it uses ar.en.train.json, which contains both arabic and english words, in order to build a cross-lingual transform, so that the vectors built in the english space with english glosses can be converted to vectors in one of the arabic spaces. in addition to copying the ar.train.json sense dictionary, making a table of all the sense-ids for each wordform, we also build a dictionary, swords, of derived values which points into that table. this includes several kinds of values: each of these substitute words points to one or more words for which we have a least one id and embedding vector. [although english offers similar possibilities, we did not build a substitution list for english before the end of the test period.] ## conclusion our earnest thanks go out to the organizers, who prepared a substantial dataset for this workshop. although our approach was not completely successful, it did better than the baseline for subtask 1, arabic definitions with sgns vectors. we have discussed several variations in section 5, some of which might improve the system performance on the other three variations of the task. we look forward to seeing designs of other workshop participants.