BITS Pilani at S em E val-2024 Task 1: Using text-embedding-3-large and L a BSE embeddings for Semantic Textual Relatedness (2024)

Abstract: Semantic Relatedness of a pair of text (sentences or words) is the degree to which theirmeanings are close. The Track A of the Semantic Textual Relatedness shared task aimsto find the semantic relatedness for the English language along with multiple other lowresource languages with the use of pretrainedlanguage models. We proposes a system tofind the Spearman coefficient of a textual pairusing pretrained embedding models like textembedding-3-large and LaBSE.

## introduction semantic relatedness is defined as the degree of closeness of textual units (sentences, words, paragraphs) @xcite . this makes semantic relatedness an important metric to understand the meaning of text. a paragraph is a string of multiple related sentences and similar paragraphs in a sequential manner form passages or documents which provides valuable information. understanding this cohesion among sentences @xcite and passages is critical for understanding meaning and therefore generating more powerful natural language processing systems. we consider text to be semantically close if there is some sort of similar meaning. we can see an example of textual relatedness in since semantic relatedness is crucial to understanding meaning, it has many use cases in various nlp tasks such as question answering and text generation to produce coherent statements @xcite . other natural language challenges like machine translation or information retrieval can be reduced to a semantic distance problem. it is also a key factor for text summarization, the relation between sentences in text will allow for more accurate summaries without too much loss of context. for track a of the semeval 2024 task 1: semantic textual relatedness (str) @xcite on codalab @xcite , we aim to create a system to automatically detect the degree of semantic relatedness between pairs of sentences with the openai text-embedding-3-large and labse text embedding models. this is for languages like english, as well as multiple low resource languages like algerian arabic, amharic, hausa, kinyarwanda, marathi, moroccan arabic, spanish, telugu. our code can be found on github at https://github.com/dipsivenkatesh/ semeval-2024-task-1 2 background ## results for evaluation, the organizers rank the system based on spearman rank correlation coefficient with the golden labels. the performance of the models on all the languages can be found in ## conclusions and limitations in our paper for the semeval task 1: semantic textual relatedness we propose a zero-shot approach for relatedness using the text-embedding-3-large and labse embedding models. it is importand to consider that text-embedding-3-large is not an open-source model and that these models may contain inherent biases in them.