ANU at MLSP -2024: Prompt-based Lexical Simplification for E nglish and S inhala (2024)

Abstract: Lexical simplification, the process of simplifying complex content in text without any modifications to the syntactical structure of text, plays a crucial role in enhancing comprehension and accessibility. This paper presents an approach to lexical simplification that relies on the capabilities of generative Artificial Intelligence (AI) models to predict the complexity of words and substitute complex words with simpler alternatives. Early lexical simplification methods predominantly relied on rule-based approaches, transitioning gradually to machine learning and deep learning techniques, leveraging contextual embeddings from large language models. However, the the emergence of generative AI models revolutionized the landscape of natural language processing, including lexical simplification. In this study, we proposed a straightforward yet effective method that employs generative AI models for both predicting lexical complexity and generating appropriate substitutions. To predict lexical complexity, we adopted three distinct types of prompt templates, while for lexical substitution, we employed three prompt templates alongside an ensemble approach. Extending our experimentation to include both English and Sinhala data, our approach demonstrated comparable performance across both languages, with particular strengths in lexical substitution.

## introduction lexical simplification, an essential component in making complex text more understandable, involves replacing complex words with simpler alternatives while preserving the meaning and syntax @xcite . this task is specifically valuable for people with limited knowledge in certain languages or domains or for people with low literacy skills @xcite . lexical simplification can be composed as a cascade of complex word identification and lexical substitution. addressing both these tasks is vital for improved language understandability. complex word identification task is the first step in lexical simplification @xcite . this task can be formulated as identifying the complex words in text or as predicting the level of lexical complexity for each word. various techniques have been employed for this task, ranging from rule-based @xcite through threshold-based @xcite to lexicon-based approaches @xcite . following these methods, researchers have also explored feature-based machine learning methods @xcite that also incorporate word embedding models and more sophisticated approaches like deep learning models such as long short-term memory (lstm) networks, modelling the problem as a sequence labelling task @xcite . recently, contextual embedding models like bidirectional encoder representation from transformers (bert) have gained attention for complex word identification due to their ability to capture nuanced contextual information @xcite . similar to complex word identification, lexical substitution is an important sub-task for lexical simplification. early methods relied on lexical resources to generate simpler, suitable, relevant substitutes for complex or target words @xcite . this evolved with the introduction of word embedding models like word2vec @xcite , global vectors for word representation (glove) @xcite , and embedding from language models (elmo) @xcite . more recently, contextual embedding models such as bert @xcite , roberta @xcite , and xlnet @xcite have become popular for lexical substitution, sometimes combined with lexical resources for improved performance @xcite . even though many natural language processing tasks have relied on more complex or sophisticated methods based on deep learning models and contextual embeddings, with the emergence of generative artificial intelligence (ai), most of the methods have shifted to exploring simpler approaches based on prompt engineering @xcite . prompt engineering presents straightforward and effective approaches for a wide range of tasks, including for lexical complexity prediction and substitution. in this study, we leveraged prompt engineering for both tasks, focusing on improving the accuracy and efficiency. ## results the results of the prompt-based lexical complexity prediction methods did not reach the performance levels of the top submissions in the lexical complexity prediction task ( in lexical simplification for english, our proposed few-shot approach showed strong performance, achieving comparable results with respect to the best submissions for the task across all metrics ( ## discussion in this paper, we have explored the applicability of prompt-templates for both lexical complexity prediction and lexical substitution for simplification in english and sinhala. our investigation primarily focused on three prompting methodologies: zero-shot, one-shot, and few-shot. the experiments demonstrated diverse performance levels across the two tasks and languages under consideration. the most effective approach of our experiments for predicting complexity in english relied on the zero-shot method, while for sinhala, the one-shot approach gave the best results. this difference may stem from differences in language data availability and the complexity of each language and task. compared to sinhala, english has more language data available, providing the model with a more extensive information base. this could be a reason why for english the zero-shot approach performed better, as the model could leverage enough contextual information. however, sinhala, being less extensively studied, likely has fewer linguistic resources and data available for training. therefore, the one-shot approach, which provides additional context, may be better suited to capture the patterns and dependencies in the language. considering the performance of the promptbased methods for complexity prediction in sinhala, the few-shot approach did not perform as well as the one-shot approach, even though more additional information was provided. this discrepancy could be attributed to the quality of the samples included in the prompt template. if the chosen examples fail to adequately represent the lexical features and patterns of the language, it may lead to a degradation in performance, resulting in poorer results compared to the one-shot approach. the results from the lexical substitution for simplification indicated varied performance. in english, out of our experiments, the few-shot approach gave the best results, closely followed by the ensemble approach, which combined results from all three prompt templates. this suggests that the few-shot approach provided good example instances that helped in capturing the lexical intricacies of the language. therefore, while the ensemble approach gave comparable performance, it did not filter the best predictions as effectively as the few-shot method. however, for sinhala lexical substitution, we only employed the ensemble approach. unfortunately, the results indicated subpar performance. this suggests that the ensemble approach did not effectively capture the lexical patterns, dependencies of sinhala language, that resulted in unsatisfactory outcomes. the findings indicate the importance of investigating the influence of the factors such as data availability, language complexity, and sample quality on the outcomes of lexical simplification tasks. additionally, refining prompt tuning methods could enhance the effectiveness and outcomes. ## conclusion in this work, we have used prompt-based methods for both lexical complexity prediction and lexical substitution for simplification, focusing on exploring the applicability of generative ai methods. the results from the different methods indicate varied performance levels across the two tasks and languages, giving evidence of challenges related to data availability, representations, quality of the samples, language complexity, and adaptability of the models for the lexical simplification task. this encourages further investigations that could potentially improve the performance differences.