Label Confidence Weighted Learning for Target-level Sentence Simplification (2024)

Abstract: Multi-level sentence simplification generates simplified sentences with varying language proficiency levels. We propose Label Confidence Weighted Learning (LCWL), a novel approach that incorporates a label confidence weighting scheme in the training loss of the encoder-decoder model, setting it apart from existing confidence-weighting methods primarily designed for classification. Experimentation on English grade-level simplification dataset shows that LCWL outperforms state-of-the-art unsupervised baselines. Fine-tuning the LCWL model on in-domain data and combining with Symmetric Cross Entropy (SCE) consistently delivers better simplifications compared to strong supervised methods. Our results highlight the effectiveness of label confidence weighting techniques for text simplification tasks with encoder-decoder architectures.

## introduction text simplification aims to reduce the linguistic complexity of a text while preserving its meaning, making it more accessible to a wider audience, such as language learners, individuals with cognitive impairments, or those with low literacy skills. multi-level sentence simplification takes this a step further by generating simplified sentences tailored to specific target audiences with varying language proficiency levels. despite the importance of multi-level sentence simplification, progress in this area has been hindered by the limited availability of labeled parallel corpora. the only available dataset for this task, newsela-auto @xcite , is relatively small, making it challenging to train high-performing models. to address this issue, previous studies have explored data augmentation techniques, such as constructing pseudo-parallel corpora using complex-to-simple sentence pairs @xcite . however, these approaches may suffer from the propagation of label errors in the augmented corpora, leading to suboptimal performance. in this paper, we propose label confidence weighted learning (lcwl), a novel approach to multi-level sentence simplification that leverages weak supervision from a large-scale paraphrase dataset and a pre-trained classifier to generate pseudo-labeled simplification data. it mitigates the impact of noisy labels through a label confidence weighting scheme in the training loss of the encoder-decoder model. this sets lcwl apart from existing confidence-weighting methods primarily designed for classification tasks. we evaluate our approach on the newselaauto dataset and demonstrate that lcwl outperforms state-of-the-art unsupervised baselines across multiple complexity levels. fine-tuning the lcwl model on in-domain data further improves performance, consistently delivering superior simplifications compared to strong supervised methods. our results highlight the effectiveness of label confidence weighting techniques for learning with noisy pseudo-labels in text simplification tasks involving encoderdecoder architectures. we provide our codes at: https://github.com/astro-jon/lcwl . ## methodology we propose label confidence weighted learning (lcwl), a novel approach that leverages weak supervision from a large-scale paraphrase dataset and a pre-trained classifier to generate pseudo-labeled simplification examples across various complexity levels. lcwl combines loss reweighting and sample selection techniques, allowing the model to mitigate the impact of noisy labels and exploit additional unlabeled data. ## evaluation metrics we use seven metrics, ∆𝑆𝐿𝐸, sari, lens rf, lens norf, 𝐹𝐾𝐺𝐿, bertscore, bleu. sari @xcite evaluates sentence simplification by comparing system output with the original sentence and references, averaging f1 scores for three editing operations. 𝑭𝑲𝑮𝑳 (flesch-kincaid grade level) @xcite measures readability gain based on word and sentence length. lens @xcite : lens is a learnable metric that evaluates text simplification quality by encoding the input, system output, and human references using transformer models. trained on human ratings with an adaptive loss focusing on the most relevant references, it predicts a quality score. lens-salsa @xcite ): lens-salsa is a reference-free automatic simplification metric, trained to predict sentenceand word-level quality simultaneously. bertscore @xcite assesses semantic preservation between system output and references, but does not reflect simplification degree. bleu @xcite calculates the similarity between system output and references based on n-gram matching, regardless of position. to comprehensively evaluate the performance of different models, we calculate their average rank across 7 metrics and 4 simplification levels. however, prior research has highlighted the potential drawbacks of using the bleu metric for evaluating sentence simplification @xcite , and bertscore, while effective in assessing semantic similarity, falls short in quantifying the degree of simplification achieved. therefore, we also report the average ranks of each model based on a subset of 5 evaluation metrics, excluding bleu and bertscore, to provide a more focused assessment of simplification quality. ## case analysis given the case studies in appendix d, we examine the simplification results across different levels and reveal several key insights into the performance of our proposed methods and baselines: ## conclusion this paper introduced label confidence weighted learning (lcwl), a novel approach to multi-level sentence simplification that leverages weak supervision from a large paraphrase dataset and a pre-trained classifier. lcwl incorporates a label confidence weighting scheme in the training loss of the encoder-decoder model, enabling it to generate high-quality simplifications across multiple complexity levels while mitigating the impact of noisy pseudo-labels. this sets lcwl apart from existing confidence-weighting methods that primarily focus on classification tasks. experiments on the newsela-auto dataset demonstrated that lcwl outperforms state-of-the-art unsupervised baselines, and when combined with fine-tuning on in-domain labeled data (lcwl+ft), it consistently delivers superior simplifications compared to strong supervised methods. the effectiveness of lcwl highlights the importance of label confidence weighting techniques for learning with noisy pseudo-labels in text simplification tasks involving encoder-decoder architectures. future work could explore extending lcwl to other text generation tasks facing similar challenges, investigate techniques to further improve pseudo-label quality, and conduct human evaluations to assess the perceived quality and usefulness of the generated simplifications for different target audiences.