Curriculum Consistency Learning for Conditional Sentence Generation (2024)

Abstract: Consistency learning (CL) has proven to be a valuable technique for improving the robustness of models in conditional sentence generation (CSG) tasks by ensuring stable predictions across various input data forms. However, models augmented with CL often face challenges in optimizing consistency features, which can detract from their efficiency and effectiveness. To address these challenges, we introduce Curriculum Consistency Learning (CCL), a novel strategy that guides models to learn consistency in alignment with their current capacity to differentiate between features. CCL is designed around the inherent aspects of CL-related losses, promoting task independence and simplifying implementation. Implemented across four representative CSG tasks, including instruction tuning (IT) for large language models and machine translation (MT) in three modalities (text, speech, and vision), CCL demonstrates marked improvements. Specifically, it delivers +2.0 average accuracy point improvement compared with vanilla IT and an average increase of +0.7 in COMET scores over traditional CL methods in MT tasks. Our comprehensive analysis further indicates that models utilizing CCL are particularly adept at managing complex instances, showcasing the effectiveness and efficiency of CCL in improving CSG models. Code and scripts are available at https://github.com/xinxinxing/Curriculum-Consistency-Learning.

## introduction consistency learning (cl) as a robust and widely utilized methodology for enhancing the performance of various conditional sentence generation (csg) models, @xcite @xcite @xcite @xcite . the primary goal of cl is to ensure uniformity in a model's predictions across * co-first and corresponding author identical or similar instances, even when faced with minor variations in data or model configurations. however, the integration of cl into models has faced challenges related to suboptimal convergence efficiency @xcite . the core issue arises from the increased computational demands required to optimize the cl loss, effectively doubling the convergence time for the models. furthermore, with the ongoing growth in the complexity and size of models, the efficiency problem becomes increasingly significant @xcite . this work is motivated by the need to enhance the learning efficiency of csg models using cl. our observations show that existing cl approaches use a uniform strategy to learn consistency features, which is suboptimal for csg models. early in training, models should prioritize developing strong feature representations over consistency learning. once proficient in feature discrimination, models can then focus on consistency modeling. this phased approach aligns with curriculum learning principles @xcite , which tailor the learning process to the model's evolving capabilities. however, integrating curriculum learning with cl methods is challenging. curriculum learning depends on task-specific difficulty metrics and training schedules @xcite @xcite , contrasting with the broad applicability of cl, adding complexity to this integration. in this paper, we introduce a novel approach, curriculum consistency learning (ccl), designed to overcome the existing limitations and broaden the applicability of cl in csg models. ccl integrates a difficulty measure and a proficiency estimator directly derived from the cl losses observed in both the training and validation phases. this innovation eliminates the dependency on task-specific metrics, streamlining the implementation process. ccl strategically guides the model to engage in cl with instances that match its current proficiency level, ensuring more effective and efficient optimization of cl-enhanced csg models, leading to improved performance. experimental evaluation in four csg tasks, including instruction tuning (it) tasks @xcite for large language models (llms) and machine translation (mt) tasks in three distinct modalities (text, vision, and speech) establishes the superior performance of ccl over existing advanced baselines and conventional cl approaches. specifically, it delivers +2.0 average accuracy point improvement compared with vanilla it and an average increase of +0.7 in comet scores over traditional cl methods in mt tasks. this advancement is supported by detailed analyses, which corroborate the effectiveness of ccl in improving the model's ability to handle complex instances, thus confirming the benefits of our method in enhancing the efficiency and efficacy of models in diverse csg tasks. our contributions are summarized as follows: ## our proposed method we propose that cl should not be the primary focus until the model has developed sufficient meaningful features and representations. applying consistency constraints too early can lead to learning incorrect features and slow optimization. to address this, we introduce ccl, which adapts consistency learning to different training stages. as shown in figure 1 , we first measure instance difficulty using the consistency loss of the training set, eliminating the need to model different modalities separately. next, we calculate the average cl loss for both the training and validation sets to gauge the model's capacity. finally, we calculate the instance weights and update model parameters, ensuring the model focuses on tasks aligned with its strengths. ## evaluation to establish the universality of our approach, we evaluate it using both unimodal and multimodal tasks. we focus on the following four csg tasks: instruction tuning (it) uses instruction data pairs to perform supervised fine-tuning on llms, which can effectively align llms with human preferences. we use the popular alpaca-gpt4 dataset @xcite and the llama-2 model family @xcite as our training sets and foundation model respectively. textual machine translation (tmt) involves translating a sentence from one language to another while retaining semantic equivalence. we em-ploy the vanilla transformer model @xcite and machine translation llm alma-7b @xcite as our baseline and test their performance on two prominent benchmarks iwslt14 german-english benchmarks and wmt22 english-chinese benchmarks respectively. multimodal machine translation (mmt) is a task aimed at using a fusion of text and image data to enhance the quality and expressive capacity of translation. we run experiments on the widelyused multi30k english-german benchmark @xcite , using vl-t5 @xcite as our baseline due to its superior performance. speech-to-text translation (st) translates source acoustic speech signals in a source language into a foreign text without any intermediate output. we use the commonly-used must-c dataset @xcite and focus on the english-german translation task. the speechut model @xcite , known for its strong performance, is chosen as our baseline. ## analysis in this section, we aim to answer the following research questions: 1) can ccl improve the efficiency and universality of cl? 2) is cl loss a better difficulty measurer for ccl? 3) how does ccl enhance csg model efficacy? we mainly evaluate four specific tasks, including tydiqa for it on llama-2-7b, iwslt for tmt, multi30k for mmt, and must-c for st. additionally, we employ model cl for mt and data cl for it, acknowledging the absence of model cl in it. for performance metrics, we apply bleu scores to assess translation quality and f1 scores for it, except where noted otherwise. ## conclusion this paper introduces curriculum consistency learning (ccl), a novel approach designed to refine the optimization process of conditional sentence generation models with consistency learning. our method proposes a novel mechanism for measuring difficulty and estimating model proficiency, grounded in the analysis of consistency learning loss observed during model training, which is userfriendly and operates autonomously. extensive experiments confirm the effectiveness of ccl in improving model performance and tackling challenges from complex instances. figure 5 : cl loss, derived automatically from the model, aligns closely with unique feature metrics across various domains that need to be designed by human experts. these results fully demonstrate the rationality of cl loss as a reasonable difficulty measurer of curriculum learning.