T hink A nswer Loss: Balancing Semantic Similarity and Exact Matching for LLM Reasoning Enhancement (2025)

Abstract: Knowledge distillation for large language models often uses Chain-of-Thought (CoT) and answer pairs, but existing methods struggle with appropriate supervision signals. Uniform constraints (e.g., cross-entropy) on CoT can enforce literal, verbose reasoning and suppress expressive diversity, while solely semantic constraints on answers can reduce accuracy in classification tasks. This paper proposes ThinkAnswer Loss, an information-theoretic differential supervision framework that decouples CoT and answer supervision. ThinkAnswer Loss applies semantic similarity constraints to the CoT portion while maintaining strict literal matching for the answer. We theoretically demonstrate its connection to mutual information maximization and derive a tight upper bound on generalization error. Experimental validation on text quality assessment and mathematical reasoning tasks shows that our method maintains answer accuracy while effectively reducing CoT length and preserving semantic content, thereby accelerating inference.

## introduction supervised fine-tuning (sft) of large language models (llms) has emerged as a core paradigm for enhancing domain adaptability and task performance through task-specific data optimization. in this process, the design of loss functions directly impacts the model's ability to fit target distributions and generalization performance. traditional approaches typically employ cross-entropy loss for maximum likelihood estimation (mle), optimizing alignment between model predictions and labeled data through literal matching @xcite . however, existing research has revealed its limitations: cross-entropy loss fails to effectively distinguish semantically similar but literally different outputs (e.g., synonym substitutions) and may impair the robustness of semantic representations by overemphasizing literal consistency when handling complex tasks (ren et al., 2024). the current development of large language models has established a mainstream technical paradigm of "training ultra-large parameter models first, then distilling to smaller models" @xcite . however, effectively preserving the reasoning capabilities of large models during this distillation process poses a critical challenge. existing loss algorithms exhibit significant deficiencies when handling chain-of-thought and answer pairs: most algorithms adopt a uniform constraint strategy for chain-of-thought (cot) and answer supervision, either enforcing literal consistency (e.g., cross-entropy loss) or solely pursuing semantic similarity (e.g., emo loss) @xcite . this fragmented supervision approach can lead to representation learning conflicts-for chain-of-thought reasoning tasks requiring semantic understanding, excessive literal matching inhibits the model's generalization ability; yet for classification tasks, strict literal consistency of answers remains a necessary condition for accuracy. for example, in mathematical problem-solving tasks, models must generate answers that exactly match the labels literally, while their reasoning processes (chain-of-thought) should allow semantically equivalent but differently expressed forms. existing methods (such as pure cross-entropy or contrastive loss) struggle to balance these dual requirements @xcite . to address these challenges, we propose thinkanswer loss: a method for enhancing llm reasoning capabilities that balances semantic similarity and exact matching, aiming to simultaneously optimize the semantic consistency of chainof-thought and the literal accuracy of answers. specifically, for the chain-of-thought portion in distilled data, we introduce semantic similarity constraints to encourage models to generate reasoning paths that are logically equivalent to the teacher model yet diverse in expression; for the answer portion, we retain strict cross-entropy loss to ensure precise matching of classification results. under the current technical paradigm of "large model distilling to small model," our method is particularly suitable for enhancing the reasoning capabilities of small-parameter models, maximizing inheritance of large models' reasoning advantages while maintaining computational efficiency. the contributions of this work can be summarized as follows: ## experiments we designed comprehensive experiments to validate the effectiveness of thinkanswer loss on both mathematical reasoning and text quality assessment tasks. our experiments utilized the deepseek-r1 distillation series models (including qwen-1.5b/7b/14b @xcite and llama-8b (ai@meta, 2024)), fine-tuned on chainof-thought datasets constructed from thucnews and mathglm, and compared against traditional loss functions such as mle and emo. our evaluation employed a multi-dimensional metric system, including reasoning chain quality assessment (structural completeness, logical correctness, etc.) and answer accuracy (exact match rate, format correctness rate). we used a largescale language model (qwen3-235b-a22b @xcite ) as an evaluation tool. experimental results demonstrate that thinkanswer loss not only significantly improved answer accuracy but also effectively reduced reasoning chain length while maintaining semantic consistency, achieving dual optimization of reasoning efficiency and accuracy. we fine-tuned the deepseek-r1-distill-qwen-1.5b, deepseek-r1-distill-qwen-7b, deepseek-r1-distill-qwen-14b, and deepseek-r1-distill-llama-8b models on the training set for 3 epochs. for the thinkanswer loss: we use the adamw optimizer with a learning rate of 5.0e-5. the batch size is fixed as 64 for all experiments. the maximum input lengths for data quality evaluation tasks and math tasks were set to 2048 and 256, respectively. we integrated thinkanswer into llamafactory and used lora for fine-tuning. for other parameters, such as lr_scheduler_type, we used the default values in llamafactory. for comparison methods, to ensure fairness, we used the default parameters from the open-source code. all our experiments were executed on the a100 gpu. ## conclusion in this paper, we propose thinkanswer loss, a differentiated supervision framework for chain-ofthought generation. by designing appropriate loss functions separately for the chain-of-thought and answer components, we achieve balanced optimization between chain-of-thought semantic consistency and answer exact matching. theoretical analysis demonstrates that our semantic similaritybased loss positively correlates with the mutual information maximization objective and provides a tighter generalization error bound. experimental results further confirm the effectiveness of our method. this work offers a novel perspective on fine-tuning large language models for reasoning tasks. future research could explore different mutual information approximation methods in chainof-thought optimization and the application of this framework to more diverse reasoning tasks.