Chain-of-Thought Tuning: Masked Language Models can also Think Step By Step in Natural Language Understanding (2023)

Abstract: Chain-of-Thought (CoT) is a technique that guides Large Language Models (LLMs) to decompose complex tasks into multi-step reasoning through intermediate steps in natural language form. Briefly, CoT enables LLMs to think step by step. However, although many Natural Language Understanding (NLU) tasks also require thinking step by step, LLMs perform less well than small-scale Masked Language Models (MLMs). To migrate CoT from LLMs to MLMs, we propose Chain-of-Thought Tuning (CoTT), a two-step reasoning framework based on prompt tuning, to implement step-by-step thinking for MLMs on NLU tasks. From the perspective of CoT, CoTT’s two-step framework enables MLMs to implement task decomposition; CoTT’s prompt tuning allows intermediate steps to be used in natural language form. Thereby, the success of CoT can be extended to NLU tasks through MLMs. To verify the effectiveness of CoTT, we conduct experiments on two NLU tasks: hierarchical classification and relation extraction, and the results show that CoTT outperforms baselines and achieves state-of-the-art performance.

## introduction chains-of-thought (cot) @xcite @xcite is a technique to help language models think step by step @xcite . through intermediate steps in natural language form, cot can guide language models to decompose complex tasks into multi-step reasoning processes. currently, cot is mainly employed in large language models (llms) @xcite @xcite @xcite , as llms demonstrate impressive complex reasoning capabilities @xcite . however, although llms achieved state-of-theart performance on a wide range of nlp tasks † corresponding author. (zhao et al., 2023), @xcite ; @xcite ; @xcite found that llms were less competitive than small-scale masked language models (mlms) @xcite @xcite in many traditional natural language understanding (nlu) tasks (e.g. tasks in glue @xcite and superglue @xcite ). the reasons for this are manifold: on the one hand, llms are typically autoregressive models and are not well-suited for nlu tasks @xcite ; on the other hand, nlu tasks usually involve rich domain knowledge @xcite and require fine tuning to master such knowledge @xcite . therefore, we wonder whether the success of cot in llms can be transferred to mlms in nlu tasks. in fact, many nlu tasks also require thinking step by step. for example, in hierarchical classification @xcite , each text should follow a pre-defined taxonomic hierarchy to generate multiple labels in turn, so that high-level labels can be considered as the intermediate step, as shown in fig. 1(a) ; in relation extraction @xcite @xcite , the entity types of the subject and the object in each instance need to be determined in advance of annotating the relation @xcite , so the entity types can be considered as the intermediate step, as shown in fig. 1( @xcite . although previous studies @xcite @xcite have also attempted to incorporate intermediate steps into language models, in the perspective of cot, these methods lack both a decomposition process for tasks (not multistep reasoning) and explicit use of intermediate steps (not in natural language form). in this study, we propose chain-of-thought tuning (cott), a two-step reasoning framework based on prompt tuning @xcite , to implement step-by-step thinking for mlms on nlu tasks. the two steps of cott are: cott effectively addresses the shortcomings of previous methods: on the one hand, cott's twostep framework enables mlm to implement task decomposition; on the other hand, cott is based on prompt tuning, which allows intermediate steps to be used in natural language form @xcite . in order to inject/generate intermediate steps flexibly in both steps, we propose convertible slot [c], a new type of template slot @xcite in prompt tuning. to evaluate the effectiveness of cott, we conduct extensive experiments on two traditional nlu tasks: hierarchical classification and relation extraction, and the experimental results reveal that cott obtains superior performance compared with baselines and achieves state-of-the-art performances. furthermore, due to the introduction of intermediate steps, cott is no longer an end-to-end method but can display the reasoning process. therefore, we can further improve the ability of cott by monitoring the reasoning process. we summarize our contributions as follows: ## preliminaries of prompt tuning formally, a text classification dataset can be denoted as @xmath0, y}, where x is the text set and y is the class set. for each instance @xmath1, it is made up of several words @xmath2, w 2 , . . . , w |x| , and is annotated with a label @xmath3. to bridge the gap between pre-training tasks and downstream tasks @xcite , prompt tuning is proposed as a cloze-style task to tune mlms. prompt tuning consists of a template t , a verbalizer ϕ y (•) and a mlm m. the template @xcite is a textual string with two slots: a text slot [t] for text x and an answer slot [a] (@xmath4. the verbalizer is an injective mapping function ϕ y : @xmath5. specifically, when the text x is injected into the text slot, we get the prompt t x . then, we can formalize the label probability by feeding the prompt t x into mlm m as: hereafter, we @xmath6. in the training process, we can maximize the learning objective @xmath7. ## chain-of-thought tuning we propose chain-of-thought tuning finally, we fuse the information from both steps to rectify mlm's prediction (section 4.4). ## conclusion in this study, we propose chain-of-thought tuning (cott), a two-step reasoning framework based on prompt tuning to implement step-by-step thinking for mlms on nlu tasks. specifically, the two-step framework of cott enables mlm to implement task decomposition; and cott based on prompt tuning, which allows intermediate steps to be used in natural language form. experiments demonstrated that cott achieved state-of-the-art performance on two nlu tasks. in the future, we aim to scale up mlms to fundamentally improve their language ability. in addition, using llms to implement multi-step reasoning in nlu tasks is also a possible research direction.