Reverse Modeling in Large Language Models (2025)

Abstract: Humans are accustomed to reading and writing in a forward manner, and this natural bias extends to text understanding in auto-regressive large language models (LLMs). This paper investigates whether LLMs, like humans, struggle with reverse modeling, specifically with reversed text inputs. We found that publicly available pre-trained LLMs cannot understand such inputs. However, LLMs trained from scratch with both forward and reverse texts can understand them equally well during inference across multiple languages.Our case study shows that different-content texts result in different losses if input (to LLMs) in different directions—some get lower losses for forward while some for reverse. This leads us to a simple and nice solution for data selection based on the loss differences between forward and reverse directions. Using our selected data in continued pretraining can boost LLMs’ performance by a large margin across different language understanding benchmarks.

## introduction llms @xcite have shown impressive capabilities in various natural language processing tasks and beyond. these capabilities are primarily attributed to the learning of extensive corpora that cover general world knowledge @xcite . these corpora are created in human society and often demonstrate human bias, including inherently forward-oriented human cognition @xcite , e.g., reasons may precede outcomes and solutions can be deduced from given information in most cases of the grad school math dataset @xcite . in contrast, reverse thinking presents more cognitive challenges due to its contradiction with innate common sense and human logic @xcite . it inspires us to explore the following questions: to study this, we simulate reverse-modeling data by directly reversing entire paragraphs or documents at the token level. please note that this is the simplest and extreme way, but may not be the optimal way of emulating reverse thinking. we train llms with these simulated texts and conduct a comprehensive analysis. overall results indicate that llms learn forward-and reversemodeling texts equally well when trained from scratch. however, performance varies across text samples. some are suited to reverse modeling, while others favor forward modeling. notably, we find that the texts suited for reverse modeling are often of high quality and more logically coherent. training on them, the original "forward-modeling" llms can be improved. we perform empirical validation on language understanding benchmarks, such as massive multitask language understanding (mmlu) @xcite . in summary, this paper has two main contributions. ## related work in this paper we utilize the reverse text for model training. previous work on reverse inputs falls into three main areas. the first area involves the use of reverse text in machine translation. studies show that using decoders to process text both left-toright and right-to-left within an encoder-decoder framework improves machine translation performance @xcite , a finding later extended to llms @xcite . concurrently, @xcite examines the relationship between error propagation and reverse direction decoding in machine translation. the second area focuses on the reversal curse @xcite , where an llm trained to understand "a is b" may struggle to generalize to "b is a". reversing the text is proposed as a solution to this problem @xcite . these two streams of work focus on machine translation or the reversal curse. third, a recent work @xcite also explores the direction of input text, but there are two key differences compared to ours: (1) our work is inspired by the concept of reverse thinking, while the reversed input is one simulating solution; @xcite we further analyze it across different domains and inference steps and discover a valuable tool for assessing data quality. our applications are partially related to the selection of training data for llms, which is divided mainly into heuristic and model-based methods @xcite . heuristic methods filter out low-quality data by defining various rules, such as the ratio of nouns and verbs @xcite @xcite . model-based methods filter data by training selection models or based on the perplexity of language models @xcite @xcite . however, our data selection method is an extra bonus derived from the reverse modeling analysis. ## experimental settings forward and reverse training. given a original text, it can be represented as a sequence after tokenization, which is used for forward training. to perform reverse training, we directly reverse the original token sequence to construct a reverse training sample. while some studies explore keeping the original orders of detected words or entities during reverse @xcite , we choose the simplest operation to avoid the various performance of detection modules in different domains and languages. the llama2-7b (touvron ## experiments rq1: can llms perform reverse modeling? to explore llms' reverse modeling capabilities, we investigate two pre-training approaches: (1) continued training from a well-trained model checkpoint and ( 2 ) pretraining from scratch with random initialization. specifically, we train models fed with forward input and reverse text using the two approaches separately. figure 1 compares training losses (average sample losses within training batches) for english using both methods on the mc4 dataset, while figure 7 in the appendix b shows analogous results for other languages. in the continued pretraining setting, the forward loss for forward-modeling remains stable due to extensive training in the initial pretraining stage. in contrast, the reverse loss for reverse modeling, initially high, decreases rapidly after a few training steps. notably, the forward loss is consistently lower than the reverse loss during continued pretraining. we speculate this occurs because the initial pretraining corpora consists entirely of forwarddirection texts, imparting a natural directional bias to the llms. consequently, the models find pro-text favoring reverse (low reverse loss) text favoring forward (low forward loss) whether you like it or not, your garden is an open park for all of nature's creatures. ... let's take a few minutes to learn all about ladybugs in your garden. are ladybugs good for your garden? ... now that you know all about ladybugs and their role in controlling the aphid population, you may be interested in attracting ladybugs to your garden. ... ubuntu manpage: phm2helix -calculate projections through a time varying phantom object. ... phm2helix -calculate projections through a time varying phantom object. ... phm2pj calculates projections through a time varying phantom object. ... ## conclusions in conclusion, our results demonstrate that llms can learn from both forward and reverse-modeling texts with comparable proficiency when trained from scratch. this study also highlights the potential benefits of incorporating training data that favors reverse modeling. our findings underscore the importance of exploring diverse reverse modeling frameworks to enhance the capabilities of llms.