Recurrent Alignment with Hard Attention for Hierarchical Text Rating (2024)

Abstract: While large language models (LLMs) excel at understanding and generating plain text, they are not tailored to handle hierarchical text structures or directly predict task-specific properties such as text rating. In fact, selectively and repeatedly grasping the hierarchical structure of large-scale text is pivotal for deciphering its essence. To this end, we propose a novel framework for hierarchical text rating utilizing LLMs, which incorporates Recurrent Alignment with Hard Attention (RAHA). Particularly, hard attention mechanism prompts a frozen LLM to selectively focus on pertinent leaf texts associated with the root text and generate symbolic representations of their relationships. Inspired by the gradual stabilization of the Markov Chain, recurrent alignment strategy involves feeding predicted ratings iteratively back into the prompts of another trainable LLM, aligning it to progressively approximate the desired target. Experimental results demonstrate that RAHA outperforms existing state-of-the-art methods on three hierarchical text rating datasets. Theoretical and empirical analysis confirms RAHA’s ability to gradually converge towards the underlying target through multiple inferences. Additional experiments on plain text rating datasets verify the effectiveness of this Markov-like alignment. Our data and code can be available in https://github.com/ECNU-Text-Computing/Markov-LLM.

## introduction scaling up llms yields significant advances in their ability to mimic human-like text comprehension and generation @xcite @xcite . they demonstrate remarkable aptitude for in-context learning (icl) @xcite @xcite across various natural language processing (nlp) tasks (qi et al., * corresponding author. ## related work the essence of human intelligence is characterized by the ability to understand abstract concepts, engage in logical reasoning, and make advanced predictions based on existing knowledge @xcite @xcite . however, in the era of natural language processing (nlp), despite impressive representation and learning capabilities of neural networks, it is still difficult for them to infer and deduce information from contexts @xcite . this landscape has been dramatically reshaped with the evolution of large language models (llms) @xcite , driven by significant upscaling in parameters, data, and computational resources @xcite @xcite . they exhibit exceptional proficiency for in-context learning (icl) @xcite @xcite across a wide range of nlp tasks @xcite @xcite . one of the key advancements in llms is the incorporation of strategies like chain of thought (cot) prompting, which empowers these models to generate reasoning steps and tackle more complex downstream application @xcite @xcite . notwithstanding the progress made in cot reasoning @xcite @xcite , there remains a notable deficiency in current methodologies regarding the processing of hierarchical structures within long text. numerous studies have focused on identifying and correcting specific thought units where the reasoning process may deviate or require additional information, aiming to produce desired outcomes @xcite @xcite . this prevailing research predominantly concentrates on purely textual content, neglecting the intrinsic hierarchical nature of certain text formats @xcite . in our work, we propose a hard attention mechanism to redress this shortfall by introducing a novel paradigm for enhancing the processing of structured text within cot reasoning. the escalation in the scale and adaptability of llms has been accompanied by significant advancements in model fine-tuning and adaptation, exemplified by the introduction of various adapter architectures @xcite @xcite . however, these adaptations have primarily focused on enhancing the model's generation capabilities and have not addressed the limitations of llms in directly generating continuous prediction values like text rating. while the prediction of structured continuous numerical values has begun to be explored in some studies @xcite , there remains a notable gap in experimentation with large language models in this area. concurrently, recent research within llms has increasingly focused on recurrent alignment, primarily through prompting techniques and iterative refinement processes @xcite . yet, these methodologies have not sufficiently capitalized on employing the properties from predictive tasks as feedback mechanisms for iterative refinement. our contribution in this regard is the formulation of a markovlike recurrent alignment strategy. it represents a novel approach in harnessing the model's output for successive iterative enhancements, thereby augmenting the predictive precision and versatility of llms. ## methodology the proposed framework, raha, is depicted in figure 2 . it includes a tree-based hard attention mechanism that enhances the ability of llms to effectively capture hierarchical structures. in addition, a trainable llm is employed to output hierarchical text rating score. moreover, we employ a markov-like recurrent alignment strategy to enable the raha to iteratively align with the ground truth of the downstream task. ## conclusion in this paper, we propose a novel framework called raha, that leverages two llms to analyze hierarchically structured text. raha incorporates a tree-based hard attention mechanism and a recurrent alignment strategy. the tree-based attention enables a frozen llm to understand the associations between the root and each leaf separately and then selectively choose significant updates for aggregation. this results in a reduction of potential noise in the hierarchical structure and improved utilization of computing resources. the iterative recurrent alignment empowers a trainable llm to revisit insights gained from previous deliberations, progressively aligning itself with the desired property for downstream tasks. in evaluations on three datasets, raha outperforms existing baselines in text rating estimation. theoretical and empirical analysis reveals that by repeated iterations of prompting the results from the preceding step, raha produces hidden representations that gradually approach the optimal representation. this study enhances the abilities of llms in handling hierarchical text and aligning with specific tasks.