I nstruct G raph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment (2024)

Abstract: Do current large language models (LLMs) better solve graph reasoning and generation tasks with parameter updates? In this paper, we propose InstructGraph , a framework that empowers LLMs with the abilities of graph reasoning and generation by instruction tuning and preference alignment. Specifically, we first propose a structured format verbalizer to unify all graph data into a universal code-like format, which can simply represent the graph without any external graph-specific encoders. Furthermore, a graph instruction tuning stage is introduced to guide LLMs in solving graph reasoning and generation tasks. Finally, we identify potential hallucination problems in graph tasks and sample negative instances for preference alignment, the target of which is to enhance the output’s reliability of the model. Extensive experiments across multiple graph-centric tasks exhibit that InstructGraph can achieve the best performance and outperform GPT-4 and LLaMA2 by more than 13% and 38%, respectively.

## introduction currently, large language models (llms) have succeeded in reasoning on textual data @xcite . however, there also exists rich information in graph data, that is difficult to represent using plain text @xcite , such as knowledge graphs @xcite , symbolic graphs (saba, 2023), and social networks @xcite , etc. to endow llms with the ability to solve graph tasks, a series of works focus on designing the interface (e.g., prompt engineering) of llms on graph data to make them understand the semantics without parameter optimization @xcite @xcite @xcite @xcite , or injecting the graph embeddings into the partial parameters of llms through graph neural networks (gnns) @xcite @xcite . despite significant progress, we explore these two challenges: 1) there still exists a semantic gap between graph and text, which may impede the llm in graph reasoning and generation. 2) llms tend to generate hallucinations which may be caused by fabricated erroneous inputs or lack of pertinent knowledge. it can be viewed as the graph hallucination problem. to overcome these challenges, we present a framework named instructgraph that boosts llms by instruction tuning and preference alignment. a straightforward approach to solve the first challenge is to use a graph description @xcite or graph embeddings @xcite , however, these methods require a large number of manual templates to describe the graph. representing a large or complex graph via embeddings may cause information loss. in addition, the responses generated by the llm with these methods are difficult to parse into actual graphs @xcite . current investigations have demonstrated that llms have a great ability for code understanding and generation @xcite @xcite . inspired by them, we can unify graph data into a code-like universal format to enhance the llm's understanding and generation performance on graph tasks. as shown in figure 1 , each graph can be converted into a code with basic variables, such as node_list (or entity_list), edge_list (or triple_list) and optional properties. to this end, a graph instruction tuning stage is introduced to train the llm on these formulated data. in addition, previous works have found that llms generate responses with hallucination when definition: given a graph, understand the graph semantic and answer the question about caption, qa, node classification, link prediction, relevance and collaboration. definition: given a passage, understand the instruction and question, and then generate a graph to satisfy the semantics or structures. definition: given a reasoning question, think step by step: 1) find a topic entity, 2) then generate a graph that express the thinking process, 3) finally output the answer. following the instructions, typically referring to fabricated erroneous inputs or lack of intrinsic knowledge @xcite . for example, the llm may derive a wrong answer when being questioned on a graph that lacks key information, or the llm may generate a graph with incorrect facts, conflicting, or missing information. however, how to reduce this effect in graph reasoning and generation is still under-explored. hence, we introduce the graph preference alignment to alleviate the hallucination problem in the llm's reasoning and generation. specifically, we follow the direct preference optimization (dpo) algorithm @xcite to optimize the llm to make better preferences. to automatically sample the negative instances in dpo, we explore various scenarios, such as unfactual graph, conflict graph and missing graph. , to simulate the graph hallucination problem. to evaluate the effectiveness of our framework, we perform extensive experiments on multiple graph reasoning and generation tasks. results reveal that the proposed instructgraph achieves the best performance on both graph-centric instruction and preference tasks and outperforms the gpt-4 (openai, 2023) and llama2 @xcite by more than 13% and 38%, respectively. ## methodology the skeleton is shown in figure 2 , which can be decomposed into three modules, i.e., graph input engineering, graph instruction tuning, and graph preference aligning. ## experiments in this section, we perform extensive experiments to evaluate the effectiveness of instructgraph over graph tasks and general nlp tasks. ## conclusion this paper proposes a novel instructgraph framework that empowers the llm with the capacity to solve graph reasoning and generation tasks. to bridge the gap between graph data and textual language models, we introduce a structured format verbalizer to transform each graph into a code-like format and continually tune the llm based on the instruction dataset, which is collected from 29 graph tasks. in addition, we also introduce a graph preference alignment stage to further mitigate the hallucination problem when reasoning on or generating a graph. extensive experiments illustrate that instructgraph substantially achieves the best performance. in our future work, we will further improve the performance on graph-centric and general nlp tasks and scale it to other llms.