GNET - QG : Graph Network for Multi-hop Question Generation (2025)

Abstract: Multi-hop question generation is a challenging task in natural language processing (NLP) that requires synthesizing information from multiple sources. We propose GNET-QG, a novel approach that integrates Graph Attention Networks (GAT) with sequence-to-sequence models, enabling structured reasoning over multiple information sources to generate complex questions. Our experiments demonstrate that GNET-QG outperforms previous state-of-the-art models across several evaluation metrics, particularly excelling in METEOR, showing its effectiveness in enhancing machine reasoning capabilities.

## introduction question generation (qg) is the task of producing a natural language question given an input context and an answer. while recent neural models have achieved considerable success in qg, they often fail to generate complex, multi-hop questions that require reasoning across multiple contexts. unlike simple questions that rely on a single fact, multihop questions demand the integration of knowledge from multiple pieces of information to formulate a coherent query. multi-hop question generation is not only a challenging task but also a critical one, with applications ranging from improving query suggestions for search engines to enhancing educational tools for reading comprehension @xcite . despite advancements in models like mulqg @xcite and cqg @xcite , existing methods still struggle to consistently generate high-quality multi-hop questions across diverse contexts. to address this gap, we introduce gnet-qg, a model that incorporates graph attention networks (gat) to identify and focus on relevant entities within a context. by enriching the input context using gat and combining it with a powerful sequence-to-sequence model, gnet-qg is able to generate more complex, coherent, and answerable questions. our method shows significant improvements over existing techniques, especially in the meteor metric, indicating better semantic alignment in generated questions. ## related work early research in question generation focused on rule-based methods for single-hop qg, transforming declarative sentences into questions via handcrafted rules @xcite . neu-ral methods soon emerged, leveraging sequenceto-sequence models to improve reading comprehension qg, but struggled with multi-hop reasoning due to their reliance on local context @xcite . recent approaches, such as mulqg @xcite and cqg @xcite , have incorporated graph-based models to address multi-hop qg. mulqg used graph convolutional networks (gcns) to represent context and perform multi-hop encoding fusion, while cqg utilized a controlled framework to focus on key entities during question generation. building upon these methods, @xcite introduced a type-aware semantics extraction-based chain-of-thought (tase-cot) approach for few-shot multi-hop qg. this approach begins by identifying question types and key semantic phrases from the provided documents and answer, then utilizes a three-step chain-of-thought template to generate multi-hop questions based on the extracted information. despite these advancements, there remains a need for models that consistently perform well across different datasets and contexts. gnet-qg builds on this body of work by integrating graph attention networks (gat) into the question generation process. gat enables our model to focus attention on important entities within the input context, providing better entity representation for complex reasoning tasks. ## implementation details in the task of multi-hop question generation, we implemented gnet-qg by integrating the gat from this github repository foot_0 with an encoder-decoder model as the main backbone. we connected the untrained gat architecture with pre-trained versions of both bart and t5 models to generate the multihop questions, leveraging the filtered hotpotqa dataset, following the preprocessing approach outlined in the mulqg @xcite . the gat and the transformer models (bart and t5) were trained end-to-end. during training, gradients were backpropagated through both the gat and the transformer model, allowing the entire network to learn jointly. this approach enables the model to effectively incorporate the structural information captured by the gat into the question generation process. our candidate pre-trained models for the proposed architecture were the bart-squad-qg-hl foot_1 version of the bart model and the t5-base-finetuned-question-generation-ap foot_2 version of the t5 model. these models were selected due to their proven efficacy in question generation tasks and their availability for fine-tuning on domain-specific datasets. ## conclusion in this work, we introduced gnet-qg, a graphbased approach to multi-hop question generation that effectively reduces model complexity without sacrificing performance. by explicitly modeling semantic relationships between entities and enriching the input context for transformer-based models, gnet-qg addresses the limitations of existing methods. a key contribution of gnet-qg is its ability to reduce the model size by approximately 7.5 million parameters compared to cqg, a highly competitive model in this space. this substantial reduction leads to improvements in computational efficiency, lower memory usage, and faster inference speeds, making gnet-qg a more practical and scalable solution for real-world applications. despite the smaller parameter size, our experimental results demonstrate that gnet-qg outperforms cqg in terms of the quality of generated questions, highlighting its effectiveness and efficiency. furthermore, we validated the versatility of our architecture by integrating well-known sequenceto-sequence frameworks such as bart and t5. the consistent performance improvements across these models underscore gnet-qg's compatibility with different large language models and its ability to generate high-quality, complex multi-hop questions requiring sophisticated reasoning over multiple interconnected pieces of information. future research could focus on enhancing the model's reasoning capabilities to better address abstract or causal questions and extending its application to other nlp tasks like summarization or machine translation. additionally, exploring gnet-qg's performance on non-english datasets could unlock its potential for multilingual question generation, further broadening its impact.