Sequence Structure Aware Retriever for Procedural Document Retrieval: A New Dataset and Baseline (2025)

Abstract: Execution failures are common in daily life when individuals perform procedural tasks, such as cooking or handicrafts making. Retrieving relevant procedural documents that align closely with both the content of steps and the overall execution sequence can help correct these failures with fewer modifications. However, existing retrieval methods, which primarily focus on declarative knowledge, often neglect the execution sequence structures inherent in procedural documents. To tackle this challenge, we introduce a new dataset Procedural Questions, and propose a retrieval model Graph-Fusion Procedural Document Retriever (GFPDR) which integrates procedural graphs with document representations. Extensive experiments demonstrate the effectiveness of GFPDR, highlighting its superior performance in procedural document retrieval compared to existing models.

## introduction in real life, humans often encounter execution failures when performing procedural tasks such as cooking or handicrafts making, as shown in figure 1 . the causes and solutions to failures can be found in procedural documents (e.g., cookbooks or operation tutorials) containing the used materials with their specific amounts, the sequence of operating steps, operating precautions, and so on @xcite . however, it is often labor-intensive and time-consuming for humans to correctly retrieve the reference procedural documents that enable solving the failures effectively. in our paper, we refer to such retrieval scenario as procedural document retrieval. current retrieval scenarios are predominantly centered around declarative knowledge @xcite . popular retrieval datasets like ms-marco @xcite and natural questions @xcite primarily consist of declarative documents (e.g., wikipedia pages) and queries are simple like "what's the largest ice sheet?". retrievers @xcite mainly focus on capturing the key elements of declarative documents, e.g., subjects, facts, and entities. however, they neglect the execution sequence of steps within procedural document, which provides crucial clues for retrieving relevant documents in procedural document retrieval. for instance, in figure 2 , although both documents contain the same topic and similar step contents (i.e., place eggs, heat water, and simmer), the sequences of these steps are executed differently. document 2 places eggs into water first, then heats the water, whereas document 1 reverses the sequence. this difference in sequence affects the simmering time for different states of eggs. since both the step contents and sequence structure in question: i'm trying to make soft-boiled eggs with jammy yolks, but the yolks i end up cooking are still too runny, and the whites are overly soft. i'm not sure what i'm doing wrong. ## methods task formulation: the procedural document retrieval task aims to search the most relevant documents given a query. formally, given a query q and a collection of procedural documents d, the task is to retrieve top-k relevant documents d @xmath0, ..., d * k from d. graph-fusion procedural document retriever (gfpdr) aims to capture sequence structure from procedural graph and fuse with document embeddings. our method has three components: 1) edge-aware graph attention module, which is designed to construct procedural graphs from procedural documents and extract the sequence structure features from the graphs. 2) graph-fusion document encoder, which encodes procedural documents and integrates with procedural graph. 3) query encoder, a language model for encoding the query. the overall architecture is shown in figure 3 . ## conclusion in this paper, we address the problem of procedural document retrieval, where both step content and execution sequence structure should be considered between queries and documents. while existing retrieval methods primarily focus on learning topical content, they overlook the complex sequence structures inherent in procedural documents. to tackle this, we introduce a new dataset for procedural document retrieval and propose a retriever that integrates procedural graphs with document representations. extensive experiments demonstrate the effectiveness of our approach.