Schema and Natural Language Aware In-Context Learning for Improved G raph QL Query Generation (2025)

Abstract: GraphQL offers a flexible alternative to REST APIs, allowing precise data retrieval across multiple sources in a single query. However, generating complex GraphQL queries remains a significant challenge. Large Language Models (LLMs), while powerful, often produce suboptimal queries due to limited exposure to GraphQL schemas and their structural intricacies.Custom prompt engineering with in-context examples is a common approach to guide LLMs, but existing methods, like randomly selecting examples, often yield unsatisfactory results. While semantic similarity-based selection is effective in other domains, it falls short for GraphQL, where understanding schema-specific nuances is crucial for accurate query formulation.To address this, we propose a Schema and NL-Aware In-context Learning (SNAIL) framework that integrates both structural and semantic information from GraphQL schemas with natural language inputs, enabling schema-aware in-context learning. Unlike existing methods, our approach captures the complexities of GraphQL schemas to improve query generation accuracy.We validate this framework on a publicly available complex GraphQL test dataset, demonstrating notable performance improvements, with specific query classes showing up to a 20% performance improvement for certain LLMs. As GraphQL adoption grows, with Gartner predicting over 60% of enterprises will use it in production by 2027, this work addresses a critical need, paving the way for more efficient and reliable GraphQL query generation in enterprise applications.

## introduction graphql is a powerful query language and runtime for apis, offering a flexible alternative to rest by allowing clients to request precise data from interconnected sources in a single query. at its core, graphql relies on a schema that defines object types, their relationships, and supported operations (queries and mutations). while this schema-driven approach enhances flexibility and efficiency, its complexity in larger systems can make generating accurate queries challenging. according to a gartner report @xcite , by 2027, over 60% of enterprises are expected to use graphql in production, up from less than 30% in 2024. this rapid adoption underscores graphql's growing significance and the need for researchers and developers to address challenges in scalability, and usability. these advancements are essential for graphql to meet the evolving demands of modern enterprises. large language models (llms) can assist by generating graphql queries from natural language (nl) inputs, leveraging the schema to fulfill user requests. however, as noted in @xcite , the scarcity of publicly available graphql datasets limits llm exposure to schema-specific patterns, reducing their effectiveness in producing valid queries. incorporating in-context examples in prompts has been shown to improve performance, but selecting these examples effectively is critical. existing methods for few-shot selection typically rely on semantic similarity between the input query and nl representations of the few-shot examples. however, in the context of graphql, the schema's structure and relationships play a pivotal role in query formulation. this raises a key research question: can the graphql schema be leveraged to refine few-shot selection and enhance contextual relevance? in this paper, we investigate this pivotal question and propose a novel framework that significantly enhances llm performance for graphql query generation. ## related work graphql has gained significant attention in academia and industry for its flexibility and efficiency in managing data interactions. while studies have explored the advantages of graphql over rest apis-such as reduced client-server interactions @xcite , improved maintainability @xcite , and optimized data fetching @xcite )-technical challenges remain. for example, @xcite investigates testing method-ologies for query validation, while @xcite examines its capacity to represent complex data structures. graphql's role in real-world applications, including data integration across heterogeneous sources, is highlighted in @xcite . recent efforts to leverage large language models (llms) for graphql query generation include several notable approaches @xcite @xcite . however, the introduction of a graphql-specific dataset in @xcite marks the first attempt to systematically address training and evaluation for such tasks. despite this, the study does not fully address the need for adaptive few-shot learning techniques that incorporate both semantic and structural schema characteristics. existing query generation systems typically rely on semantic similarity between natural language (nl) queries and examples, overlooking the critical role of schema structure. in summary, while prior research highlights graphql's strengths and challenges in api management, our work improves the graphql query generation performance of the state of the art llms by introducing an adaptive few-shot learning framework. this approach bridges gaps in existing methodologies, enabling llms to better handle the complexity and diversity of real-world graphql schemas. ## proposed snail framework we propose the schema and nl-aware in-context learning (snail) framework (figure 1 ) for generating graphql queries from natural language (nl) utterances. unlike traditional methods relying on semantic similarity, our framework dynamically selects few-shot examples based on both structural and semantic similarities, tailored to each query. as shown in figure 1 , the process starts by sampling and storing k few-shot examples (schemas, graphql queries, and corresponding utterances) from the dataset @xcite . upon receiving test samples, the framework assesses structural similarity between the test schema and fewshot examples, while also evaluating semantic similarity with the test utterance. these similarity met-rics rank and group the examples for in-context learning, which, along with instructions and the input query, is passed to large language models (llms) for final graphql query generation. by incorporating both structural and semantic similarities, snail framework improves the precision and adaptability of query generation.