SPOT : Zero-Shot Semantic Parsing Over Property Graphs (2025)

Abstract: Knowledge Graphs (KGs) have gained popularity as a means of storing structured data, with property graphs, in particular, gaining traction in recent years. Consequently, the task of semantic parsing remains crucial in enabling access to the information in these graphs via natural language queries. However, annotated data is scarce, requires significant effort to create, and is not easily transferable between different graphs. To address these challenges we introduce SPOT, a method to generate training data for semantic parsing over Property Graphs without human annotations. We generate tree patterns, match them to the KG to obtain a query program, and use a finite-state transducer to produce a proto-natural language realization of the query. Finally, we paraphrase the proto-NL with an LLM to generate samples for training a semantic parser. We demonstrate the effectiveness of SPOT on two property graph benchmarks utilizing the Cypher query language. In addition, we show that our approach can also be applied effectively to RDF graphs.

## introduction the use of knowledge graphs (kgs) is expanding as a means to store general world knowledge as well as domain-specific knowledge @xcite . in particular, property graphs (pgs) have seen growing adoption in industrial settings. this is mainly because they enable the specification of semantically rich data models for organizing and understanding connected entities and their relations. this is in contrast with the flattened rdf representations, which were the predominant prior paradigm. the information contained in a knowledge graph can be accessed through queries in specialized graph programming languages such as sparql (rdf) and cypher (pgs) @xcite . however, constructing these queries to fulfill a specific information need requires significant expertise. consequently, systems that enable querying kgs via natural language interfaces are becoming increasingly important @xcite . the task of semantic parsing focuses on automatically generating query programs from natural language inputs that express a particular intent. semantic parsing continues to be a challenging and demanding task. typically, developing a semantic parser for a target kg requires the availability of annotated training data. such data is often scarce and acquiring more can be expensive due to the considerable human effort required. moreover, these training datasets are not easily transferable. each kg defines, in principle, its own translation language, with its own schema of specific relations, concepts and properties that require domain-specific annotations. these challenges are further exacerbated when considering property graphs, which have been significantly understudied compared to rdf-style graphs. we believe that automatic data generation is the answer to many of these challenges. in this work we introduce spot (zero-shot semantic parsing over property graphs), a method that can automatically generate training data starting from a knowledge graph, without the need for human intervention. our approach begins with an ungrounded tree pattern comprising a set of anonymous nodes and edges that represent a graph query. we match this pattern to the knowledge graph, producing a grounded version with instantiated entities and attributes. from this grounded tree, we derive the corresponding executable query program. additionally, from the grounded tree pattern, we generate a natural language realization, which we refer to as proto-nl. we employ a finite-state transducer that takes as input a linearized tree and sequentially outputs natural language fragments corresponding to nodes and relations. we then use a large language model to paraphrase the proto-nl into a final natural language realization (nl). by pairing the generated queries with their corresponding nls, we can generate a training data-set that can be utilized to train a semantic parsing model. we demonstrate the effectiveness of spot on property graphs using two different datasets containing queries written in cypher. our results show that with spot we can obtain a semantic parser that significantly outperforms the de-facto alternative used for zero-shot semantic parsing over pgs (i.e. prompting an llms with the nl query and the graph schema). although spot was designed to leverage the characteristics of property graphs, we can also apply it to rdf graphs. our experiments show that in this setting our approach outperforms byokg @xcite , a state-of-the-art method for zero-shot semantic parsing over rdf graphs. the idea of sampling some form of tree pattern directly from the kg for data generation is not novel, a similar approach has been proposed in the context of rdf graphs @xcite . neither is the idea of exploiting the compositional nature of a representation to automatically generate an nl realization, similar ideas have been proposed for semantic parsing over relational databases @xcite . our unique contribution is to combine and extend those ideas to develop an effective data generation approach for pgs; to the best of our knowledge this is the first approach for zero-shot semantic parsing over pgs. our code is available at https://github.com/interact-erc/spot the paper is organized as follows: section 2 provides some preliminaries on knowledge graphs, section 3 describes our data generation approach, section 4 presents our method applied to property graphs, including the experimental setup and results, section 5 applies the same approach to rdf graphs, section 6 describes the related work and finally section 7 concludes the paper. ## preliminaries a knowledge graph (kg) is a structured representation of entities and their relations organized as a graph where nodes represent entities and edges relations. the information stored in the kg can be efficiently retrieved by executing programs (i.e. formal queries) which can be written in different graph query languages. a commonly used kg format popularized in the early 2000s is the resource description frame-work (rdf). rdf graphs consist of entity-classes, entities and relations stored in triplet format. a @xmath0, p, @xmath1, a predicate p and an object o. the predicate p defines the relation (from s to o) that connects two entities of a certain class type. these graphs are queried via the sparql query language. property graphs (pgs) also consist of nodes and edges, but unlike rdf graphs, nodes in pgs are structured objects. in addition to belonging to a class type, they can have an arbitrary list of associated properties, each represented as a key-value pair storing specific information about the entity. in general, pgs are believed to enable the specification of a rich and semantically intuitive data model. this is because they clearly distinguish between entity properties and entity relations, whereas rdf graphs conflate the two concepts: entity properties can only be expressed as relations. formal query languages for pgs include cypher @xcite and gremlin @xcite . appendix d shows a fragment of the schema of one of the pgs used in our experiments. ## generation with tree patterns and finite state transductions figure 1 gives a general overview of our data generation approach which consists of five main steps: in the next sub-sections we describe each of the steps in more detail. ## property graphs experiments 4.1 experimental setup to test our method we generate training data with spot and use it to train a transformer decodermodel. we generate 100k of samples and for each proto-nl we generate 3 paraphrasis (see appendix b for a discussion). to avoid generating samples that are semantically meaningless, we do not generate queries that contain three relations of the same type, 2 or more edge nodes without defined property, nodes that repeat the same entity values, edge nodes of the same class as the answer node without a defined property and parallel nodes of the same class that have one an instantiated property and the other not. the model is trained to predict a formal query for a given natural language question. training uses generated data only, thus it can be regarded as a zero-shot setting. note that to focus purely on the semantic parsing task we assume gold entity linking and we pass the gold entity values in the input. specifically, we use qwen3-4b @xcite and llama-3.2-3b @xcite as our base parsers. for generating the natural lan-guage realization from our proto-nl we employ qwen3-14b. we do an initial parameter selection by reserving a small subset of the data, and then proceed to train on the full dataset. we train the model on a single a100 gpu and every result reported is the average of 5 runs with different seeds. as evaluation metric we report execution accuracy, where a predicted query is considered correct if it returns the same results as the gold query when run on the corresponding kg. for the pgs we focus on the cypher graph language and evaluate on the following datasets: for zograscope we also provide results training our base parsers on the human annotated training set (while dmt does not have one). since, to the best of our knowledge, we are the first ones to explore zero-shot semantic parsing for pgs cypher graphs, we will compare our method with the natural strategies for zero-shot parsing based on llm prompting. the zero-shot setting employs a prompt (appendix f) that includes the description of the graph schema and then asks the llm to provide the cypher query for a given question. additionally, we report results for the few-shot setting, where 5 examples are included in the prompt. for zo-grascope we use the results reported in the original paper. the evaluated models include gpt-4o (gpt-4o-2024-08-06), a large proprietary model, as well as small llms that can be run locally: llama-3.2-3b, qwen3-4b @xcite and mistral-7b-v0.1 @xcite . ## related work recent years have witnessed a growing interest in natural language interfaces to knowledge graphs, focusing mainly in semantic parsing and question answering (kbqa) @xcite @xcite @xcite @xcite @xcite @xcite . some of these works leverage in-context learning approaches @xcite @xcite as well as graph rag @xcite . there seems to be a general consensus that llms do not offer satisfactory performance out of the box @xcite @xcite . zero-shot semantic parsing over knowledge graphs is a relatively newer subject that is starting to gain the interest of the research community, but it remains a relatively unexplored subject area. in the context of zero-shot semantic parsing for rdf graphs, the work that is most related to ours is byokg @xcite . byokg generates data from an rdf kg by a random traversal which generates s-expressions @xcite that are then used to generate natural language realizations via an iterative paraphrasing strategy. the authors test their data generation approach with an in-context learning strategy. besides this zeroshot approach, there have also been some contributions focusing on few-shot semantic parsing for rdf, including @xcite and @xcite . in the context of knowledge bases, there have been some methods: @xcite and @xcite that generate logical forms. however, their setting is not truly zero-shot since the method uses unlabeled queries taken from an annotated training set. strikingly, despite the recent popularity gained by property graphs in the industrial sector, there is relatively little work on semantic parsing specifically dedicated to these types of graphs. of the available literature there are some works that focus on fine-tuning @xcite @xcite or on exploiting large language models @xcite @xcite . there have also been some efforts on mapping rdf graphs to @xcite @xcite . besides these prior literature on pgs, to the best of our knowledge this is the first work specifically dedicated to zero-shot semantic parsing over pgs. ## conclusions natural language interfaces to knowledge graphs and databases have the potential to democratize data-science by enabling people with no programming experience to efficiently and precisely interact with data. the main blocker preventing the development of such interfaces is the training data bottleneck when fine-tuning semantic parsers. in this paper we addressed this bottleneck in the specific context of semantic parsing for property graphs. we presented a data-generation method that leverages the inherent compositional nature of graph queries. one key observation is that although the mappings between queries and their natural language realizations can be very complex, there might still exist one "simpler" natural language realization that can be automatically generated as a monotonic transduction of an intermediate tree pattern representation. then we can use llm paraphrasing to generate the more "complex" realizations. our experiments show the potential of the proposed approach and bring us a step closer in developing zero-shot semantic parsers that have actual practical use as industrial applications.