R^3 - NL 2 GQL : A Model Coordination and Knowledge Graph Alignment Approach for NL 2 GQL (2024)

Abstract: While current tasks of converting natural language to SQL (NL2SQL) using Foundation Models have shown impressive achievements, adapting these approaches for converting natural language to Graph Query Language (NL2GQL) encounters hurdles due to the distinct nature of GQL compared to SQL, alongside the diverse forms of GQL. Moving away from traditional rule-based and slot-filling methodologies, we introduce a novel approach, R^3 -NL2GQL, integrating both small and large Foundation Models for ranking, rewriting, and refining tasks. This method leverages the interpretative strengths of smaller models for initial ranking and rewriting stages, while capitalizing on the superior generalization and query generation prowess of larger models for the final transformation of natural language queries into GQL formats. Addressing the scarcity of datasets in this emerging field, we have developed a bilingual dataset, sourced from graph database manuals and selected open-source Knowledge Graphs (KGs). Our evaluation of this methodology on this dataset demonstrates its promising efficacy and robustness.

## introduction graph-based data structures are central to diverse areas such as financial risk management, social networking, and healthcare @xcite . to manage this data efficiently, graph databases are widely used, offering an effective means to represent and store complex, interconnected information @xcite . despite their utility, the intricacy of gql poses a challenge for those not specialized in the field, making it hard to leverage graph databases for data analysis and application development. meanwhile, although numerous nl2sql approaches where, having, order by, join expression count(), max(), strcasecmp(), timestamp(), properties() sum(), ceil(), abs(), lower(), data() have shown promise @xcite ) @xcite , their direct application to nl2gql is hindered by the differences in focus and syntactic complexity between sql and gql, as shown in regarding information retrieval in kgs, although triplet vector-based retrieval methods @xcite offer efficiency and accuracy, they compromise the graph's structural integrity, limiting their utility in complex queries. in contrast, gql-based methods maintain rich data and logical pathways, bridging the conversational and datastructured worlds, and enhancing the model's interactivity and interpretability, as shown in figure 1 . therefore, implementing a system for the nl2gql task has become particularly important, but the progress in nl2gql has been modest, with efforts predominantly concentrating on the cypher (one type of the gql). many solutions, such as text2cypher, a python library, use template-based sony panasonic apple samsung tim cook apple hold_stock related_comp any_industry upstream … graph db gql apple inc apple inc. company_ introduction apple inc. is a globally technology… listing_date december 12, 1976 company_ name apple inc. amount $10 billion company_id aapl query i want to see the equity chain of apple. methods to transform natural language into cypher, ensuring syntactic correctness but requiring extensive customization for specific data schemas. more recently, spcql @xcite introduced the text to cypher task and developed the first dedicated dataset, using seq2seq models as a baseline. however, this approach has only achieved a 2% success rate in generating accurate cypher queries, indicating significant potential for improvement, while the lack of schemas makes this dataset difficult to apply in real-world environments. the challenges in nl2gql stem from several key factors: 1) multiple model requirements: graph databases complicate gql formulation with their intricate node-edge structures. our experiments have shown that a single small model cannot learn gql syntax through few-shot or finetuning. larger models, although better at generalizing across schemas, often struggle to align with the specific schemas or data elements within graph databases, leading to errors or hallucinations, making it difficult to solve the nl2gql task with a single model. 2)limited resources: the nascent stage of nl2gql, contrasted with the well-resourced nl2sql field, leads to a scarcity of datasets @xcite @xcite and tools, hampering research and development efforts in this area. to address these issues, we developed r 3 -nl2gql, combining the specialized insights of fine-tuned smaller models with the broad adaptability of larger ones. the smaller model acts as a ranker and rewriter, while the larger model refines the gql generation. we also integrated original kg data to optimize alignment, aiming to improve the larger model's zero-shot performance. facing a lack of nl2gql datasets, we created a bilingual dataset with thousands of high-quality entries, marking a novel application of foundation models in nl2gql, the link to the project is https://github.com/zhiqix/nl2gql . we summarize our contributions as follows: ## task formulation to address the challenge of information loss in natural language schema representations, we devised a novel approach for schema and query formulation in the context. ## r -nl2gql framework the r 3 -nl2gql framework pioneers a coordination strategy, merging several models to mitigate the limitations of relying on a single model, as illustrated in figure 3 . the process initializes with a finely tuned smaller model serving as a ranker, excel at identifying key components like crud operations, clauses, and schema classes from the input. to tackle the alignment challenge, another smaller model leverages few-shot learning to fetch and validate information against the graph database, functioning as a rewriter to guarantee data precision. the outputs of these models are then further honed by a larger model, tapping into its sophisticated generalization and synthesis capabilities to ultimately generate accurate gqls. ## data design in contrast to the numerous open-source datasets for nl2sql tasks, such as spider and kaggled-bqa @xcite , gql is deficient in largescale, diverse-schema datasets that meet real-world industrial requirements. most existing datasets predominantly focus on cypher, making it challenging to create a dataset for gqls. to address this gap, we developed a multischema dataset for nl2gql. leveraging foundation models' proficiency in generating cypher, we choose ngql for our research to evaluate our approach. this section outlines our methodology for defining gql generation tasks and synthetic data generation, as shown in figure 5 . ## experiment we introduced a multi-tiered evaluation system for nl2gql tasks, covering aspects from syntax to semantics. utilizing the dataset, we test the performance of our framework against gpt family counterparts. ## conclusion our study presents a novel model coordination framework designed for the nl2gql task, leveraging the complementary strengths of larger and smaller foundation models. by delineating clear roles for each model, we markedly improve the nl2gql conversion. additionally, the development of a gql-specific bilingual dataset underscores the superior performance of our framework. these results pave the way for future advancements in the field of nl2gql, offering a robust founda-tion for further exploration and development.