Application of Generative AI as an Enterprise Wikibase Knowledge Graph Q & A System (2024)

Abstract: Generative AI and Large Language Models are increasingly used in business contexts. One application involves natural language conversations contextualized by company data, which can be accomplished by Enterprise Knowledge Graphs, standardized representations of data. This paper outlines an architecture for implementation of an Enterprise Knowledge Graph using open-source Wikibase software. Additionally, it is presented a Knowledge Graph Q&A System powered by Generative AI.

## introduction knowledge graphs (kg) are semantic networks that represent information in a graph structure, with entities as nodes and relationships as edges @xcite , built from diverse data to integrate and organize knowledge @xcite . they can be applied in areas such as the labor market @xcite , education methods @xcite , and medicine @xcite , and are valued in artificial intelligence (ai) for their clarity and flexibility @xcite . an example is the combination of kg with ai technologies, such as microsoft's azure openai, which further enhances their potential by facilitating the integration and analysis of large volumes of data more efficiently and accurately @xcite . in this context, the use of wikibase to create kg offers significant advantages. wikibase allows the integration of heterogeneous data, flexible data schema modeling, and collaborative knowledge curation, enabling the construction of comprehensive and up-to-date graphs for applications such as recommendation, analysis, and research @xcite . in brazil, kg have driven advancements in areas such as smart cities and healthcare @xcite . despite the challenges, research is exploring their potential, such as the brazilian legislation and brazilian history @xcite . large language models (llms) are reshaping the way humans interact with machines, specially through generative ai applications. known for their immense scale and intricate architecture, llms have transformed the field of natural language processing. these models undergo rigorous stages, including data gathering, preprocessing, model selection, training, and fine-tuning, all aimed at achieving peak performance @xcite . presently, experts are exploring the gen ai capacity to redefine a company's valuation and improving its cost structure, which can fully impact several business in the future @xcite . although the progress in these models is promising, they do come with limitations. large language models struggle to expand or modify their memory, lack transparency in their predictions, and may even generate "hallucinations." however, models that blend training data with company data (retrieval-based) can mitigate some of these challenges @xcite . this technique is called retrieval-augmented generation (rag) and allows expansion of knowledge, as well as inspection and interpretation of accessed information @xcite . mackenzie presbyterian institute (ipm) maintains one of the oldest institutions of education in brazil, founded in 1870. its structure encompasses a university, with campuses in 6 brazilian cities and with about 37,000 students enrolled, a school with about 9,000 students enrolled, and two hospitals, which together provide more than 2.3 million health care encounters and procedures in 2022 @xcite . in a corporate context such as ipm, data integration was being addressed with the use of datawarehouses and datalakes, trying to offer a 360-degree view of the customer and allow the board to have an integrated view of the company. however, the growing understanding of student interactions, patients and the various services offered by ipm seems to be naturally represented by a graph and well documented through business glossaries and ontologies, suggesting that knowledge graphs can offer a more complete and integrated experience of ipm data @xcite . the possibility of integrating large language models (llm) with knowledge graphs (kg) opened a new horizon for offering data to ipm end users: the possibility of asking questions about the integrated data and being answered in natural language. but how complex would it be to implement this solution for ipm? and what would be the results of the interaction of an llm with ipm data? these are some of the questions we need to answer. in this article we will detail a rag q&a system that accesses data from an enterprise knowledge graph based on wikibase, created to integrate company data. for the experiments, the graph was loaded with synthetically generated students and patients data, thus preserving the identity and privacy of both patients and students. this article is structured as follows: section 2 depicts the entire solution of the knowledge graph, including its ontologies, its data, and its architecture; section 3 discuss the tests and results obtained when adopting llm to build the q&a system; section 4 concludes the article and presents contributions, limitations, and future improvement opportunities. ## knowledge graph the kg that is being built for our company can be defined as an enterprise knowledge graph (ekg), as it is restricted to corporate use and is applied to commercial use cases. the objectives for building a kg in our context include: gain insights into students' relationships with courses, teachers, subjects and content, as well as patients' relationships with treatments, medications and medical procedures; integrate data from different sources; build the foundation of what will become a semantic data catalog, and build the foundation that supports data analysis. an important concern for our company was to provision an ekg that could demonstrate its data integration potential in the shortest possible time and in a performant manner, or, as in the words of @xcite , "deliver the right data in the right format in a timely and highperformance manner". in this sense, wikibase proved to be more advantageous compared to classic rdf kg solutions, as it offers out-of-the-box services @xcite . in just a few days of work, we had a fully operational sandbox environment provisioned on docker containers, including a sparql endpoint, a full-text search solution for concepts and attributes, and a graphical interface for sparql queries. the fact that wikibase is a solution for open knowledge graphs (okg) is still a concern for us because wikidata does not allow restricting data access according to user groups and, in a corporate environment, users should only access data relating to their activities. at least intuitively this concern can be addressed by using extensions @xcite or even developing one. in the next sections we will discuss the kg components, including the ontologies, data and technologies adopted in its construction. ## large language models we will also discuss the tests and results obtained when adopting llm to build a kg questions and answers system. ## conclusion even at the prototype stage, the knowledge graph q&a system demonstrated a good hit rate, specially in simple questions. as illustrated in figure 4 , the re-act agent performed significantly better with type 1 questions when connected to our private knowledge graph, yielding 31% more correct answers. this improvement can be attributed to our enterprise knowledge graph (ekg) being restricted to subjects of interest for our company, as opposed to wikidata, which covers a wide range of topics. the discrepancy shown in figure 4 between local ekg and wikidata for type 2 questions are possibly due to training data about the wikidata, since several pids and qids are known by gemini 1.0 pro. occasionally, the answers generated by the re-act agent connected to our private ekg contain some wikidata properties, resulting in empty results from our sparql endpoint. using a more robust llm like gemini 1.5 pro or gpt-4 and fine tuning the model to better generate sparql may fix this problem and improve the success rate. when deploying the chatbot to production, this will be our approach. also, when using premium llm versions, the providers guarantee that no private data are used to train models, making them suitable to a production version. on privacy, an opportunity to improve the kg is the segregation of access to items based on the authenticated user's profile, and the reproduction of this segregation of access for the sparql queries submitted to the sparql endpoint and to the wikibase query service (wdqs). the results obtained so far have been favorable to the adoption of kgs as a solution for data integration and llm for the construction of a search and response interface in natural language for ipm's corporate data. while adopting an open knowledge graph solution like wikibase in an enterprise environment presents the challenge of segregating data access, the gains from wikidata query service's out-of-the-box data visualization options and simplicity of horizontal scaling of wikibase's docker swarm implementation are self-demonstrable.