# CAPRAG: A Large Language Model Solution for Customer Service and Automatic Reporting using Vector and Graph Retrieval-Augmented Generation

H Landolsi, Kais Letaief, Nizar Taghouti, I Abdeljaoued-Tej

► To cite this version:

H Landolsi, Kais Letaief, Nizar Taghouti, I Abdeljaoued-Tej. CAPRAG: A Large Language Model Solution for Customer Service and Automatic Reporting using Vector and Graph Retrieval-Augmented Generation. University of Carthage, ESSAI, Ariana, Tunisia; LR16IPT09, Institut Pasteur de Tunis, Universsity Tunis-El-Manar, Tunisia. 2025. hal-04889831

HAL Id: hal-04889831

https://hal.science/hal-04889831v1

Submitted on 16 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
CAPRAG: A Large Language Model Solution for
Customer Service and Automatic Reporting using
Vector and Graph Retrieval-Augmented Generation

Hamza Landolsi¹,², Kais Letaief¹, Nizar Taghouti¹, Ines Abdeljaoued-Tej²,³

¹INETUM Tunisia, Immeuble Harbour, Avenue de la Bourse, Tunis 1053, Tunisia,
²Engineering School of Statistics and Information Analysis, University of Carthage, Ariana, Tunisia,
³Laboratory of BioInformatics bioMathematics, and bioStatistics (LR24IPT09), Institut Pasteur de Tunis,
University of Tunis El Manar, 13, place Pasteur, B.P. 74, Belvédère, 1002 Tunis, Tunisia

Correspondence: hamza.landolsi@essai.ucar.tn

Abstract

The introduction of new features and ser-
vices in the banking sector often overwhelms
customers, creating an opportunity for banks
to enhance user experience through financial
chatbots powered by large language models
(LLMs). We initiated an AI agent designed to
provide customers with relevant information
about banking services and insights from an-
nual reports. We proposed a hybrid Customer
Analysis Pipeline Retrieval-Augmented Gen-
eration (CAPRAG) that effectively addresses
both relationship-based and contextual queries,
thereby improving customer engagement in the
digital banking landscape. To implement this,
we developed a processing pipeline to refine
text data, which we utilized in two main frame-
works: Vector RAG and Graph RAG. This dual
approach enables us to populate both vector
and graph databases with processed data for
efficient retrieval. The Cypher query compo-
nent is employed to effectively query the graph
database. When a user submits a query, it is
first expanded by a query expansion module
before being routed to construct a final query
from the hybrid Knowledge Base (KB). This
final query is then sent to an open-source LLM
for response generation. Overall, our innova-
tive, designed to international banks, serves
bank's customers in an increasingly complex
digital environment, enhancing clarity and ac-
cessibility of information.

# 1 Introduction

Banks all over the world are constantly introduc-
ing new features, delivering new financial services
and providing their customer with clear booklets
to guide them on how to use their mobile applica-
tions, how to benefit from all their available ser-
vices, how to navigate their websites, etc. How-
ever when banks grow, even with the most straight-
forward user experience and with their efforts to
make all informations accessible via promotions
and accessible website, the task of naviagting and

being informed with updates made the user lost in
a maze (Ravalimanana, 2020; Shumba, 2023). As
the banking industry continues to embrace digital
transformation, the potential applications of finan-
cial chatbots with Large Language Models (LLMs)
are limitless presenting an exciting opportunity for
banks to innovate and differentiate themselves in a
rapidly changing market (Eisfeldt et al., 2023; Ooi
et al., 2023; Abbott and Smith, 2023).

In fact, LLMs and GenAI have revolutionized the
field of AI, allowing machines to create diverse
content such as text (Pandey et al., 2024), images
(Elasri et al., 2022), music (Dong, 2024) and videos
(Zhou et al., 2024). Unlike discriminative models
that classify, GenAI models generate new content
by learning patterns and relationships from human-
created datasets (Decardi-Nelson et al., 2024).

RAG framework is a solution to feed concise rele-
vant context similar to the question and then passed
to LLM to answer to a given query (Lewis et al.,
2020). This come as a solution to overcome the
context window of LLMs limitations. Recently a
graph RAG-based approach (Procko and Ochoa,
2024) led to major improvements. This method
seeks to replicate the way our minds think and rea-
son, similar to the functioning of a graph. When
faced with a question that demands deep thought
and reasoning, we begin to model the relationships
among key factors, entities, and agents that may
impact our area of interest. Many experts from
the industry claim that agentic RAG is the future
(Nguyen et al., 2024): It aims to deliver a stan-
dalone solution that can be independent relying
on multiple agents, specializing each in a defined
workflow. With agentic style we can combine pro-
cess and do multiple actions over a lot of sources
and interact seamlessly with multiple agent or "ex-
pert" in their task. It leads to more context aware
and more adaptability (Immorlica et al., 2024). In
this work, we designed an AI agent tasked to pro-
vide the customers with information related to the
bank financial services, features, and key insights about the company through its annual reports. We obtained a hybrid approach using Vector and Graph RAG, to handle efficiently the nuances of both relationship based and contextual questions. Instead of thinking in a linear way that isolates each element, we connected various elements and asked questions that relate hidden factors. After iteratively refining our experiments and testing each component we conclude to our solution Customer Analysis Pipeline RAG (CAPRAG). We discussed in next sections each workflow separately and the challenges and limitations of our solution. In Figure 5 we gave the big picture of the workflow and we mentioned a Vector and Graph RAG pipelines.

## 2 Data source and description of bank A

The data used in this study is derived from publicly available documents and corporate publications pertaining to an international bank, that we anonymized as A. Data sources include SEC filings, brochures and booklets.<fnref n="1" /> SEC filings contain comprehensive financial statements, management discussions, and risk factors, providing a detailed overview of the bank's financial performance and strategic outlook. From the SEC filings, we find annual reports containing qualitative data, such as bank A's strategic initiatives, corporate social responsibility (CSR) activities, and customer engagement programs. These qualitative insights offer a broader understanding of the bank's positioning and operational strategies beyond the financial statements. We extracted financial metrics such as total assets, net income, return on equity (ROE), and risk-weighted assets over a period of five years (2019–2023). These metrics are pivotal in evaluating the financial health of bank A and its performance relative to its peers. In addition to regulatory filings, bank A publishes various brochures and booklets that provide insights into its business operations, service offerings, and corporate governance. The brochures and booklets provided complementary information about bank A and that is very important for customers, providing every financial service offered by the bank, the criteria required, some frequent asked questions, and other relevant information.

Providing insights about these KPIs is essential for customers and the ones who are interested in in-

vesting in bank A as well. These materials, aimed at both clients and stakeholders, contribute to a qualitative data that complements the financial information.

Each of these sources provides unique insights into bank A's operations and strategy, and together they form a comprehensive dataset for analysis in this study (Table 1).

## 3 Evaluation pipeline: LLM as a judge

There are several ways to monitor and track the evolution of our work, for this project we used LLMs to help us rate and assess our metrics of interest. This technique is called "LLM as a judge".

We setup an evaluation pipeline in an automatic way. First of all, we choose the metrics that align the best with our challenges. To ensure the effectiveness and reliability of the RAG model, it is essential to evaluate its outputs using specific metrics. The three metrics we focus on are answer relevance, context relevance and groundedness.

**Definition 3.1 (Answer Relevance)** *Answer relevance measures how accurately the generated response addresses the given query. A high answer relevance score indicates that the response is directly related to the question and provides useful information.*

**Definition 3.2 (Context Relevance)** *Context relevance evaluates the extent to which the generated response aligns with the context provided. This metric ensures that the response is coherent within the given context and maintains the flow of the conversation or narrative.*

**Definition 3.3 (Groundedness)** *Groundedness assesses whether the generated response is based on factual information retrieved from reliable sources. A grounded response is one that can be traced back to authoritative references, ensuring the accuracy and trustworthiness of the information.*

The evaluation pipeline plays an important role in the development process. By systematically assessing the model's outputs based on answer relevance, context relevance, and groundedness, we can ensure that the model generates high-quality, accurate, and coherent responses (see Figure 1). We improve the user experience and builds trust in the system's capabilities.

<footnote n="1">¹SEC Filing are mandatory disclosures submitted by bank A to the U.S. Securities and Exchange Commission.</footnote>
<table><caption>Table 1: Summary of data sources and extracted information</caption><thead><tr><th>Data Source</th><th>Type of Data</th><th>Extracted Information</th></tr></thead><tbody><tr><td>SEC Filings</td><td>Quantitative</td><td>Financial metrics (ROE, assets, net income) / Strategic initiatives, CSR activities</td></tr><tr><td>Brochures and Booklets</td><td>Qualitative</td><td>Financial products/services, (loans, insurance, branches, etc.)</td></tr><tr><td>Annual Reports</td><td>Quantitative<br>Qualitative</td><td>Financial performance, governance insights</td></tr><tr><td>Sustainability Reports</td><td>Qualitative</td><td>ESG goals, sustainability programs</td></tr></tbody></table>

<figure><img src="image_2.png" /><figcaption>Figure 1: Evaluation pipeline with LLM as judge</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Figure 2: Vector RAG enhancements</figcaption></figure>

## 4 Vector RAG: Methodology and Results

As our main focus is improving customer service efficiency in the banking industry, we leveraged advanced chunking optimization techniques, query expansion, and retrieval enhancement strategies. The goal is to improve the accuracy and speed of responses to customer inquiries by optimizing how Knowledge Base (KB) articles are retrieved and processed. In this Section we detailed further the Vector RAG component (See Figure 2).

We first explored various chunking optimization techniques to segment large KB into smaller, more relevant pieces for retrieval (we have chosen the semantic chunking for our solution). Second, we utilized query expansion strategy to refine customer queries, making them more robust and comprehensive for retrieval systems. These techniques allow for a better match between the customer query and relevant chunks. For the retrieval system, we compare different retrieval enhancement techniques tailored to the banking domain. We choose finally the FlashRank re-ranker (Damodaran, 2017) as a post retrieval enhancement strategy.

We used Zephyr for generation in our Vector RAG

mechanism.<fnref n="2" />

The bar plot in Figure 3 compares the performance of various techniques used on manually created dataset to optimize the Retrieval-Augmented Generation (RAG) model across the three evaluation metrics: Answer relevance, Context relevance, and Groundedness.

**Baseline RAG.** As expected, the baseline configuration shows moderate scores across all metrics. While it performs fairly well in terms of Answer relevance (about 0.75), the Groundedness score is significantly lower (around 0.55), which indicates that the model has room for improvement when it comes to grounding answers in retrieved documents.

<footnote n="2">²(Tunstall et al., 2023) we aim to achieve comparable results with other big LLMs which offer enterprise-ready solutions in terms of scalability, security, and license compliance, especially relevant in the banking sector.</footnote>
<figure><img src="image_4.png" /><figcaption>Figure 3: Vector RAG evaluation</figcaption></figure>

**Chunking Optimization.** Introducing chunking optimization leads to an increase in both Answer relevance and Context relevance. The Answer relevance improves slightly compared to the baseline, suggesting better handling of long documents by splitting them into smaller chunks. However, the Groundedness score slightly decreases, implying that chunking may inadvertently lead to less reliance on the retrieved documents for the final answer.

**Enhancing Retrieval.** This optimization strategy yields the best overall Answer relevance score, exceeding 0.8, indicating that improved retrieval techniques help the model generate more accurate answers. However, similar to chunking, the Groundedness remains relatively lower, indicating that although retrieval improves answer quality, it may not always improve the model's alignment with retrieved evidence.

**Query Translations.** This technique leads to a balanced improvement across all metrics. Answer relevance remains high, and Context relevance sees a boost, indicating that better query formulation helps retrieve more relevant context. Additionally, Groundedness improves compared to the baseline, suggesting that query translations contribute to generating more factually grounded answers.

Overall, the results indicate that while optimizations like chunking and enhanced retrieval can improve answer quality, further work may be needed to improve the Groundedness of the model's responses, ensuring the generated answers align more closely with the retrieved documents. Techniques like query translations show promise in improving performance across the board.

# 5 Graph RAG

The first step in creating a powerful graph knowledge base is to model the data in the most appropriate way (to capture the complex relationships between chunks). The graph should be created with nodes (representing the entity) and edges (the relationship between two nodes).

## 5.1 Schema Construction

We first created nodes of label Document, where in each case we could store the document name and other important metadata as properties. The second type of node is Section, where we store sections of each document. Since documents can have sections and many hierarchical subsections, we can handle this by adding two relationships:

1. Section to Section relationship: **UNDER-SECTION** that link the subsection to its parent section;
2. Section to Document relationship: **HAS-DOCUMENT** which can link sections to the documents they are located in.

This gives a comprehensive overview for our hierarchy of the corpus layout to our knowledge base. The third type of node is Chunk. Here we made special node for Table chunks, so we preserve only table related information and separate them from other information (without losing the scope and the context around the table). We can do so by establishing new edges: **HAS_PARENT**, **CONSECUTIVE**, **PERSON_LINK**, **PRODUCT_LINK**, etc. For example, the relationship between Table and Section is given by **HAS_PARENT**; *idem* for Chunk to Section relationship. The Chunk to Chunk relationship is given by the edge **LINKED** (that link each chunk with the next chunk in corpus). This can be a solution for the problem of lengthy chunks that lead to loss of information. When we divide them to two chunks carrying the information of the same product or service (information loss or chunk corruption). The Table to Chunk relationship is given by **CONSECUTIVE**, which links Tables to their next and previous Chunks or other Tables.

Let focus on some of the powerful entities deemed insightful for our use case. Figure 4 visualizes our expanded graph at this stage. Person is often a very important entity in our corpus that we should focus on. With the length of our chunks, this crucial information may be diluted in the
<figure><img src="image_5.png" /><figcaption>Figure 4: Expanded graph schema using Graph Knowledge</figcaption></figure>

embedding process hence we lose this information. For the Location entity, we focused on the geospatial coordinate (Longitude and Latitude) rather than only working with its semantic vector representation. We made the LLM responsible for text-to-cypher mission the ability to construct Cypher queries capable of solving spatial metric related questions.

Creating a knowledge graph - ready to integrate with our RAG framework - passed through modeling our database schema and populating our graph database.

## 5.2 Methodology

Although semantic similarity is very powerful, it might fail also to identify other types of similarity such as geospatial similarity. For example, the user query might be: "I am in Ariana and I am wondering what's the nearest bank A.<fnref n="3" /> ATM branch to me?". The complexity of the graph database come with more burden and challenges. Having vector representation of embedded user question doesn't help us to query our knowledge graph. Thus, we applied LLMs for generating Cypher queries from the user query.⁴ Achieving this task can be through choosing a powerful LLM, expanding its context window with tailored prompt-

ing. The option to go for this use case is often one of the top performing LLMs in the leaderboard. Many prompting techniques have emerged to make the best of their capabilities (Wei et al., 2022; Zhang et al., 2022; Prompting Guide, 2022; Tian et al., 2024). One considerable limitation of this is that LLM try to follow our instructions and give us relevant and most of the time very accurate results. But when it comes to specifying a valid rigid format, it often fall to respect it.

For example, in our use case, we feed LLM user query and graph knowledge database schema as input with a prompt template guiding it to output a valid Cypher query in string format. As we didn't have enough resources to fine tune our own custom model or use some specialized closed source models so we designed a Cypher repository, each Cypher template in this repository with a description so that we can decide if the user query can be answered via this Cypher or not.

# 6 Customer Analysis Pipeline RAG (CAPRAG)

We aim to setup a processing pipeline that refine the text data. Than we simultaneously used this processed data in two main pipelines Vector RAG and Graph RAG, see Figure 5. Finally we populated respectively the vector database and graph database with the final ready data to use for retrieval. The router is a LLM prompted to divide the query

<footnote n="3">³We took bank A as our fictive bank customer
⁴Cypher is the language used to query a graph database, such as Neo4j (Miller, 2013).</footnote>
<table><caption>Table 2: Comparison between Vector and Knowledge RAG output</caption><thead><tr><th>User query</th><th>Vector KB (Knowledge Base) based RAG</th><th>Graph KB RAG</th></tr></thead><tbody><tr><td>Can you explain the professional background and career progression of the key executives mentioned in the bank A annual reports? Do they have any common traits or experiences that have shaped their leadership at bank A?</td><td>Based on context provided, the key executives include: Maria R. - Chief Executive Officer Jason Q. - Group Financial Director Peter M. - Deputy CEO. They have played significant roles in the company's strategic decisions and overall performance. For more detailed information, please refer to the bank A website or contact the team. There is no specific information available about common traits or experiences.</td><td>From the bank A annual reports, we have identified the following details regarding the professional background and career progression of key executives, as well as their common traits: Maria R. - Chief Executive Officer Professional Background: Maria R. has a distinguished career in both the public and private sectors. She previously served as the Director-General of South Africa's National Treasury. Career Progression: Maria joined bank A in 2009 and has since led several key initiatives, including the transformation of the bank's digital strategy and expansion into new markets. Jason Q. - Group Financial Director Professional Background: Jason Q. is a qualified Chartered Accountant with over 20 years of experience in the financial sector. Before joining bank A, he held senior positions at Deloitte. Career Progression: Jason has been with bank A since 2008, initially as the Head of Finance for Retail and Business Banking before being appointed Group Financial Director in 2016. Peter M. - Deputy CEO Professional Background: Peter M. has an extensive background in both corporate leadership and media. Prior to bank A, he was the CEO of Tiger Brands and the SABC. Career Progression: Joining bank A in 2016, Peter has been pivotal in driving the bank's growth across the African continent. Common Traits and Experiences Public and Private Sector Experience: Both Maria R and Peter M. have extensive experience in both public and private sectors, which has contributed to their versatile leadership styles. Leadership in Transformation and Expansion: All three executives have played significant roles in transformative projects and expansion initiatives within bank A, showcasing their strategic leadership and vision.</td></tr></tbody></table>

or reformulate it to each of the next components the graph RAG and the Vector RAG.

The Cypher query component is responsible for querying graph database, and we tried several techniques to handle text-to-cypher problem (Francis et al., 2018).<fnref n="5" /> So when a user enter his question, we pass the query to a Query Expansion module that will enrich our user query and send it to a router where he constructs a query from the hybrid knowledge base and finally send the final prompt to the open source LLM for generation.

<figure><img src="image_6.png" /><figcaption>Figure 5: Workflow of CAPRAG</figcaption></figure>

We used a decoder only based LLM architecture

(Brown, 2020). We employed HuggingFace H4/zephyr-7b-beta model checkpoint, configured to generate text (Jain, 2022), via HuggingFaceEndpoint. It is configured with a maximum length of 2000 tokens and a temperature of 0.01, which means we are encouraging the LLM to be more deterministic and not allowing it to be creative.<fnref n="6" /> In our use case, the LLM sticks to the provided context and reduces the hallucination as much as possible, see Table 2.

## 7 Discussion

In this study, we emphasized that using embedded user queries to retrieve chunks based solely on cosine similarity with their vector representation is limited in capturing complex relationships and geo-spatial information. Our findings highlighted that despite there are several automatic expansion techniques, opting for manual expansion can be optimal and ideal especially for a specific industry. While a lot of researchers has used graph RAG for an ontology based graph construction in banking sector, (Kang and Cheung, 2023) investigated novel methods for extracting accurate insightful entities, some others have introduced the relationship between sections and structure of the document hierarchy. Other research works focused on the

<footnote n="5">⁵We utilized a static approach due to hardware requirements and private source LLM provider API keys restrictions.</footnote>

<footnote n="6">⁶This is a hyper-parameter for adjusting the creativity of generation, i.e. pushing the model to be greedy when it comes to choose the highest probability of next predicted token.</footnote>
geospatial data and did awesome work to enhance
embedding capturing both semantic and spatial
meaning (Hu et al., 2024) or even customizing
agentic workflow to interact with geographic data
(Ning et al., 2024).

Our findings is complementing the work of (Sarmah et al., 2024) aiming to enhance the questions related to spatial terms, that needs other metrics of similarity rather than the semantic oriented like cosine similarity. For that, we are letting a router LLM to think what metric to choose and which Cypher query to run. We pave the way to other innovative works relying on other similarity metric, and tackle problems in the same way by first identifying the issue in the similarity that cannot be captured semantically, extracting the entities of problem. Finally let the LLM decide how to query these nodes of interest. The extraction of product entities can give banks and financial institutions the opportunity of either up-selling or cross-selling their products based on the customer profile. As ontology based graph knowledge is gaining a huge success proven by recent research works, there is a growing need for identifying relevant entities for each sector. For that relying on curated factors and entities from domain experts can be more powerful than giving the LLMs the responsibility of constructing the graph knowledge base alone. We may consider combining both ideally. We have developed a pipeline that is capable of smartly choosing the path and the component utilized for answering a given question by putting a router responsible of understanding the requirement of the questions.

As we don't have access to closed source mod-
els, and the hardware required to run open source
LLMs, we focused on using open source only mod-
els. We were limited by the Hugging Face maxi-
mum quota of requests. So we couldn't evaluate
our pipelines on open data benchmarks in this field
or conclude to comparisons. We could potentially
get better results in extracting entities of our focus
by experimenting with specialized LLMs or agents
for converting human-like queries to Cypher given
our proposed scheme. It may be possible to use a
structured output format more easily, using features
that are only supported in closed source models.

8 Conclusion

Our research highlights the integration of Vector
RAG and Graph RAG in an advanced pipeline, for
an application into the banking sector. By refining
text data and utilizing a hybrid approach, we have
developed a robust Customer Analysis Pipeline
RAG (CAPRAG) that effectively addresses the
complexities of customer inquiries. This innovative
solution not only enhances the retrieval of relevant
information but also fosters deeper connections be-
tween key factors and entities within the banking
ecosystem. Our AI agent serves as a valuable tool
for providing timely insights into financial services,
thereby empowering customers to make informed
decisions.

References

Michael Abbott and Keri Smith. 2023. 3 Ways Generative AI Will Transform Banking | Accenture Banking Blog. [Accessed 28-03-2024].

Tom B Brown. 2020. Language models are few-shot learners. *Preprint arXiv arXiv:2005.14165*.

Prithiviraj Damodaran. 2017. Flashrank: Re-rank your search results with sota pairwise or listwise rerankers. https://github.com/PrithivirajDamodaran/FlashRank. [Accessed 17-10-2024].

Benjamin Decardi-Nelson, Abdulelah S Alshehri, Ak-
shay Ajagekar, and Fengqi You. 2024. Generative AI
and process systems engineering: The next frontier.
*Preprint arXiv arXiv:2402.10977*.

Hao-Wen Dong. 2024. *Generative AI for Music and Audio*. Ph.D. thesis, University of California, San Diego.

Andrea L Eisfeldt, Gregor Schubert, and Miao Ben Zhang. 2023. Generative ai and firm values. Technical report, National Bureau of Economic Research.

Mohamed Elasri, Omar Elharrouss, Somaya Al-
Maadeed, and Hamid Tairi. 2022. Image generation: A
review. *Neural Processing Letters*, 54(5):4609–4646.

Nadime Francis, Alastair Green, Paolo Guagliardo, Leonid Libkin, Tobias Lindaaker, Victor Marsault, Stefan Plantikow, Mats Rydberg, Petra Selmer, and Andrés Taylor. 2018. Cypher: An evolving query language for property graphs. In *Proceedings of the 2018 international conference on management of data*, pages 1433–1445.

Lei Hu, Wenwen Li, and Yunqiang Zhu. 2024. Geometric feature enhanced knowledge graph embedding and spatial reasoning. Preprint arXiv arXiv:2410.18345.

Nicole Immorlica, Brendan Lucier, and Aleksandrs
Slivkins. 2024. Generative AI as economic agents.
*SIGecom Exch.*, 22(1):93–109.
Shashank Mohan Jain. 2022. Hugging face. *Introduc-
tion to transformers for NLP: With the hugging face
library and models to solve problems*, pages 51–67.

Wenhao Kang and Benny C.F. Cheung. 2023. Banking
system incidents analysis using knowledge graph. In-
ternational Journal of Knowledge and Systems Science,
14:1–23.

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio
Petroni, Vladimir Karpukhin, Naman Goyal, Hein-
rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock-
täschel, et al. 2020. Retrieval-augmented generation
for knowledge-intensive nlp tasks. *Advances in Neural
Information Processing Systems*, 33:9459–9474.

Justin J Miller. 2013. Graph database applications and concepts with neo4j. In *Proceedings of the southern association for information systems conference, Atlanta, GA, USA, 2324-36, pages 141–147.*

Zooey Nguyen, Anthony Annunziata, Vinh Luong, Sang
Dinh, Quynh Le, Anh Hai Ha, Chanh Le, Hong An Phan,
Shruti Raghavan, and Christopher Nguyen. 2024. En-
hancing Q&A with domain-specific fine-tuning and iter-
ative reasoning: A comparative study. *Preprint arXiv
arXiv:2404.11792*.

Huan Ning, Zhenlong Li, Temitope Akinboyewa, and M Naser Lessani. 2024. An autonomous gis agent framework for geospatial data retrieval. Preprint Researchgate.

Keng-Boon Ooi, Garry Wei-Han Tan, Mostafa Al-
Emran, Mohammed A Al-Sharafi, Alexandru Capatina,
Amrita Chakraborty, Yogesh K Dwivedi, Tzu-Ling
Huang, Arpan Kumar Kar, Voon-Hsien Lee, et al. 2023.
The potential of generative artificial intelligence across
disciplines: Perspectives and future directions. *Journal
of Computer Information Systems*, pages 1–32.

Rohit Pandey, Hetvi Waghela, Sneha Rakshit, Aparna
Rangari, Anjali Singh, Rahul Kumar, Ratnadeep Ghosal,
and Jaydip Sen. 2024. Generative ai-based text genera-
tion methods using pre-trained gpt-2 model. Preprint
arXiv arXiv:2404.01786.

Tyler Procko and Omar Ochoa. 2024. Graph retrieval-augmented generation for large language models: A survey. Available at SSRN. https://ssrn.com/abstract=4895062.

Prompting Guide. 2022. Chain-of-thought prompting.
Accessed: 2024-10-07.

Lova Ravalimanana. 2020. Impacts of the technology
on the workforce in the banking industry in switzer-
land. *Mémoire de bachelor: Haute école de gestion de
Genève, SONAR*.

Bhaskarjit Sarmah, Benika Hall, Rohan Rao, Sunil Patel, Stefano Pasquali, and Dhagash Mehta. 2024. Hybriddrag: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction. Preprint arXiv arXiv:2408.04948.

Brilliant Shumba. 2023. An evaluation of the drivers of digital transformation of the banking sector in south Africa. Faculty of Business and Media, Selinus University.

Yijun Tian, Huan Song, Zichen Wang, Haozhu Wang, Ziqing Hu, Fang Wang, Nitesh V Chawla, and Panpan Xu. 2024. Graph neural prompting with large language models. In *Proceedings of the AAAI Conference on Artificial Intelligence*, 38:17, pages 19080–19088.

Lewis Tunstall, Edward Beeching, Nathan Lam-
bert, Nazneen Rajani, Kashif Rasul, Younes Belkada,
Shengyi Huang, Leandro von Werra, Clémentine Four-
rier, Nathan Habib, et al. 2023. Zephyr: Direct distilla-
tion of LM alignment. *Preprint arXiv:2310.16944*.

Jason Wei, Xinyu Wang, Dale Schuurmans, Maarten
Bosma, and Frank Hutter. 2022. Chain-of-thought
prompting elicits reasoning in large language models.
*Preprint arXiv
arXiv:2210.03493*.

Yujia Zhang, Haoyu Liu, Yujun Chen, and Zhiqiang
Wang. 2022. Automatic chain-of-thought prompt-
ing for large language models. *Preprint arXiv
arXiv:2210.03493*.

Pengyuan Zhou, Lin Wang, Zhi Liu, Yanbin Hao,
Pan Hui, Sasu Tarkoma, and Jussi Kangasharju. 2024.
A survey on generative ai and llm for video genera-
tion, understanding, and streaming. Preprint arXiv
arXiv:2404.16038.