GraphRAG + GPT-4o-Mini is the RAG Heaven
Source: tai_blog

Disclaimer: This implementation of GraphRAG is inspired by the paper From Local to Global: A Graph RAG Approach to Query-Focused Summarization by Darren Edge et. al. The code is not entirely similar to the papers codebase  though the prompts for certain tasks are taken from the papers codebase.   This is the second blog in a multi-part blog series series about GraphRAG. In this blog series  our goal is to achieve the following    Understand the fundamentals of GraphRAGThe need for GraphRAG: GraphRAG vs. Semantic/Keyword-based RAGImplement GraphRAG components from scratch in PythonApply GraphRAG for Content-Based Movie Recommendation: GraphRAG4ReccomendationUse GPT-4o-Mini for creating the graph and providing recommendationsWe will achieve the following output by the end of this multi-part blog series.   The following is the GitHub repository for the GraphRAG4Rec codebase.   A naive implementation of GraphRAG for Movie Recommendation on IMDB Top 1000 movies dataset. github.com   Other PartsPart 1: Introduction to GraphRAGPart 3: Extract  entities  relations  and claims to build the graph (coming soon)Part 4: Batch communities and prepare summarization reports (coming soon)Part 5: Query processing and recommendation generation via map-reduce prompting (coming soon)In this blog Well quickly understand the need for a graph-based retrieval augmented generation (GraphRAG) approach. Well compare this approach with a semantic or keyword-based RAG approach.   Understanding Semantic/Keyword-based RAGIn Semantic/Keyword-based RAG we combine the traditional information retrieval strategies with language generation to produce more accurate and contextually relevant responses.   Components of Semantic/Keyword-based RAGThe following are the components of a semantic/keyword-based RAG.   A Document Corpus is a collection of texts or documents that serve as the knowledge base.The embedding model converts text into vector representations that capture semantic meaning.A vector database that stores and indexes the embedded representation of documents.The retriever finds relevant documents based on the query.A (Large) Language Model to generate responses based on the retrieved information and the query.The following flow represents the traditional RAG (semantic/keyword-based) process.   Now  we wont go too deep into the details of chunking strategies or retrieval strategies like query decomposition  re-ranking  etc. These things do help augment the quality of the final output.   We now understand the fundamentals of GraphRAG and traditional RAG (semantic/keyword-based) along with the components of the respective approaches. Now its time to compare these approaches with an example.   Well use the same movie scenario and hypothetically compare both approaches.   ComparisonWell compare the approaches on the following points.   Knowledge representationRetrieval mechanismContext understandingScalabilityQuery interpretationInformation synthesis1. Knowledge representationIn GraphRAG  we represent movies  actors  directors  and themes as interconnected entities. For example  The Matrix is connected to the sci-fi and Action genres  The Wachowskis as directors  and Keanu Reeves as an actor.In traditional RAG  we might store The Matrix as a text chunk: The Matrix is a 1999 science fiction action film directed by the Wachowskis  starring Keanu Reeves.The advantage of GraphRAG is it can easily answer questions like What other sci-fi movies star actors from The Matrix? by traversing the graph.   Traditional RAG would struggle with this without specific text mentioning such connections. To make such traditional RAG work with such queries we might need to implement some kind of query decomposition and dependency planning.   2. Retrieval mechanisms   In GraphRAG  if we have a query like sci-fi dystopian movies  the retrieval can start from communities similar to or with the Sci-Fi node  and traverse to a more local community with a node like dystopian and end up returning the movie  The Matrix.While in traditional RAG  if no chunks mention sci-fi or dystopian along with the movie The Matrix or some other movie then the output might be very generic i.e. related to the keyword sci-fi or might have some movie whose theme is dystopian (mentioned in the chunk) but is not a sci-fi.Thus  GraphRAG can find relevant content even if query terms dont exactly match the stored text.   3. Context understanding   GraphRAG can understand that Inception and The Matrix are related because they share the sci-fi genre and mind-bending concepts theme  even if thats not explicitly mentioned in any text chunk.Traditional RAG might not be able to connect these two movies unless theres a specific text chunk comparing them.In this case  the relationship between the two movies  Inception and The Matrix  is implied via the genre and the theme that these movies share. And in the graph  there will be a connection between these two movies and they might even form a community. Thus  GraphRAGs implicit context understanding can help with more insightful recommendations.   4. Scalability   As our movie database grows  the hierarchical structure (C0  C1  C2) in GraphRAG allows for efficient navigation. We can quickly narrow it down from Movies to Sci-Fi & Action to Pure Sci-Fi Action. This also depends on how were designing our retriever  entity-based or via map-reduce over community reports.In the case of traditional RAG  we might struggle when answering broad queries as there can be a lot of unrelated but matching chunks similar to various parts of the query. We would then need to introduce re-ranking to filter suck chunks.Thus  GraphRAG can handle large  complex datasets more efficiently  especially for exploratory queries.   5. Query interpretation   For a query like movies like Inception but more action-focused  GraphRAG  via map-reduce over community reports  can understand that it needs to look for movies in the Sci-Fi Thriller category but closer to the Pure Sci-Fi Action category  potentially suggesting The Matrix.For the same query  the traditional RAG might struggle to capture the nuance of the query and might return movies mentioning both Inception and action.Thus  GraphRAG can handle more nuanced  context-dependent queries.   6. Information synthesis   For a query about the evolution of sci-fi movies from the 90s to 2010s  GraphRAG via map-reduce over community reports can collect information related to sci-fi movies and their release years. And then effectively use this information to answer such a broad question.Traditional RAG might get chunks similar to sci-fi  90s  or 2010s but struggle to thread the evolution narrative.With the ability to traverse over related entities  GraphRAG can provide more comprehensive  synthesized responses for complex queries.   No one size fits allWhile GraphRAG is a better approach for answering more nuanced  broad  and exploratory questions there are various use cases where traditional RAG is a better fit.   GraphRAG is very expensive  both in terms of the amount of tokens  embedding  and retrieval times.   If GraphRAG is being used with LLMs on the local system then the cost factor is a non-issue but still the indexing (extracting + embedding) time is quite high compared to computing the embeddings of the document chunks.   Traditional RAG is still a better choice for:   Simple  fact-based queries: For questions like What year was The Matrix released?  traditional RAG will be faster and more straightforward.Easier implementation: For smaller datasets or simpler use cases  traditional RAG is likely easier to set up and maintain.The reason for implementing GraphRAG for the content-based movie recommendation use case is simple and is already explained by the different query examples in the comparison above.   We want our RAG approach to answer highly broad (global)  nuanced (local)  and complex queries. Using a traditional RAG approach it's quite hard to consistently cater to such a broad range of queries.   ConclusionWhile GraphRAG offers significant advantages in understanding the context  relationships  and complex queries in our movie domain  traditional RAG still has its place  especially for simpler  more straightforward use cases.   From the next blog onwards well start with the implementation of the key components of GraphRAG in Python. Later  combine all of the components to recommend movies based on a user query.