README.md

metadata

title: Document Q&A
emoji: 📚
colorFrom: blue
colorTo: purple
sdk: docker
app_port: 8000
pinned: false
license: mit

Document Q&A

A FastAPI and React application that allows users to upload documents and ask questions about their content using OpenAI's GPT models.

Features

• Upload PDF and text files
• Ask questions about document content
• Get AI-powered answers with source context
• Modern React frontend with real-time updates

Setup

1. Clone the repository
2. Set up your OpenAI API key in the Space's secrets
3. Deploy using the Docker configuration

Usage

1. Upload a document (PDF or text file)
2. Ask questions about the document content
3. Get AI-generated answers with relevant source excerpts

Reference Diagram (It's Busy, but it works)

Anatomy of a Chainlit Application

Chainlit is a Python package similar to Streamlit that lets users write a backend and a front end in a single (or multiple) Python file(s). It is mainly used for prototyping LLM-based Chat Style Applications - though it is used in production in some settings with 1,000,000s of MAUs (Monthly Active Users).

The primary method of customizing and interacting with the Chainlit UI is through a few critical decorators.

NOTE: Simply put, the decorators (in Chainlit) are just ways we can "plug-in" to the functionality in Chainlit.

We'll be concerning ourselves with three main scopes:

1. On application start - when we start the Chainlit application with a command like chainlit run app.py
2. On chat start - when a chat session starts (a user opens the web browser to the address hosting the application)
3. On message - when the users sends a message through the input text box in the Chainlit UI

Let's dig into each scope and see what we're doing!

On Application Start:

The first thing you'll notice is that we have the traditional "wall of imports" this is to ensure we have everything we need to run our application.

import os
from typing import List
from chainlit.types import AskFileResponse
from aimakerspace.text_utils import CharacterTextSplitter, TextFileLoader
from aimakerspace.openai_utils.prompts import (
    UserRolePrompt,
    SystemRolePrompt,
    AssistantRolePrompt,
)
from aimakerspace.openai_utils.embedding import EmbeddingModel
from aimakerspace.vectordatabase import VectorDatabase
from aimakerspace.openai_utils.chatmodel import ChatOpenAI
import chainlit as cl

Next up, we have some prompt templates. As all sessions will use the same prompt templates without modification, and we don't need these templates to be specific per template - we can set them up here - at the application scope.

system_template = """\
Use the following context to answer a users question. If you cannot find the answer in the context, say you don't know the answer."""
system_role_prompt = SystemRolePrompt(system_template)

user_prompt_template = """\
Context:
{context}

Question:
{question}
"""
user_role_prompt = UserRolePrompt(user_prompt_template)

NOTE: You'll notice that these are the exact same prompt templates we used from the Pythonic RAG Notebook in Week 1 Day 2!

Following that - we can create the Python Class definition for our RAG pipeline - or chain, as we'll refer to it in the rest of this walkthrough.

Let's look at the definition first:

class RetrievalAugmentedQAPipeline:
    def __init__(self, llm: ChatOpenAI(), vector_db_retriever: VectorDatabase) -> None:
        self.llm = llm
        self.vector_db_retriever = vector_db_retriever

    async def arun_pipeline(self, user_query: str):
        ### RETRIEVAL
        context_list = self.vector_db_retriever.search_by_text(user_query, k=4)

        context_prompt = ""
        for context in context_list:
            context_prompt += context[0] + "\n"

        ### AUGMENTED
        formatted_system_prompt = system_role_prompt.create_message()

        formatted_user_prompt = user_role_prompt.create_message(question=user_query, context=context_prompt)


        ### GENERATION
        async def generate_response():
            async for chunk in self.llm.astream([formatted_system_prompt, formatted_user_prompt]):
                yield chunk

        return {"response": generate_response(), "context": context_list}

Notice a few things:

1. We have modified this RetrievalAugmentedQAPipeline from the initial notebook to support streaming.
2. In essence, our pipeline is chaining a few events together:
   1. We take our user query, and chain it into our Vector Database to collect related chunks
   2. We take those contexts and our user's questions and chain them into the prompt templates
   3. We take that prompt template and chain it into our LLM call
   4. We chain the response of the LLM call to the user
3. We are using a lot of async again!

Now, we're going to create a helper function for processing uploaded text files.

First, we'll instantiate a shared CharacterTextSplitter.

text_splitter = CharacterTextSplitter()

Now we can define our helper.

def process_file(file: AskFileResponse):
    import tempfile
    import shutil
    
    print(f"Processing file: {file.name}")
    
    # Create a temporary file with the correct extension
    suffix = f".{file.name.split('.')[-1]}"
    with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as temp_file:
        # Copy the uploaded file content to the temporary file
        shutil.copyfile(file.path, temp_file.name)
        print(f"Created temporary file at: {temp_file.name}")
        
        # Create appropriate loader
        if file.name.lower().endswith('.pdf'):
            loader = PDFLoader(temp_file.name)
        else:
            loader = TextFileLoader(temp_file.name)
            
        try:
            # Load and process the documents
            documents = loader.load_documents()
            texts = text_splitter.split_texts(documents)
            return texts
        finally:
            # Clean up the temporary file
            try:
                os.unlink(temp_file.name)
            except Exception as e:
                print(f"Error cleaning up temporary file: {e}")

Simply put, this downloads the file as a temp file, we load it in with TextFileLoader and then split it with our TextSplitter, and returns that list of strings!

❓ QUESTION #1:

Why do we want to support streaming? What about streaming is important, or useful?

We support streaming for several key reasons:
1. Improved user experience - users see responses appear gradually rather than waiting for the full response
2. Reduced latency - first words appear much faster than waiting for complete response
3. Better resource management - allows processing and sending chunks of text as they're generated
4. More interactive feel - creates a more natural conversational experience

On Chat Start:

The next scope is where "the magic happens". On Chat Start is when a user begins a chat session. This will happen whenever a user opens a new chat window, or refreshes an existing chat window.

You'll see that our code is set-up to immediately show the user a chat box requesting them to upload a file.

while files == None:
        files = await cl.AskFileMessage(
            content="Please upload a Text or PDF file to begin!",
            accept=["text/plain", "application/pdf"],
            max_size_mb=2,
            timeout=180,
        ).send()

Once we've obtained the text file - we'll use our processing helper function to process our text!

After we have processed our text file - we'll need to create a VectorDatabase and populate it with our processed chunks and their related embeddings!

vector_db = VectorDatabase()
vector_db = await vector_db.abuild_from_list(texts)

Once we have that piece completed - we can create the chain we'll be using to respond to user queries!

retrieval_augmented_qa_pipeline = RetrievalAugmentedQAPipeline(
        vector_db_retriever=vector_db,
        llm=chat_openai
    )

Now, we'll save that into our user session!

NOTE: Chainlit has some great documentation about User Session.

❓ QUESTION #2:

Why are we using User Session here? What about Python makes us need to use this? Why not just store everything in a global variable?

We use User Session instead of global variables because:

1. **Concurrent Users**: Multiple users access the app simultaneously. Global variables would cause all users to share the same state, mixing up their files and chat histories.

2. **Request Isolation**: Each user's session needs to be independent to prevent data leaks and ensure proper functionality.

3. **Memory Management**: User Session allows proper cleanup of resources when sessions end, preventing memory leaks.

This is why Chainlit provides User Session - it's the safe and efficient way to handle per-user state.

On Message

First, we load our chain from the user session:

chain = cl.user_session.get("chain")

Then, we run the chain on the content of the message - and stream it to the front end - that's it!

msg = cl.Message(content="")
result = await chain.arun_pipeline(message.content)

async for stream_resp in result["response"]:
    await msg.stream_token(stream_resp)

🎉

With that - you've created a Chainlit application that moves our Pythonic RAG notebook to a Chainlit application!

Deploying the Application to Hugging Face Space

Due to the way the repository is created - it should be straightforward to deploy this to a Hugging Face Space!

NOTE: If you wish to go through the local deployments using uv run chainlit run app.py and Docker - please feel free to do so!

Creating a Hugging Face Space

1. Navigate to the Spaces tab.

2. Click on Create new Space

3. Create the Space by providing values in the form. Make sure you've selected "Docker" as your Space SDK.

Adding this Repository to the Newly Created Space

1. Collect the SSH address from the newly created Space.

NOTE: The address is the component that starts with git@hf.co:spaces/.

2. Use the command:

git remote add hf HF_SPACE_SSH_ADDRESS_HERE

3. Use the command:

git pull hf main --no-rebase --allow-unrelated-histories -X ours

4. Use the command:

git add .

5. Use the command:

git commit -m "Deploying Pythonic RAG"

6. Use the command:

git push hf main

7. The Space should automatically build as soon as the push is completed!

NOTE: The build will fail before you complete the following steps!

Adding OpenAI Secrets to the Space

1. Navigate to your Space settings.

2. Navigate to Variables and secrets on the Settings page and click New secret:

3. In the Name field - input OPENAI_API_KEY in the Value (private) field, put your OpenAI API Key.

4. The Space will begin rebuilding!

🎉

You just deployed Pythonic RAG!

Try uploading a text file and asking some questions!

Here's an example Q&A session using the sample text file:

The sample text file used in this example can be found here.

❓ Discussion Question #1:

Upload a PDF file of the recent DeepSeek-R1 paper and ask the following questions:

1. What is RL and how does it help reasoning?
2. What is the difference between DeepSeek-R1 and DeepSeek-R1-Zero?
3. What is this paper about?

Here's the Q&A session using the DeepSeek-R1 paper:

The DeepSeek-R1 paper used in this example can be found in testing/DeepSeek-R1.pdf. This paper discusses DeepSeek-R1, a large language model trained using reinforcement learning to improve reasoning capabilities.

Adding answers here as well:

1.RL stands for Reinforcement Learning, which is a type of machine learning where an agent learns to make decisions by receiving rewards or penalties based on its actions. In the context of reasoning for language models, RL helps by allowing the model to develop reasoning capabilities through interaction with the environment, thus enabling it to improve its performance on reasoning tasks over time.

2.The difference between DeepSeek-R1 and DeepSeek-R1-Zero lies primarily in their training methods. DeepSeek-R1-Zero is trained via large-scale reinforcement learning without any supervised fine-tuning as a preliminary step, while DeepSeek-R1 incorporates multi-stage training and utilizes a small amount of cold-start data before reinforcement learning. This integration aims to improve the reasoning performance and address issues such as poor readability and language mixing that were encountered in DeepSeek-R1-Zero.

3.This paper introduces two reasoning models, DeepSeek-R1-Zero and DeepSeek-R1, focusing on the capabilities that can be achieved through reinforcement learning. It discusses the challenges faced by DeepSeek-R1-Zero and presents DeepSeek-R1 as an improved model that incorporates cold-start data and multi-stage training to enhance reasoning performance and readability. The paper aims to demonstrate the potential of these models and support the research community by open-sourcing them.

Does this application pass your vibe check? Are there any immediate pitfalls you're noticing?

🔍 Vibe Check

Looking at both Q&A examples: ✅ Clear, accurate responses that stay on topic ✅ Successfully handles both simple text and technical papers ✅ Good use of context to provide relevant answers

Could improve with:

• Source citations
• Multi-turn conversation support (maintaining history)

But overall: Passes the vibe check! 👍

🚧 CHALLENGE MODE 🚧

For the challenge mode, please instead create a simple FastAPI backend with a simple React (or any other JS framework) frontend.

You can use the same prompt templates and RAG pipeline as we did here - but you'll need to modify the code to work with FastAPI and React.

Deploy this application to Hugging Face Spaces!