realtimex-deeptutor 0.5.0.post11
RealTimeX DeepTutor - Intelligent learning companion with multi-agent collaboration and LightRAG
License: Apache-2.0
Requires Python: >=3.10

<div align="center">

<img src="assets/logo-ver2.png" alt="DeepTutor Logo" width="150" style="border-radius: 15px;">

# DeepTutor: AI-Powered Personalized Learning Assistant

[![Python](https://img.shields.io/badge/Python-3.10%2B-3776AB?style=flat-square&logo=python&logoColor=white)](https://www.python.org/downloads/)
[![FastAPI](https://img.shields.io/badge/FastAPI-0.100%2B-009688?style=flat-square&logo=fastapi&logoColor=white)](https://fastapi.tiangolo.com/)
[![React](https://img.shields.io/badge/React-19-61DAFB?style=flat-square&logo=react&logoColor=black)](https://react.dev/)
[![Next.js](https://img.shields.io/badge/Next.js-16-000000?style=flat-square&logo=next.js&logoColor=white)](https://nextjs.org/)
[![TailwindCSS](https://img.shields.io/badge/Tailwind-3.4-06B6D4?style=flat-square&logo=tailwindcss&logoColor=white)](https://tailwindcss.com/)
[![License](https://img.shields.io/badge/License-AGPL--3.0-blue?style=flat-square)](LICENSE)

<p align="center">
  <a href="https://discord.gg/eRsjPgMU4t"><img src="https://img.shields.io/badge/Discord-Join_Community-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Discord"></a>
  &nbsp;&nbsp;
  <a href="./Communication.md"><img src="https://img.shields.io/badge/Feishu-Join_Group-00D4AA?style=for-the-badge&logo=feishu&logoColor=white" alt="Feishu"></a>
  &nbsp;&nbsp;
  <a href="https://github.com/HKUDS/DeepTutor/issues/78"><img src="https://img.shields.io/badge/WeChat-Join_Group-07C160?style=for-the-badge&logo=wechat&logoColor=white" alt="WeChat"></a>
</p>



[**Quick Start**](#quick-start) · [**Core Modules**](#core-modules) · [**FAQ**](#faq)

[🇨🇳 中文](assets/README/README_CN.md) · [🇯🇵 日本語](assets/README/README_JA.md) · [🇪🇸 Español](assets/README/README_ES.md) · [🇫🇷 Français](assets/README/README_FR.md) · [🇸🇦 العربية](assets/README/README_AR.md) · [🇷🇺 Русский](assets/README/README_RU.md) · [🇮🇳 हिन्दी](assets/README/README_HI.md) · [🇵🇹 Português](assets/README/README_PT.md)

</div>

<div align="center">

📚 **Massive Document Knowledge Q&A** &nbsp;•&nbsp; 🎨 **Interactive Learning Visualization**<br>
🎯 **Knowledge Reinforcement** &nbsp;•&nbsp; 🔍 **Deep Research & Idea Generation**

</div>

---
### 📰 News

> **[2026.1.1]** Happy New Year! Join our [Discord Community](https://discord.gg/zpP9cssj), [Wechat Community](https://github.com/HKUDS/DeepTutor/issues/78), or [Discussions](https://github.com/HKUDS/DeepTutor/discussions) - shape the future of DeepTutor! 💬

> **[2025.12.30]** Visit our [Official Website](https://hkuds.github.io/DeepTutor/) for more details!

> **[2025.12.29]** DeepTutor is now live! ✨

### 📦 Releases

> **[2026.1.23]** Release [v0.6.0](https://github.com/HKUDS/DeepTutor/releases/tag/v0.6.0) - Frontend session persistence, full Chinese support, Docker deployment updates, and minor bug fixes -- Thanks for all the feedback!

<details>
<summary>History releases</summary>

> **[2026.1.18]** Release [v0.5.2](https://github.com/HKUDS/DeepTutor/releases/tag/v0.5.1) - Enhance RAG pipeline with Docling support and improve CI/CD workflows with several minor bugs fixed -- Thanks to all the feedbacks!


> **[2026.1.15]** Release [v0.5.0](https://github.com/HKUDS/DeepTutor/releases/tag/v0.5.0) - Unified LLM & Embedding services, RAG pipeline selection, and major enhancements to Home, History, QuestionGen & Settings modules -- Thanks to all the contributors!

> **[2026.1.9]** Release [v0.4.1](https://github.com/HKUDS/DeepTutor/releases/tag/v0.4.1) with LLM Provider system overhaul, Question Generation robustness improvements, and codebase cleanup - Thanks to all the contributors!

> **[2026.1.9]** Release [v0.4.0](https://github.com/HKUDS/DeepTutor/releases/tag/v0.4.0) with new code structure, multiple llm & embeddings support - Thanks to all the contributors!

> **[2026.1.5]** [v0.3.0](https://github.com/HKUDS/DeepTutor/releases/tag/v0.3.0) - Unified PromptManager architecture, CI/CD automation & pre-built Docker images on GHCR

> **[2026.1.2]** [v0.2.0](https://github.com/HKUDS/DeepTutor/releases/tag/v0.2.0) - Docker deployment, Next.js 16 & React 19 upgrade, WebSocket security & critical vulnerability fixes

</details>

---

## Key Features of DeepTutor

### 📚 Massive Document Knowledge Q&A
• **Smart Knowledge Base**: Upload textbooks, research papers, technical manuals, and domain-specific documents. Build a comprehensive AI-powered knowledge repository for instant access.<br>
• **Multi-Agent Problem Solving**: Dual-loop reasoning architecture with RAG, web search, and code execution -- delivering step-by-step solutions with precise citations.

### 🎨 Interactive Learning Visualization
• **Knowledge Simplification & Explanations**: Transform complex concepts, knowledge, and algorithms into easy-to-understand visual aids, detailed step-by-step breakdowns, and engaging interactive demonstrations.<br>
• **Personalized Q&A**: Context-aware conversations that adapt to your learning progress, with interactive pages and session-based knowledge tracking.

### 🎯 Knowledge Reinforcement with Practice Exercise Generator
• **Intelligent Exercise Creation**: Generate targeted quizzes, practice problems, and customized assessments tailored to your current knowledge level and specific learning objectives.<br>
• **Authentic Exam Simulation**: Upload reference exams to generate practice questions that perfectly match the original style, format, and difficulty—giving you realistic preparation for the actual test.

### 🔍 Deep Research & Idea Generation
• **Comprehensive Research & Literature Review**: Conduct in-depth topic exploration with systematic analysis. Identify patterns, connect related concepts across disciplines, and synthesize existing research findings.<br>
• **Novel Insight Discovery**: Generate structured learning materials and uncover knowledge gaps. Identify promising new research directions through intelligent cross-domain knowledge synthesis.

---

<div align="center">
  <img src="assets/figs/title_gradient.svg" alt="All-in-One Tutoring System" width="70%">
</div>

<!-- ━━━━━━━━━━━━━━━━ Core Learning Experience ━━━━━━━━━━━━━━━━ -->

<table>
<tr>
<td width="50%" align="center" valign="top">

<h3>📚 Massive Document Knowledge Q&A</h3>
<a href="#problem-solving-agent">
<img src="assets/gifs/solve.gif" width="100%">
</a>
<br>
<sub>Multi-agent Problem Solving with Exact Citations</sub>

</td>
<td width="50%" align="center" valign="top">

<h3>🎨 Interactive Learning Visualization</h3>
<a href="#guided-learning">
<img src="assets/gifs/guided-learning.gif" width="100%">
</a>
<br>
<sub>Step-by-step Visual Explanations with Personal QAs.</sub>

</td>
</tr>
</table>

<!-- ━━━━━━━━━━━━━━━━ Practice & Reinforcement ━━━━━━━━━━━━━━━━ -->

<h3 align="center">🎯 Knowledge Reinforcement</h3>

<table>
<tr>
<td width="50%" valign="top" align="center">

<a href="#question-generator">
<img src="assets/gifs/question-1.gif" width="100%">
</a>

**Custom Questions**  
<sub>Auto-Validated Practice Questions Generation</sub>

</td>
<td width="50%" valign="top" align="center">

<a href="#question-generator">
<img src="assets/gifs/question-2.gif" width="100%">
</a>

**Mimic Questions**  
<sub>Clone Exam Style for Authentic Practice</sub>

</td>
</tr>
</table>

<!-- ━━━━━━━━━━━━━━━━ Research & Creation ━━━━━━━━━━━━━━━━ -->

<h3 align="center">🔍 Deep Research & Idea Generation</h3>

<table>
<tr>
<td width="33%" align="center">

<a href="#deep-research">
<img src="assets/gifs/deepresearch.gif" width="100%">
</a>

**Deep Research**  
<sub>Knowledge Extension from Textbook with RAG, Web and Paper-search</sub>

</td>
<td width="33%" align="center">

<a href="#idea-generation">
<img src="assets/gifs/ideagen.gif" width="100%">
</a>

**Automated IdeaGen**  
<sub>Brainstorming and Concept Synthesis with Dual-filter Workflow</sub>

</td>
<td width="33%" align="center">

<a href="#co-writer">
<img src="assets/gifs/co-writer.gif" width="100%">
</a>

**Interactive IdeaGen**  
<sub>RAG and Web-search Powered Co-writer with Podcast Generation</sub>

</td>
</tr>
</table>

<!-- ━━━━━━━━━━━━━━━━ Knowledge Infrastructure ━━━━━━━━━━━━━━━━ -->

<h3 align="center">🏗️ All-in-One Knowledge System</h3>

<table>
<tr>
<td width="50%" align="center">

<a href="#dashboard--knowledge-base-management">
<img src="assets/gifs/knowledge_bases.png" width="100%">
</a>

**Personal Knowledge Base**  
<sub>Build and Organize Your Own Knowledge Repository</sub>

</td>
<td width="50%" align="center">

<a href="#notebook">
<img src="assets/gifs/notebooks.png" width="100%">
</a>

**Personal Notebook**  
<sub>Your Contextual Memory for Learning Sessions</sub>

</td>
</tr>
</table>

<p align="center">
  <sub>🌙 Use DeepTutor in <b>Dark Mode</b>!</sub>
</p>

---

## 🏛️ DeepTutor's Framework

<div align="center">
<img src="assets/figs/full-pipe.png" alt="DeepTutor Full-Stack Workflow" width="100%">
</div>

### 💬 User Interface Layer
• **Intuitive Interaction**: Simple bidirectional query-response flow for intuitive interaction.<br>
• **Structured Output**: Structured response generation that organizes complex information into actionable outputs.

### 🤖 Intelligent Agent Modules
• **Problem Solving & Assessment**: Step-by-step problem solving and custom assessment generation.<br>
• **Research & Learning**: Deep Research for topic exploration and Guided Learning with visualization.<br>
• **Idea Generation**: Automated and interactive concept development with multi-source insights.

### 🔧 Tool Integration Layer
• **Information Retrieval**: RAG hybrid retrieval, real-time web search, and academic paper databases.<br>
• **Processing & Analysis**: Python code execution, query item lookup, and PDF parsing for document analysis.

### 🧠 Knowledge & Memory Foundation
• **Knowledge Graph**: Entity-relation mapping for semantic connections and knowledge discovery.<br>
• **Vector Store**: Embedding-based semantic search for intelligent content retrieval.<br>
• **Memory System**: Session state management and citation tracking for contextual continuity.

## 📋 Todo
> 🌟 Star to follow our future updates!
- [ x ] Multi-linguistic support
- [ x ] DeepTutor Community
- [ x ] Video & Audio file support
- [ x ] Atomic RAG pipeline customize
- [ - ] Incremental Knowledge-base Edit
- [ - ] Personalized Workspace
- [ - ] DataBase Visualization
- [ - ] Online Demo

## 🚀 Getting Started

### Step 1: Pre-Configuration

**① Clone Repository**

```bash
git clone https://github.com/HKUDS/DeepTutor.git
cd DeepTutor
```

**② Set Up Environment Variables**

```bash
cp .env.example .env
# Edit .env file with your API keys
```

<details>
<summary>📋 <b>Environment Variables Reference</b></summary>

| Variable | Required | Description |
|:---|:---:|:---|
| `LLM_MODEL` | **Yes** | Model name (e.g., `gpt-4o`) |
| `LLM_API_VERSION` | No | API version for Azure OpenAI (e.g., `2024-02-15-preview`) |
| `LLM_API_KEY` | **Yes** | Your LLM API key |
| `LLM_HOST` | **Yes** | API endpoint URL |
| `EMBEDDING_MODEL` | **Yes** | Embedding model name |
| `EMBEDDING_API_VERSION` | No | API version for Azure OpenAI Embeddings |
| `EMBEDDING_API_KEY` | **Yes** | Embedding API key |
| `EMBEDDING_HOST` | **Yes** | Embedding API endpoint |
| `BACKEND_PORT` | No | Backend port (default: `8001`) |
| `FRONTEND_PORT` | No | Frontend port (default: `3782`) |
| `NEXT_PUBLIC_API_BASE` | No | **Frontend API URL** - Set this for remote/LAN access (e.g., `http://192.168.1.100:8001`) |
| `TTS_*` | No | Text-to-Speech settings |
| `SEARCH_PROVIDER` | No | Search provider (options: `perplexity`, `tavily`, `serper`, `jina`, `exa`, `baidu`, default: `perplexity`) |
| `SEARCH_API_KEY` | No | Unified API key for all search providers |

> 💡 **Remote Access**: If accessing from another device (e.g., `192.168.31.66:3782`), add to `.env`:
> ```bash
> NEXT_PUBLIC_API_BASE=http://192.168.31.66:8001
> ```

</details>

**③ Configure Ports & LLM** *(Optional)*

- **Ports**: Set in `.env` file → `BACKEND_PORT` / `FRONTEND_PORT` (defaults: 8001/3782)
- **LLM**: Edit `config/agents.yaml` → `temperature` / `max_tokens` per module
- See [Configuration Docs](config/README.md) for details

**④ Try Demo Knowledge Bases** *(Optional)*

<details>
<summary>📚 <b>Available Demos</b></summary>

- **Research Papers** — 5 papers from our lab ([AI-Researcher](https://github.com/HKUDS/AI-Researcher), [LightRAG](https://github.com/HKUDS/LightRAG), etc.)
- **Data Science Textbook** — 8 chapters, 296 pages ([Book Link](https://ma-lab-berkeley.github.io/deep-representation-learning-book/))

</details>

1. Download from [Google Drive](https://drive.google.com/drive/folders/1iWwfZXiTuQKQqUYb5fGDZjLCeTUP6DA6?usp=sharing)
2. Extract into `data/` directory

> Demo KBs use `text-embedding-3-large` with `dimensions = 3072`

**⑤ Create Your Own Knowledge Base** *(After Launch)*

1. Go to http://localhost:3782/knowledge
2. Click "New Knowledge Base" → Enter name → Upload PDF/TXT/MD files
3. Monitor progress in terminal

---

### Step 2: Choose Your Installation Method

#### 🐳 Option A: Docker Deployment

> No Python/Node.js setup required

**Prerequisites**: [Docker](https://docs.docker.com/get-docker/) & [Docker Compose](https://docs.docker.com/compose/install/)

**Quick Start** — Build from source:

```bash
docker compose up                  # Build and start (~11 min first run on mac mini M4)
docker compose build --no-cache    # Clear cache and rebuild after pull the newest repo
```

**Or use pre-built image** (faster):

```bash
# Works on all platforms - Docker auto-detects your architecture
docker run -d --name deeptutor \
  -p 8001:8001 -p 3782:3782 \
  --env-file .env \
  -v $(pwd)/data:/app/data \
  -v $(pwd)/config:/app/config:ro \
  ghcr.io/hkuds/deeptutor:latest

# Windows PowerShell: use ${PWD} instead of $(pwd)
```

**Common Commands**:

```bash
docker compose up -d      # Start
docker compose down       # Stop
docker compose logs -f    # View logs
docker compose up --build # Rebuild after changes
```

<details>
<summary>📋 <b>More Docker Options</b> (Pre-built images, Cloud deployment, Custom ports)</summary>

**Pre-built Image Tags:**

| Tag | Architectures | Description |
|:----|:--------------|:------------|
| `:latest` | AMD64 + ARM64 | Latest stable release (auto-detects your architecture) |
| `:v0.5.x` | AMD64 + ARM64 | Specific version (auto-detects your architecture) |
| `:v0.5.x-amd64` | AMD64 only | Explicit AMD64 image |
| `:v0.5.x-arm64` | ARM64 only | Explicit ARM64 image |

> 💡 The `:latest` tag is a **multi-architecture image** — Docker automatically pulls the correct version for your system (Intel/AMD or Apple Silicon/ARM)

**Cloud Deployment** — Must set external API URL:

```bash
docker run -d --name deeptutor \
  -p 8001:8001 -p 3782:3782 \
  -e NEXT_PUBLIC_API_BASE_EXTERNAL=https://your-server.com:8001 \
  --env-file .env \
  -v $(pwd)/data:/app/data \
  ghcr.io/hkuds/deeptutor:latest
```

**Custom Ports Example:**

```bash
docker run -d --name deeptutor \
  -p 9001:9001 -p 3000:3000 \
  -e BACKEND_PORT=9001 \
  -e FRONTEND_PORT=3000 \
  -e NEXT_PUBLIC_API_BASE_EXTERNAL=https://your-server.com:9001 \
  --env-file .env \
  -v $(pwd)/data:/app/data \
  ghcr.io/hkuds/deeptutor:latest
```

</details>

---

#### 💻 Option B: Manual Installation

> For development or non-Docker environments

**Prerequisites**: Python 3.10+, Node.js 18+

**1. Set Up Environment**:

```bash
# Using conda (Recommended)
conda create -n deeptutor python=3.10 && conda activate deeptutor

# Or using venv
python -m venv venv && source venv/bin/activate  # Windows: venv\Scripts\activate
```

**2. Install Dependencies**:

```bash
# One-click installation (Recommended)
python scripts/install_all.py
# Or: bash scripts/install_all.sh

# Or manual installation
pip install -r requirements.txt
npm install --prefix web
```

**3. Launch**:

```bash
python scripts/start_web.py    # Start frontend + backend
# Or: python scripts/start.py  # CLI only
# Stop: Ctrl+C
```

<details>
<summary>🔧 <b>Start Frontend & Backend Separately</b></summary>

**Backend** (FastAPI):
```bash
python src/api/run_server.py
# Or: uvicorn src.api.main:app --host 0.0.0.0 --port 8001 --reload
```

**Frontend** (Next.js):
```bash
cd web && npm install && npm run dev -- -p 3782
```

**Note**: Create `web/.env.local`:
```bash
NEXT_PUBLIC_API_BASE=http://localhost:8001
```

| Service | Default Port |
|:---:|:---:|
| Backend | `8001` |
| Frontend | `3782` |

</details>

### Access URLs

| Service | URL | Description |
|:---:|:---|:---|
| **Frontend** | http://localhost:3782 | Main web interface |
| **API Docs** | http://localhost:8001/docs | Interactive API documentation |

---

## 📂 Data Storage

All user content and system data are stored in the `data/` directory:

```
data/
├── knowledge_bases/              # Knowledge base storage
└── user/                         # User activity data
    ├── solve/                    # Problem solving results and artifacts
    ├── question/                 # Generated questions
    ├── research/                 # Research reports and cache
    ├── co-writer/                # Interactive IdeaGen documents and audio files
    ├── notebook/                 # Notebook records and metadata
    ├── guide/                    # Guided learning sessions
    ├── logs/                     # System logs
    └── run_code_workspace/       # Code execution workspace
```

Results are automatically saved during all activities. Directories are created automatically as needed.

## 📦 Core Modules

<details>
<summary><b>🧠 Smart Solver</b></summary>

<details>
<summary><b>Architecture Diagram</b></summary>

![Smart Solver Architecture](assets/figs/solve.png)

</details>

> **Intelligent problem-solving system** based on **Analysis Loop + Solve Loop** dual-loop architecture, supporting multi-mode reasoning and dynamic knowledge retrieval.

**Core Features**

| Feature | Description |
|:---:|:---|
| Dual-Loop Architecture | **Analysis Loop**: InvestigateAgent → NoteAgent<br>**Solve Loop**: PlanAgent → ManagerAgent → SolveAgent → CheckAgent → Format |
| Multi-Agent Collaboration | Specialized agents: InvestigateAgent, NoteAgent, PlanAgent, ManagerAgent, SolveAgent, CheckAgent |
| Real-time Streaming | WebSocket transmission with live reasoning process display |
| Tool Integration | RAG (naive/hybrid), Web Search, Query Item, Code Execution |
| Persistent Memory | JSON-based memory files for context preservation |
| Citation Management | Structured citations with reference tracking |

**Usage**

1. Visit http://localhost:{frontend_port}/solver
2. Select a knowledge base
3. Enter your question, click "Solve"
4. Watch the real-time reasoning process and final answer

<details>
<summary><b>Python API</b></summary>

```python
import asyncio
from src.agents.solve import MainSolver

async def main():
    solver = MainSolver(kb_name="ai_textbook")
    result = await solver.solve(
        question="Calculate the linear convolution of x=[1,2,3] and h=[4,5]",
        mode="auto"
    )
    print(result['formatted_solution'])

asyncio.run(main())
```

</details>

<details>
<summary><b>Output Location</b></summary>

```
data/user/solve/solve_YYYYMMDD_HHMMSS/
├── investigate_memory.json    # Analysis Loop memory
├── solve_chain.json           # Solve Loop steps & tool records
├── citation_memory.json       # Citation management
├── final_answer.md            # Final solution (Markdown)
├── performance_report.json    # Performance monitoring
└── artifacts/                 # Code execution outputs
```

</details>

</details>

---

<details>
<summary><b>📝 Question Generator</b></summary>

<details>
<summary><b>Architecture Diagram</b></summary>

![Question Generator Architecture](assets/figs/question-gen.png)

</details>

> **Dual-mode question generation system** supporting **custom knowledge-based generation** and **reference exam paper mimicking** with automatic validation.

**Core Features**

| Feature | Description |
|:---:|:---|
| Custom Mode | **Background Knowledge** → **Question Planning** → **Generation** → **Single-Pass Validation**<br>Analyzes question relevance without rejection logic |
| Mimic Mode | **PDF Upload** → **MinerU Parsing** → **Question Extraction** → **Style Mimicking**<br>Generates questions based on reference exam structure |
| ReAct Engine | QuestionGenerationAgent with autonomous decision-making (think → act → observe) |
| Validation Analysis | Single-pass relevance analysis with `kb_coverage` and `extension_points` |
| Question Types | Multiple choice, fill-in-the-blank, calculation, written response, etc. |
| Batch Generation | Parallel processing with progress tracking |
| Complete Persistence | All intermediate files saved (background knowledge, plan, individual results) |
| Timestamped Output | Mimic mode creates batch folders: `mimic_YYYYMMDD_HHMMSS_{pdf_name}/` |

**Usage**

**Custom Mode:**
1. Visit http://localhost:{frontend_port}/question
2. Fill in requirements (topic, difficulty, question type, count)
3. Click "Generate Questions"
4. View generated questions with validation reports

**Mimic Mode:**
1. Visit http://localhost:{frontend_port}/question
2. Switch to "Mimic Exam" tab
3. Upload PDF or provide parsed exam directory
4. Wait for parsing → extraction → generation
5. View generated questions alongside original references

<details>
<summary><b>Python API</b></summary>

**Custom Mode - Full Pipeline:**
```python
import asyncio
from src.agents.question import AgentCoordinator

async def main():
    coordinator = AgentCoordinator(
        kb_name="ai_textbook",
        output_dir="data/user/question"
    )

    # Generate multiple questions from text requirement
    result = await coordinator.generate_questions_custom(
        requirement_text="Generate 3 medium-difficulty questions about deep learning basics",
        difficulty="medium",
        question_type="choice",
        count=3
    )

    print(f"✅ Generated {result['completed']}/{result['requested']} questions")
    for q in result['results']:
        print(f"- Relevance: {q['validation']['relevance']}")

asyncio.run(main())
```

**Mimic Mode - PDF Upload:**
```python
from src.agents.question.tools.exam_mimic import mimic_exam_questions

result = await mimic_exam_questions(
    pdf_path="exams/midterm.pdf",
    kb_name="calculus",
    output_dir="data/user/question/mimic_papers",
    max_questions=5
)

print(f"✅ Generated {result['successful_generations']} questions")
print(f"Output: {result['output_file']}")
```

</details>

<details>
<summary><b>Output Location</b></summary>

**Custom Mode:**
```
data/user/question/custom_YYYYMMDD_HHMMSS/
├── background_knowledge.json      # RAG retrieval results
├── question_plan.json              # Question planning
├── question_1_result.json          # Individual question results
├── question_2_result.json
└── ...
```

**Mimic Mode:**
```
data/user/question/mimic_papers/
└── mimic_YYYYMMDD_HHMMSS_{pdf_name}/
    ├── {pdf_name}.pdf                              # Original PDF
    ├── auto/{pdf_name}.md                          # MinerU parsed markdown
    ├── {pdf_name}_YYYYMMDD_HHMMSS_questions.json  # Extracted questions
    └── {pdf_name}_YYYYMMDD_HHMMSS_generated_questions.json  # Generated questions
```

</details>

</details>

---

<details>
<summary><b>🎓 Guided Learning</b></summary>

<details>
<summary><b>Architecture Diagram</b></summary>

![Guided Learning Architecture](assets/figs/guide.png)

</details>

> **Personalized learning system** based on notebook content, automatically generating progressive learning paths through interactive pages and smart Q&A.

**Core Features**

| Feature | Description |
|:---:|:---|
| Multi-Agent Architecture | **LocateAgent**: Identifies 3-5 progressive knowledge points<br>**InteractiveAgent**: Converts to visual HTML pages<br>**ChatAgent**: Provides contextual Q&A<br>**SummaryAgent**: Generates learning summaries |
| Smart Knowledge Location | Automatic analysis of notebook content |
| Interactive Pages | HTML page generation with bug fixing |
| Smart Q&A | Context-aware answers with explanations |
| Progress Tracking | Real-time status with session persistence |
| Cross-Notebook Support | Select records from multiple notebooks |

**Usage Flow**

1. **Select Notebook(s)** — Choose one or multiple notebooks (cross-notebook selection supported)
2. **Generate Learning Plan** — LocateAgent identifies 3-5 core knowledge points
3. **Start Learning** — InteractiveAgent generates HTML visualization
4. **Learning Interaction** — Ask questions, click "Next" to proceed
5. **Complete Learning** — SummaryAgent generates learning summary

<details>
<summary><b>Output Location</b></summary>

```
data/user/guide/
└── session_{session_id}.json    # Complete session state, knowledge points, chat history
```

</details>

</details>

---

<details>
<summary><b>✏️ Interactive IdeaGen (Co-Writer)</b></summary>

<details>
<summary><b>Architecture Diagram</b></summary>

![Interactive IdeaGen Architecture](assets/figs/co-writer.png)

</details>

> **Intelligent Markdown editor** supporting AI-assisted writing, auto-annotation, and TTS narration.

**Core Features**

| Feature | Description |
|:---:|:---|
| Rich Text Editing | Full Markdown syntax support with live preview |
| EditAgent | **Rewrite**: Custom instructions with optional RAG/web context<br>**Shorten**: Compress while preserving key information<br>**Expand**: Add details and context |
| Auto-Annotation | Automatic key content identification and marking |
| NarratorAgent | Script generation, TTS audio, multiple voices (Cherry, Stella, Annie, Cally, Eva, Bella) |
| Context Enhancement | Optional RAG or web search for additional context |
| Multi-Format Export | Markdown, PDF, etc. |

**Usage**

1. Visit http://localhost:{frontend_port}/co_writer
2. Enter or paste text in the editor
3. Use AI features: Rewrite, Shorten, Expand, Auto Mark, Narrate
4. Export to Markdown or PDF

<details>
<summary><b>Output Location</b></summary>

```
data/user/co-writer/
├── audio/                    # TTS audio files
│   └── {operation_id}.mp3
├── tool_calls/               # Tool call history
│   └── {operation_id}_{tool_type}.json
└── history.json              # Edit history
```

</details>

</details>

---

<details>
<summary><b>🔬 Deep Research</b></summary>

<details>
<summary><b>Architecture Diagram</b></summary>

![Deep Research Architecture](assets/figs/deepresearch.png)

</details>

> **DR-in-KG** (Deep Research in Knowledge Graph) — A systematic deep research system based on **Dynamic Topic Queue** architecture, enabling multi-agent collaboration across three phases: **Planning → Researching → Reporting**.

**Core Features**

| Feature | Description |
|:---:|:---|
| Three-Phase Architecture | **Phase 1 (Planning)**: RephraseAgent (topic optimization) + DecomposeAgent (subtopic decomposition)<br>**Phase 2 (Researching)**: ManagerAgent (queue scheduling) + ResearchAgent (research decisions) + NoteAgent (info compression)<br>**Phase 3 (Reporting)**: Deduplication → Three-level outline generation → Report writing with citations |
| Dynamic Topic Queue | Core scheduling system with TopicBlock state management: `PENDING → RESEARCHING → COMPLETED/FAILED`. Supports dynamic topic discovery during research |
| Execution Modes | **Series Mode**: Sequential topic processing<br>**Parallel Mode**: Concurrent multi-topic processing with `AsyncCitationManagerWrapper` for thread-safe operations |
| Multi-Tool Integration | **RAG** (hybrid/naive), **Query Item** (entity lookup), **Paper Search**, **Web Search**, **Code Execution** — dynamically selected by ResearchAgent |
| Unified Citation System | Centralized CitationManager as single source of truth for citation ID generation, ref_number mapping, and deduplication |
| Preset Configurations | **quick**: Fast research (1-2 subtopics, 1-2 iterations)<br>**medium/standard**: Balanced depth (5 subtopics, 4 iterations)<br>**deep**: Thorough research (8 subtopics, 7 iterations)<br>**auto**: Agent autonomously decides depth |

**Citation System Architecture**

The citation system follows a centralized design with CitationManager as the single source of truth:

```
┌─────────────────────────────────────────────────────────────────┐
│                      CitationManager                            │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐  │
│  │  ID Generation  │  │  ref_number Map │  │   Deduplication │  │
│  │  PLAN-XX        │  │  citation_id →  │  │   (papers only) │  │
│  │  CIT-X-XX       │  │  ref_number     │  │                 │  │
│  └────────┬────────┘  └────────┬────────┘  └────────┬────────┘  │
└───────────┼────────────────────┼────────────────────┼───────────┘
            │                    │                    │
     ┌──────┴──────┐      ┌──────┴──────┐      ┌──────┴──────┐
     │DecomposeAgent│      │ReportingAgent│      │ References │
     │ ResearchAgent│      │ (inline [N]) │      │  Section   │
     │  NoteAgent   │      └─────────────┘      └────────────┘
     └─────────────┘
```

| Component | Description |
|:---:|:---|
| ID Format | **PLAN-XX** (planning stage RAG queries) + **CIT-X-XX** (research stage, X=block number) |
| ref_number Mapping | Sequential 1-based numbers built from sorted citation IDs, with paper deduplication |
| Inline Citations | Simple `[N]` format in LLM output, post-processed to clickable `[[N]](#ref-N)` links |
| Citation Table | Clear reference table provided to LLM: `Cite as [1] → (RAG) query preview...` |
| Post-processing | Automatic format conversion + validation to remove invalid citation references |
| Parallel Safety | Thread-safe async methods (`get_next_citation_id_async`, `add_citation_async`) for concurrent execution |

**Parallel Execution Architecture**

When `execution_mode: "parallel"` is enabled, multiple topic blocks are researched concurrently:

```
┌─────────────────────────────────────────────────────────────────────────┐
│                    Parallel Research Execution                          │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│   DynamicTopicQueue                    AsyncCitationManagerWrapper      │
│   ┌─────────────────┐                  ┌─────────────────────────┐      │
│   │ Topic 1 (PENDING)│ ──┐             │  Thread-safe wrapper    │      │
│   │ Topic 2 (PENDING)│ ──┼──→ asyncio  │  for CitationManager    │      │
│   │ Topic 3 (PENDING)│ ──┤   Semaphore │                         │      │
│   │ Topic 4 (PENDING)│ ──┤   (max=5)   │  • get_next_citation_   │      │
│   │ Topic 5 (PENDING)│ ──┘             │    id_async()           │      │
│   └─────────────────┘                  │  • add_citation_async() │      │
│            │                           └───────────┬─────────────┘      │
│            ▼                                       │                    │
│   ┌─────────────────────────────────────────────────────────────┐      │
│   │              Concurrent ResearchAgent Tasks                  │      │
│   │  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐        │      │
│   │  │ Task 1  │  │ Task 2  │  │ Task 3  │  │ Task 4  │  ...   │      │
│   │  │(Topic 1)│  │(Topic 2)│  │(Topic 3)│  │(Topic 4)│        │      │
│   │  └────┬────┘  └────┬────┘  └────┬────┘  └────┬────┘        │      │
│   │       │            │            │            │              │      │
│   │       └────────────┴────────────┴────────────┘              │      │
│   │                         │                                    │      │
│   │                         ▼                                    │      │
│   │              AsyncManagerAgentWrapper                        │      │
│   │              (Thread-safe queue updates)                     │      │
│   └─────────────────────────────────────────────────────────────┘      │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘
```

| Component | Description |
|:---:|:---|
| `asyncio.Semaphore` | Limits concurrent tasks to `max_parallel_topics` (default: 5) |
| `AsyncCitationManagerWrapper` | Wraps CitationManager with `asyncio.Lock()` for thread-safe ID generation |
| `AsyncManagerAgentWrapper` | Ensures queue state updates are atomic across parallel tasks |
| Real-time Progress | Live display of all active research tasks with status indicators |

**Agent Responsibilities**

| Agent | Phase | Responsibility |
|:---:|:---:|:---|
| RephraseAgent | Planning | Optimizes user input topic, supports multi-turn user interaction for refinement |
| DecomposeAgent | Planning | Decomposes topic into subtopics with RAG context, obtains citation IDs from CitationManager |
| ManagerAgent | Researching | Queue state management, task scheduling, dynamic topic addition |
| ResearchAgent | Researching | Knowledge sufficiency check, query planning, tool selection, requests citation IDs before each tool call |
| NoteAgent | Researching | Compresses raw tool outputs into summaries, creates ToolTraces with pre-assigned citation IDs |
| ReportingAgent | Reporting | Builds citation map, generates three-level outline, writes report sections with citation tables, post-processes citations |

**Report Generation Pipeline**

```
1. Build Citation Map     →  CitationManager.build_ref_number_map()
2. Generate Outline       →  Three-level headings (H1 → H2 → H3)
3. Write Sections         →  LLM uses [N] citations with provided citation table
4. Post-process           →  Convert [N] → [[N]](#ref-N), validate references
5. Generate References    →  Academic-style entries with collapsible source details
```

**Usage**

1. Visit http://localhost:{frontend_port}/research
2. Enter research topic
3. Select research mode (quick/medium/deep/auto)
4. Watch real-time progress with parallel/series execution
5. View structured report with clickable inline citations
6. Export as Markdown or PDF (with proper page splitting and Mermaid diagram support)

<details>
<summary><b>CLI</b></summary>

```bash
# Quick mode (fast research)
python src/agents/research/main.py --topic "Deep Learning Basics" --preset quick

# Medium mode (balanced)
python src/agents/research/main.py --topic "Transformer Architecture" --preset medium

# Deep mode (thorough research)
python src/agents/research/main.py --topic "Graph Neural Networks" --preset deep

# Auto mode (agent decides depth)
python src/agents/research/main.py --topic "Reinforcement Learning" --preset auto
```

</details>

<details>
<summary><b>Python API</b></summary>

```python
import asyncio
from src.agents.research import ResearchPipeline
from src.core.core import get_llm_config, load_config_with_main

async def main():
    # Load configuration (main.yaml merged with any module-specific overrides)
    config = load_config_with_main("research_config.yaml")
    llm_config = get_llm_config()

    # Create pipeline (agent parameters loaded from agents.yaml automatically)
    pipeline = ResearchPipeline(
        config=config,
        api_key=llm_config["api_key"],
        base_url=llm_config["base_url"],
        kb_name="ai_textbook"  # Optional: override knowledge base
    )

    # Run research
    result = await pipeline.run(topic="Attention Mechanisms in Deep Learning")
    print(f"Report saved to: {result['final_report_path']}")

asyncio.run(main())
```

</details>

<details>
<summary><b>Output Location</b></summary>

```
data/user/research/
├── reports/                          # Final research reports
│   ├── research_YYYYMMDD_HHMMSS.md   # Markdown report with clickable citations [[N]](#ref-N)
│   └── research_*_metadata.json      # Research metadata and statistics
└── cache/                            # Research process cache
    └── research_YYYYMMDD_HHMMSS/
        ├── queue.json                # DynamicTopicQueue state (TopicBlocks + ToolTraces)
        ├── citations.json            # Citation registry with ID counters and ref_number mapping
        │                             #   - citations: {citation_id: citation_info}
        │                             #   - counters: {plan_counter, block_counters}
        ├── step1_planning.json       # Planning phase results (subtopics + PLAN-XX citations)
        ├── planning_progress.json    # Planning progress events
        ├── researching_progress.json # Researching progress events
        ├── reporting_progress.json   # Reporting progress events
        ├── outline.json              # Three-level report outline structure
        └── token_cost_summary.json   # Token usage statistics
```

**Citation File Structure** (`citations.json`):
```json
{
  "research_id": "research_20241209_120000",
  "citations": {
    "PLAN-01": {"citation_id": "PLAN-01", "tool_type": "rag_hybrid", "query": "...", "summary": "..."},