pdf2tex 0.2.0
High-performance RAG-based PDF to LaTeX conversion module
Requires Python: >=3.11
Keywords: document-conversion, latex, llm, ocr, pdf, rag

# PDF2TeX

High-performance RAG-based PDF to LaTeX conversion module for large documents (2000+ pages).

## Features

- **Intelligent PDF Extraction**: Multi-path content processing with PyMuPDF, Nougat, and PaddleOCR
- **Math-First Approach**: 95%+ accuracy on mathematical content using neural equation recognition
- **RAG-Powered Generation**: Context-aware LaTeX synthesis with Hugging Face LLMs
- **Distributed Processing**: Ray-based parallel processing for high throughput
- **Chapter-Based Output**: One `.tex` file per chapter with master document

## Architecture

```
PDF Input → Extract + OCR → Chunk + Index → RAG + LLM → LaTeX Output
                ↓                ↓              ↓
            Ray Distributed Workers Pool
                        ↓
        Qdrant | Redis | MinIO | Postgres
```

## Quick Start

### Prerequisites

- Python 3.11+
- Docker & Docker Compose
- NVIDIA GPU (recommended)
- Hugging Face API token

### Installation

```bash
# Clone repository
git clone https://github.com/pdf2tex/pdf2tex.git
cd pdf2tex

# Create virtual environment
python -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -e ".[dev]"

# Start infrastructure
docker-compose up -d

# Run conversion
pdf2tex convert input.pdf --output ./output
```

### Configuration

Create a `.env` file:

```env
# Hugging Face
HUGGINGFACE_TOKEN=hf_xxxxxxxxxxxxx

# Database
POSTGRES_URL=postgresql+asyncpg://pdf2tex:password@localhost:5432/pdf2tex

# Vector Store
QDRANT_URL=http://localhost:6333

# Redis
REDIS_URL=redis://localhost:6379

# MinIO
MINIO_ENDPOINT=localhost:9000
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin

# Ray
RAY_ADDRESS=auto
```

## Usage

### CLI

```bash
# Convert a PDF
pdf2tex convert document.pdf --output ./output

# Resume failed conversion
pdf2tex resume doc_abc123

# Check status
pdf2tex status doc_abc123
```

### API

```bash
# Start API server
uvicorn pdf2tex.api.app:app --host 0.0.0.0 --port 8000

# Submit document
curl -X POST http://localhost:8000/documents \
  -F "file=@textbook.pdf"

# Check status
curl http://localhost:8000/documents/doc_abc123
```

### Python SDK

```python
from pdf2tex import PDF2TeX

converter = PDF2TeX()
result = await converter.convert("textbook.pdf", output_dir="./output")
print(f"Converted {result.total_pages} pages in {result.duration}")
```

## Project Structure

```
pdf2tex/
├── src/pdf2tex/
│   ├── extraction/     # PDF parsing, OCR, math extraction
│   ├── chunking/       # Text splitting, chapter detection
│   ├── rag/            # Embeddings, vector store, retrieval
│   ├── generation/     # LLM integration, LaTeX synthesis
│   ├── pipeline/       # Orchestration, distributed workers
│   └── api/            # FastAPI endpoints
├── tests/
├── docker-compose.yml
└── pyproject.toml
```

## Performance

| Document Size | Processing Time | Workers |
|--------------|-----------------|---------|
| 500 pages    | ~20 min         | 10      |
| 1000 pages   | ~40 min         | 20      |
| 2000 pages   | ~72 min         | 20      |

## License

MIT License - see [LICENSE](LICENSE) for details.