RAG + Web Search Agent — AutoGen × Saf3AI

A production-ready AI research agent powered by Microsoft AutoGen 0.4.x, secured and observed by the Saf3AI SDK.
Combines a local RAG knowledge base with live web search — available as a Gradio web UI or a terminal CLI.

What It Does

Send any question in the chat. The agent will:

Search the internal knowledge base (ChromaDB + local embeddings) for relevant context
Fall back to live web search (DuckDuckGo) if the KB has nothing useful
Answer with cited sources — knowledge base file name or web URL
Block dangerous prompts and responses via Saf3AI real-time security scanning
Export full conversation traces to the Saf3AI Analyzer dashboard via OpenTelemetry

You can also attach images or documents — they are scanned and injected into the conversation automatically.

Features

Feature	Details
Multi-provider LLM	OpenAI GPT-4o mini, Google Gemini 2.5 Flash Lite, Groq LLaMA 3.3 70B
RAG knowledge base	ChromaDB + `sentence-transformers` (`all-MiniLM-L6-v2`); extend by dropping `.txt` files into `data/`
Live web search	DuckDuckGo via `duckduckgo-search` — no API key required
File upload	Images (JPEG, PNG, GIF, WebP, BMP) and documents (PDF, DOCX, TXT, CSV, JSON, code files)
Security scanning	Every prompt & response scanned for injection, jailbreaks, hate speech, PII leakage, malicious URIs
Observability	Full OTel traces (prompt, response, tokens, threats) exported to Saf3AI Analyzer
Gradio UI	Browser-based chat with sidebar provider selector and file picker
CLI mode	Lightweight terminal chat loop via `main.py`

Architecture

┌─────────────────────────────────────────────────────────┐
│                   User Interface                        │
│          Gradio Web UI (app.py)  /  CLI (main.py)       │
└────────────────────────┬────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────┐
│              AutoGen AssistantAgent                     │
│                                                         │
│   ┌─────────────────┐    ┌──────────────────────────┐  │
│   │   rag_search    │    │      web_search           │  │
│   │  ChromaDB +     │    │   DuckDuckGo (live)       │  │
│   │  sentence-      │    │   No API key needed       │  │
│   │  transformers   │    │                           │  │
│   └─────────────────┘    └──────────────────────────┘  │
│                                                         │
│          LLM: OpenAI / Google Gemini / Groq             │
└────────────────────────┬────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────┐
│                  Saf3AI SDK Layer                       │
│  (wraps every on_messages call transparently)           │
│                                                         │
│  ┌──────────────────────────┐  ┌─────────────────────┐ │
│  │   Security Scanner       │  │   OTel Exporter      │ │
│  │   scanner-dev.saf3ai.com │  │   analyzer-dev.      │ │
│  │   Prompt + response scan │  │   saf3ai.com         │ │
│  └──────────────────────────┘  └─────────────────────┘ │
└─────────────────────────────────────────────────────────┘

Prerequisites

Python 3.10 – 3.12
Saf3AI account — API key from saf3ai.com
At least one LLM API key — OpenAI, Google AI Studio, or Groq (all free tiers available)

Quick Start

1. Clone the repository

git clone https://huggingface.co/satyamsaf3ai/rag-websearch-autogen
cd rag-websearch-autogen

2. Create a virtual environment

python -m venv .venv

# Windows
.venv\Scripts\activate

# macOS / Linux
source .venv/bin/activate

3. Install the Saf3AI SDK

pip install git+https://github.com/saf3ai/saf3ai_sdk.git@satyam-dev

4. Install project dependencies

pip install -r requirements.txt

5. Configure environment

# Windows
copy .env.example .env

# macOS / Linux
cp .env.example .env

Edit .env and fill in your API keys:

LLM_PROVIDER=openai
OPENAI_API_KEY=sk-...
SAF3AI_API_KEY=your-key
SECURITY_API_KEY=your-key
SECURITY_API_ENDPOINT=https://scanner-dev.saf3ai.com
SAF3AI_COLLECTOR_AGENT=https://analyzer-dev.saf3ai.com/v1/traces

6. Run

python app.py

Open http://127.0.0.1:7865 in your browser.

(Optional) Enable PDF and Word document support

pip install pymupdf python-docx

Configuration

Copy the example file

# macOS / Linux
cp .env.example .env

# Windows
copy .env.example .env

Edit `.env`

# ── Pick your LLM provider ────────────────────────────────
LLM_PROVIDER=openai          # openai | gemini | groq

OPENAI_API_KEY=sk-...        # if LLM_PROVIDER=openai
GOOGLE_API_KEY=AIza...       # if LLM_PROVIDER=gemini
GROQ_API_KEY=gsk_...         # if LLM_PROVIDER=groq

# ── Saf3AI ────────────────────────────────────────────────
SAF3AI_API_KEY=your-key
SAF3AI_AGENT_ID=rag-websearch-autogen
SAF3AI_COLLECTOR_AGENT=https://analyzer-dev.saf3ai.com/v1/traces

SECURITY_API_ENDPOINT=https://scanner-dev.saf3ai.com
SECURITY_API_KEY=your-key
THREAT_ACTION_LEVEL=BLOCK    # BLOCK | WARN | OFF

Full variable reference

Variable	Required	Default	Description
`LLM_PROVIDER`	Yes	`openai`	Active LLM provider
`OPENAI_API_KEY`	If `openai`	—	OpenAI API key
`OPENAI_MODEL`	No	`gpt-4o-mini`	OpenAI model name
`GOOGLE_API_KEY`	If `gemini`	—	Google AI Studio key
`GEMINI_MODEL`	No	`gemini-2.5-flash-lite`	Gemini model name
`GROQ_API_KEY`	If `groq`	—	Groq API key
`GROQ_MODEL`	No	`llama-3.3-70b-versatile`	Groq model name
`SAF3AI_API_KEY`	Yes	—	Saf3AI org API key
`SAF3AI_AGENT_ID`	No	`rag-websearch-autogen`	Agent label in dashboard
`SAF3AI_COLLECTOR_AGENT`	Yes	`http://localhost:19999`	OTel traces endpoint
`SECURITY_SCAN_ENABLED`	No	`true`	Set to `false` to disable scanning
`SECURITY_API_ENDPOINT`	Yes	—	Saf3AI Scanner URL
`SECURITY_API_KEY`	Yes	—	Scanner API key (same as `SAF3AI_API_KEY`)
`SECURITY_API_TIMEOUT`	No	`30`	Scanner request timeout (seconds)
`THREAT_ACTION_LEVEL`	No	`BLOCK`	`BLOCK` / `WARN` / `OFF`
`RAG_TOP_K`	No	`4`	Documents returned per RAG query
`RAG_EMBED_MODEL`	No	`all-MiniLM-L6-v2`	Sentence-transformers model
`ENVIRONMENT`	No	`development`	Shown in Analyzer traces

Running

Gradio Web UI

python app.py

Open http://127.0.0.1:7865 in your browser.

UI walkthrough:

Left sidebar — shows active tools and lets you switch LLM provider
Chat area — type your question and press Enter or click Send
File picker — attach an image or document before sending; it is scanned and included in the message context
Clear button — resets the conversation

Terminal CLI

python main.py

============================================================
  RAG + Web Search Agent  (AutoGen + Saf3AI)
============================================================
  Provider: openai
  Type 'exit' to quit.

You: What is prompt injection?
Agent: Prompt injection is an attack where...

Adding Custom Knowledge

The agent ships with 5 built-in documents covering: Saf3AI SDK, Microsoft AutoGen, RAG, OpenTelemetry, and LLM security.

Add your own documents

Create the data/ directory if it doesn't exist:
```
mkdir data
```
Drop any .txt files into data/ — they are loaded automatically on the next startup.

To ingest files without restarting the app:

# All .txt files in data/
python scripts/ingest_docs.py

# Specific files
python scripts/ingest_docs.py path/to/file1.txt path/to/file2.txt

The agent will now prefer these documents when answering relevant questions.

Saf3AI Security & Observability

How scanning works

The Saf3AI SDK patches AutoGen's on_messages method transparently. For every user message:

User sends message
      │
      ▼
[Saf3AI] Prompt scan ──► POST /scan → scanner endpoint
      │
      ├── THREAT FOUND  →  raise ValueError  →  "🚫 Blocked by Saf3AI"
      │
      └── CLEAN  ──────→  LLM called normally
                                │
                                ▼
                         [Saf3AI] Response scan (non-blocking, logged only)
                                │
                                ▼
                         Answer shown in UI

Threat action levels

Level	Behaviour
`BLOCK`	Dangerous prompts and responses are blocked; user sees an error message
`WARN`	Threats are logged as warnings but the request proceeds
`OFF`	Threat enforcement is disabled (scanning still runs, results only logged)

File upload scanning

Images are scanned with the Saf3AI image scanner before display.
Documents are scanned with the Saf3AI document scanner before text extraction.
Unsafe files are rejected with a 🚫 blocked message — content never reaches the LLM.

Observability dashboard

Every conversation emits an OTel span to SAF3AI_COLLECTOR_AGENT containing:

Conversation ID and agent name
Full prompt and response text
Security scan results — detected categories, threat types, severity
Token usage

View all traces at the Saf3AI Analyzer dashboard.

Project Structure

rag-websearch-autogen/
│
├── app.py                  # Gradio web UI — main entry point
├── main.py                 # Terminal CLI entry point
├── requirements.txt        # Pinned dependencies
├── pyproject.toml          # Package metadata
├── .env.example            # Configuration template (copy → .env)
│
├── autogen_agent/
│   ├── agent.py            # AssistantAgent factory + Saf3AI SDK init
│   ├── llm.py              # LLM client factory (OpenAI / Gemini / Groq)
│   ├── tools.py            # rag_search + web_search FunctionTools
│   ├── security.py         # Threat policy + scanner setup
│   └── rag/
│       └── store.py        # ChromaDB in-memory vector store
│
├── scripts/
│   └── ingest_docs.py      # Batch document ingestion CLI
│
└── data/                   # Drop .txt files here to extend the KB
                            # (auto-loaded on startup)

Switching LLM Providers

Set LLM_PROVIDER in .env and restart. The UI also has a live provider selector in the sidebar.

Provider	Model	Notes
`openai`	`gpt-4o-mini`	Best tool-calling reliability
`gemini`	`gemini-2.5-flash-lite`	Fast, low cost
`groq`	`llama-3.3-70b-versatile`	Very fast inference

Troubleshooting

model_info is required error
AutoGen doesn't recognise the model name. Make sure LLM_PROVIDER matches one of openai, gemini, or groq exactly.

Chatbot.__init__() got an unexpected keyword argument
Gradio version mismatch. Run pip install gradio==6.19.0.

OSError: Cannot find empty port
Port 7865 is already in use. Change server_port in app.py or kill the existing process.

Scanner not appearing in dashboard
Check that SAF3AI_COLLECTOR_AGENT points to your Analyzer endpoint (not localhost) and that SECURITY_API_ENDPOINT is set correctly.

No documents found in RAG search
Make sure the data/ directory exists and contains .txt files, or run python scripts/ingest_docs.py to verify ingestion.

License

MIT

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support