ecdallm 0.2.9
Retrieval-Augmented Generation - (RAG)
License: MIT
Requires Python: <3.15,>=3.11
Keywords: rag, llm, nlp, data, vector

# ecdallm

[![PyPI
version](https://badge.fury.io/py/ecdallm.svg)](https://pypi.org/project/ecdallm/)
[![Python
versions](https://img.shields.io/pypi/pyversions/ecdallm.svg)](https://pypi.org/project/ecdallm/)
[![License](https://img.shields.io/pypi/l/ecdallm.svg)](LICENCE)

**ecdallm** is a lightweight Retrieval-Augmented Generation (RAG)
application that lets you chat with your own documents using either a
**local LLM** or an **external OpenAI-compatible provider**.

It combines:

-   FastAPI web interface
-   Local embedding pipeline (FastEmbed)
-   Persistent vector storage (ChromaDB)
-   Document ingestion (PDF, TXT, DOCX)
-   CLI launcher
-   Local LLM support (e.g., LM Studio)
-   External LLM support (OpenAI-compatible APIs)

The goal is to provide a simple, reproducible environment for
**document-grounded LLM interaction** with flexible model connectivity.

------------------------------------------------------------------------

## Overview

`ecdallm` allows you to:

1.  Upload documents
2.  Index them into a vector database
3.  Run semantic retrieval
4.  Query an LLM with grounded context

All embeddings and vector storage run locally.

The chat model can run:

-   locally (LM Studio, Ollama, etc.)
-   externally (OpenRouter or OpenAI-compatible APIs)

This makes the system suitable for:

-   research environments
-   private document analysis
-   offline experimentation
-   RAG prototyping
-   hybrid local/cloud workflows

------------------------------------------------------------------------

## Installation

Install from PyPI:

``` bash
pip install ecdallm
```

------------------------------------------------------------------------

## Running the application

Start the CLI:

``` bash
ecdallm
```

The CLI will:

-   find a free port (starting from 8000)
-   start the FastAPI server
-   open the browser automatically

Example output:

    ecdallm running at http://127.0.0.1:8000/
    INFO: Uvicorn running on http://127.0.0.1:8000

------------------------------------------------------------------------

## LLM Configuration

When the application starts, click **Continue** and choose:

-   Local LLM
-   External LLM

Configuration is stored in the browser session.

Embeddings always run locally using FastEmbed with:

    nomic-embed-text-v1.5

------------------------------------------------------------------------

## Using a local LLM

`ecdallm` expects an OpenAI-compatible endpoint.

For example, with **LM Studio**:

1.  Start LM Studio server
2.  Load a chat model
3.  Enable the local API server

Typical endpoint:

    http://localhost:1234/v1

Default configuration:

    Base URL: http://localhost:1234/v1
    API Key: lm-studio

The backend automatically detects the available chat model via:

    GET /models

------------------------------------------------------------------------

## Using an external LLM

`ecdallm` can connect to any **OpenAI-compatible API provider**.

Examples include:

-   OpenRouter
-   OpenAI-compatible gateways
-   Self-hosted inference APIs

Example configuration (OpenRouter):

    Base URL: https://openrouter.ai/api/v1
    Model: openrouter/aurora-alpha
    API Key: sk-or-...

Steps:

1.  Create an account with the provider
2.  Generate an API key
3.  Choose a chat model
4.  Enter the configuration in the web interface

When validating, `ecdallm`:

-   checks connectivity
-   performs a test chat completion
-   stores configuration in session storage

Your API key is sent only to your backend for validation and is **not
used directly in the browser**.

Embeddings remain local.

------------------------------------------------------------------------

## Supported document types

-   PDF
-   TXT
-   DOCX

------------------------------------------------------------------------

## Workflow

### 1. Upload documents

Use the **Upload** page to add files.

### 2. Index documents

Files are automatically indexed into ChromaDB using FastEmbed.

### 3. Chat with documents

Open the **Chat** page and ask questions.

The assistant will:

-   retrieve relevant chunks
-   build a grounded prompt
-   query the configured LLM
-   return a concise answer

------------------------------------------------------------------------

## Project structure

    ecdallm/
    ├── cli.py
    └── app/
        ├── main.py
        ├── rag.py
        ├── paths.py
        ├── search_engine.py
        ├── vector.py
        ├── templates/
        ├── static/
        ├── uploads/
        └── rag_store/

------------------------------------------------------------------------

## Notes

Embeddings and retrieval always run locally.

The chat model can be:

-   local (LM Studio, Ollama, etc.)
-   external (OpenAI-compatible providers)

This keeps the system flexible while maintaining local document
processing.

------------------------------------------------------------------------

## Erasmus Data Collaboratory

Developed by the Erasmus Data Collaboratory (ECDA). 
- Zaman Ziabakhshganji --- creator and maintainer 
- Farshad Radman --- co-author and contributor 
- Jos van Dongen --- co-author and contributor

------------------------------------------------------------------------

## License

MIT License