npm package: @grouchoab/victor
Description: miniscule vector database.
Author: dspnikder
Latest version: 0.3.5

# Victor

A miniscule vector database for Node.js, running entirely locally.

## Overview

Victor is a lightweight, file-based vector database designed for semantic search on small to medium-sized document sets (up to ~100k embeddings). It uses `@xenova/transformers` to generate embeddings locally, making it perfect for offline-first applications or projects where data privacy is a concern.

## Features

*   **Local First**: No external services or API keys required.
*   **Simple API**: Easy to index documents and perform semantic searches.
*   **File-based**: The index is a simple JSON file, making it portable and easy to manage.
*   **Customizable**: Supports various embedding models from Hugging Face.
*   **Filtering**: Supports metadata-based filtering during search.

## Installation

```bash
npm install @grouchoab/victor
```

## Usage

Here's a quick example of how to use Victor for semantic search.

First, you need to set the `DATA_ROOT` environment variable. This is where Victor will store its index.

```typescript
import { Victor } from '@grouchoab/victor';

// Make sure DATA_ROOT is set, e.g., process.env.DATA_ROOT = './data';

async function main() {
  const victor = new Victor();

  // The index is built automatically if it doesn't exist,
  // using files from `${process.env.DATA_ROOT}/examples` and `${process.env.DATA_ROOT}/requirements`.
  // You can customize this by calling `buildStandardIndex` yourself.

  const query = "what are the security requirements?";
  const results = await victor.query(query);

  console.log(`Found ${results.length} relevant files for "${query}":`);
  results.forEach(file => {
    console.log(`- ${file}`);
  });
}

main();
```

## Architecture

Victor's architecture is composed of three main components:

1.  **Local Embeddings**: Handles text chunking and embedding generation using `transformers.js`.
2.  **Vector Index**: Builds the searchable index from a collection of documents.
3.  **Vector Query Engine**: Performs the similarity search against the index.

Here's a diagram illustrating the flow:

```mermaid
graph TD
    A[Documents] --> B(Vector Index Builder);
    B --> C{index.json};
    D[Query String] --> E(Local Embeddings);
    E --> F(Vector Query Engine);
    C --> F;
    F --> G[Search Results];

    subgraph "Indexing (CLI)"
        A
        B
        C
    end

    subgraph "Querying (Runtime)"
        D
        E
        F
        G
    end
```

## Building the Index

Victor comes with a CLI for building the index from a set of files.

```bash
DATA_ROOT=/path/to/your/data node node_modules/@grouchoab/victor/dist/vector-index.js /path/to/your/docs
```

This will create an `index.json` file in `$DATA_ROOT/cache/vdb/`.

## API Reference

### `Victor`

The main class for interacting with the vector database.

#### `constructor()`

Initializes a new Victor instance. It will look for an existing index in `$DATA_ROOT/cache/vdb/index.json`. If not found, it will build a new one using the default paths.

#### `query(q: string): Promise<string[]>`

Performs a semantic search for the given query string `q`. Returns a promise that resolves to an array of file paths, sorted by relevance.

### `buildStandardIndex(victor: Victor)`

Builds the index from the default paths (`$DATA_ROOT/examples` and `$DATA_ROOT/requirements`) and adds it to the provided `victor` instance.

### `index(paths: string[])`

Builds an index from the given array of file or directory paths.