npm package: @sf-ai/rag-indexes
Description: RAG vector index helpers for HNSW and IVFFlat
Author: Dan Lynch
Latest version: 0.2.1

<p align="center" width="100%">
  <img height="250" src="https://github.com/city-of-san-francisco/.github/blob/main/assets/logo.svg" />
</p>

<p align="center" width="100%">
   <a href="https://github.com/city-of-san-francisco/license"><img height="20" src="https://img.shields.io/badge/license-SF-blue.svg"/></a>
   <a href="https://github.com/city-of-san-francisco/sf-rag-utils/actions/workflows/ci.yml"><img height="20" src="https://github.com/city-of-san-francisco/sf-rag-utils/actions/workflows/ci.yml/badge.svg" /></a>
   <a href="https://www.npmjs.com/package/@sf-ai/rag-indexes"><img height="20" src="https://img.shields.io/github/package-json/v/city-of-san-francisco/sf-rag-utils?filename=packages%2Frag-indexes%2Fpackage.json"/></a>
</p>

<p align="center" width="100%">
  <a href="https://github.com/city-of-san-francisco/sf-rag-utils">A pgpm workspace for building agentic, database-driven AI applications for the City of San Francisco.</a>
</p>

# @sf-ai/rag-indexes

RAG vector index helpers for HNSW and IVFFlat.

## Overview

This package provides SQL functions for creating and managing pgvector indexes on RAG embeddings.

## Functions

### `rag.create_hnsw_index(collection_id, model_name, m, ef_construction, distance_op)`

Create an HNSW index for fast approximate nearest neighbor search.

```sql
-- Create HNSW index with defaults (m=16, ef_construction=64, cosine distance)
SELECT rag.create_hnsw_index('collection-uuid');

-- Create with custom parameters
SELECT rag.create_hnsw_index(
  'collection-uuid',
  'text-embedding-3-small',
  32,    -- m: connections per layer
  128,   -- ef_construction: index build quality
  'l2'   -- distance: cosine, l2, or ip
);
```

**Parameters:**
- `m`: Max connections per layer (default: 16). Higher = better recall, more memory
- `ef_construction`: Build-time quality (default: 64). Higher = better recall, slower build
- `distance_op`: Distance function - `cosine`, `l2`/`euclidean`, or `ip`/`inner_product`

### `rag.create_ivfflat_index(collection_id, model_name, lists, distance_op)`

Create an IVFFlat index for fast indexing with good recall.

```sql
-- Create IVFFlat index with defaults (lists=100, cosine distance)
SELECT rag.create_ivfflat_index('collection-uuid');

-- Create with custom parameters
SELECT rag.create_ivfflat_index(
  'collection-uuid',
  'text-embedding-3-small',
  200,     -- lists: number of clusters
  'cosine' -- distance: cosine, l2, or ip
);
```

**Note:** IVFFlat requires data to be present before creating the index.

### `rag.list_vector_indexes(collection_id)`

List all vector indexes, optionally filtered by collection.

```sql
SELECT * FROM rag.list_vector_indexes();
SELECT * FROM rag.list_vector_indexes('collection-uuid');
```

### `rag.drop_vector_index(index_name)`

Drop a vector index by name.

```sql
SELECT rag.drop_vector_index('idx_rag_embedding_hnsw_...');
```

## When to Use Which Index

| Index | Best For | Trade-offs |
|-------|----------|------------|
| HNSW | High recall, easy management | Slower inserts, more memory |
| IVFFlat | Fast indexing, large datasets | Requires pre-populated data |

## Dependencies

- `@sf-ai/rag-core`

## License

[SF License](https://github.com/city-of-san-francisco/license)