langchain-semantic-cache 0.1.2
A semantic caching extension for LangChain with FAISS vector similarity search
License: MIT
Requires Python: >=3.8
Keywords: langchain, semantic, cache, faiss, vector, similarity, llm

# LangChain Semantic Cache

A high-performance semantic caching extension for LangChain that uses FAISS vector similarity search to cache and retrieve LLM responses based on semantic similarity rather than exact matches.

## Features

- **Multi-layer Caching**: Memory cache, SQLite cache, and semantic vector search
- **FAISS Integration**: Fast similarity search using Facebook's FAISS library
- **Configurable Similarity**: Adjustable similarity thresholds for cache hits
- **Quantization Support**: Optional FAISS quantization for memory efficiency
- **Async Support**: Asynchronous operations for better performance
- **Comprehensive Metrics**: Detailed cache performance monitoring
- **LRU Eviction**: Intelligent cache eviction based on usage patterns

## Installation

```bash
pip install langchain-semantic-cache
```

### Optional Dependencies

For GPU acceleration:
```bash
pip install langchain-semantic-cache[quantization]
```

For development:
```bash
pip install langchain-semantic-cache[dev]
```

## Quick Start

### Basic Usage

```python
from langchain_semantic_cache import SemanticCache, ensure_semantic_cache
from langchain_openai import OpenAI

# Method 1: Initialize cache manually
cache = SemanticCache(
    similarity_threshold=0.8,
    max_cache_size=1000
)

# Method 2: Use global cache setup (recommended)
ensure_semantic_cache()

# Use with any LangChain LLM
llm = OpenAI(temperature=0)
response = llm("What is the capital of France?")
```

### Advanced Configuration

```python
from langchain_semantic_cache import SemanticCache

cache = SemanticCache(
    database_path="./my_cache.db",
    faiss_index_path="./vector_index",
    similarity_threshold=0.85,
    max_cache_size=5000,
    memory_cache_size=200,
    batch_size=20,
    enable_quantization=True
)
```

## Configuration Options

| Parameter | Default | Description |
|-----------|---------|-------------|
| `database_path` | `.langchain.db` | Path to SQLite database |
| `faiss_index_path` | `../semantic_cache_index` | Path to FAISS index |
| `similarity_threshold` | `0.5` | Minimum similarity for cache hits |
| `max_cache_size` | `1000` | Maximum entries in vector store |
| `memory_cache_size` | `100` | Maximum entries in memory cache |
| `batch_size` | `10` | Batch size for operations |
| `enable_quantization` | `False` | Enable FAISS quantization |

## Performance Monitoring

```python
# Get cache metrics
metrics = cache.get_metrics()
print(f"Hit rate: {metrics['hit_rate']:.2%}")
print(f"Total requests: {metrics['total_requests']}")
print(f"Memory hits: {metrics['memory_hits']}")
print(f"Semantic hits: {metrics['semantic_hits']}")
```

## Cache Management

```python
# Clear all caches
cache.clear_cache()

# Get current metrics
metrics = cache.get_metrics()
```

## How It Works

1. **Memory Cache**: First checks in-memory LRU cache for fastest access
2. **SQLite Cache**: Checks persistent SQLite database for exact matches
3. **Semantic Search**: Uses FAISS to find semantically similar queries
4. **Embedding Cache**: Caches embeddings to avoid recomputation
5. **Smart Eviction**: Removes oldest entries when cache limits are reached

## Embedding Model

By default, uses `thenlper/gte-base` for embeddings. You can configure this in the source code or extend the class to use different models.

## Requirements

- Python 3.8+
- LangChain
- FAISS
- HuggingFace Transformers
- NumPy

## Contributing

1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

## License

MIT License - see LICENSE file for details.

## Changelog

### 0.1.0
- Initial release
- Multi-layer caching system
- FAISS integration
- Async support
- Comprehensive metrics

## Support

- GitHub Issues: [Report bugs or request features](https://github.com/Chrisolande/langchain-semantic-cache/issues)
- Documentation: [Full documentation](https://github.com/Chrisolande/langchain-semantic-cache#readme)

## Related Projects

- [LangChain](https://github.com/langchain-ai/langchain) - The framework this extends
- [FAISS](https://github.com/facebookresearch/faiss) - Vector similarity search library
- [Sentence Transformers](https://github.com/UKPLab/sentence-transformers) - Embedding models