coret 0.1.1
Surrogate based Concept Retrieval for Large Datasets
License: MIT
Requires Python: <3.13,>=3.9
Keywords: concept-retrieval, interpretability, xai, computer-vision, machine-learning

# Surrogate Concept Retrieval

[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

# surrogate_concept_retrieval
Implementation for the paper Concept Retrieval - What and How?

## Package Status

✅ **Added**:
- Project URLs and Documentation links
- Keywords and classifiers for PyPI
- Populated `__init__.py` for proper imports
- Documentation structure with Sphinx
- Example code
- Improved README with usage examples

🔄 **In Progress**:
- Comprehensive documentation
- Test coverage
- CI/CD setup

## Getting Started

```bash
# Install the package
pip install -e .
```

See `RECOMMENDATIONS.md` for full details on package improvements.

## Overview

This package provides tools for extracting concepts from large datasets using surrogate concept retrieval method.

## Features

- Fast embedding indexing using FAISS
- GPU-accelerated similarity computation
- Automatic concept extraction from embedding spaces
- Flexible concept filtering and refinement
- Support for projection-based concept analysis

## Installation

```bash
# Install from PyPI
pip install coret

# Install with development dependencies
pip install "coret[dev]"
```

## Quick Start

```python
import numpy as np
from coret import ConceptRetrieval

# Load your embeddings (example uses random data)
embeddings = np.random.randn(1000, 768)
embeddings = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True)
embeddings = np.ascontiguousarray(embeddings, dtype=np.float32)

# Initialize concept retrieval
concept_retriever = ConceptRetrieval()

# Fit the model with embeddings
concept_retriever.fit(embeddings=embeddings)

# Select a random query embedding for demonstration
query_index = np.random.randint(0, len(embeddings))
query_embedding = embeddings[query_index]

# Retrieve concepts for the query
concepts = concept_retriever.retrieve(
    query=query_embedding,
    number_of_concepts=5,
    number_of_samples_per_concept=5
)

# Print retrieved concepts
top_k_concepts_indices_s = concepts['top_k_concepts_indices_s']

print(f"Query index: {query_index}")
for i, concept_indices in enumerate(top_k_concepts_indices_s):
  print(f"Concept {i+1}:")
  print(f"  Indices: {concept_indices}")
  print()
```

## Requirements

- Python 3.9+
- CUDA-compatible GPU (recommended for large datasets)
- Dependencies:
  - numpy
  - faiss-gpu (or faiss-cpu)
  - scipy
  - scikit-learn
  - tqdm
  - cupy (for GPU acceleration)

## Documentation

For detailed API documentation and examples, please visit our [documentation site](https://onr.github.io/surrogate_concept_retrieval).


## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

This project is licensed under the MIT License - see the LICENSE file for details.

<!-- ## Citation

If you use this library in your research, please cite:

```bibtex
@article{author2025concept,
  title={Concept Retrieval - What and How?},
  author={Author, A.},
  journal={Journal Name},
  year={2025}
}
``` -->