taxonova 25.11.0
Deep Learning-based Taxonomic Classification for Amplicon Sequences
License: MIT
Requires Python: >=3.8
Keywords: bioinformatics, taxonomy, classification, deep-learning, amplicon, 16S, rRNA, microbiome, metagenomics

# Taxonova

[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![PyPI version](https://img.shields.io/pypi/v/taxonova.svg)](https://pypi.org/project/taxonova/)

A powerful deep learning system for accurate hierarchical taxonomic classification of bacterial amplicon sequences. Taxonova uses state-of-the-art hierarchical prototypical classification to classify 16S rRNA sequences (250-1500 bp) across multiple taxonomic levels with exceptional species-level accuracy.

## Table of Contents
- [Key Features](#key-features)
- [Prerequisites](#prerequisites)
- [Installation](#installation)
  - [Quick Install from PyPI](#quick-install-from-pypi)
  - [Development Installation](#development-installation)
- [Downloading Pre-trained Models](#downloading-pre-trained-models)
- [Usage Guide](#usage-guide)
  - [Classification Mode](#classification-mode)
  - [Training Mode](#training-mode)
- [Input Data Format](#input-data-format)
- [Output Files](#output-files)
- [Advanced Configuration](#advanced-configuration)
- [Hardware Requirements](#hardware-requirements)
- [Publishing to PyPI](#publishing-to-pypi)
- [License](#license)

## Key Features

- **Hierarchical Prototypical Classification**: Advanced few-shot learning approach optimized for species-level accuracy
- **Pre-trained Models**: Ready-to-use models for different 16S rRNA V regions (V1-V9, V3-V4, V4, full rRNA)
- **Smart Full Data Training**: Two-stage training process for optimal performance on 100% of your data
- **High Species Accuracy**: Specifically optimized to overcome data imbalance issues in taxonomic datasets
- **Hierarchical Consistency**: Enforces taxonomic relationships across all classification levels
- **GPU Acceleration**: CUDA support for fast training and prediction
- **Easy Installation**: Available via pip with automatic model downloading

## Prerequisites

- Python 3.8 or higher
- 2GB RAM (minimum, 4GB+ recommended)
- CUDA-compatible GPU (optional but recommended for training)

## Installation

### Quick Install from PyPI

Install Taxonova directly from PyPI:

```bash
pip install taxonova
```

After installation, download pre-trained models:

```bash
taxonova-setup
```

Or download models manually:

```bash
taxonova download-models --interactive
```

That's it! You're ready to classify sequences.

### Development Installation

For development or to use the latest version from source:

```bash
# Clone the repository
git clone https://github.com/YOUR_USERNAME/taxonova.git
cd taxonova

# Create a conda environment (recommended)
conda create -n taxonova python=3.8
conda activate taxonova

# Install in development mode
pip install -e .
```

## Downloading Pre-trained Models

Taxonova provides pre-trained models for different 16S rRNA variable regions. Models are stored in `~/.taxonova/models/` by default.

### Available Models

- **gg2_v1v9**: GreenGenes2 V1-V9 region model
- **gg2_v3v4**: GreenGenes2 V3-V4 region model
- **gg2_v4**: GreenGenes2 V4 region model
- **gg2_rrn**: GreenGenes2 full rRNA gene model
- **gtdb_rrn**: GTDB full rRNA gene model

### Download Methods

**Interactive selection** (recommended for first-time users):
```bash
taxonova download-models --interactive
```

**Download specific models**:
```bash
taxonova download-models --models gg2_v3v4 gg2_v4
```

**Download all models**:
```bash
taxonova download-models --models all
```

**List available and downloaded models**:
```bash
taxonova download-models --list
```

**Custom models directory**:
```bash
# Set environment variable
export TAXONOVA_MODELS_DIR=/path/to/custom/models

# Or use command option
taxonova download-models --models-dir /path/to/custom/models --models gg2_v3v4
```

## Usage Guide

Taxonova supports two main operation modes: classification and training.

### Classification Mode

Use a pre-trained model to classify unknown sequences.

#### Basic Classification:
```bash
taxonova classification \
    --input sequences.fasta \
    --model_input gg2_v3v4_model.pt \
    --output predictions.tsv \
    --verbose
```

**Note**: If the model is in `~/.taxonova/models/`, you can use just the filename. Taxonova will automatically find it.

#### Classification Parameters:
- `--input`: Path to input FASTA file with sequences to classify (required)
- `--model_input`: Path to trained model file or just the model filename (required)
- `--output`: Path to save classification results (required)
- `--confidence_threshold`: Default confidence threshold (default: 0.5)
- `--domain_threshold`, `--phylum_threshold`, etc.: Level-specific thresholds
- `--enforce_taxonomy`: Enforce taxonomic consistency (default: True)
- `--verbose`: Show detailed progress

### Training Mode

Train a new taxonomic classification model using your reference database.

#### Basic Training:
```bash
taxonova training \
    --train_db database/your_database.fasta \
    --model_output models/your_model.pt \
    --batch_size 32 \
    --epochs 30 \
    --learning_rate 0.01 \
    --kmer-length 8 \
    --verbose
```

#### Smart Full Data Training (Recommended):
For optimal performance, use the smart full data mode which automatically finds the best number of epochs:

```bash
taxonova training \
    --train_db database/your_database.fasta \
    --model_output models/your_model.pt \
    --batch_size 32 \
    --learning_rate 0.01 \
    --kmer-length 8 \
    --smart_full_data \
    --verbose
```

**Optimize for Validation Loss:**
To optimize for validation loss instead of species F1 during training:

```bash
taxonova training \
    --train_db database/your_database.fasta \
    --model_output models/your_model.pt \
    --batch_size 32 \
    --learning_rate 0.01 \
    --kmer-length 8 \
    --smart_full_data \
    --optimize_metric val_loss \
    --verbose
```

**Smart Full Data Process:**
1. **Scouting Phase**: Uses 20% validation split to find optimal epochs with early stopping
2. **Full Training Phase**: Trains on 100% of data using the optimal number of epochs

#### Training Parameters:
- `--train_db`: Path to training database FASTA file (required)
- `--model_output`: Path to save the trained model (required)
- `--batch_size`: Training batch size (default: 32)
- `--epochs`: Number of training epochs (default: 30)
- `--learning_rate`: Learning rate (default: 5e-5)
- `--kmer-length`: K-mer length for feature extraction (default: 8)
- `--smart_full_data`: Enable smart two-stage training
- `--scout_epochs`: Maximum epochs for scouting phase (default: 100)
- `--scout_val_split`: Validation split for scouting (default: 0.2)
- `--optimize_metric`: Metric to optimize during training - 'val_loss', 'species_f1', or 'overall_f1' (default: species_f1)
- `--verbose`: Show detailed progress

### Classification Mode

Use a trained model to classify unknown sequences.

```bash
taxonova classification \
    --input input/sequences.fasta \
    --model_input models/your_model.pt \
    --output output/predictions.tsv \
    --verbose
```

#### Classification Parameters:
- `--input`: Path to input FASTA file with sequences to classify (required)
- `--model_input`: Path to trained model file (required)
- `--output`: Path to save classification results (required)
- `--confidence_threshold`: Default confidence threshold (default: 0.5)
- `--domain_threshold`, `--phylum_threshold`, etc.: Level-specific thresholds
- `--enforce_taxonomy`: Enforce taxonomic consistency (default: True)
- `--verbose`: Show detailed progress

## Input Data Format

### Training Database Format
FASTA file with taxonomic information in sequence headers:
```
>sequence_id|d:domain;p:phylum;c:class;o:order;f:family;g:genus;s:species
ATCGATCGATCG...
```

### Query Sequences Format
Standard FASTA format:
```
>sequence_id
ATCGATCGATCG...
```

## Output Files

### Training Output
- `model.pt`: Unified model file containing model weights, taxonomy information, and hierarchy relationships
- Log files in `output/` directory

### Classification Output
TSV file with columns:
- `sequence_id`: Input sequence identifier
- `domain`, `phylum`, `class`, `order`, `family`, `genus`, `species`: Predicted taxa
- `domain_confidence`, `phylum_confidence`, etc.: Confidence scores for each level

## Advanced Configuration

### Taxonomic Level Weights
Customize the importance of different taxonomic levels during training:

```bash
taxonova training \
    --train_db database.fasta \
    --domain_weight 0.2 \
    --phylum_weight 0.3 \
    --class_weight 0.4 \
    --order_weight 0.6 \
    --family_weight 0.8 \
    --genus_weight 1.0 \
    --species_weight 1.5
```

### K-mer Configuration
Use multiple k-mer lengths for feature extraction:

```bash
taxonova training \
    --train_db database.fasta \
    --kmer-lengths "6,7,8"
```

## Hardware Requirements

- **Minimum**: 2GB RAM, CPU-only
- **Recommended**: 4GB+ RAM, CUDA-compatible GPU
- **Large datasets**: 16GB+ RAM, high-end GPU

Training time scales with:
- Dataset size
- Number of taxonomic classes
- Hardware specifications
- Chosen epochs/parameters

## Publishing to PyPI

To publish Taxonova to PyPI (for maintainers):

### 1. Prepare Your Models

First, upload your trained models to a hosting service (e.g., GitHub Releases):

```bash
# Create a new release on GitHub
# Upload the model files:
#   - gg2_v1v9_model.pt
#   - gg2_v3v4_model.pt
#   - gg2_v4_model.pt
#   - gg2_rrn_model.pt
#   - gtdb_rrn_model.pt
```

Update the URLs in `taxonova/utils/model_downloader.py` with the actual download URLs.

### 2. Update Package Metadata

Edit the following files before publishing:

- `setup.py`: Add your email and GitHub URL
- `pyproject.toml`: Add your email and GitHub URL
- Update version number in both files if needed

### 3. Build and Upload to PyPI

```bash
# Install build tools
pip install build twine

# Clean previous builds
rm -rf dist/ build/ *.egg-info

# Build the package
python -m build

# Upload to TestPyPI (optional, for testing)
python -m twine upload --repository testpypi dist/*

# Test installation from TestPyPI
pip install --index-url https://test.pypi.org/simple/ taxonova

# Upload to PyPI (production)
python -m twine upload dist/*
```

### 4. Create GitHub Release

Tag the release and create release notes:

```bash
git tag v25.11.0
git push origin v25.11.0
```

## Environment Variables

Taxonova supports the following environment variables:

- `TAXONOVA_MODELS_DIR`: Custom directory for storing models (default: `~/.taxonova/models/`)
- `TAXONOVA_CONFIG_DIR`: Custom configuration directory (default: `~/.taxonova/`)

Example:
```bash
export TAXONOVA_MODELS_DIR=/data/taxonova/models
taxonova classification --input sequences.fasta --model_input gg2_v3v4_model.pt --output results.tsv
```

## Troubleshooting

### Model Not Found

If you get a "Model not found" error:

```bash
# Check downloaded models
taxonova download-models --list

# Download the required model
taxonova download-models --models gg2_v3v4

# Or use full path to model
taxonova classification --model_input /full/path/to/model.pt ...
```

### Memory Issues

For large datasets, reduce batch size:

```bash
taxonova training --batch_size 16 ...
```

### CUDA Out of Memory

Switch to CPU or reduce batch size:

```bash
# Force CPU usage
CUDA_VISIBLE_DEVICES="" taxonova classification ...
```

## Citation

If you use Taxonova in your research, please cite:

```
[Add citation information here]
```

## License

This project is licensed under the MIT License - see the LICENSE file for details.