mlcontester 0.1.0
A unified Python client for ML competition platforms (Kaggle, Zindi, DrivenData, AIcrowd, and more)
License: MIT
Requires Python: >=3.9
Keywords: machine-learning, competitions, kaggle, zindi, drivendata, aicrowd

# MLContester

A unified Python client for ML competition platforms. Connect to Kaggle, Zindi, DrivenData, AIcrowd, EvalAI, CrunchDAO, Codabench, Hugging Face, Solafune, and more through a single, consistent API.

## Features

- 🔐 **Unified Authentication** - Single interface for all platforms
- 📋 **List Competitions** - Browse active competitions across platforms
- 🏆 **Join Competitions** - Participate in challenges programmatically
- 📥 **Download Data** - Fetch competition datasets with one command
- 📤 **Submit Solutions** - Upload predictions directly from your code
- 📊 **Track Progress** - View leaderboards and your submissions
- 💬 **Discussions** - Read and post in competition forums
- 🖥️ **CLI Interface** - Command-line tools for quick operations

## Supported Platforms

| Platform | Status | Features | Notes |
|----------|--------|----------|-------|
| [Kaggle](https://www.kaggle.com) | ✅ Full | All features | Uses official Kaggle API |
| [Zindi](https://zindi.africa) | ✅ Full | All features | African ML competitions |
| [DrivenData](https://www.drivendata.org) | ✅ Full | All features | Social impact competitions |
| [AIcrowd](https://www.aicrowd.com) | ✅ Full | All features | Research challenges |
| [EvalAI](https://eval.ai) | ✅ Full | All features | Open-source evaluation platform |
| [CrunchDAO](https://hub.crunchdao.com) | ⚠️ Partial | List, Download, Leaderboard | Submissions via crunch-cli |
| [Codabench](https://www.codabench.org) | ✅ Full | All features | CodaLab successor, self-hostable |
| [Hugging Face](https://huggingface.co) | ⚠️ Partial | List, Browse, Discussions | Submission varies by competition |
| [Solafune](https://solafune.com) | ✅ Full | All features | Satellite/geospatial ML |

## Installation

```bash
pip install mlcontester

# With Kaggle official API support
pip install mlcontester[kaggle]

# With development dependencies
pip install mlcontester[dev]
```

Or install from source:

```bash
git clone https://github.com/mlcontester/mlcontester.git
cd mlcontester
pip install -e .
```

## Quick Start

### Python API

```python
import asyncio
from pathlib import Path
from mlcontester import MLContester, CompetitionStatus

async def main():
    async with MLContester() as client:
        # Authenticate with platforms
        await client.authenticate("kaggle", username="your_username", api_key="your_key")
        await client.authenticate("zindi", email="you@example.com", password="your_pass")
        
        # List active competitions
        competitions = await client.list_all_competitions(status=CompetitionStatus.ACTIVE)
        for comp in competitions[:5]:
            print(f"[{comp.platform}] {comp.title} - {comp.deadline}")
        
        # Download competition data
        await client.download_data("kaggle", "titanic", Path("./data/titanic"))
        
        # Make a submission
        submission = await client.submit(
            "kaggle",
            "titanic",
            Path("./submission.csv"),
            message="My first submission"
        )
        print(f"Submitted! Score: {submission.public_score}")
        
        # Check leaderboard
        leaderboard = await client.get_leaderboard("kaggle", "titanic")
        for entry in leaderboard[:10]:
            print(f"{entry.rank}. {entry.team_name}: {entry.score}")

asyncio.run(main())
```

### Command Line Interface

```bash
# List supported platforms
mlcontester platforms

# Authenticate (uses environment variables)
mlcontester auth kaggle

# List active competitions
mlcontester competitions kaggle --active

# Get competition details
mlcontester info kaggle titanic

# Download data
mlcontester download kaggle titanic -o ./data

# Make a submission
mlcontester submit kaggle titanic submission.csv -m "My submission"

# View leaderboard
mlcontester leaderboard kaggle titanic -n 20

# List your submissions
mlcontester submissions kaggle titanic
```

## Configuration

### Environment Variables

Set credentials via environment variables:

```bash
# Kaggle
export KAGGLE_USERNAME="your_username"
export KAGGLE_KEY="your_api_key"

# Zindi
export ZINDI_EMAIL="you@example.com"
export ZINDI_PASSWORD="your_password"
# Or use token
export ZINDI_TOKEN="your_token"

# DrivenData
export DRIVENDATA_USERNAME="your_username"
export DRIVENDATA_PASSWORD="your_password"
# Or use token
export DRIVENDATA_TOKEN="your_token"

# AIcrowd
export AICROWD_API_KEY="your_api_key"

# EvalAI
export EVALAI_AUTH_TOKEN="your_auth_token"
# Or for self-hosted instances
export EVALAI_HOST_URL="https://your-evalai-instance.com"

# CrunchDAO
export CRUNCHDAO_API_KEY="your_api_key"

# Codabench
export CODABENCH_TOKEN="your_token"
# Or for self-hosted instances
export CODABENCH_URL="https://your-codabench-instance.com"

# Hugging Face
export HF_TOKEN="hf_your_access_token"
# Or
export HUGGINGFACE_TOKEN="hf_your_access_token"

# Solafune
export SOLAFUNE_TOKEN="your_token"
```

### Kaggle Configuration

You can also use the standard `~/.kaggle/kaggle.json` file:

```json
{
    "username": "your_username",
    "key": "your_api_key"
}
```

## API Reference

### MLContester Client

The main client class that aggregates all platform functionality.

```python
from mlcontester import MLContester

async with MLContester() as client:
    # Your code here
    pass
```

#### Methods

| Method | Description |
|--------|-------------|
| `authenticate(platform, **credentials)` | Authenticate with a platform |
| `authenticate_from_env(platform)` | Authenticate using environment variables |
| `list_competitions(platform, status, search)` | List competitions from a platform |
| `list_all_competitions(status, search)` | List from all authenticated platforms |
| `get_competition(platform, competition_id)` | Get competition details |
| `join_competition(platform, competition_id)` | Join a competition |
| `download_data(platform, competition_id, destination)` | Download competition data |
| `submit(platform, competition_id, file_path, message)` | Submit a solution |
| `list_submissions(platform, competition_id)` | List your submissions |
| `get_leaderboard(platform, competition_id)` | Get leaderboard |
| `list_discussions(platform, competition_id)` | List discussions |
| `post_discussion(platform, competition_id, title, content)` | Post a discussion |

### Data Models

```python
from mlcontester import Competition, Submission, LeaderboardEntry, Discussion

# Competition
comp.id          # Competition identifier
comp.title       # Competition title
comp.platform    # Platform name
comp.url         # Web URL
comp.status      # CompetitionStatus (ACTIVE, ENDED, UPCOMING)
comp.deadline    # Deadline datetime
comp.prize       # Prize amount
comp.is_joined   # Whether user has joined

# Submission
sub.id           # Submission identifier
sub.filename     # Submitted file name
sub.status       # Submission status
sub.public_score # Public leaderboard score
sub.submitted_at # Submission datetime

# LeaderboardEntry
entry.rank       # Position on leaderboard
entry.team_name  # Team or user name
entry.score      # Score value
entry.entries    # Number of submissions

# Discussion
disc.id           # Discussion identifier
disc.title        # Discussion title
disc.author       # Author name
disc.votes        # Vote count
disc.comments_count # Number of comments
```

## Platform-Specific Notes

### Kaggle

- Uses official Kaggle API under the hood
- Some competitions require accepting rules via web interface before joining
- Discussion posting not supported via API (use web interface)

### Zindi

- Full API support for all features
- Supports both email/password and token authentication

### DrivenData

- Uses session-based authentication
- Some features may require web interface interaction

### AIcrowd

- Get your API key from https://www.aicrowd.com/participants/me
- Some challenges use git-based submissions (not yet fully supported)

### EvalAI

- Open-source platform for ML evaluation challenges
- Get your auth token from Settings > Auth Token on the EvalAI dashboard
- Supports self-hosted instances via `host_url` parameter
- Full support for code and remote evaluation challenges

### CrunchDAO

- Quantitative research competition platform
- Get your API key from https://hub.crunchdao.com/settings/api
- Full submissions require the `crunch-cli` tool: `pip install crunch-cli`
- MLContester provides competition listing, data download, and leaderboard access

### Codabench

- Successor to CodaLab, fully open source
- Can be self-hosted - configure with `base_url` parameter
- Get your token from API settings in your profile
- Full support for phase-based benchmarking competitions

### Hugging Face

- ML model hub with competition Spaces and Datasets
- Get your access token from https://huggingface.co/settings/tokens
- Competitions are identified by Space/Dataset paths (e.g., `org/competition-name`)
- Submission methods vary by competition (check individual competition rules)
- Full discussion support on Spaces and Datasets

### Solafune

- Japanese platform specializing in satellite imagery and geospatial ML
- Full API support for all features
- Competitions focus on Earth observation data

## Examples

### Automated Competition Workflow

```python
import asyncio
from pathlib import Path
from mlcontester import MLContester

async def compete():
    async with MLContester() as client:
        # Setup
        await client.authenticate_from_env()
        
        # Find an interesting competition
        competitions = await client.list_competitions(
            "kaggle",
            status="active",
            search="classification"
        )
        
        comp = competitions[0]
        print(f"Working on: {comp.title}")
        
        # Join and download data
        await client.join_competition("kaggle", comp.id)
        await client.download_data("kaggle", comp.id, Path("./data"))
        
        # ... train your model ...
        
        # Submit predictions
        submission = await client.submit(
            "kaggle",
            comp.id,
            Path("./predictions.csv"),
            "Baseline model v1"
        )
        
        print(f"Score: {submission.public_score}")

asyncio.run(compete())
```

### Cross-Platform Competition Search

```python
async def find_competitions():
    async with MLContester() as client:
        # Authenticate with all platforms
        results = await client.authenticate_from_env()
        print(f"Authenticated: {[k for k, v in results.items() if v]}")
        
        # Search across all platforms
        competitions = await client.list_all_competitions(
            status="active",
            search="nlp"
        )
        
        print(f"Found {len(competitions)} NLP competitions:")
        for comp in competitions:
            print(f"  [{comp.platform}] {comp.title}")
            print(f"    Deadline: {comp.deadline}")
            print(f"    Prize: {comp.prize}")
```

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

### Adding a New Platform

1. Create a new file in `src/mlcontester/platforms/`
2. Implement the `BasePlatform` interface
3. Add to the registry in `src/mlcontester/client.py`
4. Add environment variables documentation
5. Write tests

## License

MIT License - see [LICENSE](LICENSE) for details.

## Acknowledgments

- Inspired by the ML competition community
- Thanks to all the platforms for providing APIs
- Built with [httpx](https://www.python-httpx.org/), [pydantic](https://docs.pydantic.dev/), [typer](https://typer.tiangolo.com/), and [rich](https://rich.readthedocs.io/)