semvec/pypi-packages huggingface dataset:
momahub 0.1.0
Distributed AI inference hub — Mixture of Models on Ollama
License: Apache-2.0
Requires Python: >=3.10
Keywords: distributed-inference, gpu-sharing, llm, moma, ollama

# MoMa Hub

**Distributed AI inference on consumer GPUs — Mixture of Models on Ollama**

```
pip install momahub
```

---

## What is MoMa Hub?

MoMa Hub is the infrastructure layer for the **MoMa (Mixture of Models on Ollama)** vision:
a federated network where anyone with a gaming GPU can contribute inference capacity to the
global AI commons — and route tasks to the right model on the right GPU automatically.

| Inspiration | What they shared | MoMa Hub equivalent |
|-------------|-----------------|---------------------|
| SETI@home   | Idle CPU cycles  | Idle GPU cycles |
| Airbnb      | Spare bedrooms   | Spare VRAM |
| Docker Hub  | Container images | Ollama runtime configs |
| GitHub      | Source code      | SPL prompt scripts |

A **GTX 1080 Ti** (11 GB VRAM, ~$150 used) runs any 7B model in real time.
There are millions sitting idle in gaming PCs. MoMa Hub organises them.

---

## Quick Start

```bash
# 1. Install
pip install momahub

# 2. Start the hub server
momahub serve --port 8765

# 3. Register your Ollama node (in another terminal)
momahub register --node-id home-gpu-0 \
                 --url http://localhost:11434 \
                 --gpu "GTX 1080 Ti" \
                 --vram 11 \
                 --models qwen2.5:7b --models mistral:7b

# 4. Check nodes
momahub nodes

# 5. Run inference through the hub
momahub infer --model qwen2.5:7b --prompt "Explain attention mechanisms"
```

### Python SDK

```python
import asyncio
from momahub import MoMaHub, NodeInfo, InferenceRequest

hub = MoMaHub()
hub.register(NodeInfo(
    node_id="home-gpu-0",
    url="http://localhost:11434",
    gpu_model="GTX 1080 Ti",
    models=["qwen2.5:7b"],
))

resp = asyncio.run(hub.infer(InferenceRequest(
    model="qwen2.5:7b",
    prompt="Hello from MoMa Hub!",
)))
print(resp.content)
```

---

## Architecture

```
Consumer GPUs (GTX 1080 Ti × N)
  │  Ollama serve (one per GPU)
  │
  ▼
MoMa Hub  ──  FastAPI registry + round-robin router
  │
  ▼
Client CLI / Python SDK / SPL scripts
```

See [docs/DESIGN.md](docs/DESIGN.md) for the full roadmap and hardware reference.

---

## Roadmap

| Version | Milestone |
|---------|-----------|
| **v0.1** (now) | Local MVP: register nodes, round-robin routing, CLI + SDK |
| v0.2 | Persistent registry, capability-aware routing, heartbeat daemon |
| v0.3 | LAN mDNS discovery, SPL integration (`USING HUB momahub://...`) |
| v0.4 | Internet federation, auth, public hub registry |

---

## Contributing

MoMa Hub is planned as open-source (Apache 2.0). Once the MVP is proven on 4× GTX 1080 Ti
at home, the repo will go public. Follow the GitHub for updates.

```
https://github.com/digital-duck/momahub
```

---

## License

Apache 2.0