coeai 4.1.0
Professional Python client for COE AI LLM inference API with text and vision support
License: MIT
Requires Python: >=3.7
Keywords: llm inference coeai ollama ai-client multimodal vision deepseek llama

# coeai - Professional Python Client for COE AI LLM API

[![PyPI version](https://img.shields.io/pypi/v/coeai.svg)](https://pypi.org/project/coeai/)
[![Python Support](https://img.shields.io/pypi/pyversions/coeai.svg)](https://pypi.org/project/coeai/)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

**Professional Python client for interacting with high-capacity multimodal Large Language Models hosted on the COE AI GPU cluster.**

> **⚠️ Network Requirement**: This API is **only accessible from UPES's internal network (UPESNET)**. Ensure you are connected to UPES Wi-Fi to use this package.

The `coeai` package provides seamless LLM inference over the UPES local network, supporting both text-to-text and image-to-text operations with advanced streaming capabilities.

---

## 🚀 Quick Start

```bash
pip install coeai
```

```python
from coeai import LLMinfer

# Initialize client (requires UPESNET connectivity)
llm = LLMinfer(api_key="your-api-key")

# Generate text
response = llm.generate(
    model="tinyllama:latest",
    prompt="Explain quantum computing in simple terms",
    max_tokens=256
)
print(response)
```

---

## ✨ Features

| Feature | Description |
|---------|-------------|
| **Text Generation** | Support for all available LLM models |
| **Vision Models** | Image-to-text with multimodal models |
| **Streaming** | Real-time response streaming |
| **Error Handling** | Comprehensive error messages with actionable guidance |
| **File Management** | Automatic cleanup of file handles |
| **Logging** | Optional debug logging for troubleshooting |
| **Model Discovery** | Programmatic model listing |

---

## 📋 Available Models

Get the latest list programmatically:
```python
models = llm.list_models()
print(f"Available models: {', '.join(models)}")
```

**Current models** (as of February 2026):

### Text Models
- `tinyllama:latest` - Compact model for basic tasks
- `tinyllama:1.1b` - Small efficient model
- `deepseek-r1:70b` - Advanced reasoning model
- `gpt-oss:120b` - Large general-purpose model
- `llama4:16x17b` - High-quality multimodal model
- `llama4:128x17b` - Largest available model

### Vision Models (Image + Text)
- `llama3.2-vision:11b` - Vision-capable model
- `llama4:16x17b` - Recommended for image analysis

### Embedding Model
- `bge-m3:567m` - Text embeddings model

---

## 📖 Usage Examples

### 1. Basic Text Generation

```python
from coeai import LLMinfer

llm = LLMinfer(api_key="your-api-key")

response = llm.generate(
    model="tinyllama:latest",
    prompt="Write a haiku about programming",
    max_tokens=100,
    temperature=0.8
)
print(response)
```

### 2. Streaming Response

```python
response = llm.generate(
    model="deepseek-r1:70b",
    prompt="Explain the theory of relativity",
    max_tokens=512,
    stream=True,
    print_stream=True  # Print as it generates
)
```

### 3. Single Image Analysis

```python
response = llm.generate(
    model="llama3.2-vision:11b",
    inference_type="image-to-text",
    files=["/path/to/image.jpg"],
    prompt="Describe this image in detail",
    max_tokens=512
)
print(response)
```

### 4. Multiple Image Comparison

```python
response = llm.generate(
    model="llama4:16x17b",
    inference_type="image-to-text",
    files=["/path/to/image1.jpg", "/path/to/image2.jpg"],
    prompt="Compare these images and identify differences",
    max_tokens=1024,
    temperature=0.7
)
```

### 5. Custom Conversation

```python
messages = [
    {"role": "system", "content": [{"type": "text", "text": "You are a helpful coding assistant."}]},
    {"role": "user", "content": [{"type": "text", "text": "Explain Python decorators"}]}
]

response = llm.generate(
    model="gpt-oss:120b",
    messages=messages,
    max_tokens=512
)
```

### 6. Advanced Parameters

```python
response = llm.generate(
    model="deepseek-r1:70b",
    prompt="Solve:  What is 15! (factorial)?",
    max_tokens=400,
    context_window=4096, # Increase context for complex tasks
    temperature=0.1,   # Lower = more deterministic
    top_p=0.9          # Nucleus sampling
)
```

---

## 🔧 API Reference

### LLMinfer Class

#### Initialization

```python
LLMinfer(api_key: str, host: str = "http://10.9.6.165:8000")
```

| Parameter | Type | Description | Default |
|-----------|------|-------------|---------|
| `api_key` | str | Your COE AI API key (**required**) | - |
| `host` | str | API server URL | `http://10.9.6.165:8000` |

#### Methods

##### `generate()`

```python
generate(
    model: str,
    inference_type: str = "text-to-text",
    prompt: Optional[str] = None,
    messages: Optional[List[Dict]] = None,
    files: Optional[List[str]] = None,
    max_tokens: int = 512,
    temperature: float = 0.7,
    top_p: float = 1.0,
    context_window: int = 2048,
    stream: bool = False,
    print_stream: bool = True
) -> Dict
```

| Parameter | Type | Description |
|-----------|------|-------------|
| `model` | str | Model name (required) |
| `inference_type` | str | `"text-to-text"` or `"image-to-text"` |
| `prompt` | str | Text prompt (optional if `messages` provided) |
| `messages` | list | Custom conversation history |
| `files` | list | Image file paths for vision models |
| `max_tokens` | int | Maximum tokens to generate |
| `temperature` | float | Sampling temperature (0.0–2.0) |
| `top_p` | float | Nucleus sampling (0.0–1.0) |
| `context_window` | int | Context size (num_ctx), default 2048 |
| `stream` | bool | Enable streaming response |
| `print_stream` | bool | Print stream to console |

**Returns**: Dictionary with API response

##### `list_models()`

```python
list_models() -> List[str]
```

Returns list of all available model names.

---

## 🔑 Getting an API Key

### Option 1: Web Dashboard (Recommended)
1. **Connect to UPESNET** (UPES Wi-Fi)
2. Visit **https://coeai.ddn.upes.ac.in**
3. Sign in with your UPES credentials
4. Generate an API key from the dashboard

### Option 2: Email Request
Send an email to `hpc-access@ddn.upes.ac.in` from your UPES account:

```
Subject: API Key Request for COE AI LLM Access

Dear COE AI Team,

I am requesting access to the LLM API for my project work.

Project Details:
- Project Name: <Your Project>
- Description: <Brief description>
- Expected Usage: <How you'll use the API>

Name: <Your Name>
Email: <Your UPES Email>
Department: <Your Department>

Thank you!
```

Allow **2-3 business days** for processing.

---

## ⚠️ Error Handling

The client provides detailed error messages:

```python
from coeai import LLMinfer, AuthenticationError, ModelNotFoundError, InferenceError

llm = LLMinfer(api_key="your-key")

try:
    response = llm.generate(
        model="tinyllama:latest",
        prompt="Hello world"
    )
except AuthenticationError as e:
    print(f"Auth failed: {e}")
except ModelNotFoundError as e:
    print(f"Model not found: {e}")
except InferenceError as e:
    print(f"Inference failed: {e}")
except FileNotFoundError as e:
    print(f"Image file missing: {e}")
```

### Common Errors

| Error | Cause | Solution |
|-------|-------|----------|
| `AuthenticationError` | Invalid API key | Check key or get new one |
| `ModelNotFoundError` | Model doesn't exist | Use `llm.list_models()` to see available models |
| `InferenceError` (429) | Rate limit exceeded | Wait and retry |
| `FileNotFoundError` | Image path wrong | Verify file exists |
| `ConnectionError` | Can't reach server | **Verify you're on UPESNET**, check server status |

---

## 🔍 Troubleshooting

### Enable Debug Logging

```python
import logging
logging.basicConfig(level=logging.DEBUG)

from coeai import LLMinfer
llm = LLMinfer(api_key="your-key")
```

### Check Package Version

```python
import coeai
print(f"coeai version: {coeai.__version__}")
```

### Test Connection

```python
llm = LLMinfer(api_key="your-key")
try:
    models = llm.list_models()
    print(f"✅ Connected! Found {len(models)} models")
except Exception as e:
    print(f"❌ Connection failed: {e}")
```

---

## 📊 Performance Tips

1. **Model Selection**
   - Use `tinyllama` for quick responses
   - Use `deepseek-r1:70b` for reasoning tasks
   - Use `llama4:16x17b` or `llama3.2-vision` for image analysis

2. **Temperature Settings**
   - **0.1-0.3**: Factual/technical content
   - **0.7-0.9**: Balanced creativity
   - **1.0-2.0**: Maximum creativity

3. **Token Limits**
   - Set `max_tokens` appropriately to balance quality and speed
   - Typical: 100-256 for summaries, 512-1024 for detailed responses

4. **Streaming**
   - Enable `stream=True` for long responses to see progress

---

## 🔄 Migration from v2.x

### Breaking Changes in v4.0.0


> **⚠️ Important**: Requires UPESNET (UPES internal network) connectivity

#### Update Your Code:

**Before (v2.x/v3.x)**:
```python
llm = LLMinfer(api_key="key", host="http://10.9.6.165:8001")  # Wrong port
```

**After (v4.0.0)**:
```python
# Correct - uses port 8000 (default)
llm = LLMinfer(api_key="key")

# Or explicitly specify
llm = LLMinfer(api_key="key", host="http://10.9.6.165:8000")
```

### New Features in v4.0

- ✅ Automatic file handle cleanup
- ✅ Comprehensive error handling with custom exceptions
- ✅ `list_models()` method for model discovery
- ✅ Debug logging support
- ✅ Relaxed vision model restrictions
- ✅ Version export (`coeai.__version__`)

---

## 📜 Changelog

### v4.0.0 (Current)
- **BREAKING**: Fixed default port from 8001 to 8000
- **REQUIREMENT**: Only accessible from UPES internal network (UPESNET)
- **API Keys**: Get your key from https://coeai.ddn.upes.ac.in
- Added `list_models()` method
- Improved error handling with custom exceptions
- Fixed file handle leaks
- Added logging support
- Relaxed vision model restrictions
- Added `__version__` export
- Updated documentation with current models

### v2.3.0
- Production release with text-to-text and image-to-text support
- Streaming capabilities
- Multiple image processing

---

## 📄 License

Released under the **MIT License**. See [LICENSE](LICENSE) for details.

---

## 👥 Authors

**Konal Puri** & **Sawai Pratap Khatri**  
Centre of Excellence: AI (COE AI), HPC Project, UPES

- **PyPI**: https://pypi.org/project/coeai  
- **GitHub**: https://github.com/pkonal23/COE-AI-HPC-Project  
- **API Server**: http://10.9.6.165:8000 (UPESNET only)
- **Get API Key**: https://coeai.ddn.upes.ac.in

---

## 🤝 Contributing

Contributions are welcome! Please feel free to submit issues or pull requests on GitHub.

---

## 📞 Support

For issues or questions:
- Open an issue on [GitHub](https://github.com/pkonal23/COE-AI-HPC-Project/issues)
- Contact: `hpc-access@ddn.upes.ac.in`

---

**Made with ❤️ at UPES Centre of Excellence: AI**