semvec/pypi-packages huggingface dataset:
glueco-plugin-llm 0.1.0
LLM plugin for Glueco Gateway SDK - Typed chat completions for OpenAI, Groq, Gemini
License: MIT
Requires Python: >=3.10
Keywords: ai, chat, completions, gateway, gemini, groq, llm, openai

# Glueco LLM Plugin

[![PyPI version](https://badge.fury.io/py/glueco-plugin-llm.svg)](https://pypi.org/project/glueco-plugin-llm/)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)

Typed LLM chat completions for the [Glueco Gateway SDK](https://pypi.org/project/glueco-sdk/).

## Installation

```bash
pip install glueco-plugin-llm
```

## Quick Start

```python
from glueco_sdk import GatewayClient, FileKeyStorage, FileConfigStorage
from glueco_plugin_llm import llm_client

# Setup gateway client
client = GatewayClient(
    key_storage=FileKeyStorage(".gateway/keys.json"),
    config_storage=FileConfigStorage(".gateway/config.json"),
)

# Get transport and create LLM client
transport = client.get_transport()
llm = llm_client(transport)

# Make chat completion
response = llm.chat_completions(
    provider="groq",
    model="llama-3.1-8b-instant",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello!"},
    ],
    temperature=0.7,
)

print(response.content)
```

## Streaming

```python
for chunk in llm.chat_completions_stream(
    provider="groq",
    model="llama-3.1-8b-instant",
    messages=[{"role": "user", "content": "Tell me a story"}],
):
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
```

## Supported Providers

| Provider | Resource ID | Models |
|----------|-------------|--------|
| OpenAI | `llm:openai` | gpt-4o, gpt-4o-mini, etc. |
| Groq | `llm:groq` | llama-3.1-8b-instant, llama-3.1-70b, etc. |
| Gemini | `llm:gemini` | gemini-1.5-flash, gemini-1.5-pro, etc. |

## Types

```python
from glueco_plugin_llm import (
    ChatMessage,
    ChatCompletion,
    ChatCompletionChoice,
    ChatCompletionUsage,
)

# Create typed messages
messages = [
    ChatMessage(role="system", content="You are helpful."),
    ChatMessage(role="user", content="Hello!"),
]

# Response is fully typed
response: ChatCompletion = llm.chat_completions(...)
print(response.id)
print(response.model)
print(response.choices[0].message.content)
print(response.usage.total_tokens)
```

## API Reference

### llm_client(transport)

Factory function to create an LLM client.

### LLMClient

```python
llm.chat_completions(
    provider: str,          # "openai", "groq", "gemini"
    model: str,             # Model name
    messages: list,         # Conversation messages
    temperature: float,     # Optional (0.0-2.0)
    max_tokens: int,        # Optional
    response_format: dict,  # Optional
    **kwargs,               # Provider-specific params
) -> ChatCompletion

llm.chat_completions_stream(...) -> Iterator[ChatCompletionChunk]
```

## License

MIT