llmgine 0.0.1
An application framework for building LLM-powered applications.
Requires Python: <4.0,>=3.11
Keywords: python

# 🌌 **LLMgine**

LLMgine is a _pattern-driven_ framework for building **production-grade, tool-augmented LLM applications** in Python.  
It offers a clean separation between _**engines**_ (conversation logic), _**models/providers**_ (LLM back-ends), _**tools**_ (function calling with **MCP support**), a streaming **message-bus** for commands & events, and opt-in **observability**.  
Think _FastAPI_ for web servers or _Celery_ for tasks—LLMgine plays the same role for complex, chat-oriented AI.

---

## ✨ Feature Highlights
| Area | What you get | Key files |
|------|--------------|-----------|
| **Engines** | Plug-n-play `Engine` subclasses (`SinglePassEngine`, `ToolChatEngine`, …) with session isolation, tool-loop orchestration, and CLI front-ends | `engines/*.py`, `src/llmgine/llm/engine/` |
| **Message Bus** | Async **command bus** (1 handler) + **event bus** (N listeners) + **sessions** for scoped handlers | `src/llmgine/bus/` |
| **Tooling** | Declarative function-to-tool registration, multi-provider JSON-schema parsing (OpenAI, Claude, DeepSeek), async execution pipeline, **MCP integration** for external tool servers | `src/llmgine/llm/tools/` |
| **Providers / Models** | Wrapper classes for OpenAI, OpenRouter, Gemini 2.5 Flash etc. _without locking you in_ | `src/llmgine/llm/providers/`, `src/llmgine/llm/models/` |
| **Unified Interface** | Single API for OpenAI, Anthropic, and Gemini - switch providers by changing model name | `src/llmgine/unified/` |
| **Context Management** | Simple and in-memory chat history managers, event-emitting for retrieval/update | `src/llmgine/llm/context/` |
| **UI** | Rich-powered interactive CLI (`EngineCLI`) with live spinners, confirmation prompts, tool result panes | `src/llmgine/ui/cli/` |
| **Observability** | Console + JSONL file handlers, per-event metadata, easy custom sinks | `src/llmgine/observability/` |
| **Bootstrap** | One-liner `ApplicationBootstrap` that wires logging, bus startup, and observability | `src/llmgine/bootstrap.py` |

---

## 🏗️ High-Level Architecture

```mermaid
flowchart TD
    %% Nodes
    AppBootstrap["ApplicationBootstrap"]
    Bus["MessageBus<br/>(async loop)"]
    Obs["Observability<br/>Handlers"]
    Eng["Engine(s)"]
    TM["ToolManager"]
    Tools["Local Tools"]
    MCP["MCP Servers"]
    Session["BusSession"]
    CLI["CLI / UI"]

    %% Edges
    AppBootstrap -->|starts| Bus

    Bus -->|events| Obs
    Bus -->|commands| Eng
    Bus -->|events| Session

    Eng -- status --> Bus
    Eng -- tool_calls --> TM

    TM -- executes --> Tools
    TM -. "MCP Protocol" .-> MCP
    Tools -- ToolResult --> CLI
    MCP -. "External Tools" .-> CLI

    Session --> CLI
```

*Every component communicates _only_ through the bus, so engines, tools, and UIs remain fully decoupled.*

---

## 🚀 Quick Start

### 1. Install

```bash
git clone https://github.com/your-org/llmgine.git
cd llmgine
python -m venv .venv && source .venv/bin/activate
pip install -e ".[openai]"   # extras: openai, openrouter, mcp, dev, …
export OPENAI_API_KEY="sk-…" # or OPENROUTER_API_KEY / GEMINI_API_KEY
```

### 2. Run the demo CLI

```bash
python -m llmgine.engines.single_pass_engine  # pirate translator
# or
python -m llmgine.engines.tool_chat_engine    # automatic tool loop
```

You’ll get an interactive prompt with live status updates and tool execution logs.

---

## 🧑‍💻 Building Your Own Engine

```python
from llmgine.llm.engine.engine import Engine
from llmgine.messages.commands import Command, CommandResult
from llmgine.bus.bus import MessageBus

class MyCommand(Command):
    prompt: str = ""

class MyEngine(Engine):
    def __init__(self, session_id: str):
        self.session_id = session_id
        self.bus = MessageBus()

    async def handle_command(self, cmd: MyCommand) -> CommandResult:
        await self.bus.publish(Status("thinking", session_id=self.session_id))
        # call LLM or custom logic here …
        answer = f"Echo: {cmd.prompt}"
        await self.bus.publish(Status("finished", session_id=self.session_id))
        return CommandResult(success=True, result=answer)

# Wire into CLI
from llmgine.ui.cli.cli import EngineCLI
chat = EngineCLI(session_id="demo")
chat.register_engine(MyEngine("demo"))
chat.register_engine_command(MyCommand, MyEngine("demo").handle_command)
await chat.main()
```

---

## 🔧 Tool Integration

### Local Tools in 3 Lines

```python
from llmgine.llm.tools.tool import Parameter
from llmgine.engines.tool_chat_engine import ToolChatEngine

def get_weather(city: str):
    """Return current temperature for a city.
    Args:
        city: Name of the city
    """
    return f"{city}: 17 °C"

engine = ToolChatEngine(session_id="demo")
await engine.register_tool(get_weather)               # ← introspection magic ✨
```

The engine now follows the **OpenAI function-calling loop**:

```
User → Engine → LLM (asks to call get_weather) → ToolManager → get_weather()
          ↑                                        ↓
          └───────────    context update   ────────┘ (loops until no tool calls)
```

### MCP (Model Context Protocol) Integration

Connect to external tool servers using the MCP protocol:

```python
from llmgine.llm.tools.tool_manager import ToolManager

# Initialize tool manager with MCP support
tool_manager = ToolManager()

# Register an MCP server (e.g., Notion integration)
await tool_manager.register_mcp_server(
    server_name="notion",
    command="python",
    args=["/path/to/notion_mcp_server.py"]
)

# MCP tools are now available alongside local tools
```

**MCP Features:**
- 🔌 **External Tool Servers**: Connect to any MCP-compatible tool server
- 🔄 **Dynamic Loading**: Tools are discovered and loaded at runtime
- 🎯 **Provider Agnostic**: Works with OpenAI, Anthropic, and Gemini tool formats
- ⚡ **Graceful Degradation**: Falls back silently if MCP dependencies unavailable

**Prerequisites:**
```bash
pip install mcp  # Optional: for MCP integration
```

---

## 📰 Message Bus in Depth

```python
from llmgine.bus.bus import MessageBus
from llmgine.bus.session import BusSession

bus = MessageBus()
await bus.start()

class Ping(Command): pass
class Pong(Event): msg: str = "pong!"

async def ping_handler(cmd: Ping):
    await bus.publish(Pong(session_id=cmd.session_id))
    return CommandResult(success=True)

with bus.create_session() as sess:
    sess.register_command_handler(Ping, ping_handler)
    sess.register_event_handler(Pong, lambda e: print(e.msg))
    await sess.execute_with_session(Ping())      # prints “pong!”
```

*Handlers are **auto-unregistered** when the `BusSession` exits—no leaks.*

---

## 📊 Observability

Add structured logs with zero boilerplate:

```python
from llmgine.bootstrap import ApplicationBootstrap, ApplicationConfig
config = ApplicationConfig(enable_console_handler=True,
                           enable_file_handler=True,
                           log_level="debug")
await ApplicationBootstrap(config).bootstrap()
```

The observability system uses a standalone `ObservabilityManager` that:
- Operates independently of the message bus (no circular dependencies)
- Provides synchronous handlers (see note below)
- Supports console, file, and OpenTelemetry handlers
- Can be extended with custom handlers

*All events flow through the configured handlers to console and/or timestamped `logs/events_*.jsonl` files.*

**⚠️ Performance Note**: The current implementation uses synchronous handlers which can block the event loop in high-throughput scenarios. This is suitable for development and low-volume production use. See the [Observability System documentation](src/llmgine/observability/README.md#performance-considerations) for details and planned improvements.

For OpenTelemetry support:
```bash
pip install llmgine[opentelemetry]
```

---

## 📚 Documentation

Comprehensive documentation is available in the [`docs/`](docs/) directory:

- **[Documentation Index](docs/index.md)** - Start here for navigation
- **[Product Requirements Document](docs/prd.md)** - Original project requirements
- **[Brownfield Enhancement PRD](docs/brownfield-prd/index.md)** - Production-ready enhancements
- **[User Stories](docs/stories/index.md)** - Detailed implementation stories
- **[Engine Development Guide](programs/engines/engine_guide.md)** - How to build custom engines
- **[CLAUDE.md](CLAUDE.md)** - AI assistant instructions for contributors

### Component Documentation
- **[Message Bus](src/llmgine/bus/README.md)** - Event and command bus architecture
- **[Tools System](src/llmgine/llm/tools/README.md)** - Function calling, tool registration, and MCP integration
- **[Observability System](src/llmgine/observability/README.md)** - Standalone observability architecture
- **[Observability CLI](programs/observability-cli/README.md)** - CLI monitoring tool
- **[Observability GUI](programs/observability-gui/README.md)** - React-based monitoring interface

---

## 📁 Repository Layout (abridged)

```
llmgine/
│
├─ engines/            # Turn-key example engines (single-pass, tool chat, …)
└─ src/llmgine/
   ├─ bus/             # Message bus core + sessions
   ├─ llm/
   │   ├─ context/     # Chat history & context events
   │   ├─ engine/      # Engine base + dummy
   │   ├─ models/      # Provider-agnostic model wrappers
   │   ├─ providers/   # OpenAI, OpenRouter, Gemini, Dummy, …
   │   └─ tools/       # ToolManager, parser, register, types, MCP integration
   ├─ observability/   # Console & file handlers, log events
   └─ ui/cli/          # Rich-based CLI components
```

---

## 🏁 Roadmap

- [ ] **Streaming responses** with incremental event dispatch  
- [ ] **WebSocket / FastAPI** front-end (drop-in replacement for CLI)  
- [ ] **Persistent vector memory** layer behind `ContextManager`  
- [ ] **Plugin system** for third-party Observability handlers  
- [ ] **More providers**: Anthropic, Vertex AI, etc.

---

## 🤝 Contributing

1. Fork & create a feature branch  
2. Ensure `pre-commit` passes (`ruff`, `black`, `isort`, `pytest`)  
3. Open a PR with context + screenshots/GIFs if UI-related  

---

## 📄 License

LLMgine is distributed under the **MIT License**—see [`LICENSE`](LICENSE) for details.

---

> _“Build architecturally sound LLM apps, not spaghetti code.  
> Welcome to the engine room.”_