Spaces:
Running
Running
Commit
·
c2f7da2
1
Parent(s):
645a051
docs: add P0 blockers documentation for Magentic mode implementation
Browse files- Introduced a new markdown document outlining critical blockers in the Magentic mode implementation.
- Highlighted issues such as hardcoded OpenAI models, dependency source ambiguity, and the lack of a "Free Tier" for users.
- Provided detailed impacts and required fixes for each identified issue to ensure a stable deployment.
This documentation aims to facilitate resolution of critical issues and improve the overall user experience in Magentic mode.
docs/bugs/007_magentic_p0_blockers.md
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# P0 Blockers: Magentic Mode Implementation
|
| 2 |
+
|
| 3 |
+
**Date:** November 26, 2025
|
| 4 |
+
**Status:** CRITICAL
|
| 5 |
+
**Component:** Magentic Orchestration (Phase 5)
|
| 6 |
+
|
| 7 |
+
This document outlines critical blockers identified during the implementation of the Magentic multi-agent mode. These issues must be resolved to ensure a stable and configurable production deployment.
|
| 8 |
+
|
| 9 |
+
## 1. Hardcoded OpenAI Models (High Severity)
|
| 10 |
+
|
| 11 |
+
**Issue:**
|
| 12 |
+
The agent factory functions in `src/agents/magentic_agents.py` have hardcoded model IDs (`gpt-4o` and `gpt-4o-mini`).
|
| 13 |
+
|
| 14 |
+
```python
|
| 15 |
+
# src/agents/magentic_agents.py
|
| 16 |
+
|
| 17 |
+
def create_search_agent(...):
|
| 18 |
+
client = chat_client or OpenAIChatClient(
|
| 19 |
+
model_id="gpt-4o-mini", # <--- HARDCODED
|
| 20 |
+
api_key=settings.openai_api_key,
|
| 21 |
+
)
|
| 22 |
+
|
| 23 |
+
def create_judge_agent(...):
|
| 24 |
+
client = chat_client or OpenAIChatClient(
|
| 25 |
+
model_id="gpt-4o", # <--- HARDCODED
|
| 26 |
+
api_key=settings.openai_api_key,
|
| 27 |
+
)
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
**Impact:**
|
| 31 |
+
1. **Configuration ignored:** The user's `OPENAI_MODEL` setting (from `.env`) is ignored by the agents, only used by the Manager.
|
| 32 |
+
2. **Access Failure:** Users without access to `gpt-4o` (e.g., legacy tiers) cannot run the system, even if they configure `gpt-3.5-turbo` in their env.
|
| 33 |
+
3. **Cost Control:** Users cannot downgrade to cheaper models for development/testing.
|
| 34 |
+
|
| 35 |
+
**Fix Required:**
|
| 36 |
+
Update `src/agents/magentic_agents.py` to use `settings.openai_model` (or a specific agent config) instead of hardcoded strings.
|
| 37 |
+
|
| 38 |
+
---
|
| 39 |
+
|
| 40 |
+
## 2. Dependency Source Ambiguity (High Severity)
|
| 41 |
+
|
| 42 |
+
**Issue:**
|
| 43 |
+
The `pyproject.toml` declares a dependency on `agent-framework-core` without specifying a version or path.
|
| 44 |
+
|
| 45 |
+
```toml
|
| 46 |
+
[project.optional-dependencies]
|
| 47 |
+
magentic = [
|
| 48 |
+
"agent-framework-core",
|
| 49 |
+
]
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
**Impact:**
|
| 53 |
+
1. **PyPI vs. Local Mismatch:** It is unclear if `agent-framework-core` is being pulled from PyPI or the local `reference_repos`. The local reference contains specific `MagenticBuilder` logic that may not be present or identical in a potentially stale PyPI package.
|
| 54 |
+
2. **Deployment Failure:** If the PyPI package is missing or version-mismatched, `pip install .[magentic]` will fail or install a broken version on deployment (e.g., HuggingFace Spaces).
|
| 55 |
+
|
| 56 |
+
**Fix Required:**
|
| 57 |
+
Explicitly define the source of `agent-framework-core`. If relying on the local reference, use a relative path dependency or ensure the correct version is published and pinned.
|
| 58 |
+
*Recommendation:* For this repo, since `reference_repos` is included, we should install from the local path in development, but this is hard with standard `pyproject.toml` (path dependencies don't work well for uploads).
|
| 59 |
+
*Alternative:* Verify exact PyPI version matches `reference_repos` and pin it.
|
| 60 |
+
|
| 61 |
+
---
|
| 62 |
+
|
| 63 |
+
## 3. Missing "Free Tier" for Magentic Mode (Medium Severity)
|
| 64 |
+
|
| 65 |
+
**Issue:**
|
| 66 |
+
Magentic mode is currently hard-locked to OpenAI.
|
| 67 |
+
|
| 68 |
+
```python
|
| 69 |
+
# src/orchestrator_magentic.py
|
| 70 |
+
if not settings.openai_api_key:
|
| 71 |
+
raise ConfigurationError("Magentic mode requires OPENAI_API_KEY...")
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
**Impact:**
|
| 75 |
+
Users relying on the "Free Tier" (HuggingFace Inference) cannot use the Multi-Agent features. This bifurcates the user experience:
|
| 76 |
+
* **Free User:** Simple linear search (Phase 4).
|
| 77 |
+
* **Paid User:** Advanced multi-agent loop (Phase 5).
|
| 78 |
+
|
| 79 |
+
**Mitigation:**
|
| 80 |
+
This is currently "Working as Designed" due to technical limitations of HF models with tool calling, but it should be clearly documented in the UI (which has been done in `app.py`).
|
| 81 |
+
|
| 82 |
+
---
|
| 83 |
+
|
| 84 |
+
## Action Plan
|
| 85 |
+
|
| 86 |
+
1. [IMMEDIATE] Refactor `src/agents/magentic_agents.py` to use `settings.openai_model`.
|
| 87 |
+
2. [IMMEDIATE] Verify `agent-framework-core` installation source.
|
src/agents/magentic_agents.py
CHANGED
|
@@ -22,7 +22,7 @@ def create_search_agent(chat_client: OpenAIChatClient | None = None) -> ChatAgen
|
|
| 22 |
ChatAgent configured for biomedical search
|
| 23 |
"""
|
| 24 |
client = chat_client or OpenAIChatClient(
|
| 25 |
-
model_id=
|
| 26 |
api_key=settings.openai_api_key,
|
| 27 |
)
|
| 28 |
|
|
@@ -60,7 +60,7 @@ def create_judge_agent(chat_client: OpenAIChatClient | None = None) -> ChatAgent
|
|
| 60 |
ChatAgent configured for evidence assessment
|
| 61 |
"""
|
| 62 |
client = chat_client or OpenAIChatClient(
|
| 63 |
-
model_id=
|
| 64 |
api_key=settings.openai_api_key,
|
| 65 |
)
|
| 66 |
|
|
@@ -99,7 +99,7 @@ def create_hypothesis_agent(chat_client: OpenAIChatClient | None = None) -> Chat
|
|
| 99 |
ChatAgent configured for hypothesis generation
|
| 100 |
"""
|
| 101 |
client = chat_client or OpenAIChatClient(
|
| 102 |
-
model_id=
|
| 103 |
api_key=settings.openai_api_key,
|
| 104 |
)
|
| 105 |
|
|
@@ -136,7 +136,7 @@ def create_report_agent(chat_client: OpenAIChatClient | None = None) -> ChatAgen
|
|
| 136 |
ChatAgent configured for report generation
|
| 137 |
"""
|
| 138 |
client = chat_client or OpenAIChatClient(
|
| 139 |
-
model_id=
|
| 140 |
api_key=settings.openai_api_key,
|
| 141 |
)
|
| 142 |
|
|
|
|
| 22 |
ChatAgent configured for biomedical search
|
| 23 |
"""
|
| 24 |
client = chat_client or OpenAIChatClient(
|
| 25 |
+
model_id=settings.openai_model, # Use configured model
|
| 26 |
api_key=settings.openai_api_key,
|
| 27 |
)
|
| 28 |
|
|
|
|
| 60 |
ChatAgent configured for evidence assessment
|
| 61 |
"""
|
| 62 |
client = chat_client or OpenAIChatClient(
|
| 63 |
+
model_id=settings.openai_model,
|
| 64 |
api_key=settings.openai_api_key,
|
| 65 |
)
|
| 66 |
|
|
|
|
| 99 |
ChatAgent configured for hypothesis generation
|
| 100 |
"""
|
| 101 |
client = chat_client or OpenAIChatClient(
|
| 102 |
+
model_id=settings.openai_model,
|
| 103 |
api_key=settings.openai_api_key,
|
| 104 |
)
|
| 105 |
|
|
|
|
| 136 |
ChatAgent configured for report generation
|
| 137 |
"""
|
| 138 |
client = chat_client or OpenAIChatClient(
|
| 139 |
+
model_id=settings.openai_model,
|
| 140 |
api_key=settings.openai_api_key,
|
| 141 |
)
|
| 142 |
|