VibecoderMcSwaggins commited on
Commit
c2f7da2
·
1 Parent(s): 645a051

docs: add P0 blockers documentation for Magentic mode implementation

Browse files

- Introduced a new markdown document outlining critical blockers in the Magentic mode implementation.
- Highlighted issues such as hardcoded OpenAI models, dependency source ambiguity, and the lack of a "Free Tier" for users.
- Provided detailed impacts and required fixes for each identified issue to ensure a stable deployment.

This documentation aims to facilitate resolution of critical issues and improve the overall user experience in Magentic mode.

docs/bugs/007_magentic_p0_blockers.md ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # P0 Blockers: Magentic Mode Implementation
2
+
3
+ **Date:** November 26, 2025
4
+ **Status:** CRITICAL
5
+ **Component:** Magentic Orchestration (Phase 5)
6
+
7
+ This document outlines critical blockers identified during the implementation of the Magentic multi-agent mode. These issues must be resolved to ensure a stable and configurable production deployment.
8
+
9
+ ## 1. Hardcoded OpenAI Models (High Severity)
10
+
11
+ **Issue:**
12
+ The agent factory functions in `src/agents/magentic_agents.py` have hardcoded model IDs (`gpt-4o` and `gpt-4o-mini`).
13
+
14
+ ```python
15
+ # src/agents/magentic_agents.py
16
+
17
+ def create_search_agent(...):
18
+ client = chat_client or OpenAIChatClient(
19
+ model_id="gpt-4o-mini", # <--- HARDCODED
20
+ api_key=settings.openai_api_key,
21
+ )
22
+
23
+ def create_judge_agent(...):
24
+ client = chat_client or OpenAIChatClient(
25
+ model_id="gpt-4o", # <--- HARDCODED
26
+ api_key=settings.openai_api_key,
27
+ )
28
+ ```
29
+
30
+ **Impact:**
31
+ 1. **Configuration ignored:** The user's `OPENAI_MODEL` setting (from `.env`) is ignored by the agents, only used by the Manager.
32
+ 2. **Access Failure:** Users without access to `gpt-4o` (e.g., legacy tiers) cannot run the system, even if they configure `gpt-3.5-turbo` in their env.
33
+ 3. **Cost Control:** Users cannot downgrade to cheaper models for development/testing.
34
+
35
+ **Fix Required:**
36
+ Update `src/agents/magentic_agents.py` to use `settings.openai_model` (or a specific agent config) instead of hardcoded strings.
37
+
38
+ ---
39
+
40
+ ## 2. Dependency Source Ambiguity (High Severity)
41
+
42
+ **Issue:**
43
+ The `pyproject.toml` declares a dependency on `agent-framework-core` without specifying a version or path.
44
+
45
+ ```toml
46
+ [project.optional-dependencies]
47
+ magentic = [
48
+ "agent-framework-core",
49
+ ]
50
+ ```
51
+
52
+ **Impact:**
53
+ 1. **PyPI vs. Local Mismatch:** It is unclear if `agent-framework-core` is being pulled from PyPI or the local `reference_repos`. The local reference contains specific `MagenticBuilder` logic that may not be present or identical in a potentially stale PyPI package.
54
+ 2. **Deployment Failure:** If the PyPI package is missing or version-mismatched, `pip install .[magentic]` will fail or install a broken version on deployment (e.g., HuggingFace Spaces).
55
+
56
+ **Fix Required:**
57
+ Explicitly define the source of `agent-framework-core`. If relying on the local reference, use a relative path dependency or ensure the correct version is published and pinned.
58
+ *Recommendation:* For this repo, since `reference_repos` is included, we should install from the local path in development, but this is hard with standard `pyproject.toml` (path dependencies don't work well for uploads).
59
+ *Alternative:* Verify exact PyPI version matches `reference_repos` and pin it.
60
+
61
+ ---
62
+
63
+ ## 3. Missing "Free Tier" for Magentic Mode (Medium Severity)
64
+
65
+ **Issue:**
66
+ Magentic mode is currently hard-locked to OpenAI.
67
+
68
+ ```python
69
+ # src/orchestrator_magentic.py
70
+ if not settings.openai_api_key:
71
+ raise ConfigurationError("Magentic mode requires OPENAI_API_KEY...")
72
+ ```
73
+
74
+ **Impact:**
75
+ Users relying on the "Free Tier" (HuggingFace Inference) cannot use the Multi-Agent features. This bifurcates the user experience:
76
+ * **Free User:** Simple linear search (Phase 4).
77
+ * **Paid User:** Advanced multi-agent loop (Phase 5).
78
+
79
+ **Mitigation:**
80
+ This is currently "Working as Designed" due to technical limitations of HF models with tool calling, but it should be clearly documented in the UI (which has been done in `app.py`).
81
+
82
+ ---
83
+
84
+ ## Action Plan
85
+
86
+ 1. [IMMEDIATE] Refactor `src/agents/magentic_agents.py` to use `settings.openai_model`.
87
+ 2. [IMMEDIATE] Verify `agent-framework-core` installation source.
src/agents/magentic_agents.py CHANGED
@@ -22,7 +22,7 @@ def create_search_agent(chat_client: OpenAIChatClient | None = None) -> ChatAgen
22
  ChatAgent configured for biomedical search
23
  """
24
  client = chat_client or OpenAIChatClient(
25
- model_id="gpt-4o-mini", # Fast, cheap for tool orchestration
26
  api_key=settings.openai_api_key,
27
  )
28
 
@@ -60,7 +60,7 @@ def create_judge_agent(chat_client: OpenAIChatClient | None = None) -> ChatAgent
60
  ChatAgent configured for evidence assessment
61
  """
62
  client = chat_client or OpenAIChatClient(
63
- model_id="gpt-4o", # Better model for nuanced judgment
64
  api_key=settings.openai_api_key,
65
  )
66
 
@@ -99,7 +99,7 @@ def create_hypothesis_agent(chat_client: OpenAIChatClient | None = None) -> Chat
99
  ChatAgent configured for hypothesis generation
100
  """
101
  client = chat_client or OpenAIChatClient(
102
- model_id="gpt-4o",
103
  api_key=settings.openai_api_key,
104
  )
105
 
@@ -136,7 +136,7 @@ def create_report_agent(chat_client: OpenAIChatClient | None = None) -> ChatAgen
136
  ChatAgent configured for report generation
137
  """
138
  client = chat_client or OpenAIChatClient(
139
- model_id="gpt-4o",
140
  api_key=settings.openai_api_key,
141
  )
142
 
 
22
  ChatAgent configured for biomedical search
23
  """
24
  client = chat_client or OpenAIChatClient(
25
+ model_id=settings.openai_model, # Use configured model
26
  api_key=settings.openai_api_key,
27
  )
28
 
 
60
  ChatAgent configured for evidence assessment
61
  """
62
  client = chat_client or OpenAIChatClient(
63
+ model_id=settings.openai_model,
64
  api_key=settings.openai_api_key,
65
  )
66
 
 
99
  ChatAgent configured for hypothesis generation
100
  """
101
  client = chat_client or OpenAIChatClient(
102
+ model_id=settings.openai_model,
103
  api_key=settings.openai_api_key,
104
  )
105
 
 
136
  ChatAgent configured for report generation
137
  """
138
  client = chat_client or OpenAIChatClient(
139
+ model_id=settings.openai_model,
140
  api_key=settings.openai_api_key,
141
  )
142