DevodG commited on
Commit
bf995e4
·
1 Parent(s): 02859f0

Update code

Browse files
.kiro/specs/ai-financial-intelligence-system/.config.kiro ADDED
@@ -0,0 +1 @@
 
 
1
+ {"specId": "6fa4d985-8691-4485-8edc-fea35099c0a4", "workflowType": "requirements-first", "specType": "feature"}
.kiro/specs/ai-financial-intelligence-system/design.md ADDED
@@ -0,0 +1,2715 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Design Document: MiroOrg v1.1
2
+
3
+ ## Overview
4
+
5
+ MiroOrg v1.1 is a general intelligence operating system that orchestrates multiple specialist agents, runs simulations, supports pluggable domain packs, and autonomously improves itself over time. The system merges capabilities from miroorg-basic-v2 (base architecture), impact_ai (first domain pack), MiroFish (simulation lab), and public-apis (API discovery catalog) into a unified, production-ready platform.
6
+
7
+ ### Core Principles
8
+
9
+ - **Modularity**: Clear separation between core platform, agent organization, domain packs, simulation lab, and autonomous learning
10
+ - **Extensibility**: Domain packs can be added without modifying agent orchestration layer
11
+ - **Provider Agnostic**: Abstract AI model providers (OpenRouter, Ollama, future OpenAI) behind unified interface
12
+ - **Local-First**: Single-user local deployment with production-quality code structure
13
+ - **Simulation-Ready**: Deep integration with MiroFish for scenario modeling and what-if analysis
14
+ - **Self-Improving**: Autonomous learning from internet knowledge, past cases, and successful patterns without local model training
15
+ - **Resource-Conscious**: Strict storage limits and lightweight background tasks suitable for 8GB/256GB laptop
16
+
17
+ ### System Context
18
+
19
+ The system operates as a FastAPI backend with a Next.js frontend dashboard. All state is stored locally in JSON files. External services (MiroFish, Tavily, NewsAPI, Alpha Vantage) are accessed through adapter clients. The architecture supports both synchronous analysis and asynchronous simulation workflows.
20
+
21
+ ## Architecture
22
+
23
+ ### Five-Layer Architecture
24
+
25
+ ```
26
+ ┌─────────────────────────────────────────────────────────────┐
27
+ │ Layer 5: Autonomous Knowledge Evolution (Self-Improvement) │
28
+ │ - World knowledge ingestion (compressed summaries) │
29
+ │ - Experience learning from cases │
30
+ │ - Prompt evolution and optimization │
31
+ │ - Skill distillation from patterns │
32
+ │ - Trust and freshness management │
33
+ └─────────────────────────────────────────────────────────────┘
34
+
35
+
36
+ ┌─────────────────────────────────────────────────────────────┐
37
+ │ Layer 4: Simulation Lab (MiroFish Integration) │
38
+ │ - Graph building, persona generation, simulation execution │
39
+ │ - Report generation, post-simulation chat │
40
+ └─────────────────────────────────────────────────────────────┘
41
+
42
+
43
+ ┌─────────────────────────────────────────────────────────────┐
44
+ │ Layer 3: Domain Packs (Pluggable Intelligence Modules) │
45
+ │ - Finance Pack: market data, news, entity/ticker detection │
46
+ │ - Future: Policy, Cyber, Enterprise Ops, Research, Edu │
47
+ └─────────────────────────────────────────────────────────────┘
48
+
49
+
50
+ ┌─────────────────────────────────────────────────────────────┐
51
+ │ Layer 2: Agent Organization (Multi-Agent Orchestration) │
52
+ │ - Switchboard: routing and classification │
53
+ │ - Research: context gathering and entity extraction │
54
+ │ - Planner: action plan generation │
55
+ │ - Verifier: credibility validation and uncertainty │
56
+ │ - Synthesizer: final response composition │
57
+ └─────────────────────────────────────────────────────────────┘
58
+
59
+
60
+ ┌─────────────────────────────────────────────────────────────┐
61
+ │ Layer 1: Core Platform (Infrastructure) │
62
+ │ - FastAPI backend, Next.js frontend │
63
+ │ - Config, health, memory, prompts, cases, logs │
64
+ │ - Provider abstraction layer │
65
+ └─────────────────────────────────────────────────────────────┘
66
+ ```
67
+
68
+
69
+ ### Execution Flow
70
+
71
+ ```mermaid
72
+ graph TD
73
+ A[User Input] --> B[Switchboard]
74
+ B --> C{Task Classification}
75
+ C -->|Simple| D[Solo Mode: Direct Response]
76
+ C -->|Medium| E[Standard Mode: Research + Planner]
77
+ C -->|Complex| F[Deep Mode: Full Pipeline]
78
+ C -->|Simulation| G[Simulation Mode: MiroFish]
79
+
80
+ E --> H[Research Agent]
81
+ H --> I[Planner Agent]
82
+ I --> J[Synthesizer Agent]
83
+
84
+ F --> K[Research Agent]
85
+ K --> L[Planner Agent]
86
+ L --> M[Verifier Agent]
87
+ M --> N[Synthesizer Agent]
88
+
89
+ G --> O[MiroFish Client]
90
+ O --> P[Graph Building]
91
+ P --> Q[Simulation Execution]
92
+ Q --> R[Report Generation]
93
+ R --> S[Post-Simulation Chat]
94
+
95
+ D --> T[Save Case]
96
+ J --> T
97
+ N --> T
98
+ S --> T
99
+ T --> U[Return Response]
100
+ ```
101
+
102
+ ### Repository Ownership
103
+
104
+ - **Primary Repo**: miroorg-basic-v2 (canonical architecture)
105
+ - **Domain Source**: impact_ai (reusable domain modules, not structural peer)
106
+ - **Simulation Service**: MiroFish (separate service, accessed via adapter)
107
+ - **API Catalog**: public-apis (discovery dataset, not runtime dependency)
108
+
109
+ ### Technology Stack
110
+
111
+ - **Backend**: Python 3.10+, FastAPI, LangGraph, Pydantic, httpx
112
+ - **Frontend**: Next.js 14+, React, TypeScript, Tailwind CSS
113
+ - **Storage**: Local JSON files (cases, simulations, logs)
114
+ - **AI Providers**: OpenRouter (primary), Ollama (fallback), future OpenAI
115
+ - **External APIs**: Tavily, NewsAPI, Alpha Vantage, Jina Reader
116
+ - **Simulation**: MiroFish (external service)
117
+
118
+ ## Components and Interfaces
119
+
120
+ ### Layer 1: Core Platform
121
+
122
+ #### Provider Abstraction Layer
123
+
124
+ **Purpose**: Unified interface for AI model providers with automatic fallback
125
+
126
+ **Interface**:
127
+ ```python
128
+ def call_model(
129
+ prompt: str,
130
+ mode: str = "chat", # "chat" or "reasoner"
131
+ system_prompt: Optional[str] = None,
132
+ provider_override: Optional[str] = None,
133
+ ) -> str:
134
+ """
135
+ Call AI model with automatic provider fallback.
136
+
137
+ Args:
138
+ prompt: User prompt or agent instruction
139
+ mode: "chat" for general tasks, "reasoner" for complex analysis
140
+ system_prompt: Optional system-level instructions
141
+ provider_override: Force specific provider (testing/debugging)
142
+
143
+ Returns:
144
+ Model response text
145
+
146
+ Raises:
147
+ LLMProviderError: When all providers fail
148
+ """
149
+ ```
150
+
151
+ **Implementation Strategy**:
152
+ - Current: `_call_openrouter()` and `_call_ollama()` in `backend/app/agents/_model.py`
153
+ - Enhancement: Add `_call_openai()` for future OpenAI support
154
+ - Fallback Logic: Try primary provider, catch exception, try fallback provider
155
+ - Logging: Log provider selection, fallback events, and failures
156
+
157
+
158
+ #### Configuration Management
159
+
160
+ **Purpose**: Centralized environment-based configuration
161
+
162
+ **File**: `backend/app/config.py`
163
+
164
+ **Configuration Groups**:
165
+ 1. **App Settings**: VERSION, DIRS (prompts, data, memory, simulations, logs)
166
+ 2. **Provider Settings**: PRIMARY_PROVIDER, FALLBACK_PROVIDER, model names, API keys
167
+ 3. **External API Settings**: TAVILY_API_KEY, NEWSAPI_KEY, ALPHAVANTAGE_API_KEY, JINA_READER_BASE
168
+ 4. **MiroFish Settings**: ENABLED, API_BASE, TIMEOUT, endpoint paths
169
+ 5. **Simulation Settings**: TRIGGER_KEYWORDS (configurable list)
170
+
171
+ **Enhancement Strategy**:
172
+ - Add validation on startup for required keys
173
+ - Add warnings for missing optional keys
174
+ - Add feature flags for domain packs
175
+ - Add logging configuration
176
+
177
+ #### Memory and Storage
178
+
179
+ **Purpose**: Local persistence for cases, simulations, and logs
180
+
181
+ **Directory Structure**:
182
+ ```
183
+ backend/app/data/
184
+ ├── memory/ # Case execution records (JSON)
185
+ ├── simulations/ # Simulation metadata (JSON)
186
+ └── logs/ # Application logs (rotating)
187
+ ```
188
+
189
+ **Case Storage Interface**:
190
+ ```python
191
+ def save_case(case_id: str, payload: Dict[str, Any]) -> None
192
+ def get_case(case_id: str) -> Optional[Dict[str, Any]]
193
+ def list_cases(limit: Optional[int] = None) -> List[Dict[str, Any]]
194
+ def delete_case(case_id: str) -> bool
195
+ def memory_stats() -> Dict[str, Any]
196
+ ```
197
+
198
+ **Simulation Storage Interface**:
199
+ ```python
200
+ def save_simulation(simulation_id: str, record: Dict[str, Any]) -> None
201
+ def get_simulation(simulation_id: str) -> Optional[Dict[str, Any]]
202
+ def list_simulations(limit: Optional[int] = None) -> List[Dict[str, Any]]
203
+ ```
204
+
205
+ **Case Schema**:
206
+ ```json
207
+ {
208
+ "case_id": "uuid",
209
+ "user_input": "string",
210
+ "route": {
211
+ "task_family": "normal|simulation",
212
+ "domain_pack": "finance|general|policy|custom",
213
+ "complexity": "simple|medium|complex",
214
+ "execution_mode": "solo|standard|deep"
215
+ },
216
+ "outputs": [
217
+ {
218
+ "agent": "research|planner|verifier|synthesizer",
219
+ "summary": "string",
220
+ "details": {},
221
+ "confidence": 0.0
222
+ }
223
+ ],
224
+ "final_answer": "string",
225
+ "simulation_id": "optional uuid",
226
+ "created_at": "ISO timestamp",
227
+ "updated_at": "ISO timestamp"
228
+ }
229
+ ```
230
+
231
+
232
+ ### Layer 2: Agent Organization
233
+
234
+ #### Switchboard Agent
235
+
236
+ **Purpose**: Classify tasks and route to appropriate execution mode
237
+
238
+ **Current Implementation**: `backend/app/agents/switchboard.py`
239
+
240
+ **Classification Dimensions**:
241
+ 1. **task_family**: "normal" or "simulation" (based on trigger keywords)
242
+ 2. **domain_pack**: "finance", "general", "policy", "custom" (future enhancement)
243
+ 3. **complexity**: "simple" (≤5 words), "medium" (≤25 words), "complex" (>25 words)
244
+ 4. **execution_mode**: "solo", "standard", "deep"
245
+
246
+ **Routing Logic**:
247
+ ```python
248
+ def decide_route(user_input: str) -> Dict[str, Any]:
249
+ """
250
+ Classify task and determine execution path.
251
+
252
+ Returns:
253
+ {
254
+ "task_family": str,
255
+ "domain_pack": str,
256
+ "complexity": str,
257
+ "execution_mode": str,
258
+ "risk_level": str
259
+ }
260
+ """
261
+ ```
262
+
263
+ **Enhancement Strategy**:
264
+ - Add domain_pack detection (keywords: "stock", "market", "ticker" → finance)
265
+ - Add entity extraction for domain routing
266
+ - Add confidence scoring for routing decisions
267
+ - Add routing decision logging
268
+
269
+ **Execution Mode Mapping**:
270
+ - **solo**: Simple queries, direct response, no multi-agent collaboration
271
+ - **standard**: Medium complexity, Research → Planner → Synthesizer
272
+ - **deep**: Complex queries, full pipeline with Verifier, optional simulation handoff
273
+
274
+ #### Research Agent
275
+
276
+ **Purpose**: Gather context, extract entities, fetch external information
277
+
278
+ **Current Implementation**: `backend/app/agents/research.py`
279
+
280
+ **Responsibilities**:
281
+ 1. Extract entities (companies, people, concepts)
282
+ 2. Extract tickers (stock symbols like $AAPL)
283
+ 3. Extract URLs for content reading
284
+ 4. Fetch external context (Tavily search, NewsAPI, Alpha Vantage, Jina Reader)
285
+ 5. Return structured facts, assumptions, open questions, useful signals
286
+
287
+ **Interface**:
288
+ ```python
289
+ def run_research(user_input: str, prompt_template: str) -> Dict[str, Any]:
290
+ """
291
+ Gather context and extract entities from user input.
292
+
293
+ Returns:
294
+ {
295
+ "agent": "research",
296
+ "summary": str,
297
+ "details": {
298
+ "external_context_used": bool,
299
+ "entities": List[str],
300
+ "tickers": List[str],
301
+ "urls": List[str]
302
+ },
303
+ "confidence": float
304
+ }
305
+ """
306
+ ```
307
+
308
+ **Enhancement Strategy**:
309
+ - Integrate impact_ai entity_resolver.py for better entity extraction
310
+ - Integrate impact_ai ticker_resolver.py for ticker normalization
311
+ - Add structured entity extraction (not just text summary)
312
+ - Add domain-specific context gathering (finance pack)
313
+ - Add caching for repeated queries
314
+
315
+
316
+ #### Planner Agent
317
+
318
+ **Purpose**: Convert research into practical action plans
319
+
320
+ **Current Implementation**: `backend/app/agents/planner.py`
321
+
322
+ **Responsibilities**:
323
+ 1. Synthesize research findings into actionable recommendations
324
+ 2. Highlight dependencies and risks
325
+ 3. Suggest next steps
326
+ 4. Identify when simulation mode would be more appropriate
327
+
328
+ **Interface**:
329
+ ```python
330
+ def run_planner(
331
+ user_input: str,
332
+ research_summary: str,
333
+ prompt_template: str
334
+ ) -> Dict[str, Any]:
335
+ """
336
+ Generate action plan from research findings.
337
+
338
+ Returns:
339
+ {
340
+ "agent": "planner",
341
+ "summary": str,
342
+ "details": {
343
+ "recommendations": List[str],
344
+ "risks": List[str],
345
+ "next_steps": List[str],
346
+ "simulation_suggested": bool
347
+ },
348
+ "confidence": float
349
+ }
350
+ """
351
+ ```
352
+
353
+ **Enhancement Strategy**:
354
+ - Add structured output parsing (recommendations, risks, next_steps)
355
+ - Add simulation mode detection and suggestion
356
+ - Add domain-specific planning (finance pack)
357
+ - Update prompt to include domain intelligence instructions
358
+
359
+ #### Verifier Agent
360
+
361
+ **Purpose**: Validate credibility, detect rumors/scams, surface uncertainty
362
+
363
+ **Current Implementation**: `backend/app/agents/verifier.py`
364
+
365
+ **Responsibilities**:
366
+ 1. Test credibility of information sources
367
+ 2. Detect rumors and unsupported claims
368
+ 3. Detect scams and fraudulent information
369
+ 4. Identify contradictions in research and planning
370
+ 5. Force uncertainty to be made visible
371
+
372
+ **Interface**:
373
+ ```python
374
+ def run_verifier(
375
+ user_input: str,
376
+ research_summary: str,
377
+ planner_summary: str,
378
+ prompt_template: str
379
+ ) -> Dict[str, Any]:
380
+ """
381
+ Validate credibility and detect issues.
382
+
383
+ Returns:
384
+ {
385
+ "agent": "verifier",
386
+ "summary": str,
387
+ "details": {
388
+ "credibility_score": float,
389
+ "rumors_detected": List[str],
390
+ "scams_detected": List[str],
391
+ "contradictions": List[str],
392
+ "uncertainty_areas": List[str]
393
+ },
394
+ "confidence": float
395
+ }
396
+ """
397
+ ```
398
+
399
+ **Enhancement Strategy**:
400
+ - Integrate impact_ai source_checker.py for source credibility scoring
401
+ - Integrate impact_ai rumor_detector.py for rumor detection
402
+ - Integrate impact_ai scam_detector.py for scam detection
403
+ - Add structured output parsing
404
+ - Update prompt with domain intelligence instructions
405
+ - Only run in "deep" execution mode
406
+
407
+
408
+ #### Synthesizer Agent
409
+
410
+ **Purpose**: Combine outputs into final comprehensive response
411
+
412
+ **Current Implementation**: `backend/app/agents/synthesizer.py`
413
+
414
+ **Responsibilities**:
415
+ 1. Combine research, planning, and verification outputs
416
+ 2. State uncertainty honestly
417
+ 3. Recommend next actions
418
+ 4. Suggest simulation mode when scenario analysis is appropriate
419
+
420
+ **Interface**:
421
+ ```python
422
+ def run_synthesizer(
423
+ user_input: str,
424
+ research_summary: str,
425
+ planner_summary: str,
426
+ verifier_summary: str,
427
+ prompt_template: str
428
+ ) -> Dict[str, Any]:
429
+ """
430
+ Produce final comprehensive response.
431
+
432
+ Returns:
433
+ {
434
+ "agent": "synthesizer",
435
+ "summary": str,
436
+ "details": {
437
+ "uncertainty_level": str,
438
+ "next_actions": List[str],
439
+ "simulation_recommended": bool
440
+ },
441
+ "confidence": float
442
+ }
443
+ """
444
+ ```
445
+
446
+ **Enhancement Strategy**:
447
+ - Add structured output parsing
448
+ - Add uncertainty quantification
449
+ - Add simulation mode recommendation logic
450
+ - Update prompt with domain intelligence instructions
451
+
452
+ ### Layer 3: Domain Packs
453
+
454
+ #### Domain Pack Architecture
455
+
456
+ **Purpose**: Pluggable domain intelligence modules that extend agent capabilities
457
+
458
+ **Design Pattern**:
459
+ ```
460
+ backend/app/domain_packs/
461
+ ├── __init__.py
462
+ ├── base.py # Abstract base class for domain packs
463
+ ├── finance/ # First domain pack (from impact_ai)
464
+ │ ├── __init__.py
465
+ │ ├── market_data.py # Alpha Vantage integration
466
+ │ ├── news.py # NewsAPI integration
467
+ │ ├── entity_resolver.py
468
+ │ ├── ticker_resolver.py
469
+ │ ├── source_checker.py
470
+ │ ├── rumor_detector.py
471
+ │ ├── scam_detector.py
472
+ │ ├── stance_detector.py
473
+ │ ├── event_analyzer.py
474
+ │ └── prediction.py
475
+ └── registry.py # Domain pack registration and discovery
476
+ ```
477
+
478
+ **Base Domain Pack Interface**:
479
+ ```python
480
+ class DomainPack(ABC):
481
+ """Abstract base class for domain packs."""
482
+
483
+ @property
484
+ @abstractmethod
485
+ def name(self) -> str:
486
+ """Domain pack identifier (e.g., 'finance', 'policy')."""
487
+ pass
488
+
489
+ @property
490
+ @abstractmethod
491
+ def keywords(self) -> List[str]:
492
+ """Keywords for automatic domain detection."""
493
+ pass
494
+
495
+ @abstractmethod
496
+ def enhance_research(self, user_input: str, context: Dict[str, Any]) -> Dict[str, Any]:
497
+ """Enhance research agent with domain-specific context."""
498
+ pass
499
+
500
+ @abstractmethod
501
+ def enhance_verification(self, claims: List[str], context: Dict[str, Any]) -> Dict[str, Any]:
502
+ """Enhance verifier agent with domain-specific checks."""
503
+ pass
504
+
505
+ @abstractmethod
506
+ def get_capabilities(self) -> List[str]:
507
+ """List domain-specific capabilities."""
508
+ pass
509
+ ```
510
+
511
+
512
+ #### Finance Domain Pack
513
+
514
+ **Purpose**: Financial intelligence capabilities from impact_ai
515
+
516
+ **Modules to Integrate**:
517
+
518
+ 1. **market_data.py**: Alpha Vantage client for stock quotes, historical data
519
+ 2. **news.py**: NewsAPI client for financial news
520
+ 3. **entity_resolver.py**: Extract and normalize company/organization names
521
+ 4. **ticker_resolver.py**: Resolve company names to stock tickers
522
+ 5. **source_checker.py**: Score credibility of financial news sources
523
+ 6. **rumor_detector.py**: Detect unverified market rumors
524
+ 7. **scam_detector.py**: Detect investment scams and fraud
525
+ 8. **stance_detector.py**: Analyze sentiment and stance in financial text
526
+ 9. **event_analyzer.py**: Analyze market events and their impacts
527
+ 10. **prediction.py**: Market prediction and scenario modeling
528
+
529
+ **Integration Strategy**:
530
+ - Refactor impact_ai modules to match service layer pattern
531
+ - Consolidate overlapping clients (Alpha Vantage, NewsAPI) with existing external_sources.py
532
+ - Expose capabilities through FinanceDomainPack class
533
+ - Register pack in domain pack registry
534
+ - Update agent prompts to include finance-specific instructions when pack is active
535
+
536
+ **Finance Pack Interface**:
537
+ ```python
538
+ class FinanceDomainPack(DomainPack):
539
+ name = "finance"
540
+ keywords = ["stock", "market", "ticker", "trading", "investment", "portfolio",
541
+ "earnings", "dividend", "IPO", "SEC", "financial"]
542
+
543
+ def enhance_research(self, user_input: str, context: Dict[str, Any]) -> Dict[str, Any]:
544
+ """
545
+ Add financial context:
546
+ - Extract tickers and resolve company names
547
+ - Fetch market quotes
548
+ - Fetch recent financial news
549
+ - Extract financial entities
550
+ """
551
+
552
+ def enhance_verification(self, claims: List[str], context: Dict[str, Any]) -> Dict[str, Any]:
553
+ """
554
+ Add financial verification:
555
+ - Check source credibility for financial news
556
+ - Detect market rumors
557
+ - Detect investment scams
558
+ - Analyze stance and sentiment
559
+ """
560
+ ```
561
+
562
+ #### Domain Pack Registry
563
+
564
+ **Purpose**: Centralized registration and discovery of domain packs
565
+
566
+ **Interface**:
567
+ ```python
568
+ class DomainPackRegistry:
569
+ def register(self, pack: DomainPack) -> None
570
+ def get_pack(self, name: str) -> Optional[DomainPack]
571
+ def detect_domain(self, user_input: str) -> Optional[str]
572
+ def list_packs(self) -> List[str]
573
+ def get_capabilities(self, domain: str) -> List[str]
574
+
575
+ # Global registry instance
576
+ domain_registry = DomainPackRegistry()
577
+ domain_registry.register(FinanceDomainPack())
578
+ ```
579
+
580
+ **Usage in Agents**:
581
+ ```python
582
+ # In research agent
583
+ domain = domain_registry.detect_domain(user_input)
584
+ if domain:
585
+ pack = domain_registry.get_pack(domain)
586
+ enhanced_context = pack.enhance_research(user_input, base_context)
587
+ ```
588
+
589
+
590
+ ### Layer 4: Simulation Lab
591
+
592
+ #### MiroFish Integration Architecture
593
+
594
+ **Purpose**: External simulation service for scenario modeling and what-if analysis
595
+
596
+ **Integration Pattern**: Adapter client (not direct merge)
597
+
598
+ **Current Implementation**: `backend/app/services/mirofish_client.py`
599
+
600
+ **MiroFish Capabilities**:
601
+ 1. **Graph Building**: Extract entities and relationships from seed text
602
+ 2. **Persona Generation**: Create realistic personas for simulation
603
+ 3. **Environment Setup**: Configure simulation parameters
604
+ 4. **Simulation Execution**: Run multi-agent scenario modeling
605
+ 5. **Report Generation**: Produce structured simulation reports
606
+ 6. **Post-Simulation Chat**: Deep interaction with simulation results
607
+
608
+ **Client Interface**:
609
+ ```python
610
+ def mirofish_health() -> Dict[str, Any]:
611
+ """Check MiroFish service availability."""
612
+
613
+ def run_simulation(payload: Dict[str, Any]) -> Dict[str, Any]:
614
+ """
615
+ Submit simulation request.
616
+
617
+ Payload:
618
+ {
619
+ "title": str,
620
+ "seed_text": str,
621
+ "prediction_goal": str,
622
+ "mode": "standard|deep",
623
+ "metadata": {}
624
+ }
625
+
626
+ Returns:
627
+ {
628
+ "simulation_id": str,
629
+ "status": "submitted|running|completed|failed",
630
+ "message": str
631
+ }
632
+ """
633
+
634
+ def simulation_status(simulation_id: str) -> Dict[str, Any]:
635
+ """Get simulation execution status."""
636
+
637
+ def simulation_report(simulation_id: str) -> Dict[str, Any]:
638
+ """Retrieve simulation report."""
639
+
640
+ def simulation_chat(simulation_id: str, message: str) -> Dict[str, Any]:
641
+ """Ask questions about simulation results."""
642
+ ```
643
+
644
+ **Router Implementation**: `backend/app/routers/simulation.py`
645
+
646
+ **Endpoints**:
647
+ - `GET /simulation/health`: MiroFish health check
648
+ - `POST /simulation/run`: Submit simulation
649
+ - `GET /simulation/{simulation_id}`: Get status
650
+ - `GET /simulation/{simulation_id}/report`: Get report
651
+ - `POST /simulation/{simulation_id}/chat`: Post-simulation chat
652
+
653
+ **Error Handling**:
654
+ - Graceful degradation when MiroFish is disabled
655
+ - Descriptive error messages for connection failures
656
+ - Local metadata storage even when remote service fails
657
+ - Timeout configuration via environment variables
658
+
659
+ **Frontend Integration Rule**: Frontend MUST only consume MiroOrg endpoints, never direct MiroFish calls
660
+
661
+
662
+ #### Simulation Workflow Integration
663
+
664
+ **Trigger Detection**: Switchboard detects simulation keywords and sets task_family="simulation"
665
+
666
+ **Simulation Keywords** (configurable):
667
+ - simulate, predict, model reaction, test scenarios, run digital twins
668
+ - explore "what if" outcomes, forecast, scenario analysis
669
+ - public opinion, policy impact, market impact, stakeholder reaction
670
+
671
+ **Execution Flow**:
672
+ 1. User submits query with simulation keywords
673
+ 2. Switchboard classifies as task_family="simulation", execution_mode="deep"
674
+ 3. Research agent gathers context (optional)
675
+ 4. Planner agent structures simulation parameters (optional)
676
+ 5. System submits to MiroFish via `/simulation/run`
677
+ 6. System polls `/simulation/{id}` for status
678
+ 7. When complete, system retrieves report via `/simulation/{id}/report`
679
+ 8. Synthesizer agent summarizes simulation results
680
+ 9. User can ask follow-up questions via `/simulation/{id}/chat`
681
+
682
+ **Case Linking**: Simulation metadata includes case_id, case metadata includes simulation_id
683
+
684
+ ### API Discovery Subsystem
685
+
686
+ **Purpose**: Discover and classify free APIs from public-apis catalog for future connector expansion
687
+
688
+ **Architecture**:
689
+ ```
690
+ backend/app/services/api_discovery/
691
+ ├── __init__.py
692
+ ├── catalog_loader.py # Load public-apis JSON data
693
+ ├── classifier.py # Classify APIs by category and usefulness
694
+ ├── scorer.py # Score APIs for integration priority
695
+ └── metadata_store.py # Store API metadata locally
696
+ ```
697
+
698
+ **Catalog Loader**:
699
+ ```python
700
+ def load_public_apis_catalog() -> List[Dict[str, Any]]:
701
+ """
702
+ Load public-apis catalog from GitHub or local cache.
703
+
704
+ Returns:
705
+ List of API entries with:
706
+ - API name
707
+ - Description
708
+ - Auth type (apiKey, OAuth, None)
709
+ - HTTPS support
710
+ - CORS support
711
+ - Category
712
+ - Link
713
+ """
714
+ ```
715
+
716
+ **Classifier**:
717
+ ```python
718
+ def classify_api(api_entry: Dict[str, Any]) -> Dict[str, Any]:
719
+ """
720
+ Classify API by category and potential use cases.
721
+
722
+ Categories:
723
+ - market_data: Stock, crypto, commodities
724
+ - news: News aggregation, RSS feeds
725
+ - social: Social media, sentiment
726
+ - government: Policy, regulations, open data
727
+ - weather: Weather, climate
728
+ - general: Utilities, reference data
729
+ """
730
+ ```
731
+
732
+ **Scorer**:
733
+ ```python
734
+ def score_api_usefulness(api_entry: Dict[str, Any]) -> float:
735
+ """
736
+ Score API for integration priority (0.0 - 1.0).
737
+
738
+ Factors:
739
+ - Auth simplicity (no auth > apiKey > OAuth)
740
+ - HTTPS support (required)
741
+ - CORS support (preferred)
742
+ - Category relevance to domain packs
743
+ - Description quality
744
+ """
745
+ ```
746
+
747
+ **Usage**: Discovery subsystem is for future expansion, not runtime dependency
748
+
749
+
750
+ ### Layer 5: Autonomous Knowledge Evolution
751
+
752
+ **Purpose**: Self-improving intelligence system that learns from internet knowledge, past cases, prompt evolution, and skill distillation without local model training
753
+
754
+ **Design Constraints**:
755
+ - NO local model training or fine-tuning
756
+ - NO large raw dataset storage
757
+ - Store only compressed summaries (2-4KB per item)
758
+ - Max knowledge cache: 200MB
759
+ - Lightweight background tasks only
760
+ - Battery-aware and resource-conscious
761
+ - Suitable for 8GB/256GB laptop
762
+
763
+ **Architecture**:
764
+ ```
765
+ backend/app/services/learning/
766
+ ├── __init__.py
767
+ ├── knowledge_ingestor.py # Ingest and compress external knowledge
768
+ ├── knowledge_store.py # Store compressed knowledge items
769
+ ├── learning_engine.py # Core learning logic and pattern detection
770
+ ├── prompt_optimizer.py # Prompt evolution and testing
771
+ ├── skill_distiller.py # Extract reusable skills from patterns
772
+ ├── trust_manager.py # Source reliability tracking
773
+ ├── freshness_manager.py # Information recency tracking
774
+ └── scheduler.py # Lightweight background task scheduler
775
+ ```
776
+
777
+ **Data Directories**:
778
+ ```
779
+ backend/app/data/
780
+ ├── knowledge/ # Compressed knowledge items (max 200MB)
781
+ ├── skills/ # Distilled reusable skills
782
+ ├── prompt_versions/ # Prompt version history
783
+ └── learning/ # Learning metadata and statistics
784
+ ```
785
+
786
+ #### Knowledge Ingestion
787
+
788
+ **Purpose**: Continuously pull high-signal information and compress it
789
+
790
+ **Knowledge Item Schema**:
791
+ ```python
792
+ class KnowledgeItem(BaseModel):
793
+ id: str
794
+ title: str
795
+ summary: str # 2-4KB compressed summary
796
+ entities: List[str]
797
+ claims: List[str]
798
+ source_url: str
799
+ source_type: str # "news", "api", "search", "webpage"
800
+ trust_score: float # 0.0 - 1.0
801
+ freshness_score: float # 0.0 - 1.0
802
+ domain_pack: Optional[str] # "finance", "policy", etc.
803
+ created_at: str
804
+ expires_at: str
805
+ metadata: Dict[str, Any]
806
+ ```
807
+
808
+ **Ingestion Sources**:
809
+ - Tavily search results
810
+ - Jina Reader webpage summaries
811
+ - NewsAPI articles
812
+ - Alpha Vantage market data
813
+ - Domain-specific APIs
814
+ - Public-APIs catalog metadata
815
+
816
+ **Ingestion Rules**:
817
+ - Pull only during idle periods
818
+ - Respect API rate limits
819
+ - Compress immediately (no raw storage)
820
+ - Extract entities, claims, and key facts
821
+ - Score trust and freshness
822
+ - Set expiration based on domain rules
823
+
824
+ **Interface**:
825
+ ```python
826
+ def ingest_from_search(query: str, max_items: int = 5) -> List[KnowledgeItem]:
827
+ """Ingest knowledge from search results."""
828
+
829
+ def ingest_from_url(url: str) -> Optional[KnowledgeItem]:
830
+ """Ingest knowledge from specific URL."""
831
+
832
+ def ingest_from_news(query: str, max_items: int = 5) -> List[KnowledgeItem]:
833
+ """Ingest knowledge from news sources."""
834
+
835
+ def compress_content(raw_content: str, max_length: int = 4000) -> str:
836
+ """Compress raw content to summary."""
837
+ ```
838
+
839
+ #### Knowledge Store
840
+
841
+ **Purpose**: Store and retrieve compressed knowledge items
842
+
843
+ **Storage Strategy**:
844
+ - JSON files in `backend/app/data/knowledge/`
845
+ - One file per knowledge item
846
+ - Automatic cleanup of expired items
847
+ - Hard limit: 200MB total storage
848
+ - LRU eviction when limit reached
849
+
850
+ **Interface**:
851
+ ```python
852
+ def save_knowledge(item: KnowledgeItem) -> None
853
+ def get_knowledge(item_id: str) -> Optional[KnowledgeItem]
854
+ def search_knowledge(query: str, domain: Optional[str] = None) -> List[KnowledgeItem]
855
+ def list_knowledge(limit: Optional[int] = None) -> List[KnowledgeItem]
856
+ def delete_expired_knowledge() -> int
857
+ def get_storage_stats() -> Dict[str, Any]
858
+ ```
859
+
860
+ #### Experience Learning
861
+
862
+ **Purpose**: Learn from every case execution to improve future performance
863
+
864
+ **Case Learning Metadata**:
865
+ ```python
866
+ class CaseLearning(BaseModel):
867
+ case_id: str
868
+ route_effectiveness: float # Did routing work well?
869
+ prompt_performance: Dict[str, float] # Which prompts worked?
870
+ provider_reliability: Dict[str, bool] # Which providers succeeded?
871
+ source_usefulness: Dict[str, float] # Which sources were valuable?
872
+ pattern_detected: Optional[str] # Repeated pattern?
873
+ corrections_made: List[str] # User corrections?
874
+ execution_time: float
875
+ created_at: str
876
+ ```
877
+
878
+ **Learning Rules**:
879
+ - Track every case execution
880
+ - Store only metadata (not full case duplicate)
881
+ - Detect repeated patterns across cases
882
+ - Update trust scores based on verification outcomes
883
+ - Identify successful routing strategies
884
+ - Track prompt effectiveness
885
+
886
+ **Interface**:
887
+ ```python
888
+ def learn_from_case(case_id: str, case_data: Dict[str, Any]) -> CaseLearning
889
+ def detect_patterns(min_occurrences: int = 3) -> List[Dict[str, Any]]
890
+ def get_route_effectiveness(route_type: str) -> float
891
+ def get_prompt_performance(prompt_name: str) -> float
892
+ def recommend_improvements() -> List[str]
893
+ ```
894
+
895
+ #### Prompt Evolution
896
+
897
+ **Purpose**: Controlled evolution of agent prompts through testing and comparison
898
+
899
+ **Prompt Version Schema**:
900
+ ```python
901
+ class PromptVersion(BaseModel):
902
+ name: str # "research", "planner", etc.
903
+ version: str # "v1.0", "v1.1", etc.
904
+ content: str
905
+ status: str # "active", "experimental", "archived"
906
+ win_rate: float # Success rate in tests
907
+ test_count: int
908
+ last_tested: str
909
+ created_at: str
910
+ metadata: Dict[str, Any]
911
+ ```
912
+
913
+ **Evolution Process**:
914
+ 1. Store current prompt as baseline version
915
+ 2. Generate improved variant (using provider API)
916
+ 3. Test variant on sampled tasks
917
+ 4. Compare outcomes using quality metrics
918
+ 5. Promote if better, archive if worse
919
+ 6. Never auto-promote without validation
920
+
921
+ **Quality Metrics**:
922
+ - Response relevance
923
+ - Entity extraction accuracy
924
+ - Credibility detection rate
925
+ - User satisfaction (if available)
926
+ - Execution time
927
+
928
+ **Interface**:
929
+ ```python
930
+ def create_prompt_variant(prompt_name: str, improvement_goal: str) -> PromptVersion
931
+ def test_prompt_variant(variant: PromptVersion, test_cases: List[str]) -> float
932
+ def compare_prompts(baseline: PromptVersion, variant: PromptVersion) -> Dict[str, Any]
933
+ def promote_prompt(prompt_name: str, version: str) -> None
934
+ def archive_prompt(prompt_name: str, version: str) -> None
935
+ def get_prompt_history(prompt_name: str) -> List[PromptVersion]
936
+ ```
937
+
938
+ #### Skill Distillation
939
+
940
+ **Purpose**: Convert repeated successful patterns into reusable skills
941
+
942
+ **Skill Schema**:
943
+ ```python
944
+ class Skill(BaseModel):
945
+ name: str # "financial_rumor_review", "policy_reaction_analysis"
946
+ description: str
947
+ trigger_patterns: List[str] # Keywords that activate this skill
948
+ recommended_agents: List[str] # Which agents to use
949
+ preferred_sources: List[str] # Which sources to prioritize
950
+ prompt_overrides: Dict[str, str] # Agent-specific prompt additions
951
+ success_rate: float
952
+ usage_count: int
953
+ created_at: str
954
+ last_used: str
955
+ metadata: Dict[str, Any]
956
+ ```
957
+
958
+ **Distillation Process**:
959
+ 1. Detect repeated patterns (min 3 occurrences)
960
+ 2. Extract common elements (agents, sources, prompts)
961
+ 3. Create skill record
962
+ 4. Test skill on similar tasks
963
+ 5. Activate if success rate > 0.7
964
+
965
+ **Skill Types**:
966
+ - **financial_rumor_review**: Verify unverified market claims
967
+ - **policy_reaction_analysis**: Analyze policy impact scenarios
968
+ - **earnings_impact_brief**: Summarize earnings report impacts
969
+ - **simulation_prep_pack**: Prepare simulation parameters
970
+
971
+ **Interface**:
972
+ ```python
973
+ def detect_skill_candidates(min_occurrences: int = 3) -> List[Dict[str, Any]]
974
+ def distill_skill(pattern: Dict[str, Any]) -> Skill
975
+ def test_skill(skill: Skill, test_cases: List[str]) -> float
976
+ def activate_skill(skill_name: str) -> None
977
+ def get_skill(skill_name: str) -> Optional[Skill]
978
+ def list_skills() -> List[Skill]
979
+ def apply_skill(skill_name: str, user_input: str) -> Dict[str, Any]
980
+ ```
981
+
982
+ #### Trust Management
983
+
984
+ **Purpose**: Track source reliability and learn which sources are trustworthy
985
+
986
+ **Trust Score Schema**:
987
+ ```python
988
+ class SourceTrust(BaseModel):
989
+ source_id: str # URL domain or API name
990
+ source_type: str # "news", "api", "website"
991
+ trust_score: float # 0.0 - 1.0
992
+ verification_count: int
993
+ success_count: int
994
+ failure_count: int
995
+ last_verified: str
996
+ domain_pack: Optional[str]
997
+ metadata: Dict[str, Any]
998
+ ```
999
+
1000
+ **Trust Scoring Rules**:
1001
+ - Start with neutral score (0.5)
1002
+ - Increase on successful verification
1003
+ - Decrease on failed verification or detected misinformation
1004
+ - Weight recent verifications more heavily
1005
+ - Domain-specific trust (finance sources vs general sources)
1006
+
1007
+ **Interface**:
1008
+ ```python
1009
+ def get_trust_score(source_id: str) -> float
1010
+ def update_trust(source_id: str, verification_result: bool) -> None
1011
+ def list_trusted_sources(min_score: float = 0.7) -> List[SourceTrust]
1012
+ def list_untrusted_sources(max_score: float = 0.3) -> List[SourceTrust]
1013
+ def get_trust_stats() -> Dict[str, Any]
1014
+ ```
1015
+
1016
+ #### Freshness Management
1017
+
1018
+ **Purpose**: Track information recency and identify stale knowledge
1019
+
1020
+ **Freshness Score Schema**:
1021
+ ```python
1022
+ class FreshnessScore(BaseModel):
1023
+ item_id: str
1024
+ freshness_score: float # 0.0 - 1.0
1025
+ created_at: str
1026
+ last_updated: str
1027
+ last_verified: str
1028
+ update_frequency: str # "hourly", "daily", "weekly", "static"
1029
+ domain_pack: Optional[str]
1030
+ metadata: Dict[str, Any]
1031
+ ```
1032
+
1033
+ **Freshness Rules**:
1034
+ - News: Degrades rapidly (hourly)
1035
+ - Market data: Degrades moderately (daily)
1036
+ - Policy info: Degrades slowly (weekly)
1037
+ - Reference data: Static (no degradation)
1038
+ - Recommend refresh when score < 0.5
1039
+
1040
+ **Interface**:
1041
+ ```python
1042
+ def calculate_freshness(item: KnowledgeItem) -> float
1043
+ def update_freshness(item_id: str) -> None
1044
+ def get_stale_items(max_score: float = 0.5) -> List[str]
1045
+ def recommend_refresh() -> List[str]
1046
+ def get_freshness_stats() -> Dict[str, Any]
1047
+ ```
1048
+
1049
+ #### Learning Scheduler
1050
+
1051
+ **Purpose**: Lightweight background task scheduler with safeguards
1052
+
1053
+ **Scheduler Rules**:
1054
+ - Run only when system is idle
1055
+ - Max one background job at a time
1056
+ - Small batch sizes (5-10 items per run)
1057
+ - Stop on provider errors or rate limits
1058
+ - Skip if battery low (<20%)
1059
+ - Skip if CPU usage high (>70%)
1060
+ - Respect API rate limits
1061
+
1062
+ **Scheduled Tasks**:
1063
+ - Knowledge ingestion (every 6 hours)
1064
+ - Expired knowledge cleanup (daily)
1065
+ - Trust score updates (after each case)
1066
+ - Freshness score updates (hourly)
1067
+ - Pattern detection (daily)
1068
+ - Skill distillation (weekly)
1069
+ - Prompt optimization (weekly)
1070
+
1071
+ **Interface**:
1072
+ ```python
1073
+ def schedule_task(task_name: str, interval: str, func: Callable) -> None
1074
+ def run_once(task_name: str) -> Dict[str, Any]
1075
+ def get_scheduler_status() -> Dict[str, Any]
1076
+ def pause_scheduler() -> None
1077
+ def resume_scheduler() -> None
1078
+ def is_system_idle() -> bool
1079
+ def is_battery_ok() -> bool
1080
+ ```
1081
+
1082
+ #### Learning Engine
1083
+
1084
+ **Purpose**: Core learning logic that coordinates all learning subsystems
1085
+
1086
+ **Responsibilities**:
1087
+ - Inspect past cases for patterns
1088
+ - Trigger knowledge ingestion
1089
+ - Trigger skill distillation
1090
+ - Recommend prompt upgrades
1091
+ - Update trust and freshness scores
1092
+ - Generate learning insights
1093
+
1094
+ **Interface**:
1095
+ ```python
1096
+ def run_learning_cycle() -> Dict[str, Any]:
1097
+ """Run one complete learning cycle."""
1098
+
1099
+ def analyze_cases(limit: int = 100) -> Dict[str, Any]:
1100
+ """Analyze recent cases for patterns."""
1101
+
1102
+ def generate_insights() -> Dict[str, Any]:
1103
+ """Generate learning insights and recommendations."""
1104
+
1105
+ def get_learning_status() -> Dict[str, Any]:
1106
+ """Get current learning system status."""
1107
+ ```
1108
+
1109
+ #### Integration with Existing Layers
1110
+
1111
+ **With Cases**:
1112
+ - Learn from every case execution
1113
+ - Store case learning metadata
1114
+ - Detect patterns across cases
1115
+
1116
+ **With Domain Packs**:
1117
+ - Domain-specific knowledge ingestion
1118
+ - Domain-specific trust scoring
1119
+ - Domain-specific freshness rules
1120
+
1121
+ **With Simulation**:
1122
+ - Learn from simulation outcomes
1123
+ - Store simulation insights
1124
+ - Improve simulation parameter selection
1125
+
1126
+ **With Prompts**:
1127
+ - Version all prompts
1128
+ - Test prompt variants
1129
+ - Promote better prompts
1130
+
1131
+ **With Providers**:
1132
+ - Track provider reliability
1133
+ - Learn optimal provider selection
1134
+ - Adapt to provider failures
1135
+
1136
+
1137
+ ## Data Models
1138
+
1139
+ ### Core Schemas
1140
+
1141
+ **UserTask**:
1142
+ ```python
1143
+ class UserTask(BaseModel):
1144
+ user_input: str
1145
+ ```
1146
+
1147
+ **RouteDecision**:
1148
+ ```python
1149
+ class RouteDecision(BaseModel):
1150
+ task_family: Literal["normal", "simulation"]
1151
+ domain_pack: str # "finance", "general", "policy", "custom"
1152
+ complexity: Literal["simple", "medium", "complex"]
1153
+ execution_mode: Literal["solo", "standard", "deep"]
1154
+ risk_level: str
1155
+ confidence: float = 0.0
1156
+ ```
1157
+
1158
+ **AgentOutput**:
1159
+ ```python
1160
+ class AgentOutput(BaseModel):
1161
+ agent: str
1162
+ summary: str
1163
+ details: Dict[str, Any] = Field(default_factory=dict)
1164
+ confidence: float = 0.0
1165
+ timestamp: str
1166
+ ```
1167
+
1168
+ **CaseRecord**:
1169
+ ```python
1170
+ class CaseRecord(BaseModel):
1171
+ case_id: str
1172
+ user_input: str
1173
+ route: RouteDecision
1174
+ outputs: List[AgentOutput]
1175
+ final_answer: str
1176
+ simulation_id: Optional[str] = None
1177
+ created_at: str
1178
+ updated_at: str
1179
+ ```
1180
+
1181
+ **SimulationRequest**:
1182
+ ```python
1183
+ class SimulationRunRequest(BaseModel):
1184
+ title: str
1185
+ seed_text: str
1186
+ prediction_goal: str
1187
+ mode: str = "standard"
1188
+ metadata: Dict[str, Any] = Field(default_factory=dict)
1189
+ ```
1190
+
1191
+ **SimulationRecord**:
1192
+ ```python
1193
+ class SimulationRecord(BaseModel):
1194
+ simulation_id: str
1195
+ status: Literal["submitted", "running", "completed", "failed"]
1196
+ title: str
1197
+ prediction_goal: str
1198
+ remote_payload: Dict[str, Any]
1199
+ report: Optional[Dict[str, Any]] = None
1200
+ case_id: Optional[str] = None
1201
+ created_at: str
1202
+ updated_at: str
1203
+ ```
1204
+
1205
+ ### Domain Pack Schemas
1206
+
1207
+ **EntityExtraction**:
1208
+ ```python
1209
+ class EntityExtraction(BaseModel):
1210
+ entities: List[str]
1211
+ tickers: List[str]
1212
+ companies: List[str]
1213
+ people: List[str]
1214
+ locations: List[str]
1215
+ confidence: float
1216
+ ```
1217
+
1218
+ **CredibilityScore**:
1219
+ ```python
1220
+ class CredibilityScore(BaseModel):
1221
+ source: str
1222
+ score: float # 0.0 - 1.0
1223
+ factors: Dict[str, Any]
1224
+ warnings: List[str]
1225
+ ```
1226
+
1227
+ **MarketQuote**:
1228
+ ```python
1229
+ class MarketQuote(BaseModel):
1230
+ symbol: str
1231
+ price: float
1232
+ change: float
1233
+ change_percent: float
1234
+ volume: int
1235
+ timestamp: str
1236
+ ```
1237
+
1238
+ ### Learning Layer Schemas
1239
+
1240
+ **KnowledgeItem**:
1241
+ ```python
1242
+ class KnowledgeItem(BaseModel):
1243
+ id: str
1244
+ title: str
1245
+ summary: str # 2-4KB compressed summary
1246
+ entities: List[str]
1247
+ claims: List[str]
1248
+ source_url: str
1249
+ source_type: Literal["news", "api", "search", "webpage"]
1250
+ trust_score: float # 0.0 - 1.0
1251
+ freshness_score: float # 0.0 - 1.0
1252
+ domain_pack: Optional[str]
1253
+ created_at: str
1254
+ expires_at: str
1255
+ metadata: Dict[str, Any] = Field(default_factory=dict)
1256
+ ```
1257
+
1258
+ **CaseLearning**:
1259
+ ```python
1260
+ class CaseLearning(BaseModel):
1261
+ case_id: str
1262
+ route_effectiveness: float
1263
+ prompt_performance: Dict[str, float]
1264
+ provider_reliability: Dict[str, bool]
1265
+ source_usefulness: Dict[str, float]
1266
+ pattern_detected: Optional[str]
1267
+ corrections_made: List[str]
1268
+ execution_time: float
1269
+ created_at: str
1270
+ ```
1271
+
1272
+ **PromptVersion**:
1273
+ ```python
1274
+ class PromptVersion(BaseModel):
1275
+ name: str
1276
+ version: str
1277
+ content: str
1278
+ status: Literal["active", "experimental", "archived"]
1279
+ win_rate: float
1280
+ test_count: int
1281
+ last_tested: str
1282
+ created_at: str
1283
+ metadata: Dict[str, Any] = Field(default_factory=dict)
1284
+ ```
1285
+
1286
+ **Skill**:
1287
+ ```python
1288
+ class Skill(BaseModel):
1289
+ name: str
1290
+ description: str
1291
+ trigger_patterns: List[str]
1292
+ recommended_agents: List[str]
1293
+ preferred_sources: List[str]
1294
+ prompt_overrides: Dict[str, str]
1295
+ success_rate: float
1296
+ usage_count: int
1297
+ created_at: str
1298
+ last_used: str
1299
+ metadata: Dict[str, Any] = Field(default_factory=dict)
1300
+ ```
1301
+
1302
+ **SourceTrust**:
1303
+ ```python
1304
+ class SourceTrust(BaseModel):
1305
+ source_id: str
1306
+ source_type: Literal["news", "api", "website"]
1307
+ trust_score: float # 0.0 - 1.0
1308
+ verification_count: int
1309
+ success_count: int
1310
+ failure_count: int
1311
+ last_verified: str
1312
+ domain_pack: Optional[str]
1313
+ metadata: Dict[str, Any] = Field(default_factory=dict)
1314
+ ```
1315
+
1316
+ **FreshnessScore**:
1317
+ ```python
1318
+ class FreshnessScore(BaseModel):
1319
+ item_id: str
1320
+ freshness_score: float # 0.0 - 1.0
1321
+ created_at: str
1322
+ last_updated: str
1323
+ last_verified: str
1324
+ update_frequency: Literal["hourly", "daily", "weekly", "static"]
1325
+ domain_pack: Optional[str]
1326
+ metadata: Dict[str, Any] = Field(default_factory=dict)
1327
+ ```
1328
+
1329
+
1330
+ ## Service Layer Organization
1331
+
1332
+ ### External Sources Service
1333
+
1334
+ **Purpose**: Unified interface for external API integrations
1335
+
1336
+ **Current Implementation**: `backend/app/services/external_sources.py`
1337
+
1338
+ **Capabilities**:
1339
+ - URL extraction and content reading (Jina Reader)
1340
+ - Web search (Tavily)
1341
+ - News search (NewsAPI)
1342
+ - Market quotes (Alpha Vantage)
1343
+ - Ticker extraction from text
1344
+
1345
+ **Enhancement Strategy**:
1346
+ - Consolidate with impact_ai Alpha Vantage and NewsAPI clients
1347
+ - Add connection pooling
1348
+ - Add request timeouts
1349
+ - Add caching for market quotes (5 minute TTL)
1350
+ - Add rate limiting
1351
+
1352
+ ### Agent Registry Service
1353
+
1354
+ **Purpose**: Centralized agent registration and discovery
1355
+
1356
+ **Current Implementation**: `backend/app/services/agent_registry.py`
1357
+
1358
+ **Interface**:
1359
+ ```python
1360
+ def list_agents() -> List[Dict[str, Any]]
1361
+ def get_agent(agent_name: str) -> Optional[Dict[str, Any]]
1362
+ def run_single_agent(agent: str, user_input: str, **context) -> Dict[str, Any]
1363
+ ```
1364
+
1365
+ **Enhancement Strategy**:
1366
+ - Add agent capability metadata
1367
+ - Add agent dependency tracking
1368
+ - Add agent health checks
1369
+
1370
+ ### Case Store Service
1371
+
1372
+ **Purpose**: Local persistence for case execution records
1373
+
1374
+ **Current Implementation**: `backend/app/services/case_store.py`
1375
+
1376
+ **Interface**:
1377
+ ```python
1378
+ def save_case(case_id: str, payload: Dict[str, Any]) -> None
1379
+ def get_case(case_id: str) -> Optional[Dict[str, Any]]
1380
+ def list_cases(limit: Optional[int] = None) -> List[Dict[str, Any]]
1381
+ def delete_case(case_id: str) -> bool
1382
+ def memory_stats() -> Dict[str, Any]
1383
+ ```
1384
+
1385
+ **Enhancement Strategy**:
1386
+ - Add case search by user_input
1387
+ - Add case filtering by route parameters
1388
+ - Add case export functionality
1389
+
1390
+ ### Simulation Store Service
1391
+
1392
+ **Purpose**: Local persistence for simulation metadata
1393
+
1394
+ **Current Implementation**: `backend/app/services/simulation_store.py`
1395
+
1396
+ **Interface**:
1397
+ ```python
1398
+ def save_simulation(simulation_id: str, record: Dict[str, Any]) -> None
1399
+ def get_simulation(simulation_id: str) -> Optional[Dict[str, Any]]
1400
+ def list_simulations(limit: Optional[int] = None) -> List[Dict[str, Any]]
1401
+ ```
1402
+
1403
+ ### Prompt Store Service
1404
+
1405
+ **Purpose**: Dynamic prompt management
1406
+
1407
+ **Current Implementation**: `backend/app/services/prompt_store.py`
1408
+
1409
+ **Interface**:
1410
+ ```python
1411
+ def list_prompts() -> List[str]
1412
+ def get_prompt(name: str) -> Optional[Dict[str, Any]]
1413
+ def update_prompt(name: str, content: str) -> Dict[str, Any]
1414
+ ```
1415
+
1416
+ **Enhancement Strategy**:
1417
+ - Add prompt versioning
1418
+ - Add prompt validation
1419
+ - Add domain-specific prompt templates
1420
+
1421
+ ### Health Service
1422
+
1423
+ **Purpose**: System health monitoring
1424
+
1425
+ **Current Implementation**: `backend/app/services/health_service.py`
1426
+
1427
+ **Interface**:
1428
+ ```python
1429
+ def deep_health() -> Dict[str, Any]:
1430
+ """
1431
+ Comprehensive health check.
1432
+
1433
+ Returns:
1434
+ {
1435
+ "status": "ok|degraded|error",
1436
+ "version": str,
1437
+ "checks": {
1438
+ "providers": {...},
1439
+ "external_apis": {...},
1440
+ "mirofish": {...},
1441
+ "storage": {...},
1442
+ "domain_packs": {...}
1443
+ }
1444
+ }
1445
+ """
1446
+ ```
1447
+
1448
+
1449
+ ## Frontend Architecture
1450
+
1451
+ ### Technology Stack
1452
+
1453
+ - **Framework**: Next.js 14+ with App Router
1454
+ - **UI Library**: React 18+
1455
+ - **Styling**: Tailwind CSS
1456
+ - **State Management**: React hooks + Context API
1457
+ - **HTTP Client**: fetch API with error handling
1458
+ - **Type Safety**: TypeScript
1459
+
1460
+ ### Page Structure
1461
+
1462
+ ```
1463
+ frontend/src/app/
1464
+ ├── layout.tsx # Root layout with navigation
1465
+ ├── page.tsx # Main dashboard (landing)
1466
+ ├── analyze/
1467
+ │ └── page.tsx # Analyze mode interface
1468
+ ├── cases/
1469
+ │ ├── page.tsx # Case history list
1470
+ │ └── [id]/
1471
+ │ └── page.tsx # Case detail view
1472
+ ├── simulation/
1473
+ │ ├── page.tsx # Simulation interface
1474
+ │ └── [id]/
1475
+ │ └── page.tsx # Simulation detail and chat
1476
+ ├── prompts/
1477
+ │ └── page.tsx # Prompt lab
1478
+ └── config/
1479
+ └── page.tsx # System configuration view
1480
+ ```
1481
+
1482
+ ### Component Architecture
1483
+
1484
+ ```
1485
+ frontend/src/components/
1486
+ ├── layout/
1487
+ │ ├── Header.tsx
1488
+ │ ├── Navigation.tsx
1489
+ │ └── Footer.tsx
1490
+ ├── analyze/
1491
+ │ ├── TaskInput.tsx
1492
+ │ ├── ModeSelector.tsx
1493
+ │ ├── ResultViewer.tsx
1494
+ │ └── AgentOutputPanel.tsx
1495
+ ├── cases/
1496
+ │ ├── CaseList.tsx
1497
+ │ ├── CaseCard.tsx
1498
+ │ └── CaseDetail.tsx
1499
+ ├── simulation/
1500
+ │ ├── SimulationForm.tsx
1501
+ │ ├── SimulationStatus.tsx
1502
+ │ ├── SimulationReport.tsx
1503
+ │ └── SimulationChat.tsx
1504
+ ├── prompts/
1505
+ │ ├── PromptList.tsx
1506
+ │ └── PromptEditor.tsx
1507
+ └── common/
1508
+ ├── Badge.tsx
1509
+ ├── Card.tsx
1510
+ ├── LoadingSpinner.tsx
1511
+ └── ErrorMessage.tsx
1512
+ ```
1513
+
1514
+ ### API Client
1515
+
1516
+ ```typescript
1517
+ // frontend/src/lib/api.ts
1518
+ export class MiroOrgClient {
1519
+ private baseUrl: string;
1520
+
1521
+ async runTask(input: string): Promise<CaseResult>
1522
+ async getCase(caseId: string): Promise<Case>
1523
+ async listCases(limit?: number): Promise<Case[]>
1524
+ async deleteCase(caseId: string): Promise<void>
1525
+
1526
+ async runSimulation(request: SimulationRequest): Promise<Simulation>
1527
+ async getSimulation(id: string): Promise<Simulation>
1528
+ async getSimulationReport(id: string): Promise<SimulationReport>
1529
+ async chatWithSimulation(id: string, message: string): Promise<ChatResponse>
1530
+
1531
+ async listPrompts(): Promise<string[]>
1532
+ async getPrompt(name: string): Promise<Prompt>
1533
+ async updatePrompt(name: string, content: string): Promise<Prompt>
1534
+
1535
+ async getHealth(): Promise<HealthStatus>
1536
+ async getConfig(): Promise<ConfigStatus>
1537
+ }
1538
+ ```
1539
+
1540
+ ### Design System
1541
+
1542
+ **Color Palette** (Dark Theme):
1543
+ - Background: `#0a0a0a`
1544
+ - Surface: `#1a1a1a`
1545
+ - Border: `#2a2a2a`
1546
+ - Primary: `#3b82f6` (blue)
1547
+ - Success: `#10b981` (green)
1548
+ - Warning: `#f59e0b` (amber)
1549
+ - Error: `#ef4444` (red)
1550
+ - Text Primary: `#f9fafb`
1551
+ - Text Secondary: `#9ca3af`
1552
+
1553
+ **Typography**:
1554
+ - Font Family: Inter, system-ui, sans-serif
1555
+ - Headings: font-semibold
1556
+ - Body: font-normal
1557
+ - Code: font-mono
1558
+
1559
+ **Spacing**: Tailwind default scale (4px base unit)
1560
+
1561
+ **Animations**:
1562
+ - Fade in: 200ms ease-in
1563
+ - Slide in: 300ms ease-out
1564
+ - Hover transitions: 150ms ease-in-out
1565
+
1566
+
1567
+ ## Correctness Properties
1568
+
1569
+ A property is a characteristic or behavior that should hold true across all valid executions of a system—essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.
1570
+
1571
+ ### Property Reflection
1572
+
1573
+ After analyzing all acceptance criteria, I identified the following redundancies and consolidations:
1574
+
1575
+ **Redundancy Group 1: Configuration from Environment**
1576
+ - Requirements 1.8, 6.7 both test that configuration comes from environment variables
1577
+ - Consolidated into Property 1
1578
+
1579
+ **Redundancy Group 2: Directory Storage Locations**
1580
+ - Requirements 10.9, 10.10, 10.11 all test file storage locations
1581
+ - Consolidated into Property 8
1582
+
1583
+ **Redundancy Group 3: Logging Requirements**
1584
+ - Requirements 6.6, 9.4, 9.5, 9.6, 9.7 all test logging behavior
1585
+ - Consolidated into Property 11
1586
+
1587
+ **Redundancy Group 4: Switchboard Complexity Mapping**
1588
+ - Requirements 4.2, 4.3, 4.4 all test complexity-to-execution-mode mapping
1589
+ - Consolidated into Property 3 (single comprehensive property)
1590
+
1591
+ **Redundancy Group 5: Case Storage Fields**
1592
+ - Requirements 10.2, 10.7, 10.8 all test case record structure
1593
+ - Consolidated into Property 7
1594
+
1595
+ **Redundancy Group 6: External API Integration**
1596
+ - Requirements 9.1, 9.9, 9.10 all test external API client behavior
1597
+ - Consolidated into Property 13
1598
+
1599
+ **Redundancy Group 7: Error Handling**
1600
+ - Requirements 9.3, 9.8, 9.10 all test error response structure
1601
+ - Consolidated into Property 14
1602
+
1603
+ After consolidation, 15 core properties remain that provide unique validation value.
1604
+
1605
+ ### Property 1: Configuration Environment Isolation
1606
+
1607
+ For any configuration value (API keys, provider settings, feature flags), the system should load it from environment variables, not from hardcoded values in source code.
1608
+
1609
+ **Validates: Requirements 1.8, 6.7**
1610
+
1611
+ ### Property 2: Switchboard Four-Dimensional Classification
1612
+
1613
+ For any user input, the Switchboard routing decision should contain exactly four dimensions: task_family, domain_pack, complexity, and execution_mode.
1614
+
1615
+ **Validates: Requirements 4.1**
1616
+
1617
+ ### Property 3: Complexity-to-Execution-Mode Mapping
1618
+
1619
+ For any user input, the Switchboard should map complexity to execution_mode according to the rule: simple→solo, medium→standard, complex→deep.
1620
+
1621
+ **Validates: Requirements 4.2, 4.3, 4.4**
1622
+
1623
+ ### Property 4: Simulation Keyword Triggering
1624
+
1625
+ For any user input containing simulation trigger keywords (configurable via environment), the Switchboard should classify task_family as "simulation".
1626
+
1627
+ **Validates: Requirements 4.5, 4.6**
1628
+
1629
+ ### Property 5: Provider Fallback Behavior
1630
+
1631
+ For any model call, if the primary provider fails, the system should automatically attempt the fallback provider before raising an error.
1632
+
1633
+ **Validates: Requirements 6.5**
1634
+
1635
+ ### Property 6: Case Persistence Round Trip
1636
+
1637
+ For any case execution, saving the case and then retrieving it by case_id should return an equivalent case record with all required fields.
1638
+
1639
+ **Validates: Requirements 10.1, 10.3**
1640
+
1641
+ ### Property 7: Case Record Structure Completeness
1642
+
1643
+ For any stored case, the JSON record should contain all required fields: case_id, user_input, route (with four dimensions), outputs (list of agent outputs), final_answer, and timestamps.
1644
+
1645
+ **Validates: Requirements 10.2, 10.7, 10.8**
1646
+
1647
+ ### Property 8: Data Directory Organization
1648
+
1649
+ For any data persistence operation (cases, simulations, logs), the system should store files in the correct directory: cases in memory/, simulations in simulations/, logs in logs/.
1650
+
1651
+ **Validates: Requirements 10.9, 10.10, 10.11**
1652
+
1653
+ ### Property 9: Directory Auto-Creation
1654
+
1655
+ For any missing data directory (memory, simulations, logs), the system should automatically create it on startup or first use.
1656
+
1657
+ **Validates: Requirements 10.12**
1658
+
1659
+ ### Property 10: MiroFish Adapter Isolation
1660
+
1661
+ For any simulation operation, the system should route requests through the mirofish_client adapter, never allowing direct MiroFish API calls from other components.
1662
+
1663
+ **Validates: Requirements 1.3, 3.4**
1664
+
1665
+ ### Property 11: Comprehensive Logging
1666
+
1667
+ For any agent execution, provider call, external API call, or simulation request, the system should create a log entry with timestamp and relevant context.
1668
+
1669
+ **Validates: Requirements 6.6, 9.4, 9.6, 9.7**
1670
+
1671
+ ### Property 12: Schema Validation
1672
+
1673
+ For any API request, if the request body does not match the expected Pydantic schema, the system should return a 422 validation error with details about the validation failure.
1674
+
1675
+ **Validates: Requirements 9.1, 9.2**
1676
+
1677
+ ### Property 13: External API Client Patterns
1678
+
1679
+ For any external API client, the system should implement connection pooling, request timeouts, and error handling consistently.
1680
+
1681
+ **Validates: Requirements 9.1, 9.9, 9.10**
1682
+
1683
+ ### Property 14: Error Response Sanitization
1684
+
1685
+ For any error response, the system should return a structured error with appropriate HTTP status code and descriptive message, without exposing internal implementation details or raw exceptions.
1686
+
1687
+ **Validates: Requirements 9.3, 9.8, 9.10**
1688
+
1689
+ ### Property 15: Domain Pack Extensibility
1690
+
1691
+ For any new domain pack registration, the system should support it without requiring modifications to the agent organization layer (Switchboard, Research, Planner, Verifier, Synthesizer).
1692
+
1693
+ **Validates: Requirements 2.5, 2.7**
1694
+
1695
+
1696
+ ## Error Handling
1697
+
1698
+ ### Error Categories
1699
+
1700
+ 1. **Validation Errors** (HTTP 422)
1701
+ - Invalid request schema
1702
+ - Missing required fields
1703
+ - Type mismatches
1704
+
1705
+ 2. **Client Errors** (HTTP 400)
1706
+ - Feature disabled (e.g., MiroFish disabled)
1707
+ - Invalid parameters
1708
+ - Business logic violations
1709
+
1710
+ 3. **Not Found Errors** (HTTP 404)
1711
+ - Case not found
1712
+ - Agent not found
1713
+ - Prompt not found
1714
+ - Simulation not found
1715
+
1716
+ 4. **External Service Errors** (HTTP 502)
1717
+ - Provider failures (OpenRouter, Ollama)
1718
+ - External API failures (Tavily, NewsAPI, Alpha Vantage)
1719
+ - MiroFish connection failures
1720
+
1721
+ 5. **Internal Errors** (HTTP 500)
1722
+ - Unexpected exceptions
1723
+ - Storage failures
1724
+ - System errors
1725
+
1726
+ ### Error Response Schema
1727
+
1728
+ ```python
1729
+ class ErrorResponse(BaseModel):
1730
+ error: str
1731
+ detail: str
1732
+ status_code: int
1733
+ timestamp: str
1734
+ ```
1735
+
1736
+ ### Error Handling Patterns
1737
+
1738
+ **Provider Fallback**:
1739
+ ```python
1740
+ try:
1741
+ return call_primary_provider(prompt)
1742
+ except ProviderError as e:
1743
+ logger.warning(f"Primary provider failed: {e}")
1744
+ try:
1745
+ return call_fallback_provider(prompt)
1746
+ except ProviderError as fallback_error:
1747
+ logger.error(f"Fallback provider failed: {fallback_error}")
1748
+ raise LLMProviderError("All providers failed")
1749
+ ```
1750
+
1751
+ **External API Graceful Degradation**:
1752
+ ```python
1753
+ try:
1754
+ results = tavily_search(query)
1755
+ except Exception as e:
1756
+ logger.warning(f"Tavily search failed: {e}")
1757
+ results = [] # Continue with empty results
1758
+ ```
1759
+
1760
+ **MiroFish Error Handling**:
1761
+ ```python
1762
+ if not MIROFISH_ENABLED:
1763
+ raise HTTPException(
1764
+ status_code=400,
1765
+ detail="MiroFish integration is disabled"
1766
+ )
1767
+
1768
+ try:
1769
+ result = mirofish_client.run_simulation(payload)
1770
+ except MiroFishError as e:
1771
+ raise HTTPException(
1772
+ status_code=502,
1773
+ detail=f"MiroFish service error: {str(e)}"
1774
+ )
1775
+ ```
1776
+
1777
+ ### Logging Strategy
1778
+
1779
+ **Log Levels**:
1780
+ - **DEBUG**: Detailed execution traces, variable values
1781
+ - **INFO**: Normal operations, agent executions, case saves
1782
+ - **WARNING**: Degraded functionality, fallback usage, missing optional features
1783
+ - **ERROR**: Failures that prevent operation completion
1784
+ - **CRITICAL**: System-wide failures
1785
+
1786
+ **Log Format**:
1787
+ ```
1788
+ [TIMESTAMP] [LEVEL] [MODULE] [CASE_ID] MESSAGE
1789
+ ```
1790
+
1791
+ **Log Rotation**:
1792
+ - Daily rotation
1793
+ - Keep 30 days of logs
1794
+ - Compress old logs
1795
+ - Max log file size: 100MB
1796
+
1797
+
1798
+ ## Testing Strategy
1799
+
1800
+ ### Dual Testing Approach
1801
+
1802
+ The system requires both unit tests and property-based tests for comprehensive coverage:
1803
+
1804
+ - **Unit tests**: Verify specific examples, edge cases, and error conditions
1805
+ - **Property tests**: Verify universal properties across all inputs
1806
+
1807
+ Both approaches are complementary and necessary. Unit tests catch concrete bugs in specific scenarios, while property tests verify general correctness across randomized inputs.
1808
+
1809
+ ### Unit Testing
1810
+
1811
+ **Focus Areas**:
1812
+ 1. Specific examples that demonstrate correct behavior
1813
+ 2. Integration points between components
1814
+ 3. Edge cases (empty inputs, missing fields, null values)
1815
+ 4. Error conditions (provider failures, API timeouts, invalid schemas)
1816
+
1817
+ **Unit Test Balance**:
1818
+ - Avoid writing too many unit tests for input variations
1819
+ - Property-based tests handle comprehensive input coverage
1820
+ - Unit tests should focus on specific scenarios and integration points
1821
+
1822
+ **Example Unit Tests**:
1823
+ ```python
1824
+ def test_switchboard_simple_query():
1825
+ """Test that short queries route to solo mode."""
1826
+ route = decide_route("Hello")
1827
+ assert route["complexity"] == "simple"
1828
+ assert route["execution_mode"] == "solo"
1829
+
1830
+ def test_case_storage_missing_directory():
1831
+ """Test that missing memory directory is created."""
1832
+ # Remove directory if exists
1833
+ # Save case
1834
+ # Assert directory was created
1835
+ # Assert case file exists
1836
+
1837
+ def test_provider_fallback_on_primary_failure():
1838
+ """Test that system falls back to secondary provider."""
1839
+ # Mock primary provider to fail
1840
+ # Call model
1841
+ # Assert fallback provider was called
1842
+ # Assert result is returned
1843
+ ```
1844
+
1845
+ ### Property-Based Testing
1846
+
1847
+ **Library**: pytest-hypothesis (Python), fast-check (TypeScript)
1848
+
1849
+ **Configuration**: Minimum 100 iterations per property test
1850
+
1851
+ **Property Test Structure**:
1852
+ ```python
1853
+ from hypothesis import given, strategies as st
1854
+
1855
+ @given(st.text(min_size=1, max_size=1000))
1856
+ def test_property_1_config_from_environment(user_input):
1857
+ """
1858
+ Property 1: Configuration Environment Isolation
1859
+
1860
+ For any configuration value, the system should load it from
1861
+ environment variables, not from hardcoded values.
1862
+
1863
+ Feature: ai-financial-intelligence-system, Property 1
1864
+ """
1865
+ # Test implementation
1866
+ ```
1867
+
1868
+ **Property Test Tags**: Each property test must include a comment tag:
1869
+ ```
1870
+ Feature: {feature_name}, Property {number}: {property_text}
1871
+ ```
1872
+
1873
+ ### Property Test Implementation Plan
1874
+
1875
+ **Property 1: Configuration Environment Isolation**
1876
+ - Generate random config keys
1877
+ - Verify values come from os.environ
1878
+ - Verify no hardcoded API keys in code
1879
+
1880
+ **Property 2: Switchboard Four-Dimensional Classification**
1881
+ - Generate random user inputs
1882
+ - Verify routing decision has all four dimensions
1883
+ - Verify all dimensions have valid values
1884
+
1885
+ **Property 3: Complexity-to-Execution-Mode Mapping**
1886
+ - Generate inputs of varying lengths
1887
+ - Verify complexity classification
1888
+ - Verify execution_mode matches complexity
1889
+
1890
+ **Property 4: Simulation Keyword Triggering**
1891
+ - Generate inputs with/without trigger keywords
1892
+ - Verify task_family classification
1893
+ - Verify keyword detection is case-insensitive
1894
+
1895
+ **Property 5: Provider Fallback Behavior**
1896
+ - Generate random prompts
1897
+ - Mock primary provider failures
1898
+ - Verify fallback is attempted
1899
+ - Verify result is returned or error is raised
1900
+
1901
+ **Property 6: Case Persistence Round Trip**
1902
+ - Generate random case data
1903
+ - Save case
1904
+ - Retrieve case
1905
+ - Verify equivalence
1906
+
1907
+ **Property 7: Case Record Structure Completeness**
1908
+ - Generate random case executions
1909
+ - Save cases
1910
+ - Verify all required fields present
1911
+ - Verify field types match schema
1912
+
1913
+ **Property 8: Data Directory Organization**
1914
+ - Generate random case/simulation/log data
1915
+ - Save to storage
1916
+ - Verify files in correct directories
1917
+
1918
+ **Property 9: Directory Auto-Creation**
1919
+ - Remove directories
1920
+ - Trigger storage operations
1921
+ - Verify directories created
1922
+
1923
+ **Property 10: MiroFish Adapter Isolation**
1924
+ - Scan codebase for direct MiroFish URLs
1925
+ - Verify all calls go through adapter
1926
+ - Verify frontend has no MiroFish URLs
1927
+
1928
+ **Property 11: Comprehensive Logging**
1929
+ - Generate random operations
1930
+ - Execute operations
1931
+ - Verify log entries created
1932
+ - Verify log entries have required fields
1933
+
1934
+ **Property 12: Schema Validation**
1935
+ - Generate invalid request bodies
1936
+ - Submit to endpoints
1937
+ - Verify 422 status code
1938
+ - Verify validation error details
1939
+
1940
+ **Property 13: External API Client Patterns**
1941
+ - Inspect all external API clients
1942
+ - Verify timeout configuration
1943
+ - Verify connection pooling
1944
+ - Verify error handling
1945
+
1946
+ **Property 14: Error Response Sanitization**
1947
+ - Generate various error conditions
1948
+ - Trigger errors
1949
+ - Verify error response structure
1950
+ - Verify no internal details exposed
1951
+
1952
+ **Property 15: Domain Pack Extensibility**
1953
+ - Create mock domain pack
1954
+ - Register domain pack
1955
+ - Verify agents can use pack
1956
+ - Verify no agent code modifications needed
1957
+
1958
+ ### Integration Testing
1959
+
1960
+ **Test Scenarios**:
1961
+ 1. End-to-end case execution (user input → final answer)
1962
+ 2. Simulation workflow (submission → status → report → chat)
1963
+ 3. Provider fallback in real execution
1964
+ 4. Domain pack enhancement in research agent
1965
+ 5. Case storage and retrieval
1966
+ 6. Prompt management workflow
1967
+
1968
+ ### Test Coverage Goals
1969
+
1970
+ - **Critical paths**: 90%+ coverage
1971
+ - **Service layer**: 80%+ coverage
1972
+ - **Agent logic**: 70%+ coverage
1973
+ - **Overall**: 70%+ coverage
1974
+
1975
+ ### CI/CD Pipeline
1976
+
1977
+ 1. Linting (ruff, black, isort)
1978
+ 2. Type checking (mypy)
1979
+ 3. Unit tests
1980
+ 4. Property tests
1981
+ 5. Integration tests
1982
+ 6. Coverage report
1983
+
1984
+
1985
+ ## Implementation Phases
1986
+
1987
+ ### Phase 1: Backend Consolidation and Provider Enhancement
1988
+
1989
+ **Goal**: Strengthen core platform and provider abstraction
1990
+
1991
+ **Tasks**:
1992
+ 1. Add OpenAI provider support to `backend/app/agents/_model.py`
1993
+ 2. Enhance provider fallback logging
1994
+ 3. Add provider health checks to `backend/app/services/health_service.py`
1995
+ 4. Add configuration validation on startup in `backend/app/config.py`
1996
+ 5. Add missing environment variables to `.env.example`
1997
+ 6. Update `backend/requirements.txt` with any new dependencies
1998
+
1999
+ **Files to Modify**:
2000
+ - `backend/app/agents/_model.py`
2001
+ - `backend/app/services/health_service.py`
2002
+ - `backend/app/config.py`
2003
+ - `backend/.env.example`
2004
+ - `backend/requirements.txt`
2005
+
2006
+ **Verification**:
2007
+ - All three providers (OpenRouter, Ollama, OpenAI) work
2008
+ - Fallback behavior logs correctly
2009
+ - Health check reports provider status
2010
+ - Missing config keys log warnings
2011
+
2012
+ ### Phase 2: Domain Pack Architecture
2013
+
2014
+ **Goal**: Create domain pack infrastructure and integrate finance pack
2015
+
2016
+ **Tasks**:
2017
+ 1. Create domain pack base architecture:
2018
+ - `backend/app/domain_packs/__init__.py`
2019
+ - `backend/app/domain_packs/base.py` (DomainPack abstract class)
2020
+ - `backend/app/domain_packs/registry.py` (DomainPackRegistry)
2021
+
2022
+ 2. Create finance domain pack structure:
2023
+ - `backend/app/domain_packs/finance/__init__.py`
2024
+ - `backend/app/domain_packs/finance/pack.py` (FinanceDomainPack class)
2025
+
2026
+ 3. Port impact_ai modules to finance pack:
2027
+ - `backend/app/domain_packs/finance/market_data.py` (from impact_ai alpha_vantage_client.py)
2028
+ - `backend/app/domain_packs/finance/news.py` (from impact_ai news_api.py)
2029
+ - `backend/app/domain_packs/finance/entity_resolver.py`
2030
+ - `backend/app/domain_packs/finance/ticker_resolver.py`
2031
+ - `backend/app/domain_packs/finance/source_checker.py`
2032
+ - `backend/app/domain_packs/finance/rumor_detector.py`
2033
+ - `backend/app/domain_packs/finance/scam_detector.py`
2034
+ - `backend/app/domain_packs/finance/stance_detector.py`
2035
+ - `backend/app/domain_packs/finance/event_analyzer.py`
2036
+ - `backend/app/domain_packs/finance/prediction.py`
2037
+
2038
+ 4. Consolidate external API clients:
2039
+ - Merge Alpha Vantage logic into `backend/app/services/external_sources.py`
2040
+ - Merge NewsAPI logic into `backend/app/services/external_sources.py`
2041
+ - Remove duplicates
2042
+
2043
+ 5. Register finance pack in global registry
2044
+
2045
+ **Files to Create**:
2046
+ - `backend/app/domain_packs/__init__.py`
2047
+ - `backend/app/domain_packs/base.py`
2048
+ - `backend/app/domain_packs/registry.py`
2049
+ - `backend/app/domain_packs/finance/__init__.py`
2050
+ - `backend/app/domain_packs/finance/pack.py`
2051
+ - `backend/app/domain_packs/finance/market_data.py`
2052
+ - `backend/app/domain_packs/finance/news.py`
2053
+ - `backend/app/domain_packs/finance/entity_resolver.py`
2054
+ - `backend/app/domain_packs/finance/ticker_resolver.py`
2055
+ - `backend/app/domain_packs/finance/source_checker.py`
2056
+ - `backend/app/domain_packs/finance/rumor_detector.py`
2057
+ - `backend/app/domain_packs/finance/scam_detector.py`
2058
+ - `backend/app/domain_packs/finance/stance_detector.py`
2059
+ - `backend/app/domain_packs/finance/event_analyzer.py`
2060
+ - `backend/app/domain_packs/finance/prediction.py`
2061
+
2062
+ **Files to Modify**:
2063
+ - `backend/app/services/external_sources.py`
2064
+ - `backend/app/config.py` (add finance pack config)
2065
+
2066
+ **Verification**:
2067
+ - Finance pack is registered
2068
+ - Finance pack capabilities are accessible
2069
+ - Domain detection works for finance keywords
2070
+ - External API clients are consolidated
2071
+
2072
+ ### Phase 3: Agent Enhancement with Domain Intelligence
2073
+
2074
+ **Goal**: Integrate domain pack capabilities into agents
2075
+
2076
+ **Tasks**:
2077
+ 1. Enhance Switchboard with domain detection:
2078
+ - Modify `backend/app/agents/switchboard.py`
2079
+ - Add domain_pack dimension to routing decision
2080
+ - Use domain registry for keyword detection
2081
+
2082
+ 2. Enhance Research Agent with domain capabilities:
2083
+ - Modify `backend/app/agents/research.py`
2084
+ - Call domain pack enhance_research() when domain detected
2085
+ - Add structured entity extraction
2086
+ - Update `backend/app/prompts/research.txt` with domain instructions
2087
+
2088
+ 3. Enhance Verifier Agent with domain capabilities:
2089
+ - Modify `backend/app/agents/verifier.py`
2090
+ - Call domain pack enhance_verification() when domain detected
2091
+ - Add structured credibility scoring
2092
+ - Update `backend/app/prompts/verifier.txt` with domain instructions
2093
+
2094
+ 4. Enhance Planner Agent:
2095
+ - Modify `backend/app/agents/planner.py`
2096
+ - Add simulation mode suggestion logic
2097
+ - Update `backend/app/prompts/planner.txt`
2098
+
2099
+ 5. Enhance Synthesizer Agent:
2100
+ - Modify `backend/app/agents/synthesizer.py`
2101
+ - Add uncertainty quantification
2102
+ - Add simulation recommendation logic
2103
+ - Update `backend/app/prompts/synthesizer.txt`
2104
+
2105
+ 6. Update graph execution:
2106
+ - Modify `backend/app/graph.py`
2107
+ - Pass domain pack context through pipeline
2108
+
2109
+ **Files to Modify**:
2110
+ - `backend/app/agents/switchboard.py`
2111
+ - `backend/app/agents/research.py`
2112
+ - `backend/app/agents/verifier.py`
2113
+ - `backend/app/agents/planner.py`
2114
+ - `backend/app/agents/synthesizer.py`
2115
+ - `backend/app/graph.py`
2116
+ - `backend/app/prompts/research.txt`
2117
+ - `backend/app/prompts/verifier.txt`
2118
+ - `backend/app/prompts/planner.txt`
2119
+ - `backend/app/prompts/synthesizer.txt`
2120
+
2121
+ **Verification**:
2122
+ - Switchboard detects finance domain
2123
+ - Research agent extracts entities and tickers
2124
+ - Verifier agent scores credibility
2125
+ - Agents suggest simulation mode when appropriate
2126
+ - Domain-enhanced execution produces better results
2127
+
2128
+ ### Phase 4: Simulation Integration Enhancement
2129
+
2130
+ **Goal**: Improve simulation workflow and case linking
2131
+
2132
+ **Tasks**:
2133
+ 1. Enhance simulation router:
2134
+ - Modify `backend/app/routers/simulation.py`
2135
+ - Add case_id linking
2136
+ - Improve error messages
2137
+
2138
+ 2. Enhance simulation store:
2139
+ - Modify `backend/app/services/simulation_store.py`
2140
+ - Add simulation search
2141
+ - Add simulation filtering
2142
+
2143
+ 3. Update case storage for simulation linking:
2144
+ - Modify `backend/app/services/case_store.py`
2145
+ - Add simulation_id field to case records
2146
+ - Add case-to-simulation lookup
2147
+
2148
+ 4. Add simulation workflow to graph:
2149
+ - Modify `backend/app/graph.py`
2150
+ - Add simulation handoff logic
2151
+ - Add simulation result synthesis
2152
+
2153
+ **Files to Modify**:
2154
+ - `backend/app/routers/simulation.py`
2155
+ - `backend/app/services/simulation_store.py`
2156
+ - `backend/app/services/case_store.py`
2157
+ - `backend/app/graph.py`
2158
+ - `backend/app/schemas.py` (add simulation-related schemas)
2159
+
2160
+ **Verification**:
2161
+ - Simulation requests create linked cases
2162
+ - Cases with simulations show simulation_id
2163
+ - Simulation results are synthesized into final answer
2164
+ - Simulation workflow is seamless
2165
+
2166
+ ### Phase 5: API Discovery Subsystem
2167
+
2168
+ **Goal**: Create API discovery infrastructure for future expansion
2169
+
2170
+ **Tasks**:
2171
+ 1. Create API discovery structure:
2172
+ - `backend/app/services/api_discovery/__init__.py`
2173
+ - `backend/app/services/api_discovery/catalog_loader.py`
2174
+ - `backend/app/services/api_discovery/classifier.py`
2175
+ - `backend/app/services/api_discovery/scorer.py`
2176
+ - `backend/app/services/api_discovery/metadata_store.py`
2177
+
2178
+ 2. Implement catalog loader:
2179
+ - Load public-apis JSON from GitHub or local cache
2180
+ - Parse API entries
2181
+
2182
+ 3. Implement classifier:
2183
+ - Classify APIs by category
2184
+ - Map to domain packs
2185
+
2186
+ 4. Implement scorer:
2187
+ - Score APIs by usefulness
2188
+ - Consider auth, HTTPS, CORS
2189
+
2190
+ 5. Add discovery endpoints (optional):
2191
+ - `GET /api-discovery/categories`
2192
+ - `GET /api-discovery/search?category=X`
2193
+ - `GET /api-discovery/top-scored`
2194
+
2195
+ **Files to Create**:
2196
+ - `backend/app/services/api_discovery/__init__.py`
2197
+ - `backend/app/services/api_discovery/catalog_loader.py`
2198
+ - `backend/app/services/api_discovery/classifier.py`
2199
+ - `backend/app/services/api_discovery/scorer.py`
2200
+ - `backend/app/services/api_discovery/metadata_store.py`
2201
+
2202
+ **Files to Modify** (optional):
2203
+ - `backend/app/main.py` (add discovery router)
2204
+
2205
+ **Verification**:
2206
+ - Catalog loads successfully
2207
+ - APIs are classified correctly
2208
+ - Scoring produces reasonable priorities
2209
+ - Discovery is available for future connector development
2210
+
2211
+
2212
+ ### Phase 6: Frontend Enhancement
2213
+
2214
+ **Goal**: Evolve frontend from demo to product dashboard
2215
+
2216
+ **Tasks**:
2217
+ 1. Create layout and navigation:
2218
+ - Modify `frontend/src/app/layout.tsx`
2219
+ - Create `frontend/src/components/layout/Header.tsx`
2220
+ - Create `frontend/src/components/layout/Navigation.tsx`
2221
+
2222
+ 2. Create main dashboard:
2223
+ - Modify `frontend/src/app/page.tsx`
2224
+ - Add quick stats, recent cases, system status
2225
+
2226
+ 3. Create Analyze page:
2227
+ - Create `frontend/src/app/analyze/page.tsx`
2228
+ - Create `frontend/src/components/analyze/TaskInput.tsx`
2229
+ - Create `frontend/src/components/analyze/ModeSelector.tsx`
2230
+ - Create `frontend/src/components/analyze/ResultViewer.tsx`
2231
+ - Create `frontend/src/components/analyze/AgentOutputPanel.tsx`
2232
+
2233
+ 4. Create Cases page:
2234
+ - Create `frontend/src/app/cases/page.tsx`
2235
+ - Create `frontend/src/app/cases/[id]/page.tsx`
2236
+ - Create `frontend/src/components/cases/CaseList.tsx`
2237
+ - Create `frontend/src/components/cases/CaseCard.tsx`
2238
+ - Create `frontend/src/components/cases/CaseDetail.tsx`
2239
+
2240
+ 5. Create Simulation page:
2241
+ - Create `frontend/src/app/simulation/page.tsx`
2242
+ - Create `frontend/src/app/simulation/[id]/page.tsx`
2243
+ - Create `frontend/src/components/simulation/SimulationForm.tsx`
2244
+ - Create `frontend/src/components/simulation/SimulationStatus.tsx`
2245
+ - Create `frontend/src/components/simulation/SimulationReport.tsx`
2246
+ - Create `frontend/src/components/simulation/SimulationChat.tsx`
2247
+
2248
+ 6. Create Prompt Lab page:
2249
+ - Create `frontend/src/app/prompts/page.tsx`
2250
+ - Create `frontend/src/components/prompts/PromptList.tsx`
2251
+ - Create `frontend/src/components/prompts/PromptEditor.tsx`
2252
+
2253
+ 7. Create Config page:
2254
+ - Create `frontend/src/app/config/page.tsx`
2255
+
2256
+ 8. Create API client:
2257
+ - Create `frontend/src/lib/api.ts` (MiroOrgClient class)
2258
+ - Create `frontend/src/lib/types.ts` (TypeScript types)
2259
+
2260
+ 9. Create common components:
2261
+ - Create `frontend/src/components/common/Badge.tsx`
2262
+ - Create `frontend/src/components/common/Card.tsx`
2263
+ - Create `frontend/src/components/common/LoadingSpinner.tsx`
2264
+ - Create `frontend/src/components/common/ErrorMessage.tsx`
2265
+
2266
+ 10. Update styling:
2267
+ - Modify `frontend/src/app/globals.css`
2268
+ - Implement dark theme
2269
+ - Add animations
2270
+
2271
+ **Files to Create**:
2272
+ - `frontend/src/components/layout/Header.tsx`
2273
+ - `frontend/src/components/layout/Navigation.tsx`
2274
+ - `frontend/src/app/analyze/page.tsx`
2275
+ - `frontend/src/components/analyze/TaskInput.tsx`
2276
+ - `frontend/src/components/analyze/ModeSelector.tsx`
2277
+ - `frontend/src/components/analyze/ResultViewer.tsx`
2278
+ - `frontend/src/components/analyze/AgentOutputPanel.tsx`
2279
+ - `frontend/src/app/cases/page.tsx`
2280
+ - `frontend/src/app/cases/[id]/page.tsx`
2281
+ - `frontend/src/components/cases/CaseList.tsx`
2282
+ - `frontend/src/components/cases/CaseCard.tsx`
2283
+ - `frontend/src/components/cases/CaseDetail.tsx`
2284
+ - `frontend/src/app/simulation/page.tsx`
2285
+ - `frontend/src/app/simulation/[id]/page.tsx`
2286
+ - `frontend/src/components/simulation/SimulationForm.tsx`
2287
+ - `frontend/src/components/simulation/SimulationStatus.tsx`
2288
+ - `frontend/src/components/simulation/SimulationReport.tsx`
2289
+ - `frontend/src/components/simulation/SimulationChat.tsx`
2290
+ - `frontend/src/app/prompts/page.tsx`
2291
+ - `frontend/src/components/prompts/PromptList.tsx`
2292
+ - `frontend/src/components/prompts/PromptEditor.tsx`
2293
+ - `frontend/src/app/config/page.tsx`
2294
+ - `frontend/src/lib/api.ts`
2295
+ - `frontend/src/lib/types.ts`
2296
+ - `frontend/src/components/common/Badge.tsx`
2297
+ - `frontend/src/components/common/Card.tsx`
2298
+ - `frontend/src/components/common/LoadingSpinner.tsx`
2299
+ - `frontend/src/components/common/ErrorMessage.tsx`
2300
+
2301
+ **Files to Modify**:
2302
+ - `frontend/src/app/layout.tsx`
2303
+ - `frontend/src/app/page.tsx`
2304
+ - `frontend/src/app/globals.css`
2305
+ - `frontend/package.json` (add dependencies if needed)
2306
+
2307
+ **Verification**:
2308
+ - All pages are accessible via navigation
2309
+ - Analyze workflow works end-to-end
2310
+ - Case history displays correctly
2311
+ - Simulation workflow works end-to-end
2312
+ - Prompt lab allows editing
2313
+ - Config page shows system status
2314
+ - UI is polished and professional
2315
+
2316
+ ### Phase 7: Testing and Documentation
2317
+
2318
+ **Goal**: Comprehensive testing and documentation
2319
+
2320
+ **Tasks**:
2321
+ 1. Write unit tests:
2322
+ - Test provider abstraction
2323
+ - Test domain pack registry
2324
+ - Test agent routing
2325
+ - Test case storage
2326
+ - Test simulation integration
2327
+
2328
+ 2. Write property-based tests:
2329
+ - Implement all 15 properties from Correctness Properties section
2330
+ - Configure 100+ iterations per test
2331
+ - Add property tags
2332
+
2333
+ 3. Write integration tests:
2334
+ - End-to-end case execution
2335
+ - Simulation workflow
2336
+ - Provider fallback
2337
+ - Domain pack enhancement
2338
+
2339
+ 4. Update documentation:
2340
+ - Update `README.md` with architecture overview
2341
+ - Document four-layer architecture
2342
+ - Document agent roles
2343
+ - Document domain pack integration
2344
+ - Document simulation integration
2345
+ - Add setup instructions
2346
+ - Add API endpoint documentation
2347
+ - Add environment variable reference
2348
+
2349
+ 5. Create developer documentation:
2350
+ - Create `ARCHITECTURE.md`
2351
+ - Create `DOMAIN_PACKS.md`
2352
+ - Create `TESTING.md`
2353
+ - Create `DEPLOYMENT.md`
2354
+
2355
+ **Files to Create**:
2356
+ - `backend/tests/test_providers.py`
2357
+ - `backend/tests/test_domain_packs.py`
2358
+ - `backend/tests/test_agents.py`
2359
+ - `backend/tests/test_storage.py`
2360
+ - `backend/tests/test_simulation.py`
2361
+ - `backend/tests/test_properties.py` (property-based tests)
2362
+ - `backend/tests/test_integration.py`
2363
+ - `ARCHITECTURE.md`
2364
+ - `DOMAIN_PACKS.md`
2365
+ - `TESTING.md`
2366
+ - `DEPLOYMENT.md`
2367
+
2368
+ **Files to Modify**:
2369
+ - `README.md`
2370
+
2371
+ **Verification**:
2372
+ - All tests pass
2373
+ - Coverage meets goals (70%+ overall)
2374
+ - Documentation is complete and accurate
2375
+ - Setup instructions work for new developers
2376
+
2377
+ ### Phase 8: Cleanup and Optimization
2378
+
2379
+ **Goal**: Remove dead code, optimize performance, polish
2380
+
2381
+ **Tasks**:
2382
+ 1. Remove dead code:
2383
+ - Remove unused imports
2384
+ - Remove commented code
2385
+ - Remove duplicate implementations
2386
+
2387
+ 2. Optimize performance:
2388
+ - Add caching for market quotes (5 min TTL)
2389
+ - Add connection pooling for external APIs
2390
+ - Optimize database queries (if applicable)
2391
+ - Add request timeouts
2392
+
2393
+ 3. Polish error messages:
2394
+ - Review all error messages
2395
+ - Ensure consistency
2396
+ - Ensure clarity
2397
+
2398
+ 4. Polish logging:
2399
+ - Review log levels
2400
+ - Ensure consistency
2401
+ - Add missing log entries
2402
+
2403
+ 5. Security review:
2404
+ - Ensure no API keys in code
2405
+ - Ensure error messages don't leak internals
2406
+ - Ensure input validation is comprehensive
2407
+
2408
+ 6. Performance testing:
2409
+ - Test response times
2410
+ - Test under load
2411
+ - Identify bottlenecks
2412
+
2413
+ **Verification**:
2414
+ - No dead code remains
2415
+ - Performance meets requirements (5s simple, 30s complex)
2416
+ - Error messages are clear and consistent
2417
+ - Logging is comprehensive and useful
2418
+ - Security review passes
2419
+ - Performance testing passes
2420
+
2421
+ ### Phase 9: Autonomous Knowledge Evolution Layer
2422
+
2423
+ **Goal**: Implement self-improving intelligence system that learns without local model training
2424
+
2425
+ **Tasks**:
2426
+ 1. Create learning subsystem structure:
2427
+ - Create `backend/app/services/learning/__init__.py`
2428
+ - Create `backend/app/services/learning/knowledge_ingestor.py`
2429
+ - Create `backend/app/services/learning/knowledge_store.py`
2430
+ - Create `backend/app/services/learning/learning_engine.py`
2431
+ - Create `backend/app/services/learning/prompt_optimizer.py`
2432
+ - Create `backend/app/services/learning/skill_distiller.py`
2433
+ - Create `backend/app/services/learning/trust_manager.py`
2434
+ - Create `backend/app/services/learning/freshness_manager.py`
2435
+ - Create `backend/app/services/learning/scheduler.py`
2436
+
2437
+ 2. Create data directories:
2438
+ - Create `backend/app/data/knowledge/`
2439
+ - Create `backend/app/data/skills/`
2440
+ - Create `backend/app/data/prompt_versions/`
2441
+ - Create `backend/app/data/learning/`
2442
+
2443
+ 3. Implement knowledge ingestion:
2444
+ - Implement ingest_from_search() using Tavily
2445
+ - Implement ingest_from_url() using Jina Reader
2446
+ - Implement ingest_from_news() using NewsAPI
2447
+ - Implement compress_content() for summarization
2448
+ - Add storage limit enforcement (200MB max)
2449
+
2450
+ 4. Implement knowledge store:
2451
+ - Implement save_knowledge() with JSON storage
2452
+ - Implement search_knowledge() with keyword matching
2453
+ - Implement delete_expired_knowledge() with auto-cleanup
2454
+ - Implement LRU eviction when storage limit reached
2455
+
2456
+ 5. Implement experience learning:
2457
+ - Implement learn_from_case() to extract metadata
2458
+ - Implement detect_patterns() for repeated patterns
2459
+ - Implement get_route_effectiveness() for routing insights
2460
+ - Implement get_prompt_performance() for prompt insights
2461
+
2462
+ 6. Implement prompt evolution:
2463
+ - Implement create_prompt_variant() using provider API
2464
+ - Implement test_prompt_variant() with quality metrics
2465
+ - Implement compare_prompts() for A/B testing
2466
+ - Implement promote_prompt() with validation
2467
+
2468
+ 7. Implement skill distillation:
2469
+ - Implement detect_skill_candidates() from patterns
2470
+ - Implement distill_skill() to create skill records
2471
+ - Implement test_skill() for validation
2472
+ - Implement apply_skill() for skill usage
2473
+
2474
+ 8. Implement trust and freshness management:
2475
+ - Implement get_trust_score() and update_trust()
2476
+ - Implement calculate_freshness() with domain rules
2477
+ - Implement recommend_refresh() for stale items
2478
+
2479
+ 9. Implement learning scheduler:
2480
+ - Implement schedule_task() with safeguards
2481
+ - Implement is_system_idle() and is_battery_ok()
2482
+ - Add scheduled tasks: ingestion, cleanup, pattern detection
2483
+ - Add manual trigger via run_once()
2484
+
2485
+ 10. Add learning endpoints:
2486
+ - Add GET /learning/status
2487
+ - Add POST /learning/run-once
2488
+ - Add GET /learning/insights
2489
+ - Add GET /knowledge and GET /knowledge/{item_id}
2490
+ - Add GET /knowledge/search
2491
+ - Add GET /skills and GET /skills/{skill_name}
2492
+ - Add POST /skills/distill
2493
+ - Add GET /sources/trust and GET /sources/freshness
2494
+ - Add GET /prompts/versions/{name}
2495
+ - Add POST /prompts/optimize/{name}
2496
+ - Add POST /prompts/promote/{name}/{version}
2497
+
2498
+ 11. Integrate with existing layers:
2499
+ - Hook learn_from_case() into case save flow
2500
+ - Hook knowledge search into research agent
2501
+ - Hook skill application into agent execution
2502
+ - Hook trust scores into source selection
2503
+ - Hook prompt versions into prompt loading
2504
+
2505
+ 12. Add configuration:
2506
+ - Add LEARNING_ENABLED flag
2507
+ - Add KNOWLEDGE_MAX_SIZE_MB (default 200)
2508
+ - Add LEARNING_SCHEDULE_INTERVAL
2509
+ - Add LEARNING_BATCH_SIZE
2510
+ - Add domain-specific expiration rules
2511
+
2512
+ **Verification**:
2513
+ - Learning subsystem runs without stressing laptop
2514
+ - Knowledge cache stays under 200MB
2515
+ - Scheduler respects battery and CPU constraints
2516
+ - Trust scores improve source selection
2517
+ - Prompt evolution produces better prompts
2518
+ - Skills are distilled from repeated patterns
2519
+ - Learning endpoints return useful insights
2520
+ - System improves over time without manual intervention
2521
+
2522
+ ## Implementation Priority Summary
2523
+
2524
+ 1. **Phase 1**: Backend Consolidation and Provider Enhancement
2525
+ 2. **Phase 2**: Domain Pack Architecture
2526
+ 3. **Phase 3**: Agent Enhancement with Domain Intelligence
2527
+ 4. **Phase 4**: Simulation Integration Enhancement
2528
+ 5. **Phase 5**: API Discovery Subsystem
2529
+ 6. **Phase 6**: Frontend Enhancement
2530
+ 7. **Phase 7**: Testing and Documentation
2531
+ 8. **Phase 8**: Cleanup and Optimization
2532
+ 9. **Phase 9**: Autonomous Knowledge Evolution Layer
2533
+
2534
+ Each phase builds on the previous, maintaining a runnable system at every step. The focus is on single-user local deployment with production-quality code structure, not enterprise features like auth, Kubernetes, or cloud deployment.
2535
+
2536
+
2537
+ ## Design Decisions and Rationale
2538
+
2539
+ ### Why Five Layers?
2540
+
2541
+ The five-layer architecture provides clear separation of concerns:
2542
+ - **Layer 1 (Core Platform)**: Infrastructure that any system needs
2543
+ - **Layer 2 (Agent Organization)**: Domain-agnostic orchestration
2544
+ - **Layer 3 (Domain Packs)**: Pluggable domain intelligence
2545
+ - **Layer 4 (Simulation Lab)**: External service for scenario modeling
2546
+ - **Layer 5 (Autonomous Knowledge Evolution)**: Self-improvement without local model training
2547
+
2548
+ This separation allows the system to improve itself over time while maintaining clear boundaries between operational layers and learning layers.
2549
+
2550
+ ### Why Domain Packs Instead of Monolithic Agents?
2551
+
2552
+ Domain packs provide:
2553
+ - **Modularity**: Finance intelligence can be developed independently
2554
+ - **Reusability**: Same pack can enhance multiple agents
2555
+ - **Extensibility**: New domains (policy, cyber) can be added without refactoring
2556
+ - **Testability**: Domain logic can be tested in isolation
2557
+
2558
+ ### Why Adapter Pattern for MiroFish?
2559
+
2560
+ The adapter pattern provides:
2561
+ - **Loose Coupling**: MiroOrg doesn't depend on MiroFish internals
2562
+ - **Testability**: Adapter can be mocked for testing
2563
+ - **Flexibility**: MiroFish can be replaced or upgraded independently
2564
+ - **Error Isolation**: MiroFish failures don't crash MiroOrg
2565
+
2566
+ ### Why Local Storage Instead of Database?
2567
+
2568
+ For single-user local deployment:
2569
+ - **Simplicity**: No database setup required
2570
+ - **Portability**: Data travels with the application
2571
+ - **Transparency**: JSON files are human-readable
2572
+ - **Offline**: Works without network connectivity
2573
+
2574
+ Future versions can add database support without changing the service layer interface.
2575
+
2576
+ ### Why Provider Abstraction?
2577
+
2578
+ Provider abstraction provides:
2579
+ - **Flexibility**: Switch providers without changing agent code
2580
+ - **Resilience**: Automatic fallback when primary fails
2581
+ - **Cost Optimization**: Use cheaper providers for simple tasks
2582
+ - **Future-Proofing**: New providers can be added easily
2583
+
2584
+ ### Why Property-Based Testing?
2585
+
2586
+ Property-based testing provides:
2587
+ - **Comprehensive Coverage**: Tests thousands of input combinations
2588
+ - **Bug Discovery**: Finds edge cases developers miss
2589
+ - **Specification**: Properties serve as executable specifications
2590
+ - **Regression Prevention**: Random inputs catch regressions
2591
+
2592
+ Combined with unit tests, property tests provide high confidence in correctness.
2593
+
2594
+ ### Why Autonomous Knowledge Evolution Instead of Local Model Training?
2595
+
2596
+ The Autonomous Knowledge Evolution Layer provides system-level self-improvement without the resource requirements of local model training:
2597
+
2598
+ **Resource Constraints**:
2599
+ - Local model training requires 40GB+ VRAM, terabytes of storage, and days of compute time
2600
+ - Autonomous learning requires only 200MB storage and lightweight background tasks
2601
+ - Suitable for 8GB/256GB laptop without stressing the system
2602
+
2603
+ **Learning Approach**:
2604
+ - **NOT**: Training foundation models locally
2605
+ - **YES**: Learning from compressed knowledge, case patterns, prompt evolution, and skill distillation
2606
+ - **NOT**: Storing raw datasets
2607
+ - **YES**: Storing compressed summaries (2-4KB each)
2608
+
2609
+ **Benefits**:
2610
+ - System improves from real-world usage
2611
+ - Learns which sources are trustworthy
2612
+ - Evolves prompts through controlled testing
2613
+ - Distills reusable skills from patterns
2614
+ - Adapts to user's domain and use cases
2615
+ - No manual intervention required
2616
+
2617
+ **Safeguards**:
2618
+ - Strict storage limits (200MB max)
2619
+ - Battery-aware scheduling
2620
+ - CPU-conscious background tasks
2621
+ - Controlled prompt evolution (not autonomous chaos)
2622
+ - Manual override options
2623
+
2624
+ This approach makes the system genuinely self-improving while respecting laptop constraints.
2625
+
2626
+ ## Security Considerations
2627
+
2628
+ ### API Key Management
2629
+
2630
+ - All API keys loaded from environment variables
2631
+ - Never commit `.env` files
2632
+ - Never log API keys
2633
+ - Never expose keys in API responses
2634
+ - Validate keys on startup
2635
+
2636
+ ### Input Validation
2637
+
2638
+ - All requests validated against Pydantic schemas
2639
+ - Reject invalid inputs with 422 status
2640
+ - Sanitize user input before external API calls
2641
+ - Prevent injection attacks
2642
+
2643
+ ### Error Message Sanitization
2644
+
2645
+ - Never expose internal implementation details
2646
+ - Never expose stack traces to frontend
2647
+ - Never expose raw provider exceptions
2648
+ - Provide descriptive but safe error messages
2649
+
2650
+ ### External Service Isolation
2651
+
2652
+ - All external calls have timeouts
2653
+ - All external failures are caught and handled
2654
+ - External failures don't crash the system
2655
+ - External services are optional (graceful degradation)
2656
+
2657
+ ## Performance Considerations
2658
+
2659
+ ### Response Time Targets
2660
+
2661
+ - Simple queries: < 5 seconds
2662
+ - Medium queries: < 15 seconds
2663
+ - Complex queries: < 30 seconds
2664
+ - Simulation submission: < 5 seconds
2665
+ - Simulation completion: varies (minutes to hours)
2666
+
2667
+ ### Optimization Strategies
2668
+
2669
+ - Connection pooling for external APIs
2670
+ - Caching for market quotes (5 min TTL)
2671
+ - Async/await for I/O-bound operations
2672
+ - Pagination for large result sets
2673
+ - Request timeouts to prevent hanging
2674
+
2675
+ ### Scalability Considerations
2676
+
2677
+ Current design is single-user local deployment. Future scalability improvements:
2678
+ - Replace JSON storage with database
2679
+ - Add Redis for caching
2680
+ - Add message queue for async processing
2681
+ - Add horizontal scaling for agents
2682
+ - Add load balancing
2683
+
2684
+ ## Deployment Considerations
2685
+
2686
+ ### Local Development
2687
+
2688
+ 1. Clone repository
2689
+ 2. Copy `.env.example` to `.env`
2690
+ 3. Configure API keys
2691
+ 4. Install Python dependencies: `pip install -r requirements.txt`
2692
+ 5. Install Node dependencies: `cd frontend && npm install`
2693
+ 6. Run backend: `cd backend && uvicorn app.main:app --reload`
2694
+ 7. Run frontend: `cd frontend && npm run dev`
2695
+ 8. Access at `http://localhost:3000`
2696
+
2697
+ ### Production Deployment (Future)
2698
+
2699
+ - Use production ASGI server (gunicorn + uvicorn)
2700
+ - Use production Node server (Next.js production build)
2701
+ - Use reverse proxy (nginx)
2702
+ - Use process manager (systemd, supervisor)
2703
+ - Use HTTPS
2704
+ - Use environment-specific configs
2705
+ - Use log aggregation
2706
+ - Use monitoring and alerting
2707
+
2708
+ ## Conclusion
2709
+
2710
+ This design provides a comprehensive blueprint for implementing MiroOrg v1.1 as a general intelligence operating system with pluggable domain packs, multi-agent orchestration, and simulation capabilities. The four-layer architecture ensures modularity and extensibility, while the provider abstraction and domain pack pattern enable flexibility and future growth.
2711
+
2712
+ The implementation phases provide a clear roadmap from current state to production-ready system, maintaining a runnable system at every step. The testing strategy ensures correctness through both unit tests and property-based tests, while the error handling and security considerations ensure robustness and safety.
2713
+
2714
+ The design is executable, with specific file targets, clear interfaces, and concrete implementation guidance. The system can be built incrementally, with each phase delivering value and maintaining backward compatibility.
2715
+
.kiro/specs/ai-financial-intelligence-system/requirements.md ADDED
@@ -0,0 +1,473 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Requirements Document
2
+
3
+ ## Introduction
4
+
5
+ This document specifies requirements for MiroOrg v1.1, a general intelligence operating system that orchestrates multiple specialist agents, runs simulations, supports pluggable domain packs, and autonomously improves itself over time. The system merges capabilities from miroorg-basic-v2 (base architecture), impact_ai (first domain pack), MiroFish (simulation lab), and public-apis (API discovery catalog) into a unified, production-ready platform. The architecture follows a five-layer design: Core Platform, Agent Organization, Domain Packs, Simulation Lab, and Autonomous Knowledge Evolution Layer.
6
+
7
+ ## Glossary
8
+
9
+ - **System**: MiroOrg v1.1 - The general intelligence operating system
10
+ - **Core_Platform**: Layer 1 - FastAPI backend, frontend dashboard, config, health, memory, prompts, cases, logs
11
+ - **Agent_Organization**: Layer 2 - Multi-agent orchestration framework (Switchboard, Research, Planner, Verifier, Synthesizer)
12
+ - **Domain_Packs**: Layer 3 - Pluggable domain intelligence modules (finance/news from impact_ai as first pack)
13
+ - **Simulation_Lab**: Layer 4 - MiroFish integration for simulation, digital-world modeling, and scenario analysis
14
+ - **Autonomous_Knowledge_Evolution_Layer**: Layer 5 - Self-improving intelligence system that learns from internet knowledge, past cases, prompt evolution, and skill distillation
15
+ - **Knowledge_Item**: Compressed structured record of external information with summary, entities, trust score, and freshness score
16
+ - **Skill**: Distilled reusable workflow pattern extracted from repeated successful case executions
17
+ - **Trust_Score**: Measure of source reliability learned from verification outcomes (0.0 - 1.0)
18
+ - **Freshness_Score**: Measure of information recency and relevance (0.0 - 1.0)
19
+ - **Prompt_Version**: Versioned prompt with performance metadata (win_rate, status, last_tested)
20
+ - **Switchboard**: Routing agent that classifies tasks by family, domain, complexity, and execution mode
21
+ - **Research_Agent**: Agent responsible for gathering context, extracting entities, and fetching external information
22
+ - **Planner_Agent**: Agent that converts research into practical action plans
23
+ - **Verifier_Agent**: Agent that validates credibility, detects rumors/scams, and surfaces uncertainty
24
+ - **Synthesizer_Agent**: Agent that produces final comprehensive responses with honest uncertainty
25
+ - **Provider_Layer**: Abstraction for AI model providers (OpenRouter, Ollama, future OpenAI)
26
+ - **Case**: A stored execution record containing inputs, routing decisions, agent outputs, and results
27
+ - **MiroFish**: External simulation service for graph building, environment setup, simulation, report generation, and deep interaction
28
+ - **Domain_Pack**: Pluggable module providing domain-specific intelligence (finance is first, others follow)
29
+ - **API_Discovery**: Subsystem using public-apis catalog for discovering and classifying free APIs
30
+ - **Ticker**: Stock market symbol (e.g., AAPL, TSLA)
31
+ - **Entity**: Company, organization, person, or concept mentioned in text
32
+ - **Source_Credibility**: Measure of trustworthiness for information sources
33
+ - **Analyze_Mode**: Execution mode for summarization, research, and analysis tasks
34
+ - **Organization_Mode**: Execution mode using multi-agent collaboration
35
+ - **Simulation_Mode**: Execution mode for scenario forecasting and what-if modeling
36
+
37
+ ## Requirements
38
+
39
+ ### Requirement 1: Repository Ownership and Merge Strategy
40
+
41
+ **User Story:** As a developer, I want clear repository ownership rules, so that I can merge codebases without architectural confusion.
42
+
43
+ #### Acceptance Criteria
44
+
45
+ 1. THE System SHALL use miroorg-basic-v2 as the primary repo and canonical architecture
46
+ 2. THE System SHALL treat impact_ai as a source of reusable domain modules, not as an equal structural peer
47
+ 3. THE System SHALL integrate MiroFish as a separate service through a client adapter
48
+ 4. THE System SHALL treat public-apis/public-apis as a discovery dataset for future connector expansion, not as a runtime dependency
49
+ 5. WHEN overlapping logic exists between repos, THE System SHALL choose one canonical implementation and remove dead duplicates
50
+ 6. THE System SHALL preserve the existing miroorg-basic-v2 folder structure as the base architecture
51
+ 7. THE System SHALL maintain separate directories for agents, services, routers, prompts, and core configuration
52
+ 8. THE System SHALL use environment variables for all configuration and API keys
53
+
54
+ ### Requirement 2: Five-Layer Architecture
55
+
56
+ **User Story:** As a system architect, I want a clear five-layer architecture, so that the system can scale across multiple domains, use cases, and autonomously improve over time.
57
+
58
+ #### Acceptance Criteria
59
+
60
+ 1. THE System SHALL implement Layer 1 (Core Platform) with FastAPI backend, frontend dashboard, config, health, memory, prompts, cases, and logs
61
+ 2. THE System SHALL implement Layer 2 (Agent Organization) with Switchboard, Research Agent, Planner Agent, Verifier Agent, and Synthesizer Agent
62
+ 3. THE System SHALL implement Layer 3 (Domain Packs) with finance/news intelligence from impact_ai as the first pack
63
+ 4. THE System SHALL implement Layer 4 (Simulation Lab) with MiroFish integration for simulation and digital-world modeling
64
+ 5. THE System SHALL implement Layer 5 (Autonomous Knowledge Evolution Layer) with world knowledge ingestion, experience learning, prompt evolution, skill distillation, and trust management
65
+ 6. THE System SHALL support future domain packs (policy, cyber, enterprise ops, research, education) without changing Layer 2
66
+ 7. THE System SHALL support optional future agents (Risk, Market, Simulation, Compliance) in Layer 2
67
+ 8. THE System SHALL maintain clear separation between layers with well-defined interfaces
68
+
69
+ ### Requirement 3: Product Modes
70
+
71
+ **User Story:** As a user, I want the system to support different execution modes, so that I can choose the appropriate approach for my task.
72
+
73
+ #### Acceptance Criteria
74
+
75
+ 1. THE System SHALL support Analyze Mode for summarization, research, market/news analysis, entity/ticker detection, credibility/risk evaluation, and actionable recommendations
76
+ 2. THE System SHALL support Organization Mode for multi-agent debate, planning, verification, synthesis, and case memory workflows
77
+ 3. THE System SHALL support Simulation Mode for scenario forecasting, market reaction prediction, stakeholder reaction modeling, policy/narrative/reputation simulation, and post-simulation questioning
78
+ 4. WHEN Simulation Mode is used, THE System SHALL use MiroFish through adapter endpoints, not through frontend-direct calls
79
+ 5. THE System SHALL route tasks to the appropriate mode based on Switchboard classification
80
+
81
+ ### Requirement 4: Switchboard Routing and Classification
82
+
83
+ **User Story:** As a user, I want intelligent task routing, so that my requests are handled by the appropriate execution path.
84
+
85
+ #### Acceptance Criteria
86
+
87
+ 1. THE Switchboard SHALL classify every task using four dimensions: task_family (normal or simulation), domain_pack (finance, general, policy, custom), complexity (simple, medium, complex), and execution_mode (solo, standard, deep)
88
+ 2. WHEN complexity is simple, THE Switchboard SHALL route to solo execution mode (minimal path)
89
+ 3. WHEN complexity is medium, THE Switchboard SHALL route to standard execution mode (normal multi-agent path)
90
+ 4. WHEN complexity is complex, THE Switchboard SHALL route to deep execution mode (full multi-agent path with optional verifier and optional simulation handoff)
91
+ 5. WHEN user input contains simulation trigger keywords, THE Switchboard SHALL route to Simulation Mode
92
+ 6. THE System SHALL make simulation trigger keywords environment-configurable
93
+ 7. THE Switchboard SHALL detect keywords including: simulate, predict, model reaction, test scenarios, run digital twins, explore "what if" outcomes
94
+ 8. THE Switchboard SHALL include routing decision in case metadata
95
+
96
+ ### Requirement 5: Domain Engine Integration
97
+
98
+ **User Story:** As a financial analyst, I want the system to leverage impact_ai's financial intelligence modules, so that I can perform sophisticated market and news analysis.
99
+
100
+ #### Acceptance Criteria
101
+
102
+ 1. THE System SHALL integrate impact_ai as the first domain pack
103
+ 2. THE System SHALL inspect and reuse valuable modules including: alpha_vantage_client.py, news_api.py, brain.py, event_analyzer.py, ticker_resolver.py, source_checker.py, rumor_detector.py, scam_detector.py, stance_detector.py, prediction.py, market_data.py, entity_resolver.py
104
+ 3. THE System SHALL expose domain pack capabilities through the MiroOrg service layer, not as scattered utility scripts
105
+ 4. THE System SHALL consolidate overlapping external API clients (Alpha Vantage, NewsAPI) with existing external_sources.py
106
+ 5. WHEN integrating impact_ai modules, THE System SHALL refactor code to match the existing service layer pattern
107
+ 6. THE System SHALL design the architecture to support additional domain packs later without changing the agent organization layer
108
+ 7. THE System SHALL treat finance as the first deep pack, not the only pack
109
+
110
+ ### Requirement 6: Provider Abstraction Layer
111
+
112
+ **User Story:** As a system administrator, I want a unified provider interface, so that I can switch between AI model providers without changing agent code.
113
+
114
+ #### Acceptance Criteria
115
+
116
+ 1. THE System SHALL create a provider abstraction layer with a single call_model() interface
117
+ 2. THE System SHALL support OpenRouter as a primary provider
118
+ 3. THE System SHALL support Ollama as a fallback provider
119
+ 4. THE System SHALL support future OpenAI provider integration
120
+ 5. WHEN the primary provider fails, THE System SHALL automatically fall back to the secondary provider according to environment-configured policy
121
+ 6. THE System SHALL log provider selection and fallback events
122
+ 7. THE System SHALL expose provider configuration through environment variables
123
+ 8. THE System SHALL allow per-agent model selection (chat vs reasoner models)
124
+ 9. THE System SHALL support adaptive execution depth to reduce unnecessary external model usage on trivial tasks
125
+
126
+ ### Requirement 7: Enhanced Agent Intelligence
127
+
128
+ **User Story:** As a user, I want agents to use domain intelligence capabilities, so that I receive accurate and credible analysis.
129
+
130
+ #### Acceptance Criteria
131
+
132
+ 1. THE Research_Agent SHALL gather context from prompts, APIs, and domain services
133
+ 2. THE Research_Agent SHALL extract entities, tickers, and claims from user input
134
+ 3. THE Research_Agent SHALL fetch external information where allowed
135
+ 4. THE Research_Agent SHALL return structured facts, assumptions, open questions, and useful signals
136
+ 5. THE Planner_Agent SHALL convert research into a practical response or action plan
137
+ 6. THE Planner_Agent SHALL highlight dependencies, risks, and possible next steps
138
+ 7. THE Verifier_Agent SHALL test credibility of information
139
+ 8. THE Verifier_Agent SHALL detect rumors, scams, unsupported claims, and contradictions using source_checker, rumor_detector, and scam_detector
140
+ 9. THE Verifier_Agent SHALL force uncertainty to be made visible
141
+ 10. THE Synthesizer_Agent SHALL combine outputs into one final answer
142
+ 11. THE Synthesizer_Agent SHALL state uncertainty honestly
143
+ 12. THE Synthesizer_Agent SHALL recommend next actions
144
+ 13. THE Synthesizer_Agent SHALL suggest simulation mode when scenario analysis is more appropriate
145
+ 14. THE System SHALL preserve existing agent prompt files and update them with domain intelligence instructions
146
+ 15. THE System SHALL maintain agent modularity and separation of concerns
147
+
148
+ ### Requirement 9: External API Integration and Discovery
149
+
150
+ **User Story:** As a developer, I want a structured approach to external API integration, so that I can easily add new connectors and discover useful APIs.
151
+
152
+ #### Acceptance Criteria
153
+
154
+ 1. THE System SHALL support external API integrations through a dedicated services layer
155
+ 2. THE System SHALL include initial connector support for: market/news connectors from impact_ai, OpenRouter, Ollama, Tavily, Jina Reader, NewsAPI, Alpha Vantage
156
+ 3. THE System SHALL use public-apis/public-apis as an API discovery source for future connectors
157
+ 4. THE System SHALL implement an API discovery subsystem that classifies free APIs by category
158
+ 5. THE API discovery subsystem SHALL score candidate APIs for usefulness
159
+ 6. THE API discovery subsystem SHALL store metadata such as auth requirements, HTTPS support, and CORS configuration
160
+ 7. THE API discovery subsystem SHALL support future sandbox testing and promotion into connectors
161
+ 8. THE System SHALL treat public-apis as a discovery catalog, not as a runtime dependency
162
+ 9. THE System SHALL use connection pooling for external API clients
163
+ 10. THE System SHALL implement request timeouts for all external API calls
164
+
165
+ ### Requirement 10: Case Memory System
166
+
167
+ **User Story:** As a user, I want all interactions stored, so that I can review past analyses and track system behavior.
168
+
169
+ #### Acceptance Criteria
170
+
171
+ 1. THE System SHALL persist every case execution to local storage
172
+ 2. THE System SHALL store case_id, user_input, routing decision, agent outputs, final answer, and timestamps
173
+ 3. THE System SHALL support retrieving cases by case_id
174
+ 4. THE System SHALL support listing all cases with optional limit parameter
175
+ 5. THE System SHALL support deleting cases by case_id
176
+ 6. THE System SHALL provide memory statistics including total cases and storage size
177
+ 7. THE System SHALL use JSON format for case storage
178
+ 8. WHEN a simulation is executed, THE System SHALL link simulation_id to the case record
179
+ 9. THE System SHALL store cases in backend/app/data/memory directory
180
+ 10. THE System SHALL store simulations in backend/app/data/simulations directory
181
+ 11. THE System SHALL store logs in backend/app/data/logs directory
182
+ 12. THE System SHALL create data directories automatically if they do not exist
183
+
184
+ ### Requirement 8: Simulation Integration
185
+
186
+ **User Story:** As a user, I want to run simulations and explore what-if scenarios, so that I can predict potential impacts and test hypotheses.
187
+
188
+ #### Acceptance Criteria
189
+
190
+ 1. THE System SHALL integrate MiroFish as a separate backend service, not merged directly into the MiroOrg codebase
191
+ 2. THE System SHALL implement a mirofish_client service as an adapter
192
+ 3. THE System SHALL make MiroFish API paths configurable through environment variables
193
+ 4. THE System SHALL support MiroFish health check
194
+ 5. THE System SHALL support simulation submission with title and prediction_goal
195
+ 6. THE System SHALL support simulation status retrieval by simulation_id
196
+ 7. THE System SHALL support report retrieval by simulation_id
197
+ 8. THE System SHALL support post-simulation chat by simulation_id
198
+ 9. THE System SHALL use MiroFish for graph building, entity relationship extraction, persona generation, simulation, report generation, and deep interaction
199
+ 10. THE System SHALL store simulation metadata locally in simulations directory
200
+ 11. WHEN MiroFish is disabled, THE System SHALL return appropriate error messages for simulation requests
201
+ 12. THE System SHALL handle MiroFish connection failures gracefully with descriptive errors
202
+ 13. THE frontend SHALL only consume MiroOrg endpoints for simulation, never direct MiroFish calls
203
+
204
+ ### Requirement 11: API Endpoints
205
+
206
+ **User Story:** As a frontend developer, I want comprehensive REST endpoints, so that I can build a rich user interface.
207
+
208
+ #### Acceptance Criteria
209
+
210
+ 1. THE System SHALL preserve GET /health endpoint for basic health checks
211
+ 2. THE System SHALL preserve GET /health/deep endpoint for comprehensive health status
212
+ 3. THE System SHALL preserve GET /config/status endpoint for configuration visibility
213
+ 4. THE System SHALL preserve GET /agents endpoint for listing all agents
214
+ 5. THE System SHALL preserve GET /agents/{agent_name} endpoint for agent details
215
+ 6. THE System SHALL preserve POST /run endpoint for standard execution
216
+ 7. THE System SHALL preserve POST /run/debug endpoint for detailed execution traces
217
+ 8. THE System SHALL preserve POST /run/agent endpoint for single agent execution
218
+ 9. THE System SHALL preserve GET /cases endpoint for listing cases
219
+ 10. THE System SHALL preserve GET /cases/{case_id} endpoint for case details
220
+ 11. THE System SHALL preserve DELETE /cases/{case_id} endpoint for case deletion
221
+ 12. THE System SHALL preserve GET /memory/stats endpoint for memory statistics
222
+ 13. THE System SHALL preserve GET /prompts endpoint for listing prompts
223
+ 14. THE System SHALL preserve GET /prompts/{name} endpoint for prompt retrieval
224
+ 15. THE System SHALL preserve PUT /prompts/{name} endpoint for prompt updates
225
+ 16. THE System SHALL preserve GET /simulation/health endpoint for MiroFish health
226
+ 17. THE System SHALL preserve POST /simulation/run endpoint for simulation submission
227
+ 18. THE System SHALL preserve GET /simulation/{simulation_id} endpoint for simulation status
228
+ 19. THE System SHALL preserve GET /simulation/{simulation_id}/report endpoint for simulation reports
229
+ 20. THE System SHALL preserve POST /simulation/{simulation_id}/chat endpoint for simulation chat
230
+ 21. THE frontend SHALL only consume MiroOrg endpoints, even for simulation operations
231
+
232
+ ### Requirement 9: Error Handling and Logging
233
+
234
+ **User Story:** As a developer, I want robust error handling and logging, so that I can diagnose issues and monitor system behavior.
235
+
236
+ #### Acceptance Criteria
237
+
238
+ 1. THE System SHALL use typed Pydantic schemas for all request and response models
239
+ 2. THE System SHALL validate all incoming requests against schemas
240
+ 3. THE System SHALL return structured error responses with appropriate HTTP status codes
241
+ 4. THE System SHALL log all agent executions with case_id and timestamps
242
+ 5. THE System SHALL log provider selection and fallback events
243
+ 6. THE System SHALL log external API calls and failures
244
+ 7. THE System SHALL log simulation requests and responses
245
+ 8. WHEN an external service fails, THE System SHALL return descriptive error messages without exposing internal details
246
+ 9. THE System SHALL write logs to backend/app/data/logs directory with rotation
247
+ 10. THE System SHALL never expose raw provider exceptions to the frontend
248
+
249
+ ### Requirement 12: Frontend Enhancement
250
+
251
+ **User Story:** As a user, I want a polished dashboard interface, so that I can interact with the system professionally.
252
+
253
+ #### Acceptance Criteria
254
+
255
+ 1. THE System SHALL evolve the frontend from a demo page into a product dashboard
256
+ 2. THE System SHALL provide a Main Dashboard page
257
+ 3. THE System SHALL provide an Analyze tab for analysis tasks
258
+ 4. THE System SHALL provide a Cases/History tab for reviewing past executions
259
+ 5. THE System SHALL provide a Prompt Lab tab for prompt management
260
+ 6. THE System SHALL provide a Simulation tab for scenario modeling
261
+ 7. THE System SHALL provide an input task box
262
+ 8. THE System SHALL provide a mode selector for Analyze vs Simulation
263
+ 9. THE System SHALL provide a case output viewer
264
+ 10. THE System SHALL display route/debug badges
265
+ 11. THE System SHALL display agent output panels
266
+ 12. THE System SHALL display market context panel
267
+ 13. THE System SHALL display simulation status panel
268
+ 14. THE System SHALL display confidence badges
269
+ 15. THE System SHALL use a premium dark UI with card-based structure
270
+ 16. THE System SHALL include subtle animations and transitions
271
+ 17. THE System SHALL allow users to view case details including case_id and stored metadata
272
+ 18. THE System MAY reuse impact_ai UI ideas and data panels, but SHALL consolidate into the miroorg-basic-v2 app shell
273
+
274
+ ### Requirement 13: Configuration and Deployment
275
+
276
+ **User Story:** As a system administrator, I want clear configuration and setup instructions, so that I can deploy the system reliably.
277
+
278
+ #### Acceptance Criteria
279
+
280
+ 1. THE System SHALL provide a .env.example file with all required environment variables
281
+ 2. THE System SHALL document all environment variables in README.md
282
+ 3. THE System SHALL include setup instructions for local development
283
+ 4. THE System SHALL include instructions for running backend and frontend separately
284
+ 5. THE System SHALL specify Python version requirements (3.10+)
285
+ 6. THE System SHALL specify Node.js version requirements for frontend
286
+ 7. THE System SHALL include a requirements.txt with all Python dependencies
287
+ 8. THE System SHALL include a package.json with all Node.js dependencies
288
+ 9. THE System SHALL document API endpoint usage with examples
289
+ 10. THE System SHALL document the four-layer architecture in README.md
290
+ 11. THE System SHALL document agent roles and responsibilities
291
+ 12. THE System SHALL document simulation integration setup
292
+ 13. THE System SHALL document domain pack integration approach
293
+ 14. THE System SHALL keep the backend runnable locally at every phase
294
+
295
+ ### Requirement 14: Data Persistence and Storage
296
+
297
+ **User Story:** As a user, I want my data stored locally, so that I can access historical analyses offline.
298
+
299
+ #### Acceptance Criteria
300
+
301
+ 1. THE System SHALL store cases in backend/app/data/memory directory
302
+ 2. THE System SHALL store simulations in backend/app/data/simulations directory
303
+ 3. THE System SHALL store logs in backend/app/data/logs directory
304
+ 4. THE System SHALL use JSON format for case and simulation storage
305
+ 5. THE System SHALL create data directories automatically if they do not exist
306
+ 6. THE System SHALL include data directories in .gitignore to prevent committing user data
307
+ 7. THE System SHALL support exporting cases as JSON files
308
+ 8. WHEN storage operations fail, THE System SHALL log errors and return appropriate HTTP status codes
309
+ 9. EACH case SHALL store: case_id, input, route, agent outputs, final answer, timestamps, optional simulation_id
310
+ 10. EACH simulation SHALL store: simulation_id, remote payload, local metadata, status, report snapshot
311
+
312
+ ### Requirement 13: Security and Secrets Management
313
+
314
+ **User Story:** As a security-conscious developer, I want proper secrets management, so that API keys are never exposed.
315
+
316
+ #### Acceptance Criteria
317
+
318
+ 1. THE System SHALL load all API keys from environment variables
319
+ 2. THE System SHALL never commit .env files to version control
320
+ 3. THE System SHALL include .env in .gitignore
321
+ 4. THE System SHALL provide .env.example with placeholder values
322
+ 5. THE System SHALL validate required API keys on startup
323
+ 6. WHEN required API keys are missing, THE System SHALL log warnings and disable affected features
324
+ 7. THE System SHALL never expose API keys in API responses
325
+ 8. THE System SHALL never log API keys in plain text
326
+ 9. THE System SHALL use HTTPS for all external API calls
327
+ 10. THE System SHALL sanitize error messages to prevent information leakage
328
+
329
+ ### Requirement 14: Testing and Quality Assurance
330
+
331
+ **User Story:** As a developer, I want the codebase to be testable, so that I can ensure reliability and catch regressions.
332
+
333
+ #### Acceptance Criteria
334
+
335
+ 1. THE System SHALL maintain modular service layer for unit testing
336
+ 2. THE System SHALL use dependency injection for external services
337
+ 3. THE System SHALL provide mock implementations for external APIs in tests
338
+ 4. THE System SHALL validate all Pydantic schemas with test cases
339
+ 5. THE System SHALL test provider fallback behavior
340
+ 6. THE System SHALL test agent routing logic
341
+ 7. THE System SHALL test case storage and retrieval
342
+ 8. THE System SHALL test simulation integration error handling
343
+ 9. THE System SHALL maintain code coverage above 70% for critical paths
344
+ 10. THE System SHALL run linting and type checking in CI/CD pipeline
345
+
346
+ ### Requirement 15: Performance and Scalability
347
+
348
+ **User Story:** As a user, I want fast response times, so that I can analyze information efficiently.
349
+
350
+ #### Acceptance Criteria
351
+
352
+ 1. WHEN processing simple queries, THE System SHALL respond within 5 seconds
353
+ 2. WHEN processing complex queries, THE System SHALL respond within 30 seconds
354
+ 3. THE System SHALL use connection pooling for external API clients
355
+ 4. THE System SHALL implement request timeouts for all external API calls
356
+ 5. THE System SHALL cache market quotes for 5 minutes to reduce API calls
357
+ 6. THE System SHALL limit concurrent external API requests to prevent rate limiting
358
+ 7. THE System SHALL use async/await patterns for I/O-bound operations
359
+ 8. THE System SHALL implement pagination for case listing endpoints
360
+ 9. WHEN memory usage exceeds 1GB, THE System SHALL log warnings
361
+ 10. THE System SHALL support horizontal scaling by making state storage pluggable
362
+
363
+ ### Requirement 16: Implementation Priorities
364
+
365
+ **User Story:** As a project manager, I want clear implementation priorities, so that the team can deliver value incrementally.
366
+
367
+ #### Acceptance Criteria
368
+
369
+ 1. THE System SHALL implement in this order: backend consolidation, provider abstraction, impact_ai domain integration, simulation adapter, frontend enhancement, testing and cleanup
370
+ 2. THE System SHALL keep the backend runnable locally at every phase
371
+ 3. THE System SHALL NOT prioritize enterprise auth, Kubernetes, large-scale distributed infra, full cloud deployment, or advanced multi-user features in this phase
372
+ 4. THE System SHALL focus on single-user local deployment with production-quality code structure
373
+ 5. THE System SHALL use miroorg-basic-v2 as the base repo for all implementation work
374
+ 6. THE System SHALL port valuable impact_ai modules into the service/domain layer
375
+ 7. THE System SHALL integrate MiroFish through a clean adapter and router, never direct frontend calls
376
+ 8. THE System SHALL add API discovery scaffolding using public-apis as a catalog source
377
+ 9. THE System SHALL remove dead duplicates during consolidation
378
+
379
+ ### Requirement 17: Autonomous Knowledge Evolution Layer
380
+
381
+ **User Story:** As a user, I want the system to improve itself over time by learning from internet knowledge, past cases, and successful patterns, so that it becomes smarter without requiring manual intervention or stressing my laptop.
382
+
383
+ #### Acceptance Criteria
384
+
385
+ 1. THE System SHALL implement an Autonomous Knowledge Evolution Layer as Layer 5 in the architecture
386
+ 2. THE System SHALL NOT train foundation models locally or store large raw datasets
387
+ 3. THE System SHALL store only compressed summaries, extracted facts, source metadata, trust scores, and skill records
388
+ 4. THE System SHALL respect strict storage limits: max 200MB for knowledge cache, 2-4KB per article summary
389
+ 5. THE System SHALL auto-delete stale knowledge after configurable expiration period
390
+ 6. THE System SHALL run learning tasks only when system is idle and laptop is not stressed
391
+ 7. THE System SHALL stop learning tasks if battery is low or system resources are constrained
392
+
393
+ #### World Knowledge Ingestion
394
+
395
+ 8. THE System SHALL continuously ingest high-signal information from Tavily, Jina Reader, NewsAPI, Alpha Vantage, and discovered APIs
396
+ 9. THE System SHALL compress external information into structured summaries with: title, summary, entities, source_url, source_type, trust_score, freshness_score, domain_pack, timestamps
397
+ 10. THE System SHALL NOT save raw webpage archives or full-page content
398
+ 11. THE System SHALL extract and store only: summaries, entities, claims, source metadata, freshness indicators, trust scores
399
+ 12. THE System SHALL respect API rate limits and avoid excessive external requests
400
+
401
+ #### Experience Learning
402
+
403
+ 13. THE System SHALL learn from every case execution by tracking: route effectiveness, prompt performance, provider reliability, source usefulness, answer corrections, repeated patterns
404
+ 14. THE System SHALL store case learning metadata without duplicating full case records
405
+ 15. THE System SHALL identify patterns across multiple cases to inform future routing and agent decisions
406
+ 16. THE System SHALL update trust scores for sources based on verification outcomes
407
+
408
+ #### Prompt Evolution
409
+
410
+ 17. THE System SHALL version all agent prompts with metadata: version, last_tested, win_rate, status (active/experimental/archived)
411
+ 18. THE System SHALL test improved prompt variants on sampled tasks
412
+ 19. THE System SHALL compare prompt outcomes using quality metrics
413
+ 20. THE System SHALL promote better-performing prompts to active status
414
+ 21. THE System SHALL archive underperforming prompt versions
415
+ 22. THE System SHALL NOT allow uncontrolled autonomous prompt changes without validation
416
+
417
+ #### Skill Distillation
418
+
419
+ 23. WHEN the system solves similar problems repeatedly, THE System SHALL distill patterns into reusable skills
420
+ 24. EACH skill SHALL contain: name, trigger_patterns, recommended_agents, preferred_sources, prompt_overrides
421
+ 25. THE System SHALL store skills as structured records in backend/app/data/skills directory
422
+ 26. THE System SHALL make distilled skills available to agents for future similar tasks
423
+ 27. THE System SHALL support skill types including: financial_rumor_review, policy_reaction_analysis, earnings_impact_brief, simulation_prep_pack
424
+
425
+ #### Trust and Freshness Management
426
+
427
+ 28. THE System SHALL maintain trust scores for all external sources (APIs, news outlets, websites)
428
+ 29. THE System SHALL track freshness scores to identify stale information
429
+ 30. THE System SHALL recommend source refresh when freshness degrades
430
+ 31. THE System SHALL learn which sources are reliable vs noisy over time
431
+ 32. THE System SHALL expire knowledge items based on domain-specific freshness rules
432
+
433
+ #### Storage and Resource Management
434
+
435
+ 33. THE System SHALL store knowledge in backend/app/data/knowledge directory
436
+ 34. THE System SHALL store skills in backend/app/data/skills directory
437
+ 35. THE System SHALL store prompt versions in backend/app/data/prompt_versions directory
438
+ 36. THE System SHALL store learning metadata in backend/app/data/learning directory
439
+ 37. THE System SHALL compress old cases to save space
440
+ 38. THE System SHALL enforce hard storage limits and auto-cleanup policies
441
+ 39. THE System SHALL use external provider APIs for heavy reasoning, not local computation
442
+
443
+ #### Learning Scheduler
444
+
445
+ 40. THE System SHALL implement a lightweight scheduler with safeguards: one background job at a time, small batch sizes, stop on errors, respect rate limits
446
+ 41. THE System SHALL schedule learning tasks during idle periods
447
+ 42. THE System SHALL NOT interfere with user-initiated operations
448
+ 43. THE System SHALL provide manual trigger option for immediate learning runs
449
+
450
+ #### Integration with Existing Layers
451
+
452
+ 44. THE System SHALL integrate learning insights with normal MiroOrg case executions
453
+ 45. THE System SHALL integrate learning insights with domain pack intelligence
454
+ 46. THE System SHALL learn from MiroFish simulation results and outcomes
455
+ 47. THE System SHALL integrate with prompt management system
456
+ 48. THE System SHALL integrate with provider abstraction layer
457
+
458
+ #### API Endpoints
459
+
460
+ 49. THE System SHALL provide GET /learning/status endpoint for learning system status
461
+ 50. THE System SHALL provide POST /learning/run-once endpoint for manual learning trigger
462
+ 51. THE System SHALL provide GET /learning/insights endpoint for learning statistics
463
+ 52. THE System SHALL provide GET /knowledge endpoint for listing knowledge items
464
+ 53. THE System SHALL provide GET /knowledge/{item_id} endpoint for knowledge details
465
+ 54. THE System SHALL provide GET /knowledge/search endpoint for knowledge search
466
+ 55. THE System SHALL provide GET /skills endpoint for listing distilled skills
467
+ 56. THE System SHALL provide GET /skills/{skill_name} endpoint for skill details
468
+ 57. THE System SHALL provide POST /skills/distill endpoint for manual skill distillation
469
+ 58. THE System SHALL provide GET /sources/trust endpoint for source trust scores
470
+ 59. THE System SHALL provide GET /sources/freshness endpoint for source freshness scores
471
+ 60. THE System SHALL provide GET /prompts/versions/{name} endpoint for prompt version history
472
+ 61. THE System SHALL provide POST /prompts/optimize/{name} endpoint for prompt optimization
473
+ 62. THE System SHALL provide POST /prompts/promote/{name}/{version} endpoint for promoting prompt versions
.kiro/specs/ai-financial-intelligence-system/tasks.md ADDED
@@ -0,0 +1,843 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Implementation Plan: AI Financial Intelligence System (MiroOrg v1.1)
2
+
3
+ ## Overview
4
+
5
+ This implementation plan transforms MiroOrg into a general intelligence operating system that orchestrates multiple specialist agents, runs simulations, and supports pluggable domain packs. The system merges capabilities from miroorg-basic-v2 (base architecture), impact_ai (first domain pack), MiroFish (simulation lab), and public-apis (API discovery catalog) into a unified, production-ready platform.
6
+
7
+ The implementation follows 9 phases, each building on the previous while maintaining a runnable system. The focus is on single-user local deployment with production-quality code structure.
8
+
9
+ ## Tasks
10
+
11
+ ### Phase 1: Backend Consolidation and Provider Enhancement
12
+
13
+ - [x] 1. Strengthen core platform and provider abstraction
14
+ - [x] 1.1 Add OpenAI provider support to model abstraction layer
15
+ - Implement `_call_openai()` function in `backend/app/agents/_model.py`
16
+ - Add OpenAI API key configuration in `backend/app/config.py`
17
+ - Add OpenAI to provider fallback chain
18
+ - _Requirements: 6.2, 6.4_
19
+
20
+ - [x] 1.2 Enhance provider fallback logging and health checks
21
+ - Add detailed logging for provider selection events in `backend/app/agents/_model.py`
22
+ - Add logging for fallback attempts and failures
23
+ - Implement provider health checks in `backend/app/services/health_service.py`
24
+ - Add provider status to deep health endpoint
25
+ - _Requirements: 6.5, 6.6, 9.4_
26
+
27
+ - [x] 1.3 Add configuration validation on startup
28
+ - Implement config validation in `backend/app/config.py`
29
+ - Add warnings for missing optional API keys
30
+ - Add errors for missing required configuration
31
+ - Validate provider configuration completeness
32
+ - _Requirements: 1.8, 6.7, 13.5, 13.6_
33
+
34
+ - [x] 1.4 Update environment configuration files
35
+ - Add OpenAI configuration to `backend/.env.example`
36
+ - Add domain pack feature flags
37
+ - Add simulation trigger keywords configuration
38
+ - Document all environment variables
39
+ - _Requirements: 1.8, 4.6, 13.1, 13.2_
40
+
41
+ - [x] 2. Checkpoint - Verify provider abstraction
42
+ - Ensure all three providers (OpenRouter, Ollama, OpenAI) work correctly
43
+ - Verify fallback behavior with proper logging
44
+ - Verify health check reports provider status accurately
45
+ - Ask the user if questions arise
46
+
47
+ ### Phase 2: Domain Pack Architecture
48
+
49
+ - [x] 3. Create domain pack base infrastructure
50
+ - [x] 3.1 Implement domain pack base architecture
51
+ - Create `backend/app/domain_packs/__init__.py`
52
+ - Create `backend/app/domain_packs/base.py` with DomainPack abstract base class
53
+ - Define abstract methods: name, keywords, enhance_research, enhance_verification, get_capabilities
54
+ - _Requirements: 2.3, 2.5, 2.7, 5.6_
55
+
56
+ - [x] 3.2 Implement domain pack registry
57
+ - Create `backend/app/domain_packs/registry.py` with DomainPackRegistry class
58
+ - Implement register(), get_pack(), detect_domain(), list_packs(), get_capabilities()
59
+ - Create global registry instance
60
+ - _Requirements: 2.5, 5.6_
61
+
62
+ - [x] 3.3 Create finance domain pack structure
63
+ - Create `backend/app/domain_packs/finance/__init__.py`
64
+ - Create `backend/app/domain_packs/finance/pack.py` with FinanceDomainPack class
65
+ - Implement name, keywords properties for finance domain
66
+ - _Requirements: 5.1, 5.2, 5.7_
67
+
68
+ - [x] 4. Port impact_ai modules to finance domain pack
69
+ - [x] 4.1 Port market data and news modules
70
+ - Create `backend/app/domain_packs/finance/market_data.py` from impact_ai alpha_vantage_client.py
71
+ - Create `backend/app/domain_packs/finance/news.py` from impact_ai news_api.py
72
+ - Refactor to match service layer pattern
73
+ - _Requirements: 5.2, 5.5_
74
+
75
+ - [x] 4.2 Port entity and ticker resolution modules
76
+ - Create `backend/app/domain_packs/finance/entity_resolver.py`
77
+ - Create `backend/app/domain_packs/finance/ticker_resolver.py`
78
+ - Implement entity extraction and normalization
79
+ - _Requirements: 5.2, 7.2_
80
+
81
+ - [x] 4.3 Port credibility and detection modules
82
+ - Create `backend/app/domain_packs/finance/source_checker.py`
83
+ - Create `backend/app/domain_packs/finance/rumor_detector.py`
84
+ - Create `backend/app/domain_packs/finance/scam_detector.py`
85
+ - Implement credibility scoring and detection logic
86
+ - _Requirements: 5.2, 7.8_
87
+
88
+ - [x] 4.4 Port analysis and prediction modules
89
+ - Create `backend/app/domain_packs/finance/stance_detector.py`
90
+ - Create `backend/app/domain_packs/finance/event_analyzer.py`
91
+ - Create `backend/app/domain_packs/finance/prediction.py`
92
+ - Implement sentiment analysis and prediction logic
93
+ - _Requirements: 5.2_
94
+
95
+ - [x] 5. Consolidate external API clients
96
+ - [x] 5.1 Merge Alpha Vantage and NewsAPI clients
97
+ - Consolidate Alpha Vantage logic into `backend/app/services/external_sources.py`
98
+ - Consolidate NewsAPI logic into `backend/app/services/external_sources.py`
99
+ - Remove duplicate implementations
100
+ - Add connection pooling and timeouts
101
+ - _Requirements: 5.4, 9.9, 9.10, 15.3, 15.4_
102
+
103
+ - [x] 5.2 Register finance pack and update configuration
104
+ - Register FinanceDomainPack in global registry
105
+ - Add finance pack configuration to `backend/app/config.py`
106
+ - Add feature flags for domain pack enablement
107
+ - _Requirements: 5.3, 5.6_
108
+
109
+ - [x] 6. Checkpoint - Verify domain pack infrastructure
110
+ - Ensure finance pack is registered successfully
111
+ - Verify domain detection works for finance keywords
112
+ - Verify external API clients are consolidated
113
+ - Ask the user if questions arise
114
+
115
+ ### Phase 3: Agent Enhancement with Domain Intelligence
116
+
117
+ - [ ] 7. Enhance Switchboard with domain detection
118
+ - [ ] 7.1 Add domain pack dimension to routing
119
+ - Modify `backend/app/agents/switchboard.py` to add domain_pack to routing decision
120
+ - Implement domain detection using domain registry
121
+ - Update RouteDecision schema in `backend/app/schemas.py`
122
+ - _Requirements: 4.1, 5.6_
123
+
124
+ - [ ] 7.2 Implement complexity-based routing logic
125
+ - Ensure simple queries (≤5 words) route to solo mode
126
+ - Ensure medium queries (≤25 words) route to standard mode
127
+ - Ensure complex queries (>25 words) route to deep mode
128
+ - _Requirements: 4.2, 4.3, 4.4_
129
+
130
+ - [ ] 7.3 Implement simulation keyword detection
131
+ - Add simulation trigger keyword detection
132
+ - Load keywords from environment configuration
133
+ - Set task_family="simulation" when keywords detected
134
+ - _Requirements: 4.5, 4.6, 4.7_
135
+
136
+ - [ ] 8. Enhance Research Agent with domain capabilities
137
+ - [ ] 8.1 Integrate domain pack research enhancement
138
+ - Modify `backend/app/agents/research.py` to detect domain
139
+ - Call domain pack enhance_research() when domain detected
140
+ - Add structured entity extraction output
141
+ - _Requirements: 5.3, 7.1, 7.2, 7.3, 7.4_
142
+
143
+ - [ ] 8.2 Update research agent prompt
144
+ - Update `backend/app/prompts/research.txt` with domain intelligence instructions
145
+ - Add instructions for entity and ticker extraction
146
+ - Add instructions for structured output
147
+ - _Requirements: 7.14_
148
+
149
+ - [ ] 9. Enhance Verifier Agent with domain capabilities
150
+ - [ ] 9.1 Integrate domain pack verification enhancement
151
+ - Modify `backend/app/agents/verifier.py` to detect domain
152
+ - Call domain pack enhance_verification() when domain detected
153
+ - Add structured credibility scoring output
154
+ - _Requirements: 5.3, 7.7, 7.8, 7.9_
155
+
156
+ - [ ] 9.2 Update verifier agent prompt
157
+ - Update `backend/app/prompts/verifier.txt` with domain intelligence instructions
158
+ - Add instructions for rumor and scam detection
159
+ - Add instructions for uncertainty surfacing
160
+ - _Requirements: 7.14_
161
+
162
+ - [ ] 10. Enhance Planner and Synthesizer Agents
163
+ - [ ] 10.1 Add simulation mode suggestion to Planner
164
+ - Modify `backend/app/agents/planner.py` to detect simulation opportunities
165
+ - Add simulation_suggested field to output
166
+ - Update `backend/app/prompts/planner.txt` with simulation guidance
167
+ - _Requirements: 7.6, 7.14_
168
+
169
+ - [ ] 10.2 Add uncertainty quantification to Synthesizer
170
+ - Modify `backend/app/agents/synthesizer.py` to quantify uncertainty
171
+ - Add simulation recommendation logic
172
+ - Update `backend/app/prompts/synthesizer.txt` with uncertainty instructions
173
+ - _Requirements: 7.11, 7.12, 7.13, 7.14_
174
+
175
+ - [ ] 11. Update graph execution with domain context
176
+ - [ ] 11.1 Pass domain pack context through pipeline
177
+ - Modify `backend/app/graph.py` to detect domain early
178
+ - Pass domain context to all agents
179
+ - Ensure domain-enhanced execution flows correctly
180
+ - _Requirements: 2.5, 5.3, 7.15_
181
+
182
+ - [ ] 12. Checkpoint - Verify agent enhancements
183
+ - Ensure Switchboard detects finance domain correctly
184
+ - Verify Research agent extracts entities and tickers
185
+ - Verify Verifier agent scores credibility
186
+ - Verify agents suggest simulation mode appropriately
187
+ - Ask the user if questions arise
188
+
189
+ ### Phase 4: Simulation Integration Enhancement
190
+
191
+ - [ ] 13. Enhance simulation workflow and case linking
192
+ - [ ] 13.1 Add case linking to simulation router
193
+ - Modify `backend/app/routers/simulation.py` to link case_id
194
+ - Improve error messages for MiroFish failures
195
+ - Add better status reporting
196
+ - _Requirements: 8.1, 8.11, 8.12_
197
+
198
+ - [ ] 13.2 Enhance simulation store with search and filtering
199
+ - Modify `backend/app/services/simulation_store.py`
200
+ - Add simulation search by title or prediction_goal
201
+ - Add simulation filtering by status
202
+ - _Requirements: 8.10_
203
+
204
+ - [ ] 13.3 Update case storage for simulation linking
205
+ - Modify `backend/app/services/case_store.py` to add simulation_id field
206
+ - Add case-to-simulation lookup functionality
207
+ - Update CaseRecord schema in `backend/app/schemas.py`
208
+ - _Requirements: 10.8, 14.9_
209
+
210
+ - [ ] 13.4 Add simulation workflow to graph execution
211
+ - Modify `backend/app/graph.py` to add simulation handoff logic
212
+ - Add simulation result synthesis
213
+ - Ensure simulation results flow into final answer
214
+ - _Requirements: 3.4, 8.13_
215
+
216
+ - [ ] 14. Checkpoint - Verify simulation integration
217
+ - Ensure simulation requests create linked cases
218
+ - Verify cases with simulations show simulation_id
219
+ - Verify simulation results are synthesized correctly
220
+ - Ask the user if questions arise
221
+
222
+ ### Phase 5: API Discovery Subsystem
223
+
224
+ - [ ] 15. Create API discovery infrastructure
225
+ - [ ] 15.1 Create API discovery structure
226
+ - Create `backend/app/services/api_discovery/__init__.py`
227
+ - Create `backend/app/services/api_discovery/catalog_loader.py`
228
+ - Create `backend/app/services/api_discovery/classifier.py`
229
+ - Create `backend/app/services/api_discovery/scorer.py`
230
+ - Create `backend/app/services/api_discovery/metadata_store.py`
231
+ - _Requirements: 9.3, 9.4_
232
+
233
+ - [ ] 15.2 Implement catalog loader
234
+ - Implement load_public_apis_catalog() to fetch from GitHub or local cache
235
+ - Parse API entries with name, description, auth, HTTPS, CORS, category, link
236
+ - _Requirements: 9.3, 9.6_
237
+
238
+ - [ ] 15.3 Implement API classifier and scorer
239
+ - Implement classify_api() to categorize APIs by domain
240
+ - Implement score_api_usefulness() to prioritize APIs for integration
241
+ - Consider auth simplicity, HTTPS, CORS, category relevance
242
+ - _Requirements: 9.5, 9.6_
243
+
244
+ - [ ]* 15.4 Add optional discovery endpoints
245
+ - Add `GET /api-discovery/categories` endpoint
246
+ - Add `GET /api-discovery/search?category=X` endpoint
247
+ - Add `GET /api-discovery/top-scored` endpoint
248
+ - _Requirements: 9.4_
249
+
250
+ - [ ] 16. Checkpoint - Verify API discovery
251
+ - Ensure catalog loads successfully
252
+ - Verify APIs are classified correctly
253
+ - Verify scoring produces reasonable priorities
254
+ - Ask the user if questions arise
255
+
256
+ ### Phase 6: Frontend Enhancement
257
+
258
+ - [ ] 17. Create layout and navigation infrastructure
259
+ - [ ] 17.1 Create layout components
260
+ - Create `frontend/src/components/layout/Header.tsx` with branding and navigation
261
+ - Create `frontend/src/components/layout/Navigation.tsx` with tab navigation
262
+ - Modify `frontend/src/app/layout.tsx` to use new layout components
263
+ - _Requirements: 12.1, 12.2_
264
+
265
+ - [ ] 17.2 Create common UI components
266
+ - Create `frontend/src/components/common/Badge.tsx` for status indicators
267
+ - Create `frontend/src/components/common/Card.tsx` for content containers
268
+ - Create `frontend/src/components/common/LoadingSpinner.tsx` for loading states
269
+ - Create `frontend/src/components/common/ErrorMessage.tsx` for error display
270
+ - _Requirements: 12.10, 12.11, 12.15_
271
+
272
+ - [ ] 17.3 Create API client and type definitions
273
+ - Create `frontend/src/lib/api.ts` with MiroOrgClient class
274
+ - Implement methods for all backend endpoints
275
+ - Create `frontend/src/lib/types.ts` with TypeScript interfaces
276
+ - _Requirements: 11.21_
277
+
278
+ - [ ] 18. Create Main Dashboard page
279
+ - [ ] 18.1 Implement dashboard with system overview
280
+ - Modify `frontend/src/app/page.tsx` to show quick stats
281
+ - Display recent cases summary
282
+ - Display system health status
283
+ - Add navigation to main features
284
+ - _Requirements: 12.2_
285
+
286
+ - [ ] 19. Create Analyze page and components
287
+ - [ ] 19.1 Create Analyze page structure
288
+ - Create `frontend/src/app/analyze/page.tsx` with analysis interface
289
+ - Create `frontend/src/components/analyze/TaskInput.tsx` for user input
290
+ - Create `frontend/src/components/analyze/ModeSelector.tsx` for mode selection
291
+ - _Requirements: 12.3, 12.7, 12.8_
292
+
293
+ - [ ] 19.2 Create result display components
294
+ - Create `frontend/src/components/analyze/ResultViewer.tsx` for final answers
295
+ - Create `frontend/src/components/analyze/AgentOutputPanel.tsx` for agent outputs
296
+ - Display route/debug badges and confidence indicators
297
+ - _Requirements: 12.9, 12.10, 12.11, 12.14_
298
+
299
+ - [ ] 20. Create Cases page and components
300
+ - [ ] 20.1 Create Cases history interface
301
+ - Create `frontend/src/app/cases/page.tsx` with case list
302
+ - Create `frontend/src/components/cases/CaseList.tsx` for listing cases
303
+ - Create `frontend/src/components/cases/CaseCard.tsx` for case preview
304
+ - _Requirements: 12.4, 12.17_
305
+
306
+ - [ ] 20.2 Create Case detail view
307
+ - Create `frontend/src/app/cases/[id]/page.tsx` for case details
308
+ - Create `frontend/src/components/cases/CaseDetail.tsx` for full case display
309
+ - Display case_id, routing decision, agent outputs, timestamps
310
+ - _Requirements: 12.17_
311
+
312
+ - [ ] 21. Create Simulation page and components
313
+ - [ ] 21.1 Create Simulation submission interface
314
+ - Create `frontend/src/app/simulation/page.tsx` with simulation form
315
+ - Create `frontend/src/components/simulation/SimulationForm.tsx` for input
316
+ - Create `frontend/src/components/simulation/SimulationStatus.tsx` for status display
317
+ - _Requirements: 12.6, 12.13_
318
+
319
+ - [ ] 21.2 Create Simulation detail and chat interface
320
+ - Create `frontend/src/app/simulation/[id]/page.tsx` for simulation details
321
+ - Create `frontend/src/components/simulation/SimulationReport.tsx` for report display
322
+ - Create `frontend/src/components/simulation/SimulationChat.tsx` for post-simulation chat
323
+ - _Requirements: 12.13_
324
+
325
+ - [ ] 22. Create Prompt Lab and Config pages
326
+ - [ ] 22.1 Create Prompt Lab interface
327
+ - Create `frontend/src/app/prompts/page.tsx` with prompt management
328
+ - Create `frontend/src/components/prompts/PromptList.tsx` for listing prompts
329
+ - Create `frontend/src/components/prompts/PromptEditor.tsx` for editing
330
+ - _Requirements: 12.5_
331
+
332
+ - [ ] 22.2 Create Config page
333
+ - Create `frontend/src/app/config/page.tsx` with system configuration view
334
+ - Display provider status, feature flags, health checks
335
+ - _Requirements: 12.1_
336
+
337
+ - [ ] 23. Implement dark theme and styling
338
+ - [ ] 23.1 Update global styles with dark theme
339
+ - Modify `frontend/src/app/globals.css` with dark color palette
340
+ - Implement card-based structure with subtle borders
341
+ - Add animations and transitions
342
+ - Use Inter font family
343
+ - _Requirements: 12.15, 12.16_
344
+
345
+ - [ ] 24. Checkpoint - Verify frontend functionality
346
+ - Ensure all pages are accessible via navigation
347
+ - Verify Analyze workflow works end-to-end
348
+ - Verify Case history displays correctly
349
+ - Verify Simulation workflow works end-to-end
350
+ - Verify Prompt lab allows editing
351
+ - Verify Config page shows system status
352
+ - Ask the user if questions arise
353
+
354
+ ### Phase 7: Testing and Documentation
355
+
356
+ - [ ] 25. Write unit tests for core functionality
357
+ - [ ]* 25.1 Write provider abstraction tests
358
+ - Test OpenRouter, Ollama, OpenAI provider calls
359
+ - Test provider fallback behavior
360
+ - Test provider error handling
361
+ - _Requirements: 6.5, 6.6_
362
+
363
+ - [ ]* 25.2 Write domain pack tests
364
+ - Test domain pack registration
365
+ - Test domain detection
366
+ - Test finance pack capabilities
367
+ - Test entity and ticker extraction
368
+ - _Requirements: 5.6, 7.2_
369
+
370
+ - [ ]* 25.3 Write agent routing tests
371
+ - Test Switchboard classification logic
372
+ - Test complexity-to-execution-mode mapping
373
+ - Test simulation keyword detection
374
+ - Test domain detection
375
+ - _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5_
376
+
377
+ - [ ]* 25.4 Write storage tests
378
+ - Test case save and retrieve
379
+ - Test simulation save and retrieve
380
+ - Test directory auto-creation
381
+ - Test memory statistics
382
+ - _Requirements: 10.1, 10.3, 10.12_
383
+
384
+ - [ ]* 25.5 Write simulation integration tests
385
+ - Test MiroFish client adapter
386
+ - Test simulation workflow
387
+ - Test case-simulation linking
388
+ - Test error handling for disabled MiroFish
389
+ - _Requirements: 8.1, 8.11, 8.12_
390
+
391
+ - [ ] 26. Write property-based tests
392
+ - [ ]* 26.1 Write Property 1: Configuration Environment Isolation
393
+ - **Property 1: Configuration Environment Isolation**
394
+ - **Validates: Requirements 1.8, 6.7**
395
+ - Test that all configuration values come from environment variables
396
+ - Generate random config keys and verify no hardcoded values
397
+
398
+ - [ ]* 26.2 Write Property 2: Switchboard Four-Dimensional Classification
399
+ - **Property 2: Switchboard Four-Dimensional Classification**
400
+ - **Validates: Requirements 4.1**
401
+ - Test that routing decisions contain all four dimensions
402
+ - Generate random user inputs and verify structure
403
+
404
+ - [ ]* 26.3 Write Property 3: Complexity-to-Execution-Mode Mapping
405
+ - **Property 3: Complexity-to-Execution-Mode Mapping**
406
+ - **Validates: Requirements 4.2, 4.3, 4.4**
407
+ - Test that complexity maps correctly to execution mode
408
+ - Generate inputs of varying lengths and verify mapping
409
+
410
+ - [ ]* 26.4 Write Property 4: Simulation Keyword Triggering
411
+ - **Property 4: Simulation Keyword Triggering**
412
+ - **Validates: Requirements 4.5, 4.6**
413
+ - Test that simulation keywords trigger correct classification
414
+ - Generate inputs with/without keywords and verify task_family
415
+
416
+ - [ ]* 26.5 Write Property 5: Provider Fallback Behavior
417
+ - **Property 5: Provider Fallback Behavior**
418
+ - **Validates: Requirements 6.5**
419
+ - Test that provider fallback works correctly
420
+ - Mock primary provider failures and verify fallback
421
+
422
+ - [ ]* 26.6 Write Property 6: Case Persistence Round Trip
423
+ - **Property 6: Case Persistence Round Trip**
424
+ - **Validates: Requirements 10.1, 10.3**
425
+ - Test that saved cases can be retrieved correctly
426
+ - Generate random case data and verify round trip
427
+
428
+ - [ ]* 26.7 Write Property 7: Case Record Structure Completeness
429
+ - **Property 7: Case Record Structure Completeness**
430
+ - **Validates: Requirements 10.2, 10.7, 10.8**
431
+ - Test that case records contain all required fields
432
+ - Generate random cases and verify structure
433
+
434
+ - [ ]* 26.8 Write Property 8: Data Directory Organization
435
+ - **Property 8: Data Directory Organization**
436
+ - **Validates: Requirements 10.9, 10.10, 10.11**
437
+ - Test that data is stored in correct directories
438
+ - Generate random data and verify file locations
439
+
440
+ - [ ]* 26.9 Write Property 9: Directory Auto-Creation
441
+ - **Property 9: Directory Auto-Creation**
442
+ - **Validates: Requirements 10.12**
443
+ - Test that missing directories are created automatically
444
+ - Remove directories and verify auto-creation
445
+
446
+ - [ ]* 26.10 Write Property 10: MiroFish Adapter Isolation
447
+ - **Property 10: MiroFish Adapter Isolation**
448
+ - **Validates: Requirements 1.3, 3.4**
449
+ - Test that all MiroFish calls go through adapter
450
+ - Scan codebase for direct MiroFish URLs
451
+
452
+ - [ ]* 26.11 Write Property 11: Comprehensive Logging
453
+ - **Property 11: Comprehensive Logging**
454
+ - **Validates: Requirements 6.6, 9.4, 9.6, 9.7**
455
+ - Test that all operations create log entries
456
+ - Generate random operations and verify logging
457
+
458
+ - [ ]* 26.12 Write Property 12: Schema Validation
459
+ - **Property 12: Schema Validation**
460
+ - **Validates: Requirements 9.1, 9.2**
461
+ - Test that invalid requests return 422 errors
462
+ - Generate invalid request bodies and verify errors
463
+
464
+ - [ ]* 26.13 Write Property 13: External API Client Patterns
465
+ - **Property 13: External API Client Patterns**
466
+ - **Validates: Requirements 9.1, 9.9, 9.10**
467
+ - Test that external API clients use consistent patterns
468
+ - Verify connection pooling, timeouts, error handling
469
+
470
+ - [ ]* 26.14 Write Property 14: Error Response Sanitization
471
+ - **Property 14: Error Response Sanitization**
472
+ - **Validates: Requirements 9.3, 9.8, 9.10**
473
+ - Test that error responses don't leak internals
474
+ - Generate various errors and verify sanitization
475
+
476
+ - [ ]* 26.15 Write Property 15: Domain Pack Extensibility
477
+ - **Property 15: Domain Pack Extensibility**
478
+ - **Validates: Requirements 2.5, 2.7**
479
+ - Test that new domain packs don't require agent changes
480
+ - Create mock domain pack and verify integration
481
+
482
+ - [ ] 27. Write integration tests
483
+ - [ ]* 27.1 Write end-to-end case execution test
484
+ - Test complete workflow from user input to final answer
485
+ - Verify all agents execute correctly
486
+ - Verify case is saved with correct structure
487
+ - _Requirements: 3.2_
488
+
489
+ - [ ]* 27.2 Write simulation workflow test
490
+ - Test complete simulation workflow
491
+ - Verify submission, status, report, chat
492
+ - Verify case-simulation linking
493
+ - _Requirements: 3.3, 8.1_
494
+
495
+ - [ ]* 27.3 Write provider fallback integration test
496
+ - Test fallback in real execution context
497
+ - Verify system continues working with fallback provider
498
+ - _Requirements: 6.5_
499
+
500
+ - [ ]* 27.4 Write domain pack enhancement test
501
+ - Test domain-enhanced research and verification
502
+ - Verify finance pack capabilities are used
503
+ - _Requirements: 5.3, 7.1, 7.7_
504
+
505
+ - [ ] 28. Create comprehensive documentation
506
+ - [ ] 28.1 Update main README
507
+ - Update `README.md` with architecture overview
508
+ - Document four-layer architecture
509
+ - Document agent roles and responsibilities
510
+ - Add setup instructions for local development
511
+ - Add environment variable reference
512
+ - _Requirements: 13.2, 13.3, 13.4, 13.10, 13.11, 13.12_
513
+
514
+ - [ ] 28.2 Create architecture documentation
515
+ - Create `ARCHITECTURE.md` with detailed architecture description
516
+ - Document component interactions
517
+ - Document data flow
518
+ - _Requirements: 13.10_
519
+
520
+ - [ ] 28.3 Create domain pack documentation
521
+ - Create `DOMAIN_PACKS.md` with domain pack integration guide
522
+ - Document how to create new domain packs
523
+ - Document finance pack capabilities
524
+ - _Requirements: 13.13_
525
+
526
+ - [ ] 28.4 Create testing documentation
527
+ - Create `TESTING.md` with testing strategy and guidelines
528
+ - Document unit test patterns
529
+ - Document property-based test patterns
530
+ - Document integration test patterns
531
+ - _Requirements: 14.1, 14.2, 14.3_
532
+
533
+ - [ ] 28.5 Create deployment documentation
534
+ - Create `DEPLOYMENT.md` with deployment instructions
535
+ - Document environment setup
536
+ - Document dependency installation
537
+ - Document running backend and frontend
538
+ - _Requirements: 13.4, 13.5, 13.6, 13.7, 13.8_
539
+
540
+ - [ ] 29. Checkpoint - Verify testing and documentation
541
+ - Ensure all tests pass
542
+ - Verify coverage meets goals (70%+ overall)
543
+ - Verify documentation is complete and accurate
544
+ - Verify setup instructions work for new developers
545
+ - Ask the user if questions arise
546
+
547
+ ### Phase 8: Cleanup and Optimization
548
+
549
+ - [ ] 30. Remove dead code and optimize performance
550
+ - [ ] 30.1 Clean up codebase
551
+ - Remove unused imports across all files
552
+ - Remove commented code
553
+ - Remove duplicate implementations
554
+ - _Requirements: 1.5_
555
+
556
+ - [ ] 30.2 Optimize external API performance
557
+ - Add caching for market quotes with 5 minute TTL
558
+ - Verify connection pooling is implemented
559
+ - Verify request timeouts are configured
560
+ - Add rate limiting for external APIs
561
+ - _Requirements: 15.3, 15.4, 15.5, 15.6_
562
+
563
+ - [ ] 30.3 Polish error messages and logging
564
+ - Review all error messages for clarity and consistency
565
+ - Review log levels for appropriateness
566
+ - Add missing log entries for key operations
567
+ - _Requirements: 9.3, 9.8_
568
+
569
+ - [ ] 30.4 Security review
570
+ - Verify no API keys in source code
571
+ - Verify error messages don't leak internals
572
+ - Verify input validation is comprehensive
573
+ - Verify all external API calls use HTTPS
574
+ - _Requirements: 13.1, 13.2, 13.3, 13.7, 13.8, 13.9_
575
+
576
+ - [ ]* 30.5 Performance testing
577
+ - Test response times for simple queries (target: <5s)
578
+ - Test response times for complex queries (target: <30s)
579
+ - Identify and address bottlenecks
580
+ - _Requirements: 15.1, 15.2_
581
+
582
+ - [ ] 31. Final checkpoint - System verification
583
+ - Ensure no dead code remains
584
+ - Verify performance meets requirements
585
+ - Verify error messages are clear and consistent
586
+ - Verify logging is comprehensive
587
+ - Verify security review passes
588
+ - Ask the user if questions arise
589
+
590
+ ### Phase 9: Autonomous Knowledge Evolution Layer
591
+
592
+ - [ ] 32. Create learning subsystem infrastructure
593
+ - [ ] 32.1 Create learning service structure
594
+ - Create `backend/app/services/learning/__init__.py`
595
+ - Create `backend/app/services/learning/knowledge_ingestor.py`
596
+ - Create `backend/app/services/learning/knowledge_store.py`
597
+ - Create `backend/app/services/learning/learning_engine.py`
598
+ - _Requirements: 17.1, 17.2, 17.3_
599
+
600
+ - [ ] 32.2 Create additional learning services
601
+ - Create `backend/app/services/learning/prompt_optimizer.py`
602
+ - Create `backend/app/services/learning/skill_distiller.py`
603
+ - Create `backend/app/services/learning/trust_manager.py`
604
+ - Create `backend/app/services/learning/freshness_manager.py`
605
+ - Create `backend/app/services/learning/scheduler.py`
606
+ - _Requirements: 17.1, 17.17, 17.23, 17.28, 17.40_
607
+
608
+ - [ ] 32.3 Create data directories
609
+ - Create `backend/app/data/knowledge/` directory
610
+ - Create `backend/app/data/skills/` directory
611
+ - Create `backend/app/data/prompt_versions/` directory
612
+ - Create `backend/app/data/learning/` directory
613
+ - _Requirements: 17.33, 17.34, 17.35, 17.36_
614
+
615
+ - [ ] 33. Implement knowledge ingestion and storage
616
+ - [ ] 33.1 Implement knowledge ingestion
617
+ - Implement ingest_from_search() using Tavily API
618
+ - Implement ingest_from_url() using Jina Reader
619
+ - Implement ingest_from_news() using NewsAPI
620
+ - Implement compress_content() for summarization (2-4KB limit)
621
+ - _Requirements: 17.8, 17.9, 17.10, 17.11_
622
+
623
+ - [ ] 33.2 Implement knowledge store
624
+ - Implement save_knowledge() with JSON storage
625
+ - Implement get_knowledge() and search_knowledge()
626
+ - Implement delete_expired_knowledge() with auto-cleanup
627
+ - Implement storage limit enforcement (200MB max)
628
+ - Implement LRU eviction when limit reached
629
+ - _Requirements: 17.4, 17.5, 17.33, 17.38_
630
+
631
+ - [ ] 33.3 Add knowledge schemas
632
+ - Add KnowledgeItem schema to `backend/app/schemas.py`
633
+ - Add validation for summary length (2-4KB)
634
+ - Add trust_score and freshness_score fields
635
+ - _Requirements: 17.9_
636
+
637
+ - [ ] 34. Implement experience learning
638
+ - [ ] 34.1 Implement case learning
639
+ - Implement learn_from_case() to extract metadata
640
+ - Implement detect_patterns() for repeated patterns
641
+ - Implement get_route_effectiveness() for routing insights
642
+ - Implement get_prompt_performance() for prompt insights
643
+ - _Requirements: 17.13, 17.14, 17.15, 17.16_
644
+
645
+ - [ ] 34.2 Add case learning schemas
646
+ - Add CaseLearning schema to `backend/app/schemas.py`
647
+ - Add fields for route_effectiveness, prompt_performance, provider_reliability
648
+ - _Requirements: 17.13_
649
+
650
+ - [ ] 34.3 Hook learning into case save flow
651
+ - Modify `backend/app/services/case_store.py` to call learn_from_case()
652
+ - Store case learning metadata separately
653
+ - _Requirements: 17.44_
654
+
655
+ - [ ] 35. Implement prompt evolution
656
+ - [ ] 35.1 Implement prompt versioning
657
+ - Implement create_prompt_variant() using provider API
658
+ - Implement test_prompt_variant() with quality metrics
659
+ - Implement compare_prompts() for A/B testing
660
+ - Implement promote_prompt() with validation
661
+ - Implement archive_prompt() for old versions
662
+ - _Requirements: 17.17, 17.18, 17.19, 17.20, 17.21, 17.22_
663
+
664
+ - [ ] 35.2 Add prompt version schemas
665
+ - Add PromptVersion schema to `backend/app/schemas.py`
666
+ - Add fields for version, status, win_rate, test_count
667
+ - _Requirements: 17.17_
668
+
669
+ - [ ] 35.3 Integrate with prompt management
670
+ - Hook prompt versions into prompt loading
671
+ - Store prompt history in prompt_versions directory
672
+ - _Requirements: 17.47_
673
+
674
+ - [ ] 36. Implement skill distillation
675
+ - [ ] 36.1 Implement skill detection and creation
676
+ - Implement detect_skill_candidates() from patterns
677
+ - Implement distill_skill() to create skill records
678
+ - Implement test_skill() for validation
679
+ - Implement apply_skill() for skill usage
680
+ - _Requirements: 17.23, 17.24, 17.25, 17.26, 17.27_
681
+
682
+ - [ ] 36.2 Add skill schemas
683
+ - Add Skill schema to `backend/app/schemas.py`
684
+ - Add fields for trigger_patterns, recommended_agents, preferred_sources
685
+ - _Requirements: 17.24_
686
+
687
+ - [ ] 36.3 Integrate skills with agents
688
+ - Hook skill application into agent execution
689
+ - Store skills in skills directory
690
+ - _Requirements: 17.45_
691
+
692
+ - [ ] 37. Implement trust and freshness management
693
+ - [ ] 37.1 Implement trust management
694
+ - Implement get_trust_score() and update_trust()
695
+ - Implement list_trusted_sources() and list_untrusted_sources()
696
+ - Track verification outcomes
697
+ - _Requirements: 17.28, 17.29, 17.30, 17.31, 17.32_
698
+
699
+ - [ ] 37.2 Implement freshness management
700
+ - Implement calculate_freshness() with domain-specific rules
701
+ - Implement update_freshness() and get_stale_items()
702
+ - Implement recommend_refresh() for stale items
703
+ - _Requirements: 17.28, 17.29, 17.30, 17.31_
704
+
705
+ - [ ] 37.3 Add trust and freshness schemas
706
+ - Add SourceTrust schema to `backend/app/schemas.py`
707
+ - Add FreshnessScore schema to `backend/app/schemas.py`
708
+ - _Requirements: 17.28_
709
+
710
+ - [ ] 37.4 Integrate with source selection
711
+ - Hook trust scores into research agent source selection
712
+ - Hook freshness scores into knowledge retrieval
713
+ - _Requirements: 17.46_
714
+
715
+ - [ ] 38. Implement learning scheduler
716
+ - [ ] 38.1 Implement scheduler with safeguards
717
+ - Implement schedule_task() with interval configuration
718
+ - Implement is_system_idle() to check CPU usage
719
+ - Implement is_battery_ok() to check battery level
720
+ - Implement run_once() for manual triggers
721
+ - _Requirements: 17.40, 17.41, 17.42, 17.43_
722
+
723
+ - [ ] 38.2 Add scheduled tasks
724
+ - Schedule knowledge ingestion (every 6 hours)
725
+ - Schedule expired knowledge cleanup (daily)
726
+ - Schedule pattern detection (daily)
727
+ - Schedule skill distillation (weekly)
728
+ - Schedule prompt optimization (weekly)
729
+ - _Requirements: 17.6, 17.7, 17.40_
730
+
731
+ - [ ] 38.3 Add scheduler configuration
732
+ - Add LEARNING_ENABLED flag to config
733
+ - Add KNOWLEDGE_MAX_SIZE_MB (default 200)
734
+ - Add LEARNING_SCHEDULE_INTERVAL
735
+ - Add LEARNING_BATCH_SIZE
736
+ - Add domain-specific expiration rules
737
+ - _Requirements: 17.4, 17.5, 17.6, 17.7, 17.12_
738
+
739
+ - [ ] 39. Add learning API endpoints
740
+ - [ ] 39.1 Add learning status endpoints
741
+ - Add GET /learning/status endpoint
742
+ - Add POST /learning/run-once endpoint
743
+ - Add GET /learning/insights endpoint
744
+ - _Requirements: 17.49, 17.50, 17.51_
745
+
746
+ - [ ] 39.2 Add knowledge endpoints
747
+ - Add GET /knowledge endpoint for listing
748
+ - Add GET /knowledge/{item_id} endpoint for details
749
+ - Add GET /knowledge/search endpoint with query parameter
750
+ - _Requirements: 17.52, 17.53, 17.54_
751
+
752
+ - [ ] 39.3 Add skill endpoints
753
+ - Add GET /skills endpoint for listing
754
+ - Add GET /skills/{skill_name} endpoint for details
755
+ - Add POST /skills/distill endpoint for manual distillation
756
+ - _Requirements: 17.55, 17.56, 17.57_
757
+
758
+ - [ ] 39.4 Add trust and freshness endpoints
759
+ - Add GET /sources/trust endpoint
760
+ - Add GET /sources/freshness endpoint
761
+ - _Requirements: 17.58, 17.59_
762
+
763
+ - [ ] 39.5 Add prompt evolution endpoints
764
+ - Add GET /prompts/versions/{name} endpoint
765
+ - Add POST /prompts/optimize/{name} endpoint
766
+ - Add POST /prompts/promote/{name}/{version} endpoint
767
+ - _Requirements: 17.60, 17.61, 17.62_
768
+
769
+ - [ ] 40. Integrate learning layer with existing system
770
+ - [ ] 40.1 Integrate with case execution
771
+ - Hook learn_from_case() into case save flow
772
+ - Store case learning metadata
773
+ - _Requirements: 17.44_
774
+
775
+ - [ ] 40.2 Integrate with research agent
776
+ - Hook knowledge search into research agent
777
+ - Use trust scores for source selection
778
+ - _Requirements: 17.45, 17.46_
779
+
780
+ - [ ] 40.3 Integrate with simulation
781
+ - Learn from simulation outcomes
782
+ - Store simulation insights
783
+ - _Requirements: 17.46_
784
+
785
+ - [ ] 40.4 Integrate with prompt management
786
+ - Hook prompt versions into prompt loading
787
+ - Track prompt performance
788
+ - _Requirements: 17.47_
789
+
790
+ - [ ] 41. Test and verify learning layer
791
+ - [ ]* 41.1 Test knowledge ingestion
792
+ - Test ingest_from_search() with Tavily
793
+ - Test ingest_from_url() with Jina Reader
794
+ - Test compress_content() produces 2-4KB summaries
795
+ - Test storage limit enforcement (200MB)
796
+ - _Requirements: 17.4, 17.8, 17.9, 17.10_
797
+
798
+ - [ ]* 41.2 Test experience learning
799
+ - Test learn_from_case() extracts metadata
800
+ - Test detect_patterns() finds repeated patterns
801
+ - Test trust score updates
802
+ - _Requirements: 17.13, 17.14, 17.15, 17.16_
803
+
804
+ - [ ]* 41.3 Test prompt evolution
805
+ - Test create_prompt_variant() generates improvements
806
+ - Test test_prompt_variant() measures quality
807
+ - Test promote_prompt() validates before promotion
808
+ - _Requirements: 17.17, 17.18, 17.19, 17.20_
809
+
810
+ - [ ]* 41.4 Test skill distillation
811
+ - Test detect_skill_candidates() finds patterns
812
+ - Test distill_skill() creates valid skills
813
+ - Test apply_skill() improves execution
814
+ - _Requirements: 17.23, 17.24, 17.25, 17.26_
815
+
816
+ - [ ]* 41.5 Test scheduler safeguards
817
+ - Test is_system_idle() respects CPU limits
818
+ - Test is_battery_ok() respects battery level
819
+ - Test scheduler stops on errors
820
+ - Test scheduler respects rate limits
821
+ - _Requirements: 17.6, 17.7, 17.40, 17.41, 17.42_
822
+
823
+ - [ ] 42. Final checkpoint - Learning layer verification
824
+ - Ensure learning subsystem runs without stressing laptop
825
+ - Verify knowledge cache stays under 200MB
826
+ - Verify scheduler respects battery and CPU constraints
827
+ - Verify trust scores improve source selection
828
+ - Verify prompt evolution produces better prompts
829
+ - Verify skills are distilled from repeated patterns
830
+ - Verify learning endpoints return useful insights
831
+ - Verify system improves over time
832
+ - Ask the user if questions arise
833
+
834
+ ## Notes
835
+
836
+ - Tasks marked with `*` are optional and can be skipped for faster MVP delivery
837
+ - Each task references specific requirements for traceability
838
+ - Checkpoints ensure incremental validation and user feedback
839
+ - Property tests validate universal correctness properties with 100+ iterations
840
+ - Unit tests validate specific examples and edge cases
841
+ - The system remains runnable after each phase
842
+ - Focus is on single-user local deployment with production-quality code structure
843
+ - No enterprise features (auth, Kubernetes, cloud deployment) in this phase
backend/.env.example CHANGED
@@ -1,11 +1,19 @@
 
 
 
 
 
 
1
  APP_VERSION=0.3.0
2
 
3
  # ---------- Primary model routing ----------
 
 
4
  PRIMARY_PROVIDER=openrouter
5
  FALLBACK_PROVIDER=ollama
6
 
7
  # ---------- OpenRouter ----------
8
- OPENROUTER_API_KEY=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
9
  OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
10
  OPENROUTER_CHAT_MODEL=openrouter/free
11
  OPENROUTER_REASONER_MODEL=openrouter/free
@@ -13,18 +21,34 @@ OPENROUTER_SITE_URL=http://localhost:3000
13
  OPENROUTER_APP_NAME=MiroOrg Basic
14
 
15
  # ---------- Ollama ----------
 
 
16
  OLLAMA_ENABLED=true
17
  OLLAMA_BASE_URL=http://127.0.0.1:11434/api
18
  OLLAMA_CHAT_MODEL=qwen2.5:3b-instruct
19
  OLLAMA_REASONER_MODEL=qwen2.5:3b-instruct
20
 
 
 
 
 
 
 
 
 
21
  # ---------- External research APIs ----------
 
 
 
 
22
  TAVILY_API_KEY=
23
  NEWSAPI_KEY=
24
  ALPHAVANTAGE_API_KEY=
25
  JINA_READER_BASE=https://r.jina.ai/http://
26
 
27
  # ---------- MiroFish ----------
 
 
28
  MIROFISH_ENABLED=true
29
  MIROFISH_API_BASE=http://127.0.0.1:5001
30
  MIROFISH_TIMEOUT_SECONDS=120
@@ -35,4 +59,10 @@ MIROFISH_REPORT_PATH=/simulation/{id}/report
35
  MIROFISH_CHAT_PATH=/simulation/{id}/chat
36
 
37
  # ---------- Routing ----------
38
- SIMULATION_TRIGGER_KEYWORDS=simulate,predict,what if,reaction,scenario,public opinion,policy impact,market impact,digital twin
 
 
 
 
 
 
 
1
+ # ========================================
2
+ # MiroOrg v1.1 - AI Financial Intelligence System
3
+ # Environment Configuration
4
+ # ========================================
5
+
6
+ # ---------- Application Version ----------
7
  APP_VERSION=0.3.0
8
 
9
  # ---------- Primary model routing ----------
10
+ # PRIMARY_PROVIDER: The main LLM provider to use (openrouter, ollama, or openai)
11
+ # FALLBACK_PROVIDER: The backup provider if primary fails (openrouter, ollama, or openai)
12
  PRIMARY_PROVIDER=openrouter
13
  FALLBACK_PROVIDER=ollama
14
 
15
  # ---------- OpenRouter ----------
16
+ OPENROUTER_API_KEY=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
17
  OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
18
  OPENROUTER_CHAT_MODEL=openrouter/free
19
  OPENROUTER_REASONER_MODEL=openrouter/free
 
21
  OPENROUTER_APP_NAME=MiroOrg Basic
22
 
23
  # ---------- Ollama ----------
24
+ # Ollama provides local LLM inference
25
+ # Install from: https://ollama.ai
26
  OLLAMA_ENABLED=true
27
  OLLAMA_BASE_URL=http://127.0.0.1:11434/api
28
  OLLAMA_CHAT_MODEL=qwen2.5:3b-instruct
29
  OLLAMA_REASONER_MODEL=qwen2.5:3b-instruct
30
 
31
+ # ---------- OpenAI ----------
32
+ # OpenAI provides GPT models
33
+ # Get your API key from: https://platform.openai.com/api-keys
34
+ OPENAI_API_KEY=
35
+ OPENAI_BASE_URL=https://api.openai.com/v1
36
+ OPENAI_CHAT_MODEL=gpt-4o-mini
37
+ OPENAI_REASONER_MODEL=gpt-4o
38
+
39
  # ---------- External research APIs ----------
40
+ # Tavily: AI-powered web search API - https://tavily.com
41
+ # NewsAPI: News aggregation API - https://newsapi.org
42
+ # Alpha Vantage: Financial data API - https://www.alphavantage.co
43
+ # Jina Reader: Web content extraction - https://jina.ai
44
  TAVILY_API_KEY=
45
  NEWSAPI_KEY=
46
  ALPHAVANTAGE_API_KEY=
47
  JINA_READER_BASE=https://r.jina.ai/http://
48
 
49
  # ---------- MiroFish ----------
50
+ # MiroFish is the simulation service for scenario modeling
51
+ # Repository: https://github.com/yourusername/mirofish (update with actual URL)
52
  MIROFISH_ENABLED=true
53
  MIROFISH_API_BASE=http://127.0.0.1:5001
54
  MIROFISH_TIMEOUT_SECONDS=120
 
59
  MIROFISH_CHAT_PATH=/simulation/{id}/chat
60
 
61
  # ---------- Routing ----------
62
+ # Comma-separated list of keywords that trigger simulation mode
63
+ # Examples: simulate, predict, what if, reaction, scenario, public opinion, policy impact, market impact, digital twin
64
+ SIMULATION_TRIGGER_KEYWORDS=simulate,predict,what if,reaction,scenario,public opinion,policy impact,market impact,digital twin
65
+
66
+ # ---------- Domain Packs ----------
67
+ # Enable/disable domain packs (future feature)
68
+ FINANCE_DOMAIN_PACK_ENABLED=true
backend/app/agents/_model.py CHANGED
@@ -1,4 +1,5 @@
1
  from typing import Optional, List, Dict, Any
 
2
 
3
  import httpx
4
 
@@ -15,8 +16,14 @@ from app.config import (
15
  OLLAMA_BASE_URL,
16
  OLLAMA_CHAT_MODEL,
17
  OLLAMA_REASONER_MODEL,
 
 
 
 
18
  )
19
 
 
 
20
 
21
  class LLMProviderError(Exception):
22
  pass
@@ -30,6 +37,10 @@ def _pick_ollama_model(mode: str) -> str:
30
  return OLLAMA_REASONER_MODEL if mode == "reasoner" else OLLAMA_CHAT_MODEL
31
 
32
 
 
 
 
 
33
  def _build_messages(prompt: str, system_prompt: Optional[str] = None) -> List[Dict[str, str]]:
34
  messages: List[Dict[str, str]] = []
35
  if system_prompt:
@@ -87,6 +98,30 @@ def _call_ollama(prompt: str, mode: str = "chat", system_prompt: Optional[str] =
87
  return str(message.get("content", "")).strip()
88
 
89
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
90
  def call_model(
91
  prompt: str,
92
  mode: str = "chat",
@@ -94,26 +129,48 @@ def call_model(
94
  provider_override: Optional[str] = None,
95
  ) -> str:
96
  provider = (provider_override or PRIMARY_PROVIDER).lower()
 
97
 
98
  try:
99
  if provider == "openrouter":
100
- return _call_openrouter(prompt, mode=mode, system_prompt=system_prompt)
 
 
101
  if provider == "ollama":
102
- return _call_ollama(prompt, mode=mode, system_prompt=system_prompt)
 
 
 
 
 
 
103
  raise LLMProviderError(f"Unsupported provider: {provider}")
104
  except Exception as primary_error:
 
105
  fallback = FALLBACK_PROVIDER.lower()
106
  if fallback == provider:
 
107
  raise LLMProviderError(str(primary_error))
108
 
 
109
  try:
110
  if fallback == "ollama":
111
- return _call_ollama(prompt, mode=mode, system_prompt=system_prompt)
 
 
112
  if fallback == "openrouter":
113
- return _call_openrouter(prompt, mode=mode, system_prompt=system_prompt)
 
 
 
 
 
 
114
  except Exception as fallback_error:
 
115
  raise LLMProviderError(
116
  f"Primary provider failed: {primary_error} | Fallback failed: {fallback_error}"
117
  )
118
 
 
119
  raise LLMProviderError(str(primary_error))
 
1
  from typing import Optional, List, Dict, Any
2
+ import logging
3
 
4
  import httpx
5
 
 
16
  OLLAMA_BASE_URL,
17
  OLLAMA_CHAT_MODEL,
18
  OLLAMA_REASONER_MODEL,
19
+ OPENAI_API_KEY,
20
+ OPENAI_BASE_URL,
21
+ OPENAI_CHAT_MODEL,
22
+ OPENAI_REASONER_MODEL,
23
  )
24
 
25
+ logger = logging.getLogger(__name__)
26
+
27
 
28
  class LLMProviderError(Exception):
29
  pass
 
37
  return OLLAMA_REASONER_MODEL if mode == "reasoner" else OLLAMA_CHAT_MODEL
38
 
39
 
40
+ def _pick_openai_model(mode: str) -> str:
41
+ return OPENAI_REASONER_MODEL if mode == "reasoner" else OPENAI_CHAT_MODEL
42
+
43
+
44
  def _build_messages(prompt: str, system_prompt: Optional[str] = None) -> List[Dict[str, str]]:
45
  messages: List[Dict[str, str]] = []
46
  if system_prompt:
 
98
  return str(message.get("content", "")).strip()
99
 
100
 
101
+ def _call_openai(prompt: str, mode: str = "chat", system_prompt: Optional[str] = None) -> str:
102
+ if not OPENAI_API_KEY:
103
+ raise LLMProviderError("OPENAI_API_KEY is missing.")
104
+
105
+ headers = {
106
+ "Authorization": f"Bearer {OPENAI_API_KEY}",
107
+ "Content-Type": "application/json",
108
+ }
109
+
110
+ payload = {
111
+ "model": _pick_openai_model(mode),
112
+ "messages": _build_messages(prompt, system_prompt=system_prompt),
113
+ }
114
+
115
+ with httpx.Client(timeout=90) as client:
116
+ response = client.post(f"{OPENAI_BASE_URL}/chat/completions", headers=headers, json=payload)
117
+
118
+ if response.status_code >= 400:
119
+ raise LLMProviderError(f"OpenAI error {response.status_code}: {response.text}")
120
+
121
+ data = response.json()
122
+ return data["choices"][0]["message"]["content"].strip()
123
+
124
+
125
  def call_model(
126
  prompt: str,
127
  mode: str = "chat",
 
129
  provider_override: Optional[str] = None,
130
  ) -> str:
131
  provider = (provider_override or PRIMARY_PROVIDER).lower()
132
+ logger.info(f"Calling model with provider={provider}, mode={mode}")
133
 
134
  try:
135
  if provider == "openrouter":
136
+ result = _call_openrouter(prompt, mode=mode, system_prompt=system_prompt)
137
+ logger.info(f"Provider {provider} succeeded")
138
+ return result
139
  if provider == "ollama":
140
+ result = _call_ollama(prompt, mode=mode, system_prompt=system_prompt)
141
+ logger.info(f"Provider {provider} succeeded")
142
+ return result
143
+ if provider == "openai":
144
+ result = _call_openai(prompt, mode=mode, system_prompt=system_prompt)
145
+ logger.info(f"Provider {provider} succeeded")
146
+ return result
147
  raise LLMProviderError(f"Unsupported provider: {provider}")
148
  except Exception as primary_error:
149
+ logger.warning(f"Primary provider {provider} failed: {primary_error}")
150
  fallback = FALLBACK_PROVIDER.lower()
151
  if fallback == provider:
152
+ logger.error(f"No fallback available, primary provider {provider} failed")
153
  raise LLMProviderError(str(primary_error))
154
 
155
+ logger.info(f"Attempting fallback to provider={fallback}")
156
  try:
157
  if fallback == "ollama":
158
+ result = _call_ollama(prompt, mode=mode, system_prompt=system_prompt)
159
+ logger.info(f"Fallback provider {fallback} succeeded")
160
+ return result
161
  if fallback == "openrouter":
162
+ result = _call_openrouter(prompt, mode=mode, system_prompt=system_prompt)
163
+ logger.info(f"Fallback provider {fallback} succeeded")
164
+ return result
165
+ if fallback == "openai":
166
+ result = _call_openai(prompt, mode=mode, system_prompt=system_prompt)
167
+ logger.info(f"Fallback provider {fallback} succeeded")
168
+ return result
169
  except Exception as fallback_error:
170
+ logger.error(f"Fallback provider {fallback} also failed: {fallback_error}")
171
  raise LLMProviderError(
172
  f"Primary provider failed: {primary_error} | Fallback failed: {fallback_error}"
173
  )
174
 
175
+ logger.error(f"Primary provider {provider} failed with no valid fallback")
176
  raise LLMProviderError(str(primary_error))
backend/app/agents/switchboard.py CHANGED
@@ -1,28 +1,58 @@
1
  from app.config import SIMULATION_TRIGGER_KEYWORDS
 
2
 
3
 
4
  def decide_route(user_input: str) -> dict:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  text = user_input.strip()
6
  lower = text.lower()
7
  words = len(text.split())
8
 
 
9
  task_family = "simulation" if any(k in lower for k in SIMULATION_TRIGGER_KEYWORDS) else "normal"
10
 
 
 
 
 
 
 
11
  if task_family == "simulation":
12
- execution_mode = "deep"
13
  complexity = "complex"
14
  elif words <= 5:
15
- execution_mode = "solo"
16
  complexity = "simple"
17
  elif words <= 25:
18
- execution_mode = "standard"
19
  complexity = "medium"
20
  else:
21
- execution_mode = "deep"
22
  complexity = "complex"
23
 
 
 
 
 
 
 
 
 
 
 
24
  return {
25
  "task_family": task_family,
 
26
  "complexity": complexity,
27
  "execution_mode": execution_mode,
28
  "risk_level": "medium" if execution_mode == "deep" else "low",
 
1
  from app.config import SIMULATION_TRIGGER_KEYWORDS
2
+ from app.domain_packs.registry import get_registry
3
 
4
 
5
  def decide_route(user_input: str) -> dict:
6
+ """
7
+ Classify task and determine execution path.
8
+
9
+ Classification dimensions:
10
+ 1. task_family: "normal" or "simulation"
11
+ 2. domain_pack: "finance", "general", "policy", "custom"
12
+ 3. complexity: "simple" (≤5 words), "medium" (≤25 words), "complex" (>25 words)
13
+ 4. execution_mode: "solo", "standard", "deep"
14
+
15
+ Args:
16
+ user_input: The user's query
17
+
18
+ Returns:
19
+ Dictionary with routing decision including all four dimensions
20
+ """
21
  text = user_input.strip()
22
  lower = text.lower()
23
  words = len(text.split())
24
 
25
+ # Dimension 1: Task family (simulation detection)
26
  task_family = "simulation" if any(k in lower for k in SIMULATION_TRIGGER_KEYWORDS) else "normal"
27
 
28
+ # Dimension 2: Domain pack detection
29
+ registry = get_registry()
30
+ detected_domain = registry.detect_domain(user_input)
31
+ domain_pack = detected_domain if detected_domain else "general"
32
+
33
+ # Dimension 3: Complexity based on word count
34
  if task_family == "simulation":
 
35
  complexity = "complex"
36
  elif words <= 5:
 
37
  complexity = "simple"
38
  elif words <= 25:
 
39
  complexity = "medium"
40
  else:
 
41
  complexity = "complex"
42
 
43
+ # Dimension 4: Execution mode based on complexity
44
+ if task_family == "simulation":
45
+ execution_mode = "deep"
46
+ elif complexity == "simple":
47
+ execution_mode = "solo"
48
+ elif complexity == "medium":
49
+ execution_mode = "standard"
50
+ else:
51
+ execution_mode = "deep"
52
+
53
  return {
54
  "task_family": task_family,
55
+ "domain_pack": domain_pack,
56
  "complexity": complexity,
57
  "execution_mode": execution_mode,
58
  "risk_level": "medium" if execution_mode == "deep" else "low",
backend/app/config.py CHANGED
@@ -32,6 +32,11 @@ OLLAMA_BASE_URL = os.getenv("OLLAMA_BASE_URL", "http://127.0.0.1:11434/api")
32
  OLLAMA_CHAT_MODEL = os.getenv("OLLAMA_CHAT_MODEL", "qwen2.5:3b-instruct")
33
  OLLAMA_REASONER_MODEL = os.getenv("OLLAMA_REASONER_MODEL", "qwen2.5:3b-instruct")
34
 
 
 
 
 
 
35
  TAVILY_API_KEY = os.getenv("TAVILY_API_KEY", "")
36
  NEWSAPI_KEY = os.getenv("NEWSAPI_KEY", "")
37
  ALPHAVANTAGE_API_KEY = os.getenv("ALPHAVANTAGE_API_KEY", "")
@@ -54,3 +59,87 @@ SIMULATION_TRIGGER_KEYWORDS = [
54
  ).split(",")
55
  if item.strip()
56
  ]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
  OLLAMA_CHAT_MODEL = os.getenv("OLLAMA_CHAT_MODEL", "qwen2.5:3b-instruct")
33
  OLLAMA_REASONER_MODEL = os.getenv("OLLAMA_REASONER_MODEL", "qwen2.5:3b-instruct")
34
 
35
+ OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "")
36
+ OPENAI_BASE_URL = os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1")
37
+ OPENAI_CHAT_MODEL = os.getenv("OPENAI_CHAT_MODEL", "gpt-4o-mini")
38
+ OPENAI_REASONER_MODEL = os.getenv("OPENAI_REASONER_MODEL", "gpt-4o")
39
+
40
  TAVILY_API_KEY = os.getenv("TAVILY_API_KEY", "")
41
  NEWSAPI_KEY = os.getenv("NEWSAPI_KEY", "")
42
  ALPHAVANTAGE_API_KEY = os.getenv("ALPHAVANTAGE_API_KEY", "")
 
59
  ).split(",")
60
  if item.strip()
61
  ]
62
+
63
+ # Domain pack configuration
64
+ FINANCE_DOMAIN_PACK_ENABLED = os.getenv("FINANCE_DOMAIN_PACK_ENABLED", "true").lower() == "true"
65
+
66
+
67
+ # Configuration validation
68
+ import logging
69
+ import sys
70
+
71
+ logger = logging.getLogger(__name__)
72
+
73
+
74
+ def validate_config():
75
+ """Validate configuration on startup and log warnings/errors."""
76
+ errors = []
77
+ warnings = []
78
+
79
+ # Validate primary provider configuration
80
+ primary = PRIMARY_PROVIDER.lower()
81
+ if primary not in ["openrouter", "ollama", "openai"]:
82
+ errors.append(f"PRIMARY_PROVIDER '{PRIMARY_PROVIDER}' is not supported. Must be one of: openrouter, ollama, openai")
83
+
84
+ if primary == "openrouter" and not OPENROUTER_API_KEY:
85
+ errors.append("PRIMARY_PROVIDER is 'openrouter' but OPENROUTER_API_KEY is missing")
86
+
87
+ if primary == "openai" and not OPENAI_API_KEY:
88
+ errors.append("PRIMARY_PROVIDER is 'openai' but OPENAI_API_KEY is missing")
89
+
90
+ if primary == "ollama" and not OLLAMA_ENABLED:
91
+ errors.append("PRIMARY_PROVIDER is 'ollama' but OLLAMA_ENABLED is false")
92
+
93
+ # Validate fallback provider configuration
94
+ fallback = FALLBACK_PROVIDER.lower()
95
+ if fallback not in ["openrouter", "ollama", "openai"]:
96
+ errors.append(f"FALLBACK_PROVIDER '{FALLBACK_PROVIDER}' is not supported. Must be one of: openrouter, ollama, openai")
97
+
98
+ if fallback == "openrouter" and not OPENROUTER_API_KEY:
99
+ warnings.append("FALLBACK_PROVIDER is 'openrouter' but OPENROUTER_API_KEY is missing - fallback will fail")
100
+
101
+ if fallback == "openai" and not OPENAI_API_KEY:
102
+ warnings.append("FALLBACK_PROVIDER is 'openai' but OPENAI_API_KEY is missing - fallback will fail")
103
+
104
+ if fallback == "ollama" and not OLLAMA_ENABLED:
105
+ warnings.append("FALLBACK_PROVIDER is 'ollama' but OLLAMA_ENABLED is false - fallback will fail")
106
+
107
+ # Validate optional API keys
108
+ if not TAVILY_API_KEY:
109
+ warnings.append("TAVILY_API_KEY is missing - web search functionality will be limited")
110
+
111
+ if not NEWSAPI_KEY:
112
+ warnings.append("NEWSAPI_KEY is missing - news research functionality will be limited")
113
+
114
+ if not ALPHAVANTAGE_API_KEY:
115
+ warnings.append("ALPHAVANTAGE_API_KEY is missing - financial data functionality will be limited")
116
+
117
+ # Validate MiroFish configuration
118
+ if MIROFISH_ENABLED and not MIROFISH_API_BASE:
119
+ warnings.append("MIROFISH_ENABLED is true but MIROFISH_API_BASE is missing")
120
+
121
+ # Validate data directories
122
+ try:
123
+ DATA_DIR.mkdir(parents=True, exist_ok=True)
124
+ MEMORY_DIR.mkdir(parents=True, exist_ok=True)
125
+ SIMULATION_DIR.mkdir(parents=True, exist_ok=True)
126
+ except Exception as e:
127
+ errors.append(f"Failed to create data directories: {e}")
128
+
129
+ # Log results
130
+ if errors:
131
+ logger.error("Configuration validation failed with errors:")
132
+ for error in errors:
133
+ logger.error(f" - {error}")
134
+ sys.exit(1)
135
+
136
+ if warnings:
137
+ logger.warning("Configuration validation completed with warnings:")
138
+ for warning in warnings:
139
+ logger.warning(f" - {warning}")
140
+ else:
141
+ logger.info("Configuration validation passed")
142
+
143
+
144
+ # Run validation on import (startup)
145
+ validate_config()
backend/app/domain_packs/__init__.py ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Domain Packs - Pluggable domain-specific intelligence modules.
3
+
4
+ Domain packs extend the base MiroOrg system with specialized capabilities
5
+ for specific domains (finance, healthcare, legal, etc.) without requiring
6
+ changes to the core agent architecture.
7
+ """
8
+
9
+ from app.domain_packs.base import DomainPack
10
+ from app.domain_packs.registry import DomainPackRegistry, get_registry
11
+
12
+ __all__ = ["DomainPack", "DomainPackRegistry", "get_registry"]
backend/app/domain_packs/base.py ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Base class for domain packs.
3
+
4
+ Domain packs provide specialized capabilities for specific domains
5
+ without requiring changes to core agents.
6
+ """
7
+
8
+ from abc import ABC, abstractmethod
9
+ from typing import List, Dict, Any, Optional
10
+
11
+
12
+ class DomainPack(ABC):
13
+ """Abstract base class for domain packs."""
14
+
15
+ @property
16
+ @abstractmethod
17
+ def name(self) -> str:
18
+ """Return the domain pack name (e.g., 'finance', 'healthcare')."""
19
+ pass
20
+
21
+ @property
22
+ @abstractmethod
23
+ def keywords(self) -> List[str]:
24
+ """Return keywords that trigger this domain pack."""
25
+ pass
26
+
27
+ @abstractmethod
28
+ def enhance_research(self, query: str, context: Dict[str, Any]) -> Dict[str, Any]:
29
+ """
30
+ Enhance research phase with domain-specific capabilities.
31
+
32
+ Args:
33
+ query: The user's query
34
+ context: Current research context
35
+
36
+ Returns:
37
+ Enhanced context with domain-specific data
38
+ """
39
+ pass
40
+
41
+ @abstractmethod
42
+ def enhance_verification(self, claims: List[str], context: Dict[str, Any]) -> Dict[str, Any]:
43
+ """
44
+ Enhance verification phase with domain-specific capabilities.
45
+
46
+ Args:
47
+ claims: Claims to verify
48
+ context: Current verification context
49
+
50
+ Returns:
51
+ Enhanced context with domain-specific verification
52
+ """
53
+ pass
54
+
55
+ @abstractmethod
56
+ def get_capabilities(self) -> Dict[str, Any]:
57
+ """
58
+ Return domain pack capabilities and metadata.
59
+
60
+ Returns:
61
+ Dictionary describing pack capabilities
62
+ """
63
+ pass
backend/app/domain_packs/finance/__init__.py ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ """
2
+ Finance domain pack - specialized capabilities for financial intelligence.
3
+ """
4
+
5
+ from app.domain_packs.finance.pack import FinanceDomainPack
6
+
7
+ __all__ = ["FinanceDomainPack"]
backend/app/domain_packs/finance/entity_resolver.py ADDED
@@ -0,0 +1,150 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Entity resolver for finance domain pack.
3
+
4
+ Extracts and normalizes financial entities (companies, people, organizations)
5
+ from text.
6
+ """
7
+
8
+ import re
9
+ from typing import List, Dict, Any, Set
10
+ import logging
11
+
12
+ logger = logging.getLogger(__name__)
13
+
14
+
15
+ # Common financial entity patterns
16
+ COMPANY_SUFFIXES = [
17
+ "Inc", "Corp", "Corporation", "Ltd", "Limited", "LLC", "LP", "LLP",
18
+ "Co", "Company", "Group", "Holdings", "Partners", "Capital", "Ventures",
19
+ "Technologies", "Systems", "Solutions", "Services", "Enterprises"
20
+ ]
21
+
22
+ # Known major companies (expandable)
23
+ KNOWN_COMPANIES = {
24
+ "apple": "Apple Inc.",
25
+ "microsoft": "Microsoft Corporation",
26
+ "google": "Alphabet Inc.",
27
+ "alphabet": "Alphabet Inc.",
28
+ "amazon": "Amazon.com Inc.",
29
+ "meta": "Meta Platforms Inc.",
30
+ "facebook": "Meta Platforms Inc.",
31
+ "tesla": "Tesla Inc.",
32
+ "nvidia": "NVIDIA Corporation",
33
+ "berkshire": "Berkshire Hathaway Inc.",
34
+ "jpmorgan": "JPMorgan Chase & Co.",
35
+ "visa": "Visa Inc.",
36
+ "walmart": "Walmart Inc.",
37
+ "exxon": "Exxon Mobil Corporation",
38
+ "johnson": "Johnson & Johnson",
39
+ }
40
+
41
+
42
+ def extract_entities(text: str) -> List[Dict[str, Any]]:
43
+ """
44
+ Extract financial entities from text.
45
+
46
+ Args:
47
+ text: Input text
48
+
49
+ Returns:
50
+ List of extracted entities with metadata
51
+ """
52
+ entities = []
53
+ seen: Set[str] = set()
54
+
55
+ # Extract company names with suffixes
56
+ for suffix in COMPANY_SUFFIXES:
57
+ pattern = rf'\b([A-Z][a-zA-Z&\s]+)\s+{suffix}\b'
58
+ matches = re.finditer(pattern, text)
59
+ for match in matches:
60
+ full_name = match.group(0)
61
+ if full_name not in seen:
62
+ entities.append({
63
+ "text": full_name,
64
+ "type": "company",
65
+ "confidence": 0.9,
66
+ "source": "pattern_match"
67
+ })
68
+ seen.add(full_name)
69
+
70
+ # Check for known companies
71
+ text_lower = text.lower()
72
+ for key, canonical_name in KNOWN_COMPANIES.items():
73
+ if key in text_lower and canonical_name not in seen:
74
+ entities.append({
75
+ "text": canonical_name,
76
+ "type": "company",
77
+ "confidence": 1.0,
78
+ "source": "known_entity"
79
+ })
80
+ seen.add(canonical_name)
81
+
82
+ # Extract potential CEO/executive names (capitalized names near titles)
83
+ exec_pattern = r'\b(CEO|CFO|CTO|COO|President|Chairman|Director|Executive)\s+([A-Z][a-z]+\s+[A-Z][a-z]+)'
84
+ matches = re.finditer(exec_pattern, text)
85
+ for match in matches:
86
+ title = match.group(1)
87
+ name = match.group(2)
88
+ if name not in seen:
89
+ entities.append({
90
+ "text": name,
91
+ "type": "person",
92
+ "role": title,
93
+ "confidence": 0.8,
94
+ "source": "title_pattern"
95
+ })
96
+ seen.add(name)
97
+
98
+ logger.info(f"Extracted {len(entities)} entities from text")
99
+ return entities
100
+
101
+
102
+ def normalize_company_name(name: str) -> str:
103
+ """
104
+ Normalize company name to canonical form.
105
+
106
+ Args:
107
+ name: Company name
108
+
109
+ Returns:
110
+ Normalized company name
111
+ """
112
+ # Check known companies first
113
+ name_lower = name.lower()
114
+ for key, canonical in KNOWN_COMPANIES.items():
115
+ if key in name_lower:
116
+ return canonical
117
+
118
+ # Otherwise return cleaned version
119
+ # Remove extra whitespace
120
+ normalized = " ".join(name.split())
121
+
122
+ # Capitalize properly
123
+ words = normalized.split()
124
+ normalized = " ".join(
125
+ word.upper() if word.upper() in ["LLC", "LP", "LLP", "USA", "UK"]
126
+ else word.capitalize()
127
+ for word in words
128
+ )
129
+
130
+ return normalized
131
+
132
+
133
+ def resolve_entity(entity_text: str) -> Dict[str, Any]:
134
+ """
135
+ Resolve entity to canonical form with metadata.
136
+
137
+ Args:
138
+ entity_text: Entity text to resolve
139
+
140
+ Returns:
141
+ Dictionary with resolved entity information
142
+ """
143
+ normalized = normalize_company_name(entity_text)
144
+
145
+ return {
146
+ "original": entity_text,
147
+ "normalized": normalized,
148
+ "type": "company" if any(suffix in normalized for suffix in COMPANY_SUFFIXES) else "unknown",
149
+ "confidence": 0.7,
150
+ }
backend/app/domain_packs/finance/event_analyzer.py ADDED
@@ -0,0 +1,212 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Event analyzer for finance domain pack.
3
+
4
+ Analyzes financial events and their potential market impact.
5
+ """
6
+
7
+ from typing import Dict, Any, List
8
+ import re
9
+ import logging
10
+
11
+ logger = logging.getLogger(__name__)
12
+
13
+
14
+ # Event categories and their typical impact
15
+ EVENT_CATEGORIES = {
16
+ "earnings": {
17
+ "keywords": ["earnings", "quarterly results", "q1", "q2", "q3", "q4", "eps", "revenue"],
18
+ "typical_impact": "high",
19
+ "volatility": "high",
20
+ },
21
+ "merger_acquisition": {
22
+ "keywords": ["merger", "acquisition", "takeover", "buyout", "deal"],
23
+ "typical_impact": "very_high",
24
+ "volatility": "very_high",
25
+ },
26
+ "regulatory": {
27
+ "keywords": ["sec", "investigation", "lawsuit", "fine", "penalty", "regulation"],
28
+ "typical_impact": "high",
29
+ "volatility": "high",
30
+ },
31
+ "product_launch": {
32
+ "keywords": ["launch", "release", "unveil", "announce", "new product"],
33
+ "typical_impact": "medium",
34
+ "volatility": "medium",
35
+ },
36
+ "executive_change": {
37
+ "keywords": ["ceo", "cfo", "resign", "appoint", "hire", "fire", "step down"],
38
+ "typical_impact": "medium",
39
+ "volatility": "medium",
40
+ },
41
+ "guidance": {
42
+ "keywords": ["guidance", "forecast", "outlook", "projection", "estimate"],
43
+ "typical_impact": "high",
44
+ "volatility": "high",
45
+ },
46
+ "dividend": {
47
+ "keywords": ["dividend", "payout", "distribution", "yield"],
48
+ "typical_impact": "low",
49
+ "volatility": "low",
50
+ },
51
+ "fed_policy": {
52
+ "keywords": ["federal reserve", "fed", "interest rate", "monetary policy", "fomc"],
53
+ "typical_impact": "very_high",
54
+ "volatility": "very_high",
55
+ },
56
+ }
57
+
58
+
59
+ def detect_event_type(text: str) -> List[Dict[str, Any]]:
60
+ """
61
+ Detect financial event types in text.
62
+
63
+ Args:
64
+ text: Text to analyze
65
+
66
+ Returns:
67
+ List of detected event types with metadata
68
+ """
69
+ text_lower = text.lower()
70
+ detected_events = []
71
+
72
+ for event_type, info in EVENT_CATEGORIES.items():
73
+ matches = []
74
+ for keyword in info["keywords"]:
75
+ if keyword in text_lower:
76
+ matches.append(keyword)
77
+
78
+ if matches:
79
+ detected_events.append({
80
+ "event_type": event_type,
81
+ "matched_keywords": matches,
82
+ "typical_impact": info["typical_impact"],
83
+ "volatility": info["volatility"],
84
+ "confidence": min(len(matches) * 0.3, 1.0),
85
+ })
86
+
87
+ logger.info(f"Detected {len(detected_events)} event types")
88
+ return detected_events
89
+
90
+
91
+ def analyze_event_impact(text: str, event_types: List[Dict[str, Any]] = None) -> Dict[str, Any]:
92
+ """
93
+ Analyze potential market impact of events.
94
+
95
+ Args:
96
+ text: Text describing the event
97
+ event_types: Pre-detected event types (optional)
98
+
99
+ Returns:
100
+ Impact analysis
101
+ """
102
+ if event_types is None:
103
+ event_types = detect_event_type(text)
104
+
105
+ if not event_types:
106
+ return {
107
+ "impact_level": "unknown",
108
+ "volatility_level": "unknown",
109
+ "confidence": 0.0,
110
+ }
111
+
112
+ # Aggregate impact levels
113
+ impact_scores = {
114
+ "very_high": 1.0,
115
+ "high": 0.75,
116
+ "medium": 0.5,
117
+ "low": 0.25,
118
+ "unknown": 0.0,
119
+ }
120
+
121
+ impacts = [impact_scores.get(e["typical_impact"], 0.0) for e in event_types]
122
+ avg_impact = sum(impacts) / len(impacts) if impacts else 0.0
123
+
124
+ # Determine impact level
125
+ if avg_impact >= 0.85:
126
+ impact_level = "very_high"
127
+ elif avg_impact >= 0.65:
128
+ impact_level = "high"
129
+ elif avg_impact >= 0.4:
130
+ impact_level = "medium"
131
+ else:
132
+ impact_level = "low"
133
+
134
+ # Aggregate volatility
135
+ volatility_scores = {
136
+ "very_high": 1.0,
137
+ "high": 0.75,
138
+ "medium": 0.5,
139
+ "low": 0.25,
140
+ "unknown": 0.0,
141
+ }
142
+
143
+ volatilities = [volatility_scores.get(e["volatility"], 0.0) for e in event_types]
144
+ avg_volatility = sum(volatilities) / len(volatilities) if volatilities else 0.0
145
+
146
+ if avg_volatility >= 0.85:
147
+ volatility_level = "very_high"
148
+ elif avg_volatility >= 0.65:
149
+ volatility_level = "high"
150
+ elif avg_volatility >= 0.4:
151
+ volatility_level = "medium"
152
+ else:
153
+ volatility_level = "low"
154
+
155
+ # Calculate confidence
156
+ confidences = [e["confidence"] for e in event_types]
157
+ avg_confidence = sum(confidences) / len(confidences) if confidences else 0.0
158
+
159
+ return {
160
+ "impact_level": impact_level,
161
+ "volatility_level": volatility_level,
162
+ "confidence": avg_confidence,
163
+ "detected_events": event_types,
164
+ "event_count": len(event_types),
165
+ }
166
+
167
+
168
+ def extract_event_timeline(text: str) -> List[Dict[str, Any]]:
169
+ """
170
+ Extract timeline information from event description.
171
+
172
+ Args:
173
+ text: Text to analyze
174
+
175
+ Returns:
176
+ List of timeline markers
177
+ """
178
+ timeline = []
179
+
180
+ # Date patterns
181
+ date_patterns = [
182
+ r'\b(\d{1,2}/\d{1,2}/\d{2,4})\b', # MM/DD/YYYY
183
+ r'\b(January|February|March|April|May|June|July|August|September|October|November|December)\s+\d{1,2},?\s+\d{4}\b',
184
+ r'\b(Q[1-4]\s+\d{4})\b', # Q1 2024
185
+ ]
186
+
187
+ for pattern in date_patterns:
188
+ matches = re.finditer(pattern, text, re.IGNORECASE)
189
+ for match in matches:
190
+ timeline.append({
191
+ "date_text": match.group(0),
192
+ "position": match.start(),
193
+ "type": "date",
194
+ })
195
+
196
+ # Time indicators
197
+ time_indicators = [
198
+ "today", "tomorrow", "yesterday", "next week", "next month",
199
+ "this quarter", "next quarter", "this year", "next year",
200
+ "upcoming", "soon", "recently", "last week", "last month",
201
+ ]
202
+
203
+ text_lower = text.lower()
204
+ for indicator in time_indicators:
205
+ if indicator in text_lower:
206
+ timeline.append({
207
+ "date_text": indicator,
208
+ "type": "relative_time",
209
+ })
210
+
211
+ logger.info(f"Extracted {len(timeline)} timeline markers")
212
+ return timeline
backend/app/domain_packs/finance/market_data.py ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Market data module for finance domain pack.
3
+
4
+ Provides access to market quotes, historical data, and financial metrics
5
+ via Alpha Vantage API.
6
+ """
7
+
8
+ from typing import Dict, Any, Optional
9
+ import logging
10
+
11
+ import httpx
12
+
13
+ from app.config import ALPHAVANTAGE_API_KEY
14
+
15
+ logger = logging.getLogger(__name__)
16
+
17
+
18
+ def get_quote(symbol: str) -> Dict[str, Any]:
19
+ """
20
+ Get real-time quote for a stock symbol.
21
+
22
+ Args:
23
+ symbol: Stock ticker symbol (e.g., 'AAPL', 'TSLA')
24
+
25
+ Returns:
26
+ Dictionary with quote data or empty dict if unavailable
27
+ """
28
+ if not ALPHAVANTAGE_API_KEY or not symbol:
29
+ logger.warning("Alpha Vantage API key missing or symbol empty")
30
+ return {}
31
+
32
+ try:
33
+ params = {
34
+ "function": "GLOBAL_QUOTE",
35
+ "symbol": symbol.upper(),
36
+ "apikey": ALPHAVANTAGE_API_KEY,
37
+ }
38
+ with httpx.Client(timeout=30) as client:
39
+ response = client.get("https://www.alphavantage.co/query", params=params)
40
+
41
+ if response.status_code >= 400:
42
+ logger.error(f"Alpha Vantage API error {response.status_code}")
43
+ return {}
44
+
45
+ data = response.json()
46
+ quote = data.get("Global Quote", {})
47
+
48
+ if quote:
49
+ logger.info(f"Retrieved quote for {symbol}")
50
+ else:
51
+ logger.warning(f"No quote data for {symbol}")
52
+
53
+ return quote
54
+ except Exception as e:
55
+ logger.error(f"Error fetching quote for {symbol}: {e}")
56
+ return {}
57
+
58
+
59
+ def get_company_overview(symbol: str) -> Dict[str, Any]:
60
+ """
61
+ Get company overview and fundamental data.
62
+
63
+ Args:
64
+ symbol: Stock ticker symbol
65
+
66
+ Returns:
67
+ Dictionary with company data or empty dict if unavailable
68
+ """
69
+ if not ALPHAVANTAGE_API_KEY or not symbol:
70
+ return {}
71
+
72
+ try:
73
+ params = {
74
+ "function": "OVERVIEW",
75
+ "symbol": symbol.upper(),
76
+ "apikey": ALPHAVANTAGE_API_KEY,
77
+ }
78
+ with httpx.Client(timeout=30) as client:
79
+ response = client.get("https://www.alphavantage.co/query", params=params)
80
+
81
+ if response.status_code >= 400:
82
+ return {}
83
+
84
+ data = response.json()
85
+ logger.info(f"Retrieved company overview for {symbol}")
86
+ return data
87
+ except Exception as e:
88
+ logger.error(f"Error fetching company overview for {symbol}: {e}")
89
+ return {}
90
+
91
+
92
+ def search_symbol(keywords: str) -> list[Dict[str, Any]]:
93
+ """
94
+ Search for stock symbols by company name or keywords.
95
+
96
+ Args:
97
+ keywords: Search keywords
98
+
99
+ Returns:
100
+ List of matching symbols with metadata
101
+ """
102
+ if not ALPHAVANTAGE_API_KEY or not keywords:
103
+ return []
104
+
105
+ try:
106
+ params = {
107
+ "function": "SYMBOL_SEARCH",
108
+ "keywords": keywords,
109
+ "apikey": ALPHAVANTAGE_API_KEY,
110
+ }
111
+ with httpx.Client(timeout=30) as client:
112
+ response = client.get("https://www.alphavantage.co/query", params=params)
113
+
114
+ if response.status_code >= 400:
115
+ return []
116
+
117
+ data = response.json()
118
+ matches = data.get("bestMatches", [])
119
+ logger.info(f"Found {len(matches)} symbol matches for '{keywords}'")
120
+ return matches
121
+ except Exception as e:
122
+ logger.error(f"Error searching symbols for '{keywords}': {e}")
123
+ return []
backend/app/domain_packs/finance/news.py ADDED
@@ -0,0 +1,144 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ News module for finance domain pack.
3
+
4
+ Provides access to financial news via NewsAPI.
5
+ """
6
+
7
+ from typing import List, Dict, Any
8
+ from datetime import datetime, timedelta
9
+ import logging
10
+
11
+ import httpx
12
+
13
+ from app.config import NEWSAPI_KEY
14
+
15
+ logger = logging.getLogger(__name__)
16
+
17
+
18
+ def search_news(
19
+ query: str,
20
+ page_size: int = 10,
21
+ language: str = "en",
22
+ sort_by: str = "publishedAt"
23
+ ) -> List[Dict[str, Any]]:
24
+ """
25
+ Search for news articles.
26
+
27
+ Args:
28
+ query: Search query
29
+ page_size: Number of results (max 100)
30
+ language: Language code (default: 'en')
31
+ sort_by: Sort order ('publishedAt', 'relevancy', 'popularity')
32
+
33
+ Returns:
34
+ List of news articles
35
+ """
36
+ if not NEWSAPI_KEY:
37
+ logger.warning("NewsAPI key missing")
38
+ return []
39
+
40
+ try:
41
+ params = {
42
+ "q": query,
43
+ "pageSize": min(page_size, 100),
44
+ "language": language,
45
+ "sortBy": sort_by,
46
+ "apiKey": NEWSAPI_KEY,
47
+ }
48
+ with httpx.Client(timeout=30) as client:
49
+ response = client.get("https://newsapi.org/v2/everything", params=params)
50
+
51
+ if response.status_code >= 400:
52
+ logger.error(f"NewsAPI error {response.status_code}: {response.text}")
53
+ return []
54
+
55
+ data = response.json()
56
+ articles = data.get("articles", [])
57
+ logger.info(f"Found {len(articles)} news articles for '{query}'")
58
+ return articles
59
+ except Exception as e:
60
+ logger.error(f"Error searching news for '{query}': {e}")
61
+ return []
62
+
63
+
64
+ def get_top_headlines(
65
+ category: str = "business",
66
+ country: str = "us",
67
+ page_size: int = 10
68
+ ) -> List[Dict[str, Any]]:
69
+ """
70
+ Get top headlines by category.
71
+
72
+ Args:
73
+ category: News category ('business', 'technology', etc.)
74
+ country: Country code (default: 'us')
75
+ page_size: Number of results (max 100)
76
+
77
+ Returns:
78
+ List of top headline articles
79
+ """
80
+ if not NEWSAPI_KEY:
81
+ logger.warning("NewsAPI key missing")
82
+ return []
83
+
84
+ try:
85
+ params = {
86
+ "category": category,
87
+ "country": country,
88
+ "pageSize": min(page_size, 100),
89
+ "apiKey": NEWSAPI_KEY,
90
+ }
91
+ with httpx.Client(timeout=30) as client:
92
+ response = client.get("https://newsapi.org/v2/top-headlines", params=params)
93
+
94
+ if response.status_code >= 400:
95
+ logger.error(f"NewsAPI error {response.status_code}")
96
+ return []
97
+
98
+ data = response.json()
99
+ articles = data.get("articles", [])
100
+ logger.info(f"Retrieved {len(articles)} top headlines for {category}/{country}")
101
+ return articles
102
+ except Exception as e:
103
+ logger.error(f"Error fetching top headlines: {e}")
104
+ return []
105
+
106
+
107
+ def get_company_news(company_name: str, days_back: int = 7) -> List[Dict[str, Any]]:
108
+ """
109
+ Get recent news about a specific company.
110
+
111
+ Args:
112
+ company_name: Company name to search for
113
+ days_back: Number of days to look back (default: 7)
114
+
115
+ Returns:
116
+ List of news articles about the company
117
+ """
118
+ if not NEWSAPI_KEY:
119
+ return []
120
+
121
+ try:
122
+ from_date = (datetime.now() - timedelta(days=days_back)).strftime("%Y-%m-%d")
123
+
124
+ params = {
125
+ "q": company_name,
126
+ "from": from_date,
127
+ "sortBy": "publishedAt",
128
+ "language": "en",
129
+ "pageSize": 20,
130
+ "apiKey": NEWSAPI_KEY,
131
+ }
132
+ with httpx.Client(timeout=30) as client:
133
+ response = client.get("https://newsapi.org/v2/everything", params=params)
134
+
135
+ if response.status_code >= 400:
136
+ return []
137
+
138
+ data = response.json()
139
+ articles = data.get("articles", [])
140
+ logger.info(f"Found {len(articles)} articles about '{company_name}' in last {days_back} days")
141
+ return articles
142
+ except Exception as e:
143
+ logger.error(f"Error fetching company news for '{company_name}': {e}")
144
+ return []
backend/app/domain_packs/finance/pack.py ADDED
@@ -0,0 +1,122 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Finance domain pack implementation.
3
+
4
+ Provides specialized capabilities for financial intelligence including:
5
+ - Market data integration
6
+ - Entity and ticker resolution
7
+ - Credibility scoring for financial sources
8
+ - Rumor and scam detection
9
+ - Sentiment analysis and predictions
10
+ """
11
+
12
+ from typing import List, Dict, Any
13
+ import logging
14
+
15
+ from app.domain_packs.base import DomainPack
16
+
17
+ logger = logging.getLogger(__name__)
18
+
19
+
20
+ class FinanceDomainPack(DomainPack):
21
+ """Finance domain pack for financial intelligence."""
22
+
23
+ @property
24
+ def name(self) -> str:
25
+ return "finance"
26
+
27
+ @property
28
+ def keywords(self) -> List[str]:
29
+ return [
30
+ # Markets and trading
31
+ "stock", "stocks", "market", "markets", "trading", "trader",
32
+ "equity", "equities", "shares", "ticker", "nasdaq", "nyse",
33
+ "dow", "s&p", "index", "indices",
34
+
35
+ # Financial instruments
36
+ "bond", "bonds", "derivative", "derivatives", "option", "options",
37
+ "futures", "etf", "mutual fund", "portfolio",
38
+
39
+ # Companies and entities
40
+ "earnings", "revenue", "profit", "loss", "quarterly", "annual report",
41
+ "sec filing", "10-k", "10-q", "ipo", "merger", "acquisition",
42
+
43
+ # Economic indicators
44
+ "fed", "federal reserve", "interest rate", "inflation", "gdp",
45
+ "unemployment", "jobs report", "cpi", "ppi",
46
+
47
+ # Crypto (if applicable)
48
+ "bitcoin", "ethereum", "crypto", "cryptocurrency", "blockchain",
49
+
50
+ # Financial news and events
51
+ "earnings call", "analyst", "rating", "upgrade", "downgrade",
52
+ "price target", "bull", "bear", "rally", "crash", "correction",
53
+
54
+ # Risk and compliance
55
+ "fraud", "scam", "ponzi", "insider trading", "sec investigation",
56
+ "bankruptcy", "default", "credit rating",
57
+ ]
58
+
59
+ def enhance_research(self, query: str, context: Dict[str, Any]) -> Dict[str, Any]:
60
+ """
61
+ Enhance research with finance-specific capabilities.
62
+
63
+ This will be implemented in Phase 2 Task 4 when we port impact_ai modules.
64
+ For now, return context unchanged.
65
+ """
66
+ logger.info(f"Finance pack enhancing research for query: {query[:100]}")
67
+
68
+ # Placeholder - will be implemented with impact_ai modules
69
+ enhanced = context.copy()
70
+ enhanced["domain"] = "finance"
71
+ enhanced["finance_capabilities"] = [
72
+ "market_data",
73
+ "entity_resolution",
74
+ "ticker_resolution",
75
+ "credibility_scoring",
76
+ "rumor_detection",
77
+ "scam_detection",
78
+ ]
79
+
80
+ return enhanced
81
+
82
+ def enhance_verification(self, claims: List[str], context: Dict[str, Any]) -> Dict[str, Any]:
83
+ """
84
+ Enhance verification with finance-specific capabilities.
85
+
86
+ This will be implemented in Phase 2 Task 4 when we port impact_ai modules.
87
+ For now, return context unchanged.
88
+ """
89
+ logger.info(f"Finance pack enhancing verification for {len(claims)} claims")
90
+
91
+ # Placeholder - will be implemented with impact_ai modules
92
+ enhanced = context.copy()
93
+ enhanced["domain"] = "finance"
94
+ enhanced["verification_methods"] = [
95
+ "source_credibility_check",
96
+ "rumor_detection",
97
+ "scam_detection",
98
+ "cross_reference_market_data",
99
+ ]
100
+
101
+ return enhanced
102
+
103
+ def get_capabilities(self) -> Dict[str, Any]:
104
+ """Return finance pack capabilities."""
105
+ return {
106
+ "name": self.name,
107
+ "version": "1.0.0",
108
+ "description": "Financial intelligence domain pack",
109
+ "features": [
110
+ "Market data integration (Alpha Vantage)",
111
+ "News aggregation (NewsAPI)",
112
+ "Entity and ticker resolution",
113
+ "Source credibility scoring",
114
+ "Rumor detection",
115
+ "Scam detection",
116
+ "Sentiment analysis",
117
+ "Event impact analysis",
118
+ "Price prediction support",
119
+ ],
120
+ "keywords_count": len(self.keywords),
121
+ "status": "active",
122
+ }
backend/app/domain_packs/finance/prediction.py ADDED
@@ -0,0 +1,200 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Prediction support module for finance domain pack.
3
+
4
+ Provides structured support for financial predictions and forecasts.
5
+ Note: This module does NOT make actual predictions, but helps structure
6
+ prediction-related analysis and uncertainty quantification.
7
+ """
8
+
9
+ from typing import Dict, Any, List, Optional
10
+ import logging
11
+
12
+ logger = logging.getLogger(__name__)
13
+
14
+
15
+ def structure_prediction_context(
16
+ query: str,
17
+ entities: List[Dict[str, Any]],
18
+ events: List[Dict[str, Any]],
19
+ stance: Dict[str, Any],
20
+ sources: List[str]
21
+ ) -> Dict[str, Any]:
22
+ """
23
+ Structure context for prediction-related queries.
24
+
25
+ Args:
26
+ query: User's prediction query
27
+ entities: Extracted entities
28
+ events: Detected events
29
+ stance: Market stance analysis
30
+ sources: Information sources
31
+
32
+ Returns:
33
+ Structured prediction context
34
+ """
35
+ from app.domain_packs.finance.source_checker import aggregate_source_scores
36
+
37
+ # Assess source credibility
38
+ source_assessment = aggregate_source_scores(sources)
39
+
40
+ # Determine prediction type
41
+ query_lower = query.lower()
42
+ prediction_type = "unknown"
43
+
44
+ if any(word in query_lower for word in ["price", "stock", "value", "worth"]):
45
+ prediction_type = "price_movement"
46
+ elif any(word in query_lower for word in ["earnings", "revenue", "profit"]):
47
+ prediction_type = "financial_performance"
48
+ elif any(word in query_lower for word in ["market", "sector", "industry"]):
49
+ prediction_type = "market_trend"
50
+ elif any(word in query_lower for word in ["merger", "acquisition", "deal"]):
51
+ prediction_type = "corporate_action"
52
+
53
+ # Calculate uncertainty factors
54
+ uncertainty_factors = []
55
+
56
+ if source_assessment["average_score"] < 0.7:
57
+ uncertainty_factors.append("low_source_credibility")
58
+
59
+ if stance.get("confidence", 0) < 0.6:
60
+ uncertainty_factors.append("mixed_market_sentiment")
61
+
62
+ if len(events) == 0:
63
+ uncertainty_factors.append("no_clear_catalysts")
64
+
65
+ if len(entities) == 0:
66
+ uncertainty_factors.append("unclear_target_entities")
67
+
68
+ uncertainty_level = "high" if len(uncertainty_factors) >= 3 else \
69
+ "medium" if len(uncertainty_factors) >= 1 else \
70
+ "low"
71
+
72
+ return {
73
+ "prediction_type": prediction_type,
74
+ "target_entities": entities,
75
+ "relevant_events": events,
76
+ "market_stance": stance,
77
+ "source_credibility": source_assessment,
78
+ "uncertainty_level": uncertainty_level,
79
+ "uncertainty_factors": uncertainty_factors,
80
+ "recommendation": "high_confidence_analysis" if uncertainty_level == "low" else
81
+ "moderate_confidence_analysis" if uncertainty_level == "medium" else
82
+ "low_confidence_analysis",
83
+ }
84
+
85
+
86
+ def quantify_prediction_uncertainty(
87
+ prediction_context: Dict[str, Any],
88
+ additional_factors: Optional[Dict[str, Any]] = None
89
+ ) -> Dict[str, Any]:
90
+ """
91
+ Quantify uncertainty in prediction context.
92
+
93
+ Args:
94
+ prediction_context: Structured prediction context
95
+ additional_factors: Additional uncertainty factors
96
+
97
+ Returns:
98
+ Uncertainty quantification
99
+ """
100
+ base_uncertainty = 0.5 # Start with 50% uncertainty
101
+
102
+ # Adjust based on source credibility
103
+ source_score = prediction_context.get("source_credibility", {}).get("average_score", 0.5)
104
+ base_uncertainty -= (source_score - 0.5) * 0.3
105
+
106
+ # Adjust based on market stance confidence
107
+ stance_confidence = prediction_context.get("market_stance", {}).get("confidence", 0.5)
108
+ base_uncertainty -= (stance_confidence - 0.5) * 0.2
109
+
110
+ # Adjust based on event clarity
111
+ event_count = len(prediction_context.get("relevant_events", []))
112
+ if event_count > 0:
113
+ base_uncertainty -= 0.1
114
+
115
+ # Adjust based on entity clarity
116
+ entity_count = len(prediction_context.get("target_entities", []))
117
+ if entity_count > 0:
118
+ base_uncertainty -= 0.1
119
+
120
+ # Apply additional factors
121
+ if additional_factors:
122
+ if additional_factors.get("high_volatility"):
123
+ base_uncertainty += 0.15
124
+ if additional_factors.get("conflicting_signals"):
125
+ base_uncertainty += 0.2
126
+ if additional_factors.get("limited_data"):
127
+ base_uncertainty += 0.15
128
+
129
+ # Clamp to 0-1 range
130
+ uncertainty_score = max(0.0, min(1.0, base_uncertainty))
131
+
132
+ confidence_score = 1.0 - uncertainty_score
133
+
134
+ return {
135
+ "uncertainty_score": uncertainty_score,
136
+ "confidence_score": confidence_score,
137
+ "uncertainty_level": prediction_context.get("uncertainty_level", "unknown"),
138
+ "factors": prediction_context.get("uncertainty_factors", []),
139
+ "recommendation": "proceed_with_caution" if uncertainty_score >= 0.7 else
140
+ "moderate_confidence" if uncertainty_score >= 0.4 else
141
+ "reasonable_confidence",
142
+ }
143
+
144
+
145
+ def suggest_simulation_scenarios(prediction_context: Dict[str, Any]) -> List[Dict[str, Any]]:
146
+ """
147
+ Suggest simulation scenarios based on prediction context.
148
+
149
+ Args:
150
+ prediction_context: Structured prediction context
151
+
152
+ Returns:
153
+ List of suggested simulation scenarios
154
+ """
155
+ scenarios = []
156
+
157
+ prediction_type = prediction_context.get("prediction_type", "unknown")
158
+ events = prediction_context.get("relevant_events", [])
159
+
160
+ if prediction_type == "price_movement":
161
+ scenarios.append({
162
+ "scenario": "bull_case",
163
+ "description": "Optimistic price movement scenario",
164
+ "parameters": {"sentiment": "positive", "volatility": "moderate"},
165
+ })
166
+ scenarios.append({
167
+ "scenario": "bear_case",
168
+ "description": "Pessimistic price movement scenario",
169
+ "parameters": {"sentiment": "negative", "volatility": "moderate"},
170
+ })
171
+ scenarios.append({
172
+ "scenario": "base_case",
173
+ "description": "Neutral price movement scenario",
174
+ "parameters": {"sentiment": "neutral", "volatility": "low"},
175
+ })
176
+
177
+ if prediction_type == "market_trend":
178
+ scenarios.append({
179
+ "scenario": "sector_rotation",
180
+ "description": "Capital flows between sectors",
181
+ "parameters": {"market_phase": "rotation"},
182
+ })
183
+
184
+ # Add event-specific scenarios
185
+ for event in events:
186
+ event_type = event.get("event_type")
187
+ if event_type == "earnings":
188
+ scenarios.append({
189
+ "scenario": "earnings_beat",
190
+ "description": "Company beats earnings expectations",
191
+ "parameters": {"event": "earnings", "outcome": "positive"},
192
+ })
193
+ scenarios.append({
194
+ "scenario": "earnings_miss",
195
+ "description": "Company misses earnings expectations",
196
+ "parameters": {"event": "earnings", "outcome": "negative"},
197
+ })
198
+
199
+ logger.info(f"Suggested {len(scenarios)} simulation scenarios")
200
+ return scenarios
backend/app/domain_packs/finance/rumor_detector.py ADDED
@@ -0,0 +1,138 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Rumor detector for finance domain pack.
3
+
4
+ Detects potential rumors and unverified claims in financial content.
5
+ """
6
+
7
+ import re
8
+ from typing import List, Dict, Any
9
+ import logging
10
+
11
+ logger = logging.getLogger(__name__)
12
+
13
+
14
+ # Rumor indicator patterns
15
+ RUMOR_INDICATORS = [
16
+ # Hedging language
17
+ r'\b(allegedly|reportedly|rumor|rumored|speculation|speculated|unconfirmed|unverified)\b',
18
+ r'\b(sources say|sources claim|insider|insiders|anonymous source)\b',
19
+ r'\b(could be|might be|may be|possibly|potentially)\b',
20
+
21
+ # Vague attribution
22
+ r'\b(some say|people say|word is|buzz is|chatter|whispers)\b',
23
+ r'\b(according to rumors|according to speculation)\b',
24
+
25
+ # Sensational language
26
+ r'\b(shocking|bombshell|explosive|leaked|secret)\b',
27
+ ]
28
+
29
+ # Verification indicators (opposite of rumors)
30
+ VERIFICATION_INDICATORS = [
31
+ r'\b(confirmed|verified|official|announced|disclosed|filed)\b',
32
+ r'\b(sec filing|press release|earnings report|official statement)\b',
33
+ r'\b(ceo said|cfo said|company announced)\b',
34
+ ]
35
+
36
+
37
+ def detect_rumor_indicators(text: str) -> Dict[str, Any]:
38
+ """
39
+ Detect rumor indicators in text.
40
+
41
+ Args:
42
+ text: Text to analyze
43
+
44
+ Returns:
45
+ Dictionary with rumor detection results
46
+ """
47
+ text_lower = text.lower()
48
+
49
+ # Count rumor indicators
50
+ rumor_matches = []
51
+ for pattern in RUMOR_INDICATORS:
52
+ matches = re.finditer(pattern, text_lower, re.IGNORECASE)
53
+ for match in matches:
54
+ rumor_matches.append({
55
+ "text": match.group(0),
56
+ "position": match.start(),
57
+ "type": "rumor_indicator",
58
+ })
59
+
60
+ # Count verification indicators
61
+ verification_matches = []
62
+ for pattern in VERIFICATION_INDICATORS:
63
+ matches = re.finditer(pattern, text_lower, re.IGNORECASE)
64
+ for match in matches:
65
+ verification_matches.append({
66
+ "text": match.group(0),
67
+ "position": match.start(),
68
+ "type": "verification_indicator",
69
+ })
70
+
71
+ # Calculate rumor score (0-1, higher = more likely rumor)
72
+ rumor_count = len(rumor_matches)
73
+ verification_count = len(verification_matches)
74
+
75
+ if rumor_count == 0 and verification_count == 0:
76
+ rumor_score = 0.5 # Neutral
77
+ else:
78
+ # Score based on ratio
79
+ total = rumor_count + verification_count
80
+ rumor_score = rumor_count / total if total > 0 else 0.5
81
+
82
+ # Adjust for absolute counts
83
+ if rumor_count >= 3:
84
+ rumor_score = min(rumor_score + 0.2, 1.0)
85
+ if verification_count >= 2:
86
+ rumor_score = max(rumor_score - 0.2, 0.0)
87
+
88
+ assessment = "likely_rumor" if rumor_score >= 0.7 else \
89
+ "possible_rumor" if rumor_score >= 0.5 else \
90
+ "likely_verified"
91
+
92
+ logger.info(f"Rumor detection: score={rumor_score:.2f}, assessment={assessment}")
93
+
94
+ return {
95
+ "rumor_score": rumor_score,
96
+ "assessment": assessment,
97
+ "rumor_indicators": rumor_matches,
98
+ "verification_indicators": verification_matches,
99
+ "rumor_count": rumor_count,
100
+ "verification_count": verification_count,
101
+ }
102
+
103
+
104
+ def check_claim_verification(claim: str, sources: List[str]) -> Dict[str, Any]:
105
+ """
106
+ Check if a claim is verified by credible sources.
107
+
108
+ Args:
109
+ claim: Claim to verify
110
+ sources: List of source URLs
111
+
112
+ Returns:
113
+ Verification assessment
114
+ """
115
+ from app.domain_packs.finance.source_checker import aggregate_source_scores
116
+
117
+ # Detect rumor indicators in the claim itself
118
+ rumor_detection = detect_rumor_indicators(claim)
119
+
120
+ # Check source credibility
121
+ source_assessment = aggregate_source_scores(sources)
122
+
123
+ # Combine assessments
124
+ is_verified = (
125
+ rumor_detection["rumor_score"] < 0.5 and
126
+ source_assessment["average_score"] >= 0.7
127
+ )
128
+
129
+ confidence = (1 - rumor_detection["rumor_score"]) * source_assessment["average_score"]
130
+
131
+ return {
132
+ "claim": claim,
133
+ "is_verified": is_verified,
134
+ "confidence": confidence,
135
+ "rumor_detection": rumor_detection,
136
+ "source_assessment": source_assessment,
137
+ "recommendation": "trust" if is_verified else "verify_further" if confidence >= 0.5 else "skeptical",
138
+ }
backend/app/domain_packs/finance/scam_detector.py ADDED
@@ -0,0 +1,159 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Scam detector for finance domain pack.
3
+
4
+ Detects potential financial scams and fraudulent schemes.
5
+ """
6
+
7
+ import re
8
+ from typing import List, Dict, Any
9
+ import logging
10
+
11
+ logger = logging.getLogger(__name__)
12
+
13
+
14
+ # Scam indicator patterns
15
+ SCAM_PATTERNS = [
16
+ # Get-rich-quick schemes
17
+ r'\b(get rich quick|make money fast|guaranteed returns|no risk)\b',
18
+ r'\b(double your money|triple your investment|10x returns)\b',
19
+ r'\b(passive income|work from home|financial freedom)\b',
20
+
21
+ # Pressure tactics
22
+ r'\b(act now|limited time|urgent|don\'t miss out|last chance)\b',
23
+ r'\b(exclusive offer|secret|insider tip|hidden opportunity)\b',
24
+
25
+ # Unrealistic promises
26
+ r'\b(guaranteed profit|risk-free|100% return|never lose)\b',
27
+ r'\b(foolproof|can\'t lose|sure thing|no-brainer)\b',
28
+
29
+ # Pyramid/MLM indicators
30
+ r'\b(recruit|downline|upline|multi-level|network marketing)\b',
31
+ r'\b(join my team|be your own boss|financial independence)\b',
32
+
33
+ # Crypto scams
34
+ r'\b(airdrop|free crypto|token giveaway|pump and dump)\b',
35
+ r'\b(send.*receive back|double your bitcoin)\b',
36
+
37
+ # Phishing/fraud
38
+ r'\b(verify your account|suspended account|unusual activity)\b',
39
+ r'\b(click here immediately|update payment|confirm identity)\b',
40
+ ]
41
+
42
+ # Known scam keywords
43
+ HIGH_RISK_KEYWORDS = [
44
+ "ponzi", "pyramid scheme", "advance fee", "419 scam",
45
+ "pump and dump", "rug pull", "exit scam", "phishing",
46
+ ]
47
+
48
+
49
+ def detect_scam_indicators(text: str) -> Dict[str, Any]:
50
+ """
51
+ Detect scam indicators in text.
52
+
53
+ Args:
54
+ text: Text to analyze
55
+
56
+ Returns:
57
+ Dictionary with scam detection results
58
+ """
59
+ text_lower = text.lower()
60
+
61
+ # Find pattern matches
62
+ matches = []
63
+ for pattern in SCAM_PATTERNS:
64
+ found = re.finditer(pattern, text_lower, re.IGNORECASE)
65
+ for match in found:
66
+ matches.append({
67
+ "text": match.group(0),
68
+ "position": match.start(),
69
+ "type": "scam_pattern",
70
+ })
71
+
72
+ # Check for high-risk keywords
73
+ high_risk_found = []
74
+ for keyword in HIGH_RISK_KEYWORDS:
75
+ if keyword in text_lower:
76
+ high_risk_found.append(keyword)
77
+
78
+ # Calculate scam score (0-1, higher = more likely scam)
79
+ pattern_score = min(len(matches) * 0.15, 0.8)
80
+ keyword_score = min(len(high_risk_found) * 0.3, 0.9)
81
+
82
+ scam_score = max(pattern_score, keyword_score)
83
+
84
+ # Adjust for multiple indicators
85
+ if len(matches) >= 5:
86
+ scam_score = min(scam_score + 0.2, 1.0)
87
+
88
+ risk_level = "high_risk" if scam_score >= 0.7 else \
89
+ "medium_risk" if scam_score >= 0.4 else \
90
+ "low_risk"
91
+
92
+ logger.info(f"Scam detection: score={scam_score:.2f}, risk={risk_level}")
93
+
94
+ return {
95
+ "scam_score": scam_score,
96
+ "risk_level": risk_level,
97
+ "pattern_matches": matches,
98
+ "high_risk_keywords": high_risk_found,
99
+ "match_count": len(matches),
100
+ "keyword_count": len(high_risk_found),
101
+ }
102
+
103
+
104
+ def check_investment_legitimacy(
105
+ description: str,
106
+ promised_return: float = None,
107
+ timeframe: str = None
108
+ ) -> Dict[str, Any]:
109
+ """
110
+ Check if an investment opportunity appears legitimate.
111
+
112
+ Args:
113
+ description: Investment description
114
+ promised_return: Promised return percentage (if specified)
115
+ timeframe: Timeframe for returns (if specified)
116
+
117
+ Returns:
118
+ Legitimacy assessment
119
+ """
120
+ # Detect scam indicators
121
+ scam_detection = detect_scam_indicators(description)
122
+
123
+ # Check for unrealistic returns
124
+ unrealistic_return = False
125
+ if promised_return is not None:
126
+ # Returns over 20% annually are suspicious
127
+ # Returns over 50% are highly suspicious
128
+ if promised_return > 50:
129
+ unrealistic_return = True
130
+ scam_detection["scam_score"] = min(scam_detection["scam_score"] + 0.3, 1.0)
131
+ elif promised_return > 20:
132
+ unrealistic_return = True
133
+ scam_detection["scam_score"] = min(scam_detection["scam_score"] + 0.15, 1.0)
134
+
135
+ # Check for short timeframes with high returns
136
+ suspicious_timeframe = False
137
+ if timeframe and promised_return:
138
+ timeframe_lower = timeframe.lower()
139
+ if any(word in timeframe_lower for word in ["day", "days", "week", "weeks"]):
140
+ if promised_return > 10:
141
+ suspicious_timeframe = True
142
+ scam_detection["scam_score"] = min(scam_detection["scam_score"] + 0.2, 1.0)
143
+
144
+ is_legitimate = scam_detection["scam_score"] < 0.4
145
+
146
+ return {
147
+ "is_legitimate": is_legitimate,
148
+ "scam_detection": scam_detection,
149
+ "unrealistic_return": unrealistic_return,
150
+ "suspicious_timeframe": suspicious_timeframe,
151
+ "recommendation": "avoid" if scam_detection["scam_score"] >= 0.7 else
152
+ "investigate_thoroughly" if scam_detection["scam_score"] >= 0.4 else
153
+ "proceed_with_caution",
154
+ "warnings": [
155
+ "Unrealistic return promises" if unrealistic_return else None,
156
+ "Suspicious timeframe" if suspicious_timeframe else None,
157
+ f"{scam_detection['match_count']} scam indicators found" if scam_detection['match_count'] > 0 else None,
158
+ ],
159
+ }
backend/app/domain_packs/finance/source_checker.py ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Source credibility checker for finance domain pack.
3
+
4
+ Evaluates the credibility of financial news sources and information.
5
+ """
6
+
7
+ from typing import Dict, Any
8
+ from urllib.parse import urlparse
9
+ import logging
10
+
11
+ logger = logging.getLogger(__name__)
12
+
13
+
14
+ # Trusted financial news sources (expandable)
15
+ TRUSTED_SOURCES = {
16
+ # Tier 1: Highly trusted
17
+ "bloomberg.com": {"tier": 1, "score": 0.95, "category": "financial_news"},
18
+ "reuters.com": {"tier": 1, "score": 0.95, "category": "news_wire"},
19
+ "wsj.com": {"tier": 1, "score": 0.95, "category": "financial_news"},
20
+ "ft.com": {"tier": 1, "score": 0.95, "category": "financial_news"},
21
+
22
+ # Tier 2: Trusted
23
+ "cnbc.com": {"tier": 2, "score": 0.85, "category": "financial_news"},
24
+ "marketwatch.com": {"tier": 2, "score": 0.85, "category": "financial_news"},
25
+ "barrons.com": {"tier": 2, "score": 0.85, "category": "financial_news"},
26
+ "economist.com": {"tier": 2, "score": 0.85, "category": "business_news"},
27
+ "forbes.com": {"tier": 2, "score": 0.80, "category": "business_news"},
28
+
29
+ # Tier 3: Generally reliable
30
+ "yahoo.com": {"tier": 3, "score": 0.70, "category": "aggregator"},
31
+ "seekingalpha.com": {"tier": 3, "score": 0.70, "category": "analysis"},
32
+ "investopedia.com": {"tier": 3, "score": 0.75, "category": "education"},
33
+
34
+ # Official sources
35
+ "sec.gov": {"tier": 1, "score": 1.0, "category": "regulatory"},
36
+ "federalreserve.gov": {"tier": 1, "score": 1.0, "category": "regulatory"},
37
+ "treasury.gov": {"tier": 1, "score": 1.0, "category": "regulatory"},
38
+ }
39
+
40
+ # Red flag domains (known for misinformation)
41
+ UNTRUSTED_SOURCES = {
42
+ "example-scam.com": {"score": 0.1, "reason": "known_scam"},
43
+ # Add more as identified
44
+ }
45
+
46
+
47
+ def check_source_credibility(url: str) -> Dict[str, Any]:
48
+ """
49
+ Check the credibility of a source URL.
50
+
51
+ Args:
52
+ url: Source URL to check
53
+
54
+ Returns:
55
+ Dictionary with credibility assessment
56
+ """
57
+ try:
58
+ parsed = urlparse(url)
59
+ domain = parsed.netloc.lower()
60
+
61
+ # Remove www. prefix
62
+ if domain.startswith("www."):
63
+ domain = domain[4:]
64
+
65
+ # Check if it's a trusted source
66
+ if domain in TRUSTED_SOURCES:
67
+ info = TRUSTED_SOURCES[domain]
68
+ logger.info(f"Source {domain} is trusted (tier {info['tier']})")
69
+ return {
70
+ "url": url,
71
+ "domain": domain,
72
+ "credibility_score": info["score"],
73
+ "tier": info["tier"],
74
+ "category": info["category"],
75
+ "trusted": True,
76
+ "reason": "known_trusted_source",
77
+ }
78
+
79
+ # Check if it's an untrusted source
80
+ if domain in UNTRUSTED_SOURCES:
81
+ info = UNTRUSTED_SOURCES[domain]
82
+ logger.warning(f"Source {domain} is untrusted: {info['reason']}")
83
+ return {
84
+ "url": url,
85
+ "domain": domain,
86
+ "credibility_score": info["score"],
87
+ "trusted": False,
88
+ "reason": info["reason"],
89
+ }
90
+
91
+ # Unknown source - neutral score
92
+ logger.info(f"Source {domain} is unknown, assigning neutral score")
93
+ return {
94
+ "url": url,
95
+ "domain": domain,
96
+ "credibility_score": 0.5,
97
+ "trusted": None,
98
+ "reason": "unknown_source",
99
+ }
100
+
101
+ except Exception as e:
102
+ logger.error(f"Error checking source credibility for {url}: {e}")
103
+ return {
104
+ "url": url,
105
+ "credibility_score": 0.3,
106
+ "trusted": False,
107
+ "reason": "parse_error",
108
+ }
109
+
110
+
111
+ def aggregate_source_scores(sources: list[str]) -> Dict[str, Any]:
112
+ """
113
+ Aggregate credibility scores from multiple sources.
114
+
115
+ Args:
116
+ sources: List of source URLs
117
+
118
+ Returns:
119
+ Aggregated credibility assessment
120
+ """
121
+ if not sources:
122
+ return {
123
+ "average_score": 0.0,
124
+ "trusted_count": 0,
125
+ "untrusted_count": 0,
126
+ "unknown_count": 0,
127
+ }
128
+
129
+ scores = []
130
+ trusted_count = 0
131
+ untrusted_count = 0
132
+ unknown_count = 0
133
+
134
+ for url in sources:
135
+ result = check_source_credibility(url)
136
+ scores.append(result["credibility_score"])
137
+
138
+ if result.get("trusted") is True:
139
+ trusted_count += 1
140
+ elif result.get("trusted") is False:
141
+ untrusted_count += 1
142
+ else:
143
+ unknown_count += 1
144
+
145
+ average_score = sum(scores) / len(scores) if scores else 0.0
146
+
147
+ return {
148
+ "average_score": average_score,
149
+ "trusted_count": trusted_count,
150
+ "untrusted_count": untrusted_count,
151
+ "unknown_count": unknown_count,
152
+ "total_sources": len(sources),
153
+ "assessment": "high_credibility" if average_score >= 0.8 else
154
+ "medium_credibility" if average_score >= 0.6 else
155
+ "low_credibility",
156
+ }
backend/app/domain_packs/finance/stance_detector.py ADDED
@@ -0,0 +1,143 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Stance detector for finance domain pack.
3
+
4
+ Detects sentiment and stance (bullish/bearish) in financial content.
5
+ """
6
+
7
+ import re
8
+ from typing import Dict, Any
9
+ import logging
10
+
11
+ logger = logging.getLogger(__name__)
12
+
13
+
14
+ # Bullish indicators
15
+ BULLISH_KEYWORDS = [
16
+ "bullish", "buy", "long", "upgrade", "outperform", "strong buy",
17
+ "positive", "growth", "rally", "surge", "soar", "climb", "gain",
18
+ "beat expectations", "exceed", "record high", "all-time high",
19
+ "momentum", "breakout", "uptrend", "optimistic", "confident",
20
+ ]
21
+
22
+ # Bearish indicators
23
+ BEARISH_KEYWORDS = [
24
+ "bearish", "sell", "short", "downgrade", "underperform", "strong sell",
25
+ "negative", "decline", "fall", "drop", "plunge", "crash", "loss",
26
+ "miss expectations", "disappoint", "record low", "downturn",
27
+ "weakness", "breakdown", "downtrend", "pessimistic", "concerned",
28
+ ]
29
+
30
+ # Neutral indicators
31
+ NEUTRAL_KEYWORDS = [
32
+ "hold", "neutral", "maintain", "unchanged", "stable", "flat",
33
+ "sideways", "range-bound", "wait and see", "cautious",
34
+ ]
35
+
36
+
37
+ def detect_stance(text: str) -> Dict[str, Any]:
38
+ """
39
+ Detect financial stance (bullish/bearish/neutral) in text.
40
+
41
+ Args:
42
+ text: Text to analyze
43
+
44
+ Returns:
45
+ Dictionary with stance detection results
46
+ """
47
+ text_lower = text.lower()
48
+
49
+ # Count keyword occurrences
50
+ bullish_count = sum(1 for keyword in BULLISH_KEYWORDS if keyword in text_lower)
51
+ bearish_count = sum(1 for keyword in BEARISH_KEYWORDS if keyword in text_lower)
52
+ neutral_count = sum(1 for keyword in NEUTRAL_KEYWORDS if keyword in text_lower)
53
+
54
+ total_count = bullish_count + bearish_count + neutral_count
55
+
56
+ if total_count == 0:
57
+ # No clear indicators
58
+ stance = "neutral"
59
+ confidence = 0.3
60
+ sentiment_score = 0.5
61
+ else:
62
+ # Calculate sentiment score (-1 to 1)
63
+ # Positive = bullish, negative = bearish
64
+ sentiment_score = (bullish_count - bearish_count) / total_count
65
+
66
+ # Normalize to 0-1 range
67
+ sentiment_score = (sentiment_score + 1) / 2
68
+
69
+ # Determine stance
70
+ if bullish_count > bearish_count and bullish_count > neutral_count:
71
+ stance = "bullish"
72
+ confidence = bullish_count / total_count
73
+ elif bearish_count > bullish_count and bearish_count > neutral_count:
74
+ stance = "bearish"
75
+ confidence = bearish_count / total_count
76
+ else:
77
+ stance = "neutral"
78
+ confidence = max(neutral_count / total_count, 0.5)
79
+
80
+ logger.info(f"Stance detection: {stance} (confidence={confidence:.2f}, sentiment={sentiment_score:.2f})")
81
+
82
+ return {
83
+ "stance": stance,
84
+ "confidence": confidence,
85
+ "sentiment_score": sentiment_score,
86
+ "bullish_count": bullish_count,
87
+ "bearish_count": bearish_count,
88
+ "neutral_count": neutral_count,
89
+ "total_indicators": total_count,
90
+ }
91
+
92
+
93
+ def analyze_price_action_language(text: str) -> Dict[str, Any]:
94
+ """
95
+ Analyze language describing price action.
96
+
97
+ Args:
98
+ text: Text to analyze
99
+
100
+ Returns:
101
+ Price action analysis
102
+ """
103
+ text_lower = text.lower()
104
+
105
+ # Detect magnitude words
106
+ strong_movement = any(word in text_lower for word in [
107
+ "surge", "soar", "plunge", "crash", "skyrocket", "plummet"
108
+ ])
109
+
110
+ moderate_movement = any(word in text_lower for word in [
111
+ "rise", "fall", "climb", "drop", "gain", "loss"
112
+ ])
113
+
114
+ weak_movement = any(word in text_lower for word in [
115
+ "inch", "edge", "slip", "dip", "tick"
116
+ ])
117
+
118
+ # Detect direction
119
+ upward = any(word in text_lower for word in [
120
+ "up", "higher", "gain", "rise", "climb", "rally", "surge"
121
+ ])
122
+
123
+ downward = any(word in text_lower for word in [
124
+ "down", "lower", "loss", "fall", "drop", "decline", "plunge"
125
+ ])
126
+
127
+ magnitude = "strong" if strong_movement else \
128
+ "moderate" if moderate_movement else \
129
+ "weak" if weak_movement else \
130
+ "unclear"
131
+
132
+ direction = "upward" if upward and not downward else \
133
+ "downward" if downward and not upward else \
134
+ "mixed" if upward and downward else \
135
+ "unclear"
136
+
137
+ return {
138
+ "magnitude": magnitude,
139
+ "direction": direction,
140
+ "strong_movement": strong_movement,
141
+ "moderate_movement": moderate_movement,
142
+ "weak_movement": weak_movement,
143
+ }
backend/app/domain_packs/finance/ticker_resolver.py ADDED
@@ -0,0 +1,171 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Ticker resolver for finance domain pack.
3
+
4
+ Resolves company names to stock ticker symbols and vice versa.
5
+ """
6
+
7
+ import re
8
+ from typing import Optional, List, Dict, Any
9
+ import logging
10
+
11
+ from app.domain_packs.finance.market_data import search_symbol
12
+
13
+ logger = logging.getLogger(__name__)
14
+
15
+
16
+ # Ticker pattern: $SYMBOL or standalone uppercase 1-5 letters
17
+ TICKER_PATTERN = re.compile(r'\$([A-Z]{1,5})\b')
18
+ STANDALONE_TICKER_PATTERN = re.compile(r'\b([A-Z]{2,5})\b')
19
+
20
+ # Known ticker mappings (expandable)
21
+ KNOWN_TICKERS = {
22
+ "AAPL": "Apple Inc.",
23
+ "MSFT": "Microsoft Corporation",
24
+ "GOOGL": "Alphabet Inc.",
25
+ "GOOG": "Alphabet Inc.",
26
+ "AMZN": "Amazon.com Inc.",
27
+ "META": "Meta Platforms Inc.",
28
+ "TSLA": "Tesla Inc.",
29
+ "NVDA": "NVIDIA Corporation",
30
+ "BRK.A": "Berkshire Hathaway Inc.",
31
+ "BRK.B": "Berkshire Hathaway Inc.",
32
+ "JPM": "JPMorgan Chase & Co.",
33
+ "V": "Visa Inc.",
34
+ "WMT": "Walmart Inc.",
35
+ "XOM": "Exxon Mobil Corporation",
36
+ "JNJ": "Johnson & Johnson",
37
+ }
38
+
39
+ # Reverse mapping
40
+ COMPANY_TO_TICKER = {v: k for k, v in KNOWN_TICKERS.items()}
41
+
42
+
43
+ def extract_tickers(text: str) -> List[str]:
44
+ """
45
+ Extract stock ticker symbols from text.
46
+
47
+ Args:
48
+ text: Input text
49
+
50
+ Returns:
51
+ List of ticker symbols found
52
+ """
53
+ tickers = []
54
+
55
+ # Find $SYMBOL patterns
56
+ dollar_tickers = TICKER_PATTERN.findall(text)
57
+ tickers.extend(dollar_tickers)
58
+
59
+ # Find standalone uppercase symbols (more conservative)
60
+ # Only if they're known tickers to avoid false positives
61
+ standalone = STANDALONE_TICKER_PATTERN.findall(text)
62
+ for symbol in standalone:
63
+ if symbol in KNOWN_TICKERS:
64
+ tickers.append(symbol)
65
+
66
+ # Remove duplicates while preserving order
67
+ seen = set()
68
+ unique_tickers = []
69
+ for ticker in tickers:
70
+ if ticker not in seen:
71
+ unique_tickers.append(ticker)
72
+ seen.add(ticker)
73
+
74
+ logger.info(f"Extracted {len(unique_tickers)} tickers from text: {unique_tickers}")
75
+ return unique_tickers
76
+
77
+
78
+ def resolve_ticker(ticker: str) -> Optional[Dict[str, Any]]:
79
+ """
80
+ Resolve ticker symbol to company information.
81
+
82
+ Args:
83
+ ticker: Stock ticker symbol
84
+
85
+ Returns:
86
+ Dictionary with company information or None
87
+ """
88
+ ticker = ticker.upper()
89
+
90
+ # Check known tickers first
91
+ if ticker in KNOWN_TICKERS:
92
+ return {
93
+ "ticker": ticker,
94
+ "company_name": KNOWN_TICKERS[ticker],
95
+ "source": "known_mapping",
96
+ "confidence": 1.0,
97
+ }
98
+
99
+ # Try Alpha Vantage search
100
+ try:
101
+ results = search_symbol(ticker)
102
+ if results:
103
+ best_match = results[0]
104
+ return {
105
+ "ticker": best_match.get("1. symbol", ticker),
106
+ "company_name": best_match.get("2. name", "Unknown"),
107
+ "region": best_match.get("4. region", "Unknown"),
108
+ "currency": best_match.get("8. currency", "Unknown"),
109
+ "source": "alpha_vantage",
110
+ "confidence": 0.8,
111
+ }
112
+ except Exception as e:
113
+ logger.error(f"Error resolving ticker {ticker}: {e}")
114
+
115
+ return None
116
+
117
+
118
+ def resolve_company_to_ticker(company_name: str) -> Optional[str]:
119
+ """
120
+ Resolve company name to ticker symbol.
121
+
122
+ Args:
123
+ company_name: Company name
124
+
125
+ Returns:
126
+ Ticker symbol or None
127
+ """
128
+ # Check known mappings first
129
+ if company_name in COMPANY_TO_TICKER:
130
+ ticker = COMPANY_TO_TICKER[company_name]
131
+ logger.info(f"Resolved '{company_name}' to ticker {ticker}")
132
+ return ticker
133
+
134
+ # Try Alpha Vantage search
135
+ try:
136
+ results = search_symbol(company_name)
137
+ if results:
138
+ best_match = results[0]
139
+ ticker = best_match.get("1. symbol")
140
+ logger.info(f"Resolved '{company_name}' to ticker {ticker} via Alpha Vantage")
141
+ return ticker
142
+ except Exception as e:
143
+ logger.error(f"Error resolving company '{company_name}' to ticker: {e}")
144
+
145
+ return None
146
+
147
+
148
+ def enrich_with_tickers(entities: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
149
+ """
150
+ Enrich entity list with ticker symbols.
151
+
152
+ Args:
153
+ entities: List of entities from entity_resolver
154
+
155
+ Returns:
156
+ Enriched entities with ticker information
157
+ """
158
+ enriched = []
159
+
160
+ for entity in entities:
161
+ enriched_entity = entity.copy()
162
+
163
+ if entity.get("type") == "company":
164
+ company_name = entity.get("text", "")
165
+ ticker = resolve_company_to_ticker(company_name)
166
+ if ticker:
167
+ enriched_entity["ticker"] = ticker
168
+
169
+ enriched.append(enriched_entity)
170
+
171
+ return enriched
backend/app/domain_packs/init_packs.py ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Initialize and register domain packs.
3
+
4
+ This module should be imported at application startup to register
5
+ all available domain packs.
6
+ """
7
+
8
+ import logging
9
+ from app.config import os
10
+
11
+ logger = logging.getLogger(__name__)
12
+
13
+
14
+ def init_domain_packs():
15
+ """Initialize and register all domain packs."""
16
+ from app.domain_packs.registry import get_registry
17
+
18
+ # Check if finance pack is enabled
19
+ finance_enabled = os.getenv("FINANCE_DOMAIN_PACK_ENABLED", "true").lower() == "true"
20
+
21
+ if finance_enabled:
22
+ try:
23
+ from app.domain_packs.finance import FinanceDomainPack
24
+
25
+ registry = get_registry()
26
+ finance_pack = FinanceDomainPack()
27
+ registry.register(finance_pack)
28
+
29
+ logger.info("Finance domain pack registered successfully")
30
+ except Exception as e:
31
+ logger.error(f"Failed to register finance domain pack: {e}")
32
+ else:
33
+ logger.info("Finance domain pack is disabled")
34
+
35
+ # Future domain packs can be registered here
36
+ # Example:
37
+ # if healthcare_enabled:
38
+ # from app.domain_packs.healthcare import HealthcareDomainPack
39
+ # registry.register(HealthcareDomainPack())
backend/app/domain_packs/registry.py ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Domain pack registry for managing and discovering domain packs.
3
+ """
4
+
5
+ from typing import Dict, List, Optional
6
+ import logging
7
+
8
+ from app.domain_packs.base import DomainPack
9
+
10
+ logger = logging.getLogger(__name__)
11
+
12
+
13
+ class DomainPackRegistry:
14
+ """Registry for managing domain packs."""
15
+
16
+ def __init__(self):
17
+ self._packs: Dict[str, DomainPack] = {}
18
+
19
+ def register(self, pack: DomainPack) -> None:
20
+ """Register a domain pack."""
21
+ name = pack.name
22
+ if name in self._packs:
23
+ logger.warning(f"Domain pack '{name}' is already registered, overwriting")
24
+ self._packs[name] = pack
25
+ logger.info(f"Registered domain pack: {name}")
26
+
27
+ def get_pack(self, name: str) -> Optional[DomainPack]:
28
+ """Get a domain pack by name."""
29
+ return self._packs.get(name)
30
+
31
+ def detect_domain(self, query: str) -> Optional[str]:
32
+ """
33
+ Detect which domain pack matches the query based on keywords.
34
+
35
+ Args:
36
+ query: The user's query
37
+
38
+ Returns:
39
+ Domain pack name if detected, None otherwise
40
+ """
41
+ query_lower = query.lower()
42
+
43
+ for name, pack in self._packs.items():
44
+ for keyword in pack.keywords:
45
+ if keyword.lower() in query_lower:
46
+ logger.info(f"Detected domain '{name}' from keyword '{keyword}'")
47
+ return name
48
+
49
+ return None
50
+
51
+ def list_packs(self) -> List[str]:
52
+ """List all registered domain pack names."""
53
+ return list(self._packs.keys())
54
+
55
+ def get_capabilities(self) -> Dict[str, Any]:
56
+ """Get capabilities of all registered domain packs."""
57
+ return {
58
+ name: pack.get_capabilities()
59
+ for name, pack in self._packs.items()
60
+ }
61
+
62
+
63
+ # Global registry instance
64
+ _registry = DomainPackRegistry()
65
+
66
+
67
+ def get_registry() -> DomainPackRegistry:
68
+ """Get the global domain pack registry."""
69
+ return _registry
backend/app/main.py CHANGED
@@ -30,6 +30,10 @@ logger = logging.getLogger(__name__)
30
 
31
  app = FastAPI(title="MiroOrg Basic", version=APP_VERSION)
32
 
 
 
 
 
33
  app.add_middleware(
34
  CORSMiddleware,
35
  allow_origins=["http://localhost:3000", "http://127.0.0.1:3000"],
 
30
 
31
  app = FastAPI(title="MiroOrg Basic", version=APP_VERSION)
32
 
33
+ # Initialize domain packs on startup
34
+ from app.domain_packs.init_packs import init_domain_packs
35
+ init_domain_packs()
36
+
37
  app.add_middleware(
38
  CORSMiddleware,
39
  allow_origins=["http://localhost:3000", "http://127.0.0.1:3000"],
backend/app/schemas.py CHANGED
@@ -2,6 +2,15 @@ from typing import List, Dict, Any, Optional
2
  from pydantic import BaseModel, Field
3
 
4
 
 
 
 
 
 
 
 
 
 
5
  class UserTask(BaseModel):
6
  user_input: str
7
 
 
2
  from pydantic import BaseModel, Field
3
 
4
 
5
+ class RouteDecision(BaseModel):
6
+ """Routing decision from Switchboard agent."""
7
+ task_family: str = Field(..., description="Task family: 'normal' or 'simulation'")
8
+ domain_pack: str = Field(..., description="Domain pack: 'finance', 'general', 'policy', 'custom'")
9
+ complexity: str = Field(..., description="Complexity: 'simple', 'medium', 'complex'")
10
+ execution_mode: str = Field(..., description="Execution mode: 'solo', 'standard', 'deep'")
11
+ risk_level: str = Field(default="low", description="Risk level: 'low', 'medium', 'high'")
12
+
13
+
14
  class UserTask(BaseModel):
15
  user_input: str
16
 
backend/app/services/health_service.py CHANGED
@@ -1,6 +1,7 @@
1
  from pathlib import Path
2
  from importlib.util import find_spec
3
  from typing import Dict, Any
 
4
 
5
  from app.config import (
6
  APP_VERSION,
@@ -9,7 +10,11 @@ from app.config import (
9
  PRIMARY_PROVIDER,
10
  FALLBACK_PROVIDER,
11
  OPENROUTER_API_KEY,
 
12
  OLLAMA_ENABLED,
 
 
 
13
  TAVILY_API_KEY,
14
  NEWSAPI_KEY,
15
  ALPHAVANTAGE_API_KEY,
@@ -17,6 +22,8 @@ from app.config import (
17
  )
18
  from app.services.mirofish_client import mirofish_health
19
 
 
 
20
 
21
  REQUIRED_PROMPTS = ["research.txt", "planner.txt", "verifier.txt", "synthesizer.txt"]
22
 
@@ -33,6 +40,52 @@ def _memory_dir_writable() -> bool:
33
  return False
34
 
35
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
  def deep_health() -> Dict[str, Any]:
37
  prompt_checks = {
38
  prompt: (Path(PROMPTS_DIR) / prompt).exists()
@@ -41,13 +94,23 @@ def deep_health() -> Dict[str, Any]:
41
 
42
  mirofish = mirofish_health() if MIROFISH_ENABLED else {"reachable": False, "status_code": None, "body": "disabled"}
43
 
 
 
 
 
 
 
 
44
  checks = {
45
  "memory_dir_writable": _memory_dir_writable(),
46
  "prompt_files": prompt_checks,
47
  "prompts_loaded": all(prompt_checks.values()),
48
  "primary_provider": PRIMARY_PROVIDER,
 
49
  "fallback_provider": FALLBACK_PROVIDER,
 
50
  "openrouter_key_present": bool(OPENROUTER_API_KEY),
 
51
  "ollama_enabled": OLLAMA_ENABLED,
52
  "tavily_enabled": bool(TAVILY_API_KEY),
53
  "newsapi_enabled": bool(NEWSAPI_KEY),
 
1
  from pathlib import Path
2
  from importlib.util import find_spec
3
  from typing import Dict, Any
4
+ import logging
5
 
6
  from app.config import (
7
  APP_VERSION,
 
10
  PRIMARY_PROVIDER,
11
  FALLBACK_PROVIDER,
12
  OPENROUTER_API_KEY,
13
+ OPENROUTER_BASE_URL,
14
  OLLAMA_ENABLED,
15
+ OLLAMA_BASE_URL,
16
+ OPENAI_API_KEY,
17
+ OPENAI_BASE_URL,
18
  TAVILY_API_KEY,
19
  NEWSAPI_KEY,
20
  ALPHAVANTAGE_API_KEY,
 
22
  )
23
  from app.services.mirofish_client import mirofish_health
24
 
25
+ logger = logging.getLogger(__name__)
26
+
27
 
28
  REQUIRED_PROMPTS = ["research.txt", "planner.txt", "verifier.txt", "synthesizer.txt"]
29
 
 
40
  return False
41
 
42
 
43
+ def _check_provider_health(provider: str) -> Dict[str, Any]:
44
+ """Check if a provider is configured and reachable."""
45
+ import httpx
46
+
47
+ provider = provider.lower()
48
+
49
+ if provider == "openrouter":
50
+ if not OPENROUTER_API_KEY:
51
+ return {"configured": False, "reachable": False, "error": "API key missing"}
52
+ try:
53
+ with httpx.Client(timeout=5) as client:
54
+ response = client.get(f"{OPENROUTER_BASE_URL}/models", headers={
55
+ "Authorization": f"Bearer {OPENROUTER_API_KEY}"
56
+ })
57
+ return {"configured": True, "reachable": response.status_code < 500, "status_code": response.status_code}
58
+ except Exception as e:
59
+ logger.warning(f"OpenRouter health check failed: {e}")
60
+ return {"configured": True, "reachable": False, "error": str(e)}
61
+
62
+ elif provider == "ollama":
63
+ if not OLLAMA_ENABLED:
64
+ return {"configured": False, "reachable": False, "error": "Ollama disabled"}
65
+ try:
66
+ with httpx.Client(timeout=5) as client:
67
+ response = client.get(f"{OLLAMA_BASE_URL.replace('/api', '')}/api/tags")
68
+ return {"configured": True, "reachable": response.status_code == 200, "status_code": response.status_code}
69
+ except Exception as e:
70
+ logger.warning(f"Ollama health check failed: {e}")
71
+ return {"configured": True, "reachable": False, "error": str(e)}
72
+
73
+ elif provider == "openai":
74
+ if not OPENAI_API_KEY:
75
+ return {"configured": False, "reachable": False, "error": "API key missing"}
76
+ try:
77
+ with httpx.Client(timeout=5) as client:
78
+ response = client.get(f"{OPENAI_BASE_URL}/models", headers={
79
+ "Authorization": f"Bearer {OPENAI_API_KEY}"
80
+ })
81
+ return {"configured": True, "reachable": response.status_code < 500, "status_code": response.status_code}
82
+ except Exception as e:
83
+ logger.warning(f"OpenAI health check failed: {e}")
84
+ return {"configured": True, "reachable": False, "error": str(e)}
85
+
86
+ return {"configured": False, "reachable": False, "error": "Unknown provider"}
87
+
88
+
89
  def deep_health() -> Dict[str, Any]:
90
  prompt_checks = {
91
  prompt: (Path(PROMPTS_DIR) / prompt).exists()
 
94
 
95
  mirofish = mirofish_health() if MIROFISH_ENABLED else {"reachable": False, "status_code": None, "body": "disabled"}
96
 
97
+ # Check provider health
98
+ primary_health = _check_provider_health(PRIMARY_PROVIDER)
99
+ fallback_health = _check_provider_health(FALLBACK_PROVIDER)
100
+
101
+ logger.info(f"Primary provider {PRIMARY_PROVIDER} health: {primary_health}")
102
+ logger.info(f"Fallback provider {FALLBACK_PROVIDER} health: {fallback_health}")
103
+
104
  checks = {
105
  "memory_dir_writable": _memory_dir_writable(),
106
  "prompt_files": prompt_checks,
107
  "prompts_loaded": all(prompt_checks.values()),
108
  "primary_provider": PRIMARY_PROVIDER,
109
+ "primary_provider_health": primary_health,
110
  "fallback_provider": FALLBACK_PROVIDER,
111
+ "fallback_provider_health": fallback_health,
112
  "openrouter_key_present": bool(OPENROUTER_API_KEY),
113
+ "openai_key_present": bool(OPENAI_API_KEY),
114
  "ollama_enabled": OLLAMA_ENABLED,
115
  "tavily_enabled": bool(TAVILY_API_KEY),
116
  "newsapi_enabled": bool(NEWSAPI_KEY),
backend/test_requirements.py ADDED
@@ -0,0 +1,259 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Test script to verify all requirements for Task 7 are met.
3
+
4
+ Requirements being tested:
5
+ - 4.1: Switchboard classifies using four dimensions (task_family, domain_pack, complexity, execution_mode)
6
+ - 4.2: Simple queries (≤5 words) route to solo mode
7
+ - 4.3: Medium queries (≤25 words) route to standard mode
8
+ - 4.4: Complex queries (>25 words) route to deep mode
9
+ - 4.5: Simulation trigger keywords detected
10
+ - 4.6: Keywords are environment-configurable
11
+ - 4.7: task_family="simulation" when keywords detected
12
+ - 5.6: Domain pack detection using domain registry
13
+ """
14
+
15
+ from app.agents.switchboard import decide_route
16
+ from app.domain_packs.init_packs import init_domain_packs
17
+ from app.domain_packs.registry import get_registry
18
+ from app.config import SIMULATION_TRIGGER_KEYWORDS
19
+
20
+
21
+ def test_requirement_4_1():
22
+ """Test Requirement 4.1: Four-dimension classification"""
23
+ print("\n" + "="*60)
24
+ print("Testing Requirement 4.1: Four-dimension classification")
25
+ print("="*60)
26
+
27
+ init_domain_packs()
28
+
29
+ result = decide_route("What is the stock market doing today?")
30
+
31
+ required_keys = ["task_family", "domain_pack", "complexity", "execution_mode"]
32
+
33
+ for key in required_keys:
34
+ if key in result:
35
+ print(f"✅ {key}: {result[key]}")
36
+ else:
37
+ print(f"❌ Missing dimension: {key}")
38
+ return False
39
+
40
+ print("✅ Requirement 4.1 PASSED: All four dimensions present")
41
+ return True
42
+
43
+
44
+ def test_requirement_4_2():
45
+ """Test Requirement 4.2: Simple queries (≤5 words) route to solo mode"""
46
+ print("\n" + "="*60)
47
+ print("Testing Requirement 4.2: Simple queries route to solo mode")
48
+ print("="*60)
49
+
50
+ init_domain_packs()
51
+
52
+ test_cases = [
53
+ "Hello",
54
+ "Hi there",
55
+ "What is this?",
56
+ "Tell me more",
57
+ "Show me data",
58
+ ]
59
+
60
+ all_passed = True
61
+ for query in test_cases:
62
+ result = decide_route(query)
63
+ word_count = len(query.split())
64
+
65
+ if word_count <= 5:
66
+ if result["complexity"] == "simple" and result["execution_mode"] == "solo":
67
+ print(f"✅ '{query}' ({word_count} words) -> solo mode")
68
+ else:
69
+ print(f"❌ '{query}' ({word_count} words) -> {result['execution_mode']} (expected solo)")
70
+ all_passed = False
71
+
72
+ if all_passed:
73
+ print("✅ Requirement 4.2 PASSED")
74
+ return all_passed
75
+
76
+
77
+ def test_requirement_4_3():
78
+ """Test Requirement 4.3: Medium queries (≤25 words) route to standard mode"""
79
+ print("\n" + "="*60)
80
+ print("Testing Requirement 4.3: Medium queries route to standard mode")
81
+ print("="*60)
82
+
83
+ init_domain_packs()
84
+
85
+ test_cases = [
86
+ "Can you tell me about the weather today?",
87
+ "What are the latest developments in artificial intelligence and machine learning?",
88
+ "I would like to know more about the current economic situation in the United States.",
89
+ ]
90
+
91
+ all_passed = True
92
+ for query in test_cases:
93
+ result = decide_route(query)
94
+ word_count = len(query.split())
95
+
96
+ if 5 < word_count <= 25:
97
+ if result["complexity"] == "medium" and result["execution_mode"] == "standard":
98
+ print(f"✅ '{query[:50]}...' ({word_count} words) -> standard mode")
99
+ else:
100
+ print(f"❌ '{query[:50]}...' ({word_count} words) -> {result['execution_mode']} (expected standard)")
101
+ all_passed = False
102
+
103
+ if all_passed:
104
+ print("✅ Requirement 4.3 PASSED")
105
+ return all_passed
106
+
107
+
108
+ def test_requirement_4_4():
109
+ """Test Requirement 4.4: Complex queries (>25 words) route to deep mode"""
110
+ print("\n" + "="*60)
111
+ print("Testing Requirement 4.4: Complex queries route to deep mode")
112
+ print("="*60)
113
+
114
+ init_domain_packs()
115
+
116
+ query = "I need a comprehensive analysis of the current market conditions including economic indicators, sector performance, and potential risks that could impact my investment portfolio over the next quarter with detailed recommendations."
117
+
118
+ result = decide_route(query)
119
+ word_count = len(query.split())
120
+
121
+ if word_count > 25:
122
+ if result["complexity"] == "complex" and result["execution_mode"] == "deep":
123
+ print(f"✅ Query ({word_count} words) -> deep mode")
124
+ print("✅ Requirement 4.4 PASSED")
125
+ return True
126
+ else:
127
+ print(f"❌ Query ({word_count} words) -> {result['execution_mode']} (expected deep)")
128
+ return False
129
+ else:
130
+ print(f"❌ Test query has only {word_count} words (need >25)")
131
+ return False
132
+
133
+
134
+ def test_requirement_4_5_and_4_7():
135
+ """Test Requirements 4.5 & 4.7: Simulation keyword detection and task_family setting"""
136
+ print("\n" + "="*60)
137
+ print("Testing Requirements 4.5 & 4.7: Simulation keyword detection")
138
+ print("="*60)
139
+
140
+ init_domain_packs()
141
+
142
+ # Test with various simulation keywords
143
+ test_cases = [
144
+ "simulate the market reaction",
145
+ "predict the outcome",
146
+ "what if scenario analysis",
147
+ "model the reaction",
148
+ "test different scenarios",
149
+ ]
150
+
151
+ all_passed = True
152
+ for query in test_cases:
153
+ result = decide_route(query)
154
+
155
+ if result["task_family"] == "simulation":
156
+ print(f"✅ '{query}' -> task_family=simulation")
157
+ else:
158
+ print(f"❌ '{query}' -> task_family={result['task_family']} (expected simulation)")
159
+ all_passed = False
160
+
161
+ if all_passed:
162
+ print("✅ Requirements 4.5 & 4.7 PASSED")
163
+ return all_passed
164
+
165
+
166
+ def test_requirement_4_6():
167
+ """Test Requirement 4.6: Keywords are environment-configurable"""
168
+ print("\n" + "="*60)
169
+ print("Testing Requirement 4.6: Keywords are environment-configurable")
170
+ print("="*60)
171
+
172
+ # Check that keywords are loaded from config
173
+ if SIMULATION_TRIGGER_KEYWORDS:
174
+ print(f"✅ Simulation keywords loaded from config: {len(SIMULATION_TRIGGER_KEYWORDS)} keywords")
175
+ print(f" Sample keywords: {SIMULATION_TRIGGER_KEYWORDS[:5]}")
176
+ print("✅ Requirement 4.6 PASSED")
177
+ return True
178
+ else:
179
+ print("❌ No simulation keywords found in config")
180
+ return False
181
+
182
+
183
+ def test_requirement_5_6():
184
+ """Test Requirement 5.6: Domain pack detection using domain registry"""
185
+ print("\n" + "="*60)
186
+ print("Testing Requirement 5.6: Domain pack detection")
187
+ print("="*60)
188
+
189
+ init_domain_packs()
190
+ registry = get_registry()
191
+
192
+ # Test finance domain detection
193
+ test_cases = [
194
+ ("What is the stock market doing?", "finance"),
195
+ ("Tell me about NASDAQ", "finance"),
196
+ ("How is the weather?", "general"),
197
+ ("What are earnings reports?", "finance"),
198
+ ]
199
+
200
+ all_passed = True
201
+ for query, expected_domain in test_cases:
202
+ result = decide_route(query)
203
+ detected = result["domain_pack"]
204
+
205
+ if detected == expected_domain:
206
+ print(f"✅ '{query}' -> domain={detected}")
207
+ else:
208
+ print(f"❌ '{query}' -> domain={detected} (expected {expected_domain})")
209
+ all_passed = False
210
+
211
+ if all_passed:
212
+ print("✅ Requirement 5.6 PASSED")
213
+ return all_passed
214
+
215
+
216
+ def run_all_tests():
217
+ """Run all requirement tests"""
218
+ print("\n" + "="*60)
219
+ print("TASK 7 REQUIREMENTS VERIFICATION")
220
+ print("="*60)
221
+
222
+ tests = [
223
+ ("4.1", test_requirement_4_1),
224
+ ("4.2", test_requirement_4_2),
225
+ ("4.3", test_requirement_4_3),
226
+ ("4.4", test_requirement_4_4),
227
+ ("4.5 & 4.7", test_requirement_4_5_and_4_7),
228
+ ("4.6", test_requirement_4_6),
229
+ ("5.6", test_requirement_5_6),
230
+ ]
231
+
232
+ results = {}
233
+ for req_id, test_func in tests:
234
+ try:
235
+ results[req_id] = test_func()
236
+ except Exception as e:
237
+ print(f"\n❌ Requirement {req_id} FAILED with exception: {e}")
238
+ results[req_id] = False
239
+
240
+ # Summary
241
+ print("\n" + "="*60)
242
+ print("SUMMARY")
243
+ print("="*60)
244
+
245
+ passed = sum(1 for v in results.values() if v)
246
+ total = len(results)
247
+
248
+ for req_id, passed_test in results.items():
249
+ status = "✅ PASSED" if passed_test else "❌ FAILED"
250
+ print(f"Requirement {req_id}: {status}")
251
+
252
+ print(f"\nTotal: {passed}/{total} requirements passed")
253
+
254
+ return all(results.values())
255
+
256
+
257
+ if __name__ == "__main__":
258
+ success = run_all_tests()
259
+ exit(0 if success else 1)
backend/test_switchboard.py ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Test script for switchboard routing enhancements.
3
+ """
4
+
5
+ from app.agents.switchboard import decide_route
6
+ from app.domain_packs.init_packs import init_domain_packs
7
+
8
+
9
+ def test_routing():
10
+ """Test the enhanced switchboard routing."""
11
+
12
+ # Initialize domain packs
13
+ init_domain_packs()
14
+
15
+ # Test cases
16
+ test_cases = [
17
+ # Simple queries (≤5 words)
18
+ {
19
+ "input": "Hello world",
20
+ "expected": {
21
+ "complexity": "simple",
22
+ "execution_mode": "solo",
23
+ "task_family": "normal",
24
+ "domain_pack": "general"
25
+ }
26
+ },
27
+ {
28
+ "input": "What is AAPL?",
29
+ "expected": {
30
+ "complexity": "simple",
31
+ "execution_mode": "solo",
32
+ "task_family": "normal",
33
+ "domain_pack": "general" # AAPL alone doesn't trigger finance keywords
34
+ }
35
+ },
36
+
37
+ # Medium queries (≤25 words)
38
+ {
39
+ "input": "Can you tell me about the latest stock market trends and what's happening with tech stocks?",
40
+ "expected": {
41
+ "complexity": "medium",
42
+ "execution_mode": "standard",
43
+ "task_family": "normal",
44
+ "domain_pack": "finance"
45
+ }
46
+ },
47
+ {
48
+ "input": "What are the best practices for software development in modern teams?",
49
+ "expected": {
50
+ "complexity": "medium",
51
+ "execution_mode": "standard",
52
+ "task_family": "normal",
53
+ "domain_pack": "general"
54
+ }
55
+ },
56
+
57
+ # Complex queries (>25 words)
58
+ {
59
+ "input": "I need a comprehensive analysis of the current market conditions including economic indicators, sector performance, and potential risks that could impact my investment portfolio over the next quarter.",
60
+ "expected": {
61
+ "complexity": "complex",
62
+ "execution_mode": "deep",
63
+ "task_family": "normal",
64
+ "domain_pack": "finance"
65
+ }
66
+ },
67
+
68
+ # Simulation queries
69
+ {
70
+ "input": "Simulate the market reaction to a Fed rate hike",
71
+ "expected": {
72
+ "complexity": "complex",
73
+ "execution_mode": "deep",
74
+ "task_family": "simulation",
75
+ "domain_pack": "finance"
76
+ }
77
+ },
78
+ {
79
+ "input": "What if the company announces bankruptcy?",
80
+ "expected": {
81
+ "complexity": "complex",
82
+ "execution_mode": "deep",
83
+ "task_family": "simulation",
84
+ "domain_pack": "finance" # "bankruptcy" is a finance keyword
85
+ }
86
+ },
87
+ ]
88
+
89
+ print("Testing Switchboard Routing\n" + "="*50)
90
+
91
+ passed = 0
92
+ failed = 0
93
+
94
+ for i, test in enumerate(test_cases, 1):
95
+ user_input = test["input"]
96
+ expected = test["expected"]
97
+
98
+ result = decide_route(user_input)
99
+
100
+ print(f"\nTest {i}:")
101
+ print(f" Input: {user_input}")
102
+ print(f" Result: {result}")
103
+
104
+ # Check each expected field
105
+ test_passed = True
106
+ for key, expected_value in expected.items():
107
+ if result.get(key) != expected_value:
108
+ print(f" ❌ FAILED: {key} = {result.get(key)}, expected {expected_value}")
109
+ test_passed = False
110
+
111
+ if test_passed:
112
+ print(f" ✅ PASSED")
113
+ passed += 1
114
+ else:
115
+ failed += 1
116
+
117
+ print(f"\n{'='*50}")
118
+ print(f"Results: {passed} passed, {failed} failed out of {len(test_cases)} tests")
119
+
120
+ return failed == 0
121
+
122
+
123
+ if __name__ == "__main__":
124
+ success = test_routing()
125
+ exit(0 if success else 1)