Update README.md

https://cdn-uploads.huggingface.co/production/uploads/69e8826eb1347b4a2120bea7/HMTW0-vhwRSfdGLoECQY1.mp4

README.md

@@ -1,3 +1,432 @@
----
-license: mit
----
+---
+license: mit
+language:
+- en
+base_model:
+- Qwen/Qwen3.6-27B
+- Qwen/Qwen3.6-35B-A3B
+pipeline_tag: image-to-text
+tags:
+- medical
+---
+<div align="center">
+
+<img src="https://img.shields.io/badge/AMD_Instinct-MI300X-ED1C24?style=for-the-badge&logo=amd&logoColor=white" />
+<img src="https://img.shields.io/badge/ROCm-Stack-ED1C24?style=for-the-badge&logo=amd&logoColor=white" />
+<img src="https://img.shields.io/badge/vLLM-Inference-6D28D9?style=for-the-badge" />
+<img src="https://img.shields.io/badge/Qwen-Multimodal-0EA5E9?style=for-the-badge" />
+<img src="https://img.shields.io/badge/FastAPI-0.115-009688?style=for-the-badge&logo=fastapi&logoColor=white" />
+<img src="https://img.shields.io/badge/Python-3.12+-3776AB?style=for-the-badge&logo=python&logoColor=white" />
+
+<br /><br />
+
+# 🏥 MediAgent
+
+### Autonomous Multi-Agent Medical Imaging Analysis System
+
+**Five specialized AI agents. One radiological verdict. Running entirely on AMD.**
+
+*AMD Developer Hackathon 2026 · Track: Vision & Multimodal AI*
+
+<br />
+
+> Built by **Ramyar** — Security researcher & full-stack developer, Sulaymaniyah, Iraq
+
+</div>
+
+---
+
+## What Is MediAgent?
+
+MediAgent is a production-grade autonomous AI system that analyzes medical images — X-rays, MRI scans, CT scans — through a five-agent pipeline and generates structured, peer-reviewed clinical radiology reports in real time.
+
+Upload an image. Watch five AI agents execute live. Get a formal radiology report with differential diagnoses, ICD-10 codes, a quality score, and a FHIR R4 export ready for any EMR system.
+
+**No cloud APIs. No OpenAI. No Nvidia.**
+Pure AMD MI300X inference. Local. Private. Fast.
+
+---
+
+## The Pipeline
+
+```
+┌─────────────────────────────────────────────────────────────────────┐
+│                        IMAGE UPLOAD                                 │
+│              PNG / JPG / DICOM (.dcm) — up to 20 MB                │
+└──────────────────────────┬──────────────────────────────────────────┘
+                           │
+          ┌────────────────┴────────────────┐
+          │         PARALLEL STAGE          │
+          ▼                                 ▼
+┌─────────────────┐               ┌─────────────────┐
+│  INTAKE AGENT   │               │  VISION AGENT   │
+│                 │               │                 │
+│ • Validates     │               │ • Multimodal    │
+│   image payload │               │   Qwen analysis │
+│ • Normalizes    │               │ • Anatomical    │
+│   clinical text │               │   findings      │
+│ • Extracts      │               │ • Severity per  │
+│   demographics  │               │   region        │
+│ • Safety triage │               │ • Confidence    │
+│   (16 keywords) │               │   scoring       │
+│ • Modality hint │               │ • Anomaly flags │
+└────────┬────────┘               └────────┬────────┘
+         └──────────────┬──────────────────┘
+                        │
+                        ▼
+            ┌───────────────────────┐
+            │    RESEARCH AGENT     │
+            │                       │
+            │ • KB cross-reference  │
+            │   (15 conditions)     │
+            │ • Demographic weight  │
+            │ • Ranked differentials│
+            │ • ICD-10 codes        │
+            │ • Match probabilities │
+            └───────────┬───────────┘
+                        │
+                        ▼
+            ┌───────────────────────┐
+            │     REPORT AGENT      │
+            │                       │
+            │ • ACR/NICE format     │
+            │ • Clinical history    │
+            │ • Technique section   │
+            │ • Findings narrative  │
+            │ • Impression + top Dx │
+            │ • Recommendations     │
+            └───────────┬───────────┘
+                        │
+                        ▼
+            ┌───────────────────────┐
+            │     CRITIC AGENT      │
+            │                       │
+            │ • Cross-validates     │
+            │   report vs findings  │
+            │ • Quality score 0-100 │
+            │ • Uncertainty flags   │
+            │ • Disclaimer enforce  │
+            └───────────┬───────────┘
+                        │
+                        ▼
+┌─────────────────────────────────────────────────────────────────────┐
+│                      FINAL REPORT                                   │
+│         Structured JSON · PDF Export · FHIR R4 DiagnosticReport    │
+└─────────────────────────────────────────────────────────────────────┘
+```
+
+INTAKE and VISION execute **concurrently** — cutting wall-clock latency by running the two most expensive operations in parallel. Everything downstream sequences after both complete.
+
+---
+
+## AMD Hardware Stack
+
+| Component | Technology |
+|---|---|
+| **GPU** | AMD Instinct MI300X |
+| **GPU Software** | ROCm — AMD's open-source GPU compute platform |
+| **Inference Server** | vLLM (ROCm build) at `localhost:8000/v1` |
+| **Model** | Qwen multimodal — native vision + text |
+| **Backend** | FastAPI 0.115 + Uvicorn |
+| **Frontend** | Vanilla JS + Tailwind CSS + SSE streaming |
+
+This project is a direct proof of concept that AMD's ROCm stack is **production-viable for real-world medical AI**. Every inference call — vision analysis, clinical normalization, report synthesis, peer review, post-report chat — runs on AMD MI300X. Zero CUDA dependency. Zero cloud API calls.
+
+---
+
+## Key Features
+
+### 🔴 Real-Time SSE Streaming
+Watch the pipeline execute live, agent by agent. Every status transition — WAITING → RUNNING → DONE — streams to the dashboard as it happens via Server-Sent Events. Per-agent runtime counters track exactly how long each step takes.
+
+### 👁️ Multimodal Vision Analysis
+Qwen processes the raw medical image natively. It returns structured JSON: detected modality, technical quality assessment, per-region findings with anatomical names, radiological descriptions, severity levels (NORMAL / INCIDENTAL / SIGNIFICANT / CRITICAL), confidence scores (0–100), and anomaly flags.
+
+### 🔬 Medical Knowledge Base + ICD-10 Mapping
+The Research Agent cross-references vision findings against 15 curated clinical conditions spanning pulmonary, neurological, abdominal, musculoskeletal, and vascular pathology. Every differential diagnosis comes with an ICD-10 code, match probability, and a sentence explaining exactly why the condition matches the findings.
+
+### 🛡️ Critic Agent QA
+Every report goes through a peer-review pass before delivery. The Critic checks that all anomalies from the Vision Agent appear in the report, flags low-confidence findings, assigns a quality score (completeness 30% + accuracy 40% + safety 20% + compliance 10%), and hard-caps the score at 40/100 if a core agent failed.
+
+### 🏥 DICOM Support
+Upload real `.dcm` files. MediAgent extracts 20+ metadata fields — patient name, study date, institution, modality, body part, KVP, slice thickness, pixel spacing, image dimensions — and pre-populates the intake form automatically. MONOCHROME1 inversion and multi-frame handling included.
+
+### 📋 FHIR R4 Export
+Every report can be exported as a fully conformant HL7 FHIR R4 DiagnosticReport resource. Includes an inline Patient resource, Observation resources, LOINC and SNOMED CT codes, severity mapping, full report text in `presentedForm`, and custom extensions for AI quality score and pipeline status. Ready to import into Epic, Cerner, or any FHIR-capable EMR.
+
+### 💬 Post-Report Clinical Chat
+After the report is delivered, a ClinicalAdvisorAgent is available for follow-up questions. It answers in 2–4 sentences with direct reference to the report findings. Qwen's thinking/reasoning mode is explicitly disabled — answers are fast, direct, and clinical.
+
+### 🔒 Hard Safety Enforcement
+- **16 deterministic safety keywords** — chest pain, stroke symptoms, acute trauma, hemoptysis, sepsis, spinal trauma, and more — trigger urgent flags regardless of LLM output.
+- **Age-based alerts** — pediatric (<18) and geriatric (>75) cases are automatically flagged for expert review.
+- **Mandatory AI disclaimer** — enforced at two independent layers (Report Agent + Critic Agent) and cannot be bypassed or modified by the LLM.
+- **Graceful degradation** — the pipeline produces a report even if individual agents fail, always marking what succeeded and what didn't.
+
+### 📄 Client-Side PDF Export
+Full radiology report exported as a formatted PDF directly in the browser using jsPDF — severity color banner, all six report sections, DICOM metadata, QA score. No server round-trip needed.
+
+---
+
+## Agent Architecture
+
+### IntakeAgent
+Validates the image payload (minimum size, valid base64), applies deterministic safety triage, and normalizes clinical language. For simple inputs under 120 characters it skips the LLM entirely and uses a built-in layman-to-medical term map (22 entries: "can't breathe" → "dyspnea", "lump" → "mass/nodule", "dizzy" → "dizziness/vertigo", etc.). Only calls the LLM for complex clinical narratives with comorbidities or medical history. Falls back cleanly to raw input preservation if the LLM is unavailable.
+
+### VisionAgent
+Sends the base64 image and clinical context to Qwen at temperature 0.0 with a strict JSON schema enforced via system prompt. Handles malformed enum values from the LLM with safe conversion fallbacks — a single bad field never drops a finding. Tracks token usage and anomaly counts in the output metadata.
+
+### ResearchAgent
+Pre-filters the knowledge base to only conditions compatible with the detected modality before sending to the LLM — reducing prompt size and improving accuracy. Enforces strict output rules: only conditions from the KB, 2–4 differentials maximum, 5% minimum probability, exact ICD-10 codes, and evidence sentences that actually explain the match.
+
+### ReportAgent
+Builds a structured prompt with clearly labeled sections — clinical history, imaging technique, findings block, differentials block — and asks the LLM to synthesize them into a formal ACR/NICE radiology report. The disclaimer is overwritten to the exact regulatory string after LLM generation, unconditionally.
+
+### CriticAgent
+Operates at temperature 0.0 for fully deterministic QA. Receives the draft report and the full pipeline state including raw vision findings. Checks every anomaly is accounted for, flags low-confidence observations, and appends a `[QUALITY ASSESSMENT]` block to the recommendations section with score, issues, and uncertainty warnings.
+
+### ClinicalAdvisorAgent
+Activated only after report delivery, scoped to the specific report's findings. Strips all Qwen thinking output via multi-layer regex before returning the answer — handles `<think>` XML blocks, markdown think fences, and plain-text reasoning preambles.
+
+---
+
+## LLM Client
+
+The `LLMClient` wraps the OpenAI Python SDK pointed at the local vLLM endpoint. It handles:
+
+- Text completions with optional JSON mode enforcement
+- Multimodal completions with base64 image injection
+- Token-level streaming with an `on_token` callback
+- 3-attempt retry loop with 1-second flat backoff
+- 90-second timeout per call
+- Dual-strategy JSON extraction: direct parse first, then character-by-character brace-matching fallback for responses where the LLM adds conversational padding
+
+---
+
+## Medical Knowledge Base
+
+15 conditions covering the most common radiological findings across all supported modalities:
+
+| Condition | ICD-10 | Modalities | Severity |
+|---|---|---|---|
+| Community-Acquired Pneumonia | J18.9 | X-RAY, CT | SIGNIFICANT |
+| Cardiogenic Pulmonary Edema | J81.0 | X-RAY, CT | CRITICAL |
+| Pleural Effusion | J90 | X-RAY, CT, MRI | SIGNIFICANT |
+| Spontaneous Pneumothorax | J93.9 | X-RAY, CT | CRITICAL |
+| Intracerebral Hemorrhage | I61.9 | CT, MRI | CRITICAL |
+| Ischemic Stroke | I63.9 | CT, MRI | CRITICAL |
+| Intracranial Neoplasm | C71.9 | MRI, CT | SIGNIFICANT |
+| Abdominal Aortic Aneurysm | I71.4 | CT, MRI | CRITICAL |
+| Nephrolithiasis | N20.0 | CT, X-RAY | SIGNIFICANT |
+| Small Bowel Obstruction | K56.6 | X-RAY, CT | SIGNIFICANT |
+| Long Bone Fracture | S82.902 | X-RAY, CT | SIGNIFICANT |
+| Degenerative Joint Disease | M19.90 | X-RAY, MRI | INCIDENTAL |
+| Hepatic Steatosis | K76.0 | CT, MRI | INCIDENTAL |
+| Herniated Disc | M51.16 | MRI, CT | SIGNIFICANT |
+| Pulmonary Nodule | R91.1 | X-RAY, CT | SIGNIFICANT |
+
+---
+
+## API Reference
+
+| Method | Endpoint | Description |
+|---|---|---|
+| `GET` | `/` | Clinical dashboard UI |
+| `GET` | `/health` | System health, version, active sessions |
+| `GET` | `/metrics/gpu` | Live AMD GPU metrics (util, VRAM, temp, power) |
+| `POST` | `/analyze` | Synchronous pipeline → full JSON report |
+| `POST` | `/analyze/stream` | Real-time SSE streaming pipeline |
+| `GET` | `/status/{report_id}` | Poll live pipeline state |
+| `POST` | `/chat/{report_id}` | Post-report clinical Q&A |
+| `GET` | `/api/docs` | Swagger UI |
+| `GET` | `/api/redoc` | ReDoc UI |
+
+### `/analyze/stream` — SSE Event Types
+
+```json
+// Agent status update (emitted on every state transition)
+{"agent": "VISION", "status": "RUNNING"}
+{"agent": "VISION", "status": "DONE"}
+
+// Final report (emitted when pipeline completes)
+{"type": "report", "data": {...}, "report_id": "REP-A3F9C2D1B4E7"}
+
+// Error
+{"type": "error", "message": "Pipeline produced no report"}
+```
+
+### Form Fields (`/analyze`, `/analyze/stream`)
+
+| Field | Type | Required | Notes |
+|---|---|---|---|
+| `image` | File | ✅ | PNG, JPG, or DICOM (.dcm), max 20 MB |
+| `symptoms` | string | — | Free-text chief complaint |
+| `age` | integer | — | 0–120 |
+| `sex` | string | — | `M`, `F`, or `O` |
+| `clinical_context` | string | — | Medical history, referral details |
+
+---
+
+## Data Models
+
+```
+PatientInput
+    └── image_base64, symptoms, age, sex, clinical_context
+
+PipelineState
+    ├── agent_statuses: {INTAKE, VISION, RESEARCH, REPORT, CRITIC}
+    ├── intake_output: IntakeOutput
+    ├── vision_output: VisionOutput
+    │       └── findings: [VisionFinding, ...]
+    │               └── anatomical_region, description, severity,
+    │                   confidence, confidence_score, is_anomaly
+    ├── research_output: ResearchOutput
+    │       └── differential_diagnoses: [KnowledgeMatch, ...]
+    │               └── condition_name, match_probability,
+    │                   supporting_evidence, differential_rank, icd10_code
+    ├── report_draft: ReportSection
+    │       └── clinical_history, technique, findings, impression,
+    │           recommendations, disclaimer
+    └── final_report: FinalReport
+            └── report_id, patient_metadata, sections, vision_summary,
+                research_summary, overall_severity, agent_pipeline_status,
+                generation_timestamp
+```
+
+---
+
+## Project Structure
+
+```
+mediagent/
+├── main.py                  ← FastAPI server, all routes, SSE orchestration
+├── core/
+│   ├── llm.py               ← LLM client (retry, vision, streaming, JSON extraction)
+│   ├── models.py            ← All Pydantic v2 data models
+│   ├── pipeline.py          ← Parallel pipeline orchestrator
+│   ├── dicom.py             ← DICOM parser (pydicom + numpy + Pillow)
+│   └── fhir.py              ← FHIR R4 DiagnosticReport builder
+├── agents/
+│   ├── intake.py            ← Input validation, normalization, safety triage
+│   ├── vision.py            ← Multimodal image analysis
+│   ├── research.py          ← KB matching, ICD-10, differential diagnosis
+│   ├── report.py            ← ACR/NICE radiology report synthesis
+│   ├── critic.py            ← QA validation, quality scoring
+│   └── advisor.py           ← Post-report clinical Q&A
+├── static/
+│   └── index.html           ← Full dashboard (Tailwind + Chart.js + SSE)
+├── requirements.txt
+└── .env.example
+```
+
+---
+
+## Getting Started
+
+### Prerequisites
+
+- Python 3.12+
+- vLLM running a Qwen multimodal model on ROCm, accessible at `http://localhost:8000/v1`
+- ROCm-compatible AMD GPU (MI300X recommended)
+
+### Installation
+
+```bash
+# Clone the repository
+git clone https://github.com/Ramyar2007/mediagent
+cd mediagent
+
+# Install Python dependencies
+pip install -r requirements.txt
+
+# Configure environment
+cp .env.example .env
+# Edit .env and set LLM_BASE_URL to your vLLM endpoint
+```
+
+### Environment Variables
+
+```env
+LLM_BASE_URL=http://localhost:8000/v1   # vLLM OpenAI-compatible endpoint
+LLM_MODEL=/model                         # Model path served by vLLM
+APP_PORT=8090                            # Server port
+```
+
+### Run
+
+```bash
+python main.py
+```
+
+Dashboard available at **http://localhost:8090**
+
+Swagger docs at **http://localhost:8090/api/docs**
+
+---
+
+## Dependencies
+
+| Package | Version | Purpose |
+|---|---|---|
+| `fastapi` | 0.115.6 | Web framework |
+| `uvicorn[standard]` | 0.34.0 | ASGI server |
+| `openai` | 1.58.1 | SDK for vLLM OpenAI-compatible API |
+| `python-multipart` | 0.0.20 | Multipart form / file upload |
+| `pydantic` | 2.10.5 | Data validation and serialization |
+| `Pillow` | 11.1.0 | Image processing for DICOM conversion |
+| `pydicom` | 2.4.4 | DICOM file parsing and metadata extraction |
+| `numpy` | 1.26.4 | Pixel array normalization for DICOM |
+
+Optional: `amdsmi` Python library — used automatically when available for more accurate GPU metrics than the `rocm-smi` CLI fallback.
+
+---
+
+## Clinical Safety
+
+MediAgent is built with clinical safety as a first-class concern, not an afterthought.
+
+**Mandatory disclaimer** — enforced at two independent code layers and cannot be overridden by any LLM output:
+
+> *"This analysis is AI-generated and must be reviewed by a licensed radiologist before any clinical decisions are made."*
+
+**Hard safety rules that run deterministically, without LLM involvement:**
+- 16 urgent clinical keywords trigger immediate flags before any AI processing
+- Pediatric and geriatric age thresholds auto-flag for specialist review
+- Quality score is hard-capped at 40/100 if core agents (Vision, Report) fail
+- Low-confidence findings are always flagged with confirmatory imaging recommendations
+- The disclaimer is re-enforced after every LLM call, unconditionally
+
+**This system is a decision support tool, not a clinical decision maker.** Every output is intended to assist, not replace, a licensed radiologist.
+
+---
+
+## Dashboard Preview
+
+The single-page clinical dashboard provides:
+
+- **Live pipeline panel** — real-time agent status cards with per-step runtime counters
+- **Analytics tab** — severity distribution donut chart, differential diagnosis confidence bar chart, agent timing bar chart — all populated from structured model output
+- **Report panel** — severity banner, safety flags, all six report sections, finding cards color-coded by severity
+- **DICOM metadata card** — study date, institution, modality, body part, technical parameters
+- **PDF export** — full formatted report generated client-side
+- **Clinical chat** — slide-up Q&A panel backed by the ClinicalAdvisorAgent
+- **AMD GPU panel** — live util %, VRAM used/total, temperature, power draw — polling every 3 seconds
+
+---
+
+## Built For
+
+**AMD Developer Hackathon 2026**
+Track: Vision & Multimodal AI
+
+This project demonstrates that AMD's ROCm ecosystem is a complete, production-viable alternative for serious AI workloads. Medical imaging analysis — with real multimodal vision, structured clinical reasoning, and standards-compliant output — running fully on AMD MI300X without a single NVIDIA or cloud dependency.
+
+---
+
+<div align="center">
+
+**Built by Ramyar · Sulaymaniyah, Iraq**
+
+*#AMDDevChallenge · AMD Instinct MI300X · ROCm · vLLM · Qwen*
+
+</div>