Skill: shuyuew1991/videomp3word-mcp (version 2.0.3)

--- name: "videomp3word-mcp" description: "Structured knowledge extraction MCP server for ClawHub/OpenClaw. Converts remote media URLs into summary, topics, action items, Q&A, flashcards, entities, confidence scores, and workflow traces through one task-oriented tool." env: VIDEOMP3WORD_SESSION_COOKIE: "Required. Session cookie for the upstream videomp3word account used for transcription." VIDEOMP3WORD_BASE_URL: "Optional. Upstream site URL. Defaults to https://videomp3word.com." VIDEOMP3WORD_ALLOWED_UPSTREAM_HOSTS: "Optional. Comma-separated allowlist for VIDEOMP3WORD_BASE_URL." VIDEOMP3WORD_API_KEY: "Optional. Upstream API key sent alongside the session cookie." KNOWLEDGE_MODEL_API_KEY: "Optional. Explicit API key for the structured knowledge model. Falls back to DASHSCOPE_API_KEY." DASHSCOPE_API_KEY: "Optional. DashScope API key used by the original videomp3word knowledge flows when KNOWLEDGE_MODEL_API_KEY is unset." KNOWLEDGE_MODEL_API_BASE: "Optional. OpenAI-compatible base URL for the knowledge model. Defaults to https://dashscope.aliyuncs.com/compatible-mode/v1." KNOWLEDGE_MODEL_NAME: "Optional. Model name for structured knowledge extraction. Defaults to qwen-plus." MONGO_URI: "Optional. MongoDB connection string for persistent resources." MONGO_DB_NAME: "Optional. MongoDB database name." MCP_ACCESS_KEYS: "Optional. Comma-separated bearer keys that gate the main tool and REST endpoint." PUBLIC_BASE_URL: "Optional. Public base URL of this deployment." HOST: "Optional. Bind host." PORT: "Optional. Listen port." --- # Videomp3word Structured Knowledge Engine Use this MCP server when the task is to turn a remote audio or video URL into structured, reusable knowledge for downstream products or automations. ## Primary Tool - `video_to_knowledge` Inputs: - `media_url` - `outputs`: any combination of `summary`, `qa`, `flashcards`, `tasks`, `topics` - `mode`: `fast`, `balanced`, or `high_accuracy` Outputs include: - summary - topics - key points - action items - Q&A pairs - flashcards - entities - confidence scores - workflow trace with models and chunk references ## Positioning Prefer this server when an agent needs: - one task-oriented MCP call instead of many small tools - structured JSON for automations - traceability for enterprise workflows - export-ready artifacts for markdown or Notion - cacheable processing with persistent resources - ClawHub-compatible stdio support ## Notes - The server keeps a single high-level MCP tool to stay commercially productized and easier to publish. - The upstream session cookie is sensitive and should come from a dedicated account. - Transcript text, chunk context, and media URLs are sent to the configured upstream transcription service and, when enabled, to the configured knowledge model endpoint. Audit and trust those endpoints before deployment. - Installation is manual for this bundle: run `npm install`, `npm run build`, and `npm start` or launch `node dist/index.js stdio`. - The default knowledge-model base matches the original videomp3word deployment and points to DashScope-compatible OpenAI APIs unless you override `KNOWLEDGE_MODEL_API_BASE`. - Non-local `VIDEOMP3WORD_BASE_URL` and `KNOWLEDGE_MODEL_API_BASE` values should use HTTPS because credential-bearing requests are sent to those services. - Configure `MCP_ACCESS_KEYS` before exposing the server publicly.

path: README.md
content: # videomp3word-mcp

`videomp3word-mcp` is a production-oriented MCP and HTTP server that turns unstructured audio/video into structured, monetizable knowledge artifacts through one API.

## What Changed

The previous flat tool proxy has been replaced with a task-oriented knowledge engine:

- one high-level REST endpoint: `POST /video_to_knowledge`
- one primary MCP tool: `video_to_knowledge`
- structured JSON outputs instead of free text
- persistent resources for `MediaAsset`, `Transcript`, `SemanticChunk`, and `KnowledgeUnit`
- DAG workflow execution with retries, checkpoints, and step traces
- modular agents for planning, transcription, semantics, knowledge generation, and evaluation
- export formats for `json`, `markdown`, and Notion-ready blocks
- minimal browser UI at `/`

## Architecture

```text
src/
  agents/        planner, transcription, semantic, knowledge, evaluation
  platform/      orchestration container + video-to-knowledge service
  services/      upstream transcription, model client, exporters
  storage/       repository abstraction + memory/mongodb implementations
  workflow/      DAG executor and workflow types
  ui/            minimal browser playground
  utils/         hashing, safe URL validation, text utilities
  schemas.ts     request/response schemas
  httpServer.ts  REST + MCP-over-HTTP adapter
  mcpServer.ts   MCP stdio/http registration
  index.ts       server bootstrap
```

## Core Endpoint

### `POST /video_to_knowledge`

Request:

```json
{
  "media_url": "https://example.com/demo.mp4",
  "outputs": ["summary", "qa", "flashcards", "tasks", "topics"],
  "mode": "balanced",
  "export_formats": ["json", "markdown", "notion"]
}
```

Response fields:

- `summary`
- `topics`
- `key_points`
- `action_items`
- `qa_pairs`
- `flashcards`
- `entities`
- `confidence_scores`
- `trace`
- `exports`

Sample output: `examples/sample_output.json`

## MCP Surface

- `video_to_knowledge`: primary task tool for ClawHub/OpenClaw style clients
- `videomp3word://knowledge-schema`: resource describing the contract and modes

The package still supports both transports:

- HTTP streamable MCP at `POST /mcp`
- stdio via `npx videomp3word-mcp stdio`

## Modes

- `fast`: lowest cost, larger chunks, shallower reasoning
- `balanced`: default production mode
- `high_accuracy`: smaller chunks, deeper extraction, richer outputs

Mode controls:

- chunk size
- output depth
- model labels in the trace
- transcription settings used for the upstream media call

## Persistence Layer

Collections and indexes:

- `media_assets`: unique index on `normalizedSourceUrl`
- `transcripts`: unique index on `mediaAssetId + transcriptionFingerprint`
- `semantic_chunks`: unique index on `transcriptId + chunkConfigHash + index`
- `knowledge_units`: unique index on `transcriptId + requestHash`
- `workflow_runs`: indexed by `requestHash + createdAt`

If `MONGO_URI` is not configured, the server uses an in-memory repository for local development and smoke tests.

## Environment

Required:

- `VIDEOMP3WORD_SESSION_COOKIE`: upstream videomp3word session used for transcription. Use a dedicated upstream account and inject this at runtime only; never commit it into the package or a checked-in `.env` file.

Recommended:

- `MCP_ACCESS_KEYS`: bearer keys for the REST endpoint and MCP tool
- `MONGO_URI`: MongoDB connection string for persistent storage
- `KNOWLEDGE_MODEL_API_KEY`: explicit API key for the structured knowledge model
- `DASHSCOPE_API_KEY`: fallback key for the original videomp3word DashScope-backed knowledge flow when `KNOWLEDGE_MODEL_API_KEY` is unset
- `KNOWLEDGE_MODEL_API_BASE`: OpenAI-compatible model endpoint, defaults to `https://dashscope.aliyuncs.com/compatible-mode/v1` to match the original videomp3word deployment
- `KNOWLEDGE_MODEL_NAME`: knowledge model name, default `qwen-plus`
- `PUBLIC_BASE_URL`: public base URL of this deployment

## Install And Run

This package ships runnable code, but deployment remains manual: install dependencies, build the TypeScript sources, and then start the server yourself.

```bash
npm install
npm run build
npm start
```

Security notes:

- `VIDEOMP3WORD_BASE_URL` and `KNOWLEDGE_MODEL_API_BASE` must use `https` for non-local deployments so credential-bearing requests are not sent over cleartext HTTP.
- Transcript text, chunk context, and media URLs are sent to the configured upstream transcription service and, when enabled, to the configured knowledge model endpoint. Audit and trust those endpoints before deployment.
- `MCP_ACCESS_KEYS` should be configured before exposing the HTTP or MCP endpoints outside a trusted network.

Local endpoints:

- `GET /health`
- `GET /docs`
- `GET /`
- `POST /video_to_knowledge`
- `POST /mcp`

## Python SDK

Wrapper module: `sdk/python/videomp3word_client.py`

Example:

```python
from sdk.python.videomp3word_client import Videomp3wordClient

client = Videomp3wordClient(base_url="http://localhost:3000", bearer_token="your-access-key")
result = client.video_to_knowledge(
    media_url="https://example.com/demo.mp4",
    outputs=["summary", "qa", "flashcards", "tasks", "topics"],
    mode="balanced",
)
print(result["summary"])
```

## Example Scripts

- `examples/video_to_blog_post.py`
- `examples/meeting_to_action_items.py`
- `examples/podcast_to_summary.py`

## Commercial Positioning

This server is designed as a structured knowledge extraction engine rather than a thin media utility wrapper. That makes it more suitable for SaaS monetization because one call can produce:

- summary content for blogs and newsletters
- Q&A assets for search and support experiences
- flashcards for education products
- action items for meetings and internal productivity workflows
- traceable, exportable JSON for downstream automation

## Verification

```bash
npm run build
npm run smoke
```


path: examples/meeting_to_action_items.py
content: from sdk.python.videomp3word_client import Videomp3wordClient

client = Videomp3wordClient(base_url="http://localhost:3000", bearer_token="your-access-key")
result = client.video_to_knowledge(
    media_url="https://example.com/meeting-recording.mp4",
    outputs=["summary", "tasks", "topics"],
    mode="balanced",
    export_formats=["json", "notion"],
)

for item in result["action_items"]:
    print(f"- {item['task']} | owner={item['owner']} | deadline={item['deadline']}")


path: examples/podcast_to_summary.py
content: from sdk.python.videomp3word_client import Videomp3wordClient

client = Videomp3wordClient(base_url="http://localhost:3000", bearer_token="your-access-key")
result = client.video_to_knowledge(
    media_url="https://example.com/podcast.mp3",
    outputs=["summary", "topics", "qa"],
    mode="fast",
    export_formats=["json"],
)

print(result["summary"])
print(result["topics"])


path: examples/sample_output.json
content: {
  "request_id": "request_123",
  "mode": "balanced",
  "requested_outputs": ["summary", "qa", "flashcards", "tasks", "topics"],
  "summary": "The speaker outlines a roadmap for an AI workflow platform and assigns follow-up work for chunking, traceability, and exports.",
  "topics": ["AI workflows", "semantic chunking", "traceability", "export formats"],
  "key_points": [
    "The platform should move from raw tools to a single task-oriented API.",
    "Workflow execution needs checkpointing, retries, and reusable resources.",
    "Structured exports create stronger monetizable outputs for SaaS use cases."
  ],
  "action_items": [
    {
      "task": "Alice will send the roadmap by next Friday.",
      "owner": "Alice",
      "deadline": "next Friday"
    }
  ],
  "qa_pairs": [
    {
      "question": "What does the transcript say about semantic chunking?",
      "answer": "Semantic chunking is treated as part of the structured knowledge extraction workflow."
    }
  ],
  "flashcards": [
    {
      "front": "Explain semantic chunking",
      "back": "Semantic chunking breaks a transcript into reusable knowledge blocks for downstream extraction."
    }
  ],
  "entities": [
    {
      "name": "Alice",
      "type": "person_or_concept"
    }
  ],
  "confidence_scores": {
    "overall": 0.89,
    "summary": 0.92,
    "topics": 0.85
  },
  "trace": {
    "workflow_run_id": "workflow_123",
    "steps": [
      {
        "step": "ingest_media",
        "status": "completed",
        "attempt": 1,
        "duration_ms": 5,
        "cached": false
      }
    ],
    "models_used": ["planner-balanced:v1", "videomp3word-video2word:balanced", "knowledge-model"],
    "chunk_references": ["chunk_1", "chunk_2"],
    "resource_ids": {
      "media_asset_id": "media_123",
      "transcript_id": "transcript_123",
      "knowledge_unit_id": "knowledge_123"
    }
  },
  "exports": {
    "json": "{...}",
    "markdown": "# Video To Knowledge",
    "notion": {
      "title": "Video To Knowledge",
      "blocks": []
    }
  }
}


path: examples/video_to_blog_post.py
content: from sdk.python.videomp3word_client import Videomp3wordClient

client = Videomp3wordClient(base_url="http://localhost:3000", bearer_token="your-access-key")
result = client.video_to_knowledge(
    media_url="https://example.com/podcast-episode.mp4",
    outputs=["summary", "topics", "qa", "flashcards"],
    mode="high_accuracy",
    export_formats=["json", "markdown"],
)

print("Blog-ready summary:\n")
print(result["summary"])
print("\nMarkdown export:\n")
print(result["exports"]["markdown"])


path: package.json
content: {
  "name": "videomp3word-mcp",
  "version": "2.0.3",
  "mcp": {
    "command": "node",
    "args": [
      "dist/index.js",
      "stdio"
    ],
    "env": {
      "VIDEOMP3WORD_SESSION_COOKIE": {
        "description": "Required session cookie for the upstream videomp3word account used for media transcription.",
        "required": true
      },
      "VIDEOMP3WORD_BASE_URL": {
        "description": "Optional upstream site URL, defaults to https://videomp3word.com",
        "required": false
      },
      "VIDEOMP3WORD_ALLOWED_UPSTREAM_HOSTS": {
        "description": "Optional comma-separated allowed hostnames for VIDEOMP3WORD_BASE_URL",
        "required": false
      },
      "VIDEOMP3WORD_API_KEY": {
        "description": "Optional upstream API key sent alongside the required session cookie when available",
        "required": false
      },
      "KNOWLEDGE_MODEL_API_KEY": {
        "description": "Optional explicit API key for the structured knowledge model. Falls back to DASHSCOPE_API_KEY when unset.",
        "required": false
      },
      "DASHSCOPE_API_KEY": {
        "description": "Optional DashScope API key used by the original videomp3word knowledge flows when KNOWLEDGE_MODEL_API_KEY is unset.",
        "required": false
      },
      "KNOWLEDGE_MODEL_API_BASE": {
        "description": "Optional OpenAI-compatible base URL for the knowledge model. Defaults to https://dashscope.aliyuncs.com/compatible-mode/v1 to match the original videomp3word deployment.",
        "required": false
      },
      "KNOWLEDGE_MODEL_NAME": {
        "description": "Optional model name for knowledge extraction, defaults to qwen-plus",
        "required": false
      },
      "MONGO_URI": {
        "description": "Optional MongoDB connection string. When unset, the server uses an in-memory repository for local development.",
        "required": false
      },
      "MONGO_DB_NAME": {
        "description": "Optional MongoDB database name, defaults to videomp3word_mcp",
        "required": false
      },
      "MCP_ACCESS_KEYS": {
        "description": "Optional comma-separated bearer keys that gate the production endpoint and MCP tool",
        "required": false
      },
      "PUBLIC_BASE_URL": {
        "description": "Optional public base URL of this MCP deployment",
        "required": false
      },
      "HOST": {
        "description": "Optional bind host, defaults to 0.0.0.0",
        "required": false
      },
      "PORT": {
        "description": "Optional port, defaults to 3000",
        "required": false
      }
    }
  },
  "description": "Production-ready MCP and HTTP server that turns audio/video into structured knowledge artifacts through a single API.",
  "author": "Emzan Technology Ltd.",
  "license": "MIT",
  "private": false,
  "type": "module",
  "main": "dist/index.js",
  "bin": {
    "videomp3word-mcp": "dist/index.js"
  },
  "files": [
    "dist",
    "SKILL.md",
    "sdk",
    "examples"
  ],
  "keywords": [
    "mcp",
    "mcp-server",
    "videomp3word",
    "knowledge-extraction",
    "transcription",
    "video",
    "audio",
    "structured-output"
  ],
  "scripts": {
    "dev": "tsx watch src/index.ts",
    "build": "tsc -p tsconfig.json",
    "smoke": "npm run build && node --test dist/smoke.test.js",
    "start": "node dist/index.js",
    "lint": "tsc --noEmit",
    "typecheck": "tsc --noEmit"
  },
  "engines": {
    "node": ">=20"
  },
  "dependencies": {
    "@modelcontextprotocol/sdk": "^1.17.5",
    "express": "^5.1.0",
    "mongodb": "^6.17.0",
    "zod": "^3.25.76"
  },
  "devDependencies": {
    "@types/express": "^5.0.3",
    "@types/node": "^24.5.2",
    "tsx": "^4.20.5",
    "typescript": "^5.9.2"
  }
}


path: sdk/python/videomp3word_client.py
content: from __future__ import annotations

import json
import urllib.request
from typing import Iterable, Optional


class Videomp3wordClient:
    def __init__(self, base_url: str, bearer_token: Optional[str] = None, timeout: int = 300):
        self.base_url = base_url.rstrip("/")
        self.bearer_token = bearer_token
        self.timeout = timeout

    def video_to_knowledge(
        self,
        media_url: str,
        outputs: Iterable[str],
        mode: str = "balanced",
        export_formats: Optional[Iterable[str]] = None,
        metadata: Optional[dict] = None,
    ) -> dict:
        payload = {
            "media_url": media_url,
            "outputs": list(outputs),
            "mode": mode,
            "export_formats": list(export_formats or ["json"]),
        }
        if metadata:
            payload["metadata"] = metadata

        request = urllib.request.Request(
            url=f"{self.base_url}/video_to_knowledge",
            data=json.dumps(payload).encode("utf-8"),
            headers=self._headers(),
            method="POST",
        )
        with urllib.request.urlopen(request, timeout=self.timeout) as response:
            return json.loads(response.read().decode("utf-8"))

    def _headers(self) -> dict:
        headers = {"Content-Type": "application/json"}
        if self.bearer_token:
            headers["Authorization"] = f"Bearer {self.bearer_token}"
        return headers


path: src/agents/evaluationAgent.ts
content: import type { KnowledgeResponse } from "../schemas.js";
import type { SemanticChunkRecord, TranscriptRecord } from "../storage/types.js";
import type { ExecutionPlan } from "./plannerAgent.js";

export class EvaluationAgent {
  evaluate(
    transcript: TranscriptRecord,
    chunks: SemanticChunkRecord[],
    payload: Omit<KnowledgeResponse, "request_id" | "trace" | "exports">,
    plan: ExecutionPlan,
  ): Record<string, number> {
    const transcriptLength = Math.max(1, transcript.text.length);
    const summaryCoverage = Math.min(1, payload.summary.length / Math.min(400, transcriptLength));
    const topicCoverage = Math.min(1, payload.topics.length / Math.max(1, plan.modeProfile.maxTopics));
    const keyPointCoverage = Math.min(1, payload.key_points.length / Math.max(1, plan.modeProfile.maxKeyPoints));
    const entityCoverage = Math.min(1, payload.entities.length / 6);
    const qaCoverage = plan.requestedOutputs.includes("qa") ? Math.min(1, payload.qa_pairs.length / Math.max(1, plan.modeProfile.qaPairs)) : 1;
    const flashcardCoverage = plan.requestedOutputs.includes("flashcards") ? Math.min(1, payload.flashcards.length / Math.max(1, plan.modeProfile.flashcards)) : 1;
    const taskCoverage = plan.requestedOutputs.includes("tasks") ? Math.min(1, payload.action_items.length / Math.max(1, Math.min(4, plan.modeProfile.maxKeyPoints))) : 1;
    const chunkCoverage = Math.min(1, chunks.length / 3);
    const overall = Number(((summaryCoverage + topicCoverage + keyPointCoverage + entityCoverage + qaCoverage + flashcardCoverage + taskCoverage + chunkCoverage) / 8).toFixed(4));

    return {
      overall,
      summary: Number(summaryCoverage.toFixed(4)),
      topics: Number(topicCoverage.toFixed(4)),
      key_points: Number(keyPointCoverage.toFixed(4)),
      entities: Number(entityCoverage.toFixed(4)),
      qa_pairs: Number(qaCoverage.toFixed(4)),
      flashcards: Number(flashcardCoverage.toFixed(4)),
      action_items: Number(taskCoverage.toFixed(4)),
      chunk_coverage: Number(chunkCoverage.toFixed(4)),
    };
  }
}


path: src/agents/knowledgeAgent.ts
content: import type { ActionItem, Entity, Flashcard, KnowledgeResponse, QAPair } from "../schemas.js";
import { clipText, extractActionItems, splitIntoSentences, uniqueStrings } from "../utils/text.js";
import { createId } from "../utils/hash.js";
import type { PlatformRepository, KnowledgeUnitRecord, SemanticChunkRecord, TranscriptRecord } from "../storage/types.js";
import type { ExecutionPlan } from "./plannerAgent.js";
import { ModelClient } from "../services/modelClient.js";

type StructuredKnowledgeDraft = {
  summary?: string;
  topics?: string[];
  key_points?: string[];
  action_items?: ActionItem[];
  qa_pairs?: QAPair[];
  flashcards?: Flashcard[];
  entities?: Entity[];
};

export class KnowledgeAgent {
  constructor(
    private readonly repository: PlatformRepository,
    private readonly modelClient: ModelClient,
  ) {}

  async execute(transcript: TranscriptRecord, chunks: SemanticChunkRecord[], plan: ExecutionPlan) {
    const cached = await this.repository.getKnowledgeUnit(transcript.id, plan.requestHash);
    if (cached) {
      return { knowledgeUnit: cached, cached: true, modelUsed: cached.modelsUsed, chunkReferences: cached.chunkReferences };
    }

    const draft = await this.generateKnowledgeDraft(transcript, chunks, plan);
    const now = new Date().toISOString();
    const knowledgeUnit: KnowledgeUnitRecord = {
      id: createId("knowledge"),
      transcriptId: transcript.id,
      mediaAssetId: transcript.mediaAssetId,
      requestHash: plan.requestHash,
      mode: plan.modeProfile.mode,
      requestedOutputs: plan.requestedOutputs,
      result: {
        mode: plan.modeProfile.mode,
        requested_outputs: plan.requestedOutputs,
        summary: draft.summary || "",
        topics: draft.topics || [],
        key_points: draft.key_points || [],
        action_items: draft.action_items || [],
        qa_pairs: draft.qa_pairs || [],
        flashcards: draft.flashcards || [],
        entities: draft.entities || [],
        confidence_scores: {},
      },
      traceSteps: [],
      modelsUsed: [this.modelClient.isEnabled() ? "knowledge-model" : "heuristic-knowledge"],
      chunkReferences: chunks.map((chunk) => chunk.id),
      createdAt: now,
      updatedAt: now,
    };

    await this.repository.upsertKnowledgeUnit(knowledgeUnit);
    return { knowledgeUnit, cached: false, modelUsed: knowledgeUnit.modelsUsed, chunkReferences: knowledgeUnit.chunkReferences };
  }

  async persistEvaluation(knowledgeUnit: KnowledgeUnitRecord, updatedResult: Omit<KnowledgeResponse, "request_id" | "trace" | "exports">) {
    const updated: KnowledgeUnitRecord = {
      ...knowledgeUnit,
      result: updatedResult,
      updatedAt: new Date().toISOString(),
    };
    await this.repository.upsertKnowledgeUnit(updated);
    return updated;
  }

  private async generateKnowledgeDraft(transcript: TranscriptRecord, chunks: SemanticChunkRecord[], plan: ExecutionPlan): Promise<StructuredKnowledgeDraft> {
    const modelDraft = await this.tryModelDraft(transcript, chunks, plan);
    if (modelDraft) return this.sanitizeDraft(modelDraft, chunks, plan);
    return this.generateHeuristicDraft(transcript, chunks, plan);
  }

  private async tryModelDraft(transcript: TranscriptRecord, chunks: SemanticChunkRecord[], plan: ExecutionPlan): Promise<StructuredKnowledgeDraft | null> {
    if (!this.modelClient.isEnabled()) return null;

    const chunkContext = chunks.map((chunk) => `Chunk ${chunk.index + 1}: ${clipText(chunk.text, 600)}`).join("\n\n");
    const messages = [
      {
        role: "system" as const,
        content: "You are a structured knowledge extraction engine. Return JSON only. Preserve fidelity to the transcript and avoid fabrication. Use empty arrays or empty strings when a requested field has insufficient evidence.",
      },
      {
        role: "user" as const,
        content: [
          `Requested outputs: ${plan.requestedOutputs.join(", ")}`,
          `Execution mode: ${plan.modeProfile.mode}`,
          "Return this JSON shape exactly:",
          JSON.stringify({
            summary: "",
            topics: [],
            key_points: [],
            action_items: [{ task: "", owner: "", deadline: "" }],
            qa_pairs: [{ question: "", answer: "" }],
            flashcards: [{ front: "", back: "" }],
            entities: [{ name: "", type: "" }],
          }),
          "Transcript excerpt:",
          clipText(transcript.text, 8000),
          "Chunk context:",
          chunkContext,
        ].join("\n\n"),
      },
    ];
    return this.modelClient.generateJson<StructuredKnowledgeDraft>(messages, "qwen-plus");
  }

  private sanitizeDraft(draft: StructuredKnowledgeDraft, chunks: SemanticChunkRecord[], plan: ExecutionPlan): StructuredKnowledgeDraft {
    const entities = uniqueEntityList([...(draft.entities || []), ...chunks.flatMap((chunk) => chunk.entities)], 12);
    return {
      summary: typeof draft.summary === "string" ? draft.summary.trim() : "",
      topics: uniqueStrings(draft.topics || [], plan.modeProfile.maxTopics),
      key_points: uniqueStrings(draft.key_points || [], plan.modeProfile.maxKeyPoints),
      action_items: sanitizeActionItems(draft.action_items || []).slice(0, plan.modeProfile.maxKeyPoints),
      qa_pairs: sanitizeQaPairs(draft.qa_pairs || []).slice(0, plan.modeProfile.qaPairs),
      flashcards: sanitizeFlashcards(draft.flashcards || []).slice(0, plan.modeProfile.flashcards),
      entities,
    };
  }

  private generateHeuristicDraft(transcript: TranscriptRecord, chunks: SemanticChunkRecord[], plan: ExecutionPlan): StructuredKnowledgeDraft {
    const sentences = splitIntoSentences(transcript.text);
    const keywords = uniqueStrings(chunks.flatMap((chunk) => chunk.keywords), plan.modeProfile.maxTopics);
    const keyPoints = uniqueStrings(
      [
        ...chunks.slice(0, plan.modeProfile.maxKeyPoints).map((chunk) => clipText(chunk.text, 180)),
        ...sentences.filter((sentence) => /\b(decide|plan|important|because|need|will|must|should)\b/i.test(sentence)).map((sentence) => clipText(sentence, 180)),
      ],
      plan.modeProfile.maxKeyPoints,
    );
    const summary = keyPoints.slice(0, Math.max(3, Math.min(5, keyPoints.length))).join(" ") || clipText(transcript.text, 320);
    const actionItems = extractActionItems(sentences, plan.modeProfile.maxKeyPoints);
    const qaPairs = buildQaPairs(keywords, keyPoints, plan.modeProfile.qaPairs);
    const flashcards = buildFlashcards(keywords, keyPoints, plan.modeProfile.flashcards);
    const entities = uniqueEntityList(chunks.flatMap((chunk) => chunk.entities), 12);

    return {
      summary,
      topics: keywords,
      key_points: keyPoints,
      action_items: plan.requestedOutputs.includes("tasks") ? actionItems : [],
      qa_pairs: plan.requestedOutputs.includes("qa") ? qaPairs : [],
      flashcards: plan.requestedOutputs.includes("flashcards") ? flashcards : [],
      entities,
    };
  }
}

function uniqueEntityList(values: Entity[], limit: number): Entity[] {
  const seen = new Set<string>();
  const output: Entity[] = [];
  for (const entity of values) {
    const key = `${entity.name.toLowerCase()}:${entity.type.toLowerCase()}`;
    if (seen.has(key)) continue;
    seen.add(key);
    output.push(entity);
    if (output.length >= limit) break;
  }
  return output;
}

function sanitizeActionItems(values: ActionItem[]): ActionItem[] {
  return values
    .filter((value) => value && typeof value.task === "string")
    .map((value) => ({
      task: value.task.trim(),
      owner: (value.owner || "unassigned").trim() || "unassigned",
      deadline: (value.deadline || "unspecified").trim() || "unspecified",
    }))
    .filter((value) => Boolean(value.task));
}

function sanitizeQaPairs(values: QAPair[]): QAPair[] {
  return values
    .filter((value) => value && typeof value.question === "string" && typeof value.answer === "string")
    .map((value) => ({ question: value.question.trim(), answer: value.answer.trim() }))
    .filter((value) => Boolean(value.question) && Boolean(value.answer));
}

function sanitizeFlashcards(values: Flashcard[]): Flashcard[] {
  return values
    .filter((value) => value && typeof value.front === "string" && typeof value.back === "string")
    .map((value) => ({ front: value.front.trim(), back: value.back.trim() }))
    .filter((value) => Boolean(value.front) && Boolean(value.back));
}

function buildQaPairs(topics: string[], keyPoints: string[], limit: number): QAPair[] {
  const outputs: QAPair[] = [];
  for (let index = 0; index < Math.min(limit, topics.length); index += 1) {
    outputs.push({
      question: `What does the transcript say about ${topics[index]}?`,
      answer: keyPoints[index] || `The transcript treats ${topics[index]} as a core topic.`,
    });
  }
  return outputs;
}

function buildFlashcards(topics: string[], keyPoints: string[], limit: number): Flashcard[] {
  const outputs: Flashcard[] = [];
  for (let index = 0; index < Math.min(limit, Math.max(topics.length, keyPoints.length)); index += 1) {
    outputs.push({
      front: topics[index] ? `Explain ${topics[index]}` : `Key point ${index + 1}`,
      back: keyPoints[index] || `Review the transcript discussion for item ${index + 1}.`,
    });
  }
  return outputs;
}


path: src/agents/plannerAgent.ts
content: import type { ExecutionMode, OutputType, VideoToKnowledgeRequest } from "../schemas.js";
import { sha256 } from "../utils/hash.js";

export type ModeProfile = {
  mode: ExecutionMode;
  chunkSize: number;
  maxTopics: number;
  maxKeyPoints: number;
  qaPairs: number;
  flashcards: number;
  processingDepth: "shallow" | "standard" | "deep";
  modelLabels: {
    planning: string;
    transcription: string;
    semantic: string;
    knowledge: string;
    evaluation: string;
  };
};

export type ExecutionPlan = {
  requestHash: string;
  modeProfile: ModeProfile;
  requestedOutputs: OutputType[];
  transcriptionFingerprint: string;
  chunkConfigHash: string;
};

const MODE_PROFILES: Record<ExecutionMode, ModeProfile> = {
  fast: {
    mode: "fast",
    chunkSize: 1800,
    maxTopics: 4,
    maxKeyPoints: 5,
    qaPairs: 4,
    flashcards: 4,
    processingDepth: "shallow",
    modelLabels: {
      planning: "planner-fast:v1",
      transcription: "transcriber-fast:v1",
      semantic: "semantic-fast:v1",
      knowledge: "knowledge-fast:v1",
      evaluation: "evaluation-fast:v1",
    },
  },
  balanced: {
    mode: "balanced",
    chunkSize: 1200,
    maxTopics: 6,
    maxKeyPoints: 7,
    qaPairs: 6,
    flashcards: 6,
    processingDepth: "standard",
    modelLabels: {
      planning: "planner-balanced:v1",
      transcription: "transcriber-balanced:v1",
      semantic: "semantic-balanced:v1",
      knowledge: "knowledge-balanced:v1",
      evaluation: "evaluation-balanced:v1",
    },
  },
  high_accuracy: {
    mode: "high_accuracy",
    chunkSize: 800,
    maxTopics: 8,
    maxKeyPoints: 10,
    qaPairs: 8,
    flashcards: 10,
    processingDepth: "deep",
    modelLabels: {
      planning: "planner-high-accuracy:v1",
      transcription: "transcriber-high-accuracy:v1",
      semantic: "semantic-high-accuracy:v1",
      knowledge: "knowledge-high-accuracy:v1",
      evaluation: "evaluation-high-accuracy:v1",
    },
  },
};

export class PlannerAgent {
  plan(request: VideoToKnowledgeRequest): ExecutionPlan {
    const requestedOutputs = [...new Set(request.outputs)] as OutputType[];
    const modeProfile = MODE_PROFILES[request.mode];
    return {
      requestHash: sha256(JSON.stringify({ media_url: request.media_url, outputs: requestedOutputs, mode: request.mode })),
      modeProfile,
      requestedOutputs,
      transcriptionFingerprint: sha256(JSON.stringify({ media_url: request.media_url, mode: request.mode, transcription: modeProfile.modelLabels.transcription })),
      chunkConfigHash: sha256(JSON.stringify({ chunkSize: modeProfile.chunkSize, semantic: modeProfile.modelLabels.semantic, processingDepth: modeProfile.processingDepth })),
    };
  }

  getModeProfiles(): Record<ExecutionMode, ModeProfile> {
    return MODE_PROFILES;
  }
}


path: src/agents/semanticAgent.ts
content: import { chunkText, buildHashEmbedding, detectEntities, extractKeywords } from "../utils/text.js";
import { deterministicId } from "../utils/hash.js";
import type { PlatformRepository, SemanticChunkRecord, TranscriptRecord } from "../storage/types.js";
import type { ExecutionPlan } from "./plannerAgent.js";

export class SemanticAgent {
  constructor(private readonly repository: PlatformRepository) {}

  async segmentTranscript(transcript: TranscriptRecord, plan: ExecutionPlan) {
    const existing = await this.repository.getSemanticChunks(transcript.id, plan.chunkConfigHash);
    if (existing.length > 0) return { chunks: existing, cached: true };

    const now = new Date().toISOString();
    const chunks = chunkText(transcript.text, plan.modeProfile.chunkSize).map((text, index) => ({
      id: deterministicId("chunk", `${transcript.id}:${plan.chunkConfigHash}:${index}`),
      transcriptId: transcript.id,
      mediaAssetId: transcript.mediaAssetId,
      index,
      text,
      chunkConfigHash: plan.chunkConfigHash,
      embeddingModel: "hash-embedding:v1",
      embedding: [],
      keywords: [],
      entities: [],
      createdAt: now,
      updatedAt: now,
    } satisfies SemanticChunkRecord));

    await this.repository.replaceSemanticChunks(chunks);
    return { chunks, cached: false };
  }

  async enrichChunks(transcript: TranscriptRecord, plan: ExecutionPlan) {
    const existing = await this.repository.getSemanticChunks(transcript.id, plan.chunkConfigHash);
    if (existing.length === 0) {
      throw new Error("Semantic chunks are missing for transcript enrichment.");
    }
    if (existing.every((chunk) => chunk.embedding.length > 0)) {
      return { chunks: existing, cached: true };
    }

    const now = new Date().toISOString();
    const enriched = existing.map((chunk) => ({
      ...chunk,
      embedding: buildHashEmbedding(chunk.text),
      keywords: extractKeywords(chunk.text, 5),
      entities: detectEntities(chunk.text, 6),
      updatedAt: now,
    }));
    await this.repository.replaceSemanticChunks(enriched);
    return { chunks: enriched, cached: false };
  }
}


path: src/agents/transcriptionAgent.ts
content: import type { ExecutionPlan } from "./plannerAgent.js";
import { createId, sha256 } from "../utils/hash.js";
import type { PlatformRepository, TranscriptRecord } from "../storage/types.js";
import { UpstreamTranscriptionClient } from "../services/upstreamClient.js";

export class TranscriptionAgent {
  constructor(
    private readonly repository: PlatformRepository,
    private readonly upstreamClient: UpstreamTranscriptionClient,
  ) {}

  async execute(mediaAssetId: string, sourceUrl: string, plan: ExecutionPlan) {
    const cached = await this.repository.getTranscriptByFingerprint(mediaAssetId, plan.transcriptionFingerprint);
    if (cached) {
      return { transcript: cached, cached: true, modelUsed: [cached.model] };
    }

    const result = await this.upstreamClient.transcribe(sourceUrl, plan.modeProfile.mode);
    const now = new Date().toISOString();
    const transcript: TranscriptRecord = {
      id: createId("transcript"),
      mediaAssetId,
      sourceUrl: result.sourceUrl,
      transcriptionFingerprint: plan.transcriptionFingerprint,
      text: result.transcript,
      model: result.modelLabel,
      contentHash: sha256(result.transcript),
      createdAt: now,
      updatedAt: now,
      metadata: {
        route: result.route,
        kind: result.kind,
        execution_mode: plan.modeProfile.mode,
      },
    };

    await this.repository.upsertTranscript(transcript);
    return { transcript, cached: false, modelUsed: [transcript.model] };
  }
}


path: src/config.ts
content: export type ServerConfig = {
  host: string;
  port: number;
  baseUrl: string;
  publicBaseUrl?: string;
  sessionCookie: string;
  upstreamApiKey?: string;
  accessKeys: Set<string>;
  mongoUri?: string;
  mongoDbName: string;
  knowledgeModelApiBase: string;
  knowledgeModelApiKey?: string;
  knowledgeModelName: string;
  knowledgeEvaluationModelName: string;
};

const DEFAULT_KNOWLEDGE_MODEL_API_BASE = "https://dashscope.aliyuncs.com/compatible-mode/v1";
const DEFAULT_KNOWLEDGE_MODEL_NAME = "qwen-plus";

function isLocalHostname(hostname: string): boolean {
  const normalized = hostname.trim().toLowerCase();
  return normalized === "localhost" || normalized === "127.0.0.1" || normalized === "::1";
}

function parseAbsoluteUrl(value: string, envName: string): URL {
  let parsed: URL;
  try {
    parsed = new URL(value);
  } catch {
    throw new Error(`${envName} must be a valid absolute URL.`);
  }
  if (!["http:", "https:"].includes(parsed.protocol)) {
    throw new Error(`${envName} must use http or https.`);
  }
  return parsed;
}

function requireHttpsForRemoteUrl(parsed: URL, envName: string): void {
  if (parsed.protocol !== "https:" && !isLocalHostname(parsed.hostname)) {
    throw new Error(`${envName} must use https outside local development.`);
  }
}

export function getConfig(): ServerConfig {
  const configuredBaseUrl = process.env.VIDEOMP3WORD_BASE_URL?.trim() || "https://videomp3word.com";
  const sessionCookie = process.env.VIDEOMP3WORD_SESSION_COOKIE?.trim();
  if (!sessionCookie) {
    throw new Error("VIDEOMP3WORD_SESSION_COOKIE is required. Use a dedicated upstream videomp3word account for this deployment.");
  }
  if (!sessionCookie.includes("=")) {
    throw new Error("VIDEOMP3WORD_SESSION_COOKIE must be a valid cookie header value such as session=your-cookie.");
  }

  const parsedBaseUrl = parseAbsoluteUrl(configuredBaseUrl, "VIDEOMP3WORD_BASE_URL");
  requireHttpsForRemoteUrl(parsedBaseUrl, "VIDEOMP3WORD_BASE_URL");

  const allowedUpstreamHosts = new Set(
    String(process.env.VIDEOMP3WORD_ALLOWED_UPSTREAM_HOSTS || "videomp3word.com,www.videomp3word.com")
      .split(",")
      .map((value) => value.trim().toLowerCase())
      .filter(Boolean)
  );
  const upstreamHost = parsedBaseUrl.hostname.toLowerCase();
  if (!allowedUpstreamHosts.has(upstreamHost)) {
    throw new Error(`VIDEOMP3WORD_BASE_URL host "${upstreamHost}" is not allowed. Configure VIDEOMP3WORD_ALLOWED_UPSTREAM_HOSTS to permit it.`);
  }

  // Keep the MCP defaults aligned with the original videomp3word deployment.
  const knowledgeModelApiBase = parseAbsoluteUrl(
    process.env.KNOWLEDGE_MODEL_API_BASE?.trim() || DEFAULT_KNOWLEDGE_MODEL_API_BASE,
    "KNOWLEDGE_MODEL_API_BASE",
  );
  requireHttpsForRemoteUrl(knowledgeModelApiBase, "KNOWLEDGE_MODEL_API_BASE");

  const publicBaseUrl = process.env.PUBLIC_BASE_URL?.trim();
  if (publicBaseUrl) {
    parseAbsoluteUrl(publicBaseUrl, "PUBLIC_BASE_URL");
  }

  const portValue = Number(process.env.PORT || 3000);
  if (!Number.isInteger(portValue) || portValue < 0 || portValue > 65535) {
    throw new Error("PORT must be an integer between 0 and 65535.");
  }

  const accessKeys = new Set(
    String(process.env.MCP_ACCESS_KEYS || "")
      .split(",")
      .map((value) => value.trim())
      .filter(Boolean)
  );
  const nodeEnv = String(process.env.NODE_ENV || "").trim().toLowerCase();
  if (nodeEnv === "production" && accessKeys.size === 0) {
    throw new Error("MCP_ACCESS_KEYS must be configured in production.");
  }

  return {
    host: process.env.HOST || "0.0.0.0",
    port: portValue,
    baseUrl: parsedBaseUrl.toString().replace(/\/+$/, ""),
    publicBaseUrl: publicBaseUrl?.replace(/\/+$/, ""),
    sessionCookie,
    upstreamApiKey: process.env.VIDEOMP3WORD_API_KEY?.trim(),
    accessKeys,
    mongoUri: process.env.MONGO_URI?.trim(),
    mongoDbName: process.env.MONGO_DB_NAME?.trim() || "videomp3word_mcp",
    knowledgeModelApiBase: knowledgeModelApiBase.toString().replace(/\/+$/, ""),
    knowledgeModelApiKey: process.env.KNOWLEDGE_MODEL_API_KEY?.trim() || process.env.DASHSCOPE_API_KEY?.trim(),
    knowledgeModelName: process.env.KNOWLEDGE_MODEL_NAME?.trim() || DEFAULT_KNOWLEDGE_MODEL_NAME,
    knowledgeEvaluationModelName:
      process.env.KNOWLEDGE_EVALUATION_MODEL_NAME?.trim() ||
      process.env.KNOWLEDGE_MODEL_NAME?.trim() ||
      DEFAULT_KNOWLEDGE_MODEL_NAME,
  };
}

export const serverConfig = getConfig();


path: src/httpServer.ts
content: import express, { type Request, type Response as ExpressResponse } from "express";
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
import type { ServerConfig } from "./config.js";
import { createMcpServer } from "./mcpServer.js";
import { renderPlaygroundHtml } from "./ui/playground.js";
import type { VideoToKnowledgePlatform } from "./platform/videoToKnowledgePlatform.js";

function jsonRpcError(id: unknown, code: number, message: string) {
  return { jsonrpc: "2.0", error: { code, message }, id: id ?? null };
}

function isAuthorizedRequest(req: Request, config: ServerConfig): boolean {
  if (config.accessKeys.size === 0) return true;
  const authHeader = req.get("authorization") || "";
  const match = authHeader.match(/^Bearer\s+(.+)$/i);
  if (!match) return false;
  return config.accessKeys.has(match[1].trim());
}

function isRestrictedToolCall(req: Request): boolean {
  return req.body?.method === "tools/call" && req.body?.params?.name === "video_to_knowledge";
}

export function createHttpApp(config: ServerConfig, platform: VideoToKnowledgePlatform) {
  const app = express();
  app.set("trust proxy", true);
  app.use(express.json({ limit: "2mb" }));
  app.use((req, res, next) => {
    res.setHeader("Access-Control-Allow-Origin", "*");
    res.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
    res.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
    if (req.method === "OPTIONS") {
      res.status(204).end();
      return;
    }
    next();
  });

  app.get("/", (_req, res) => {
    res.setHeader("content-type", "text/html; charset=utf-8");
    res.send(renderPlaygroundHtml());
  });

  app.get("/health", (_req, res) => {
    res.json({
      ok: true,
      name: "videomp3word-mcp",
      auth_required: config.accessKeys.size > 0,
    });
  });

  app.get("/docs", (_req, res) => {
    res.json({
      endpoint: "POST /video_to_knowledge",
      transports: ["http", "mcp-streamable-http", "mcp-stdio"],
      request_example: {
        media_url: "https://example.com/demo.mp4",
        outputs: ["summary", "qa", "flashcards", "tasks", "topics"],
        mode: "balanced",
        export_formats: ["json", "markdown", "notion"],
      },
      modes: platform.getModeProfiles(),
    });
  });

  app.post("/video_to_knowledge", async (req, res) => {
    if (!isAuthorizedRequest(req, config)) {
      res.status(401).json({ error: "Unauthorized. Provide a valid bearer key." });
      return;
    }
    try {
      const result = await platform.run(req.body);
      res.json(result);
    } catch (error) {
      const message = error instanceof Error ? error.message : "Video to knowledge failed.";
      res.status(400).json({ error: message });
    }
  });

  app.get("/mcp", (_req, res) => {
    res.status(405).json(jsonRpcError(null, -32000, "Use HTTP POST for MCP requests."));
  });

  app.post("/mcp", async (req: Request, res: ExpressResponse) => {
    if (isRestrictedToolCall(req) && !isAuthorizedRequest(req, config)) {
      res.status(401).json(jsonRpcError(req.body?.id ?? null, -32001, "Unauthorized. Provide a valid bearer key."));
      return;
    }

    try {
      const server = createMcpServer(config, platform);
      const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined, enableJsonResponse: true });

      res.on("close", () => {
        transport.close().catch(() => undefined);
        server.close().catch(() => undefined);
      });

      await server.connect(transport);
      await transport.handleRequest(req, res, req.body);
    } catch (error) {
      const message = error instanceof Error ? error.message : "Internal server error.";
      if (!res.headersSent) {
        res.status(500).json(jsonRpcError(req.body?.id ?? null, -32603, message));
      }
    }
  });

  return app;
}


path: src/index.ts
content: #!/usr/bin/env node
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { createHttpApp } from "./httpServer.js";
import { createMcpServer } from "./mcpServer.js";
import { serverConfig } from "./config.js";
import { createPlatform } from "./platform/serviceContainer.js";
import { logger } from "./logger.js";

const isStdio = process.argv.includes("stdio");

async function main() {
  const platform = await createPlatform(serverConfig);
  if (isStdio) {
    const server = createMcpServer(serverConfig, platform);
    const transport = new StdioServerTransport();
    await server.connect(transport);
    logger.info("videomp3word-mcp listening on stdio", { baseUrl: serverConfig.baseUrl });
    return;
  }

  const app = createHttpApp(serverConfig, platform);
  app.listen(serverConfig.port, serverConfig.host, () => {
    logger.info("videomp3word-mcp listening on http", {
      host: serverConfig.host,
      port: serverConfig.port,
      baseUrl: serverConfig.baseUrl,
    });
  });
}

main().catch((error) => {
  logger.error("Fatal error starting server", {
    message: error instanceof Error ? error.message : String(error),
  });
  process.exit(1);
});


path: src/logger.ts
content: type LogLevel = "info" | "warn" | "error";

function emit(level: LogLevel, message: string, meta?: Record<string, unknown>) {
  const payload = {
    level,
    message,
    timestamp: new Date().toISOString(),
    ...(meta ? { meta } : {}),
  };
  console.error(JSON.stringify(payload));
}

export const logger = {
  info: (message: string, meta?: Record<string, unknown>) => emit("info", message, meta),
  warn: (message: string, meta?: Record<string, unknown>) => emit("warn", message, meta),
  error: (message: string, meta?: Record<string, unknown>) => emit("error", message, meta),
};


path: src/mcpServer.ts
content: import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import type { ServerConfig } from "./config.js";
import { KnowledgeResponseSchema, VideoToKnowledgeRequestSchema } from "./schemas.js";
import type { VideoToKnowledgePlatform } from "./platform/videoToKnowledgePlatform.js";

export function createMcpServer(config: ServerConfig, platform: VideoToKnowledgePlatform) {
  const server = new McpServer({ name: "videomp3word-mcp", version: "2.0.2" });

  server.registerResource(
    "videomp3word-knowledge-schema",
    "videomp3word://knowledge-schema",
    {
      title: "Knowledge Extraction Schema",
      description: "Schema, modes, and operational notes for the single `video_to_knowledge` entrypoint.",
      mimeType: "application/json",
    },
    async (uri) => ({
      contents: [
        {
          uri: uri.href,
          mimeType: "application/json",
          text: JSON.stringify(
            {
              endpoint: "/video_to_knowledge",
              modes: platform.getModeProfiles(),
              request_fields: Object.keys(VideoToKnowledgeRequestSchema.shape),
              response_fields: Object.keys(KnowledgeResponseSchema.shape),
              deployment: { base_url: config.publicBaseUrl || null },
            },
            null,
            2,
          ),
        },
      ],
    }),
  );

  server.registerTool(
    "video_to_knowledge",
    {
      title: "Video To Knowledge",
      description: "Convert an audio or video URL into structured knowledge artifacts through one production workflow.",
      inputSchema: VideoToKnowledgeRequestSchema,
      outputSchema: KnowledgeResponseSchema,
      annotations: {
        idempotentHint: true,
        readOnlyHint: false,
        destructiveHint: false,
        openWorldHint: true,
      },
      _meta: {
        transport: ["stdio", "streamable-http"],
        publishable_on: "clawhub",
      },
    },
    async (args) => {
      const result = await platform.run(args);
      return {
        structuredContent: result,
        content: [{ type: "text", text: JSON.stringify(result, null, 2) }],
      };
    },
  );

  return server;
}


path: src/platform/serviceContainer.ts
content: import type { ServerConfig } from "../config.js";
import { createRepository } from "../storage/index.js";
import { PlannerAgent } from "../agents/plannerAgent.js";
import { TranscriptionAgent } from "../agents/transcriptionAgent.js";
import { SemanticAgent } from "../agents/semanticAgent.js";
import { KnowledgeAgent } from "../agents/knowledgeAgent.js";
import { EvaluationAgent } from "../agents/evaluationAgent.js";
import { UpstreamTranscriptionClient } from "../services/upstreamClient.js";
import { ModelClient } from "../services/modelClient.js";
import { ExportService } from "../services/exportService.js";
import { VideoToKnowledgePlatform } from "./videoToKnowledgePlatform.js";

export async function createPlatform(config: ServerConfig) {
  const repository = createRepository(config);
  const plannerAgent = new PlannerAgent();
  const transcriptionAgent = new TranscriptionAgent(repository, new UpstreamTranscriptionClient(config));
  const semanticAgent = new SemanticAgent(repository);
  const knowledgeAgent = new KnowledgeAgent(repository, new ModelClient(config));
  const evaluationAgent = new EvaluationAgent();
  const exportService = new ExportService();
  const platform = new VideoToKnowledgePlatform(
    config,
    repository,
    plannerAgent,
    transcriptionAgent,
    semanticAgent,
    knowledgeAgent,
    evaluationAgent,
    exportService,
  );
  await platform.initialize();
  return platform;
}


path: src/schemas.ts
content: import { z } from "zod";

export const OutputTypeSchema = z.enum(["summary", "qa", "flashcards", "tasks", "topics"]);
export const ExecutionModeSchema = z.enum(["fast", "balanced", "high_accuracy"]);
export const ExportFormatSchema = z.enum(["json", "markdown", "notion"]);

export const VideoToKnowledgeRequestSchema = z.object({
  media_url: z.string().url(),
  outputs: z.array(OutputTypeSchema).min(1),
  mode: ExecutionModeSchema.default("balanced"),
  export_formats: z.array(ExportFormatSchema).default(["json"]),
  metadata: z.record(z.unknown()).optional(),
});

export const ActionItemSchema = z.object({
  task: z.string(),
  owner: z.string(),
  deadline: z.string(),
});

export const QAPairSchema = z.object({
  question: z.string(),
  answer: z.string(),
});

export const FlashcardSchema = z.object({
  front: z.string(),
  back: z.string(),
});

export const EntitySchema = z.object({
  name: z.string(),
  type: z.string(),
});

export const WorkflowStepTraceSchema = z.object({
  step: z.string(),
  status: z.enum(["completed", "failed"]),
  attempt: z.number().int().min(1),
  duration_ms: z.number().nonnegative(),
  cached: z.boolean(),
  note: z.string().optional(),
});

export const TraceSchema = z.object({
  workflow_run_id: z.string(),
  steps: z.array(WorkflowStepTraceSchema),
  models_used: z.array(z.string()),
  chunk_references: z.array(z.string()),
  resource_ids: z.object({
    media_asset_id: z.string(),
    transcript_id: z.string(),
    knowledge_unit_id: z.string(),
  }),
});

export const KnowledgeResponseSchema = z.object({
  request_id: z.string(),
  mode: ExecutionModeSchema,
  requested_outputs: z.array(OutputTypeSchema),
  summary: z.string(),
  topics: z.array(z.string()),
  key_points: z.array(z.string()),
  action_items: z.array(ActionItemSchema),
  qa_pairs: z.array(QAPairSchema),
  flashcards: z.array(FlashcardSchema),
  entities: z.array(EntitySchema),
  confidence_scores: z.record(z.number()),
  trace: TraceSchema,
  exports: z.record(z.unknown()),
});

export type OutputType = z.infer<typeof OutputTypeSchema>;
export type ExecutionMode = z.infer<typeof ExecutionModeSchema>;
export type ExportFormat = z.infer<typeof ExportFormatSchema>;
export type VideoToKnowledgeRequest = z.infer<typeof VideoToKnowledgeRequestSchema>;
export type ActionItem = z.infer<typeof ActionItemSchema>;
export type QAPair = z.infer<typeof QAPairSchema>;
export type Flashcard = z.infer<typeof FlashcardSchema>;
export type Entity = z.infer<typeof EntitySchema>;
export type WorkflowStepTrace = z.infer<typeof WorkflowStepTraceSchema>;
export type KnowledgeResponse = z.infer<typeof KnowledgeResponseSchema>;


path: src/services/exportService.ts
content: import type { KnowledgeResponse } from "../schemas.js";

export class ExportService {
  buildExports(result: Omit<KnowledgeResponse, "exports">, formats: string[]): Record<string, unknown> {
    const exports: Record<string, unknown> = {};
    if (formats.includes("json")) exports.json = JSON.stringify(result, null, 2);
    if (formats.includes("markdown")) exports.markdown = this.toMarkdown(result);
    if (formats.includes("notion")) exports.notion = this.toNotion(result);
    return exports;
  }

  private toMarkdown(result: Omit<KnowledgeResponse, "exports">): string {
    const lines = [
      "# Video To Knowledge",
      "",
      "## Summary",
      result.summary || "No summary generated.",
      "",
      "## Topics",
      ...(result.topics.length ? result.topics.map((topic) => `- ${topic}`) : ["- None"]),
      "",
      "## Key Points",
      ...(result.key_points.length ? result.key_points.map((point) => `- ${point}`) : ["- None"]),
      "",
      "## Action Items",
      ...(result.action_items.length ? result.action_items.map((item) => `- ${item.task} | owner: ${item.owner} | deadline: ${item.deadline}`) : ["- None"]),
      "",
      "## Q&A",
      ...(result.qa_pairs.length ? result.qa_pairs.map((pair) => `- Q: ${pair.question}\n  A: ${pair.answer}`) : ["- None"]),
      "",
      "## Flashcards",
      ...(result.flashcards.length ? result.flashcards.map((card) => `- Front: ${card.front}\n  Back: ${card.back}`) : ["- None"]),
      "",
      "## Entities",
      ...(result.entities.length ? result.entities.map((entity) => `- ${entity.name} (${entity.type})`) : ["- None"]),
    ];
    return lines.join("\n");
  }

  private toNotion(result: Omit<KnowledgeResponse, "exports">): Record<string, unknown> {
    const blocks = [
      { type: "heading_1", text: "Video To Knowledge" },
      { type: "paragraph", text: result.summary || "No summary generated." },
      { type: "heading_2", text: "Topics" },
      ...result.topics.map((topic) => ({ type: "bulleted_list_item", text: topic })),
      { type: "heading_2", text: "Key Points" },
      ...result.key_points.map((point) => ({ type: "bulleted_list_item", text: point })),
      { type: "heading_2", text: "Action Items" },
      ...result.action_items.map((item) => ({ type: "to_do", text: `${item.task} | ${item.owner} | ${item.deadline}` })),
    ];
    return { title: "Video To Knowledge", blocks };
  }
}


path: src/storage/index.ts
content: import type { ServerConfig } from "../config.js";
import { MemoryRepository } from "./memoryRepository.js";
import { MongoRepository } from "./mongoRepository.js";
import type { PlatformRepository } from "./types.js";

export function createRepository(config: ServerConfig): PlatformRepository {
  if (config.mongoUri) {
    return new MongoRepository(config.mongoUri, config.mongoDbName);
  }
  return new MemoryRepository();
}


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)