Skill: zhcolin0313/literature-recommendation (version 1.0.3)

--- name: literature-recommendation description: "用于论文推荐、文献推荐、论文匹配、按偏好找论文、根据研究方向推荐论文；从数据库读取成员偏好，召回候选论文并输出给OpenClaw。" license: MIT --- # Literature Recommendation ## What it does This skill implements a minimal literature recommendation workflow for a team: - store member research profiles - fetch candidate papers from arXiv - score papers with lightweight rules - persist run history and recommendation output in PostgreSQL - output personal card payload for OpenClaw delivery ## 触发词 优先使用中文触发，建议包括： - 论文推荐 - 文献推荐 - 论文匹配 - 按偏好找论文 - 根据研究方向推荐论文 - 给我推荐相关论文 - 帮我推荐几篇论文 ## Usage Run the CLI from the project root: ```bash python entrypoints/run_paper_matching.py --dry-run ``` ## Workflow 1. Load member profiles from PostgreSQL. 2. Fetch candidate papers from arXiv. 3. Score papers with direction and keyword matching. 4. Store papers and recommendations in PostgreSQL. 5. Build delivery payloads: personal card JSON only. 6. Return payload to OpenClaw for actual Feishu sending. ## 职责边界（规范） - 本 Skill 负责： - 从数据库读取成员偏好 - 从 arXiv 召回候选论文 - 用硬规则打分并给出规则理由（rule_reason） - 输出 openclaw_rerank_payload 与 personal_cards - OpenClaw 负责： - 读取 openclaw_rerank_payload 并调用 LLM 做语义复排 - 判定是否推荐（keep/drop）与最终排序 - 生成最终推荐理由（可覆盖 rule_reason） - 推荐理由需解释论文摘要核心内容，并指出与成员偏好的匹配点 - 执行私聊发送和定时调度 ## 输出契约（推荐结果） - 候选输入给 OpenClaw：openclaw_rerank_payload - strategy: hard_rule_recall - llm_reason_contract: - language: zh-CN - reason_style: explanatory - required_source: - paper.title - paper.abstract - profile.primary_direction - profile.keywords - forbidden: - title_only_reason - reason_generation_steps: - 先阅读 paper.abstract 提取论文核心方法/任务 - 再对照成员偏好字段判断匹配点 - 最后生成一句概括 + 一句匹配说明 - must_include: - 论文核心内容一句话概括 - 与该成员偏好匹配的具体点 - max_length: 120 - by_member[record_id].candidates[*]: - paper_id - rule_score - rule_reason - paper (title/abstract/tags/url 等) - llm_output_schema: - by_member[record_id].items[*]: - paper_id - keep - rank - score - reason - 最终发送：delivery_payload.personal_cards - record_id - paper_id - rank - score - reason - paper_title - paper_abstract - card ## Notes - The MVP uses rule-based matching only. - The weekly push is part of the MVP, not a later iteration. - The payload now targets personal direct messages only. - Feishu sending, Bitable sync, and feedback storage are handled by the data-hub skill. - This skill is invoked manually from chat or by OpenClaw orchestration; it does not schedule itself. - If you need timed execution, let OpenClaw or an external scheduler trigger this skill on a schedule.

path: requirements.txt
content: requests>=2.31.0
python-dotenv>=1.0.0
psycopg[binary]>=3.2.0


path: env-example.txt
content: # Required (core runtime)
# PostgreSQL DSN, example: postgresql://user:password@host:5432/database
OPENCLAW_DB_DSN=
OPENCLAW_OUTPUT_DIR=./outputs

# Optional: hard-rule recall pool size before OpenClaw reranking
OPENCLAW_RECALL_CANDIDATE_POOL=20


path: skill.yaml
content: name: literature-recommendation
version: 1.0.3
description: 用于论文推荐、文献推荐和论文匹配；从数据库读取成员偏好，召回候选论文并输出给OpenClaw。
license: MIT
author:
  - OpenClaw
required_environment_variables:
  - OPENCLAW_DB_DSN
  - OPENCLAW_OUTPUT_DIR
environment_file: ~/.config/literature-recommendation/.env


path: paper_matching/config.py
content: """Configuration helpers for the recommender MVP."""

from __future__ import annotations

import os
from dataclasses import dataclass
from pathlib import Path

try:
    from dotenv import load_dotenv
except ImportError:  # pragma: no cover - optional at import time
    load_dotenv = None


def _expand_path(value: str) -> Path:
    return Path(os.path.expanduser(value)).resolve()


def _load_environment() -> None:
    root = Path(__file__).resolve().parents[1]
    env_path = os.getenv("OPENCLAW_ENV_FILE", os.path.expanduser("~/.config/literature-recommendation/.env"))
    if load_dotenv is None:
        return
    load_dotenv(env_path, override=False)
    load_dotenv(root / ".env", override=False)


@dataclass(frozen=True)
class Settings:
    db_dsn: str
    output_dir: Path
    recall_candidate_pool: int


def load_settings(
    db_dsn: str | None = None,
    output_dir: str | None = None,
) -> Settings:
    _load_environment()
    root = Path(__file__).resolve().parents[1]
    resolved_dsn = (db_dsn or os.getenv("OPENCLAW_DB_DSN", "")).strip()
    if not resolved_dsn:
        raise RuntimeError("OPENCLAW_DB_DSN is required for PostgreSQL backend.")
    return Settings(
        db_dsn=resolved_dsn,
        output_dir=_expand_path(output_dir or os.getenv("OPENCLAW_OUTPUT_DIR", str(root / "outputs"))),
        recall_candidate_pool=max(5, int(os.getenv("OPENCLAW_RECALL_CANDIDATE_POOL", "20"))),
    )


path: paper_matching/fetchers.py
content: """Paper source fetchers."""

from __future__ import annotations

import html
import re
import xml.etree.ElementTree as ET
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from typing import Iterable
from urllib.parse import quote_plus

import requests

from .models import Paper


ARXIV_API = "https://export.arxiv.org/api/query"


def _clean_text(value: str) -> str:
    value = re.sub(r"\s+", " ", value or "")
    return html.unescape(value).strip()


def _parse_arxiv_time(value: str) -> str:
    try:
        dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
    except ValueError:
        dt = datetime.now(timezone.utc)
    return dt.astimezone(timezone.utc).isoformat()


@dataclass
class ArxivFetcher:
    timeout: int = 20

    def search(self, query: str, max_results: int = 20) -> list[Paper]:
        params = {
            "search_query": query,
            "start": 0,
            "max_results": max_results,
            "sortBy": "submittedDate",
            "sortOrder": "descending",
        }
        response = requests.get(ARXIV_API, params=params, timeout=self.timeout)
        response.raise_for_status()
        return self._parse_feed(response.text)

    def _parse_feed(self, xml_text: str) -> list[Paper]:
        root = ET.fromstring(xml_text)
        ns = {
            "atom": "http://www.w3.org/2005/Atom",
            "arxiv": "http://arxiv.org/schemas/atom",
        }
        papers: list[Paper] = []
        for entry in root.findall("atom:entry", ns):
            paper_id = (entry.findtext("atom:id", default="", namespaces=ns) or "").split("/")[-1]
            title = _clean_text(entry.findtext("atom:title", default="", namespaces=ns))
            summary = _clean_text(entry.findtext("atom:summary", default="", namespaces=ns))
            published = _parse_arxiv_time(entry.findtext("atom:published", default="", namespaces=ns))
            links = entry.findall("atom:link", ns)
            url = ""
            for link in links:
                if link.attrib.get("rel") == "alternate":
                    url = link.attrib.get("href", "")
                    break
            if not url and paper_id:
                url = f"https://arxiv.org/abs/{paper_id}"
            authors = [
                _clean_text(author.findtext("atom:name", default="", namespaces=ns))
                for author in entry.findall("atom:author", ns)
            ]
            tags = [
                tag.attrib.get("term", "")
                for tag in entry.findall("atom:category", ns)
                if tag.attrib.get("term")
            ]
            papers.append(
                Paper(
                    paper_id=paper_id or url,
                    title=title,
                    abstract=summary,
                    authors=[author for author in authors if author],
                    published_at=published,
                    source="arxiv",
                    url=url,
                    tags=tags,
                    raw={"entry_id": paper_id, "source": "arxiv"},
                )
            )
        return papers

    def fetch_for_keywords(self, keywords: Iterable[str], max_results: int = 20) -> list[Paper]:
        terms = [term.strip() for term in keywords if term and term.strip()]
        if not terms:
            return []
        query = " OR ".join(f'all:"{term}"' for term in terms[:6])
        return self.search(query, max_results=max_results)


path: paper_matching/models.py
content: """Domain models used by the recommender."""

from __future__ import annotations

from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from typing import Any


def _json_list(value: Any) -> list[str]:
    if value is None:
        return []
    if isinstance(value, list):
        return [str(item).strip() for item in value if str(item).strip()]
    if isinstance(value, str):
        return [part.strip() for part in value.split(",") if part.strip()]
    return [str(value).strip()]


@dataclass
class MemberProfile:
    record_id: str
    display_name: str
    team_name: str = ""
    primary_direction: str = ""
    secondary_directions: list[str] = field(default_factory=list)
    keywords: list[str] = field(default_factory=list)
    excluded_topics: list[str] = field(default_factory=list)
    feishu_user_id: str = ""
    source: str = "manual"
    confidence: float = 1.0
    version: int = 1
    updated_at: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())

    @classmethod
    def from_dict(cls, data: dict[str, Any]) -> "MemberProfile":
        return cls(
            record_id=str(data.get("record_id") or data.get("member_id") or data.get("id") or data.get("name") or "").strip(),
            display_name=str(data.get("display_name") or data.get("name") or data.get("record_id") or data.get("member_id") or "").strip(),
            team_name=str(data.get("team_name") or "").strip(),
            primary_direction=str(data.get("primary_direction") or "").strip(),
            secondary_directions=_json_list(data.get("secondary_directions")),
            keywords=_json_list(data.get("keywords")),
            excluded_topics=_json_list(data.get("excluded_topics")),
            feishu_user_id=str(data.get("feishu_user_id") or "").strip(),
            source=str(data.get("source") or "manual").strip(),
            confidence=float(data.get("confidence") or 1.0),
            version=int(data.get("version") or 1),
            updated_at=str(data.get("updated_at") or datetime.now(timezone.utc).isoformat()),
        )

    def to_dict(self) -> dict[str, Any]:
        return asdict(self)


@dataclass
class Paper:
    paper_id: str
    title: str
    abstract: str
    authors: list[str]
    published_at: str
    source: str
    url: str
    tags: list[str] = field(default_factory=list)
    raw: dict[str, Any] = field(default_factory=dict)

    @classmethod
    def from_dict(cls, data: dict[str, Any]) -> "Paper":
        return cls(
            paper_id=str(data["paper_id"]),
            title=str(data.get("title") or "").strip(),
            abstract=str(data.get("abstract") or "").strip(),
            authors=_json_list(data.get("authors")),
            published_at=str(data.get("published_at") or datetime.now(timezone.utc).isoformat()),
            source=str(data.get("source") or "unknown").strip(),
            url=str(data.get("url") or "").strip(),
            tags=_json_list(data.get("tags")),
            raw=dict(data.get("raw") or {}),
        )

    def to_dict(self) -> dict[str, Any]:
        return asdict(self)


@dataclass
class Recommendation:
    record_id: str
    paper_id: str
    score: float
    rank: int
    reason: str
    channel: str = "personal_dm"
    created_at: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())

    def to_dict(self) -> dict[str, Any]:
        return asdict(self)


path: paper_matching/__init__.py
content: """OpenClaw literature recommender MVP."""


path: paper_matching/matching.py
content: """Rule-based matching and ranking."""

from __future__ import annotations

from datetime import datetime, timezone

from .models import MemberProfile, Paper, Recommendation


def _lower_tokens(value: str) -> list[str]:
    value = (value or "").lower().replace("/", " ").replace("-", " ")
    return [token for token in value.split() if token]


def _contains_any(text: str, keywords: list[str]) -> list[str]:
    haystack = (text or "").lower()
    matched = []
    for keyword in keywords:
        needle = keyword.lower().strip()
        if needle and needle in haystack:
            matched.append(keyword)
    return matched


def score_paper(profile: MemberProfile, paper: Paper) -> tuple[float, str, list[str]]:
    title = paper.title.lower()
    abstract = paper.abstract.lower()
    tags = [tag.lower() for tag in paper.tags]
    keywords = profile.keywords or []
    directions = [profile.primary_direction, *profile.secondary_directions]
    excluded = profile.excluded_topics or []

    excluded_hits = _contains_any(title + " " + abstract, excluded)
    if excluded_hits:
        return 0.0, f"filtered by excluded topics: {', '.join(excluded_hits[:3])}", excluded_hits

    matched_keywords = _contains_any(title, keywords) + _contains_any(abstract, keywords)
    matched_directions = _contains_any(title, directions) + _contains_any(abstract, directions)
    matched_tags = [tag for tag in tags if any(keyword.lower() in tag for keyword in keywords)]

    score = 0.0
    reasons: list[str] = []

    if matched_directions:
        score += 6.0 + min(3.0, len(set(matched_directions)) * 1.5)
        reasons.append(f"matches direction: {', '.join(sorted(set(matched_directions))[:3])}")
    if matched_keywords:
        score += 4.0 + min(2.0, len(set(matched_keywords)) * 0.8)
        reasons.append(f"matches keywords: {', '.join(sorted(set(matched_keywords))[:4])}")
    if matched_tags:
        score += 1.5 + min(1.5, len(set(matched_tags)) * 0.5)
        reasons.append(f"matches tags: {', '.join(sorted(set(matched_tags))[:4])}")

    if not reasons:
        title_tokens = _lower_tokens(profile.primary_direction)
        overlap = [token for token in title_tokens if token in title or token in abstract]
        if overlap:
            score += 2.5
            reasons.append(f"direction token overlap: {', '.join(overlap[:3])}")

    try:
        published = datetime.fromisoformat(paper.published_at.replace("Z", "+00:00")).astimezone(timezone.utc)
        age_days = max(0.0, (datetime.now(timezone.utc) - published).total_seconds() / 86400.0)
    except Exception:
        age_days = 14.0

    novelty_bonus = max(0.0, 3.5 - age_days * 0.25)
    score += novelty_bonus
    if novelty_bonus > 0:
        reasons.append(f"novelty bonus: {novelty_bonus:.1f}")

    if profile.confidence:
        score *= max(0.7, min(1.3, profile.confidence))

    if not reasons:
        reasons.append("light lexical similarity")

    return round(score, 3), "; ".join(reasons), []


def rank_for_member(profile: MemberProfile, papers: list[Paper], top_k: int = 5) -> list[Recommendation]:
    scored = []
    seen = set()
    for paper in papers:
        if paper.paper_id in seen:
            continue
        seen.add(paper.paper_id)
        score, reason, _ = score_paper(profile, paper)
        if score <= 0:
            continue
        scored.append((score, paper, reason))

    scored.sort(key=lambda item: (-item[0], item[1].published_at, item[1].paper_id))
    recommendations: list[Recommendation] = []
    for index, (score, paper, reason) in enumerate(scored[:top_k], start=1):
        recommendations.append(
            Recommendation(
                record_id=profile.record_id,
                paper_id=paper.paper_id,
                score=score,
                rank=index,
                reason=reason,
                channel="personal_dm",
            )
        )
    return recommendations


path: paper_matching/pipeline.py
content: """End-to-end recommendation pipeline."""

from __future__ import annotations

import json
from datetime import datetime, timezone
from typing import Any
from uuid import uuid4

from .config import Settings
from .fetchers import ArxivFetcher
from .matching import rank_for_member
from .models import MemberProfile, Paper
from .storage import Storage


def _profile_queries(profile: MemberProfile) -> list[str]:
    terms: list[str] = []
    if profile.primary_direction:
        terms.append(profile.primary_direction)
    terms.extend(profile.secondary_directions)
    terms.extend(profile.keywords)
    cleaned = []
    seen = set()
    for term in terms:
        value = term.strip()
        if value and value.lower() not in seen:
            cleaned.append(value)
            seen.add(value.lower())
    return cleaned[:8]


def _resolve_profiles(storage: Storage, settings: Settings, profile_source: str) -> list[MemberProfile]:
    profiles = storage.load_profiles()
    if not profiles:
        raise RuntimeError("No profiles found in database. Please sync profiles first.")
    return profiles


def run_pipeline(
    settings: Settings,
    days: int = 7,
    top_k: int = 5,
    max_results_per_member: int = 20,
    profile_source: str = "auto",
    dry_run: bool = False,
) -> dict[str, Any]:
    settings.output_dir.mkdir(parents=True, exist_ok=True)
    storage = Storage(settings.db_dsn)
    profiles = _resolve_profiles(storage, settings, profile_source)

    fetcher = ArxivFetcher()
    run_id = f"run_{datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%S')}_{uuid4().hex[:8]}"
    started_at = datetime.now(timezone.utc).isoformat()
    storage.save_run_status(run_id, "running", started_at)

    all_papers: dict[str, Paper] = {}
    fetch_log: list[dict[str, Any]] = []
    for profile in profiles:
        queries = _profile_queries(profile)
        if not queries:
            continue
        papers = fetcher.fetch_for_keywords(queries, max_results=max_results_per_member)
        fetch_log.append({"record_id": profile.record_id, "queries": queries, "count": len(papers)})
        for paper in papers:
            all_papers[paper.paper_id] = paper

    storage.save_papers(list(all_papers.values()))

    recommendations_by_member: dict[str, list] = {}
    rerank_payload_by_member: dict[str, dict[str, Any]] = {}
    recommendations_to_store = []
    for profile in profiles:
        candidate_top_k = max(top_k, settings.recall_candidate_pool)
        candidate_recommendations = rank_for_member(profile, list(all_papers.values()), top_k=candidate_top_k)
        member_recommendations = candidate_recommendations[:top_k]

        recommendations_by_member[profile.record_id] = member_recommendations
        recommendations_to_store.extend(member_recommendations)
        rerank_payload_by_member[profile.record_id] = {
            "profile": profile.to_dict(),
            "candidates": [
                {
                    "paper_id": rec.paper_id,
                    "rule_score": rec.score,
                    "rule_reason": rec.reason,
                    "paper": all_papers[rec.paper_id].to_dict() if rec.paper_id in all_papers else {},
                }
                for rec in candidate_recommendations
            ],
        }

    storage.save_recommendations(run_id, recommendations_to_store)

    member_payloads = {}
    for profile in profiles:
        member_payloads[profile.record_id] = {
            "profile": profile.to_dict(),
            "recommendations": [rec.to_dict() for rec in recommendations_by_member.get(profile.record_id, [])],
        }

    report = {
        "run_id": run_id,
        "started_at": started_at,
        "finished_at": datetime.now(timezone.utc).isoformat(),
        "days": days,
        "profile_source": profile_source,
        "profiles": [profile.to_dict() for profile in profiles],
        "fetched_papers": [paper.to_dict() for paper in all_papers.values()],
        "fetch_log": fetch_log,
        "openclaw_rerank_payload": {
            "strategy": "hard_rule_recall",
            "note": "OpenClaw should send candidates to LLM and return reranked top_k.",
            "candidate_pool": settings.recall_candidate_pool,
            "llm_reason_contract": {
                "language": "zh-CN",
                "reason_style": "explanatory",
                "required_source": ["paper.title", "paper.abstract", "profile.primary_direction", "profile.keywords"],
                "forbidden": ["title_only_reason"],
                "reason_generation_steps": [
                    "先阅读 paper.abstract 提取论文核心方法/任务",
                    "再对照成员偏好字段判断匹配点",
                    "最后生成一句概括 + 一句匹配说明",
                ],
                "must_include": [
                    "论文核心内容一句话概括",
                    "与该成员偏好匹配的具体点",
                ],
                "max_length": 120,
                "example": "该文聚焦多模态医学影像分割，并提出轻量级跨模态对齐模块；与你关注的医学AI与多模态学习方向高度匹配。",
            },
            "llm_output_schema": {
                "by_member": {
                    "record_id": {
                        "items": [
                            {
                                "paper_id": "string",
                                "keep": "boolean",
                                "rank": "integer",
                                "score": "number",
                                "reason": "string",
                            }
                        ]
                    }
                }
            },
            "by_member": rerank_payload_by_member,
        },
        "member_reports": member_payloads,
        "delivery_payload": _build_delivery_payload(report_context={
            "run_id": run_id,
            "member_reports": member_payloads,
            "fetched_papers": [paper.to_dict() for paper in all_papers.values()],
        }, settings=settings),
    }

    storage.save_run_status(run_id, "finished", started_at, report["finished_at"], notes=f"papers={len(all_papers)}, members={len(profiles)}")

    report_path = settings.output_dir / f"{run_id}.json"
    report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
    return report


def _build_delivery_payload(report_context: dict[str, Any], settings: Settings) -> dict[str, Any]:
    run_id = report_context["run_id"]
    paper_by_id = {item["paper_id"]: item for item in report_context["fetched_papers"]}

    personal_cards = []
    for record_id, payload in report_context["member_reports"].items():
        profile = payload["profile"]
        for rec in payload["recommendations"]:
            paper = paper_by_id.get(rec["paper_id"], {})
            title = paper.get("title") or rec["paper_id"]
            abstract = paper.get("abstract") or ""
            raw_url = paper.get("url") or ""

            card = {
                "config": {"wide_screen_mode": True},
                "header": {
                    "template": "blue",
                    "title": {
                        "tag": "plain_text",
                        "content": f"OpenClaw 推荐 · {profile.get('display_name')} · {rec['rank']}",
                    },
                },
                "elements": [
                    {
                        "tag": "div",
                        "text": {
                            "tag": "lark_md",
                            "content": (
                                f"**{title}**\\n"
                                f"score={rec['score']} | {rec['reason']}\\n"
                                f"[打开论文]({raw_url})"
                            ),
                        },
                    },
                ],
            }

            personal_cards.append(
                {
                    "record_id": record_id,
                    "display_name": profile.get("display_name"),
                    "feishu_user_id": profile.get("feishu_user_id", ""),
                    "paper_id": rec["paper_id"],
                    "rank": rec["rank"],
                    "score": rec["score"],
                    "reason": rec["reason"],
                    "paper_title": title,
                    "paper_abstract": abstract,
                    "paper_url": raw_url,
                    "card": card,
                }
            )

    return {
        "run_id": run_id,
        "personal_cards": personal_cards,
    }


path: paper_matching/storage.py
content: """PostgreSQL storage layer for profiles, papers, and recommendations."""

from __future__ import annotations

import json
from contextlib import contextmanager
from datetime import datetime, timezone
from typing import Any, Iterator

import psycopg
from psycopg.rows import dict_row

from .models import MemberProfile, Paper, Recommendation


SCHEMA_POSTGRES = """
CREATE TABLE IF NOT EXISTS member_profiles (
    record_id TEXT PRIMARY KEY,
    display_name TEXT NOT NULL,
    team_name TEXT,
    primary_direction TEXT,
    secondary_directions JSONB DEFAULT '[]'::jsonb,
    keywords JSONB DEFAULT '[]'::jsonb,
    excluded_topics JSONB DEFAULT '[]'::jsonb,
    feishu_user_id TEXT,
    source TEXT DEFAULT 'manual',
    confidence REAL DEFAULT 1.0,
    version INTEGER DEFAULT 1,
    updated_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE IF NOT EXISTS papers (
    paper_id TEXT PRIMARY KEY,
    title TEXT NOT NULL,
    abstract TEXT NOT NULL,
    authors JSONB NOT NULL DEFAULT '[]'::jsonb,
    published_at TIMESTAMPTZ,
    source TEXT NOT NULL,
    url TEXT NOT NULL,
    tags JSONB NOT NULL DEFAULT '[]'::jsonb,
    raw_json JSONB NOT NULL DEFAULT '{}'::jsonb,
    fetched_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE IF NOT EXISTS recommendations (
    run_id TEXT NOT NULL,
    record_id TEXT NOT NULL,
    paper_id TEXT NOT NULL,
    score DOUBLE PRECISION NOT NULL,
    rank INTEGER NOT NULL,
    reason TEXT NOT NULL,
    channel TEXT NOT NULL,
    created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (run_id, record_id, paper_id, channel)
);

CREATE TABLE IF NOT EXISTS run_history (
    run_id TEXT PRIMARY KEY,
    started_at TIMESTAMPTZ NOT NULL,
    finished_at TIMESTAMPTZ,
    status TEXT NOT NULL,
    notes TEXT DEFAULT ''
);

CREATE TABLE IF NOT EXISTS behavior_events (
    event_id TEXT PRIMARY KEY,
    record_id TEXT NOT NULL,
    paper_id TEXT NOT NULL,
    event_type TEXT NOT NULL,
    event_value TEXT DEFAULT '',
    channel TEXT DEFAULT '',
    metadata_json JSONB NOT NULL DEFAULT '{}'::jsonb,
    created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP
);
"""


def _as_json_text(value: Any) -> str:
    return json.dumps(value, ensure_ascii=False)


def _load_json_field(value: Any, fallback: Any) -> Any:
    if value is None:
        return fallback
    if isinstance(value, (list, dict)):
        return value
    if isinstance(value, str):
        text = value.strip()
        if not text:
            return fallback
        return json.loads(text)
    return fallback


def _to_iso(value: Any) -> str:
    if value is None:
        return datetime.now(timezone.utc).isoformat()
    if isinstance(value, datetime):
        if value.tzinfo is None:
            return value.replace(tzinfo=timezone.utc).isoformat()
        return value.isoformat()
    return str(value)


class Storage:
    def __init__(self, db_dsn: str):
        self.db_dsn = db_dsn.strip()
        if not self.db_dsn:
            raise RuntimeError("OPENCLAW_DB_DSN is required.")
        self._init_schema()

    @contextmanager
    def connection(self) -> Iterator[Any]:
        conn = psycopg.connect(self.db_dsn, row_factory=dict_row)
        try:
            yield conn
            conn.commit()
        finally:
            conn.close()

    def _init_schema(self) -> None:
        with self.connection() as conn:
            conn.execute(SCHEMA_POSTGRES)

    def load_profiles(self) -> list[MemberProfile]:
        with self.connection() as conn:
            rows = conn.execute("SELECT * FROM member_profiles ORDER BY display_name").fetchall()

        return [
            MemberProfile.from_dict(
                {
                    "record_id": row["record_id"],
                    "display_name": row["display_name"],
                    "team_name": row.get("team_name") or "",
                    "primary_direction": row.get("primary_direction") or "",
                    "secondary_directions": _load_json_field(row.get("secondary_directions"), []),
                    "keywords": _load_json_field(row.get("keywords"), []),
                    "excluded_topics": _load_json_field(row.get("excluded_topics"), []),
                    "feishu_user_id": row.get("feishu_user_id") or "",
                    "source": row.get("source") or "manual",
                    "confidence": row.get("confidence") if row.get("confidence") is not None else 1.0,
                    "version": row.get("version") if row.get("version") is not None else 1,
                    "updated_at": _to_iso(row.get("updated_at")),
                }
            )
            for row in rows
        ]

    def upsert_paper(self, paper: Paper) -> None:
        fetched_at = datetime.now(timezone.utc).isoformat()
        with self.connection() as conn:
            conn.execute(
                """
                INSERT INTO papers (
                    paper_id, title, abstract, authors, published_at,
                    source, url, tags, raw_json, fetched_at
                ) VALUES (%s, %s, %s, %s::jsonb, %s, %s, %s, %s::jsonb, %s::jsonb, %s)
                ON CONFLICT(paper_id) DO UPDATE SET
                    title=EXCLUDED.title,
                    abstract=EXCLUDED.abstract,
                    authors=EXCLUDED.authors,
                    published_at=EXCLUDED.published_at,
                    source=EXCLUDED.source,
                    url=EXCLUDED.url,
                    tags=EXCLUDED.tags,
                    raw_json=EXCLUDED.raw_json,
                    fetched_at=EXCLUDED.fetched_at
                """,
                (
                    paper.paper_id,
                    paper.title,
                    paper.abstract,
                    _as_json_text(paper.authors),
                    paper.published_at,
                    paper.source,
                    paper.url,
                    _as_json_text(paper.tags),
                    _as_json_text(paper.raw),
                    fetched_at,
                ),
            )

    def save_papers(self, papers: list[Paper]) -> None:
        for paper in papers:
            self.upsert_paper(paper)

    def load_recent_papers(self, limit: int = 500) -> list[Paper]:
        with self.connection() as conn:
            rows = conn.execute(
                "SELECT * FROM papers ORDER BY published_at DESC NULLS LAST LIMIT %s",
                (limit,),
            ).fetchall()

        return [
            Paper.from_dict(
                {
                    "paper_id": row["paper_id"],
                    "title": row["title"],
                    "abstract": row["abstract"],
                    "authors": _load_json_field(row.get("authors"), []),
                    "published_at": _to_iso(row.get("published_at")),
                    "source": row["source"],
                    "url": row["url"],
                    "tags": _load_json_field(row.get("tags"), []),
                    "raw": _load_json_field(row.get("raw_json"), {}),
                }
            )
            for row in rows
        ]

    def save_recommendations(self, run_id: str, recommendations: list[Recommendation]) -> None:
        with self.connection() as conn:
            for rec in recommendations:
                conn.execute(
                    """
                    INSERT INTO recommendations (
                        run_id, record_id, paper_id, score, rank, reason, channel, created_at
                    ) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)
                    ON CONFLICT (run_id, record_id, paper_id, channel) DO UPDATE SET
                        score=EXCLUDED.score,
                        rank=EXCLUDED.rank,
                        reason=EXCLUDED.reason,
                        created_at=EXCLUDED.created_at
                    """,
                    (
                        run_id,
                        rec.record_id,
                        rec.paper_id,
                        rec.score,
                        rec.rank,
                        rec.reason,
                        rec.channel,
                        rec.created_at,
                    ),
                )

    def save_run_status(self, run_id: str, status: str, started_at: str, finished_at: str | None = None, notes: str = "") -> None:
        with self.connection() as conn:
            conn.execute(
                """
                INSERT INTO run_history (run_id, started_at, finished_at, status, notes)
                VALUES (%s, %s, %s, %s, %s)
                ON CONFLICT(run_id) DO UPDATE SET
                    started_at=EXCLUDED.started_at,
                    finished_at=EXCLUDED.finished_at,
                    status=EXCLUDED.status,
                    notes=EXCLUDED.notes
                """,
                (run_id, started_at, finished_at, status, notes),
            )


path: entrypoints/run_paper_matching.py
content: """CLI entrypoint for paper matching and recommendation MVP."""

from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path


ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))

from paper_matching.config import load_settings  # noqa: E402
from paper_matching.pipeline import run_pipeline  # noqa: E402


def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(description="Run the paper matching recommendation MVP")
    parser.add_argument("--db-dsn", help="PostgreSQL DSN, e.g. postgresql://user:pass@host:5432/db")
    parser.add_argument("--output-dir", help="Directory to write run reports")
    parser.add_argument("--profile-source", choices=["db"], default="db", help="Load member profiles from database")
    parser.add_argument("--days", type=int, default=7, help="Lookback window in days")
    parser.add_argument("--top-k", type=int, default=5, help="Recommendations per member")
    parser.add_argument("--max-results-per-member", type=int, default=20, help="ArXiv fetch limit per member")
    parser.add_argument("--dry-run", action="store_true", help="Generate report without sending notifications")
    return parser


def main() -> int:
    parser = build_parser()
    args = parser.parse_args()
    settings = load_settings(db_dsn=args.db_dsn, output_dir=args.output_dir)
    report = run_pipeline(
        settings=settings,
        days=args.days,
        top_k=args.top_k,
        max_results_per_member=args.max_results_per_member,
        profile_source=args.profile_source,
        dry_run=args.dry_run,
    )
    print(json.dumps(report, ensure_ascii=False, indent=2))
    return 0


if __name__ == "__main__":
    raise SystemExit(main())


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)