Skill: xuanwuskill/has-anonymizer (version 1.0.3)

--- name: has-anonymizer description: "HaS (Hide and Seek) on-device text and image anonymization. Text: 8 languages (zh/en/fr/de/es/pt/ja/ko), open-set entity types. Image: 21 privacy categories (face, fingerprint, ID card, passport, license plate, etc.). Use when: (1) anonymizing text before sending to cloud LLMs then restoring the response, (2) anonymizing documents, code, emails, or messages before sharing, (3) scanning text or images for sensitive content, (4) anonymizing logs before handing to ops/support, (5) masking faces/IDs/plates in photos before publishing or sharing." metadata: { "openclaw": { "emoji": "🔒", "requires": { "bins": ["llama-server", "uv"], "env": { "HAS_TEXT_MODEL_PATH": { "description": "Absolute path to the HaS text GGUF model (default: ~/.openclaw/tools/has-anonymizer/models/has_text_model.gguf)", "required": false }, "HAS_IMAGE_MODEL": { "description": "Absolute path to the HaS image YOLO model (default: ~/.openclaw/tools/has-anonymizer/models/sensitive_seg_best.pt)", "required": false }, "HAS_TEXT_MAX_PARALLEL_REQUESTS": { "description": "Maximum parallel inference requests for scan/hide/restore batch operations (default: 4, max: 4)", "required": false } } }, "install": [ { "id": "brew-uv", "kind": "brew", "os": ["darwin"], "formula": "uv", "bins": ["uv"], "label": "Install uv (brew)", }, { "id": "brew-llama", "kind": "brew", "os": ["darwin"], "formula": "llama.cpp", "bins": ["llama-server"], "label": "Install llama.cpp (brew)", }, { "id": "download-model", "kind": "download", "url": "https://huggingface.co/xuanwulab/HaS_Text_0209_0.6B_Q8/resolve/main/has_text_model.gguf", "targetDir": "models", "label": "Download HaS text model Q8 (639 MB)", }, { "id": "download-image-model", "kind": "download", "url": "https://huggingface.co/xuanwulab/HaS_Image_0209_FP32/resolve/main/sensitive_seg_best.pt", "targetDir": "models", "label": "Download HaS image model FP32 (119 MB)", }, ], }, } --- # HaS Privacy HaS exposes a single umbrella CLI: - `has text ...` for text anonymization, restoration, and scanning - `has image ...` for image scanning, masking, and category discovery Use it when you need to remove private data locally before sending content elsewhere, inspect a directory for privacy risks, or mask visual privacy targets in photos and screenshots. ## Agent Decision Guidelines - Prefer `has text` for plaintext and `has image` for raster images. For mixed directories, run both and combine the results into one report. - For PDFs, Word documents, or scanned pages, extract text first and then use `has text`. For screenshots/photos where the goal is simply to hide visible carriers such as faces, screens, paper, labels, or QR codes, use `has image`. If the goal is to reason about the text content inside an image, run OCR first and then use `has text`. - Do not overwrite or delete the original files. Text commands can restore later, image masking is irreversible. - Proactively mention configurable knobs when the user intent is clear: `has text` uses repeated `--type`; `has image` uses repeated `--type`, plus `--method` and `--strength`. - If the user intent is ambiguous, start with `scan` before `hide`. - After batch scans, summarize text file count, image file count, findings by type/category, high-risk items, and the suggested next step. - If timing matters to the user, add `--timing` and report the elapsed result in plain language afterward. - For `qr_code` and `barcode`, the default mosaic strength is automatically raised based on the detection size to ensure the encoding is destroyed. The agent does not need to manually increase `--strength` for these categories. If a detection output includes `effective_strength`, report it to the user. ## Shared CLI Contract The current CLI contract is designed for agents first: - Success returns compact JSON. - Failure also returns compact JSON with `error.code` and `error.message`. - Returned path fields are absolute. - This includes `file`, `output`, `mapping_output`, and `skipped[].file`. - Invalid combinations fail fast instead of silently falling back. - Directory mode is non-recursive. Only immediate children are processed. - Batch results can include `skipped` and `skipped_count`. - Treat `skipped` entries as unprocessed files, not as clean files. Shared command layout: ```bash {baseDir}/scripts/has.sh <text|image> <command> [options] ``` Shared options can be placed before or after the subcommand. --- # Part 1: `has text` `has text` is the plaintext namespace. It supports: - `scan` - `hide` - `restore` It runs entirely on-device and uses a local `llama-server` plus the HaS text model when model inference is required. ## Core Text Concepts ### Semantic tags Anonymized text uses semantic tags such as: ```text <EntityType[ID].Category.Attribute> ``` This preserves structure better than a flat `[REDACTED]` token and is the reason restored downstream LLM output can remain usable. ### Open-set types Repeated `--type` flags are open-set. They are not limited to a fixed catalog. Natural language type names such as `"person name"`, `"address"`, `"phone number"`, or `"numeric values (transaction amounts)"` are valid. ### Public/private distinction Type wording matters. For example, `"personal location"` is usually safer than `"location"` if you want to preserve public places but hide private addresses. Public/private person-name distinctions remain less stable and should not be trusted without verification. ### Multilingual support The text model supports Chinese, English, French, German, Spanish, Portuguese, Japanese, and Korean, including mixed-language text. ### Type name language Match the `--type` language to the source text language: - **Chinese text** → use Chinese type names: `--type "人名" --type "电话号码" --type "地址"` - **Non-Chinese text** (English, French, German, etc.) → use English type names: `--type "person name" --type "phone number" --type "address"` ## Text Runtime Prerequisites `has text` auto-starts a local `llama-server` when needed. - Default model path: `~/.openclaw/tools/has-anonymizer/models/has_text_model.gguf` - Override model path: `HAS_TEXT_MODEL_PATH=/abs/path/to/has_text_model.gguf` - Override parallel cap: `HAS_TEXT_MAX_PARALLEL_REQUESTS` - If HuggingFace downloads fail, see **Model Download Mirrors** below. ## Text Usage ```bash {baseDir}/scripts/has.sh text [--timing] [--verbose] <scan|hide|restore> [options] ``` Namespace options: | Option | Description | |--------|-------------| | `--timing` | Include `elapsed_ms` in the JSON output | | `--verbose` | Emit runtime status and progress messages to stderr | Input methods: | Method | Description | |--------|-------------| | `--text '<text>'` | Pass text directly | | `--file <path>` | Read text from a file | | `--dir <path>` | Process immediate plaintext files in a directory | | stdin | For single-text mode when no `--text`, `--file`, or `--dir` is provided | Rules: - `--text`, `--file`, and `--dir` are mutually exclusive. - Empty `--type` values are rejected. - Directory mode only accepts batch output flags. - Single-file `hide` requires `--mapping-output`. - Single-file `restore` requires `--mapping`. - In text directory mode, `skipped` can include unprocessed files (binary, encoding, or read errors). ## `has text scan` Finds sensitive entities without replacing them. ```bash {baseDir}/scripts/has.sh text scan --type "person name" --type "phone number" --file report.txt {baseDir}/scripts/has.sh text scan --type "person name" --type "phone number" --dir ./reports/ ``` Parameters: | Parameter | Required | Description | |-----------|:--------:|-------------| | `--type` | yes | Entity type to scan for; repeat to add more | | `--text` / `--file` / `--dir` | one input | Input source | | `--max-chunk-tokens` | | Max tokens per chunk, default `5000` | | `--max-parallel-requests` | | Max scan chunks in parallel, default `4` | Output: - Single-text mode returns `{"entities": ...}` - Directory mode returns `{"results":[...],"count":N,"summary":{...}}` - Batch output may include `skipped` and `skipped_count` ## `has text hide` Replaces sensitive entities with semantic tags. ```bash {baseDir}/scripts/has.sh text hide --type "person name" --type "address" --text "John lives in Brooklyn" --mapping-output ./mapping.json {baseDir}/scripts/has.sh text hide --type "person name" --file note.txt --output ./note.anonymized.txt --mapping-output ./note.mapping.json {baseDir}/scripts/has.sh text hide --type "person name" --dir ./docs/ ``` Parameters: | Parameter | Required | Description | |-----------|:--------:|-------------| | `--type` | yes | Entity type to anonymize; repeat to add more | | `--text` / `--file` / `--dir` | one input | Input source | | `--mapping-output` | single-file: yes | Output path for generated mapping JSON | | `--output` | single-file | Output path for anonymized text | | `--mapping` | single-file | Existing mapping JSON file for incremental anonymization | | `--output-dir` | batch | Output directory for anonymized files (default: `<dir>/.has/anonymized/`) | | `--mapping-dir` | batch | Output directory for per-file mapping JSON files (default: `<output-dir>/mappings/`) | | `--max-chunk-tokens` | | Max tokens per chunk, default `3000` | | `--max-parallel-requests` | | Max files in parallel for `--dir`, default `4` | | `--no-tool-pair` | | Disable diff-based pair extraction; always use Model-Pair (slower but more robust) | Behavior: - Single-file mode never emits the mapping table inline. - Single-file mode returns either: - `{"text":"...","mapping_output":"/abs/path/to/map.json"}` - `{"output":"/abs/path/to/out.txt","mapping_output":"/abs/path/to/map.json"}` - Batch mode does not accept shared `--mapping`. - Mapping files are sensitive assets. Protect them. ## `has text restore` Restores anonymized text using mapping JSON. ```bash {baseDir}/scripts/has.sh text restore --mapping mapping.json --text "<person name[1].personal.name> lives in ..." {baseDir}/scripts/has.sh text restore --mapping mapping.json --file anonymized.txt --output restored.txt {baseDir}/scripts/has.sh text restore --dir ./.has/anonymized/ --output-dir ./.has/restored/ ``` Parameters: | Parameter | Required | Description | |-----------|:--------:|-------------| | `--mapping` | single-file: yes | Mapping JSON file path | | `--text` / `--file` / `--dir` | one input | Input source | | `--output` | single-file | Output path for restored text | | `--mapping-dir` | batch | Per-file mapping directory (default: `<dir>/mappings/`) | | `--output-dir` | batch | Output directory for restored files (default: sibling `restored/` under `.has/`, or `<dir>/.has/restored/`) | | `--max-chunk-tokens` | | Max tokens per chunk when model restore is needed, default `3000` | | `--max-parallel-requests` | | Max model-backed restore chunks in parallel | Behavior: - Single-file mode returns inline `text` unless `--output` is provided. - `restore --dir` uses per-file mapping JSON files. It does not accept a shared `--mapping`. - `restore --dir` expects mapping files at `<mapping-dir>/<filename>.mapping.json` (matching the naming convention produced by `hide --dir`). ## Typical Text Workflow Anonymize text before sending it to a cloud LLM, then restore the answer: 1. `hide` to produce anonymized text plus mapping 2. send anonymized text to the cloud model **with a tag-format explanation** (see below) 3. `restore` the model response with the mapping For multi-line text, prefer file-based intermediates over shell variables. ### Prompting the cloud LLM with anonymized text When forwarding anonymized text to a cloud LLM, the agent **must** prepend a brief explanation of the tag format so the model understands and preserves the tags. Include wording equivalent to the following (adjust language to match the conversation): > The text below has been anonymized. Sensitive entities are replaced by tags in the format `<EntityType[ID].Category.Attribute>`: > > - **EntityType** — the kind of entity (matches the `--type` value, e.g. `person name`, `address`, `phone number`). > - **[ID]** — a numeric identifier. The same type + same ID always refers to the **same** real-world entity (e.g. every `<person name[1]>` is the same person; `<person name[2]>` is a different person). > - **.Category.Attribute** — additional semantic classification of the entity. > > **Rules:** > 1. Preserve every tag exactly as-is in your response — do not modify, translate, paraphrase, omit, or expand any tag. > 2. When referring to an anonymized entity, reuse the original tag with the correct ID. > 3. Do not attempt to guess the real values behind the tags. Omitting this explanation may cause the cloud model to strip, rewrite, or misinterpret the tags, which will break the `restore` step. ## Model Download Mirrors If HuggingFace downloads fail, use these ModelScope mirrors: - text model: `https://modelscope.cn/models/TencentXuanwu/HaS_Text_0209_0.6B_Q8` - image model: `https://modelscope.cn/models/TencentXuanwu/HaS_Image_0209_FP32` --- # Part 2: `has image` `has image` is the image namespace. It supports: - `scan` - `hide` - `categories` It loads the YOLO segmentation model directly and does not require `llama-server`. ## Image Usage ```bash {baseDir}/scripts/has.sh image [--timing] [--model MODEL] <scan|hide|categories> [options] ``` Namespace options: | Option | Applies to | Description | |--------|------------|-------------| | `--timing` | all image commands | Include `elapsed_ms` in the JSON output | | `--model PATH` | `scan`, `hide` | Override the image model path | ## Image Privacy Categories Common categories include `biometric_face`, `id_card`, `passport`, `license_plate`, `qr_code`, `mobile_screen`, and `paper`. Use `has image categories` when you need the full catalog of 21 supported classes. `--type` accepts: - English names - Chinese names - numeric IDs - unique partial matches such as `face` Rules: - Empty `--type` values are rejected. - Ambiguous partial matches fail fast. - Omit `--type` to scan or mask all supported categories. - In image directory mode, `skipped` can include unprocessed files. ## `has image scan` Finds privacy regions without modifying the image. ```bash {baseDir}/scripts/has.sh image scan --image photo.jpg --type face --type id_card {baseDir}/scripts/has.sh image scan --dir ./photos/ --type face ``` Parameters: | Parameter | Required | Description | |-----------|:--------:|-------------| | `--image` / `--dir` | one input | Single image or batch directory | | `--type` | | Category filter; repeat to add more | | `--conf` | | Confidence threshold, default `0.25` | | `--model` | | Override image model path | Output: - Single-image mode returns `detections` and `summary` - Directory mode returns `results`, `count`, `summary`, and optional `skipped` ## `has image hide` Detects and masks privacy regions in images. ```bash {baseDir}/scripts/has.sh image hide --image photo.jpg --type face --method blur --strength 25 {baseDir}/scripts/has.sh image hide --dir ./photos/ ``` Parameters: | Parameter | Required | Description | |-----------|:--------:|-------------| | `--image` / `--dir` | one input | Single image or batch directory | | `--output` | single-image | Output image path | | `--output-dir` | batch | Output directory | | `--type` | | Category filter; repeat to add more | | `--method` | | `mosaic`, `blur`, or `fill`; default `mosaic` | | `--strength` | | Mosaic block size or blur radius; default `15` | | `--fill-color` | | Fill color for `fill`; default `#000000` | | `--conf` | | Confidence threshold; default `0.25` | | `--model` | | Override image model path | Behavior: - Refuses to overwrite the source image. - Directory mode accepts `--output-dir`, not `--output`. - For `qr_code` and `barcode` detections with `--method mosaic`, the block size is automatically raised to `max(strength, bbox_short_side // 10, 20)` to prevent the encoding from surviving pixelation. After masking, a lightweight verification confirms the code is no longer machine-readable; if it is, the strength is escalated further (up to a fill fallback). Each affected detection includes an `effective_strength` field in the output. - A cv2-based fallback supplements YOLO detection for QR codes and barcodes. When YOLO misses a code (e.g. large codes on plain backgrounds), `cv2.QRCodeDetector` and `cv2.barcode.BarcodeDetector` provide additional coverage. When YOLO misclassifies a code region as a different category (e.g. `monitor_screen`), cv2 corrects the category before `--type` filtering, so `--type qr_code` catches all QR codes regardless of YOLO's label. Corrected detections include a `"corrected_from"` field; new detections include `"cv2_fallback": true`. ## `has image categories` Lists all supported image privacy categories. ```bash {baseDir}/scripts/has.sh image categories {baseDir}/scripts/has.sh image categories --timing ``` Behavior: - Returns `{"categories":[...]}` - Supports `--timing` ## Suggested Combined Scan For a mixed workspace: 1. run `has text scan ... --dir <dir>` for plaintext 2. run `has image scan --dir <dir>` for images 3. merge the two JSON results into one privacy report If the user wants masking after that, use `hide` on the specific files or directories you already identified.

path: references/eval/eval.py
content: #!/usr/bin/env python3
"""Run scan → hide → restore on test cases and collect evidence for LLM judge.

Infrastructure checks only: command success, JSON validity, file existence,
leftover tags. All semantic evaluation (entity coverage, anonymization quality,
restoration fidelity) is left to the LLM judge reading results.json.
"""

from __future__ import annotations

import argparse
import json
import re
import shutil
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any

TAG_RE = re.compile(r"<[^<>]+>")

_EVAL_DIR = Path(__file__).resolve().parent
_SKILL_ROOT = _EVAL_DIR.parents[1]
_TEST_CASE_DIR = _EVAL_DIR / "test_case"


# ---------------------------------------------------------------------------
# CLI runner
# ---------------------------------------------------------------------------


@dataclass
class CmdResult:
    command: list[str]
    exit_code: int
    stdout: str
    stderr: str
    payload: Any | None
    error: str | None


def run_cmd(command: list[str], *, cwd: Path) -> CmdResult:
    """Run a command, parse stdout as JSON, return structured result."""
    proc = subprocess.run(
        command, cwd=cwd, text=True, capture_output=True, check=False
    )
    stdout = proc.stdout.strip()
    payload, error = None, None
    if stdout:
        try:
            payload = json.loads(stdout)
        except json.JSONDecodeError as exc:
            error = str(exc)
    else:
        error = "empty stdout"
    return CmdResult(command, proc.returncode, proc.stdout, proc.stderr, payload, error)


# ---------------------------------------------------------------------------
# Single-case evaluation
# ---------------------------------------------------------------------------


def evaluate_case(
    case: dict[str, Any],
    has_cli: Path,
    case_dir: Path,
    skill_root: Path,
) -> dict[str, Any]:
    """Run scan/hide/restore and collect evidence. Only infrastructure checks."""

    text = case["text"]
    expected = case["expected_entities"]
    has_expected = any(len(v) > 0 for v in expected.values())
    type_args = sum((["--type", t] for t in case["types"]), [])

    (case_dir / "original.txt").write_text(text, encoding="utf-8")

    errors: list[str] = []

    # --- scan ---
    scan_cmd = [str(has_cli), "text", "scan", *type_args, "--text", text]
    scan = run_cmd(scan_cmd, cwd=skill_root)
    _save_cmd(case_dir / "scan.cmd.json", scan)

    scan_entities: dict[str, list[str]] = {}
    if scan.exit_code != 0:
        errors.append("scan_failed")
    elif scan.error:
        errors.append("scan_not_json")
    elif isinstance(scan.payload, dict):
        raw = scan.payload.get("entities", {})
        if isinstance(raw, dict):
            scan_entities = {
                str(k): [str(i) for i in v]
                for k, v in raw.items()
                if isinstance(v, list)
            }
        else:
            errors.append("scan_no_entities_field")

    # --- hide ---
    mapping_path = case_dir / "mapping.json"
    hide_cmd = [
        str(has_cli), "text", "hide", *type_args,
        "--text", text, "--mapping-output", str(mapping_path),
    ]
    hide = run_cmd(hide_cmd, cwd=skill_root)
    _save_cmd(case_dir / "hide.cmd.json", hide)

    hide_text = ""
    if isinstance(hide.payload, dict):
        hide_text = str(hide.payload.get("text", ""))

    if hide.exit_code != 0:
        errors.append("hide_failed")
    elif hide.error:
        errors.append("hide_not_json")
    elif not hide_text:
        errors.append("hide_no_text")

    mapping: dict[str, Any] = {}
    if not mapping_path.exists():
        if hide.exit_code == 0:
            errors.append("mapping_missing")
    else:
        try:
            mapping = json.loads(mapping_path.read_text(encoding="utf-8"))
        except Exception:
            errors.append("mapping_invalid")

    # --- restore ---
    restore_ran = False
    restore_text = ""
    leftover_tags: list[str] = []

    if hide.exit_code == 0 and hide_text:
        restore_cmd = [
            str(has_cli), "text", "restore",
            "--mapping", str(mapping_path), "--text", hide_text,
        ]
        restore = run_cmd(restore_cmd, cwd=skill_root)
        _save_cmd(case_dir / "restore.cmd.json", restore)
        restore_ran = True

        if restore.exit_code != 0:
            errors.append("restore_failed")
        elif restore.error:
            errors.append("restore_not_json")
        else:
            if isinstance(restore.payload, dict):
                restore_text = str(restore.payload.get("text", ""))
            if not restore_text:
                errors.append("restore_no_text")

        leftover_tags = TAG_RE.findall(restore_text)
        if leftover_tags:
            errors.append("restore_has_leftover_tags")
    elif has_expected:
        errors.append("restore_skipped")

    status = "error" if errors else "ok"

    return {
        "id": case["id"],
        "language": case["language"],
        "status": status,
        "errors": errors,
        "types": case["types"],
        "original_text": text,
        "expected_entities": expected,
        "scan_entities": scan_entities,
        "hide_text": hide_text,
        "mapping": mapping,
        "restore_ran": restore_ran,
        "restore_text": restore_text,
        "restore_leftover_tags": leftover_tags,
    }


# ---------------------------------------------------------------------------
# I/O helpers
# ---------------------------------------------------------------------------


def _save_cmd(path: Path, result: CmdResult) -> None:
    """Save raw command execution details for debugging."""
    path.write_text(
        json.dumps(
            {
                "command": result.command,
                "exit_code": result.exit_code,
                "stdout": result.stdout,
                "stderr": result.stderr,
                "error": result.error,
            },
            ensure_ascii=False,
            indent=2,
        )
        + "\n",
        encoding="utf-8",
    )


def _write_json(path: Path, data: Any) -> None:
    path.write_text(
        json.dumps(data, ensure_ascii=False, indent=2, sort_keys=True) + "\n",
        encoding="utf-8",
    )


# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------


def main() -> int:
    parser = argparse.ArgumentParser(
        prog="eval",
        description=(
            "Run has text scan/hide/restore on test cases and collect evidence. "
            "Only infrastructure checks are applied; semantic evaluation is "
            "delegated to the LLM judge reading results.json."
        ),
    )
    parser.add_argument(
        "--cases",
        default=str(_TEST_CASE_DIR / "text_short_cases.json"),
        help="Path to the test case dataset JSON.",
    )
    parser.add_argument(
        "--has-cli",
        default=str(_SKILL_ROOT / "scripts" / "has.sh"),
        help="Path to the has CLI entry point.",
    )
    parser.add_argument(
        "--work-dir",
        default="/tmp/has-eval",
        help="Directory for artifacts and reports.",
    )
    parser.add_argument(
        "--case-id",
        action="append",
        dest="case_ids",
        help="Run only specific case IDs (repeatable).",
    )
    parser.add_argument(
        "--limit",
        type=int,
        help="Run only the first N cases.",
    )
    parser.add_argument(
        "--keep",
        action="store_true",
        help="Keep existing work directory.",
    )
    args = parser.parse_args()

    cases_path = Path(args.cases).resolve()
    has_cli = Path(args.has_cli).resolve()
    work_dir = Path(args.work_dir).resolve()

    dataset = json.loads(cases_path.read_text(encoding="utf-8"))
    cases: list[dict[str, Any]] = dataset["cases"]

    if args.case_ids:
        ids = set(args.case_ids)
        cases = [c for c in cases if c["id"] in ids]
    if args.limit is not None:
        cases = cases[: args.limit]

    if not args.keep and work_dir.exists():
        shutil.rmtree(work_dir)
    work_dir.mkdir(parents=True, exist_ok=True)

    counts = {"ok": 0, "error": 0}
    results: list[dict[str, Any]] = []

    for i, case in enumerate(cases, 1):
        case_dir = work_dir / "cases" / case["id"]
        case_dir.mkdir(parents=True, exist_ok=True)

        print(
            f"[{i}/{len(cases)}] {case['id']} ...",
            end="", flush=True, file=sys.stderr,
        )

        result = evaluate_case(case, has_cli, case_dir, _SKILL_ROOT)

        icon = "✓" if result["status"] == "ok" else "✗"
        print(f" {icon} {result['status']}", file=sys.stderr)
        if result["errors"]:
            print(f"       {result['errors']}", file=sys.stderr)

        counts[result["status"]] += 1
        results.append(result)
        _write_json(case_dir / "result.json", result)

    summary = {
        "dataset": str(cases_path),
        "work_dir": str(work_dir),
        "case_count": len(results),
        "counts": counts,
        "error_ids": [r["id"] for r in results if r["status"] == "error"],
    }

    _write_json(work_dir / "summary.json", summary)
    _write_json(work_dir / "results.json", results)

    print(json.dumps(summary, ensure_ascii=False, separators=(",", ":")))
    return 0


if __name__ == "__main__":
    sys.exit(main())


path: references/eval/test_case/text_long_cases.json
content: {
  "cases": [
    {
      "id": "zh-long-01",
      "language": "zh-CN",
      "types": [
        "人名",
        "电话号码",
        "工号",
        "地址",
        "邮箱"
      ],
      "text": "【机密】天远信息技术有限公司2025年度内部审计专项报告\n报告编号：IA-2025-Q4-017\n审计领域：供应商管理与采购合规性\n\n一、审计背景与范围\n\n根据公司2025年度审计工作计划的安排，内部审计部于第四季度对公司采购中心执行了一次专项审计。本次审计的重点在于审查过去十八个月内涉及信息技术基础设施类采购项目的合规性，评估采购流程中的内部控制设计和执行效果。审计组由内审部高级审计师周建华担任组长，成员包括两名审计专员和一名外聘的信息技术顾问。审计工作于2025年10月15日正式启动，历时八周，于12月10日完成现场审计程序和初步报告撰写。\n\n审计的范围覆盖了从供应商准入审查、报价评审和比选、合同签署与履约管理到验收结算与付款的完整采购业务循环。此外，审计组还对采购项目中涉及的资产登记和信息安全管理事项进行了延伸测试。在整个审计过程中，审计组共审阅了采购档案147份、供应商评审资料53份、合同文本39份、付款凭证216笔。为了全面了解业务操作流程的实际执行情况，审计组还对采购中心全部12名岗位人员进行了面对面访谈，每次访谈时长约45分钟至一小时不等。所有访谈均按照标准化的审计访谈模板进行，访谈记录经被访谈人签字确认后归入审计底稿。\n\n二、关键审计发现\n\n（一）供应商准入审查中存在的不足\n\n在对53份新供应商准入评审资料的系统性复核中，审计组发现有7份资料存在不同程度的文件瑕疵。详细分析如下：\n\n其中3份供应商准入卷宗中缺少资质证书有效期的交叉验证记录。公司采购管理制度明确要求，在供应商注册登记时，经办人员应当通过国家企业信用信息公示系统或行业主管部门网站对供应商提交的营业执照、资质证书和安全许可证的有效期进行独立验证，并将验证截屏保存在审批卷宗中。审计组在复核这三份卷宗时发现，经办人员只是将供应商自行提供的证照扫描件归档，但没有附上独立验证的截屏记录。虽然事后审计组通过独立查询确认了这三家供应商的资质均在有效期内，但文件记录的缺失意味着审批时的控制活动没有得到有效执行。\n\n另有2份卷宗缺少实地考察报告或替代的远程视频验证记录。公司制度允许对于年采购额预计低于50万元的新供应商采用视频验证代替现场考察，但这两份卷宗既没有现场考察报告也没有视频验证记录。经办人员在访谈中解释说，当时因为出差行程冲突，原计划的考察被推迟了，但后续没有跟进安排补充验证。审计组注意到，这两家供应商目前仍在公司的合格供应商名录中并有在途的采购业务。\n\n还有2份卷宗的信用评估报告引用了超过两年前的征信数据而未注明数据的时效性及其局限。公司信用评估指引要求引用的征信数据原则上不应超过最近12个月，如因特殊原因使用较早期的数据，应在评估报告中注明理由和相关风险提示。审计组认为，使用过期征信数据而不加说明可能影响信用评估结论的可靠性，特别是在当前经济环境下企业的信用状况变化较为频繁。\n\n需要特别说明的是，上述7家存在文件瑕疵的供应商中，有4家属于公司的长期合作伙伴，在准入评审时采用了针对老供应商续约的简化流程。内审部理解提高业务效率的考量，但认为即便是续约评审也应当保留关键控制节点的验证记录，以满足监管合规和内部治理的基本要求。审计组建议采购中心在2026年第一季度内完成对简化流程适用条件的重新评估，并更新相关操作指引。\n\n（一之一）供应商管理制度的演进历史\n\n为了更全面地理解当前供应商准入评审中存在的问题，审计组回顾了公司供应商管理制度的历史沿革。天远信息的第一版《供应商管理制度》制定于2018年，当时公司的年采购规模约为4500万元，在册供应商不超过30家。这一版本的制度对供应商准入评审的要求相对简单，主要包括营业执照验证、法人代表身份确认和基本的财务状况审查三个步骤。\n\n2020年，随着公司业务规模的扩大和信息技术采购需求的快速增长，采购中心对制度进行了第一次重大修订。修订后的版本增加了资质证书独立验证、实地考察或视频验证、信用评估报告和行业口碑调查四个新的审查环节。同时引入了供应商分级管理的概念：将供应商按年采购额和业务关键性分为战略级、重要级和一般级三个层级，不同层级适用不同深度的审查要求。\n\n2022年底，公司在引入新的企业资源管理系统后，对供应商管理流程进行了数字化改造。所有的准入评审环节被集成到系统中的电子化审批流程中，要求每个审查步骤都在系统中留下操作记录和附件。这次系统化改造本应显著提升文件记录的完整性，但审计组在本次审计中发现，系统设计中存在一个疏漏：系统虽然设置了审批节点，但各节点的附件上传被设计为非强制性，即经办人员可以在不上传验证截屏的情况下完成审批流转。这一系统设计上的缺陷直接导致了前述7份卷宗中文件记录缺失的问题。\n\n审计组建议信息技术部门在下一次系统维护窗口中，将供应商准入审批流程中所有涉及独立验证环节的附件上传设置为强制性必填项。在技术实现上，这一修改的工作量较小（预计开发和测试时间约一周），但在合规效果上可以从根本上避免文件记录缺失的问题复发。\n\n（二）报价评审与竞争性采购方面的问题\n\n在审阅的147份采购档案中，有23份涉及单一来源采购方式，占全部采购项目的比例约为15.6%。按照公司《采购管理制度》第十四条的规定，单一来源采购应当由采购经理提出书面申请，说明选择单一来源的理由和依据，经采购总监审批后方可执行。审计组逐一核对后发现，有8份单一来源采购缺少完整的审批记录。具体表现为：4份缺少采购总监的签字批准，2份有审批签字但缺少详细的理由说明，另有2份的审批表格中仅填写了\"技术需要\"四个字作为全部理由，缺乏具体的技术分析和市场调查说明。\n\n审计组选取了其中一个典型案例进行了深入调查。该案例涉及采购中心经理方志远主导的一次网络安全设备的完整采购项目。方志远的工号为EMP-2024-0731，于2024年7月入职公司，在加入天远信息之前有八年的IT采购工作经验。该采购项目的标的金额为人民币187万元，合同对手方为鼎创科技有限公司。鼎创科技是一家成立于2019年的网络安全解决方案提供商，注册地址位于深圳市南山区科技园路88号鼎创大厦12层，主要业务包括企业级防火墙、入侵检测系统和终端安全管理平台的研发与销售。\n\n方志远在采购审批表的单一来源理由栏中注明该供应商为\"唯一满足技术规格要求的厂商\"。然而，审计组在调阅该项目的技术评审记录时发现，评审委员会实际上曾收到过另外两家供应商提交的技术方案。在技术评审评分表上，三家供应商在八项核心指标中的得分如下：鼎创科技总分82.5分（满分100分），第二名供应商的总分为79分，第三名供应商的总分为71分。特别值得注意的是，排名第二的供应商在其中三项核心指标的得分与鼎创科技的差距均在5分以内。审计组因此认为，将该次采购定性为单一来源的依据不够充分，因为至少有一家其他供应商在技术上具备竞争力。\n\n审计组随后对方志远进行了详细的专项访谈。在访谈中，方志远解释说，之所以最终选择鼎创科技，是因为该厂商在前一年为公司的兄弟单位提供过同类型的网络安全设备，现场运行十个月以来表现稳定，设备可用率保持在99.7%以上。此外，鼎创科技承诺能够提供比其他两家供应商更短的交货周期。方志远还提到，鼎创科技的售后响应时效也优于竞争对手，合同中承诺了四小时内现场响应的服务条款。\n\n方志远同时坦承，对于采购审批流程中文件记录的不完整，他确实存在工作疏忽。他表示自己在填写单一来源理由时过于简略，没有充分展开说明技术评审的详细情况和选择鼎创科技的综合考量。但他强调，采购决策本身是基于技术评分、交货周期、售后服务和兄弟单位使用经验等多维度的综合比较做出的，不存在利益冲突或不当行为。审计组联系方志远时使用的手机号码为13912345678，该号码同时也是他在公司通讯录中登记的工作联系电话。\n\n审计组进一步调查发现，鼎创科技的客户经理何明在该项目的投标准备和评审过程中，曾多次与方志远进行沟通。何明的工作手机号为18765432100。根据方志远提供的手机通话记录和企业微信聊天截图，这些沟通主要涉及以下内容：设备技术参数的详细澄清、现场网络环境的兼容性确认、交货时间和安装排期的协调、以及报价方案的细节讨论。审计组在审阅这些通信记录后，未发现存在明显违反职业道德或公司利益冲突政策的内容。沟通的时间和频率也与项目的正常进展节奏基本一致。\n\n尽管如此，审计组仍然建议，对于涉及大额采购的项目，采购人员与潜在供应商之间的实质性业务沟通应当通过公司指定的采购协作平台或正式的企业邮件进行，并在采购卷宗中保留完整的沟通记录，以增强采购过程的透明度和可追溯性。\n\n（二之一）其他单一来源案例的详细审查\n\n除方志远主导的网络安全设备项目外，审计组还对其余7个缺少完整审批记录的单一来源采购案例逐一进行了复核。以下是审计组认为值得特别关注的另外两个案例：\n\n第二个典型案例涉及一项企业资源管理系统的年度维保和升级服务。该服务的年度合同金额为68万元，供应商为系统的原厂开发商。采购经办人员以\"维保服务只能由原厂提供\"为由申请了单一来源采购。审计组审查后认为，虽然原厂维保在技术上确实具有一定的排他性，但市场上存在经原厂授权的第三方服务商可以提供同等级别的维保服务，且报价通常比原厂低15%至25%。经办人员在审批材料中没有留存市场调研的证据，也没有说明为什么没有考虑授权第三方服务商的方案。\n\n第三个典型案例涉及为公司新办公区域采购的视频会议系统。该项目的合同金额为92万元，采购的理由是为了与公司总部和其他分支机构已有的视频会议系统保持品牌和技术标准的一致性。审计组认为这一理由虽然具有一定的合理性——统一品牌确实可以降低后续运维的复杂度和培训成本——但经办人员仍然应当提供一份书面的成本效益分析，量化说明统一品牌的维护成本节约与引入竞品的价格差异之间的权衡关系。现有的审批材料中完全缺少这方面的分析。\n\n审计组对全部23个单一来源采购项目按金额大小进行了排序。其中，金额最大的三个项目合计占单一来源采购总金额的58%。审计组建议，对于金额超过80万元的单一来源采购，应当在现有的审批流程基础上增加合规部门的独立审核环节。合规部门的职责不是评价采购决策的技术合理性，而是确认审批材料的完整性和流程的规范性。\n\n（二之二）供应商沟通管理现状评估\n\n审计组在对方志远与何明沟通记录审阅的基础上，进一步调查了采购中心整体的供应商沟通管理现状。调查结果显示，采购中心目前有4种主要的供应商沟通渠道：企业邮件、企业微信、电话和现场会议。\n\n在企业邮件方面，审计组抽查了采购中心12名员工过去六个月内与供应商之间的邮件往来记录（在获得相关授权的前提下）。抽查结果显示，绝大多数正式的报价确认、技术方案提交和合同条款协商都通过企业邮件进行，符合公司的沟通管理规范。但审计组注意到，有些重要的决策性沟通最初通过企业微信或电话达成一致后，并没有在邮件中进行正式的书面确认，这导致部分关键决策缺乏可追溯的书面依据。\n\n在企业微信方面，由于企业微信的聊天记录保留期限受限于公司的存储策略（目前设置为保留最近六个月），超过6个月的聊天记录在未主动导出的情况下将被系统自动清除。审计组发现，在方志远的采购项目中，部分早期的技术讨论记录已经因为超出保留期限而不可追溯。建议公司评估延长企业微信关键业务群组聊天记录保留期限的可行性，或者制定要求涉及重大采购决策的微信沟通应在24小时内通过邮件确认的操作规程。\n\n在电话沟通方面，公司虽然有通讯费用报销的详细记录，但通话内容本身没有留存。对于金额较大或争议性较高的采购项目，审计组建议采购人员在重要电话沟通后编写简要的通话备忘录，记录沟通的主要议题、达成的共识和后续行动事项。\n\n（三）合同管理与付款合规性检查\n\n在审计组对39份合同文本进行逐一复核的过程中，发现了以下几个方面的问题，涉及合同签署时效、验收标准设定和付款条件执行三个领域。\n\n关于合同签署时效方面，有5份合同的签署日期晚于实际到货日期或服务开始日期。换言之，供应商已经开始交付货物或提供服务，但正式的合同文本尚未完成签署程序。最极端的一个案例中，合同签署日期比实际到货日期晚了整整27天。这类操作虽然在某些紧急采购场景下可能有一定的合理业务理由，但从合规和风险管理的角度审视，这种做法显著增加了公司的法律敞口。在没有生效合同保护的情况下，如果交付的货物存在质量问题，或者服务未达到预期标准，公司在主张权利时将面临法律依据不足的困境。\n\n关于验收标准的问题，审计组发现有3份合同缺少明确的、可量化的验收标准。以一份IT运维外包服务合同为例，合同中的验收条款仅写明\"甲方对服务质量满意即视为验收通过\"，既没有定义具体的服务级别协议关键指标，也没有规定验收测试的具体流程和判定标准。审计组认为，这种模糊的验收条款不仅难以为日后可能出现的质量争议提供客观判定依据，也使得项目管理团队在日常监督供应商服务质量时缺乏明确的考核基准。\n\n在付款合规性方面，审计组发现有2份合同的实际付款节奏与合同约定的付款条件不一致。两份合同均约定为\"验收合格后30日内支付全部款项\"，但实际付款在验收报告签署后第7天即已完成，比合同约定时间提前了23天。财务分析师陈丽娜在接受审计访谈时对此作出了详细解释。陈丽娜的工号为EMP-2023-0415，她于2023年4月加入天远信息的财务部门，主要负责采购项目的财务分析和付款审批。陈丽娜表示，这两次提前付款是应供应商方面的紧急请求而做出的特殊安排。供应商在设备交付完成后不久即面临年底的资金周转压力，希望能够尽快收到货款以缓解现金流紧张。陈丽娜称她在处理提前付款时获得了财务总监的口头同意，但未按照公司财务审批流程的要求留存书面审批记录。\n\n审计组就此事与财务总监进行了确认。财务总监表示确实口头同意过这两笔提前付款，理由是维护与战略供应商的良好合作关系。但他也承认，口头审批的做法不符合公司的财务管理制度，今后将确保所有偏离合同付款条款的特殊安排都通过正式的书面审批流程。审计组再次强调，即便是出于商业合理性的特殊情况下的提前付款，也必须通过标准化的线上审批流程完成，并将审批记录自动关联至合同管理系统和财务核算系统，确保完整的审计跟踪链条。\n\n（三之三）供应商绩效考核体系评估\n\n审计组对采购中心现行的供应商绩效考核体系进行了系统性评估。目前，公司对合格供应商实行年度绩效评价制度，评价维度包括交货准时率、产品质量合格率、售后响应时效和商务配合度四个方面，各占25%的权重。\n\n审计组对最近两个年度的供应商绩效评价数据进行了统计分析。2024年度共有72家供应商接受了绩效评价，其中评价等级为\"优秀\"的有18家，\"良好\"的有39家，\"合格\"的有13家，\"需改进\"的有2家。没有供应商的评价结果为\"不合格\"。2025年度截至审计实施时已完成评价的有65家供应商，等级分布与上一年度基本相似。\n\n审计组注意到以下几个值得关注的现象：\n\n第一，绩效评价的分数分布过于集中。两个年度中，超过85%的供应商得分集中在75分至90分的区间内，标准差仅为8.3分。这种高度集中的分布可能说明评分标准不够精细，无法有效区分不同供应商的实际表现差异。审计组建议细化评分标准的分级描述，引入更多可量化的考核指标。\n\n第二，\"交货准时率\"这一指标的计算方式存在定义模糊的问题。目前的计算公式以合同约定的交货日期为基准，但在实际操作中，有些项目的交货日期曾经过双方协商进行了调整（延期或提前），而调整后的日期并没有统一反映到绩效考核系统中。这导致部分供应商的交货准时率数据不够准确。例如，鼎创科技在2025年度有一笔交货记录被系统判定为\"迟交3天\"，但方志远提供的邮件记录证明，双方已经在合同补充条款中将交货期延长了5天，系统中的数据未及时更新。\n\n第三，绩效考核结果的应用机制有待加强。审计组发现，过去两年中没有任何供应商因绩效考核结果较差而被暂停合作或从合格供应商名录中移除。2024年评为\"需改进\"的2家供应商在收到整改通知后提交了整改计划书，但采购中心没有对整改落实情况进行跟踪验证，这两家供应商目前仍在正常承接新的采购业务。审计组建议建立绩效考核结果与供应商合作状态之间的刚性联动机制——例如连续两年评为\"需改进\"的供应商应当自动触发资格重审程序。\n\n审计组还建议在现有的四个评价维度基础上增加\"合规配合度\"作为第五个维度，用于衡量供应商在采购流程中配合公司合规要求的表现。具体的评估内容可以包括：供应商是否按时提交合规所需的资质文件更新、是否配合公司的审计和检查要求、以及供应商自身是否发生过合规违规事件等。\n\n（三之一）合同风险敞口的量化评估\n\n审计组对前述5份\"先到货后签合同\"案例中的法律和财务风险进行了初步量化评估。5份合同的合计金额为413万元。在合同签署完成前的空窗期（平均为14天，最长为27天），公司在法律上缺乏明确的合同保护，面临的潜在风险包括：\n\n第一，货物质量瑕疵风险。如果在空窗期内发现到货设备存在质量问题，由于正式合同尚未签署，公司主张退换货的法律依据较为薄弱。虽然可以依据采购订单和供应商的报价确认函主张权利，但这些文件的法律约束力不如正式合同。审计组按照行业统计数据估算，大约有2%至3%的IT设备在到货后30天内发现隐性质量问题。以413万元的合同总金额和3%的瑕疵概率计算，潜在的质量索赔风险敞口约为12.4万元。\n\n第二，知识产权归属风险。在涉及软件定制开发和系统集成的项目中，如果合同未及时签署，软件著作权和技术方案的知识产权归属可能处于模糊状态。审计组审查的5份合同中有2份涉及一定程度的定制开发内容，涉及金额约为145万元。在正式合同未签署的情况下，供应商保留对其开发成果的知识产权主张权利的理论可能性，尽管在实务中这种争议发生的概率较低。\n\n第三，价格锁定风险。在合同空窗期内，如果上游原材料价格发生显著波动，供应商可能以未签署合同为由申请调整报价。虽然在实际操作中，供应商单方面调价的情况极为罕见，但从合规专业审计的角度，审计组仍然需要指出这一理论上的风险,并建议通过制度设计予以防范。\n\n审计组与法务部门就上述风险评估进行了交叉确认。法务部门认为审计组的风险识别方向基本准确，但在风险量化方面建议采用更为保守的估算方法。法务部门特别指出，虽然在实务中\"先到货后签合同\"的操作在IT行业并不罕见，但公司作为上市公司的全资子公司，在内控合规方面应当保持高于行业一般水平的标准。\n\n（三之二）付款流程的内部控制测试\n\n在对付款合规性进行审查的基础上，审计组还对公司的整体付款流程执行了一组标准化的内部控制测试。测试覆盖了付款流程的五个关键控制节点：发票验证、合同匹配、预算额度检查、审批授权和银行账号核对。\n\n测试方法采用的是随机抽样法，从全年1247笔采购付款（不含固定期间付款）中随机抽取50笔，逐一检验每笔付款在五个控制节点上的合规情况。测试结果如下：\n\n发票验证方面，50笔付款全部附有供应商开具的合规增值税专用发票或普通发票，发票信息与采购订单信息一致。合同匹配方面，48笔付款能够与现行合同完整匹配，2笔存在前述的金额偏差或分期偏差。预算额度检查方面，49笔付款在申请时系统自动完成了预算余额检查并通过，1笔由于跨年度预算调整的时间差，系统检查显示预算余额不足，但实际上调整后的预算可以覆盖该笔支出，财务人员手动确认后予以放行。审批授权方面，50笔付款全部有完整的电子审批流程记录。银行账号核对方面，50笔付款中有48笔的收款银行账号与供应商在公司系统中的注册账号一致，2笔因为供应商近期更换了开户银行，经办人员在付款前通过电话与供应商重新确认了新的账号信息。\n\n综合上述测试结果，审计组评估付款流程关键控制节点的整体有效性为\"有效\"级别。存在的少量偏差属于流程执行层面的个别问题，不构成系统性的控制缺陷。但审计组仍然建议，对于供应商变更银行账号的场景，应当建立书面确认和二次授权机制。目前依靠电话确认的方式存在被社会工程学攻击利用的风险——如果攻击者冒充供应商财务人员来电要求变更收款账号，仅凭电话确认可能无法有效防范此类欺诈。\n\n（四）资产验收与登记管理\n\n审计组对前述187万元网络安全设备采购项目的资产验收和固定资产登记情况进行了重点跟踪。验收报告显示，该批设备于2025年6月20日到货并于当日和次日完成了开箱检验和初步功能测试。初步功能测试的内容包括设备加电自检、网络连通性测试和管理界面登录验证。测试结果显示全部设备功能正常，与采购订单的规格清单一致。然而，正式的验收签字程序直到2025年8月3日才完成，比设备到货日期晚了整整44天。\n\n在这44天的间隙期内，设备实际上已经被部署到公司的生产网络环境中并上线运行，为公司的核心业务系统提供网络安全防护。但在公司的固定资产管理系统中，这批设备始终处于\"待登记\"状态。这种已经投入使用但尚未完成资产登记的情形，不仅影响公司固定资产统计的准确性和完整性，还可能对设备的保险理赔和会计折旧计算产生实质性影响。如果在这44天的无保障期内设备发生损坏，公司可能因为资产登记信息的缺失而面临保险理赔困难。\n\n审计组进一步调查了验收延迟的具体原因。根据公司内部规定，大额采购项目的正式验收必须由全部三名指定的验收委员签字确认。在该项目中，三名验收委员分别是采购中心的技术主管、信息安全部的安全架构师和财务部的资产管理专员。审计组发现，信息安全部的安全架构师在设备到货时正在休年假，从6月18日到7月10日不在岗。年假结束后，该人员又被紧急调派到公司的另一个安全项目中支援了两周，直到7月底才回到本职岗位。在此期间，没有人启动验收委员的替补程序，也没有人就验收延期向管理层提出预警。\n\n审计组认为，公司目前的验收管理制度存在单点依赖的风险，任何一名验收委员的缺席都可能导致整个验收流程停滞。审计组建议引入验收委员的预设替补机制，当指定委员因年假、出差或项目借调等原因无法在设备到货后五个工作日内参与验收时，自动启用预设的替补委员代行验收职责。同时，审计组建议在固定资产管理系统中增设一项自动提醒功能，当设备到货超过十五个工作日仍未完成资产登记时，系统自动向资产管理部门和相关业务主管发送催办邮件。\n\n（四之一）历史采购数据的趋势分析\n\n为了更好地理解上述发现的系统性背景，审计组对采购中心过去三年的采购数据进行了趋势分析。数据显示，2023年至2025年间，公司采购支出总额从1.2亿元增长至1.85亿元，年均复合增长率约为24%。其中，信息技术基础设施类采购的增速尤为突出，三年间从2800万元增长至7200万元，占总采购支出的比重从23%上升至39%。\n\n与采购规模快速扩张形成对比的是，采购中心的人员编制在同一时期仅从9人增加到12人，增幅约为33%。审计组计算了人均管理的采购项目数量和金额：2023年每人平均管理约12个项目、金额约1330万元，到2025年则上升到每人平均管理约18个项目、金额约1540万元。项目数量增长了50%，远超过人员增幅。审计组认为，采购中心面临的工作负荷压力是导致部分流程执行不到位的重要背景因素。\n\n从供应商分布的角度来看，公司目前的合格供应商名录中共有87家供应商。其中，前10家供应商的采购额合计占总采购支出的约62%，前20家则占到约81%，呈现出较为集中的供应商依赖结构。鼎创科技在所有供应商中的采购额排名第七位，年度采购额约为350万元。审计组注意到，排名靠前的供应商中，有3家的合格供应商准入评审材料存在前述章节中提到的文件瑕疵问题，这意味着公司在重要供应商的管理方面存在一定的控制风险。\n\n审计组还对单一来源采购的行业对标数据进行了横向比较。根据中国内部审计协会2024年发布的《信息技术采购内部审计实务指南》，国内同等规模的科技企业中，单一来源采购占全部IT采购的比例中位数约为12%至18%。天远信息的单一来源采购占比为15.6%，处于行业中位水平。但审计组认为，更值得关注的不是比例本身，而是审批记录的完整性。行业指南明确指出，单一来源采购的审批文件应当包含技术独占性论证、市场调研记录和风险评估三个核心要素，而公司有超过三分之一的单一来源采购缺少这些核心要素中的一个或多个。\n\n（四之二）方志远采购项目的财务影响评估\n\n审计组对方志远主导的187万元网络安全设备采购项目进行了详细的财务影响评估。评估的目的不是质疑采购决策本身的商业合理性，而是评估该项目中发现的流程控制缺陷可能带来的财务后果。方志远在后续的补充访谈中提供了该项目的完整商务比较表，显示了三家供应商在价格、交货期和售后服务方面的差异。\n\n从价格比较来看，鼎创科技的报价187万元在三家供应商中处于中间位置——排名第三的供应商报价为168万元，排名第二的供应商则报价为201万元。鼎创科技的报价比最低价高出约11.3%。按照常规的竞争性比价流程，采购团队通常会要求报价较低的供应商进行技术答辩，确认其低价是否以牺牲产品质量或服务范围为代价。在该项目中，虽然方志远表示确实对排名第三的供应商进行了技术方案评审，但评审记录中没有专门针对\"低价是否影响产品质量\"这一议题的书面分析。\n\n审计组估算，如果该项目按照标准的竞争性采购流程执行——即组织正式的技术答辩、要求供应商提交最终报价、并由采购委员会做出集体决策——可能会产生约8至15万元的成本节约空间。当然，这一估算是基于假设条件的推演，实际的谈判结果取决于多种因素。审计组明确指出，上述估算不构成对方志远采购决策的负面评价，而是用于说明标准的竞争性采购流程在成本管理方面的潜在价值。\n\n方志远对审计组的补充调查给予了充分的配合。他提供了自己与三家供应商沟通的全部邮件记录和会议纪要。审计组在审阅这些材料后确认，方志远在商务谈判过程中确实为公司争取了包括两年免费维保、现场培训等在内的附加条款，这些附加条款按市场价格估算约值12至18万元。综合考虑附加条款的价值，该项目的整体采购性价比在审计组看来属于合理范围，但流程控制的缺陷仍然需要在制度层面进行纠正。\n\n（四之三）财务控制环节的深入分析\n\n在对陈丽娜负责的付款审批环节进行深入分析后，审计组发现了更广泛的财务控制弱点。除了前述的2笔提前付款外，审计组还对陈丽娜在2025年经手的全部78笔采购付款进行了抽样测试（覆盖率约为40%，共测试31笔）。测试结果显示，31笔付款中有28笔完全符合合同约定和公司付款审批流程，3笔存在不同程度的偏差：\n\n第一笔是前述的提前付款之一，偏差天数为23天。第二笔涉及一个软件许可证续约项目，合同约定分三期付款，但实际在第二期付款时将第二期和第三期合并支付，原因是供应商的财务系统升级期间无法分两次开具发票。第三笔涉及一个办公设备采购项目，付款金额比合同约定金额多出了3200元，原因是运输费用在合同签署后发生了调整，但合同补充协议在审批付款时尚未完成签署。\n\n陈丽娜在访谈中对这些偏差逐一进行了说明，并表示每一笔偏差都有相应的业务理由。审计组认为，单独来看，每笔偏差的金额和风险等级都不高，但从内部控制设计的角度来看，这些偏差反映出公司在\"例外情况管理\"方面缺乏系统化的处理机制。目前公司的财务管理系统缺少一项关键功能：当付款申请的金额、时间或分期方式与合同约定不一致时，系统应当自动触发额外的审批节点，要求经办人员说明偏差原因并获得上一级管理者的书面批准。\n\n审计组建议财务部在下一轮系统升级中，将\"合同条款偏差检测\"功能纳入开发计划。该功能应至少覆盖以下偏差场景：付款金额偏差超过合同金额的2%或5000元（取较低值）、付款时间偏差超过合同约定的5个工作日、分期次数或分期比例的变更、以及付款对象与合同签约方不一致等。\n\n（四之四）跨部门协作中的信息流转分析\n\n审计组在审计过程中还观察到采购中心与其他部门之间在信息流转方面存在的一些问题。这些问题虽然不够单独构成审计发现，但作为改善建议的背景信息一并记录。\n\n在采购需求确认阶段，技术部门提出的采购需求书通常以邮件形式发送给采购中心。审计组抽查了20份采购需求书，发现有6份的技术规格描述不够详细，导致采购人员在准备招标文件时需要反复与技术部门沟通确认。每个项目平均往返沟通3至4轮，耗时约5至7个工作日。审计组建议技术部门制定标准化的采购需求模板，明确必填字段和技术参数的详细程度要求。\n\n在供应商评审阶段，技术评审和商务评审分别由不同的部门执行，但两个评审流程的衔接机制不够紧密。在个别案例中，技术评审已经完成但商务评审尚未启动，中间的等待时间长达两周。又如前述方志远主导的项目中，技术评审委员会的评分结果在传递给商务评审团队时，只传递了最终得分而没有附上评分的详细依据和评审专家的个别意见，使得商务团队在综合评价时缺乏完整的信息支撑。\n\n在验收和付款阶段，验收报告从项目现场提交到财务部门获批的平均周转时间为12个工作日。其中，验收报告的编写和签署平均需要6个工作日，报告在部门间流转和审核需要另外6个工作日。审计组注意到，如果公司能够将验收报告的审签流程从纸质文件改为电子化审批，预计可以将周转时间缩短至7个工作日以内。\n\n（五之一）采购中心历年审计整改跟踪情况\n\n为了评估本次审计发现的问题是否具有反复性，审计组回顾了采购中心过去三年接受的历次内部审计和外部审计的结果及整改情况。\n\n2023年第二季度，公司委托外部事务所对采购流程进行了一次第三方审计。该次审计提出了6项整改建议，其中3项涉及供应商管理，2项涉及合同管理，1项涉及付款控制。截至本次审计启动时，这6项建议中有4项已经落实到位，1项部分落实，1项尚未启动整改。尚未启动整改的那项建议恰好与本次审计中发现的\"单一来源采购审批记录不完整\"问题高度重叠，说明该问题至少已存在两年以上而未得到有效解决。审计组在报告中对此进行了特别标注，并建议管理层给予优先关注。\n\n2024年第一季度，内审部对采购中心执行了一次针对合同管理的专项检查。检查发现了2个与本次审计类似的问题：合同签署滞后于实际履约和验收标准不够量化。当时的整改承诺是在2024年第三季度前完成合同模板的修订和系统上线。审计组在本次审计中验证后发现，合同模板确实进行了修订并增加了验收标准的标准化条款，但在实际应用层面，新模板的使用率约为70%。仍有约30%的合同使用了旧版模板，原因是部分项目在新模板发布前已经启动了商务谈判，项目经办人员认为在谈判过程中更换模板可能引起供应商的困惑。审计组理解过渡期的困难，但认为应当设定一个明确的截止日期，过了截止日期后必须全面使用新版本模板。\n\n审计组还查阅了2024年第三季度的跟踪审计报告。该报告确认了合同模板修订工作已经完成，但同时指出新模板在推广应用方面的力度不够，建议加强对项目经办人员的培训。对照本次审计的发现来看，培训效果确实有所提升——使用旧模板的比例已从2024年第三季度的约45%下降至本次审计覆盖期间的约30%。但审计组认为30%的残留比例仍然过高，建议在系统层面设置强制约束,当经办人员尝试上传旧版合同模板时，系统应当弹出提示并要求说明使用旧版模板的特殊理由。\n\n从整改管理的角度来看，审计组对采购中心的整改态度给予了积极评价——每次审计提出的建议都被认真研究并制定了整改方案。但整改的执行深度和到位率仍有提升空间，特别是涉及跨部门协作和系统改造的整改事项往往因为资源协调困难而进度滞后。审计组建议公司层面建立审计整改的项目化管理机制，将重要的审计整改事项纳入部门年度绩效考核体系，明确整改责任人、完成时限和验收标准。\n\n（五之二）合规培训与意识提升评估\n\n审计组对采购中心2025年度的合规培训记录进行了审查。记录显示，采购中心在本年度共组织了3次合规相关的培训活动：一次关于《采购管理制度》修订内容的宣讲会（3月份，时长两小时），一次来自法务部的反商业贿赂合规培训（6月份，时长三小时），以及一次由财务部主办的报销与付款流程规范培训（9月份，时长一小时半）。三次培训的参加率分别为83%、92%和75%。\n\n审计组认为培训的覆盖面基本足够，但在培训有效性方面提出以下观察：\n\n首先，三次培训均采用集中授课的形式，缺乏互动环节和案例研讨。采购合规培训如果能够加入真实案例的讨论（当然需要对案例进行脱敏处理），教学效果会显著优于单向的制度宣讲。审计组建议在2026年的培训计划中，至少安排一次以案例研讨为主要形式的专题培训。\n\n其次，培训内容侧重于制度条文的讲解，缺少关于\"灰色地带\"决策判断的指导。例如，什么情况下可以使用企业微信与供应商讨论技术方案？供应商在工作时间之外通过私人电话讨论报价细节是否合规？这些都是采购人员在日常工作中经常遇到但现行制度中没有明确规定的情境。审计组建议合规培训应当增加对这类常见场景的具体指导,帮助采购人员建立清晰的行为边界认知。\n\n第三，培训后的效果评估机制较为简单。目前仅在培训结束后发放一份满意度问卷。审计组建议增加知识掌握的检验环节，例如在培训后一周内进行一次简短的在线测试，覆盖培训的核心知识点和典型场景判断。\n\n（五）信息安全与数据保护领域的延伸审计发现\n\n在对采购流程的审计过程中，审计组还对采购中心的信息安全管理实务进行了延伸性测试。这一测试并非本次审计的主要目标，但审计组在执行审计程序时偶然发现了一些值得关注的情况。\n\n首先，审计组发现采购中心有3名员工在日常工作中使用个人电子邮箱传输包含供应商报价信息和合同草稿的文件。按照公司信息安全政策的规定，涉及商业敏感信息的文件传输应当通过公司提供的企业邮箱或经批准的安全文件共享平台进行，明确禁止使用个人邮箱传输未经脱敏处理的业务文件。审计组在访谈中了解到，这3名员工使用个人邮箱的原因是企业邮箱附件大小限制为20兆字节，而部分包含高清设备照片和详细技术图纸的文件超出了这一限制。该问题已在信息安全管理的另一份报告中进行了专项记录和跟踪。\n\n其次，审计组注意到采购中心的IT安全管理员刘洋在采购项目的技术评审过程中，曾通过他的工作邮箱[REDACTED_SECRET]向参与评审的外部技术顾问发送了包含公司内部网络拓扑简图和安全策略概述的文件。虽然刘洋在发送前已对文件进行了一定程度的脱敏处理，但审计组认为，即便是经过脱敏的网络架构信息，在通过邮件发送给外部人员时，也应当事先获得信息安全部门负责人的书面批准，并使用公司指定的加密传输方式。刘洋在访谈中表示，他当时认为脱敏后的信息不属于高敏感级别，因此没有走正式的外发审批流程。审计组已将此事项通报给信息安全部，由信息安全部根据公司的数据分类分级标准做进一步评估。\n\n三、审计结论与综合评价\n\n基于上述审计发现，审计组对采购中心内部控制的整体有效性给出\"基本有效，需要改进\"的综合评价意见。这一评价反映了审计组的判断：采购中心在基本的采购业务操作框架方面建立了较为完整的制度体系，大部分采购交易能够按照规定的流程执行。但在制度执行的一致性、文件记录的完整性和特殊情况处理的规范性方面，仍然存在若干需要改进的薄弱环节。\n\n审计组的主要改进建议概括如下：\n\n第一，强化供应商准入评审标准的执行力度。即使是长期合作伙伴的续约评审，也应保持关键验证节点的完整记录。建议采购中心制订一份针对续约评审的简化版检查清单，在提高效率的同时确保合规底线得到保障。完成时限建议为2026年3月31日前。\n\n第二，修订单一来源采购的审批流程和标准。当技术评审结果表明有两家或以上供应商满足基本技术要求且评分差距在合理范围内时，不应简单地将采购定性为单一来源。建议增加技术评审委员会出具的独立意见书环节，明确说明是否存在有效竞争以及选择特定供应商的客观理由。完成时限建议为2026年3月31日前。\n\n第三，建立合同签署时效的监控和预警机制。将合同签署完成时间纳入采购流程的关键绩效指标体系，设置到货前必须完成合同签署的硬性约束条件。对于确实需要紧急采购的情况，应当建立简化但仍然有效的应急审批通道。完成时限建议为2026年6月30日前。\n\n第四，完善偏离合同条款的特殊审批制度。所有偏离合同约定付款条件的提前支付或延迟支付均应通过标准化的线上审批流程完成，审批记录自动归入合同管理系统。口头审批不再作为有效的授权方式。完成时限建议为2026年3月31日前。\n\n第五，引入验收委员的替补机制和资产登记的超期预警功能。当指定的验收委员因故无法在规定时限内参与验收时，由预设的替补人员代行验收职责，确保资产管理流程的连续性。完成时限建议为2026年6月30日前。\n\n四、管理层回应\n\n采购中心总监和分管副总裁已对上述审计发现和改进建议进行了全面回应。对于供应商准入评审和单一来源采购相关的发现，采购中心表示将在2026年第一季度内完成制度修订和全员培训。对于合同管理、提前付款和资产登记方面的问题，采购中心和财务部门承诺将在2026年6月底前完成系统改造和流程更新。对于信息安全方面的延伸发现，信息安全部已启动专项评估程序。\n\n审计组将在2026年第三季度对上述整改措施的落实情况进行跟踪审计，届时将重新测试相关控制点的有效性，并就整改完成情况出具跟踪审计报告。\n\n审计组组长：周建华\n报告日期：2025年12月20日\n\n联系方式：如对本报告内容有疑问，请联系审计组组长周建华，工作手机13912345678或发送邮件至审计部公共邮箱。涉及鼎创科技相关事项的补充信息，可联系鼎创科技客户经理何明，手机号码18765432100。",
      "expected_entities": {
        "人名": [
          "周建华",
          "方志远",
          "陈丽娜",
          "何明",
          "刘洋"
        ],
        "电话号码": [
          "13912345678",
          "18765432100"
        ],
        "工号": [
          "EMP-2024-0731",
          "EMP-2023-0415"
        ],
        "地址": [
          "深圳市南山区科技园路88号鼎创大厦12层"
        ],
        "邮箱": [
          "[REDACTED_SECRET]"
        ]
      }
    }
  ]
}


path: references/eval/test_case/text_medium_cases.json
content: {
  "cases": [
    {
      "id": "zh-medium-01",
      "language": "zh-CN",
      "text": "上周三下午，客服二组在处理退款工单 TK-20260315-042 时发现该工单被反复转派了三次但始终没有最终归口。根据系统备注，最初的投诉由用户在 APP 端提交，原因是签收当天蓝牙耳机出现充电异常，连续三次充满后仅能使用不到两小时。一组的同事尝试过远程指导恢复出厂设置，但问题并未解决，随后建议走换货流程。组长在内部沟通群里确认情况后，将该工单重新指派给售后专员李明负责后续跟进。工单附件中记录了李明的直拨手机号 13912345678，用户备注希望在下午三点以后再联系，因为上午通常在开会，短信回复也比较慢。由于距首次投诉已超过 72 小时，按照服务承诺应优先在次日上午安排电话确认，并将处理进展同步更新到工单备注栏。主管要求本周五之前完成闭环，如果涉及产品质量问题还需同步通知供应链团队做批次排查。",
      "types": [
        "人名",
        "电话号码"
      ],
      "expected_entities": {
        "人名": [
          "李明"
        ],
        "电话号码": [
          "13912345678"
        ]
      }
    },
    {
      "id": "zh-medium-02",
      "language": "zh-CN",
      "text": "本季度产品改版计划已经进入设计评审阶段。上周五的跨部门会议上，前端组提出希望在四月底之前完成首页信息流的改版上线，但设计侧目前只完成了移动端的视觉定稿，桌面端的交互方案还在迭代中。会上确定由产品经理张婷负责协调排期冲突，她会在本周内把修订后的里程碑计划发给各方确认。如果需要提前获取高保真原型图或组件标注文件，可以发邮件到 [REDACTED_SECRET] 申请权限，张婷会在收到后一个工作日内回复。后端接口方面，用户画像和推荐模块的联调窗口目前排在五月第二周，如果前端进度提前，需要在四月中旬就启动联调环境搭建。设计团队建议在正式上线前安排两轮用户可用性测试，测试人群从内部灰度用户中抽取，每轮不少于 20 人。会议结论已整理到飞书文档中，各组本周内完成响应。",
      "types": [
        "人名",
        "邮箱"
      ],
      "expected_entities": {
        "人名": [
          "张婷"
        ],
        "邮箱": [
          "[REDACTED_SECRET]"
        ]
      }
    },
    {
      "id": "zh-medium-03",
      "language": "zh-CN",
      "text": "法务部本周需要将劳动仲裁答辩状的正式文本和相关证据材料寄出。这批材料总共有三份，分别对应三位前员工提出的加班工资争议。其中第一份材料涉及的争议金额最大，证据附件也最多，需要额外准备两套副本供仲裁庭存档。行政助理已经确认了快递账号和发件流程，但收件信息还差一位当事人的最新地址。经过人事核实后确认，王磊目前的收件地址已更新为上海市浦东新区丁香路88号，旧办公室那边已经没有人代收个人快件了。法务同事提醒，所有寄出材料的封面页上不应出现非当事人的个人信息，回执单据需要单独扫描归档到案件文件夹中。快递建议选择当日达或次日达的方式，并要求签收时拍照留底，以便在仲裁庭上作为送达回证使用。寄出前请再次核对材料清单和收件信息，避免遗漏或寄错地址。",
      "types": [
        "人名",
        "地址"
      ],
      "expected_entities": {
        "人名": [
          "王磊"
        ],
        "地址": [
          "上海市浦东新区丁香路88号"
        ]
      }
    },
    {
      "id": "zh-medium-04",
      "language": "zh-CN",
      "text": "研发区的门禁权限调整申请本周二开始集中批处理。物业管理系统里累积了十二条待审批记录，其中大部分是新入职员工的初始权限开通，还有几条是跨楼层访问权限的临时追加。安全运维在逐条复核时注意到，有一条记录的申请描述不够完整——只写了「恢复权限」，没有说明原因和范围。经过与人事确认，这条记录对应的是星河科技派驻到联合实验室的陈雪，工号 EMP-2048。她之前因为项目阶段调整被临时冻结了研发区和机房的通行权限，现在项目重新启动，需要恢复原有的全部区域访问。安全运维要求补充申请表中的审批人和有效期字段，并在通行记录里标注本次恢复的原因编码。权限变更生效后还需要回邮件给人事确认，确保门禁卡的物理状态与系统记录一致。整批权限申请预计在本周四之前全部处理完毕。",
      "types": [
        "人名",
        "组织机构",
        "工号"
      ],
      "expected_entities": {
        "人名": [
          "陈雪"
        ],
        "组织机构": [
          "星河科技"
        ],
        "工号": [
          "EMP-2048"
        ]
      }
    },
    {
      "id": "zh-medium-05",
      "language": "zh-CN",
      "text": "仓库管理组在上午的分拣会上通报了本周的出库异常情况。三月以来，B 区仓库的订单拣货准确率从 99.2% 下降到了 97.8%，主要集中在生鲜品类的温控标签缺失问题上。运营负责人要求从本周起对所有生鲜订单在封箱前增加一道核验步骤，逐件扫码确认温控标签是否粘贴到位。在此过程中还发现一笔优先级较高的订单需要特别处理——订单号 ORD-20260320-018 是一批实验设备的紧急调拨件，收货方指定由赵敏签收，不接受代签。仓库调度已联系承运方确认到站时间预计在明天下午两点左右，如果赵敏当时不在场需要提前改约。司机到站前应先确认交付窗口是否仍然有效，避免到了之后无人接收又要返回站点重新安排第二天派送。出库异常的整改报告需要在本月底之前提交给运营总监。",
      "types": [
        "人名",
        "订单号"
      ],
      "expected_entities": {
        "人名": [
          "赵敏"
        ],
        "订单号": [
          "ORD-20260320-018"
        ]
      }
    },
    {
      "id": "zh-medium-06",
      "language": "zh-CN",
      "text": "财务部门在月末批量审核报销单据时发现了几起填写不规范的情况。最常见的问题是出差住宿费的发票抬头与报销申请人不一致，另外还有几笔餐费报销超出了差旅标准但没有附主管审批意见。复核人员特别标注了一笔报销单需要退回修改：刘洋提交的差旅报销申请中，把收款银行卡号 6222021234567890123 直接写进了申请表的备注栏，而不是填在系统的收款账户字段里。这样做的问题是备注栏内容会出现在审批流的邮件通知中，意味着每一级审批人的邮箱里都会留下一份含有完整卡号的邮件记录。财务主管提醒，以后所有收款信息必须通过系统的加密字段录入，不要再写进任何自由文本区域。刘洋的这笔报销已经退回要求重新提交，同时需要请 IT 协助检查已发出的审批通知邮件是否可以从邮件服务器上批量清理。",
      "types": [
        "人名",
        "银行卡号"
      ],
      "expected_entities": {
        "人名": [
          "刘洋"
        ],
        "银行卡号": [
          "6222021234567890123"
        ]
      }
    },
    {
      "id": "zh-medium-07",
      "language": "zh-CN",
      "text": "公司差旅服务台收到了一个改签申请的加急工单。由于客户方临时将原定于下周二的技术交流会推迟到了周四，对应的出差行程需要全部顺延两天。差旅助理在操作改签时发现，航空公司的系统要求重新验证乘客信息，主要是因为原始订票记录中的证件有效期即将到期。系统提示乘客周宁的护照号 E12345678 对应的有效期截止到今年六月底，虽然在出行日期范围内仍然有效，但部分航司对剩余有效期不足六个月的证件会触发额外审核。差旅助理已经把这个情况通知了周宁，建议他确认证件更新进度，如果在出发前完成换证就需要同步更新差旅系统里的证件信息。改签本身的费用差价为 320 元，按照规定由部门预算承担，已提交审批等待主管确认。机票改签的操作窗口截止到本周五下午六点，逾期将产生额外手续费。",
      "types": [
        "人名",
        "护照号"
      ],
      "expected_entities": {
        "人名": [
          "周宁"
        ],
        "护照号": [
          "E12345678"
        ]
      }
    },
    {
      "id": "zh-medium-08",
      "language": "zh-CN",
      "text": "园区物业在今天上午的早会上通报了近期访客车辆管理的改进事项。由于地下停车场 B 区正在施工改造，临时访客车位从原来的 40 个缩减到了 25 个，导致上周连续三天出现访客无车位可停的情况，有两位来访的合作伙伴不得不在园区外公共停车场步行十分钟才到达会议楼。为了缓解这个问题，物业决定在 A 区划出 10 个临时车位专供访客使用，有效期到施工结束。同时更新访客登记流程——接待人需要在访客到达前一天通过内部系统提交车辆信息。今天的预登记记录中有一条来自研发部的申请，接待人注明来访者是合作方工程师孙浩，驾驶车牌为京A12345的深灰色大众帕萨特，预计上午十点到达。安保前台在车辆进场时只需核对预登记信息放行，不再需要现场手写登记单，交接班时也不保留含车牌号的纸质记录。",
      "types": [
        "人名",
        "车牌号"
      ],
      "expected_entities": {
        "人名": [
          "孙浩"
        ],
        "车牌号": [
          "京A12345"
        ]
      }
    },
    {
      "id": "zh-medium-09",
      "language": "zh-CN",
      "text": "人事部门正在进行本月新入职员工的材料复核。按照集团的合规要求，所有入职材料必须在员工报到后五个工作日内完成三级审核——用人部门初审、人事专员复核、合规组终审。目前这一批共有八位新员工的材料在走流程，大部分已经完成了前两级审核，只剩下合规终审环节。复核过程中发现一处需要修正的地方：林晓的入职登记表上身份证号填写为 11010119900101999X，但提交的身份证复印件扫描质量偏低，末位的 X 在系统 OCR 识别时被误读成了数字 8。人事专员已经与林晓本人电话确认了正确号码，并在系统中手动修正。合规组提醒，身份证号这类证件信息只用于背调阶段的核验和社保开户登记，复核完成后原始扫描件应移入加密档案库，不要保留在共享文件夹或邮件附件中。整理新员工名册的对外版本时，需要先做脱敏处理再分发给其他部门。",
      "types": [
        "人名",
        "身份证号"
      ],
      "expected_entities": {
        "人名": [
          "林晓"
        ],
        "身份证号": [
          "11010119900101999X"
        ]
      }
    },
    {
      "id": "zh-medium-10",
      "language": "zh-CN",
      "text": "技术沙龙筹备组在这周的协调会上确认了嘉宾邀约和场地布置的最新进展。目前已有六位嘉宾确认出席，其中两位来自外部合作企业，需要提前办理园区访客通行证。场地方面，三楼多功能厅的灯光和音响设备已经完成检修，但投影幕布的遥控器电池需要更换，行政组会在活动前一天处理。关于线上直播的安排，运营组建议用公司的视频号做同步推流，预计观看人数在 200 到 500 之间。活动当天的签到和嘉宾接待由吴倩统筹负责，如果有关于议程时间调整、展位安排或茶歇方面的问题，可以添加她的微信号 wq_test_2026 直接沟通。吴倩会在签到开始前两小时把会议链接和签到二维码单独发送到筹备群里。对外宣传物料上只列活动官方联系邮箱，不要出现个人的联系方式。筹备组还讨论了活动后的内容整理计划，包括演讲录像剪辑和要点摘要的发布节奏。",
      "types": [
        "人名",
        "社交账号"
      ],
      "expected_entities": {
        "人名": [
          "吴倩"
        ],
        "社交账号": [
          "wq_test_2026"
        ]
      }
    },
    {
      "id": "zh-medium-11",
      "language": "zh-CN",
      "text": "宽带运维团队在处理本月的用户故障工单时，发现有一组连续的断线投诉集中在同一个接入层交换机端口上。初步排查显示该交换机固件版本偏旧，在高并发时段存在已知的端口震荡问题。运维工程师远程登录到接入设备后确认了故障端口的状态日志，发现近七天内该端口累计重连超过 200 次。工单系统匹配到的受影响用户中，赵杰的投诉频率最高，过去两周已经报修了四次。根据装机记录，赵杰家的宽带终端设备序列号为 SN-A9K2-7781，绑定的公网 IP 地址是 203.0.113.17。工程师判断问题根因不在用户侧设备上，而是接入层交换机需要升级固件。已经提交了固件升级申请，预计在本周末维护窗口期间实施。升级完成后需要主动回访受影响用户，确认断线问题是否彻底解决，并在工单中闭环记录处理结论。",
      "types": [
        "人名",
        "IP地址",
        "设备标识"
      ],
      "expected_entities": {
        "人名": [
          "赵杰"
        ],
        "IP地址": [
          "203.0.113.17"
        ],
        "设备标识": [
          "SN-A9K2-7781"
        ]
      }
    },
    {
      "id": "zh-medium-12",
      "language": "zh-CN",
      "text": "呼吸内科病区今天上午的交接班会议按照惯例在八点准时开始。夜班值班护士汇报了过夜期间的重点巡查情况：三楼西区共有 28 位在院患者，夜间有两位患者出现了发热症状，已按医嘱给予退热处理并记录在护理日志中。其中一位是术后第三天的患者韩梅，腕带编号 PT-99381，凌晨三点体温升至 38.4°C，给予物理降温后四十分钟回落到 37.6°C，目前状态平稳。主治医生在晨查房时会重点关注她的恢复情况，如果体温再次反复可能需要调整抗感染方案。另外，昨天新收治了两位患者，床位信息已更新到电子白板上，护士站的排班也做了相应调整。交接班记录提醒，本周四有一批医疗耗材供应商要来院做产品演示，科室会整理部分典型案例作为讨论素材，但发给外部供应商的版本只保留病程讨论和处理结论，不能包含任何患者的身份信息或腕带编号。",
      "types": [
        "人名",
        "患者编号"
      ],
      "expected_entities": {
        "人名": [
          "韩梅"
        ],
        "患者编号": [
          "PT-99381"
        ]
      }
    },
    {
      "id": "zh-medium-13",
      "language": "zh-CN",
      "text": "中介在整理本周的租房签约进度时发现，有三套房源的签约流程都卡在了付款确认环节。其中两套是因为租客的公司转账审批流程还没走完，预计下周一可以到款。另外一套的情况有些特殊——房东和租客双方已经就租期、押金和维修责任达成一致，但房东对支付方式有额外要求。根据沟通记录，房东许晨希望租客把首月租金和押金合并后直接打到其个人银行账号 6217003810045219988，不走中介的托管账户。中介经理认为这种方式不符合公司的资金监管流程，建议还是通过平台代收后再转付给房东。目前双方还在协商这个问题，中介已经把平台代收的手续费减免政策发给了许晨供参考。如果周末之前能达成一致，下周一就可以安排正式签约。内部台账只需记录签约状态和到款情况，不应在共享文档中保留房东的原始银行账号。",
      "types": [
        "人名",
        "账号"
      ],
      "expected_entities": {
        "人名": [
          "许晨"
        ],
        "账号": [
          "6217003810045219988"
        ]
      }
    },
    {
      "id": "zh-medium-14",
      "language": "zh-CN",
      "text": "法务台账的月度清理会议上，团队逐一过了一遍当前在途合同的状态。本月共有十七份合同处于不同阶段，其中五份在等待对方盖章回寄，三份在内部法务审阅中，其余九份已经归档完毕。沟通效率最低的是一份与境外供应商的技术许可协议，双方在知识产权归属条款上已经来回修改了五轮，预计还需要两到三周才能定稿。另外，唐静负责跟进的那份服务外包框架合同——合同编号 CT-2026-0441——上周已经完成了对方的签章流程，对方通过快递把双签原件寄回，法务助理在周三收到后已完成归档扫描。唐静接下来需要把合同起止日期和关键条款摘要更新到项目管理系统中，供业务组在执行阶段查阅。转给其他组查看时只附摘要信息，不应在转发正文里保留完整合同编号和签约方的详细联系人信息。",
      "types": [
        "人名",
        "合同编号"
      ],
      "expected_entities": {
        "人名": [
          "唐静"
        ],
        "合同编号": [
          "CT-2026-0441"
        ]
      }
    },
    {
      "id": "zh-medium-15",
      "language": "zh-CN",
      "text": "每月的发票核销工作通常在月初第一周集中处理。财务组的核销流程分三步：先由提交人上传电子发票和对应的费用说明，然后由财务专员逐张验证发票真伪和抬头信息，最后与报销系统中的对应记录做金额匹配。本月核销量比上月增加了约 15%，主要是因为上季度积压了一批差旅报销没有及时清理。在这批积压中，有一张发票引起了核销人员的注意：高原提交的一张增值税普通发票，发票编号 INV-2026-11873，金额为 2860 元，备注栏写的是「技术咨询服务费」。核销人员在税务验证平台上查询后发现这张发票的开票日期与费用发生日期相差超过 90 天，按照公司财务制度需要补交一份延迟报销说明。财务组已经通知高原补充材料，预计在本周内可以完成核销。核销摘要归档时只需说明已处理状态和审核结论，不需要在归档摘要中展示原始票据编号。",
      "types": [
        "人名",
        "发票号"
      ],
      "expected_entities": {
        "人名": [
          "高原"
        ],
        "发票号": [
          "INV-2026-11873"
        ]
      }
    },
    {
      "id": "zh-medium-16",
      "language": "zh-CN",
      "text": "客服质检组在上周的升级工单回顾中，梳理了九月中旬以来所有从一线升级至二线的工单。总量共 47 件，其中 31 件在二线处理完毕并已闭环，12 件仍在跟进中，还有 4 件因为需要技术侧介入而转到了工程支持组。质检组重点看了其中三件处理时间最长的工单，分析延迟原因。排在第一位的是陈诺关联的升级工单，案例编号 CASE-2026-0915，该工单从一线升级至今已经超过十四个工作日。延迟的主要原因是涉及到的技术问题需要联合产品组和运维组共同排查，三方排期对齐花了较多时间。目前根因已经定位到了后端缓存层的一个竞态条件，修复补丁正在测试环境验证中，预计本周可以部署到生产环境。部署完成后需要联系陈诺确认问题是否彻底解决，并完成用户满意度回访。质检报告在跨团队同步时只展示统计数据和改进建议，不需要列出完整的案例编号和关联用户信息。",
      "types": [
        "人名",
        "案件编号"
      ],
      "expected_entities": {
        "人名": [
          "陈诺"
        ],
        "案件编号": [
          "CASE-2026-0915"
        ]
      }
    },
    {
      "id": "zh-medium-17",
      "language": "zh-CN",
      "text": "采购部在进行新一轮供应商资质审核时，需要核验每家供应商提交的营业执照、税务登记信息和银行开户证明。本轮审核涵盖了十二家候选供应商，其中八家是已有合作关系的续期审核，另外四家是新引入的候选方。审核流程要求每位采购专员至少负责三家，并在系统里逐项填写核验结论。负责消耗品品类的采购专员在核查第三家供应商时注意到一处异常：由马可提交的供应商资料中，统一社会信用代码填写为 91310110MA1K3F8Q2L，但营业执照扫描件上该编码的第九位数字不太清晰，看起来像是 K 也像是 R。专员已经联系马可要求重新提交一份高清扫描件进行确认。在此期间这家供应商的审核状态暂时标记为「待补充材料」。审核完成后的供应商入库清单对外共享时，需要隐去具体的税号和联系人信息，只保留公司名称和审核通过状态。",
      "types": [
        "人名",
        "税号"
      ],
      "expected_entities": {
        "人名": [
          "马可"
        ],
        "税号": [
          "91310110MA1K3F8Q2L"
        ]
      }
    },
    {
      "id": "zh-medium-18",
      "language": "zh-CN",
      "text": "本季度的城市创意市集将在下月初正式开放报名。根据活动组委会公布的方案，这次市集选址在市中心的滨河文创园区，预计设置 80 到 100 个摊位，分为手作工艺、独立设计、本地美食和互动体验四个主题区域。市集时间安排在每周六和周日的上午十点到下午六点，持续三个完整周末。目前已经有超过 200 个团队和个人提交了摊位申请，组委会将根据产品类型多样性和过往参展经验综合筛选，预计在报名截止后一周内公布入选名单。入选摊主需要自行准备展台搭建和产品陈列所需的物料，组委会负责提供统一的电力接口和废弃物回收服务。现场将配备流动安保和医疗急救站，并设置三个公共卫生间服务点。有关活动详细规则和报名入口的信息会在本周内发布到组委会的官方公众号上，届时也会同步发送到各社区创业服务群。",
      "types": [
        "人名",
        "电话号码",
        "地址"
      ],
      "expected_entities": {
        "人名": [],
        "电话号码": [],
        "地址": []
      }
    },
    {
      "id": "en-medium-01",
      "language": "en",
      "text": "The accounts payable team escalated a payment discrepancy that had been open for nearly two weeks without resolution. The original issue was flagged during a routine batch reconciliation when the system detected a mismatch between the approved reimbursement amount and the actual disbursement recorded by the bank. After reviewing the transaction logs, the team traced the discrepancy to a manual entry error in the wire transfer instructions — the intermediary bank code had been copied from an outdated template. Ryan Cooper, who submitted the original reimbursement request, confirmed during a follow-up call that the routing number on file should be 021000021 and that the funds were intended for a domestic account. The corrected transfer has been resubmitted and is currently awaiting clearance, which typically takes one to two business days. The finance operations manager has asked the team to update their reference templates this quarter to prevent similar issues and to include a secondary verification step for any manual wire transfers exceeding five thousand dollars.",
      "types": [
        "person name",
        "routing number"
      ],
      "expected_entities": {
        "person name": [
          "Ryan Cooper"
        ],
        "routing number": [
          "021000021"
        ]
      }
    },
    {
      "id": "en-medium-02",
      "language": "en",
      "text": "The quarterly account health review surfaced several cases where customer records had not been updated following recent subscription changes. Most of these were minor — expired promotional codes that should have been removed, or plan tier labels that still reflected last quarter's naming convention. However, one case required direct follow-up with the customer success team. The record for Nina Patel, associated with customer ID CUST-884201, showed an active enterprise subscription but had no designated billing contact on file. This typically happens when the original contract signer leaves the client organization and the transition paperwork is not completed. The customer success manager reached out to the client's new operations lead to confirm the current billing contact and verify that the invoicing address is still correct. In the meantime, the system has flagged the account to prevent any automated payment retries until the contact information is updated. Future status updates shared with partner teams should reference only the account health status and resolution steps, not the raw customer identifier.",
      "types": [
        "person name",
        "customer ID"
      ],
      "expected_entities": {
        "person name": [
          "Nina Patel"
        ],
        "customer ID": [
          "CUST-884201"
        ]
      }
    },
    {
      "id": "en-medium-03",
      "language": "en",
      "text": "The regional distribution center reported that a batch of shipments dispatched last Thursday experienced unexpected delays due to a customs hold at the transit hub. Out of the forty-seven packages in that batch, forty-four have since cleared customs and are back on schedule, but three remain held pending additional documentation. The logistics coordinator has been working with the customs broker to expedite clearance, and two of the three are expected to be released by end of day. The remaining package is a replacement unit for a laboratory instrument and requires a separate import classification code that was not included in the original shipping manifest. The recipient, Daniel Kim, has been notified about the delay and was given the tracking number ZX204668913US to monitor the shipment status directly. The customs broker estimates clearance will take an additional two to three business days once the corrected documentation is submitted. The warehouse team has already prepared the updated classification paperwork and will submit it through the broker's electronic filing system this afternoon.",
      "types": [
        "person name",
        "shipment tracking number"
      ],
      "expected_entities": {
        "person name": [
          "Daniel Kim"
        ],
        "shipment tracking number": [
          "ZX204668913US"
        ]
      }
    },
    {
      "id": "en-medium-04",
      "language": "en",
      "text": "The claims processing team has been reviewing a backlog of property damage reports submitted during the early March storm season. Most of the new claims involve minor wind damage to roofing and siding, and the adjusters have been able to process them within the standard five-day turnaround. One claim, however, required additional investigation because the reported damage exceeded the threshold for expedited review. Olivia Brooks filed the claim under insurance policy number POL-77-AC-20481, reporting significant water damage to a finished basement after a sump pump failure coincided with heavy rainfall. The adjuster who conducted the on-site inspection confirmed visible water damage to drywall, flooring, and several pieces of furniture, and recommended approving the claim for the full assessed amount. The claims supervisor has signed off on the assessment, and the settlement offer is being prepared for delivery to the policyholder. The internal case summary shared with the underwriting team for loss-ratio reporting should describe the damage category and resolution outcome but should not carry the raw policy number or personal contact details.",
      "types": [
        "person name",
        "insurance policy number"
      ],
      "expected_entities": {
        "person name": [
          "Olivia Brooks"
        ],
        "insurance policy number": [
          "POL-77-AC-20481"
        ]
      }
    },
    {
      "id": "en-medium-05",
      "language": "en",
      "text": "Corporate travel recently processed a batch of itinerary changes triggered by the rescheduling of the annual Asia-Pacific partner summit. The event was originally set for the last week of April but has been moved to mid-May to accommodate a scheduling conflict with two keynote speakers. As a result, twenty-three employee travel bookings need to be modified, including flights, hotel reservations, and ground transportation. The travel coordinator has been working through the list in order of departure date proximity. Most changes have been straightforward — same routing with shifted dates and minimal fare differences. One booking, however, requires special attention because it involves a multi-city itinerary with a stopover that falls on a weekend. Ethan Cole's reservation, referenced under booking code BR7K2P, includes flights from San Francisco to Tokyo and then onward to Singapore, with a planned weekend layover in Tokyo for client meetings. Shifting the entire itinerary by three weeks changes the fare class availability, and the coordinator is waiting for approval on the fare difference before confirming the rebooking. The deadline for rebooking is this Friday at noon Pacific time.",
      "types": [
        "person name",
        "booking reference"
      ],
      "expected_entities": {
        "person name": [
          "Ethan Cole"
        ],
        "booking reference": [
          "BR7K2P"
        ]
      }
    },
    {
      "id": "en-medium-06",
      "language": "en",
      "text": "The service recovery team held its weekly review of escalated guest complaints and identified three cases that require compensation beyond the standard goodwill gesture. Each case involves a service failure during a peak travel period where the guest experienced a significant disruption that was not resolved during their stay. The most time-sensitive case involves Sarah Gomez, who reported that her confirmed suite reservation was downgraded to a standard room upon arrival due to an overbooking error. The front desk offered a complimentary room upgrade for the following night, but the guest had already made dinner and event arrangements based on the suite layout. The service recovery manager reviewed the case details and approved a points credit of fifty thousand to the guest's loyalty account number LQ-5520-8841, along with a written apology from the general manager. The guest relations coordinator will reach out directly to confirm the resolution. When this case is included in the monthly service quality report for regional leadership, it should reference the incident type and recovery actions taken without including the guest's membership identifier or direct contact information.",
      "types": [
        "person name",
        "loyalty account number"
      ],
      "expected_entities": {
        "person name": [
          "Sarah Gomez"
        ],
        "loyalty account number": [
          "LQ-5520-8841"
        ]
      }
    },
    {
      "id": "en-medium-07",
      "language": "en",
      "text": "The registrar's office is in the middle of its spring semester enrollment audit, a process that reviews all active student records for discrepancies between course registrations, financial aid awards, and degree progress records. This year's audit flagged a higher than usual number of records with mismatched credit hours, largely due to a system migration that took place over winter break. Most mismatches were minor — a one-credit lab section showing as two credits, or a cross-listed course appearing under both department codes — and have been corrected in batch. One case, however, required individual follow-up with the student's academic advisor. Jacob Li, listed under student ID STU-2026-4419, appears to have completed all required coursework for a minor in data science, but the minor declaration was never formally recorded in the degree audit system. The advisor confirmed that the declaration form was submitted on time but was not processed due to a clerical backlog. The registrar has now updated the record to reflect the declared minor, and the corrected degree audit will be available in the student portal within two business days.",
      "types": [
        "person name",
        "student ID"
      ],
      "expected_entities": {
        "person name": [
          "Jacob Li"
        ],
        "student ID": [
          "STU-2026-4419"
        ]
      }
    },
    {
      "id": "en-medium-08",
      "language": "en",
      "text": "The procurement operations team wrapped up its monthly vendor performance scorecard review on Wednesday. The review covers delivery reliability, invoice accuracy, and issue resolution timeliness for all active vendors in the indirect materials category. Out of the thirty-eight vendors evaluated this quarter, thirty-two met or exceeded the target thresholds across all three metrics. Four vendors fell below the delivery reliability target due to supply chain disruptions in the semiconductor components category, and two had invoice accuracy scores below the acceptable range. The team flagged one vendor for a more detailed performance conversation. Michael Reed, the category manager for electronic components, is responsible for the relationship with this vendor, which is registered in the system under vendor ID VEND-04128. The vendor's delivery reliability score dropped from ninety-four percent to eighty-one percent over the past two quarters, primarily due to recurring lead-time extensions that were not communicated in advance. Michael has scheduled a quarterly business review with the vendor's account team for next Tuesday to discuss corrective actions and agree on an improvement plan.",
      "types": [
        "person name",
        "vendor ID"
      ],
      "expected_entities": {
        "person name": [
          "Michael Reed"
        ],
        "vendor ID": [
          "VEND-04128"
        ]
      }
    },
    {
      "id": "en-medium-09",
      "language": "en",
      "text": "The IT service desk received a support ticket last Monday from an employee reporting persistent Wi-Fi connectivity issues on a company-issued laptop. The user described intermittent disconnections occurring three to four times per hour, primarily during video calls, which had been affecting productivity for over a week. A help desk technician performed initial remote diagnostics and confirmed that the laptop's wireless network adapter was experiencing frequent reassociations with the access point, indicating either a driver compatibility issue or a hardware fault. The ticket was escalated to the network engineering team for deeper investigation. After correlating the laptop's wireless logs with the access point controller data, the engineers identified that the device belonging to Ava Thompson, which has a wireless MAC address of 00:1A:2B:3C:4D:5E, was consistently failing the 802.1X reauthentication handshake approximately every fifteen minutes. The root cause appears to be a known bug in the wireless adapter's firmware that was introduced in a recent driver update. The engineering team has rolled back the driver to the previous stable version and is monitoring the connection stability over the next forty-eight hours before closing the ticket.",
      "types": [
        "person name",
        "MAC address"
      ],
      "expected_entities": {
        "person name": [
          "Ava Thompson"
        ],
        "MAC address": [
          "00:1A:2B:3C:4D:5E"
        ]
      }
    },
    {
      "id": "ja-medium-01",
      "language": "ja",
      "text": "今月の安全運転管理点検の一環として、総務部は全社の社用車ユーザーリストと運転免許情報の照合を行っています。この点検は年に二回、四月と十月に実施され、免許の有効期限確認、違反歴の自己申告、及び車両保険の付保状況を一括で確認する目的で行われます。今回の対象者は全国五つの拠点を合わせた合計84名で、そのうち79名はすでに必要書類の提出を完了しています。残りの五名のうち、四名は出張中のため来週提出予定ですが、一名については書類の記載内容に確認が必要な点がありました。営業部の山田太郎から提出された免許証コピーでは、免許番号がD-4021-778845と記載されていますが、前回の記録と末尾二桁が異なっており、更新時の番号変更か記載ミスかを判断するため、本人への再確認を依頼しています。確認が取れ次第、管理台帳を更新し、点検完了として総務部長に報告します。社外に共有する安全運転レポートには統計データのみを掲載し、個人の免許番号や氏名は含めません。",
      "types": [
        "person name",
        "driver license number"
      ],
      "expected_entities": {
        "person name": [
          "山田太郎"
        ],
        "driver license number": [
          "D-4021-778845"
        ]
      }
    },
    {
      "id": "fr-medium-01",
      "language": "fr",
      "text": "La directrice des opérations a envoyé un récapitulatif interne concernant le partenariat en cours de négociation avec un studio de design basé à Lille. Les discussions portent sur un contrat de prestation de services créatifs d'une durée de dix-huit mois, couvrant la refonte de l'identité visuelle de trois gammes de produits. Lors de la dernière réunion, tenue mardi dernier en visioconférence, les deux parties se sont mises d'accord sur le périmètre fonctionnel et le calendrier général, mais les conditions de propriété intellectuelle restent à finaliser. La coordination côté interne est assurée par Claire Martin, qui travaille chez Atelier Nord en tant que responsable du pôle création. Elle a transmis la semaine dernière une première version du cahier des charges technique, accompagnée d'exemples de livrables attendus. L'équipe juridique interne a demandé quelques ajustements sur les clauses de confidentialité et de sous-traitance avant de passer à la rédaction finale du contrat. Une prochaine réunion est prévue vendredi pour valider les derniers points en suspens.",
      "types": [
        "person name",
        "organization"
      ],
      "expected_entities": {
        "person name": [
          "Claire Martin"
        ],
        "organization": [
          "Atelier Nord"
        ]
      }
    },
    {
      "id": "es-medium-01",
      "language": "es",
      "text": "El equipo de logística interna está revisando las entregas pendientes del mes de marzo para asegurarse de que todos los envíos prioritarios se completen antes del cierre trimestral. En total hay veintitrés paquetes en cola, de los cuales dieciocho ya tienen etiqueta de envío generada y están esperando recogida por parte del servicio de mensajería. Los cinco restantes están pendientes de verificación de dirección porque el sistema detectó discrepancias entre la dirección registrada en el perfil del destinatario y la que aparece en la solicitud de envío. El coordinador de logística ha comenzado a resolver estos casos uno por uno contactando directamente a los solicitantes. El primero que se resolvió fue el de Lucía Torres, cuyo paquete contiene documentación notarial que debe entregarse en mano. Después de confirmar con ella por teléfono, se verificó que la dirección correcta de entrega es Calle Mayor 15, Madrid, ya que su oficina anterior dejó de recibir correspondencia a nombre de terceros el mes pasado. El mensajero tiene instrucciones de solicitar firma en persona y no dejar el paquete en recepción si la destinataria no está presente.",
      "types": [
        "person name",
        "address"
      ],
      "expected_entities": {
        "person name": [
          "Lucía Torres"
        ],
        "address": [
          "Calle Mayor 15, Madrid"
        ]
      }
    }
  ]
}


path: references/eval/test_case/text_short_cases.json
content: {
  "cases": [
    {
      "id": "zh-01",
      "language": "zh-CN",
      "text": "张伟的手机号是13866667777，有事请在工作时间联系。",
      "types": [
        "人名",
        "电话号码"
      ],
      "expected_entities": {
        "人名": [
          "张伟"
        ],
        "电话号码": [
          "13866667777"
        ]
      }
    },
    {
      "id": "zh-02",
      "language": "zh-CN",
      "text": "请将合同初稿发送到林芳的邮箱 [REDACTED_SECRET]。",
      "types": [
        "人名",
        "邮箱"
      ],
      "expected_entities": {
        "人名": [
          "林芳"
        ],
        "邮箱": [
          "[REDACTED_SECRET]"
        ]
      }
    },
    {
      "id": "zh-03",
      "language": "zh-CN",
      "text": "入职登记表显示，陈磊的身份证号为310101199503156718。",
      "types": [
        "人名",
        "身份证号"
      ],
      "expected_entities": {
        "人名": [
          "陈磊"
        ],
        "身份证号": [
          "310101199503156718"
        ]
      }
    },
    {
      "id": "zh-04",
      "language": "zh-CN",
      "text": "请将样品寄到黄丽的地址：深圳市南山区科技园路22号。",
      "types": [
        "人名",
        "地址"
      ],
      "expected_entities": {
        "人名": [
          "黄丽"
        ],
        "地址": [
          "深圳市南山区科技园路22号"
        ]
      }
    },
    {
      "id": "zh-05",
      "language": "zh-CN",
      "text": "订单 ORD-20260315-007 需要杨帅签收，联系电话15011223344。",
      "types": [
        "订单号",
        "人名",
        "电话号码"
      ],
      "expected_entities": {
        "订单号": [
          "ORD-20260315-007"
        ],
        "人名": [
          "杨帅"
        ],
        "电话号码": [
          "15011223344"
        ]
      }
    },
    {
      "id": "zh-06",
      "language": "zh-CN",
      "text": "报销单上的收款账户是周琳的银行卡6228480321456789012。",
      "types": [
        "人名",
        "银行卡号"
      ],
      "expected_entities": {
        "人名": [
          "周琳"
        ],
        "银行卡号": [
          "6228480321456789012"
        ]
      }
    },
    {
      "id": "zh-07",
      "language": "zh-CN",
      "text": "孙鹏在华创软件的工号是 EMP-3156，请核实门禁权限。",
      "types": [
        "人名",
        "组织机构",
        "工号"
      ],
      "expected_entities": {
        "人名": [
          "孙鹏"
        ],
        "组织机构": [
          "华创软件"
        ],
        "工号": [
          "EMP-3156"
        ]
      }
    },
    {
      "id": "zh-08",
      "language": "zh-CN",
      "text": "访客车辆登记：郑凯，车牌号沪B56789。",
      "types": [
        "人名",
        "车牌号"
      ],
      "expected_entities": {
        "人名": [
          "郑凯"
        ],
        "车牌号": [
          "沪B56789"
        ]
      }
    },
    {
      "id": "zh-09",
      "language": "zh-CN",
      "text": "患者吴芳的腕带编号是 PT-70214，请核对用药记录。",
      "types": [
        "人名",
        "患者编号"
      ],
      "expected_entities": {
        "人名": [
          "吴芳"
        ],
        "患者编号": [
          "PT-70214"
        ]
      }
    },
    {
      "id": "zh-10",
      "language": "zh-CN",
      "text": "本周末市图书馆将举办免费阅读分享会，欢迎市民报名参加。",
      "types": [
        "人名",
        "电话号码"
      ],
      "expected_entities": {
        "人名": [],
        "电话号码": []
      }
    },
    {
      "id": "en-01",
      "language": "en",
      "text": "Passenger Noah Bennett is booked under passport number GH7724518.",
      "types": [
        "person name",
        "passport number"
      ],
      "expected_entities": {
        "person name": [
          "Noah Bennett"
        ],
        "passport number": [
          "GH7724518"
        ]
      }
    },
    {
      "id": "en-02",
      "language": "en",
      "text": "Wire the deposit to account 443322110099 with routing number 071000013.",
      "types": [
        "account number",
        "routing number"
      ],
      "expected_entities": {
        "account number": [
          "443322110099"
        ],
        "routing number": [
          "071000013"
        ]
      }
    },
    {
      "id": "en-03",
      "language": "en",
      "text": "Olivia Chen filed a claim under insurance policy number POL-88-BK-30952.",
      "types": [
        "person name",
        "insurance policy number"
      ],
      "expected_entities": {
        "person name": [
          "Olivia Chen"
        ],
        "insurance policy number": [
          "POL-88-BK-30952"
        ]
      }
    },
    {
      "id": "en-04",
      "language": "en",
      "text": "The laptop assigned to Ryan Torres has MAC address 5C:3A:1B:9D:7E:4F.",
      "types": [
        "person name",
        "MAC address"
      ],
      "expected_entities": {
        "person name": [
          "Ryan Torres"
        ],
        "MAC address": [
          "5C:3A:1B:9D:7E:4F"
        ]
      }
    },
    {
      "id": "en-05",
      "language": "en",
      "text": "Academic records for Maya Singh, student ID STU-2026-5583, are ready for review.",
      "types": [
        "person name",
        "student ID"
      ],
      "expected_entities": {
        "person name": [
          "Maya Singh"
        ],
        "student ID": [
          "STU-2026-5583"
        ]
      }
    },
    {
      "id": "en-06",
      "language": "en",
      "text": "The annual technology conference will take place at the convention center on November 15th.",
      "types": [
        "person name",
        "phone number"
      ],
      "expected_entities": {
        "person name": [],
        "phone number": []
      }
    },
    {
      "id": "ja-01",
      "language": "ja",
      "text": "田中一郎の連絡先は 03-5678-1234 です。",
      "types": [
        "person name",
        "phone number"
      ],
      "expected_entities": {
        "person name": [
          "田中一郎"
        ],
        "phone number": [
          "03-5678-1234"
        ]
      }
    },
    {
      "id": "ja-02",
      "language": "ja",
      "text": "佐藤健一の運転免許証番号は 302847556791 です。",
      "types": [
        "person name",
        "driver license number"
      ],
      "expected_entities": {
        "person name": [
          "佐藤健一"
        ],
        "driver license number": [
          "302847556791"
        ]
      }
    },
    {
      "id": "ja-03",
      "language": "ja",
      "text": "資料の送付先は鈴木花子のメールアドレス [REDACTED_SECRET] です。",
      "types": [
        "person name",
        "email"
      ],
      "expected_entities": {
        "person name": [
          "鈴木花子"
        ],
        "email": [
          "[REDACTED_SECRET]"
        ]
      }
    },
    {
      "id": "ko-01",
      "language": "ko",
      "text": "김민준 씨의 연락처는 010-9876-5432입니다.",
      "types": [
        "person name",
        "phone number"
      ],
      "expected_entities": {
        "person name": [
          "김민준"
        ],
        "phone number": [
          "010-9876-5432"
        ]
      }
    },
    {
      "id": "ko-02",
      "language": "ko",
      "text": "택배 수령지는 이서연, 서울특별시 강남구 역삼로 123입니다.",
      "types": [
        "person name",
        "address"
      ],
      "expected_entities": {
        "person name": [
          "이서연"
        ],
        "address": [
          "서울특별시 강남구 역삼로 123"
        ]
      }
    },
    {
      "id": "ko-03",
      "language": "ko",
      "text": "박지훈 고객의 회원번호는 CUST-660214입니다.",
      "types": [
        "person name",
        "customer ID"
      ],
      "expected_entities": {
        "person name": [
          "박지훈"
        ],
        "customer ID": [
          "CUST-660214"
        ]
      }
    },
    {
      "id": "fr-01",
      "language": "fr",
      "text": "Le courriel de Marc Lefèvre est [REDACTED_SECRET].",
      "types": [
        "person name",
        "email"
      ],
      "expected_entities": {
        "person name": [
          "Marc Lefèvre"
        ],
        "email": [
          "[REDACTED_SECRET]"
        ]
      }
    },
    {
      "id": "fr-02",
      "language": "fr",
      "text": "Lucas Dupont est responsable du contrat CT-2026-0587.",
      "types": [
        "person name",
        "contract number"
      ],
      "expected_entities": {
        "person name": [
          "Lucas Dupont"
        ],
        "contract number": [
          "CT-2026-0587"
        ]
      }
    },
    {
      "id": "de-01",
      "language": "de",
      "text": "Die Telefonnummer von Markus Weber ist +49 170 9876543.",
      "types": [
        "person name",
        "phone number"
      ],
      "expected_entities": {
        "person name": [
          "Markus Weber"
        ],
        "phone number": [
          "+49 170 9876543"
        ]
      }
    },
    {
      "id": "de-02",
      "language": "de",
      "text": "Die Steuernummer von Anna Fischer lautet 82 491 372 501.",
      "types": [
        "person name",
        "tax ID"
      ],
      "expected_entities": {
        "person name": [
          "Anna Fischer"
        ],
        "tax ID": [
          "82 491 372 501"
        ]
      }
    },
    {
      "id": "es-01",
      "language": "es",
      "text": "El paquete de Carlos Ruiz debe enviarse a Avenida Libertad 42, Barcelona.",
      "types": [
        "person name",
        "address"
      ],
      "expected_entities": {
        "person name": [
          "Carlos Ruiz"
        ],
        "address": [
          "Avenida Libertad 42, Barcelona"
        ]
      }
    },
    {
      "id": "es-02",
      "language": "es",
      "text": "La empresa inaugurará una nueva sede en el centro de la ciudad el próximo mes.",
      "types": [
        "person name",
        "phone number"
      ],
      "expected_entities": {
        "person name": [],
        "phone number": []
      }
    },
    {
      "id": "pt-01",
      "language": "pt",
      "text": "O número de telefone de João Silva é +55 11 91234-5678.",
      "types": [
        "person name",
        "phone number"
      ],
      "expected_entities": {
        "person name": [
          "João Silva"
        ],
        "phone number": [
          "+55 11 91234-5678"
        ]
      }
    },
    {
      "id": "pt-02",
      "language": "pt",
      "text": "A conta do Instagram de Ana Costa é anacosta_design.",
      "types": [
        "person name",
        "social media account"
      ],
      "expected_entities": {
        "person name": [
          "Ana Costa"
        ],
        "social media account": [
          "anacosta_design"
        ]
      }
    }
  ]
}


path: scripts/has-image.sh
content: #!/usr/bin/env bash
# Wrapper script for HaS Image CLI — invoked as {baseDir}/scripts/has-image
# Delegates to has_image.py via uv run for automatic dependency management.

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"

exec uv run "$SCRIPT_DIR/has_image.py" "$@"


path: scripts/has-text.sh
content: #!/usr/bin/env bash
# Wrapper script for HaS Text CLI — invoked as {baseDir}/scripts/has-text
# Delegates to has_text_entry.py via uv run for automatic dependency management.

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"

exec uv run "$SCRIPT_DIR/has_text_entry.py" "$@"


path: scripts/has.sh
content: #!/usr/bin/env bash
# Umbrella HaS CLI — dispatches to text/image sub-CLIs.

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"

print_help() {
  cat <<'EOF'
usage: has <text|image> ...

Namespaces:
  text   Text anonymization and restoration
  image  Image privacy scanning and masking

Examples:
  has text scan --type "person name" --file note.txt
  has text hide --type "person name" --file note.txt --mapping-output note.mapping.json
  has text restore --mapping mapping.json --file anonymized.txt
  has image scan --type face --image photo.jpg
  has image hide --type face --image photo.jpg
  has image categories
EOF
}

emit_error() {
  local code="$1"
  local message="$2"
  python3 - "$code" "$message" <<'PY'
import json
import sys

payload = {
    "schema_version": "1",
    "error": {
        "code": sys.argv[1],
        "message": sys.argv[2],
    },
}
print(json.dumps(payload, ensure_ascii=False, separators=(",", ":")))
PY
}

if [[ $# -eq 0 ]]; then
  emit_error "missing_namespace" "Choose a namespace: text or image."
  exit 1
fi

case "$1" in
  -h|--help|help)
    print_help
    exit 0
    ;;
  text)
    shift
    export HAS_CLI_PROG="has text"
    exec "$SCRIPT_DIR/has-text.sh" "$@"
    ;;
  image)
    shift
    export HAS_CLI_PROG="has image"
    exec "$SCRIPT_DIR/has-image.sh" "$@"
    ;;
  *)
    emit_error "invalid_namespace" "Unknown namespace '$1'. Choose text or image."
    exit 1
    ;;
esac


path: scripts/has_image.py
content: #!/usr/bin/env python3
# /// script
# requires-python = ">=3.10"
# dependencies = [
#     "ultralytics>=8.3.0",
#     "opencv-python-headless>=4.8.0",
#     "pillow>=10.0.0",
# ]
# ///
"""HaS Image — Privacy anonymization for images via YOLO11 instance segmentation.

Usage:
    has-image scan --image photo.jpg [--type face] [--type id_card] [--conf 0.5]
    has-image hide --image photo.jpg [--output masked.jpg] [--method mosaic]
    has-image hide --dir ./photos/ [--output-dir ./.has/masked/]

See `has-image <command> --help` for details.
"""

from __future__ import annotations

import argparse
from dataclasses import dataclass
import json
import os
import sys
import time
from pathlib import Path
from typing import Any

# ---------------------------------------------------------------------------
# Category definitions (21 classes)
# ---------------------------------------------------------------------------

CATEGORIES: list[dict[str, str]] = [
    {"id": "0",  "name": "biometric_face",        "zh": "人脸"},
    {"id": "1",  "name": "biometric_fingerprint",  "zh": "指纹"},
    {"id": "2",  "name": "biometric_palmprint",    "zh": "掌纹"},
    {"id": "3",  "name": "id_card",                "zh": "身份证"},
    {"id": "4",  "name": "hk_macau_permit",        "zh": "港澳通行证"},
    {"id": "5",  "name": "passport",               "zh": "护照"},
    {"id": "6",  "name": "employee_badge",         "zh": "工牌"},
    {"id": "7",  "name": "license_plate",          "zh": "车牌"},
    {"id": "8",  "name": "bank_card",              "zh": "银行卡"},
    {"id": "9",  "name": "physical_key",           "zh": "钥匙"},
    {"id": "10", "name": "receipt",                 "zh": "收据"},
    {"id": "11", "name": "shipping_label",          "zh": "快递单"},
    {"id": "12", "name": "official_seal",           "zh": "公章"},
    {"id": "13", "name": "whiteboard",              "zh": "白板"},
    {"id": "14", "name": "sticky_note",             "zh": "便签"},
    {"id": "15", "name": "mobile_screen",           "zh": "手机屏幕"},
    {"id": "16", "name": "monitor_screen",          "zh": "显示器屏幕"},
    {"id": "17", "name": "medical_wristband",       "zh": "医用腕带"},
    {"id": "18", "name": "qr_code",                 "zh": "二维码"},
    {"id": "19", "name": "barcode",                 "zh": "条形码"},
    {"id": "20", "name": "paper",                   "zh": "纸张"},
]

# Build lookup helpers
_ID_TO_CAT = {int(c["id"]): c for c in CATEGORIES}
_NAME_TO_ID = {c["name"]: int(c["id"]) for c in CATEGORIES}
_ZH_TO_ID = {c["zh"]: int(c["id"]) for c in CATEGORIES}
ALL_NAMES = [c["name"] for c in CATEGORIES]
SCHEMA_VERSION = "1"

# Categories with structured encoding patterns that need adaptive mosaic strength.
# Default mosaic block size (~15px) can align with the encoding module size,
# leaving the code machine-readable after masking.
_ADAPTIVE_MOSAIC_CATEGORIES: set[int] = {
    _NAME_TO_ID["qr_code"],   # 18
    _NAME_TO_ID["barcode"],   # 19
}


@dataclass(frozen=True)
class CLIError(Exception):
    code: str
    message: str

    def __str__(self) -> str:
        return self.message


class StructuredArgumentParser(argparse.ArgumentParser):
    """ArgumentParser that reports machine-readable errors."""

    def error(self, message: str) -> None:
        raise CLIError("invalid_arguments", message)


def _emit_json(payload: dict[str, Any], *, stream=None) -> None:
    target = stream or sys.stdout
    print(json.dumps(payload, ensure_ascii=False, separators=(",", ":")), file=target)


def _error_payload(code: str, message: str) -> dict[str, Any]:
    return {
        "schema_version": SCHEMA_VERSION,
        "error": {
            "code": code,
            "message": message,
        },
    }


def _absolute_path(path: Path | str) -> str:
    """Return a stable absolute path string without resolving symlink targets."""
    return str(Path(path).expanduser().absolute())


def _resolve_types(type_values: list[str] | None) -> set[int] | None:
    """Parse repeated --type flags into a set of class IDs, or None (= all)."""
    if not type_values:
        return None
    ids: set[int] = set()
    for raw_token in type_values:
        [REDACTED_SECRET])
        if not token:
            _die("invalid_type", "--type values must be non-empty strings.")
        # Try numeric ID
        if token.isdigit():
            cid = int(token)
            if cid in _ID_TO_CAT:
                ids.add(cid)
            else:
                _die("unknown_type", f"Unknown class ID: {cid}")
        # Try english name
        elif token in _NAME_TO_ID:
            ids.add(_NAME_TO_ID[token])
        # Try chinese name
        elif token in _ZH_TO_ID:
            ids.add(_ZH_TO_ID[token])
        # Try partial match (e.g. "face" -> "biometric_face")
        else:
            matches = [n for n in ALL_NAMES if token in n]
            if len(matches) == 1:
                ids.add(_NAME_TO_ID[matches[0]])
            elif len(matches) > 1:
                _die("ambiguous_type", f"Ambiguous type '{token}', matches: {matches}")
            else:
                _die(
                    "unknown_type",
                    f"Unknown type '{token}'. "
                    f"Valid types: {', '.join(ALL_NAMES)}"
                )
    return ids if ids else None


def _die(code: str, msg: str) -> None:
    raise CLIError(code, msg)


def _verbose(message: str) -> None:
    """Emit a progress message to stderr when --verbose is active."""
    if os.environ.get("HAS_IMAGE_VERBOSE") == "1":
        print(message, file=sys.stderr)


def _is_within_directory(path: Path, directory: Path) -> bool:
    """Return whether `path` resolves within `directory`."""
    try:
        path.relative_to(directory)
        return True
    except ValueError:
        return False


# ---------------------------------------------------------------------------
# Model loading
# ---------------------------------------------------------------------------

_MODEL = None
_MODEL_LOCK = __import__("threading").Lock()
_DEFAULT_MODEL_PATH = os.path.expanduser(
    "~/.openclaw/tools/has-anonymizer/models/sensitive_seg_best.pt"
)


def _load_model(model_path: str | None = None):
    """Load YOLO11 segmentation model (lazy singleton, thread-safe)."""
    global _MODEL
    if _MODEL is not None:
        return _MODEL

    with _MODEL_LOCK:
        # Double-check after acquiring the lock.
        if _MODEL is not None:
            return _MODEL

        from ultralytics import YOLO

        path = model_path or os.environ.get("HAS_IMAGE_MODEL", _DEFAULT_MODEL_PATH)
        if not os.path.isfile(path):
            _die(
                "model_not_found",
                f"Model file not found: {path}\n"
                f"Download it via: openclaw install has-anonymizer "
                f"(or manually from HuggingFace)"
            )
        _verbose(f"Loading image model from {path}...")
        _MODEL = YOLO(path)
        _verbose("Image model loaded.")
        return _MODEL


# ---------------------------------------------------------------------------
# Detection
# ---------------------------------------------------------------------------

def _run_detection(
    image_path: str,
    model_path: str | None,
    conf: float,
    type_ids: set[int] | None,
) -> dict[str, Any]:
    """Run YOLO detection on a single image and return structured results."""
    import cv2

    model = _load_model(model_path)
    results = model(image_path, conf=conf, verbose=False)

    # Phase 1: Collect ALL YOLO detections (unfiltered) for cv2 correction
    yolo_regions: list[tuple[int, list[int], dict[str, Any]]] = []
    if results and results[0].boxes is not None:
        result = results[0]
        for index, bbox, det in _iter_detection_regions(result, type_ids=None):
            yolo_regions.append((index, bbox, det))

    # Phase 2: cv2 code detection → correct YOLO + find new codes
    image = cv2.imread(image_path)
    cv2_codes = _run_cv2_code_detection(image) if image is not None else []
    new_cv2_dets = _apply_cv2_corrections(cv2_codes, yolo_regions)

    # Phase 3: Apply --type filtering AFTER correction
    detections = []
    summary: dict[str, int] = {}

    for _, _, det in yolo_regions:
        cls_id = _NAME_TO_ID.get(det["category"])
        if type_ids is not None and cls_id not in type_ids:
            continue
        detections.append(det)
        summary[det["category"]] = summary.get(det["category"], 0) + 1

    for det in new_cv2_dets:
        cls_id = _NAME_TO_ID.get(det["category"])
        if type_ids is not None and cls_id not in type_ids:
            continue
        detections.append(det)
        summary[det["category"]] = summary.get(det["category"], 0) + 1

    return {"detections": detections, "summary": summary}


def _iter_detection_regions(result, type_ids: set[int] | None):
    """Yield filtered detection metadata shared by scan/hide."""

    if result.boxes is None:
        return

    for index, box in enumerate(result.boxes):
        cls_id = int(box.cls[0].item())
        if type_ids is not None and cls_id not in type_ids:
            continue

        cat = _ID_TO_CAT.get(cls_id, {"name": f"unknown_{cls_id}", "zh": "未知"})
        confidence = float(box.conf[0].item())
        bbox = [int(x) for x in box.xyxy[0].tolist()]
        has_mask = result.masks is not None and index < len(result.masks.data)

        yield index, bbox, {
            "category": cat["name"],
            "category_zh": cat["zh"],
            "confidence": round(confidence, 4),
            "bbox": bbox,
            "has_mask": has_mask,
        }


def _build_detection_mask(result, index: int, bbox: list[int], image_shape: tuple[int, int]):
    """Build a segmentation or bbox mask for a detection."""

    import cv2
    import numpy as np

    h, w = image_shape
    has_mask = result.masks is not None and index < len(result.masks.data)
    if has_mask:
        seg_mask = result.masks.data[index].cpu().numpy()
        return cv2.resize(seg_mask, (w, h), interpolation=cv2.INTER_NEAREST).astype(np.uint8)

    mask = np.zeros((h, w), dtype=np.uint8)
    x1, y1, x2, y2 = bbox
    mask[y1:y2, x1:x2] = 1
    return mask


def _resolve_output_path(image_path: str, output_path: str | None) -> str:
    """Resolve the output path and refuse to overwrite the source image."""

    image = Path(image_path)
    target = Path(output_path) if output_path else image.parent / ".has" / "masked" / f"{image.stem}{image.suffix}"

    if target.resolve(strict=False) == image.resolve(strict=False):
        _die("refusing_overwrite", "Refusing to overwrite the original image; choose a different output path")

    return _absolute_path(target)


# ---------------------------------------------------------------------------
# Masking strategies
# ---------------------------------------------------------------------------

def _apply_mosaic(image, mask, strength: int):
    """Apply mosaic (pixelation) to masked region."""
    import cv2
    import numpy as np

    h, w = image.shape[:2]
    block = max(strength, 2)

    # Downscale then upscale to create pixelation
    small = cv2.resize(image, (max(w // block, 1), max(h // block, 1)),
                       interpolation=cv2.INTER_LINEAR)
    mosaic = cv2.resize(small, (w, h), interpolation=cv2.INTER_NEAREST)

    # Apply only within mask
    mask_bool = mask.astype(bool)
    image[mask_bool] = mosaic[mask_bool]
    return image


def _apply_blur(image, mask, strength: int):
    """Apply Gaussian blur to masked region."""
    import cv2
    import numpy as np

    # Kernel size must be odd; ensure a monotonic relationship with strength
    ksize = max(strength | 1, 3)

    blurred = cv2.GaussianBlur(image, (ksize, ksize), 0)
    mask_bool = mask.astype(bool)
    image[mask_bool] = blurred[mask_bool]
    return image


def _apply_fill(image, mask, color: tuple[int, int, int]):
    """Apply solid color fill to masked region."""
    mask_bool = mask.astype(bool)
    image[mask_bool] = color
    return image


def _adaptive_mosaic_strength(bbox: list[int], base_strength: int) -> int:
    """Calculate adaptive mosaic strength for structured-code categories.

    Ensures the mosaic block size is large enough to destroy encoding
    structure while preserving visual recognizability of the region.
    Formula: max(base_strength, bbox_short_side // 10, 20)
    """
    bbox_w = bbox[2] - bbox[0]
    bbox_h = bbox[3] - bbox[1]
    bbox_dim = min(bbox_w, bbox_h)
    return max(base_strength, bbox_dim // 10, 20)


def _verify_code_destroyed(image, bbox: list[int], cls_id: int | None = None) -> bool:
    """Check that a QR/barcode region is no longer machine-readable.

    Crops the bbox region (with small padding) and runs the appropriate
    detector:  cv2.QRCodeDetector for QR codes, cv2.barcode.BarcodeDetector
    for barcodes.  When cls_id is None, both detectors are tried.
    Returns True if the code is destroyed (good), False if still readable (bad).
    Cost: ~15ms per region — negligible vs. YOLO inference.
    """
    import cv2

    h, w = image.shape[:2]
    x1, y1, x2, y2 = bbox
    pad = 10
    roi = image[max(y1 - pad, 0):min(y2 + pad, h),
                max(x1 - pad, 0):min(x2 + pad, w)].copy()

    barcode_id = _NAME_TO_ID.get("barcode")  # 19

    # Check QR (unless explicitly barcode-only)
    if cls_id != barcode_id:
        qr_det = cv2.QRCodeDetector()
        ok, info, _, _ = qr_det.detectAndDecodeMulti(roi)
        if ok and any(info):
            return False

    # Check barcode (unless explicitly qr-only)
    if cls_id is None or cls_id == barcode_id:
        try:
            bar_det = cv2.barcode.BarcodeDetector()
            ok, decoded, _, _ = bar_det.detectAndDecodeMulti(roi)
            if ok and decoded is not None and any(decoded):
                return False
        except Exception:
            pass  # BarcodeDetector not available in older OpenCV builds

    return True


def _bbox_iou(b1: list[int], b2: list[int]) -> float:
    """Compute Intersection-over-Union between two [x1,y1,x2,y2] bboxes."""
    x1 = max(b1[0], b2[0])
    y1 = max(b1[1], b2[1])
    x2 = min(b1[2], b2[2])
    y2 = min(b1[3], b2[3])
    inter = max(0, x2 - x1) * max(0, y2 - y1)
    area1 = (b1[2] - b1[0]) * (b1[3] - b1[1])
    area2 = (b2[2] - b2[0]) * (b2[3] - b2[1])
    union = area1 + area2 - inter
    return inter / union if union > 0 else 0.0


def _run_cv2_code_detection(image) -> list[tuple[list[int], str]]:
    """Detect QR codes and barcodes using OpenCV's algorithmic detectors.

    Returns a list of (bbox, category_name) tuples.  Runs both
    cv2.QRCodeDetector and cv2.barcode.BarcodeDetector.
    Cost: ~35-165 ms depending on image size.
    """
    import cv2

    codes: list[tuple[list[int], str]] = []

    # QR codes
    try:
        qr_det = cv2.QRCodeDetector()
        found, points = qr_det.detectMulti(image)
        if found and points is not None:
            for i in range(len(points)):
                pts = points[i].astype(int)
                bbox = [
                    int(pts[:, 0].min()), int(pts[:, 1].min()),
                    int(pts[:, 0].max()), int(pts[:, 1].max()),
                ]
                codes.append((bbox, "qr_code"))
    except Exception as exc:
        _verbose(f"cv2 QR detection error: {exc}")

    # Barcodes
    try:
        bar_det = cv2.barcode.BarcodeDetector()
        found, _decoded, _types, points = bar_det.detectAndDecodeMulti(image)
        if found and points is not None:
            for i in range(len(points)):
                pts = points[i].astype(int)
                bbox = [
                    int(pts[:, 0].min()), int(pts[:, 1].min()),
                    int(pts[:, 0].max()), int(pts[:, 1].max()),
                ]
                codes.append((bbox, "barcode"))
    except Exception as exc:
        _verbose(f"cv2 barcode detection error: {exc}")

    return codes


def _apply_cv2_corrections(
    cv2_codes: list[tuple[list[int], str]],
    yolo_regions: list[tuple[int, list[int], dict[str, Any]]],
    iou_threshold: float = 0.3,
) -> list[dict[str, Any]]:
    """Use cv2 code detections to correct YOLO and find missed codes.

    Two effects:
    1. **Correction**: If a cv2 code overlaps (IoU > threshold) a YOLO
       detection that is NOT already a code category, the YOLO detection's
       category is corrected in-place.  A ``corrected_from`` field is added.
    2. **New detections**: cv2 codes that don't overlap ANY YOLO detection
       are returned as new detection dicts (``cv2_fallback: true``).

    Corrections happen BEFORE ``--type`` filtering so the user gets the
    behaviour they expect from their ``--type`` flags.
    """
    if not cv2_codes:
        return []

    # Track which cv2 codes matched an existing YOLO region
    cv2_matched: set[int] = set()

    for cv2_idx, (cv2_bbox, cv2_cat_name) in enumerate(cv2_codes):
        for _region_idx, (_yolo_index, yolo_bbox, yolo_det) in enumerate(yolo_regions):
            if _bbox_iou(cv2_bbox, yolo_bbox) <= iou_threshold:
                continue
            cv2_matched.add(cv2_idx)
            # Only correct if YOLO's category is NOT already a code
            yolo_cat_id = _NAME_TO_ID.get(yolo_det["category"])
            if yolo_cat_id in _ADAPTIVE_MOSAIC_CATEGORIES:
                continue  # already correct
            cv2_cat_id = _NAME_TO_ID[cv2_cat_name]
            cat = _ID_TO_CAT[cv2_cat_id]
            old_cat = yolo_det["category"]
            yolo_det["corrected_from"] = old_cat
            yolo_det["category"] = cat["name"]
            yolo_det["category_zh"] = cat["zh"]
            _verbose(f"cv2 type correction: {old_cat} → {cat['name']}")
            break  # one correction per cv2 code

    # Build new-detection dicts for cv2 codes with no YOLO overlap at all
    new_dets: list[dict[str, Any]] = []
    for cv2_idx, (cv2_bbox, cv2_cat_name) in enumerate(cv2_codes):
        if cv2_idx in cv2_matched:
            continue
        cat = _ID_TO_CAT[_NAME_TO_ID[cv2_cat_name]]
        new_dets.append({
            "category": cat["name"],
            "category_zh": cat["zh"],
            "confidence": 1.0,
            "bbox": cv2_bbox,
            "has_mask": False,
            "cv2_fallback": True,
        })

    if new_dets:
        _verbose(f"cv2 fallback found {len(new_dets)} additional code(s)")
    return new_dets


def _parse_color(color_str: str) -> tuple[int, int, int]:
    """Parse hex color string to BGR tuple (OpenCV format)."""
    color_str = color_str.lstrip("#")
    if len(color_str) != 6:
        _die("invalid_color", f"Invalid color format: #{color_str}. Expected #RRGGBB")
    try:
        r = int(color_str[0:2], 16)
        g = int(color_str[2:4], 16)
        b = int(color_str[4:6], 16)
    except ValueError:
        _die("invalid_color", f"Invalid color format: #{color_str}. Expected #RRGGBB")
    return (b, g, r)  # BGR for OpenCV


def _resolve_fill_color(method: str, fill_color: str) -> tuple[int, int, int] | None:
    """Validate fill settings before any model work starts."""
    if method != "fill":
        return None
    return _parse_color(fill_color)


# ---------------------------------------------------------------------------
# Per-detection masking (shared by YOLO and cv2 fallback paths)
# ---------------------------------------------------------------------------

def _apply_masking_strategy(
    image,
    mask,
    det: dict[str, Any],
    method: str,
    strength: int,
    fill_color: tuple[int, int, int] | None,
    image_shape: tuple[int, int],
) -> tuple[Any, dict[str, Any]]:
    """Apply the chosen masking method to a single detection region.

    Returns the (possibly modified) image and updated detection dict.
    For code categories (qr_code, barcode) with mosaic method, applies
    adaptive strength and post-masking verification.
    """
    bbox = det["bbox"]
    cls_id = _NAME_TO_ID.get(det["category"])
    is_code = cls_id is not None and cls_id in _ADAPTIVE_MOSAIC_CATEGORIES

    if method == "mosaic":
        effective = (_adaptive_mosaic_strength(bbox, strength)
                     if is_code else strength)
        if effective != strength:
            _verbose(f"{det['category']}: adaptive strength "
                     f"{strength} → {effective}")
        image = _apply_mosaic(image, mask, effective)

        # Post-masking verification for code categories
        if is_code and not _verify_code_destroyed(image, bbox, cls_id):
            escalated = max(effective * 2, 60)
            _verbose(f"{det['category']}: still readable, "
                     f"escalating {effective} → {escalated}")
            image = _apply_mosaic(image, mask, escalated)
            if not _verify_code_destroyed(image, bbox, cls_id):
                _verbose(f"{det['category']}: still readable "
                         f"after escalation, applying fill")
                image = _apply_fill(image, mask, (0, 0, 0))
                det["fill_fallback"] = True
            else:
                effective = escalated
        det["effective_strength"] = effective
    elif method == "blur":
        image = _apply_blur(image, mask, strength)
    elif method == "fill":
        if fill_color is None:
            _die("missing_fill_color",
                 "Fill color is required when --method=fill")
        image = _apply_fill(image, mask, fill_color)

    return image, det


# ---------------------------------------------------------------------------
# Hide (mask) a single image
# ---------------------------------------------------------------------------

def _run_hide(
    image_path: str,
    output_path: str | None,
    model_path: str | None,
    conf: float,
    type_ids: set[int] | None,
    method: str,
    strength: int,
    fill_color: tuple[int, int, int] | None,
) -> dict[str, Any]:
    """Detect and mask privacy regions in a single image."""
    import cv2
    import numpy as np

    model = _load_model(model_path)
    results = model(image_path, conf=conf, verbose=False)

    image = cv2.imread(image_path)
    if image is None:
        _die("read_failed", f"Failed to read image: {image_path}")

    h, w = image.shape[:2]
    original = image.copy()  # preserve for cv2 fallback

    # Phase 1: Collect ALL YOLO detections (unfiltered) for cv2 correction
    yolo_regions: list[tuple[int, list[int], dict[str, Any]]] = []
    result = None
    if results and results[0].boxes is not None:
        result = results[0]
        for index, bbox, det in _iter_detection_regions(result, type_ids=None):
            yolo_regions.append((index, bbox, det))

    # Phase 2: cv2 code detection on ORIGINAL → correct YOLO + find new
    cv2_codes = _run_cv2_code_detection(original)
    new_cv2_dets = _apply_cv2_corrections(cv2_codes, yolo_regions)

    # Phase 3: Apply --type filtering AFTER correction, then mask
    detections = []
    summary: dict[str, int] = {}

    for index, bbox, det in yolo_regions:
        cls_id = _NAME_TO_ID.get(det["category"])
        if type_ids is not None and cls_id not in type_ids:
            continue
        if result is not None:
            mask = _build_detection_mask(result, index, bbox, (h, w))
        else:
            mask = np.zeros((h, w), dtype=np.uint8)
            x1, y1, x2, y2 = bbox
            mask[y1:y2, x1:x2] = 1
        image, det = _apply_masking_strategy(
            image, mask, det, method, strength, fill_color, (h, w),
        )
        detections.append(det)
        summary[det["category"]] = summary.get(det["category"], 0) + 1

    for det in new_cv2_dets:
        cls_id = _NAME_TO_ID.get(det["category"])
        if type_ids is not None and cls_id not in type_ids:
            continue
        bbox = det["bbox"]
        x1, y1, x2, y2 = bbox
        mask = np.zeros((h, w), dtype=np.uint8)
        mask[y1:y2, x1:x2] = 1
        image, det = _apply_masking_strategy(
            image, mask, det, method, strength, fill_color, (h, w),
        )
        detections.append(det)
        summary[det["category"]] = summary.get(det["category"], 0) + 1

    output_path = _resolve_output_path(image_path, output_path)

    Path(output_path).parent.mkdir(parents=True, exist_ok=True)
    if not cv2.imwrite(output_path, image):
        _die("write_failed", f"Failed to write masked image: {output_path}")

    return {
        "output": _absolute_path(output_path),
        "detections": detections,
        "summary": summary,
        "method": method,
        "strength": strength,
    }


# ---------------------------------------------------------------------------
# Batch processing
# ---------------------------------------------------------------------------

IMAGE_EXTENSIONS = {".jpg", ".jpeg", ".png", ".bmp", ".webp", ".tiff", ".tif"}


def _collect_images(dir_path: str) -> tuple[list[str], list[dict[str, str]]]:
    """Collect immediate image files from a directory plus skipped entries."""
    d = Path(dir_path)
    if not d.is_dir():
        _die("invalid_directory", f"Not a directory: {dir_path}")
    root = d.resolve()
    image_paths: list[str] = []
    skipped: list[dict[str, str]] = []
    for f in sorted(d.iterdir()):
        if not f.is_file():
            continue
        if f.suffix.lower() not in IMAGE_EXTENSIONS:
            skipped.append({"file": _absolute_path(f), "reason": "unsupported_extension"})
            continue
        try:
            resolved = f.resolve()
        except OSError as exc:
            skipped.append({"file": _absolute_path(f), "reason": str(exc)})
            continue
        if not _is_within_directory(resolved, root):
            skipped.append({"file": _absolute_path(f), "reason": "symlink_escape"})
            continue
        image_paths.append(_absolute_path(f))
    return image_paths, skipped


def run_scan_batch(
    image_paths: list[str],
    model_path: str | None,
    conf: float,
    type_ids: set[int] | None,
    skipped: list[dict[str, str]] | None = None,
) -> dict[str, Any]:
    """Scan a batch of images serially while preserving input order."""
    if not image_paths:
        result: dict[str, Any] = {"results": [], "count": 0, "summary": {}}
        if skipped:
            result["skipped"] = skipped
            result["skipped_count"] = len(skipped)
        return result

    results = []
    for idx, image_path in enumerate(image_paths):
        _verbose(f"Scanning image {idx + 1}/{len(image_paths)}: {Path(image_path).name}")
        result = _run_detection(image_path, model_path, conf, type_ids)
        result["file"] = image_path
        results.append(result)

    merged_summary: dict[str, int] = {}
    for result in results:
        for cat, count in result.get("summary", {}).items():
            merged_summary[cat] = merged_summary.get(cat, 0) + count

    result = {
        "results": results,
        "count": len(results),
        "summary": merged_summary,
    }
    if skipped:
        result["skipped"] = skipped
        result["skipped_count"] = len(skipped)
    return result


def run_hide_batch(
    image_paths: list[str],
    output_dir: str,
    model_path: str | None,
    conf: float,
    type_ids: set[int] | None,
    method: str,
    strength: int,
    fill_color: tuple[int, int, int] | None,
    skipped: list[dict[str, str]] | None = None,
) -> dict[str, Any]:
    """Hide privacy regions in a batch of images serially while preserving input order."""
    if not image_paths:
        result: dict[str, Any] = {"results": [], "count": 0}
        if skipped:
            result["skipped"] = skipped
            result["skipped_count"] = len(skipped)
        return result

    results = []
    for idx, image_path in enumerate(image_paths):
        _verbose(f"Masking image {idx + 1}/{len(image_paths)}: {Path(image_path).name}")
        output_path = str(Path(output_dir) / Path(image_path).name)
        result = _run_hide(
            image_path,
            output_path,
            model_path,
            conf,
            type_ids,
            method,
            strength,
            fill_color,
        )
        result["file"] = image_path
        results.append(result)

    result = {"results": results, "count": len(results)}
    if skipped:
        result["skipped"] = skipped
        result["skipped_count"] = len(skipped)
    return result


# ---------------------------------------------------------------------------
# Subcommand: detect
# ---------------------------------------------------------------------------

def cmd_scan(args: argparse.Namespace) -> None:
    type_ids = _resolve_types(args.type)

    t0 = time.time()
    if args.dir:
        image_paths, skipped = _collect_images(args.dir)
        batch_result = run_scan_batch(
            image_paths,
            args.model,
            args.conf,
            type_ids,
            skipped,
        )
        elapsed_ms = round((time.time() - t0) * 1000)
        _output("scan", batch_result, timing=args.timing, elapsed_ms=elapsed_ms)
    else:
        # Single image mode
        result = _run_detection(args.image, args.model, args.conf, type_ids)
        elapsed_ms = round((time.time() - t0) * 1000)
        _output("scan", result, timing=args.timing, elapsed_ms=elapsed_ms)


# ---------------------------------------------------------------------------
# Subcommand: hide
# ---------------------------------------------------------------------------

def cmd_hide(args: argparse.Namespace) -> None:
    type_ids = _resolve_types(args.type)
    fill_color = _resolve_fill_color(args.method, args.fill_color)

    t0 = time.time()
    if args.dir:
        if args.output:
            _die(
                "invalid_output_usage",
                "hide --dir does not support --output; use --output-dir instead.",
            )
        output_dir = args.output_dir or str(Path(args.dir) / ".has" / "masked")
        image_paths, skipped = _collect_images(args.dir)
        batch_result = run_hide_batch(
            image_paths,
            output_dir,
            args.model,
            args.conf,
            type_ids,
            args.method,
            args.strength,
            fill_color,
            skipped,
        )
        elapsed_ms = round((time.time() - t0) * 1000)
        _output("hide", batch_result, timing=args.timing, elapsed_ms=elapsed_ms)
    else:
        if args.output_dir:
            _die(
                "invalid_output_usage",
                "hide --image does not support --output-dir; use --output instead.",
            )
        # Single image mode
        result = _run_hide(
            args.image, args.output, args.model,
            args.conf, type_ids, args.method,
            args.strength, fill_color,
        )
        elapsed_ms = round((time.time() - t0) * 1000)
        _output("hide", result, timing=args.timing, elapsed_ms=elapsed_ms)


# ---------------------------------------------------------------------------
# Subcommand: categories
# ---------------------------------------------------------------------------

def cmd_categories(args: argparse.Namespace) -> None:
    if args.model is not None:
        _die(
            "invalid_model_usage",
            "categories does not support --model because it does not load the detection model.",
        )
    t0 = time.time()
    elapsed_ms = round((time.time() - t0) * 1000)
    _output("categories", {"categories": CATEGORIES}, timing=args.timing, elapsed_ms=elapsed_ms)


# ---------------------------------------------------------------------------
# Output
# ---------------------------------------------------------------------------

def _output(
    command: str,
    data: dict[str, Any],
    *,
    timing: bool = False,
    elapsed_ms: int | None = None,
) -> None:
    payload: dict[str, Any] = {
        "schema_version": SCHEMA_VERSION,
        "command": command,
    }
    payload.update(data)
    if timing and elapsed_ms is not None:
        payload["elapsed_ms"] = elapsed_ms
    _emit_json(payload)


# ---------------------------------------------------------------------------
# Argument parser
# ---------------------------------------------------------------------------

def build_parser() -> argparse.ArgumentParser:
    prog_name = os.environ.get("HAS_CLI_PROG", "has-image")
    shared_options = argparse.ArgumentParser(add_help=False)
    shared_options.add_argument(
        "--timing",
        action="store_true",
        default=argparse.SUPPRESS,
        help="Include elapsed_ms in the JSON output",
    )
    shared_options.add_argument(
        "--verbose",
        action="store_true",
        default=argparse.SUPPRESS,
        help="Emit runtime status and progress messages to stderr",
    )
    model_option = argparse.ArgumentParser(add_help=False)
    model_option.add_argument(
        "--model",
        default=argparse.SUPPRESS,
        help=f"Model file path (env: HAS_IMAGE_MODEL, default: {_DEFAULT_MODEL_PATH})",
    )
    parser = StructuredArgumentParser(
        parents=[shared_options, model_option],
        prog=prog_name,
        description="HaS Image — Privacy anonymization for images (YOLO11)",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog=(
            "Examples:\n"
            f"  {prog_name} scan --image photo.jpg --type face --type id_card\n"
            f"  {prog_name} hide --image photo.jpg --method mosaic --strength 20\n"
            f"  {prog_name} hide --dir ./photos/ --output-dir .has/masked/ --type face\n"
            f"  {prog_name} categories\n"
        ),
    )

    subparsers = parser.add_subparsers(
        dest="command",
        help="Available commands",
        parser_class=StructuredArgumentParser,
    )

    # --- scan ---
    scan_parser = subparsers.add_parser(
        "scan",
        parents=[shared_options, model_option],
        help="Scan image for privacy regions (no masking)",
        formatter_class=argparse.RawDescriptionHelpFormatter,
    )
    scan_img_group = scan_parser.add_mutually_exclusive_group(required=True)
    scan_img_group.add_argument("--image", help="Input image path")
    scan_img_group.add_argument("--dir", help="Input directory for batch scanning")
    scan_parser.add_argument(
        "--type", action="append", default=None,
        help="Category filter; repeat the flag to add more categories. Default: all",
    )
    scan_parser.add_argument(
        "--conf", type=float, default=0.25,
        help="Confidence threshold (default: 0.25)",
    )
    scan_parser.set_defaults(func=cmd_scan)

    # --- hide ---
    hide_parser = subparsers.add_parser(
        "hide",
        parents=[shared_options, model_option],
        help="Detect and mask privacy regions",
        formatter_class=argparse.RawDescriptionHelpFormatter,
    )
    img_group = hide_parser.add_mutually_exclusive_group(required=True)
    img_group.add_argument("--image", help="Input image path")
    img_group.add_argument("--dir", help="Input directory for batch processing")
    hide_parser.add_argument("--output", default=None, help="Output image path")
    hide_parser.add_argument("--output-dir", default=None, help="Output directory (batch mode)")
    hide_parser.add_argument(
        "--type", action="append", default=None,
        help="Category filter; repeat the flag to add more categories. Default: all",
    )
    hide_parser.add_argument(
        "--method", choices=["mosaic", "blur", "fill"], default="mosaic",
        help="Masking method (default: mosaic)",
    )
    hide_parser.add_argument(
        "--strength", type=int, default=15,
        help="Mosaic block size or blur radius (default: 15)",
    )
    hide_parser.add_argument(
        "--fill-color", default="#000000",
        help="Fill color for 'fill' method, hex format (default: #000000)",
    )
    hide_parser.add_argument(
        "--conf", type=float, default=0.25,
        help="Confidence threshold (default: 0.25)",
    )
    hide_parser.set_defaults(func=cmd_hide)

    # --- categories ---
    cat_parser = subparsers.add_parser(
        "categories",
        parents=[shared_options],
        help="List all supported privacy categories",
    )
    cat_parser.set_defaults(func=cmd_categories)

    return parser


def main() -> None:
    parser = build_parser()
    args: argparse.Namespace | None = None
    try:
        args = parser.parse_args()
        args.timing = getattr(args, "timing", False)
        args.verbose = getattr(args, "verbose", False)
        args.model = getattr(args, "model", None)
        if not args.command:
            raise CLIError("missing_command", "Choose a subcommand: scan, hide, or categories.")
        if args.verbose:
            os.environ["HAS_IMAGE_VERBOSE"] = "1"
        else:
            os.environ.pop("HAS_IMAGE_VERBOSE", None)
        args.func(args)
    except CLIError as exc:
        _emit_json(_error_payload(exc.code, exc.message))
        sys.exit(1)
    except (OSError, RuntimeError, ValueError) as exc:
        _emit_json(_error_payload("runtime_error", str(exc)))
        sys.exit(1)


if __name__ == "__main__":
    main()


path: scripts/has_text/__init__.py
content: """HaS (Hide and Seek) — Text anonymization and restoration CLI."""


path: scripts/has_text/__main__.py
content: #!/usr/bin/env python3
"""Allow running as `python -m has_text`."""

from .has_text import main

main()


path: scripts/has_text/chunker.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Text chunking with sentence-boundary awareness.

Splits long text into chunks that fit within the model's context window,
ensuring cuts happen at sentence boundaries.
"""

from __future__ import annotations

import re
from dataclasses import dataclass
from typing import Callable, List, Optional

# Default max tokens per chunk (conservative, leaves room for prompt overhead + output)
DEFAULT_MAX_CHUNK_TOKENS = 3000

# Sentence boundary pattern: split on Chinese/English sentence-ending punctuation
# Priority: paragraph break > period/exclamation/question > semicolon > comma
_SENTENCE_BOUNDARIES = re.compile(
    r"(?<=[\n])"           # newline (paragraph break)
    r"|(?<=[。！？!?])"    # Chinese/English sentence enders
    r"|(?<=[；;])"         # semicolons
)

# Fallback: split on any punctuation including commas
_FALLBACK_BOUNDARIES = re.compile(
    r"(?<=[，,。！？!?；;：:\n])"
)


@dataclass
class TextChunk:
    """A chunk of text with its position metadata."""
    text: str
    index: int          # 0-based chunk index
    start_char: int     # character offset in original text
    end_char: int       # character offset end (exclusive)
    token_count: int    # estimated token count


def take_chunk(
    text: str,
    count_tokens: Callable[[str], int],
    max_tokens: int = DEFAULT_MAX_CHUNK_TOKENS,
    *,
    index: int = 0,
    start_char: int = 0,
) -> Optional[TextChunk]:
    """Return the next chunk from text that fits within max_tokens."""
    if not text:
        return None

    total_tokens = count_tokens(text)
    if total_tokens <= max_tokens:
        return TextChunk(
            text=text,
            index=index,
            start_char=start_char,
            end_char=start_char + len(text),
            token_count=total_tokens,
        )

    # Estimate character position for max_tokens
    # Use ratio: chars_per_token ≈ len(text) / total_tokens
    chars_per_token = len(text) / total_tokens
    estimated_chars = int(max_tokens * chars_per_token)
    # Add a small buffer to search around the boundary
    search_end = min(estimated_chars + 50, len(text))
    search_text = text[:search_end]

    # Find the best split point
    split_pos = _find_split_point(search_text, count_tokens, max_tokens)

    if split_pos <= 0:
        # Emergency: hard cut at estimated position
        split_pos = max(1, estimated_chars)

    chunk_text_str = text[:split_pos]
    chunk_tokens = count_tokens(chunk_text_str)

    return TextChunk(
        text=chunk_text_str,
        index=index,
        start_char=start_char,
        end_char=start_char + split_pos,
        token_count=chunk_tokens,
    )


def chunk_text(
    text: str,
    count_tokens: Callable[[str], int],
    max_tokens: int = DEFAULT_MAX_CHUNK_TOKENS,
) -> List[TextChunk]:
    """Split text into chunks that fit within max_tokens.

    Strategy:
    1. If the entire text fits, return as single chunk.
    2. Otherwise, find the last sentence boundary before max_tokens.
    3. If no sentence boundary found, fall back to punctuation boundary.
    4. If still no boundary, hard-cut at max_tokens worth of characters.

    Args:
        text: The text to chunk.
        count_tokens: Function that returns token count for a string.
        max_tokens: Maximum tokens per chunk.

    Returns:
        List of TextChunk objects.
    """
    if not text or not text.strip():
        return []

    chunks: List[TextChunk] = []
    remaining = text
    offset = 0
    chunk_idx = 0

    while remaining:
        chunk = take_chunk(
            remaining,
            count_tokens,
            max_tokens,
            index=chunk_idx,
            start_char=offset,
        )
        if chunk is None:
            break

        chunks.append(chunk)
        offset = chunk.end_char
        remaining = remaining[len(chunk.text):]
        chunk_idx += 1

    return chunks


def _find_split_point(
    text: str,
    count_tokens: Callable[[str], int],
    max_tokens: int,
) -> int:
    """Find the best split point in text that stays within max_tokens.

    Tries sentence boundaries first, then fallback punctuation.

    Returns:
        Character position to split at, or 0 if no good split found.
    """
    # Find all sentence boundaries
    boundaries = [m.start() for m in _SENTENCE_BOUNDARIES.finditer(text)]

    # Try sentence boundaries (from right to left)
    best = _try_boundaries(boundaries, text, count_tokens, max_tokens)
    if best > 0:
        return best

    # Fallback: try punctuation boundaries
    boundaries = [m.start() for m in _FALLBACK_BOUNDARIES.finditer(text)]
    best = _try_boundaries(boundaries, text, count_tokens, max_tokens)
    if best > 0:
        return best

    return 0


def _try_boundaries(
    boundaries: List[int],
    text: str,
    count_tokens: Callable[[str], int],
    max_tokens: int,
) -> int:
    """Try split points from rightmost boundary, return first that fits."""
    for pos in reversed(boundaries):
        if pos <= 0:
            continue
        candidate = text[:pos]
        if count_tokens(candidate) <= max_tokens:
            return pos
    return 0


path: scripts/has_text/cli_utils.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Shared CLI utilities for has_text commands.

Contains argument validation, I/O helpers, error handling, and output
formatting used by all three subcommands (hide, restore, scan).
"""

from __future__ import annotations

import argparse
from dataclasses import dataclass
import json
import os
from pathlib import Path
import sys
import time
from typing import Any, Dict, List, Optional, Tuple

from .mapping import has_tags, load_mapping, save_mapping
from .parallel import DEFAULT_MAX_PARALLEL_REQUESTS, resolve_parallel_workers


# Rough token estimate used only for pre-server slot planning.
# Actual chunking always uses the real tokenizer via HaSClient.count_tokens.
def estimate_tokens_for_planning(text: str) -> int:
    """Conservative character-based estimate, used only before the server starts."""
    if not text:
        return 0
    # ~0.5 tokens per character is conservative for mixed CJK/Latin text.
    return max(1, int(len(text) * 0.5))


# Fallback context size when the default 8K is too small (mapping overflow).
FALLBACK_CONTEXT_PER_SLOT = 16384

SCHEMA_VERSION = "1"


@dataclass(frozen=True)
class CLIError(Exception):
    code: str
    message: str

    def __str__(self) -> str:
        return self.message


class StructuredArgumentParser(argparse.ArgumentParser):
    """ArgumentParser that reports machine-readable errors."""

    def error(self, message: str) -> None:
        raise CLIError("invalid_arguments", message)


def emit_json(payload: dict[str, Any], *, stream=None) -> None:
    target = stream or sys.stdout
    print(json.dumps(payload, ensure_ascii=False, separators=(",", ":")), file=target)


def error_payload(code: str, message: str) -> dict[str, Any]:
    return {
        "schema_version": SCHEMA_VERSION,
        "error": {
            "code": code,
            "message": message,
        },
    }


def fatal(code: str, message: str) -> None:
    raise CLIError(code, message)


def absolute_path(path: Path | str) -> str:
    """Return a stable absolute path string without resolving symlink targets."""
    return str(Path(path).expanduser().absolute())


def read_text(args: argparse.Namespace) -> str:
    """Read input text from --text, --file, or stdin."""
    if hasattr(args, "text") and args.text:
        return args.text
    if hasattr(args, "file") and args.file:
        with open(args.file, "r", encoding="utf-8") as f:
            return f.read()
    # Try stdin
    if not sys.stdin.isatty():
        return sys.stdin.read()
    fatal("missing_input", "Provide exactly one input source via --text, --file, --dir, or stdin.")


def parse_types(type_values: Optional[List[str]]) -> List[str]:
    """Parse repeated --type flags into a normalized list."""
    if not type_values:
        fatal("missing_type", "Provide at least one --type value.")

    parsed: List[str] = []
    for raw_value in type_values:
        value = str(raw_value).strip()
        if not value:
            fatal("invalid_type", "--type values must be non-empty strings.")
        if value not in parsed:
            parsed.append(value)
    return parsed


def load_mapping_file(path: str) -> Dict[str, List[str]]:
    mapping_path = Path(path)
    if not mapping_path.is_file():
        fatal("mapping_not_found", f"Mapping file not found: {path}")
    return load_mapping(str(mapping_path))


def output(
    command: str,
    result: dict[str, Any],
    *,
    timing: bool = False,
    elapsed_ms: Optional[int] = None,
) -> None:
    payload: dict[str, Any] = {
        "schema_version": SCHEMA_VERSION,
        "command": command,
    }
    payload.update(result)
    if timing and elapsed_ms is not None:
        payload["elapsed_ms"] = elapsed_ms
    emit_json(payload)


def parallel_default() -> int:
    """Read the shared parallel request budget from the environment."""
    raw = os.environ.get("HAS_TEXT_MAX_PARALLEL_REQUESTS")
    if raw is None:
        return DEFAULT_MAX_PARALLEL_REQUESTS

    try:
        value = int(raw)
    except ValueError:
        fatal("invalid_env", "HAS_TEXT_MAX_PARALLEL_REQUESTS must be an integer >= 1")

    if value < 1:
        fatal("invalid_env", "HAS_TEXT_MAX_PARALLEL_REQUESTS must be >= 1")

    return value


def required_slots(total_requests: int, max_parallel_requests: int) -> int:
    """Clamp server slots to the actual model work for this command."""
    if total_requests <= 0:
        return 0
    return resolve_parallel_workers(total_requests, max_parallel_requests)


def read_utf8_text_file(path: Path) -> Tuple[Optional[str], Optional[str]]:
    """Read a plaintext file, rejecting binary and non-UTF-8 content."""
    try:
        with path.open("rb") as fh:
            sample = fh.read(8192)
    except OSError as exc:
        return None, str(exc)

    if b"\x00" in sample:
        return None, "binary"

    try:
        return path.read_text(encoding="utf-8"), None
    except UnicodeDecodeError:
        return None, "non_utf8"
    except OSError as exc:
        return None, str(exc)


def is_within_directory(path: Path, directory: Path) -> bool:
    """Return whether `path` resolves within `directory`."""
    try:
        path.relative_to(directory)
        return True
    except ValueError:
        return False


def collect_text_documents(dir_path: str) -> Tuple[List[Dict[str, str]], List[Dict[str, str]]]:
    """Collect immediate plaintext files from a directory."""
    directory = Path(dir_path)
    if not directory.is_dir():
        fatal("invalid_directory", f"Not a directory: {dir_path}")
    directory_root = directory.resolve()

    documents: List[Dict[str, str]] = []
    skipped: List[Dict[str, str]] = []

    for entry in sorted(directory.iterdir()):
        if not entry.is_file():
            continue
        try:
            resolved_entry = entry.resolve()
        except OSError as exc:
            skipped.append({"file": absolute_path(entry), "reason": str(exc)})
            continue
        if not is_within_directory(resolved_entry, directory_root):
            skipped.append({"file": absolute_path(entry), "reason": "symlink_escape"})
            continue
        text, skip_reason = read_utf8_text_file(entry)
        if text is None:
            skipped.append({"file": absolute_path(entry), "reason": skip_reason or "unreadable"})
            continue
        documents.append({"file": absolute_path(entry), "text": text})

    return documents, skipped


def write_text_output(path: Path, text: str) -> None:
    """Write UTF-8 text with restrictive permissions (0600).

    Both anonymized and restored text may contain sensitive information.
    Match the restrictive permissions used by save_mapping.
    """
    import os as _os

    path.parent.mkdir(parents=True, exist_ok=True)
    fd = _os.open(str(path), _os.O_WRONLY | _os.O_CREAT | _os.O_TRUNC, 0o600)
    try:
        if hasattr(_os, "fchmod"):
            _os.fchmod(fd, 0o600)
        else:
            _os.chmod(str(path), 0o600)
        f = _os.fdopen(fd, "w", encoding="utf-8")
    except Exception:
        _os.close(fd)
        raise
    with f:
        f.write(text)


def resolve_single_output_path(
    path_value: Optional[str],
    *,
    input_path: Optional[Path] = None,
    input_label: str = "input file",
) -> Optional[Path]:
    """Resolve a single-file output path and refuse obvious clobbers."""
    if not path_value:
        return None

    output_path = Path(path_value)
    if input_path is not None and output_path.resolve(strict=False) == input_path.resolve(strict=False):
        fatal("refusing_overwrite", f"Refusing to overwrite {input_label}: {input_path}")
    return output_path


def default_seek_mapping_dir(dir_path: str) -> Path:
    """Return the default per-file mapping directory for batch restore."""
    return Path(dir_path) / "mappings"


def default_restore_output_dir(dir_path: str) -> Path:
    """Return the default output directory for batch restore.

    If the input directory sits directly under a `.has/` parent
    (e.g. `docs/.has/anonymized/`), place `restored/` as a sibling
    under the same `.has/` tree → `docs/.has/restored/`.
    Otherwise fall back to `<input-dir>/.has/restored/`.
    """
    d = Path(dir_path)
    if d.parent.name == ".has":
        return d.parent / "restored"
    return d / ".has" / "restored"


def seek_mapping_path(mapping_dir: Path, source_path: Path) -> Path:
    """Return the per-file mapping path generated by `hide --dir`."""
    return mapping_dir / f"{source_path.name}.mapping.json"


path: scripts/has_text/client.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""HTTP client for llama-server (OpenAI-compatible API)."""

from __future__ import annotations

from typing import Dict, List

DEFAULT_SERVER = "http://127.0.0.1:8080"


class ContextOverflowError(RuntimeError):
    """Raised when the request exceeds or fills the model's context window.

    Covers both cases:
    - Prompt alone exceeds context (HTTP 400, exceed_context_size_error)
    - Prompt fits but output is truncated (finish_reason: "length")
    """

    def __init__(self, message: str, *, prompt_tokens: int = 0, ctx_size: int = 0):
        super().__init__(message)
        self.prompt_tokens = prompt_tokens
        self.ctx_size = ctx_size


def _load_requests():
    """Import requests lazily so tests can import command modules without it."""
    try:
        import requests
    except ModuleNotFoundError as exc:
        raise RuntimeError(
            "The 'requests' package is required to talk to llama-server. "
            "Run via has-text/has_text_entry.py or install requests first."
        ) from exc
    return requests


class HaSClient:
    """Thin wrapper around llama-server's OpenAI-compatible API."""

    def __init__(self, server_url: str = DEFAULT_SERVER):
        self.base_url = server_url.rstrip("/")
        self._requests = _load_requests()
        self._session = self._requests.Session()

    # ------------------------------------------------------------------
    # Chat completions
    # ------------------------------------------------------------------

    def chat(self, messages: List[Dict[str, str]]) -> str:
        """Send a chat completion request and return the assistant reply.

        Args:
            messages: List of {"role": "user"|"assistant", "content": "..."}

        Returns:
            The model's response text.

        Raises:
            RuntimeError: If the server is unreachable or returns an error.
            ContextOverflowError: If the prompt exceeds context or output
                is truncated due to context exhaustion.
        """
        url = f"{self.base_url}/v1/chat/completions"
        payload = {"messages": messages}
        try:
            resp = self._session.post(url, json=payload, timeout=120)
        except self._requests.ConnectionError:
            raise RuntimeError(
                f"Cannot connect to llama-server at {self.base_url}\n"
                f"Please start llama-server first:\n"
                f"  llama-server -m has_text_model.gguf -ngl 999 -c 8192 -fa on -ctk q8_0 -ctv q8_0 --port 8080\n"
                f"For parallel scan/seek, add --parallel N (or -np N) and scale -c to 8192 * N "
                f"so each slot keeps the full 8K context budget."
            )

        # Detect prompt overflow (HTTP 400 with exceed_context_size_error)
        if resp.status_code == 400:
            try:
                err = resp.json().get("error", {})
            except Exception:
                err = {}
            if err.get("type") == "exceed_context_size_error":
                raise ContextOverflowError(
                    f"Prompt ({err.get('n_prompt_tokens', '?')} tokens) exceeds "
                    f"context window ({err.get('n_ctx', '?')} tokens)",
                    prompt_tokens=err.get("n_prompt_tokens", 0),
                    ctx_size=err.get("n_ctx", 0),
                )

        try:
            resp.raise_for_status()
        except self._requests.HTTPError as e:
            raise RuntimeError(
                f"llama-server returned {e.response.status_code}: {e.response.text}"
            ) from e

        data = resp.json()
        choice = data["choices"][0]

        # Detect output truncation (finish_reason: "length")
        if choice.get("finish_reason") == "length":
            usage = data.get("usage", {})
            raise ContextOverflowError(
                f"Output truncated: context full "
                f"(prompt {usage.get('prompt_tokens', '?')} + "
                f"completion {usage.get('completion_tokens', '?')} = "
                f"{usage.get('total_tokens', '?')} tokens)",
                prompt_tokens=usage.get("prompt_tokens", 0),
                ctx_size=usage.get("total_tokens", 0),
            )

        return choice["message"]["content"]

    # ------------------------------------------------------------------
    # Tokenize (for chunking)
    # ------------------------------------------------------------------

    def tokenize(self, text: str) -> List[int]:
        """Tokenize text using llama-server's /tokenize endpoint.

        Args:
            text: Text to tokenize.

        Returns:
            List of token IDs.
        """
        url = f"{self.base_url}/tokenize"
        try:
            resp = self._session.post(url, json={"content": text}, timeout=30)
            resp.raise_for_status()
        except self._requests.ConnectionError:
            raise RuntimeError(
                f"Cannot connect to llama-server at {self.base_url} for tokenization.\n"
                f"Token counting requires a running llama-server."
            )
        except self._requests.HTTPError as e:
            raise RuntimeError(
                f"llama-server tokenize returned {e.response.status_code}: {e.response.text}"
            ) from e

        data = resp.json()
        return data.get("tokens", [])

    def count_tokens(self, text: str) -> int:
        """Count the number of tokens in a text string.

        Args:
            text: Text to count tokens for.

        Returns:
            Number of tokens.
        """
        return len(self.tokenize(text))

    # ------------------------------------------------------------------
    # Health check
    # ------------------------------------------------------------------

    def health(self) -> bool:
        """Check if llama-server is reachable."""
        try:
            resp = self._session.get(f"{self.base_url}/health", timeout=5)
            return resp.status_code == 200
        except Exception:
            return False


path: scripts/has_text/commands/scan.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""scan command — Identify sensitive entities in text (NER only, no anonymization)."""

from __future__ import annotations

import argparse
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from dataclasses import dataclass
from typing import Any, Callable, Dict, List, Optional

from ..chunker import chunk_text
from ..client import HaSClient
from ..mapping import parse_json_tolerant
from ..parallel import DEFAULT_MAX_PARALLEL_REQUESTS, resolve_parallel_workers
from ..prompts import build_ner_messages
from ..validation import normalize_entity_map

# scan only produces a small JSON entity list (no full-text rewrite),
# so chunks can be larger than the 3000 default used by hide/seek.
DEFAULT_SCAN_MAX_CHUNK_TOKENS = 5000


@dataclass(frozen=True)
class ScanDocument:
    """Plaintext input for batch scanning."""

    source: str
    text: str


@dataclass(frozen=True)
class _ScanTask:
    document_index: int
    chunk_index: int
    text: str


def _scan_single_chunk(
    client: HaSClient,
    chunk_text_str: str,
    types: List[str],
    chunk_index: int,
) -> Dict[str, List[str]]:
    """Run NER for a single chunk."""
    messages = build_ner_messages(chunk_text_str, types)
    raw_output = client.chat(messages)
    return normalize_entity_map(
        parse_json_tolerant(raw_output),
        context=f"NER output for chunk {chunk_index + 1}",
    )


def _clone_scan_client(client: HaSClient) -> HaSClient:
    """Create a fresh client per worker to avoid sharing HTTP sessions."""
    client_cls = client.__class__
    try:
        return client_cls(client.base_url)
    except TypeError:
        return HaSClient(client.base_url)


def _merge_entities(chunk_results: List[Dict[str, List[str]]]) -> Dict[str, List[str]]:
    """Merge chunk results while preserving the first-seen order."""
    merged_entities: Dict[str, List[str]] = {}

    for ner_result in chunk_results:
        for entity_type, entities in ner_result.items():
            if entity_type not in merged_entities:
                merged_entities[entity_type] = []
            for entity in entities:
                entity_str = str(entity).strip()
                if entity_str and entity_str not in merged_entities[entity_type]:
                    merged_entities[entity_type].append(entity_str)

    return merged_entities


def estimate_scan_request_count(
    text: str,
    max_chunk_tokens: int = DEFAULT_SCAN_MAX_CHUNK_TOKENS,
    *,
    count_tokens: Callable[[str], int],
) -> int:
    """Estimate how many model requests scan will issue for one text."""
    if not text or not text.strip():
        return 0
    return len(chunk_text(text, count_tokens, max_chunk_tokens))


def estimate_scan_batch_request_count(
    documents: List[ScanDocument],
    max_chunk_tokens: int = DEFAULT_SCAN_MAX_CHUNK_TOKENS,
    *,
    count_tokens: Callable[[str], int],
) -> int:
    """Estimate total scan requests across a batch of documents."""
    return sum(
        estimate_scan_request_count(
            document.text,
            max_chunk_tokens=max_chunk_tokens,
            count_tokens=count_tokens,
        )
        for document in documents
    )


def run_scan(
    client: HaSClient,
    text: str,
    types: List[str],
    max_chunk_tokens: int = DEFAULT_SCAN_MAX_CHUNK_TOKENS,
    max_parallel_requests: int = DEFAULT_MAX_PARALLEL_REQUESTS,
) -> Dict[str, List[str]]:
    """Scan text for sensitive entities.

    For short text, runs a single NER call.
    For long text, chunks and merges NER results.

    Returns:
        {"entities": {"人名": ["张三", "李四"], "地址": ["北京"], ...}}
    """
    if max_parallel_requests < 1:
        raise ValueError("max_parallel_requests must be >= 1")

    chunks = chunk_text(text, client.count_tokens, max_chunk_tokens)
    if not chunks:
        return {"entities": {}}

    if len(chunks) == 1 or max_parallel_requests == 1:
        chunk_results = [
            _scan_single_chunk(client, chunk.text, types, chunk.index)
            for chunk in chunks
        ]
        return {"entities": _merge_entities(chunk_results)}

    max_workers = resolve_parallel_workers(len(chunks), max_parallel_requests)
    chunk_results: List[Optional[Dict[str, List[str]]]] = [None] * len(chunks)

    with ThreadPoolExecutor(max_workers=max_workers) as executor:
        futures = {
            executor.submit(
                _scan_single_chunk,
                _clone_scan_client(client),
                chunk.text,
                types,
                chunk.index,
            ): chunk.index
            for chunk in chunks
        }

        try:
            for future in as_completed(futures):
                chunk_index = futures[future]
                chunk_results[chunk_index] = future.result()
        except BaseException:
            for future in futures:
                future.cancel()
            raise

    # Merge in original chunk order so first-seen semantics remain stable.
    return {"entities": _merge_entities([result for result in chunk_results if result is not None])}


def run_scan_batch(
    client: HaSClient,
    documents: List[ScanDocument],
    types: List[str],
    max_chunk_tokens: int = DEFAULT_SCAN_MAX_CHUNK_TOKENS,
    max_parallel_requests: int = DEFAULT_MAX_PARALLEL_REQUESTS,
) -> Dict[str, Any]:
    """Scan multiple plaintext documents with a shared global request pool."""
    if max_parallel_requests < 1:
        raise ValueError("max_parallel_requests must be >= 1")

    if not documents:
        return {"results": [], "count": 0, "summary": {}}

    chunk_results: List[List[Optional[Dict[str, List[str]]]]] = []
    tasks: List[_ScanTask] = []

    for document_index, document in enumerate(documents):
        chunks = chunk_text(document.text, client.count_tokens, max_chunk_tokens)
        doc_results: List[Optional[Dict[str, List[str]]]] = [None] * len(chunks)
        chunk_results.append(doc_results)
        for chunk in chunks:
            tasks.append(
                _ScanTask(
                    document_index=document_index,
                    chunk_index=chunk.index,
                    text=chunk.text,
                )
            )

    if len(tasks) == 1 or max_parallel_requests == 1:
        for task in tasks:
            chunk_results[task.document_index][task.chunk_index] = _scan_single_chunk(
                client,
                task.text,
                types,
                task.chunk_index,
            )
    elif tasks:
        max_workers = resolve_parallel_workers(len(tasks), max_parallel_requests)
        with ThreadPoolExecutor(max_workers=max_workers) as executor:
            futures = {
                executor.submit(
                    _scan_single_chunk,
                    _clone_scan_client(client),
                    task.text,
                    types,
                    task.chunk_index,
                ): task
                for task in tasks
            }

            try:
                for future in as_completed(futures):
                    task = futures[future]
                    chunk_results[task.document_index][task.chunk_index] = future.result()
            except BaseException:
                for future in futures:
                    future.cancel()
                raise

    results: List[Dict[str, Any]] = []
    summary: Dict[str, int] = {}

    for document_index, document in enumerate(documents):
        entities = _merge_entities(
            [result for result in chunk_results[document_index] if result is not None]
        )
        results.append({"file": document.source, "entities": entities})
        for entity_type, values in entities.items():
            summary[entity_type] = summary.get(entity_type, 0) + len(values)

    return {
        "results": results,
        "count": len(results),
        "summary": summary,
    }


# ======================================================================
# CLI handler
# ======================================================================


def cmd_scan(args: argparse.Namespace) -> None:
    """Execute the scan (NER) command."""
    from ..cli_utils import (
        collect_text_documents,
        estimate_tokens_for_planning,
        output,
        parse_types,
        read_text,
        required_slots,
    )
    from ..client import DEFAULT_SERVER
    from ..server_runtime import acquire_server

    types = parse_types(args.type)
    dir_path = getattr(args, "dir", None)

    t0 = time.time()
    if dir_path:
        raw_documents, skipped = collect_text_documents(dir_path)
        documents = [
            ScanDocument(source=item["file"], text=item["text"])
            for item in raw_documents
        ]
        request_count = estimate_scan_batch_request_count(
            documents,
            max_chunk_tokens=args.max_chunk_tokens,
            count_tokens=estimate_tokens_for_planning,
        )

        if documents and request_count > 0:
            with acquire_server(
                DEFAULT_SERVER,
                required_slots=required_slots(request_count, args.max_parallel_requests),
            ) as lease:
                client = lease.create_client()
                result = run_scan_batch(
                    client,
                    documents,
                    types,
                    max_chunk_tokens=args.max_chunk_tokens,
                    max_parallel_requests=args.max_parallel_requests,
                )
        elif documents:
            result = {
                "results": [
                    {"file": document.source, "entities": {}}
                    for document in documents
                ],
                "count": len(documents),
                "summary": {},
            }
        else:
            result = {"results": [], "count": 0, "summary": {}}

        if skipped:
            result["skipped"] = skipped
            result["skipped_count"] = len(skipped)
    else:
        text = read_text(args)
        request_count = estimate_scan_request_count(
            text,
            max_chunk_tokens=args.max_chunk_tokens,
            count_tokens=estimate_tokens_for_planning,
        )
        if request_count > 0:
            with acquire_server(
                DEFAULT_SERVER,
                required_slots=required_slots(request_count, args.max_parallel_requests),
            ) as lease:
                client = lease.create_client()
                result = run_scan(
                    client,
                    text,
                    types,
                    max_chunk_tokens=args.max_chunk_tokens,
                    max_parallel_requests=args.max_parallel_requests,
                )
        else:
            result = {"entities": {}}
    elapsed_ms = round((time.time() - t0) * 1000)

    output("scan", result, timing=args.timing, elapsed_ms=elapsed_ms)



path: scripts/has_text/lang.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Language detection for determining if two texts are in the same language.

Self-contained implementation using Unicode script heuristics.
No external dependencies (langid, etc.).
"""

from __future__ import annotations

import re
from typing import Dict, Optional, Tuple

# Strip anonymized tags before detecting language
_TAG_RE = re.compile(r"<[^<>\[\]]+\[\d+\]\.[^<>\[\].]+\.[^<>\[\].]+>")


def strip_tags(text: str) -> str:
    """Remove anonymized tags from text."""
    return _TAG_RE.sub("", text or "")


def _script_counts(text: str) -> Dict[str, int]:
    """Count characters by Unicode script."""
    counts = {
        "latin": 0,
        "han": 0,
        "hiragana": 0,
        "katakana": 0,
        "hangul": 0,
        "cyrillic": 0,
        "arabic": 0,
        "devanagari": 0,
    }
    for ch in text:
        code = ord(ch)
        if 65 <= code <= 90 or 97 <= code <= 122:
            counts["latin"] += 1
        elif 0x4E00 <= code <= 0x9FFF:
            counts["han"] += 1
        elif 0x3040 <= code <= 0x309F:
            counts["hiragana"] += 1
        elif 0x30A0 <= code <= 0x30FF:
            counts["katakana"] += 1
        elif 0xAC00 <= code <= 0xD7AF:
            counts["hangul"] += 1
        elif 0x0400 <= code <= 0x04FF:
            counts["cyrillic"] += 1
        elif 0x0600 <= code <= 0x06FF:
            counts["arabic"] += 1
        elif 0x0900 <= code <= 0x097F:
            counts["devanagari"] += 1
    return counts


def detect_script_lang(text: str) -> Tuple[Optional[str], float]:
    """Detect primary language of text using script heuristics.

    Returns:
        (language_code, confidence)
        Language codes: "zh", "ja", "ko", "en", "ru", "ar", "hi", or None
    """
    cleaned = strip_tags(text).strip()
    if not cleaned:
        return None, 0.0

    counts = _script_counts(cleaned)
    total = sum(counts.values()) or 1
    ratios = {k: v / total for k, v in counts.items()}

    # Japanese: significant hiragana/katakana presence
    if ratios["hiragana"] + ratios["katakana"] > 0.15 and ratios["han"] < 0.6:
        return "ja", 0.8

    # Korean: significant hangul
    if ratios["hangul"] > 0.2:
        return "ko", 0.8

    # Chinese: significant han, low Japanese kana
    if ratios["han"] > 0.2 and ratios["hiragana"] + ratios["katakana"] < 0.1:
        return "zh", 0.7

    # Cyrillic-based (Russian, etc.)
    if ratios["cyrillic"] > 0.2:
        return "ru", 0.75

    # Arabic
    if ratios["arabic"] > 0.2:
        return "ar", 0.75

    # Devanagari (Hindi, etc.)
    if ratios["devanagari"] > 0.2:
        return "hi", 0.75

    # Latin-based (English, French, German, etc.)
    if ratios["latin"] > 0.4:
        return "en", 0.6

    return None, 0.0


# Instruction patterns that hint at target language
_HINT_PATTERNS = [
    re.compile(r"翻译(为|成)\s*([\w\u4e00-\u9fff]+)"),
    re.compile(r"请用\s*([\w\u4e00-\u9fff]+)\s*(回答|作答|输出)"),
    re.compile(r"(answer in|reply in|respond in)\s+([A-Za-z]+)", re.IGNORECASE),
    re.compile(r"translate .* to\s+([A-Za-z]+)", re.IGNORECASE),
]

_LANG_HINT_MAP: Dict[str, str] = {
    "中文": "zh", "汉语": "zh", "简体": "zh", "繁体": "zh",
    "Chinese": "zh", "Mandarin": "zh",
    "英文": "en", "英语": "en", "English": "en",
    "日文": "ja", "日语": "ja", "Japanese": "ja",
    "韩文": "ko", "韩语": "ko", "Korean": "ko",
    "法语": "fr", "French": "fr",
    "德语": "de", "German": "de",
    "俄语": "ru", "Russian": "ru",
    "西班牙语": "es", "Spanish": "es",
    "葡萄牙语": "pt", "Portuguese": "pt",
}


def _extract_target_lang_hint(text: str) -> Optional[str]:
    """Extract target language hint from translation instructions in text."""
    for pat in _HINT_PATTERNS:
        match = pat.search(text or "")
        if not match:
            continue
        [REDACTED_SECRET] or 1).strip()
        lang = _LANG_HINT_MAP.get(token) or _LANG_HINT_MAP.get(token.capitalize())
        if lang:
            return lang
    return None


def is_same_language(text_a: str, text_b: str) -> bool:
    """Determine if two texts are in the same language.

    Compares the dominant script/language of both texts.
    Also checks for translation instruction hints in text_a.

    Args:
        text_a: Original/source text (may contain translation instructions).
        text_b: Processed text to compare against.

    Returns:
        True if both texts appear to be in the same language.
    """
    lang_a, conf_a = detect_script_lang(text_a)
    lang_b, conf_b = detect_script_lang(text_b)

    # If either detection is low confidence, assume same language
    # (safer to use Tool-Seek, which has a self-check anyway)
    if lang_a is None or lang_b is None:
        return True

    if conf_a < 0.5 or conf_b < 0.5:
        return True

    # Check if text_a contains a translation instruction
    target_hint = _extract_target_lang_hint(text_a)
    if target_hint and target_hint != lang_a:
        # Source text says "translate to X" — expect text_b to be in X
        # So if text_b IS in X, it's a different language from source
        if lang_b == target_hint:
            return False

    return lang_a == lang_b


path: scripts/has_text/parallel.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Shared concurrency settings for model-backed has_text requests."""

from __future__ import annotations

DEFAULT_MAX_PARALLEL_REQUESTS = 4


def resolve_parallel_workers(total_units: int, max_parallel_requests: int) -> int:
    """Clamp worker count to the configured parallel request budget."""
    if max_parallel_requests < 1:
        raise ValueError("max_parallel_requests must be >= 1")
    return min(total_units, max_parallel_requests)


path: scripts/has_text/tests/__init__.py
content: # has_text test suite


path: scripts/has_text/validation.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Validation helpers for model outputs."""

from __future__ import annotations

from typing import Any, Dict, List


def normalize_entity_map(data: Any, *, context: str) -> Dict[str, List[str]]:
    """Validate and normalize NER output into ``Dict[str, List[str]]``."""

    if not isinstance(data, dict):
        raise RuntimeError(f"{context} did not return a JSON object")

    normalized: Dict[str, List[str]] = {}

    for raw_key, raw_values in data.items():
        key = str(raw_key).strip()
        if not key:
            raise RuntimeError(f"{context} returned an empty entity type")

        if isinstance(raw_values, str):
            values_source = [raw_values]
        elif isinstance(raw_values, list):
            values_source = raw_values
        else:
            raise RuntimeError(
                f"{context} returned an invalid value for entity type {key!r}"
            )

        values: List[str] = []
        for raw_value in values_source:
            value = str(raw_value).strip()
            if value and value not in values:
                values.append(value)

        normalized[key] = values

    return normalized


path: scripts/has_text_entry.py
content: #!/usr/bin/env python3
# /// script
# requires-python = ">=3.10"
# dependencies = [
#     "requests>=2.31.0",
# ]
# ///
"""Launcher for HaS Text CLI with uv-managed dependencies."""

from __future__ import annotations

import sys
from pathlib import Path

# Ensure the local has_text package under scripts/ is importable.
SCRIPT_DIR = Path(__file__).resolve().parent
if str(SCRIPT_DIR) not in sys.path:
    sys.path.insert(0, str(SCRIPT_DIR))

from has_text.has_text import main


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)