Skill: swaylq/humanize-chinese (version 2.4.0)

--- name: humanize-chinese description: > Detect and humanize AI-generated Chinese text. 20+ detection categories, weighted 0-100 scoring with sentence-level analysis, 7 style transforms (casual/zhihu/xiaohongshu/wechat/academic/literary/weibo), sentence restructuring, context-aware replacement. Academic paper AIGC reduction for CNKI/VIP/Wanfang (知网/维普/万方 AIGC 检测降重), 10 academic detection dimensions, 120+ scholarly expression replacements, hedging language injection. Pure Python, no dependencies. v2.1.0 Use when user says: "去AI味", "降AIGC", "人性化文本", "humanize chinese", "AI检测", "AIGC降重", "去除AI痕迹", "文本改写", "论文降重", "知网检测", "维普检测", "AI写作检测", "让文字更自然", "detect AI text", "humanize text", "reduce AIGC score", "make text human-like", "去ai化", "改成人话", "去机器味", "降低AI率", "过AIGC检测" allowed-tools: - Read - Write - Edit - exec --- # Humanize Chinese AI Text v2.1 检测和改写中文 AI 生成文本的完整工具链。可独立运行（CLI），也可作为 LLM prompt 指南使用。 ## CLI Tools 所有脚本在 `scripts/` 目录下，纯 Python，无依赖。 ```bash # 检测 AI 模式（20+ 维度，0-100 分） python scripts/detect_cn.py text.txt python scripts/detect_cn.py text.txt -v # 详细 + 最可疑句子 python scripts/detect_cn.py text.txt -s # 仅评分 python scripts/detect_cn.py text.txt -j # JSON 输出 # 改写 python scripts/humanize_cn.py text.txt -o clean.txt python scripts/humanize_cn.py text.txt --scene social -a # 社交场景 + 激进模式 python scripts/humanize_cn.py text.txt --style xiaohongshu # 先改写再转风格 # 风格转换 python scripts/style_cn.py text.txt --style zhihu -o out.txt # 前后对比 python scripts/compare_cn.py text.txt --scene tech -a # 学术论文 AIGC 降重 python scripts/academic_cn.py paper.txt -o clean.txt --compare python scripts/academic_cn.py paper.txt -o clean.txt -a --compare # 激进 ``` ### 评分标准 | 分数 | 等级 | 含义 | |------|------|------| | 0-24 | LOW | 基本像人写的 | | 25-49 | MEDIUM | 有些 AI 痕迹 | | 50-74 | HIGH | 大概率 AI 生成 | | 75-100 | VERY HIGH | 几乎确定是 AI | ### 参数速查 | 参数 | 说明 | |------|------| | `-v` | 详细模式，显示可疑句子 | | `-s` | 仅评分 | | `-j` | JSON 输出 | | `-o` | 输出文件 | | `-a` | 激进模式 | | `--seed N` | 固定随机种子 | | `--scene` | general / social / tech / formal / chat | | `--style` | casual / zhihu / xiaohongshu / wechat / academic / literary / weibo | | `--compare` | 前后对比 | ### 工作流 ```bash # 1. 检测 python scripts/detect_cn.py document.txt -v # 2. 改写 + 对比 python scripts/compare_cn.py document.txt -a -o clean.txt # 3. 验证 python scripts/detect_cn.py clean.txt -s # 4. 可选：转风格 python scripts/style_cn.py clean.txt --style zhihu -o final.txt ``` --- ## LLM 直接使用指南 当用户要求"去 AI 味"、"降 AIGC"、"人性化文本"、"改成人话"时，如果无法运行 CLI 工具，按以下流程手动处理。 ### 第一步：检测 AI 写作模式 扫描文本中的以下模式，按严重程度分类： #### 🔴 高危模式（一眼就能看出是 AI） **三段式套路：** - 首先…其次…最后 - 一方面…另一方面 - 第一…第二…第三 **机械连接词：** 值得注意的是、综上所述、不难发现、总而言之、与此同时、由此可见、不仅如此、换句话说、更重要的是、不可否认、显而易见、不言而喻、归根结底 **空洞宏大词：** 赋能、闭环、数字化转型、协同增效、降本增效、深度融合、全方位、多维度、系统性、高质量发展、新质生产力 #### 🟠 中危模式 **AI 高频词：** 助力、彰显、凸显、底层逻辑、抓手、触达、沉淀、复盘、迭代、破圈、颠覆 **填充废话：** 值得一提的是、众所周知、毫无疑问、具体来说、简而言之 **模板句式：** - 随着…的不断发展 - 在当今…时代 - 在…的背景下 - 作为…的重要组成部分 - 这不仅…更是… **平衡论述套话：** 虽然…但是…同时、既有…也有…更有 #### 🟡 低危模式 - 犹豫语过多（在一定程度上、某种程度上 出现 >5 次） - 列举成瘾（动辄①②③④⑤） - 标点滥用（大量分号、破折号） - 修辞堆砌（排比对偶过多） #### ⚪ 风格信号 - 段落长度高度一致 - 句子长度单调 - 情感表达平淡 - 开头方式重复 - 信息熵低（用词可预测） ### 第二步：改写策略 按以下顺序处理： **1. 砍掉三段式** 把"首先…其次…最后"打散，用自然过渡代替。不是每个论点都要编号。 **2. 替换 AI 套话** - 综上所述 → 总之 / 说到底 / （直接删掉） - 值得注意的是 → （直接删掉，后面的话自己能说清楚） - 赋能 → 帮助 / 支持 / 提升 - 数字化转型 → 信息化改造 / 技术升级 - 不难发现 → 可以看到 / （删掉） - 助力 → 帮 / 推动 **3. 句式重组** - 过短的句子合并（"他很累。他决定休息。" → "他累了，干脆歇会儿。"） - 过长的句子拆开（在"但是""不过""同时"等转折处断开） - 打破均匀节奏（长短句交替，不要每句差不多长） **4. 减少重复用词** 同一个词出现 3 次以上就换同义词。比如"进行"可以换成"做""搞""开展""着手"。 **5. 注入人味** - 加一两句口语化表达（场景允许的话） - 用具体的例子代替抽象概括 - 偶尔加个反问或感叹 - 不要每段都总分总结构 **6. 段落节奏** 打破每段差不多长的格局。有的段落 2 句话，有的 5 句话，像人写东西时自然的长短变化。 ### 第三步：学术论文特殊处理 当文本是学术论文时，改写规则不同——不能口语化，要保持学术严谨性： **学术专用检测维度：** 1. AI 学术措辞（"本文旨在""具有重要意义""进行了深入分析"） 2. 被动句式过度（"被广泛应用""被认为是"） 3. 段落结构过于整齐（每段总-分-总） 4. 连接词密度异常 5. 同义表达匮乏（"研究"出现 8 次） 6. 引用整合度低（每个引用都是"XX（2020）指出…"） 7. 数据论述模板化（"从表中可以看出"） 8. 过度列举（①②③④ 频繁出现） 9. 结论过于圆满（只说好不说局限） 10. 语气过于确定（"必然""毫无疑问"） **学术改写策略：** - **替换 AI 学术套话（保持学术性）：** - 本文旨在 → 本文尝试 / 本研究关注 - 具有重要意义 → 值得关注 / 有一定参考价值 - 研究表明 → 前人研究发现 / 已有文献显示 / 笔者观察到 - 进行了深入分析 → 做了初步探讨 / 展开了讨论 - 取得了显著成效 → 产生了一定效果 / 初见成效 - **减少被动句：** - 被广泛应用 → 得到较多运用 / 在多个领域有所应用 - 被认为是 → 通常被看作 / 一般认为 - **注入学术犹豫语（hedging）：** 在过于绝对的判断前加"可能""在一定程度上""就目前而言""初步来看" - **增强作者主体性：** - 研究表明 → 笔者认为 / 本研究发现 - 可以认为 → 笔者倾向于认为 - **补充局限性：** 如果结论段没有提到局限，补一句"当然，本研究也存在一定局限…" - **打破结构均匀度：** 调整段落长度，避免每段都一样。合并过短的段落，拆分过长的。 ### 第四步：验证 改写完成后，用 CLI 工具验证效果： ```bash python scripts/detect_cn.py output.txt -s ``` 目标：通用文本降到 25 分以下，学术论文降到 30 分以下。 --- ## 配置说明 所有检测模式和替换规则在 `scripts/patterns_cn.json`，可自定义： - 添加新 AI 词汇 - 调整权重 - 增加替换规则 - 修改正则匹配 ## 外部配置字段 ``` critical_patterns — 高权重检测（三段式、连接词、空洞词） high_signal_patterns — 中权重检测（AI 高频词、模板句） replacements — 替换词库（正则 + 纯文本） academic_patterns — 学术专用检测与替换 scoring — 权重和阈值配置 ```

path: README.md
content: # 🔧 中文 AI 文本去痕迹工具

**免费、本地运行、零依赖的中文 AI 文本检测与改写工具。**

[![GitHub stars](https://img.shields.io/github/stars/voidborne-d/humanize-chinese?style=flat-square)](https://github.com/voidborne-d/humanize-chinese)
[![ClawHub](https://img.shields.io/badge/clawhub-humanize--chinese-blue?style=flat-square)](https://clawhub.com/skills/humanize-chinese)
[![License: MIT](https://img.shields.io/badge/License-MIT-green?style=flat-square)](LICENSE)
[![Python](https://img.shields.io/badge/Python-3.6+-blue?style=flat-square)](https://python.org)

检测 20+ 种 AI 写作模式 + N-gram 困惑度统计分析，量化评分 0-100，支持 7 种风格转换，学术论文 AIGC 降重专用模式。纯 Python，不需要安装任何依赖包。

---

## Features at a Glance

| 功能 | 说明 |
|------|------|
| 🔍 AI 检测 | 20+ 规则维度 + N-gram 困惑度统计，加权 0-100 评分，精确到句子级 |
| 📈 统计分析 | 困惑度 / 突发度 / 段落熵均匀度，从概率分布层面检测 AI |
| ✏️ 智能改写 | 上下文感知替换，句式重组，节奏变化，**困惑度引导选词** |
| 🎓 学术降重 | 10 维度检测 + 120 条学术替换词库，针对知网/维普/万方 |
| 🎨 风格转换 | 知乎、小红书、公众号、微博、学术、文艺、口语 7 种 |
| 📊 前后对比 | 一键对比改写前后评分变化 |
| 🔄 可复现 | `--seed` 参数保证相同输入输出一致 |
| 📦 零依赖 | 纯 Python 标准库，下载即用 |

---

## Comparison / 对比

和目前最流行的中文去 AI 工具 [Humanizer-zh](https://github.com/op7418/Humanizer-zh) 的对比：

| 对比项 | humanize-chinese（本项目） | Humanizer-zh |
|--------|--------------------------|--------------|
| 运行方式 | ✅ 独立 CLI 工具，终端直接跑 | ❌ 纯 Skill prompt，必须在 Claude Code 内用 |
| 依赖 | ✅ 零依赖（纯 Python 标准库） | 需要 Claude Code 环境 |
| 量化评分 | ✅ 0-100 分，4 级评估 | ❌ 无评分，靠感觉判断 |
| 统计检测 | ✅ N-gram 困惑度 + 突发度 + 熵分析 | ❌ 无统计特征 |
| 句子级分析 | ✅ 定位最可疑的句子 | ❌ 只做整体改写 |
| 学术论文模式 | ✅ 专用 10 维度检测 + 120 条替换 | ❌ 无学术专用功能 |
| 风格转换 | ✅ 7 种中文写作风格 | ❌ 无 |
| 结果可复现 | ✅ `--seed` 固定随机数 | ❌ 每次结果不同 |
| 批量处理 | ✅ CLI 管道，脚本化 | ❌ 只能单篇交互 |
| 模式配置 | ✅ JSON 外部配置，可自定义 | ❌ 硬编码在 prompt 里 |
| 免费 | ✅ 完全免费 | ⚠️ 需要 Claude API 额度 |
| 也可作为 Skill | ✅ 支持 OpenClaw / Claude Code | ✅ 支持 Claude Code |

简单说：Humanizer-zh 是个很好的 prompt，但它只能在 Claude Code 里用，没法量化效果，也没有学术论文专用功能。我们这个是独立工具，可以在任何环境里跑，有评分有对比，学术场景也覆盖了。

---

## Install / 安装

**方式一：ClawHub（推荐）**

```bash
clawhub install humanize-chinese
```

**方式二：Git Clone**

```bash
git clone https://github.com/voidborne-d/humanize-chinese.git
cd humanize-chinese
```

**方式三：Claude Code Skill**

```bash
# 通过 npx 安装
npx skills add https://github.com/voidborne-d/humanize-chinese.git
```

**方式四：手动下载**

直接下载 `scripts/` 目录下的 Python 文件和 `patterns_cn.json`，放到同一个目录就能用。

---

## Quick Start

### 场景一：通用文本去 AI 味

```bash
# 先检测，看看 AI 味有多重
python scripts/detect_cn.py text.txt -v

# 改写 + 对比
python scripts/compare_cn.py text.txt -a -o clean.txt

# 验证效果
python scripts/detect_cn.py clean.txt -s
```

### 场景二：学术论文 AIGC 降重

```bash
# 检测学术 AI 痕迹
python scripts/academic_cn.py paper.txt

# 改写并对比（推荐）
python scripts/academic_cn.py paper.txt -o clean.txt --compare

# 激进模式（降得更狠）
python scripts/academic_cn.py paper.txt -o clean.txt -a --compare
```

### 场景三：风格转换

```bash
# 转小红书风格
python scripts/style_cn.py text.txt --style xiaohongshu

# 转知乎风格
python scripts/style_cn.py text.txt --style zhihu -o zhihu.txt

# 先去 AI 味再转风格
python scripts/humanize_cn.py text.txt --style xiaohongshu -o xhs.txt
```

---

## Examples / 改写效果展示

### 通用文本

**改写前**（AI 评分：87/100 VERY HIGH）：

> 综上所述，人工智能技术在教育领域具有重要的应用价值和广阔的发展前景。值得注意的是，随着技术的不断发展，AI 将在个性化学习、智能评估等方面发挥越来越重要的作用，为教育行业的数字化转型赋能。

**改写后**（AI 评分：12/100 LOW）：

> 从目前的实践来看，AI 辅助教学在个性化推荐方面效果比较明显，但在情感交互层面仍有不少局限。技术落地不是一蹴而就的事，教育场景的复杂性远超一般的商业应用。

### 学术论文

**改写前**（AIGC 评分：100/100 VERY HIGH）：

> 随着社会的不断发展，人工智能技术在教育领域的应用引起了广泛关注。本文旨在探讨人工智能对高等教育教学模式的影响，具有重要的理论意义和实践价值。研究表明，人工智能技术已被广泛应用于课堂教学、学生评估和个性化学习等多个方面，发挥着重要作用。

**改写后**（AIGC 评分：26/100 MEDIUM，降低 74 分）：

> 近年来社会变迁加速，人工智能技术在教育领域的应用逐渐进入研究者的视野。本文尝试探讨人工智能对高等教育教学模式的影响，兼具理论探索与实践参考的双重价值。前人研究发现，人工智能技术已广泛用于课堂教学、学生评估和个性化学习等多个方面，扮演着不可或缺的角色。

### 社交媒体

**改写前**（AI 评分：72/100 HIGH）：

> 在当今快节奏的生活中，时间管理具有至关重要的意义。通过合理规划时间，不仅能够提升工作效率，更能实现工作与生活的平衡。值得一提的是，良好的时间管理习惯需要长期培养和不断优化。

**改写后**（小红书风格，AI 评分：8/100 LOW）：

> 姐妹们！说真的，时间管理这事我踩过太多坑了 😭 之前天天加班到半夜，后来摸索出一套方法，现在居然能准点下班了？！重点就是别给自己排太满，留点缓冲时间。这个习惯我养了大概三个月才稳定下来，急不来的～

---

## Scoring / 评分标准

| 分数 | 等级 | 含义 |
|------|------|------|
| 0-24 | 🟢 LOW | 基本像人写的 |
| 25-49 | 🟡 MEDIUM | 有些 AI 痕迹 |
| 50-74 | 🟠 HIGH | 大概率是 AI 生成 |
| 75-100 | 🔴 VERY HIGH | 几乎可以确定是 AI |

### 检测维度

**🔴 高权重（容易暴露）：** 三段式套路（首先/其次/最后）、机械连接词（综上所述/值得注意的是）、空洞宏大词（赋能/闭环/数字化转型）

**🟠 中权重：** AI 高频词（助力/彰显/底层逻辑）、填充短语、平衡论述模板、套话句式

**🟡 低权重：** 犹豫语过多、列举成瘾、标点滥用、修辞堆砌

**⚪ 风格信号：** 段落长度均匀、句长单调、情感平淡、开头重复、信息熵低

**📊 统计特征（v2.2 新增）：** 基于字符级 N-gram 语言模型的统计检测。不再只"看词"，而是"看分布"——从概率层面检测 AI 生成痕迹：

| 指标 | 说明 | AI 文本 | 人类文本 |
|------|------|---------|----------|
| 困惑度 (Perplexity) | 文本在 N-gram 模型下的可预测程度 | ~231（低 = 更可预测） | ~533（高 = 更自然） |
| 突发度 (Burstiness) | 困惑度在滑动窗口间的变异系数 | 均匀（AI 复杂度恒定） | 有起伏（人类时简时繁） |
| 段落熵均匀度 | 段落间信息熵的变异系数 | 各段结构高度一致 | 段落间差异大 |

为什么加这个？传统规则检测只抓"像不像 AI 的样子"（表面特征），统计检测抓的是"是不是从语言模型采样的"（分布特征）。两层叠加，漏网率更低。

统计特征占总评分的 20-25%，作为规则检测的补充信号。基于预计算的中文字符 bigram/trigram 频率表（`scripts/ngram_freq_cn.json`），覆盖新闻、学术、社交、文学等多种文体。

独立使用统计分析：

```bash
python scripts/ngram_model.py your_text.txt
```

### 改写引擎的困惑度反馈（v2.3 新增）

改写不再是盲目替换。每次替换词/短语时，引擎会：

1. 列出所有候选替换（比如"综上所述"可以换成"总之"、"回顾以上讨论"、"简单说"等）
2. 逐一计算替换后句子的困惑度
3. **选困惑度最高的那个**（最不可预测 = 最像人写的）

另外还有两个机制：
- **Burstiness 守卫**：句式重组（合并/拆分）后如果文本 burstiness 下降超 20%（变得更均匀了），自动回退，不做这次重组
- **验证循环**：改写完成后整体测困惑度，如果还是太低，对最差的句子做第二轮替换

实测效果（学术论文改写）：

| | 原文 | 纯规则改写 | **v2.3 统计引导改写** |
|---|---|---|---|
| Perplexity | 173.9 | 185.2 | **212.0** (+22%) |
| Burstiness | 0.539 | 0.443 | **0.583** (+31%) |

用 `--no-stats` 可以关闭统计优化，回退到纯规则替换（更快，但效果差一些）。

---

## 🎓 学生党必看：免费降 AIGC 率

用 ChatGPT 写了论文初稿，担心知网/维普查出来？这个工具就是干这个的。

### 三步走

```bash
# 1. 先看看你的论文 AIGC 率多高
python scripts/academic_cn.py 论文.txt

# 2. 一键改写
python scripts/academic_cn.py 论文.txt -o 改后.txt --compare

# 3. 不放心就开激进模式
python scripts/academic_cn.py 论文.txt -o 改后.txt -a --compare
```

### 这工具干了啥

- 把"本文旨在""具有重要意义"这种 AI 套话换成正常的学术表达
- 减少被动句（"被广泛应用"→"得到较多运用"）
- 打破每段都一样长的结构
- 加入学术犹豫语（"可能""在一定程度上"），让语气没那么绝对
- 把"研究表明"换成"笔者认为""前人研究发现"，增加作者存在感
- 给结论补上局限性讨论

### 注意

⚠️ 工具只做文本风格调整，**不会改变你的论证逻辑和学术内容**。改完以后通读一遍，确认：
- 专业术语没被误改
- 论述逻辑没打乱
- 引用格式还对着
- 读起来通顺

建议改完以后自己用知网 AMLC 或维普跑一遍，确认效果。

---

## Writing Styles / 风格转换

| 风格 | 命令 | 适合场景 |
|------|------|----------|
| 口语化 | `--style casual` | 聊天、朋友圈 |
| 知乎 | `--style zhihu` | 深度回答、分析 |
| 小红书 | `--style xiaohongshu` | 种草、测评、生活分享 |
| 公众号 | `--style wechat` | 推文、专栏 |
| 学术 | `--style academic` | 论文、报告 |
| 文艺 | `--style literary` | 散文、随笔 |
| 微博 | `--style weibo` | 短评、热点 |

---

## CLI Reference

```bash
# 检测
python scripts/detect_cn.py [file] [-v] [-s] [-j] [--sentences N]

# 改写
python scripts/humanize_cn.py [file] [-o out] [--scene S] [--style S] [-a] [--seed N]

# 风格转换
python scripts/style_cn.py [file] --style S [-o out] [--seed N] [--list]

# 学术降重
python scripts/academic_cn.py [file] [-o out] [--detect-only] [-a] [--compare] [-j] [-s] [-v]

# 对比
python scripts/compare_cn.py [file] [-o out] [--scene S] [--style S] [-a]
```

### 常用参数

| 参数 | 说明 |
|------|------|
| `-v` | 详细模式，显示最可疑的句子 |
| `-s` | 只输出评分（如 `72/100 (high)`） |
| `-j` | JSON 格式输出 |
| `-o` | 输出文件路径 |
| `-a` | 激进模式，改写力度更大 |
| `--seed N` | 固定随机种子，保证结果可复现 |
| `--scene` | 场景：general / social / tech / formal / chat |
| `--style` | 风格：casual / zhihu / xiaohongshu / wechat / academic / literary / weibo |
| `--compare` | 显示改写前后对比 |
| `--no-stats` | 关闭困惑度统计优化（更快，纯规则替换） |

---

## Customization / 自定义

所有检测模式、替换词库、权重配置都在 `scripts/patterns_cn.json` 里，可以自己改：

- 加新的 AI 词汇模式
- 调整评分权重
- 添加替换规则
- 修改正则匹配模式

---

## Batch Processing / 批量处理

```bash
# 批量检测
for f in *.txt; do echo "=== $f ===" && python scripts/detect_cn.py "$f" -s; done

# 批量改写
for f in *.md; do python scripts/humanize_cn.py "$f" -a -o "${f%.md}_clean.md"; done
```

---

## License

MIT — 随便用，不用付钱，不用署名。


path: evals/evals.json
content: prompt: 帮我检测一下这段文字是不是AI写的：随着人工智能技术的不断发展，AI在医疗领域展现出了巨大的潜力。首先，值得注意的是，通过深度学习和大数据分析，AI能够助力医生进行更精准的诊断。
expected_behavior: Agent runs detect_cn.py, reports HIGH or VERY HIGH score with specific issues found (三段式, 机械连接词, 模板句式)

prompt: 把这段AI味很重的文字改成人话：综上所述，在数字化转型的背景下，赋能传统行业已成为不可逆转的趋势。值得注意的是，深度融合是实现高质量发展的必由之路。
expected_behavior: Agent runs humanize_cn.py (possibly with -a flag), output removes 综上所述/赋能/深度融合/高质量发展 etc., then verifies with detect_cn.py showing lower score

prompt: 把这篇文章改成小红书风格
expected_behavior: Agent runs style_cn.py --style xiaohongshu or humanize_cn.py --style xiaohongshu, output has emoji, short paragraphs, enthusiastic tone, hashtags

prompt: 我的论文AIGC率太高了，帮我降一下
expected_behavior: Agent runs academic_cn.py with --compare flag, shows before/after AIGC scores, outputs rewritten academic text maintaining scholarly tone

prompt: detect if this Chinese text is AI-generated
expected_behavior: Agent runs detect_cn.py with -v flag for verbose analysis, explains the scoring and detected patterns in English or Chinese based on user language

path: examples/sample_academic.txt
content: 随着社会经济的不断发展，乡村振兴战略的实施对农村基层治理提出了新的要求。本文旨在探讨数字技术在农村基层治理中的应用及其效果，具有重要的理论意义和实践价值。

近年来，数字技术在农村治理领域的应用引起了学术界的广泛关注。研究表明，数字平台已被广泛应用于政务公开、民意收集和公共服务供给等多个方面，发挥着重要作用（王某某，2023）。李某某（2022）指出，数字治理能够有效提升基层政府的行政效率和公共服务质量。张某某（2023）认为，数字技术为农村治理现代化提供了有力的技术支撑。

本研究采用混合研究方法，选取东部地区 5 个典型村庄作为研究对象，通过问卷调查和深度访谈相结合的方式收集数据。首先，对数字治理平台的使用现状进行了系统的调查分析；其次，对村干部和村民的使用体验进行了深入的访谈研究；最后，运用统计分析方法对收集的数据进行了全面的处理和分析。

从表中可以看出，数字治理平台的使用率呈现出逐年上升的趋势。数据显示，2023 年平台活跃用户数较 2021 年增长了 156%，增幅显著。此外，村民满意度也从 2021 年的 62.3% 提升至 2023 年的 85.7%。

综上所述，数字技术在农村基层治理中取得了显著的成效。本研究的发现不仅丰富了数字治理的理论体系，更为农村基层治理的实践提供了重要的参考依据。毫无疑问，数字技术必将在乡村振兴进程中发挥越来越重要的作用，为实现农村治理现代化注入强大动力。


path: examples/sample_general.txt
content: 综上所述，人工智能技术在教育领域具有重要的应用价值和广阔的发展前景。值得注意的是，随着技术的不断发展，AI 将在个性化学习、智能评估和自适应教学等方面发挥越来越重要的作用。

首先，人工智能能够助力教师进行精准教学。通过大数据分析和深度学习算法，AI 系统可以全方位地了解每个学生的学习特点，从而实现真正意义上的因材施教。其次，智能评估系统能够多维度地评判学生的综合素质，不仅关注知识掌握程度，更注重能力的培养和素质的提升。最后，自适应学习平台能够根据学生的实时表现动态调整学习路径，为每位学习者提供个性化的学习体验。

不可否认，人工智能在教育领域的应用也面临着诸多挑战。一方面，技术的成熟度和可靠性仍有待提高；另一方面，教育工作者的数字化素养也需要同步提升。然而，这些挑战并不能阻挡 AI 赋能教育的大趋势。

在数字化转型的背景下，深度融合人工智能与教育教学已成为高质量发展的必由之路。我们有理由相信，通过各方的协同增效，人工智能必将为教育事业注入新的活力，开创更加美好的未来。


path: examples/sample_social.txt
content: 在当今快节奏的生活中，时间管理对于每个人来说都具有至关重要的意义。值得注意的是，通过合理规划和科学管理时间，不仅能够显著提升工作效率，更能够实现工作与生活的完美平衡，从而全方位提升整体生活质量。

首先，制定明确的目标是时间管理的基础。一个清晰的目标能够赋能我们的日常决策，帮助我们更好地分配时间和精力，避免在不重要的事情上浪费宝贵的时间资源。不难发现，那些在时间管理上取得显著成效的人，无一例外都拥有清晰的目标体系。

其次，善于利用工具是提升时间管理效率的关键抓手。随着数字化转型的不断深入，各类时间管理应用和工具层出不穷，为我们提供了多维度的解决方案。通过深度融合这些数字化工具，我们能够更加高效地进行任务规划和进度追踪，实现真正意义上的降本增效。

此外，学会拒绝也是时间管理中不可或缺的重要环节。面对各种社交邀约和工作请求，我们需要学会区分轻重缓急，将有限的时间和精力投入到真正重要的事情上。事实上，这种取舍能力本身就是一种核心竞争力。

综上所述，良好的时间管理习惯需要长期培养和不断优化迭代。只有持之以恒地践行科学的时间管理方法，才能真正实现个人效能的最大化，迎来更加充实和有意义的人生。毫无疑问，时间管理将成为每个现代人的必备技能。


path: package.json
content: {
  "name": "humanize-chinese",
  "version": "2.1.0",
  "description": "Detect and humanize AI-generated Chinese text. 20+ detection patterns, weighted 0-100 scoring, sentence-level analysis, 7 style transforms, academic AIGC reduction for CNKI/VIP/Wanfang. Pure Python, no dependencies.",
  "main": "SKILL.md",
  "scripts": {
    "detect": "python scripts/detect_cn.py",
    "humanize": "python scripts/humanize_cn.py",
    "compare": "python scripts/compare_cn.py",
    "style": "python scripts/style_cn.py",
    "academic": "python scripts/academic_cn.py"
  },
  "keywords": [
    "ai",
    "chinese",
    "humanize",
    "detector",
    "clawhub",
    "skill",
    "ai-detector",
    "chinese-ai",
    "deai",
    "去ai化",
    "aigc",
    "降重",
    "知网",
    "维普",
    "万方",
    "academic",
    "论文",
    "humanizer",
    "ai-writing",
    "text-humanizer"
  ],
  "author": "voidborne-d",
  "license": "MIT",
  "repository": {
    "type": "git",
    "url": "https://github.com/voidborne-d/humanize-chinese.git"
  },
  "bugs": {
    "url": "https://github.com/voidborne-d/humanize-chinese/issues"
  },
  "homepage": "https://github.com/voidborne-d/humanize-chinese#readme"
}


path: scripts/academic_cn.py
content: #!/usr/bin/env python3
"""
Chinese Academic Paper AIGC Detector & Humanizer v1.0
Specialized for Chinese academic papers — targets CNKI/VIP/Wanfang AIGC detection patterns.
Features: 10 detection dimensions, 100+ replacement rules, hedging injection, structure variation.
Pure Python, no external dependencies.
"""

import sys
import re
import random
import json
import os
import argparse
from collections import defaultdict
from math import log2

# Import n-gram statistical model
try:
    from ngram_model import analyze_text as ngram_analyze
except ImportError:
    try:
        from scripts.ngram_model import analyze_text as ngram_analyze
    except ImportError:
        ngram_analyze = None

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
PATTERNS_FILE = os.path.join(SCRIPT_DIR, 'patterns_cn.json')


def load_config():
    if os.path.exists(PATTERNS_FILE):
        with open(PATTERNS_FILE, 'r', encoding='utf-8') as f:
            return json.load(f)
    return None


CONFIG = load_config()
ACAD = CONFIG.get('academic_patterns', {}) if CONFIG else {}

# ═══════════════════════════════════════════════════════════════════
#  Detection Patterns — 10 Dimensions
# ═══════════════════════════════════════════════════════════════════

# --- 1. AI 典型学术措辞 ---
AI_ACADEMIC_PHRASES = ACAD.get('ai_academic_phrases', {}).get('phrases', [
    '本文旨在', '研究表明', '具有重要意义', '进行了深入分析',
    '具有重要的理论意义和实践价值', '为此本文', '鉴于此',
    '对此进行了系统的', '进行了全面的分析', '进行了系统的研究',
    '取得了显著的成效', '具有广阔的应用前景', '发挥着重要作用',
    '引起了广泛关注', '得到了广泛的应用', '提供了有力的支撑',
    '提出了切实可行的', '进行了有益的探索', '奠定了坚实的基础',
    '提供了重要的参考价值', '提供了新的思路', '开辟了新的途径',
    '进行了详细的阐述', '做出了重要贡献', '具有一定的参考价值',
    '受到了学术界的广泛关注', '成为学术界研究的热点',
    '具有十分重要的现实意义', '有着不可忽视的作用',
    '对其进行深入研究', '有待进一步研究',
])

# --- 2. 过度被动句式 ---
PASSIVE_PATTERNS = ACAD.get('passive_patterns', {}).get('regex', [
    r'被广泛(应用|使用|采用|运用|关注|认可|接受)',
    r'被认为是',
    r'被(视为|看作|称为|誉为)',
    r'被(普遍|学界|学术界|研究者)(认为|接受|采纳)',
    r'被(充分|有效|合理)(利用|运用|开发)',
    r'被(证明|证实|验证)(为|是)',
    r'受到了?(广泛|高度|普遍|学界的?)(关注|重视|认可)',
])

# --- 3. 段落结构整齐度 ---
# (检测逻辑：每段首句总述 + N 个分论点 + 总结)
PARAGRAPH_OPENER_PATTERNS = [
    r'^(总的来看|总体而言|概括来说|从总体上看)',
    r'^(在\S{2,8}方面)',
    r'^(就\S{2,8}而言)',
    r'^(关于\S{2,8}(的问题|方面|层面))',
    r'^(从\S{2,8}(角度|视角|层面)(来看|来说|出发))',
]

# --- 4. 逻辑连接词 ---
LOGIC_CONNECTORS = ACAD.get('logic_connectors', {}).get('phrases', [
    '因此', '所以', '由此可见', '综上所述', '总而言之',
    '此外', '另外', '与此同时', '不仅如此', '更重要的是',
    '换言之', '也就是说', '具体而言', '进而', '从而',
    '然而', '但是', '不过', '尽管如此', '相比之下',
    '一方面', '另一方面', '首先', '其次', '最后',
    '事实上', '实际上', '显然', '毫无疑问', '值得注意的是',
    '需要指出的是', '值得一提的是', '由此', '基于此',
])

# --- 5. 同义表达匮乏 --- (检测逻辑在函数中)

# --- 6. 引用整合度 ---
CITATION_TEMPLATES = ACAD.get('citation_templates', {}).get('regex', [
    r'[（(]\d{4}[）)]指出',
    r'[（(]\d{4}[）)]认为',
    r'[（(]\d{4}[）)]的研究(表明|发现|指出|证实)',
    r'根据\S{1,6}[（(]\d{4}[）)]的研究',
    r'正如\S{1,6}[（(]\d{4}[）)]所(指出|述|言)',
    r'\S{1,6}等[（(]\d{4}[）)](的研究)?(表明|发现|认为)',
])

# --- 7. 数据论述模板化 ---
DATA_TEMPLATES = ACAD.get('data_templates', {}).get('phrases', [
    '数据显示', '数据表明', '从表中可以看出', '如表所示',
    '从图中可以看出', '如图所示', '由表可知', '由图可知',
    '根据表中数据', '从上表可以看出', '通过数据分析可知',
    '统计结果表明', '分析结果显示', '实验结果表明',
    '调查结果显示', '从数据中不难发现', '数据分析结果表明',
    '表中数据表明', '从图表中可以清晰地看出',
])

# --- 8. 过度列举 ---
ENUMERATION_REGEX = [
    r'(?:(?:①|②|③|④|⑤|⑥).*?[。；;]){3,}',
    r'(?:(?:第[一二三四五六七八九十])[，,].*?[。；;]){3,}',
    r'(?:(?:\([一二三四五六]\)).*?[。；;]){3,}',
    r'(?:(?:\d\.)\s*.*?[。；;]){4,}',
]

# --- 9. 结论过于圆满（缺少局限性） ---
PERFECT_CONCLUSION_PHRASES = ACAD.get('perfect_conclusion', {}).get('phrases', [
    '取得了显著成效', '取得了丰硕成果', '取得了重大突破',
    '圆满完成', '成效显著', '效果明显', '成果丰硕',
    '为.*?提供了有力支撑', '为.*?奠定了坚实基础',
    '具有重要的理论意义和实践价值', '前景广阔',
])
LIMITATION_MARKERS = [
    '局限', '不足', '有待', '仍需', '尚未', '尚需',
    '未来研究', '后续研究', '进一步探讨', '存在一定的',
    '样本量', '推广性', '外部效度', '内部效度',
]

# --- 10. 语气过于确定 ---
CERTAINTY_MARKERS = ACAD.get('certainty_markers', {}).get('phrases', [
    '必然', '一定会', '毫无疑问', '无疑', '显然',
    '毋庸置疑', '不可否认', '确定无疑', '势必',
    '必将', '注定', '无一例外', '绝对',
])
HEDGING_MARKERS = [
    '可能', '或许', '也许', '在一定条件下', '在某种程度上',
    '一般而言', '通常', '大致', '似乎', '倾向于',
    '有可能', '在一定范围内', '初步', '大体上',
]


# ═══════════════════════════════════════════════════════════════════
#  Utility
# ═══════════════════════════════════════════════════════════════════

def count_chinese(text):
    return len(re.findall(r'[\u4e00-\u9fff]', text))


def split_sentences(text):
    parts = re.split(r'([。！？\n])', text)
    sentences = []
    for i in range(0, len(parts) - 1, 2):
        s = parts[i].strip()
        if s:
            sentences.append(s + (parts[i + 1] if i + 1 < len(parts) else ''))
    if len(parts) % 2 == 1 and parts[-1].strip():
        sentences.append(parts[-1].strip())
    return [s for s in sentences if len(s) > 1]


def char_entropy(text):
    chars = re.findall(r'[\u4e00-\u9fff]', text)
    if len(chars) < 10:
        return 5.0
    bigrams = defaultdict(int)
    for i in range(len(chars) - 1):
        bigrams[chars[i] + chars[i + 1]] += 1
    total = sum(bigrams.values())
    if total == 0:
        return 5.0
    entropy = 0
    for count in bigrams.values():
        p = count / total
        if p > 0:
            entropy -= p * log2(p)
    return entropy


# ═══════════════════════════════════════════════════════════════════
#  Detection Engine — 10 Dimensions
# ═══════════════════════════════════════════════════════════════════

DIMENSION_WEIGHTS = {
    'ai_academic_phrases':   8,   # AI 典型学术措辞
    'passive_overuse':       5,   # 被动句式过度
    'paragraph_uniformity':  6,   # 段落结构过于整齐
    'connector_density':     7,   # 逻辑连接词密度
    'synonym_poverty':       6,   # 同义表达匮乏
    'citation_integration':  4,   # 引用整合度低
    'data_template':         5,   # 数据论述模板化
    'over_enumeration':      5,   # 过度列举
    'perfect_conclusion':    6,   # 结论过于圆满
    'certainty_overuse':     7,   # 语气过于确定
    # Statistical dimensions (n-gram perplexity)
    'stat_low_perplexity':   0,   # scored separately
    'stat_low_burstiness':   0,   # scored separately
    'stat_uniform_entropy':  0,   # scored separately
}

# Statistical feature weights (contribute up to ~25% of final score)
STATISTICAL_WEIGHTS = {
    'stat_low_perplexity': 12,
    'stat_low_burstiness': 10,
    'stat_uniform_entropy': 8,
}


def detect_academic(text):
    """Run all 10 academic AIGC detection dimensions."""
    issues = defaultdict(list)
    char_count = count_chinese(text)
    sentences = split_sentences(text)
    paragraphs = [p.strip() for p in text.split('\n') if p.strip() and len(p.strip()) > 10]

    # ── 1. AI 典型学术措辞 ──
    for phrase in AI_ACADEMIC_PHRASES:
        cnt = text.count(phrase)
        if cnt > 0:
            issues['ai_academic_phrases'].append({
                'text': phrase, 'count': cnt, 'severity': 'critical'})

    # ── 2. 被动句式过度 ──
    passive_total = 0
    for pat in PASSIVE_PATTERNS:
        for m in re.finditer(pat, text):
            passive_total += 1
            if len(issues['passive_overuse']) < 10:
                issues['passive_overuse'].append({
                    'text': m.group()[:40], 'severity': 'high'})
    if passive_total == 0:
        pass  # no issue

    # ── 3. 段落结构整齐度 ──
    if len(paragraphs) >= 3:
        lengths = [count_chinese(p) for p in paragraphs]
        avg = sum(lengths) / len(lengths) if lengths else 1
        if avg > 0:
            cv = (sum((l - avg) ** 2 for l in lengths) / len(lengths)) ** 0.5 / avg
            if cv < 0.18:
                issues['paragraph_uniformity'].append({
                    'text': f'段落长度 CV={cv:.2f}，结构过于整齐',
                    'severity': 'high'})

        # Check for repeating structure: opener pattern similarity
        opener_count = 0
        for p in paragraphs:
            first_sent = p.split('。')[0] if '。' in p else p[:50]
            for pat in PARAGRAPH_OPENER_PATTERNS:
                if re.match(pat, first_sent.strip()):
                    opener_count += 1
                    break
        if opener_count >= 3:
            issues['paragraph_uniformity'].append({
                'text': f'{opener_count} 个段落使用相似的起始结构',
                'severity': 'high'})

    # ── 4. 逻辑连接词密度 ──
    connector_count = 0
    found_connectors = []
    for c in LOGIC_CONNECTORS:
        cnt = text.count(c)
        if cnt > 0:
            connector_count += cnt
            found_connectors.append(c)
    if char_count > 0:
        density = connector_count / (char_count / 100)
        threshold = 3.0  # per 100 chars
        if density > threshold:
            issues['connector_density'].append({
                'text': f'逻辑连接词密度 {density:.1f}/百字（阈值 {threshold}），共 {connector_count} 个',
                'count': connector_count, 'severity': 'high'})
            # show top ones
            for c in found_connectors[:5]:
                issues['connector_density'].append({
                    'text': f'  · {c} ×{text.count(c)}', 'severity': 'info'})

    # ── 5. 同义表达匮乏 ──
    # Find words repeated 3+ times that should have synonyms
    academic_repeat_words = [
        '研究', '分析', '探讨', '论述', '阐述', '表明',
        '发现', '指出', '认为', '提出', '进行', '开展',
        '影响', '作用', '意义', '价值', '问题', '方面',
        '水平', '能力', '效果', '结果', '方法', '策略',
        '措施', '途径', '机制', '模式', '体系', '框架',
    ]
    repeat_issues = []
    for w in academic_repeat_words:
        cnt = text.count(w)
        if cnt >= 4:
            repeat_issues.append((w, cnt))
    repeat_issues.sort(key=lambda x: x[1], reverse=True)
    for w, cnt in repeat_issues[:5]:
        issues['synonym_poverty'].append({
            'text': f'"{w}" 重复 {cnt} 次', 'count': cnt, 'severity': 'medium'})

    # ── 6. 引用整合度低 ──
    cite_template_count = 0
    for pat in CITATION_TEMPLATES:
        for m in re.finditer(pat, text):
            cite_template_count += 1
    # Count total citations
    total_cites = len(re.findall(r'[（(]\d{4}[）)]', text))
    if total_cites >= 3 and cite_template_count >= 3:
        ratio = cite_template_count / total_cites if total_cites > 0 else 0
        if ratio > 0.6:
            issues['citation_integration'].append({
                'text': f'{cite_template_count}/{total_cites} 处引用使用模板化表述（比例 {ratio:.0%}）',
                'severity': 'medium'})

    # ── 7. 数据论述模板化 ──
    for phrase in DATA_TEMPLATES:
        cnt = text.count(phrase)
        if cnt > 0:
            issues['data_template'].append({
                'text': phrase, 'count': cnt, 'severity': 'medium'})

    # ── 8. 过度列举 ──
    for pat in ENUMERATION_REGEX:
        for m in re.finditer(pat, text, re.DOTALL):
            issues['over_enumeration'].append({
                'text': m.group()[:60] + '...', 'severity': 'medium'})

    # ── 9. 结论过于圆满 ──
    # Check last 20% of text for perfect conclusion without limitations
    tail = text[int(len(text) * 0.8):]
    has_limitation = any(m in tail for m in LIMITATION_MARKERS)
    perfect_count = 0
    for phrase in PERFECT_CONCLUSION_PHRASES:
        if re.search(phrase, tail):
            perfect_count += 1
    if perfect_count >= 2 and not has_limitation:
        issues['perfect_conclusion'].append({
            'text': f'结论部分有 {perfect_count} 处圆满表述，且缺少局限性讨论',
            'severity': 'high'})

    # ── 10. 语气过于确定 ──
    certain_count = sum(text.count(w) for w in CERTAINTY_MARKERS)
    hedge_count = sum(text.count(w) for w in HEDGING_MARKERS)
    if char_count > 200:
        if certain_count >= 3 and hedge_count < 2:
            issues['certainty_overuse'].append({
                'text': f'确定性表述 {certain_count} 次，学术留白仅 {hedge_count} 次',
                'severity': 'high'})
        elif certain_count > 0 and hedge_count == 0:
            issues['certainty_overuse'].append({
                'text': f'确定性表述 {certain_count} 次，完全缺少学术犹豫语',
                'severity': 'medium'})

    # ── 11. 统计特征：N-gram 困惑度 ──
    ngram_stats = None
    if ngram_analyze and char_count >= 100:
        ngram_stats = ngram_analyze(text)
        indicators = ngram_stats.get('indicators', {})

        if indicators.get('low_perplexity'):
            ppl = ngram_stats['perplexity']
            issues['stat_low_perplexity'].append({
                'text': f'文本困惑度 {ppl:.1f}（学术 AI 文本通常在此范围内）',
                'severity': 'statistical'})

        if indicators.get('low_burstiness'):
            burst = ngram_stats['burstiness']
            issues['stat_low_burstiness'].append({
                'text': f'困惑度变异系数 {burst:.3f}（过于均匀，缺少写作起伏）',
                'severity': 'statistical'})

        if indicators.get('uniform_entropy'):
            ent_cv = ngram_stats['entropy_cv']
            issues['stat_uniform_entropy'].append({
                'text': f'段落熵变异系数 {ent_cv:.3f}（段落间复杂度过于一致）',
                'severity': 'statistical'})

    # ── Metrics ──
    entropy = char_entropy(text)
    metrics = {
        'char_count': char_count,
        'sentence_count': len(sentences),
        'paragraph_count': len(paragraphs),
        'entropy': entropy if char_count > 200 else None,
        'connector_density': connector_count / (char_count / 100) if char_count > 0 else 0,
        'passive_count': passive_total,
        'certainty_count': certain_count,
        'hedging_count': hedge_count,
        'citation_count': total_cites,
    }

    # Add statistical metrics if available
    if ngram_stats:
        metrics['perplexity'] = ngram_stats['perplexity']
        metrics['burstiness'] = ngram_stats['burstiness']
        metrics['entropy_cv'] = ngram_stats['entropy_cv']

    return issues, metrics


def calculate_academic_score(issues):
    """Calculate AIGC probability score 0-100 for academic text.
    
    Scoring composition:
      - Rule-based dimensions: ~75% weight
      - Statistical features (perplexity/burstiness/entropy): ~25% weight
    """
    raw = 0
    for dim, items in issues.items():
        # Skip statistical items — scored separately
        if dim.startswith('stat_'):
            continue
        weight = DIMENSION_WEIGHTS.get(dim, 3)
        for item in items:
            sev = item.get('severity', 'medium')
            if sev == 'info':
                continue
            count = item.get('count', 1)
            sev_mult = {'critical': 1.5, 'high': 1.0, 'medium': 0.6, 'low': 0.3}.get(sev, 0.6)
            raw += weight * sev_mult * min(count, 5)
    
    # Rule-based score (cap at ~75 points)
    rule_score = min(75, int(raw * 0.7))
    
    # Statistical score (up to 25 points)
    stat_score = 0
    for dim, items in issues.items():
        if dim.startswith('stat_') and items:
            stat_score += STATISTICAL_WEIGHTS.get(dim, 5)
    stat_score = min(25, stat_score)
    
    return min(100, rule_score + stat_score)


def score_to_level(score):
    if score >= 75:
        return 'very_high'
    elif score >= 50:
        return 'high'
    elif score >= 25:
        return 'medium'
    return 'low'


# ═══════════════════════════════════════════════════════════════════
#  Academic Humanization Engine
# ═══════════════════════════════════════════════════════════════════

# Replacement dictionary: AI expression -> list of more natural academic alternatives
ACADEMIC_REPLACEMENTS = ACAD.get('replacements', {})
if not ACADEMIC_REPLACEMENTS:
    ACADEMIC_REPLACEMENTS = {
        # --- AI 典型学术措辞替换 (50+) ---
        '本文旨在': ['本研究聚焦于', '本文着重', '本文尝试'],
        '研究表明': ['已有文献指出', '前人研究发现', '相关研究揭示'],
        '具有重要意义': ['不容忽视', '值得关注', '有其独特价值'],
        '具有重要的理论意义和实践价值': ['兼具理论探索与实践参考的双重价值', '在学理与应用两个层面均有所推进'],
        '进行了深入分析': ['作了较为细致的考察', '做了逐层剖析', '展开了多角度的审视'],
        '进行了全面的分析': ['从多个维度加以考察', '较为系统地做了梳理'],
        '进行了系统的研究': ['围绕该问题做了较为完整的探索', '对此做了逐步推进的研究'],
        '取得了显著的成效': ['收到了一定的效果', '初步达到了预期目标'],
        '具有广阔的应用前景': ['在应用层面仍有可拓展的空间', '其实践潜力有待进一步挖掘'],
        '发挥着重要作用': ['扮演着不可或缺的角色', '在其中起到了关键的支撑'],
        '引起了广泛关注': ['逐渐进入研究者的视野', '日益受到学界的重视'],
        '得到了广泛的应用': ['已在多个领域落地', '实际应用范围持续扩大'],
        '提供了有力的支撑': ['为后续工作铺垫了基础', '在技术层面给予了支持'],
        '提出了切实可行的': ['给出了具有操作性的', '拟定了较为务实的'],
        '进行了有益的探索': ['做了初步的尝试', '在这一方向上迈出了一步'],
        '奠定了坚实的基础': ['打下了一定的基础', '在基础层面做了铺垫'],
        '提供了重要的参考价值': ['对相关研究有所启发', '可为后续研究提供借鉴'],
        '提供了新的思路': ['打开了一个新的切入角度', '在思路上有所创新'],
        '开辟了新的途径': ['探索出一条可能的路径', '尝试了一种新的进路'],
        '进行了详细的阐述': ['做了较为充分的说明', '逐一加以论述'],
        '做出了重要贡献': ['在该领域有所推进', '对这一问题的认识起到了补充作用'],
        '具有一定的参考价值': ['可供后来者参考', '对同领域研究有所帮助'],
        '受到了学术界的广泛关注': ['近年来已成为学界讨论的议题之一'],
        '成为学术界研究的热点': ['在学界引发了持续的讨论'],
        '具有十分重要的现实意义': ['对当下的实际问题有着直接的观照意义'],
        '有着不可忽视的作用': ['其作用不宜低估', '在其中发挥着不小的影响'],
        '对其进行深入研究': ['对此做进一步的追问', '就此展开更细致的考察'],
        '有待进一步研究': ['仍有继续追问的空间', '后续尚可深入'],
        '鉴于此': ['基于上述考虑', '考虑到这一背景'],
        '为此本文': ['在此基础上，本文', '因此本文尝试'],

        # --- 被动句式替换 (15+) ---
        '被广泛应用于': ['已广泛运用于', '广泛用于'],
        '被广泛应用': ['已在多个场景中得到使用', '使用范围较广'],
        '被广泛使用': ['使用颇为普遍', '在实际中频繁出现'],
        '被广泛采用': ['已为多数研究者所采纳', '采用率较高'],
        '被广泛关注': ['持续吸引着研究者的目光', '关注度逐年上升'],
        '被认为是': ['一般视为', '通常归入', '多看作'],
        '被视为': ['常被看作', '往往归类为'],
        '被看作': ['通常被理解为', '多半被归为'],
        '被称为': ['学界习惯称之为', '一般称其为'],
        '被普遍认为': ['多数学者倾向于认为', '主流观点认为'],
        '被证明': ['经验证', '已有证据支持'],
        '被证实': ['得到了实证支持', '验证结果支持'],
        '受到了广泛关注': ['引发了较多讨论', '获得了较高的关注度'],
        '受到了高度重视': ['得到了相当程度的重视'],

        # --- 数据论述替换 (18+) ---
        '数据显示': ['从数据来看', '数据层面反映出', '统计数字指向'],
        '数据表明': ['数据所呈现的趋势是', '对数据的解读提示'],
        '从表中可以看出': ['观察该表', '表中所列数据反映出', '对比表中各列'],
        '如表所示': ['参见下表', '该表汇总了', '以下表格列出了'],
        '从图中可以看出': ['对照该图', '图中曲线显示', '观察图形走势'],
        '如图所示': ['参见下图', '图中呈现的规律是'],
        '由表可知': ['该表揭示了', '梳理表中数据后发现'],
        '由图可知': ['图形走势反映出', '从该图可以初步判断'],
        '根据表中数据': ['依据表内数据', '结合表中各项指标'],
        '从上表可以看出': ['上表所列数据提示', '回顾上表'],
        '通过数据分析可知': ['经过数据梳理后发现', '对数据做初步分析后'],
        '统计结果表明': ['统计分析给出的结论是', '数据的统计特征指向'],
        '分析结果显示': ['分析结果提示', '根据分析输出'],
        '实验结果表明': ['实验所得结果指向', '从实验数据来看'],
        '调查结果显示': ['问卷反馈汇总后发现', '受访者的回答反映出'],
        '从数据中不难发现': ['稍加留意便可发现', '数据中隐含着'],
        '数据分析结果表明': ['对数据做深入分析后的结论是'],
        '从图表中可以清晰地看出': ['图表较为直观地呈现了'],

        # --- 逻辑连接词柔化 (20+) ---
        '综上所述': ['回顾以上讨论', '梳理上文的论述'],
        '总而言之': ['概括地说', '归结起来'],
        '由此可见': ['从这一点延伸来看', '这也提示我们'],
        '不难发现': ['稍加留意便可发现', '细看之下'],
        '值得注意的是': ['需要留意的一点是', '这里有一个值得关注的细节'],
        '需要指出的是': ['这里需要做一个说明', '有必要补充的是'],
        '值得一提的是': ['此外还有一个值得提及的细节', '顺便一提'],
        '不仅如此': ['除此之外', '在此基础上还'],
        '与此同时': ['同一时期', '在此过程中'],
        '在此基础上': ['以此为起点', '在上述结论的延长线上'],
        '毫无疑问': ['应当承认', '较为确定的是'],
        '事实上': ['实际情况是', '就实际而言'],
        '显而易见': ['较为明显的是', '比较直观的一点是'],
        '不可否认': ['应当承认', '确实'],
        '更重要的是': ['更关键的一层是', '进一步说'],
        '换言之': ['换一种表述', '用另一种方式理解'],
        '具体而言': ['展开来说', '逐一来看'],
        '此外': ['另外还有', '除上述之外'],
        '进而': ['由此推进', '在此基础上进一步'],
        '从而': ['进而使得', '以此达到'],
        '基于此': ['出于这一考虑', '以此为出发点'],
        '总的来说': ['整体看来', '从宏观上概括'],

        # --- 确定性语气柔化 (10+) ---
        '必然会': ['很可能会', '大概率会'],
        '必然': ['大概率', '在多数情况下'],
        '一定会': ['很可能会', '大概率将'],
        '毫无疑问': ['应当承认', '较为确定的是'],
        '无疑': ['在很大程度上', '应当说'],
        '毋庸置疑': ['较为明确的是', '可以说'],
        '确定无疑': ['在现有证据下可以认为', '目前的结论倾向于'],
        '势必': ['很可能', '有较大概率'],
        '必将': ['预计将', '有望'],
        '注定': ['大概率会', '从趋势看可能'],
        '无一例外': ['在绝大多数情况下', '几乎所有案例中'],
        '绝对': ['在很大程度上', '基本上'],
        '显然': ['从现有资料来看', '比较明确的是'],

        # --- 模板句式替换 (10+) ---
        '随着社会的不断发展': ['近年来社会变迁加速', '在社会持续演进的过程中'],
        '在当今社会': ['当前', '在眼下的社会语境中'],
        '在当今时代': ['当下', '就当前阶段而言'],
        '具有重要的现实意义': ['对当下的问题有着切实的回应', '与当前的现实需求直接相关'],
        '本文主要研究': ['本文聚焦的问题是', '本文所考察的对象是'],
        '本文的研究目的': ['本文试图解答的问题', '本文关心的核心议题'],
    }

# Hedging expressions to inject (only multi-char phrases that read well between commas)
# Module-level flag: whether to use stats optimization
_USE_STATS = True


def pick_best_replacement(sentence, old, candidates):
    """从多个候选替换中选 perplexity 最高的（最不可预测 = 最像人写的）。
    当 _USE_STATS 关闭、ngram_analyze 不可用、或只有单候选时回退到 random.choice。"""
    if not _USE_STATS or not ngram_analyze or len(candidates) <= 1:
        return random.choice(candidates)

    best_candidate = candidates[0]
    best_ppl = 0

    for candidate in candidates:
        new_sentence = sentence.replace(old, candidate, 1)
        stats = ngram_analyze(new_sentence)
        ppl = stats.get('perplexity', 0)
        if ppl > best_ppl:
            best_ppl = ppl
            best_candidate = candidate

    return best_candidate


def _compute_burstiness(text):
    """计算文本的 burstiness。"""
    if not _USE_STATS or not ngram_analyze:
        return None
    stats = ngram_analyze(text)
    return stats.get('burstiness', None)


# Import perplexity-boosting strategies from humanize_cn
try:
    from humanize_cn import (reduce_high_freq_bigrams, randomize_sentence_lengths,
                             inject_noise_expressions)
except ImportError:
    try:
        from scripts.humanize_cn import (reduce_high_freq_bigrams, randomize_sentence_lengths,
                                         inject_noise_expressions)
    except ImportError:
        reduce_high_freq_bigrams = None
        randomize_sentence_lengths = None
        inject_noise_expressions = None

# Module-level flag: whether to apply noise strategies
_USE_NOISE = True

HEDGING_INJECTIONS = [
    '在一定程度上', '从某种角度看', '初步来看', '大体上',
    '就目前而言', '在多数情况下', '就现有研究来看',
    '从目前的情况看', '在一定条件下', '大致可以认为',
]

# Author subjectivity expressions
AUTHOR_VOICE = [
    '笔者认为', '笔者以为', '本研究发现', '就笔者的观察',
    '从笔者的分析来看', '据笔者了解', '笔者倾向于',
    '在笔者看来', '笔者注意到',
]


def _replace_academic_phrases(text, aggressive=False):
    """Replace AI academic phrases with natural alternatives.
    
    Uses per-character sentinel interleaving to prevent cascade replacements
    where a replacement's text contains another pattern as substring.
    Uses pick_best_replacement for perplexity-optimized selection when stats enabled.
    """
    SENTINEL = '\x00'  # Null byte — won't appear in normal Chinese text
    sorted_phrases = sorted(ACADEMIC_REPLACEMENTS.keys(), key=len, reverse=True)

    for phrase in sorted_phrases:
        alternatives = ACADEMIC_REPLACEMENTS[phrase]
        if isinstance(alternatives, str):
            alternatives = [alternatives]

        while phrase in text:
            replacement = pick_best_replacement(text, phrase, alternatives)
            # Interleave sentinels between each character to prevent substring matching
            protected = SENTINEL.join(replacement)
            text = text.replace(phrase, protected, 1)

    # Strip all sentinels
    text = text.replace(SENTINEL, '')
    return text


def _inject_hedging(text, aggressive=False):
    """Add academic hedging language to overly certain statements.
    Processes paragraph by paragraph to preserve paragraph breaks."""
    paragraphs = text.split('\n\n')
    result_paragraphs = []
    injected = 0
    total_sents = len(split_sentences(text))
    max_inject = max(2, total_sents // 8) if not aggressive else max(3, total_sents // 5)

    for para in paragraphs:
        sentences = split_sentences(para)
        result = []

        for sent in sentences:
            has_certainty = any(m in sent for m in CERTAINTY_MARKERS)
            has_hedging = any(m in sent for m in HEDGING_MARKERS)

            if has_certainty and not has_hedging and injected < max_inject:
                still_certain = any(m in sent for m in CERTAINTY_MARKERS)
                if still_certain:
                    for cm in CERTAINTY_MARKERS:
                        if cm in sent:
                            hedge = random.choice(HEDGING_INJECTIONS)
                            sent = sent.replace(cm, hedge, 1)
                            injected += 1
                            break
            elif not has_hedging and random.random() < (0.08 if not aggressive else 0.15) and injected < max_inject:
                # Don't inject hedging right after structural words
                skip_starters = ['首先', '其次', '最后', '第一', '第二', '第三']
                starts_structural = any(sent.strip().startswith(s) for s in skip_starters)
                if '，' in sent and not starts_structural:
                    parts = sent.split('，', 1)
                    # Only inject if first part is substantial (>4 chars)
                    if count_chinese(parts[0]) > 4:
                        hedge = random.choice(HEDGING_INJECTIONS)
                        sent = parts[0] + '，' + hedge + '，' + parts[1]
                        injected += 1

            result.append(sent)

        result_paragraphs.append(''.join(result))

    return '\n\n'.join(result_paragraphs)


def _add_author_voice(text, aggressive=False):
    """Replace impersonal 'research shows' with author voice.
    Only replaces when the phrase appears at sentence-start positions
    (after period, newline, or at the very beginning)."""
    impersonal = ['研究表明', '研究发现', '研究显示', '研究指出', '分析表明']
    replaced = 0
    max_replace = 3 if not aggressive else 5

    for phrase in impersonal:
        # Only replace at natural sentence boundaries
        for prefix in ['。', '\n', '，']:
            target = prefix + phrase
            while target in text and replaced < max_replace:
                voice = random.choice(AUTHOR_VOICE)
                text = text.replace(target, prefix + voice, 1)
                replaced += 1
        # Also check sentence start
        if text.startswith(phrase) and replaced < max_replace:
            voice = random.choice(AUTHOR_VOICE)
            text = voice + text[len(phrase):]
            replaced += 1

    return text


def _break_uniform_structure(text):
    """Vary paragraph structure to break AI-like uniformity.
    Preserves paragraph breaks (\n\n)."""
    paragraphs = text.split('\n\n')
    if len(paragraphs) < 3:
        return text

    result = []
    i = 0
    while i < len(paragraphs):
        para = paragraphs[i]
        if not para.strip():
            result.append(para)
            i += 1
            continue

        # Occasionally merge two short adjacent paragraphs
        if (i + 1 < len(paragraphs) and
                count_chinese(para) < 60 and
                count_chinese(paragraphs[i + 1]) < 60 and
                paragraphs[i + 1].strip() and
                random.random() < 0.25):
            merged = para.rstrip() + '\n' + paragraphs[i + 1].lstrip()
            result.append(merged)
            i += 2
            continue

        # Vary sentence structure within paragraph
        sentences = split_sentences(para)
        if len(sentences) >= 4:
            if random.random() < 0.15 and len(sentences) >= 3:
                mid = len(sentences) // 2
                sentences[mid], sentences[mid - 1] = sentences[mid - 1], sentences[mid]

        result.append(''.join(sentences))
        i += 1

    return '\n\n'.join(result)


def _reduce_connectors(text, aggressive=False):
    """Reduce density of logical connectors."""
    # Only remove some, not all — keep academic coherence
    removable = ['此外，', '另外，', '与此同时，', '不仅如此，', '事实上，', '实际上，']
    removed = 0
    max_remove = 3 if not aggressive else 6

    for conn in removable:
        while conn in text and removed < max_remove:
            if random.random() < 0.5:
                text = text.replace(conn, '', 1)
                removed += 1
            else:
                break

    return text


def _shorten_long_sentences(text, max_chars=90):
    """Split overly long academic sentences at natural breakpoints, with burstiness guard."""
    burst_before = _compute_burstiness(text)

    sentences = re.split(r'([。])', text)
    result = []

    for i in range(0, len(sentences) - 1, 2):
        sent = sentences[i]
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''

        if count_chinese(sent) > max_chars:
            # Find a split point
            split_words = ['但是', '不过', '然而', '同时', '而且', '因此', '所以', '并且']
            split_done = False
            for sw in split_words:
                pos = sent.find('，' + sw)
                if pos > 10 and pos < len(sent) - 10:
                    result.append(sent[:pos] + '。' + sent[pos + 1:] + punct)
                    split_done = True
                    break
            if not split_done:
                # Split at a comma near middle
                commas = [m.start() for m in re.finditer('，', sent)]
                if commas:
                    mid = len(sent) // 2
                    best = min(commas, key=lambda x: abs(x - mid))
                    if best > 10 and best < len(sent) - 10:
                        result.append(sent[:best] + '。' + sent[best + 1:] + punct)
                    else:
                        result.append(sent + punct)
                else:
                    result.append(sent + punct)
        else:
            result.append(sent + punct)

    if len(sentences) % 2 == 1 and sentences[-1].strip():
        result.append(sentences[-1])

    new_text = ''.join(result)

    # Burstiness guard: if splitting made text more uniform, revert
    if burst_before is not None:
        burst_after = _compute_burstiness(new_text)
        if burst_after is not None and burst_after < burst_before * 0.8:
            return text

    return new_text


def _diversify_data_narration(text):
    """Make data narration less template-like."""
    # Already handled by ACADEMIC_REPLACEMENTS
    return text


def _add_limitation_markers(text, aggressive=False):
    """If conclusion section lacks limitations, add mild hedging."""
    tail_start = int(len(text) * 0.8)
    tail = text[tail_start:]

    has_limitation = any(m in tail for m in LIMITATION_MARKERS)
    if has_limitation:
        return text

    # Check if there are perfect conclusion phrases
    has_perfect = any(re.search(p, tail) for p in PERFECT_CONCLUSION_PHRASES)
    if not has_perfect:
        return text

    # Add a mild limitation note
    limitations = [
        '当然，本研究也存在一定的局限：',
        '需要指出的是，本研究仍有若干不足之处。',
        '受限于研究条件，本文的结论在推广性方面尚有不足。',
        '本研究的样本和方法存在一定的局限，后续研究可进一步完善。',
    ]
    addition = random.choice(limitations)

    # Insert before last sentence
    last_period = tail.rfind('。')
    if last_period > 0:
        new_tail = tail[:last_period + 1] + addition + tail[last_period + 1:]
        text = text[:tail_start] + new_tail

    return text


def humanize_academic(text, aggressive=False, seed=None):
    """Apply all academic humanization transforms."""
    if seed is not None:
        random.seed(seed)

    # 1. Replace AI academic phrases
    text = _replace_academic_phrases(text, aggressive)

    # 2. Reduce connector density
    text = _reduce_connectors(text, aggressive)

    # 3. Add author voice
    text = _add_author_voice(text, aggressive)

    # 4. Inject hedging language
    text = _inject_hedging(text, aggressive)

    # 5. Break uniform structure
    text = _break_uniform_structure(text)

    # 6. Shorten long sentences
    text = _shorten_long_sentences(text)

    # 7. Diversify data narration (via replacements)
    text = _diversify_data_narration(text)

    # 8. Add limitation markers if needed
    text = _add_limitation_markers(text, aggressive)

    # ── NEW: Three perplexity-boosting strategies ──
    
    # Strategy 1: Low-frequency bigram injection (always active)
    if reduce_high_freq_bigrams:
        bigram_strength = 0.5 if aggressive else 0.3
        text = reduce_high_freq_bigrams(text, strength=bigram_strength)
    
    # Strategy 2 & 3: Noise injection (skipped with --no-noise)
    if _USE_NOISE:
        # Strategy 3: Noise expression injection (academic style — restrained)
        if inject_noise_expressions:
            noise_density = 0.2 if aggressive else 0.1
            text = inject_noise_expressions(text, density=noise_density, style='academic')
        
        # Strategy 2: Sentence length randomization
        if randomize_sentence_lengths:
            text = randomize_sentence_lengths(text, aggressive=aggressive, seed=seed)

    # Clean up
    text = re.sub(r'[，,]{2,}', '，', text)
    text = re.sub(r'[。]{2,}', '。', text)
    text = re.sub(r'\n{3,}', '\n\n', text)
    text = re.sub(r'，。', '。', text)
    text = re.sub(r'。，', '。', text)
    # Fix doubled characters from replacement artifacts
    text = re.sub(r'已已', '已', text)
    text = re.sub(r'会会', '会', text)
    text = re.sub(r'了了', '了', text)
    text = re.sub(r'的的', '的', text)
    text = re.sub(r'是是', '是', text)
    text = re.sub(r'在在', '在', text)
    # Fix phrase-level duplications (e.g. "为后续研究为后续研究")
    text = re.sub(r'([\u4e00-\u9fff]{4,12})\1', r'\1', text)

    # ── Final verification loop (stats-optimized) ──
    if _USE_STATS and ngram_analyze:
        stats = ngram_analyze(text)
        ppl = stats.get('perplexity', 0)
        if 0 < ppl < 200 and count_chinese(text) >= 100:
            sentences = split_sentences(text)
            sent_scores = []
            for i, s in enumerate(sentences):
                if count_chinese(s) < 5:
                    continue
                s_stats = ngram_analyze(s)
                sent_scores.append((i, s_stats.get('perplexity', 0)))

            if sent_scores:
                sent_scores.sort(key=lambda x: x[1])
                n_fix = min(5, max(1, len(sent_scores) // 5))

                if seed is not None:
                    random.seed(seed + 1)

                sorted_phrases = sorted(ACADEMIC_REPLACEMENTS.keys(), key=len, reverse=True)
                for idx, _ in sent_scores[:n_fix]:
                    sent = sentences[idx]
                    for phrase in sorted_phrases:
                        if phrase in sent:
                            alternatives = ACADEMIC_REPLACEMENTS[phrase]
                            if isinstance(alternatives, str):
                                alternatives = [alternatives]
                            best = pick_best_replacement(sent, phrase, alternatives)
                            sentences[idx] = sent.replace(phrase, best, 1)
                            break

                text = ''.join(sentences)

    return text.strip()


# ═══════════════════════════════════════════════════════════════════
#  Output Formatting
# ═══════════════════════════════════════════════════════════════════

DIMENSION_NAMES = {
    'ai_academic_phrases':   ('🔴', 'AI 学术措辞'),
    'passive_overuse':       ('🟠', '被动句式过度'),
    'paragraph_uniformity':  ('🟠', '段落结构整齐'),
    'connector_density':     ('🟠', '逻辑连接词密集'),
    'synonym_poverty':       ('🟡', '同义表达匮乏'),
    'citation_integration':  ('🟡', '引用整合度低'),
    'data_template':         ('🟡', '数据论述模板化'),
    'over_enumeration':      ('🟡', '过度列举'),
    'perfect_conclusion':    ('🟠', '结论过于圆满'),
    'certainty_overuse':     ('🟠', '语气过于确定'),
    # Statistical features
    'stat_low_perplexity':   ('📊', '困惑度异常低'),
    'stat_low_burstiness':   ('📊', '困惑度变化均匀'),
    'stat_uniform_entropy':  ('📊', '段落熵值均匀'),
}


def format_detect_output(issues, metrics, score, as_json=False, score_only=False, verbose=False):
    level = score_to_level(score)
    total_issues = sum(len(v) for v in issues.values())

    if score_only:
        return f'{score}/100 ({level})'

    if as_json:
        result = {
            'score': score,
            'level': level,
            'metrics': metrics,
            'total_issues': total_issues,
            'issues': {},
        }
        for cat, items in issues.items():
            result['issues'][cat] = [
                {'text': it['text'], 'count': it.get('count', 1), 'severity': it['severity']}
                for it in items
            ]
        return json.dumps(result, ensure_ascii=False, indent=2)

    lines = []
    bar_len = 20
    filled = int(score / 100 * bar_len)
    bar = '█' * filled + '░' * (bar_len - filled)
    lines.append(f'学术 AIGC 评分: {score}/100 [{bar}] {level.upper().replace("_", " ")}')
    lines.append(f'字符: {metrics["char_count"]} | 句子: {metrics["sentence_count"]} | 段落: {metrics["paragraph_count"]}')
    if metrics.get('entropy'):
        lines.append(f'信息熵: {metrics["entropy"]:.2f} | 连接词密度: {metrics["connector_density"]:.1f}/百字')
    lines.append(f'确定性表述: {metrics["certainty_count"]} | 学术留白: {metrics["hedging_count"]} | 引用: {metrics["citation_count"]}')
    if metrics.get('perplexity'):
        lines.append(f'困惑度: {metrics["perplexity"]:.1f} | 突发度: {metrics.get("burstiness", 0):.3f} | 段落熵CV: {metrics.get("entropy_cv", 0):.3f}')
    lines.append(f'问题总数: {total_issues}')
    lines.append('')

    dim_order = [
        'ai_academic_phrases', 'passive_overuse', 'paragraph_uniformity',
        'connector_density', 'synonym_poverty', 'citation_integration',
        'data_template', 'over_enumeration', 'perfect_conclusion', 'certainty_overuse',
        'stat_low_perplexity', 'stat_low_burstiness', 'stat_uniform_entropy',
    ]
    for dim in dim_order:
        if dim not in issues or not issues[dim]:
            continue
        icon, name = DIMENSION_NAMES.get(dim, ('⚪', dim))
        items = issues[dim]
        # Count non-info items
        real_items = [it for it in items if it.get('severity') != 'info']
        lines.append(f'{icon} {name} ({len(real_items)})')
        show_count = 8 if verbose else 4
        for item in items[:show_count]:
            sev = item.get('severity', 'medium')
            if sev == 'info':
                lines.append(f'   {item["text"]}')
            else:
                count_str = f' ×{item["count"]}' if item.get('count', 1) > 1 else ''
                lines.append(f'   {item["text"]}{count_str}')
        if len(items) > show_count:
            lines.append(f'   ... 还有 {len(items) - show_count} 项')
        lines.append('')

    return '\n'.join(lines)


def format_comparison(before_issues, before_metrics, before_score,
                      after_issues, after_metrics, after_score):
    """Format before/after comparison."""
    lines = []
    b_level = score_to_level(before_score)
    a_level = score_to_level(after_score)

    b_bar = '█' * int(before_score / 100 * 20) + '░' * (20 - int(before_score / 100 * 20))
    a_bar = '█' * int(after_score / 100 * 20) + '░' * (20 - int(after_score / 100 * 20))

    lines.append('═══ 学术 AIGC 对比结果 ═══\n')
    lines.append(f'原文:   {before_score:3d}/100 [{b_bar}] {b_level.upper().replace("_"," ")}')
    lines.append(f'改写后: {after_score:3d}/100 [{a_bar}] {a_level.upper().replace("_"," ")}')

    diff = before_score - after_score
    if diff > 0:
        lines.append(f'\n✅ 降低了 {diff} 分')
    elif diff == 0:
        lines.append(f'\n⚠️  分数未变化')
    else:
        lines.append(f'\n❌ 分数上升了 {abs(diff)} 分')

    # Dimension breakdown
    all_dims = set(list(before_issues.keys()) + list(after_issues.keys()))
    if all_dims:
        lines.append('\n── 各维度变化 ──')
        for dim in ['ai_academic_phrases', 'passive_overuse', 'paragraph_uniformity',
                     'connector_density', 'synonym_poverty', 'citation_integration',
                     'data_template', 'over_enumeration', 'perfect_conclusion', 'certainty_overuse']:
            if dim not in all_dims:
                continue
            b_count = len([i for i in before_issues.get(dim, []) if i.get('severity') != 'info'])
            a_count = len([i for i in after_issues.get(dim, []) if i.get('severity') != 'info'])
            if b_count == 0 and a_count == 0:
                continue
            _, name = DIMENSION_NAMES.get(dim, ('', dim))
            status = '✅' if a_count < b_count else ('➖' if a_count == b_count else '❌')
            lines.append(f'  {status} {name}: {b_count} → {a_count}')

    # Key metrics
    lines.append('\n── 关键指标 ──')
    lines.append(f'  确定性表述: {before_metrics["certainty_count"]} → {after_metrics["certainty_count"]}')
    lines.append(f'  学术留白:   {before_metrics["hedging_count"]} → {after_metrics["hedging_count"]}')
    lines.append(f'  连接词密度: {before_metrics["connector_density"]:.1f} → {after_metrics["connector_density"]:.1f}/百字')

    return '\n'.join(lines)


# ═══════════════════════════════════════════════════════════════════
#  Main
# ═══════════════════════════════════════════════════════════════════

def main():
    parser = argparse.ArgumentParser(
        description='中文学术论文 AIGC 检测与改写工具 v1.0 — 针对知网/维普检测标准优化',
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog='''示例:
  python academic_cn.py paper.txt                    # 检测 + 改写
  python academic_cn.py paper.txt --detect-only      # 仅检测
  python academic_cn.py paper.txt -o clean.txt       # 改写并保存
  python academic_cn.py paper.txt -o clean.txt -a    # 激进模式
  python academic_cn.py paper.txt -o clean.txt --compare  # 对比模式
''')
    parser.add_argument('file', nargs='?', help='输入文件路径（不指定则从 stdin 读取）')
    parser.add_argument('-o', '--output', help='输出文件路径')
    parser.add_argument('--detect-only', action='store_true', help='仅检测，不改写')
    parser.add_argument('-a', '--aggressive', action='store_true', help='激进模式（改写更多）')
    parser.add_argument('--compare', action='store_true', help='对比模式（显示改写前后评分变化）')
    parser.add_argument('-j', '--json', action='store_true', help='JSON 输出')
    parser.add_argument('-s', '--score', action='store_true', help='仅输出分数')
    parser.add_argument('-v', '--verbose', action='store_true', help='详细模式')
    parser.add_argument('--seed', type=int, help='随机种子（可复现）')
    parser.add_argument('--no-stats', action='store_true',
                       help='跳过统计优化（困惑度反馈），回退到纯规则替换')
    parser.add_argument('--no-noise', action='store_true',
                       help='跳过噪声策略（句长随机化 + 噪声表达插入）')

    args = parser.parse_args()

    # Toggle stats optimization
    global _USE_STATS
    _USE_STATS = not args.no_stats

    # Toggle noise strategies
    global _USE_NOISE
    _USE_NOISE = not args.no_noise

    # Read input
    if args.file:
        try:
            with open(args.file, 'r', encoding='utf-8') as f:
                text = f.read()
        except FileNotFoundError:
            print(f'错误: 文件未找到 {args.file}', file=sys.stderr)
            sys.exit(1)
    else:
        text = sys.stdin.read()

    if not text.strip():
        print('错误: 输入为空', file=sys.stderr)
        sys.exit(1)

    # Detect
    issues, metrics = detect_academic(text)
    score = calculate_academic_score(issues)

    if args.detect_only or (not args.output and not args.compare):
        # Detection only mode
        output = format_detect_output(issues, metrics, score,
                                      as_json=args.json, score_only=args.score, verbose=args.verbose)
        print(output)
        return

    # Humanize
    humanized = humanize_academic(text, aggressive=args.aggressive, seed=args.seed)

    if args.compare:
        # Run detection on humanized text
        after_issues, after_metrics = detect_academic(humanized)
        after_score = calculate_academic_score(after_issues)
        print(format_comparison(issues, metrics, score, after_issues, after_metrics, after_score))

    # Save output
    if args.output:
        with open(args.output, 'w', encoding='utf-8') as f:
            f.write(humanized)
        mode = ' (激进模式)' if args.aggressive else ''
        print(f'✓ 已保存到 {args.output}{mode}')

        if not args.compare:
            # Quick score comparison
            after_issues, after_metrics = detect_academic(humanized)
            after_score = calculate_academic_score(after_issues)
            diff = score - after_score
            print(f'  原文评分: {score}/100 → 改写后: {after_score}/100', end='')
            if diff > 0:
                print(f'  ✅ 降低了 {diff} 分')
            else:
                print()
    else:
        # No output file, no compare — print humanized text
        print(humanized)


if __name__ == '__main__':
    main()


path: scripts/compare_cn.py
content: #!/usr/bin/env python3
"""
Compare AI detection scores before and after humanization v2.0
Shows detailed diff of score changes by category
"""

import sys
import subprocess
import os
import json
import argparse

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))

def run_detect(text, as_json=True):
    """Run detect_cn.py and get results"""
    detect_script = os.path.join(SCRIPT_DIR, 'detect_cn.py')
    cmd = ['python3', detect_script]
    if as_json:
        cmd.append('-j')
    
    try:
        result = subprocess.run(
            cmd, input=text, capture_output=True,
            text=True, encoding='utf-8', timeout=30
        )
        if as_json and result.returncode == 0:
            return json.loads(result.stdout)
        return result.stdout.strip()
    except Exception as e:
        return {'error': str(e)}

def run_humanize(text, scene='general', aggressive=False, style=None):
    """Run humanize_cn.py and get result"""
    humanize_script = os.path.join(SCRIPT_DIR, 'humanize_cn.py')
    cmd = ['python3', humanize_script, '--scene', scene]
    if aggressive:
        cmd.append('-a')
    if style:
        cmd.extend(['--style', style])
    
    try:
        result = subprocess.run(
            cmd, input=text, capture_output=True,
            text=True, encoding='utf-8', timeout=30
        )
        return result.stdout
    except Exception as e:
        return f'error: {e}'

def format_comparison(before, after):
    """Format detailed comparison"""
    lines = []
    
    b_score = before.get('score', 0)
    a_score = after.get('score', 0)
    b_level = before.get('level', 'unknown')
    a_level = after.get('level', 'unknown')
    
    # Score comparison bar
    b_bar_len = int(b_score / 100 * 20)
    a_bar_len = int(a_score / 100 * 20)
    
    lines.append('═══ 对比结果 ═══\n')
    lines.append(f'原文:   {b_score:3d}/100 [{"█" * b_bar_len}{"░" * (20 - b_bar_len)}] {b_level.upper()}')
    lines.append(f'改写后: {a_score:3d}/100 [{"█" * a_bar_len}{"░" * (20 - a_bar_len)}] {a_level.upper()}')
    
    diff = b_score - a_score
    if diff > 0:
        lines.append(f'\n✅ 降低了 {diff} 分')
    elif diff == 0:
        lines.append(f'\n⚠️  分数未变化')
    else:
        lines.append(f'\n❌ 分数上升了 {abs(diff)} 分')
    
    # Category breakdown
    b_issues = before.get('issues', {})
    a_issues = after.get('issues', {})
    all_cats = set(list(b_issues.keys()) + list(a_issues.keys()))
    
    if all_cats:
        lines.append('\n── 分类变化 ──')
        for cat in sorted(all_cats):
            b_count = len(b_issues.get(cat, []))
            a_count = len(a_issues.get(cat, []))
            
            if b_count == 0 and a_count == 0:
                continue
            
            if a_count < b_count:
                status = '✅'
            elif a_count == b_count:
                status = '➖'
            else:
                status = '❌'
            
            lines.append(f'  {status} {cat}: {b_count} → {a_count}')
    
    # Metrics comparison
    b_metrics = before.get('metrics', {})
    a_metrics = after.get('metrics', {})
    
    if b_metrics and a_metrics:
        lines.append('\n── 指标变化 ──')
        b_emo = b_metrics.get('emotional_density', 0)
        a_emo = a_metrics.get('emotional_density', 0)
        lines.append(f'  情感密度: {b_emo:.2f}% → {a_emo:.2f}%')
        
        b_ent = b_metrics.get('entropy')
        a_ent = a_metrics.get('entropy')
        if b_ent and a_ent:
            lines.append(f'  信息熵:   {b_ent:.2f} → {a_ent:.2f}')
    
    return '\n'.join(lines)

def main():
    parser = argparse.ArgumentParser(description='中文 AI 文本对比分析 v2.0')
    parser.add_argument('file', nargs='?', help='输入文件路径')
    parser.add_argument('-o', '--output', help='保存改写结果')
    parser.add_argument('--scene', default='general',
                       choices=['general', 'social', 'tech', 'formal', 'chat'],
                       help='场景')
    parser.add_argument('--style', help='写作风格')
    parser.add_argument('-a', '--aggressive', action='store_true', help='激进模式')
    
    args = parser.parse_args()
    
    # Read input
    if args.file:
        try:
            with open(args.file, 'r', encoding='utf-8') as f:
                original_text = f.read()
        except FileNotFoundError:
            print(f'错误: 文件未找到 {args.file}', file=sys.stderr)
            sys.exit(1)
    else:
        original_text = sys.stdin.read()
    
    if not original_text.strip():
        print('错误: 输入为空', file=sys.stderr)
        sys.exit(1)
    
    # Detect original
    print('⏳ 检测原文...')
    before = run_detect(original_text, as_json=True)
    
    # Humanize
    print('⏳ 人性化改写...')
    humanized_text = run_humanize(original_text, args.scene, args.aggressive, args.style)
    
    # Detect humanized
    print('⏳ 检测改写后...')
    after = run_detect(humanized_text, as_json=True)
    
    # Show comparison
    if isinstance(before, dict) and isinstance(after, dict) and 'error' not in before:
        print(format_comparison(before, after))
    else:
        print(f'\n原文分数: {before}')
        print(f'改写后分数: {after}')
    
    # Save output
    if args.output:
        with open(args.output, 'w', encoding='utf-8') as f:
            f.write(humanized_text)
        print(f'\n改写结果已保存到: {args.output}')
    else:
        print('\n═══ 改写后文本 ═══\n')
        print(humanized_text)

if __name__ == '__main__':
    main()


path: scripts/detect_cn.py
content: #!/usr/bin/env python3
"""
Chinese AI Text Detector v2.0
Scans for AI-generated patterns in Chinese text
Features: 20+ detection categories, weighted 0-100 scoring, sentence-level analysis
"""

import sys
import json
import re
import os
import argparse
from collections import defaultdict
from math import log2

# Import n-gram statistical model
try:
    from ngram_model import analyze_text as ngram_analyze
except ImportError:
    try:
        from scripts.ngram_model import analyze_text as ngram_analyze
    except ImportError:
        ngram_analyze = None

# Load patterns from JSON config
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
PATTERNS_FILE = os.path.join(SCRIPT_DIR, 'patterns_cn.json')

def load_patterns():
    """Load patterns from external config, fall back to built-in"""
    if os.path.exists(PATTERNS_FILE):
        with open(PATTERNS_FILE, 'r', encoding='utf-8') as f:
            return json.load(f)
    return None

CONFIG = load_patterns()

# ─── Detection Patterns ───

CRITICAL_PHRASES = {
    'mechanical_connectors': CONFIG['critical_patterns']['mechanical_connectors']['phrases'] if CONFIG else [
        '值得注意的是', '综上所述', '不难发现', '总而言之',
        '与此同时', '在此基础上', '由此可见', '不仅如此',
        '换句话说', '更重要的是', '需要强调的是', '不可否认',
        '显而易见', '不言而喻', '正如我们所知', '归根结底',
    ],
    'empty_grand_words': CONFIG['critical_patterns']['empty_grand_words']['phrases'] if CONFIG else [
        '赋能', '闭环', '智慧时代', '数字化转型', '生态',
        '愿景', '顶层设计', '协同增效', '降本增效', '打通壁垒',
        '深度融合', '创新驱动', '全方位', '多维度', '系统性',
    ],
}

CRITICAL_REGEX = CONFIG['critical_patterns']['three_part_structure']['regex'] if CONFIG else [
    r'首先[，,].*?其次[，,].*?最后',
    r'一方面[，,].*?另一方面',
    r'第一[，,].*?第二[，,].*?第三',
    r'其一[，,].*?其二[，,].*?其三',
]

HIGH_SIGNAL_PHRASES = {
    'ai_high_freq_words': CONFIG['high_signal_patterns']['ai_high_freq_words']['phrases'] if CONFIG else [
        '助力', '彰显', '凸显', '焕发', '深度剖析',
        '加持', '赛道', '破圈', '出圈', '颠覆', '革新',
        '底层逻辑', '抓手', '链路', '触达', '心智',
        '沉淀', '对齐', '拉通', '复盘', '迭代',
    ],
    'filler_phrases': CONFIG['high_signal_patterns']['filler_phrases']['phrases'] if CONFIG else [
        '值得一提的是', '需要指出的是', '不得不说',
        '毫无疑问', '众所周知', '正因如此',
        '具体来说', '简而言之', '换言之',
    ],
}

HIGH_SIGNAL_REGEX = {
    'balanced_arguments': CONFIG['high_signal_patterns']['balanced_arguments']['regex'] if CONFIG else [
        r'虽然.*?但是.*?同时',
        r'一方面.*?另一方面.*?总的来说',
    ],
    'template_sentences': CONFIG['high_signal_patterns']['template_sentences']['regex'] if CONFIG else [
        r'随着.*?的(不断)?发展',
        r'在.*?的背景下',
        r'在当今.*?时代',
        r'作为.*?的重要(组成部分|环节|手段)',
        r'对于.*?而言[，,].*?至关重要',
        r'这不仅.*?更是',
        r'从.*?角度(来看|来说|而言)',
        r'无论是.*?还是.*?都',
        r'可以说[，,]',
        r'总的来说[，,]',
    ],
}

HEDGING_PHRASES = CONFIG['medium_signal_patterns']['hedging_language']['phrases'] if CONFIG else [
    '在一定程度上', '或许', '某种程度上', '相对而言',
    '总体来说', '一般来说', '通常情况下',
]

EMOTIONAL_WORDS = CONFIG['style_signals']['emotional_words'] if CONFIG else [
    '愤怒', '高兴', '难过', '失望', '惊讶', '担心',
    '开心', '郁闷', '焦虑', '兴奋', '害怕', '感动',
    '烦躁', '痛苦', '崩溃', '无奈', '委屈', '舒服',
]

PERSONAL_MARKERS = CONFIG['style_signals']['personal_markers'] if CONFIG else [
    '我觉得', '我认为', '我个人', '在我看来', '说实话',
    '坦白讲', '老实说', '不瞒你说',
]

# ─── Utility ───

def count_chinese_chars(text):
    return len(re.findall(r'[\u4e00-\u9fff]', text))

def split_sentences(text):
    """Split text into sentences, handling Chinese punctuation"""
    parts = re.split(r'([。！？\n])', text)
    sentences = []
    for i in range(0, len(parts) - 1, 2):
        s = parts[i].strip()
        if s:
            sentences.append(s + (parts[i+1] if i+1 < len(parts) else ''))
    # Handle last part if odd
    if len(parts) % 2 == 1 and parts[-1].strip():
        sentences.append(parts[-1].strip())
    return [s for s in sentences if len(s) > 1]

def char_entropy(text):
    """Calculate character-level entropy (proxy for perplexity)"""
    chars = re.findall(r'[\u4e00-\u9fff]', text)
    if len(chars) < 10:
        return 5.0  # Default
    
    # Bigram entropy
    bigrams = defaultdict(int)
    for i in range(len(chars) - 1):
        bigrams[chars[i] + chars[i+1]] += 1
    
    total = sum(bigrams.values())
    if total == 0:
        return 5.0
    
    entropy = 0
    for count in bigrams.values():
        p = count / total
        if p > 0:
            entropy -= p * log2(p)
    
    return entropy

# ─── Detection Engine ───

def detect_patterns(text):
    """Detect AI patterns in Chinese text. Returns (issues, metrics)"""
    issues = defaultdict(list)
    char_count = count_chinese_chars(text)
    sentences = split_sentences(text)
    
    # ── Critical: Three-part structure ──
    for pattern in CRITICAL_REGEX:
        for match in re.finditer(pattern, text, re.DOTALL):
            issues['three_part_structure'].append({
                'text': match.group()[:60],
                'severity': 'critical',
            })
    
    # ── Critical: Mechanical connectors ──
    for phrase in CRITICAL_PHRASES['mechanical_connectors']:
        count = text.count(phrase)
        if count > 0:
            issues['mechanical_connectors'].append({
                'text': phrase,
                'count': count,
                'severity': 'critical',
            })
    
    # ── Critical: Empty grand words ──
    for word in CRITICAL_PHRASES['empty_grand_words']:
        count = text.count(word)
        if count > 0:
            issues['empty_grand_words'].append({
                'text': word,
                'count': count,
                'severity': 'critical',
            })
    
    # ── High: AI high-frequency words ──
    for word in HIGH_SIGNAL_PHRASES['ai_high_freq_words']:
        count = text.count(word)
        if count > 0:
            issues['ai_high_freq_words'].append({
                'text': word,
                'count': count,
                'severity': 'high',
            })
    
    # ── High: Filler phrases ──
    for phrase in HIGH_SIGNAL_PHRASES['filler_phrases']:
        count = text.count(phrase)
        if count > 0:
            issues['filler_phrases'].append({
                'text': phrase,
                'count': count,
                'severity': 'high',
            })
    
    # ── High: Balanced arguments (AI loves both-sides) ──
    for pattern in HIGH_SIGNAL_REGEX.get('balanced_arguments', []):
        for match in re.finditer(pattern, text, re.DOTALL):
            issues['balanced_arguments'].append({
                'text': match.group()[:80],
                'severity': 'high',
            })
    
    # ── High: Template sentences ──
    for pattern in HIGH_SIGNAL_REGEX.get('template_sentences', []):
        for match in re.finditer(pattern, text):
            issues['template_sentences'].append({
                'text': match.group()[:60],
                'severity': 'high',
            })
    
    # ── Medium: Hedging/over-cautious language ──
    hedge_count = 0
    for phrase in HEDGING_PHRASES:
        c = text.count(phrase)
        if c > 0:
            hedge_count += c
    threshold = CONFIG['medium_signal_patterns']['hedging_language'].get('threshold', 5) if CONFIG else 5
    if hedge_count >= threshold:
        issues['hedging_language'].append({
            'text': f'谨慎用语 {hedge_count} 次（阈值 {threshold}）',
            'count': hedge_count,
            'severity': 'medium',
        })
    
    # ── Medium: List addiction ──
    list_patterns = [
        r'(?:(?:①|②|③|④|⑤).*?\n){3,}',
        r'(?:(?:\d\.)\s.*?\n){3,}',
        r'(?:(?:[-•·])\s.*?\n){4,}',
    ]
    for pattern in list_patterns:
        for match in re.finditer(pattern, text):
            issues['list_addiction'].append({
                'text': '列举过多',
                'severity': 'medium',
            })
    
    # ── Medium: Punctuation overuse ──
    if char_count > 0:
        em_dash_count = text.count('—')
        semicolon_count = text.count('；')
        
        if em_dash_count / char_count * 100 > 1.0:
            issues['punctuation_overuse'].append({
                'text': f'破折号 {em_dash_count} 个',
                'severity': 'medium',
            })
        if semicolon_count / char_count * 100 > 0.5:
            issues['punctuation_overuse'].append({
                'text': f'分号 {semicolon_count} 个',
                'severity': 'medium',
            })
    
    # ── Medium: Parallel structures (excessive rhetoric) ──
    parallel_pattern = r'[，,][^，,。！？]{4,10}[；;，,][^，,。！？]{4,10}[。！？]'
    parallels = re.findall(parallel_pattern, text)
    if len(parallels) > 2:
        issues['excessive_rhetoric'].append({
            'text': f'对偶/排比 {len(parallels)} 处',
            'severity': 'medium',
        })
    
    # ── Style: Uniform paragraph lengths ──
    paragraphs = [p.strip() for p in text.split('\n\n') if p.strip() and len(p.strip()) > 20]
    if len(paragraphs) >= 3:
        lengths = [len(p) for p in paragraphs]
        avg_len = sum(lengths) / len(lengths)
        if avg_len > 0:
            cv = (sum((l - avg_len) ** 2 for l in lengths) / len(lengths)) ** 0.5 / avg_len
            if cv < 0.2:  # Coefficient of variation < 20%
                issues['uniform_paragraphs'].append({
                    'text': f'段落长度 CV={cv:.2f}，过于均匀',
                    'severity': 'style',
                })
    
    # ── Style: Low burstiness (sentence length uniformity) ──
    if len(sentences) > 5:
        sent_lens = [count_chinese_chars(s) for s in sentences]
        avg_sl = sum(sent_lens) / len(sent_lens)
        if avg_sl > 0:
            cv_sent = (sum((l - avg_sl) ** 2 for l in sent_lens) / len(sent_lens)) ** 0.5 / avg_sl
            if cv_sent < 0.25:
                issues['low_burstiness'].append({
                    'text': f'句子长度 CV={cv_sent:.2f}，缺少节奏变化',
                    'severity': 'style',
                })
    
    # ── Style: Emotional flatness ──
    emotional_count = sum(text.count(w) for w in EMOTIONAL_WORDS)
    personal_count = sum(text.count(w) for w in PERSONAL_MARKERS)
    if char_count > 300:
        emotional_density = (emotional_count + personal_count) / char_count * 100
        if emotional_density < 0.15:
            issues['emotional_flatness'].append({
                'text': f'情感/个人表达密度 {emotional_density:.2f}%，偏低',
                'severity': 'style',
            })
    
    # ── Style: Repetitive sentence starters ──
    if len(sentences) > 5:
        starters = defaultdict(int)
        for s in sentences:
            s = s.strip()
            if len(s) >= 2:
                starters[s[:2]] += 1
        max_repeat = max(starters.values()) if starters else 0
        if max_repeat >= 3:
            top_starter = max(starters, key=starters.get)
            issues['repetitive_starters'].append({
                'text': f'"{top_starter}..." 出现 {max_repeat} 次',
                'severity': 'style',
            })
    
    # ── Style: Low entropy (predictable) ──
    entropy = char_entropy(text)
    if entropy < 6.0 and char_count > 200:
        issues['low_entropy'].append({
            'text': f'字符熵 {entropy:.2f}（越低越可预测）',
            'severity': 'style',
        })
    
    # ── Statistical: N-gram perplexity features ──
    ngram_stats = None
    if ngram_analyze and char_count >= 100:
        ngram_stats = ngram_analyze(text)
        indicators = ngram_stats.get('indicators', {})
        
        if indicators.get('low_perplexity'):
            ppl = ngram_stats['perplexity']
            issues['stat_low_perplexity'].append({
                'text': f'文本困惑度 {ppl:.1f}（AI 文本通常在此范围内）',
                'severity': 'statistical',
            })
        
        if indicators.get('low_burstiness'):
            burst = ngram_stats['burstiness']
            issues['stat_low_burstiness'].append({
                'text': f'困惑度变异系数 {burst:.3f}（过于均匀，缺少人类写作的起伏）',
                'severity': 'statistical',
            })
        
        if indicators.get('uniform_entropy'):
            ent_cv = ngram_stats['entropy_cv']
            issues['stat_uniform_entropy'].append({
                'text': f'段落熵变异系数 {ent_cv:.3f}（段落间复杂度过于一致）',
                'severity': 'statistical',
            })
    
    # ── Compute metrics ──
    metrics = {
        'char_count': char_count,
        'sentence_count': len(sentences),
        'paragraph_count': len(paragraphs),
        'entropy': entropy if char_count > 200 else None,
        'emotional_density': (emotional_count + personal_count) / char_count * 100 if char_count > 0 else 0,
    }
    
    # Add statistical metrics if available
    if ngram_stats:
        metrics['perplexity'] = ngram_stats['perplexity']
        metrics['burstiness'] = ngram_stats['burstiness']
        metrics['entropy_cv'] = ngram_stats['entropy_cv']
    
    return issues, metrics

# ─── Scoring ───

SEVERITY_WEIGHTS = {
    'critical': 8,
    'high': 4,
    'medium': 2,
    'style': 1.5,
    'statistical': 0,  # statistical features scored separately below
}

# Statistical feature weights (scored independently, contribute 20-30% of final score)
STATISTICAL_WEIGHTS = {
    'stat_low_perplexity': 12,
    'stat_low_burstiness': 10,
    'stat_uniform_entropy': 8,
}

def calculate_score(issues, metrics):
    """
    Calculate AI probability score (0-100).
    Higher = more likely AI-generated.
    
    Scoring composition:
      - Rule-based patterns: ~70-80% weight
      - Statistical features (perplexity/burstiness/entropy): ~20-30% weight
    """
    raw = 0
    
    for category, items in issues.items():
        # Skip statistical items — scored separately
        if category.startswith('stat_'):
            continue
        for item in items:
            severity = item.get('severity', 'medium')
            weight = SEVERITY_WEIGHTS.get(severity, 2)
            count = item.get('count', 1)
            # Diminishing returns for repeated items
            raw += weight * min(count, 5)
    
    # Rule-based score (cap contribution at ~75 points)
    rule_score = min(75, int(raw * 1.0))
    
    # Statistical score (up to 25 points)
    stat_score = 0
    for category, items in issues.items():
        if category.startswith('stat_') and items:
            stat_score += STATISTICAL_WEIGHTS.get(category, 5)
    stat_score = min(25, stat_score)
    
    score = rule_score + stat_score
    
    # Bonus penalties
    if metrics.get('emotional_density', 1) < 0.1 and metrics['char_count'] > 500:
        score = min(100, score + 5)
    
    if metrics.get('entropy') and metrics['entropy'] < 5.5:
        score = min(100, score + 5)
    
    return min(100, score)

def score_to_level(score):
    """Convert numeric score to level"""
    if score >= 75:
        return 'very_high'
    elif score >= 50:
        return 'high'
    elif score >= 25:
        return 'medium'
    else:
        return 'low'

# ─── Sentence-level Analysis ───

def analyze_sentences(text, top_n=5):
    """Find the most AI-like sentences"""
    sentences = split_sentences(text)
    scored = []
    
    all_bad_phrases = (
        CRITICAL_PHRASES['mechanical_connectors'] +
        CRITICAL_PHRASES['empty_grand_words'] +
        HIGH_SIGNAL_PHRASES['ai_high_freq_words'] +
        HIGH_SIGNAL_PHRASES['filler_phrases']
    )
    
    all_bad_regex = CRITICAL_REGEX + HIGH_SIGNAL_REGEX.get('template_sentences', [])
    
    for sent in sentences:
        s = 0
        reasons = []
        
        # Check phrases
        for phrase in all_bad_phrases:
            if phrase in sent:
                s += 3
                reasons.append(phrase)
        
        # Check regex
        for pattern in all_bad_regex:
            if re.search(pattern, sent):
                s += 5
                reasons.append(f'模板: {pattern[:20]}')
        
        if s > 0:
            scored.append({
                'sentence': sent[:80],
                'score': s,
                'reasons': reasons[:3],
            })
    
    scored.sort(key=lambda x: x['score'], reverse=True)
    return scored[:top_n]

# ─── Output Formatting ───

CATEGORY_NAMES = {
    'three_part_structure': ('🔴', '三段式套路'),
    'mechanical_connectors': ('🔴', '机械连接词'),
    'empty_grand_words': ('🔴', '空洞宏大词'),
    'ai_high_freq_words': ('🟠', 'AI 高频词'),
    'filler_phrases': ('🟠', '套话/废话'),
    'balanced_arguments': ('🟠', '过度两面论'),
    'template_sentences': ('🟠', '模板句式'),
    'hedging_language': ('🟡', '过度谨慎'),
    'list_addiction': ('🟡', '列举上瘾'),
    'punctuation_overuse': ('🟡', '标点过度'),
    'excessive_rhetoric': ('🟡', '修辞过多'),
    'uniform_paragraphs': ('⚪', '段落均匀'),
    'low_burstiness': ('⚪', '节奏单一'),
    'emotional_flatness': ('⚪', '情感平淡'),
    'repetitive_starters': ('⚪', '句首重复'),
    'low_entropy': ('⚪', '低信息熵'),
    # Statistical features (n-gram perplexity)
    'stat_low_perplexity': ('📊', '困惑度异常低'),
    'stat_low_burstiness': ('📊', '困惑度变化均匀'),
    'stat_uniform_entropy': ('📊', '段落熵值均匀'),
}

def format_output(issues, metrics, score, sentences=None, as_json=False, score_only=False, verbose=False):
    total_issues = sum(len(v) for v in issues.values())
    level = score_to_level(score)
    
    if score_only:
        return f'{score}/100 ({level})'
    
    if as_json:
        result = {
            'score': score,
            'level': level,
            'metrics': metrics,
            'total_issues': total_issues,
            'issues': {},
        }
        for cat, items in issues.items():
            result['issues'][cat] = [
                {'text': it['text'], 'count': it.get('count', 1), 'severity': it['severity']}
                for it in items
            ]
        if sentences:
            result['worst_sentences'] = sentences
        return json.dumps(result, ensure_ascii=False, indent=2)
    
    # Human-readable
    lines = []
    
    # Score bar
    bar_len = 20
    filled = int(score / 100 * bar_len)
    bar = '█' * filled + '░' * (bar_len - filled)
    lines.append(f'AI 评分: {score}/100 [{bar}] {level.upper().replace("_", " ")}')
    lines.append(f'字符: {metrics["char_count"]} | 句子: {metrics["sentence_count"]} | 段落: {metrics["paragraph_count"]}')
    if metrics.get('entropy'):
        lines.append(f'信息熵: {metrics["entropy"]:.2f} | 情感密度: {metrics["emotional_density"]:.2f}%')
    lines.append(f'问题总数: {total_issues}')
    lines.append('')
    
    # Statistical metrics line
    if metrics.get('perplexity'):
        lines.append(f'困惑度: {metrics["perplexity"]:.1f} | 突发度: {metrics.get("burstiness", 0):.3f} | 段落熵CV: {metrics.get("entropy_cv", 0):.3f}')
        lines.append('')
    
    # Issues by severity
    for cat in ['three_part_structure', 'mechanical_connectors', 'empty_grand_words',
                'ai_high_freq_words', 'filler_phrases', 'balanced_arguments', 'template_sentences',
                'hedging_language', 'list_addiction', 'punctuation_overuse', 'excessive_rhetoric',
                'stat_low_perplexity', 'stat_low_burstiness', 'stat_uniform_entropy',
                'uniform_paragraphs', 'low_burstiness', 'emotional_flatness', 'repetitive_starters', 'low_entropy']:
        if cat not in issues or not issues[cat]:
            continue
        icon, name = CATEGORY_NAMES.get(cat, ('⚪', cat))
        items = issues[cat]
        lines.append(f'{icon} {name} ({len(items)})')
        show_count = 5 if verbose else 3
        for item in items[:show_count]:
            count_str = f' ×{item["count"]}' if item.get('count', 1) > 1 else ''
            lines.append(f'   {item["text"]}{count_str}')
        if len(items) > show_count:
            lines.append(f'   ... 还有 {len(items) - show_count} 个')
        lines.append('')
    
    # Worst sentences
    if sentences and verbose:
        lines.append('── 最可疑句子 ──')
        for i, s in enumerate(sentences, 1):
            lines.append(f'  {i}. [{s["score"]}分] {s["sentence"]}')
            lines.append(f'     原因: {", ".join(s["reasons"])}')
        lines.append('')
    
    return '\n'.join(lines)

# ─── Main ───

def main():
    parser = argparse.ArgumentParser(description='中文 AI 文本检测 v2.0')
    parser.add_argument('file', nargs='?', help='输入文件路径（不指定则从 stdin 读取）')
    parser.add_argument('-j', '--json', action='store_true', help='JSON 输出')
    parser.add_argument('-s', '--score', action='store_true', help='仅输出分数')
    parser.add_argument('-v', '--verbose', action='store_true', help='详细模式（含逐句分析）')
    parser.add_argument('--sentences', type=int, default=5, help='显示最可疑的 N 个句子')
    
    args = parser.parse_args()
    
    # Read input
    if args.file:
        try:
            with open(args.file, 'r', encoding='utf-8') as f:
                text = f.read()
        except FileNotFoundError:
            print(f'错误: 文件未找到 {args.file}', file=sys.stderr)
            sys.exit(1)
    else:
        text = sys.stdin.read()
    
    if not text.strip():
        print('错误: 输入为空', file=sys.stderr)
        sys.exit(1)
    
    # Detect
    issues, metrics = detect_patterns(text)
    score = calculate_score(issues, metrics)
    
    # Sentence analysis (verbose mode)
    worst_sentences = None
    if args.verbose or args.json:
        worst_sentences = analyze_sentences(text, args.sentences)
    
    # Output
    output = format_output(issues, metrics, score, worst_sentences,
                          as_json=args.json, score_only=args.score, verbose=args.verbose)
    print(output)

if __name__ == '__main__':
    main()


path: scripts/humanize_cn.py
content: #!/usr/bin/env python3
"""
Chinese AI Text Humanizer v2.0
Transforms AI-generated Chinese text to sound more natural
Features: sentence restructuring, rhythm variation, context-aware replacement, multi-pass
"""

import sys
import re
import random
import json
import os
import argparse

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))

# Module-level flag: whether to apply noise strategies (strategies 2 & 3)
_USE_NOISE = True

# Import n-gram statistical model for perplexity feedback
try:
    from ngram_model import analyze_text as ngram_analyze
except ImportError:
    try:
        from scripts.ngram_model import analyze_text as ngram_analyze
    except ImportError:
        ngram_analyze = None

# Module-level flag: whether to use stats optimization (can be toggled by CLI)
_USE_STATS = True
PATTERNS_FILE = os.path.join(SCRIPT_DIR, 'patterns_cn.json')

def load_config():
    if os.path.exists(PATTERNS_FILE):
        with open(PATTERNS_FILE, 'r', encoding='utf-8') as f:
            return json.load(f)
    return None

CONFIG = load_config()

# ─── Replacement Mappings ───

PHRASE_REPLACEMENTS = CONFIG['replacements'] if CONFIG else {
    '值得注意的是': ['注意', '要提醒的是', '特别说一下'],
    '综上所述': ['总之', '说到底', '简单讲'],
    '不难发现': ['可以看到', '很明显'],
    '总而言之': ['总之', '总的来说'],
    '与此同时': ['同时', '这时候'],
    '赋能': ['帮助', '提升', '支持'],
    '闭环': ['完整流程', '全链路'],
    '助力': ['帮助', '支持'],
}

# Regex-based replacements (key is regex pattern)
_REGEX_REPLACEMENTS = {}
PLAIN_REPLACEMENTS = {}

for key, val in PHRASE_REPLACEMENTS.items():
    # Check if key contains regex special chars suggesting it's a pattern
    if any(c in key for c in ['.*', '.+', '[', '(', '|', '\\']):
        _REGEX_REPLACEMENTS[key] = val
    else:
        PLAIN_REPLACEMENTS[key] = val

# Sort regex replacements by key length descending (longer patterns first)
REGEX_REPLACEMENTS = dict(sorted(_REGEX_REPLACEMENTS.items(), key=lambda x: len(x[0]), reverse=True))

# ─── Scene Configurations ───

SCENES = {
    'general': {
        'casualness': 0.3,
        'merge_short': True,
        'split_long': True,
        'rhythm_variation': True,
    },
    'social': {
        'casualness': 0.7,
        'merge_short': True,
        'split_long': True,
        'shorten_paragraphs': True,
        'add_casual': True,
        'rhythm_variation': True,
    },
    'tech': {
        'casualness': 0.3,
        'merge_short': True,
        'split_long': True,
        'keep_technical': True,
        'rhythm_variation': True,
    },
    'formal': {
        'casualness': 0.1,
        'merge_short': True,
        'split_long': True,
        'reduce_rhetoric': True,
        'rhythm_variation': True,
    },
    'chat': {
        'casualness': 0.8,
        'merge_short': True,
        'split_long': True,
        'shorten_paragraphs': True,
        'add_casual': True,
        'rhythm_variation': True,
    },
}

# ─── Stats-Optimized Selection ───

def pick_best_replacement(sentence, old, candidates):
    """从多个候选替换中选 perplexity 最高的（最不可预测 = 最像人写的）。
    当 _USE_STATS 关闭、ngram_analyze 不可用、或只有单候选时回退到 random.choice。"""
    if not _USE_STATS or not ngram_analyze or len(candidates) <= 1:
        return random.choice(candidates)

    best_candidate = candidates[0]
    best_ppl = 0

    for candidate in candidates:
        new_sentence = sentence.replace(old, candidate, 1)
        stats = ngram_analyze(new_sentence)
        ppl = stats.get('perplexity', 0)
        if ppl > best_ppl:
            best_ppl = ppl
            best_candidate = candidate

    return best_candidate


def _compute_burstiness(text):
    """计算文本的 burstiness（困惑度变异系数），用于句式重组判断。"""
    if not _USE_STATS or not ngram_analyze:
        return None
    stats = ngram_analyze(text)
    return stats.get('burstiness', None)


# ═══════════════════════════════════════════════════════════════════
#  Strategy 1: Low-frequency bigram injection — WORD_SYNONYMS table
# ═══════════════════════════════════════════════════════════════════

WORD_SYNONYMS = {
    # ── 逻辑连接 / 转折 ──
    '因此': ['所以', '因而', '为此', '故而'],
    '然而': ['不过', '但', '可是', '只是'],
    '但是': ['不过', '可是', '只是', '然而'],
    '虽然': ['尽管', '即便', '就算', '纵然'],
    '所以': ['因此', '因而', '故而', '于是'],
    '而且': ['并且', '况且', '何况', '再说'],
    '或者': ['要么', '抑或', '或是', '还是'],
    '如果': ['倘若', '假如', '若是', '要是'],
    '因为': ['由于', '缘于', '出于', '鉴于'],
    '尽管': ['虽然', '即便', '纵使', '就算'],
    # ── 动词 / 行为 ──
    '能够': ['可以', '得以', '足以', '有能力'],
    '进行': ['开展', '实施', '做', '搞'],
    '实现': ['达成', '做到', '完成', '办到'],
    '提高': ['提升', '增强', '改善', '拉高'],
    '发展': ['推进', '进展', '演进', '推动'],
    '影响': ['波及', '冲击', '左右', '触动'],
    '研究': ['探究', '考察', '审视', '钻研'],
    '表明': ['显示', '说明', '反映', '揭示'],
    '认为': ['觉得', '以为', '判断', '主张'],
    '需要': ['有必要', '须', '要', '得'],
    '使用': ['运用', '采用', '用', '动用'],
    '具有': ['带有', '兼具', '拥有', '含有'],
    '导致': ['引发', '造成', '招致', '引起'],
    '提供': ['给出', '供给', '拿出', '呈上'],
    '分析': ['剖析', '解读', '拆解', '审视'],
    '促进': ['推动', '助推', '带动', '催动'],
    '利用': ['借用', '运用', '动用', '凭借'],
    '建立': ['搭建', '构筑', '组建', '创设'],
    '引起': ['招来', '激起', '触发', '挑起'],
    '采取': ['采用', '动用', '使出', '施行'],
    '包括': ['涵盖', '囊括', '含', '包含'],
    '产生': ['催生', '引出', '萌生', '冒出'],
    '增加': ['添加', '追加', '扩充', '加大'],
    '减少': ['缩减', '削减', '降低', '裁减'],
    '保持': ['维持', '守住', '留住', '持续'],
    '解决': ['化解', '处置', '破解', '攻克'],
    '改变': ['改动', '变更', '扭转', '调整'],
    '选择': ['挑选', '择取', '选用', '拣'],
    '支持': ['撑持', '扶持', '相挺', '力挺'],
    '组成': ['构成', '拼成', '组合', '凑成'],
    '形成': ['催生', '铸成', '生成', '酿成'],
    '获得': ['取得', '赢得', '得到', '揽获'],
    '确定': ['敲定', '锁定', '明确', '定下'],
    '发现': ['察觉', '觉察', '识破', '看出'],
    '推动': ['驱动', '助推', '催动', '拉动'],
    '加强': ['强化', '增强', '夯实', '巩固'],
    '体现': ['彰显', '凸显', '映射', '折射'],
    '满足': ['达到', '契合', '符合', '迎合'],
    '存在': ['有', '潜伏', '隐含', '客观上有'],
    '属于': ['归属', '算是', '属', '归入'],
    '考虑': ['斟酌', '权衡', '琢磨', '思量'],
    '处理': ['打理', '应对', '料理', '处置'],
    '参与': ['加入', '介入', '参加', '投身'],
    '创造': ['缔造', '开创', '营造', '打造'],
    '描述': ['刻画', '勾勒', '叙述', '描绘'],
    '强调': ['着重', '突出', '力陈', '重申'],
    '反映': ['映射', '折射', '体现', '呈现'],
    '应用': ['运用', '采用', '使用', '施用'],
    '结合': ['融合', '配合', '糅合', '衔接'],
    '关注': ['留意', '聚焦', '在意', '着眼'],
    '涉及': ['牵涉', '关乎', '触及', '波及'],
    '依据': ['按照', '参照', '凭', '根据'],
    '采用': ['选用', '沿用', '取用', '引用'],
    # ── 副词 / 程度 ──
    '目前': ['眼下', '当前', '现阶段', '如今'],
    '同时': ['与此同时', '此外', '另外', '并且'],
    '通过': ['借助', '凭借', '经由', '依靠'],
    '根据': ['按照', '依据', '参照', '依照'],
    '有效': ['管用', '奏效', '见效', '起作用'],
    '基于': ['立足于', '依托', '以…为基础', '仰赖'],
    '对于': ['针对', '就', '关于', '面对'],
    '非常': ['极其', '十分', '很', '格外'],
    '已经': ['早已', '业已', '已', '早就'],
    '完全': ['彻底', '全然', '纯粹', '压根'],
    '不断': ['持续', '始终', '一再', '反复'],
    '逐渐': ['渐渐', '慢慢', '一步步', '日渐'],
    '主要': ['核心', '关键', '首要', '最要紧的'],
    '一般': ['通常', '往常', '照例', '大抵'],
    '大量': ['海量', '大批', '众多', '成堆的'],
    '进一步': ['更', '再', '深入', '继续'],
    '充分': ['尽情', '透彻', '淋漓', '饱满'],
    '直接': ['径直', '当面', '立刻', '干脆'],
    '特别': ['尤其', '格外', '极', '分外'],
    '一定': ['某种', '相当', '一些', '多少'],
    '必须': ['得', '务必', '非得', '须'],
    '可能': ['也许', '兴许', '或许', '大概'],
    # ── 名词 / 概念 ──
    '重要': ['关键', '核心', '要紧', '紧要'],
    '显著': ['明显', '突出', '可观', '醒目'],
    '问题': ['难题', '困境', '麻烦', '症结'],
    '方面': ['层面', '维度', '角度', '面向'],
    '情况': ['状况', '形势', '境况', '局面'],
    '特点': ['特征', '属性', '标志', '特色'],
    '方法': ['办法', '手段', '途径', '招数'],
    '过程': ['历程', '进程', '流程', '经过'],
    '结果': ['后果', '成果', '产物', '结局'],
    '条件': ['前提', '条件', '要件', '门槛'],
    '作用': ['功用', '效用', '效能', '功能'],
    '内容': ['要素', '成分', '要点', '素材'],
    '程度': ['幅度', '力度', '地步', '深浅'],
    '原因': ['缘由', '根源', '起因', '来由'],
    '目标': ['目的', '指向', '靶心', '方向'],
    '水平': ['档次', '层次', '段位', '高度'],
    '范围': ['领域', '地带', '区间', '覆盖面'],
    '趋势': ['走向', '苗头', '势头', '倾向'],
    '能力': ['本事', '实力', '功底', '才干'],
    '优势': ['长处', '强项', '亮点', '好处'],
    '资源': ['资产', '家底', '储备', '本钱'],
    '环境': ['氛围', '生态', '场景', '背景'],
    '系统': ['体系', '架构', '格局', '框架'],
    '策略': ['打法', '路线', '方案', '对策'],
}

# ═══════════════════════════════════════════════════════════════════
#  Strategy 3: Noise expression injection — expression table
# ═══════════════════════════════════════════════════════════════════

NOISE_EXPRESSIONS = {
    'hedging': ['说实话', '坦白讲', '客观地说', '实事求是地讲', '平心而论',
                '老实说', '不夸张地说', '公正地看'],
    'self_correction': ['或者说', '准确地讲', '换个角度看', '严格来说',
                        '更确切地说', '往深了讲', '细想一下'],
    'uncertainty': ['大概', '差不多', '似乎', '或许', '多少有些',
                    '约莫', '估摸着', '八成'],
    'transition_casual': ['话说回来', '反过来看', '换句话说', '说到这里',
                          '再往下想', '回过头看', '顺着这个思路'],
    'filler': ['当然了', '其实', '说到底', '怎么说呢', '不瞒你说',
               '你别说', '讲真', '这么说吧'],
    'personal': ['我觉得', '在我看来', '依我之见', '以我的经验',
                 '在我的理解里', '就我所知', '我个人倾向于'],
}

# Academic-safe categories (no oral fillers or personal opinions)
NOISE_ACADEMIC_CATEGORIES = ['hedging', 'self_correction', 'uncertainty']
# Academic-specific hedging (more formal)
NOISE_ACADEMIC_EXPRESSIONS = {
    'hedging': ['客观地说', '实事求是地讲', '平心而论', '公正地看'],
    'self_correction': ['准确地讲', '严格来说', '更确切地说', '往深了讲'],
    'uncertainty': ['大致', '似乎', '或许', '多少', '在一定程度上'],
}


def _load_bigram_freq():
    """Load bigram frequencies from the n-gram frequency table."""
    try:
        from ngram_model import _load_freq
    except ImportError:
        try:
            from scripts.ngram_model import _load_freq
        except ImportError:
            return {}
    freq = _load_freq()
    return freq.get('bigrams', {})


def reduce_high_freq_bigrams(text, strength=0.3):
    """
    策略1: 扫描文本中的高频 bigram，尝试用低频同义替换降低可预测性。
    strength: 0-1，控制替换比例。
    
    使用基于词的替换（非位置），避免长度变化导致的错位问题。
    """
    bigram_freq = _load_bigram_freq()
    if not bigram_freq:
        return _simple_synonym_pass(text, strength)

    chars = re.findall(r'[\u4e00-\u9fff]', text)
    if len(chars) < 4:
        return text

    # Step 1: Score each WORD_SYNONYMS word by its surrounding bigram frequency
    word_scores = []  # (word, total_bigram_freq, count_in_text)
    for word in WORD_SYNONYMS:
        count = text.count(word)
        if count == 0:
            continue
        # Compute bigram frequency of this word's characters
        word_chars = re.findall(r'[\u4e00-\u9fff]', word)
        total_freq = 0
        for i in range(len(word_chars) - 1):
            bg = word_chars[i] + word_chars[i + 1]
            total_freq += bigram_freq.get(bg, 0)
        word_scores.append((word, total_freq, count))

    if not word_scores:
        return text

    # Step 2: Sort by bigram frequency (highest first)
    word_scores.sort(key=lambda x: x[1], reverse=True)

    # Step 3: Replace top N unique words (controlled by strength)
    n_replace = max(1, int(len(word_scores) * strength))
    replaced_words = set()

    for word, freq_score, count in word_scores[:n_replace]:
        if word in replaced_words:
            continue

        candidates = WORD_SYNONYMS[word]

        # Pick the candidate with lowest bigram frequency
        best_candidate = candidates[0]
        best_freq = float('inf')

        for candidate in candidates:
            cand_chars = re.findall(r'[\u4e00-\u9fff]', candidate)
            if not cand_chars:
                continue
            total_f = 0
            for i in range(len(cand_chars) - 1):
                total_f += bigram_freq.get(cand_chars[i] + cand_chars[i + 1], 0)
            if total_f < best_freq:
                best_freq = total_f
                best_candidate = candidate

        # Replace all occurrences of this word (use sentinel to avoid chains)
        SENTINEL = '\x00'
        protected = SENTINEL.join(best_candidate)
        text = text.replace(word, protected)
        replaced_words.add(word)

        # Also mark synonyms of the same word to avoid replacing the replacement
        for syn in candidates:
            if syn != best_candidate and syn in WORD_SYNONYMS:
                replaced_words.add(syn)

    # Strip sentinels
    text = text.replace('\x00', '')

    return text


def _simple_synonym_pass(text, strength=0.3):
    """Fallback: replace a fraction of WORD_SYNONYMS matches randomly."""
    found = []
    for word in WORD_SYNONYMS:
        start = 0
        while True:
            pos = text.find(word, start)
            if pos < 0:
                break
            found.append((word, pos))
            start = pos + len(word)
    if not found:
        return text
    n_replace = max(1, int(len(found) * strength))
    random.shuffle(found)
    replaced_positions = set()
    for word, pos in found[:n_replace]:
        if any(p in replaced_positions for p in range(pos, pos + len(word))):
            continue
        candidate = random.choice(WORD_SYNONYMS[word])
        text = text[:pos] + candidate + text[pos + len(word):]
        for p in range(pos, pos + len(candidate)):
            replaced_positions.add(p)
    return text


# ═══════════════════════════════════════════════════════════════════
#  Strategy 2: Sentence length randomization
# ═══════════════════════════════════════════════════════════════════

def randomize_sentence_lengths(text, aggressive=False, seed=None):
    """
    策略2: 刻意制造不均匀的句子长度分布。
    - 随机选 20% 的短句保持极短
    - 随机选 10% 的句子通过合并拉长
    - 制造"短-长-短-长-特长-短"的节奏
    """
    if seed is not None:
        random.seed(seed)

    # Split into sentences preserving punctuation
    parts = re.split(r'([。！？])', text)
    sentences = []
    for i in range(0, len(parts) - 1, 2):
        s = parts[i]
        p = parts[i + 1] if i + 1 < len(parts) else ''
        if s.strip():
            sentences.append((s, p))
    # Handle trailing text
    if len(parts) % 2 == 1 and parts[-1].strip():
        sentences.append((parts[-1], ''))

    if len(sentences) < 4:
        return text

    merge_rate = 0.15 if not aggressive else 0.25
    truncate_rate = 0.15 if not aggressive else 0.25

    result = []
    i = 0
    while i < len(sentences):
        s, p = sentences[i]
        cn_len = len(re.findall(r'[\u4e00-\u9fff]', s))

        # Strategy A: merge short adjacent sentences into a long one
        if (i + 1 < len(sentences) and random.random() < merge_rate):
            s2, p2 = sentences[i + 1]
            cn_len2 = len(re.findall(r'[\u4e00-\u9fff]', s2))
            # Merge if combined length reasonable (< 100 chars)
            if cn_len + cn_len2 < 100:
                merged = s.rstrip() + '，' + s2.lstrip()
                result.append(merged + p2)
                i += 2
                continue

        # Strategy B: truncate longer sentences to their first clause (creates short punchy sentences)
        if cn_len > 20 and cn_len < 50 and random.random() < truncate_rate:
            # Truncate to first clause (split at first comma), keep rest as next sentence
            comma_pos = s.find('，')
            if comma_pos > 5 and comma_pos < len(s) - 5:
                first_part = s[:comma_pos]
                rest_part = s[comma_pos + 1:]
                result.append(first_part + p)
                # Push the rest as a new "sentence" to be processed
                if rest_part.strip():
                    result.append(rest_part + '。')
                i += 1
                continue

        result.append(s + p)
        i += 1

    return ''.join(result)


# ═══════════════════════════════════════════════════════════════════
#  Strategy 3: Noise expression injection
# ═══════════════════════════════════════════════════════════════════

def inject_noise_expressions(text, density=0.15, style='general'):
    """
    策略3: 在句子间或句中适当位置插入噪声表达。
    density: 大约每多少句插入一个（0.15 ≈ 每 6-7 句一个）
    style: general / academic
    """
    if style == 'academic':
        categories = NOISE_ACADEMIC_CATEGORIES
        expressions = NOISE_ACADEMIC_EXPRESSIONS
    else:
        categories = list(NOISE_EXPRESSIONS.keys())
        expressions = NOISE_EXPRESSIONS

    # Split into sentences
    parts = re.split(r'([。！？])', text)
    sentences = []
    for i in range(0, len(parts) - 1, 2):
        s = parts[i]
        p = parts[i + 1] if i + 1 < len(parts) else ''
        if s.strip():
            sentences.append([s, p])
    if len(parts) % 2 == 1 and parts[-1].strip():
        sentences.append([parts[-1], ''])

    if len(sentences) < 3:
        return text

    injected = 0
    for i in range(len(sentences)):
        # Skip the last sentence (avoid orphaned expressions)
        if i >= len(sentences) - 1:
            continue
        # Skip very short sentences
        s_text = sentences[i][0]
        if len(re.findall(r'[\u4e00-\u9fff]', s_text)) < 8:
            continue
        if random.random() > density:
            continue

        cat = random.choice(categories)
        expr_list = expressions.get(cat, [])
        if not expr_list:
            continue
        expr = random.choice(expr_list)

        s, p = sentences[i]

        # Decide insertion position
        if cat in ('hedging', 'filler', 'personal', 'transition_casual'):
            # Insert at sentence beginning
            s = expr + '，' + s.lstrip()
        elif cat in ('self_correction', 'uncertainty'):
            # Insert mid-sentence at a comma
            comma_pos = s.find('，')
            if comma_pos > 3:
                s = s[:comma_pos + 1] + expr + '，' + s[comma_pos + 1:]
            else:
                s = expr + '，' + s.lstrip()

        sentences[i] = [s, p]
        injected += 1

    return ''.join(s + p for s, p in sentences)


# ─── Core Transforms ───

def remove_three_part_structure(text):
    """Remove 首先/其次/最后, 第一/第二/第三 patterns"""
    # Don't just delete — replace with natural transitions
    replacements = [
        (r'首先[，,]\s*', ''),
        (r'其次[，,]\s*', lambda m: random.choice(['再说，', '另外，', ''])),
        (r'最后[，,]\s*', lambda m: random.choice(['还有，', '最后说一点，', ''])),
        (r'第一[，,、]\s*', ''),
        (r'第二[，,、]\s*', lambda m: random.choice(['接着，', '然后，', ''])),
        (r'第三[，,、]\s*', lambda m: random.choice(['还有，', '再就是，', ''])),
        (r'第[四五六七八九][，,、]\s*', lambda m: random.choice(['另外，', ''])),
        (r'其一[，,、]\s*', ''),
        (r'其二[，,、]\s*', lambda m: random.choice(['另外，', ''])),
        (r'其三[，,、]\s*', lambda m: random.choice(['还有，', ''])),
    ]
    
    for pattern, repl in replacements:
        if callable(repl):
            text = re.sub(pattern, repl, text)
        else:
            text = re.sub(pattern, repl, text)
    
    return text

def replace_phrases(text, casualness=0.3):
    """Replace AI phrases with natural alternatives (context-aware)"""
    # Apply regex replacements FIRST (per-sentence, max 1 regex replacement per sentence)
    # Split by sentence-ending punctuation to handle multiple templates in same line
    parts = re.split(r'([。！？\n])', text)
    rebuilt = []
    for part in parts:
        replaced = False
        for pattern, alternatives in REGEX_REPLACEMENTS.items():
            if replaced:
                break
            if isinstance(alternatives, str):
                alternatives = [alternatives]
            try:
                match = re.search(pattern, part)
                if match:
                    replacement = random.choice(alternatives)
                    part = part[:match.start()] + replacement + part[match.end():]
                    replaced = True
            except re.error:
                pass
        rebuilt.append(part)
    text = ''.join(rebuilt)
    
    # Then plain replacements, sorted by length (longest first) to avoid partial matches
    sorted_phrases = sorted(PLAIN_REPLACEMENTS.keys(), key=len, reverse=True)
    
    for phrase in sorted_phrases:
        alternatives = PLAIN_REPLACEMENTS[phrase]
        if isinstance(alternatives, str):
            alternatives = [alternatives]
        
        if phrase in text:
            # Find the sentence containing this phrase for stats-optimized selection
            # Use pick_best_replacement to choose the highest-perplexity candidate
            replacement = pick_best_replacement(text, phrase, alternatives)
            text = text.replace(phrase, replacement, 1)  # Replace first occurrence
            # For subsequent occurrences, use different alternatives
            while phrase in text:
                replacement = pick_best_replacement(text, phrase, alternatives)
                text = text.replace(phrase, replacement, 1)
    
    return text

def merge_short_sentences(text, min_len=8):
    """Merge overly short consecutive sentences, with burstiness guard."""
    # Measure burstiness before restructuring
    burst_before = _compute_burstiness(text)

    sentences = re.split(r'([。！？])', text)
    if len(sentences) < 4:
        return text
    
    result = []
    i = 0
    while i < len(sentences) - 1:
        sent = sentences[i]
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''
        
        # Check if this and next sentence are both short
        next_sent = sentences[i + 2] if i + 2 < len(sentences) else ''
        
        if len(sent.strip()) < min_len and len(next_sent.strip()) < min_len and next_sent.strip():
            # Merge with comma
            merged = sent.strip() + '，' + next_sent.strip()
            next_punct = sentences[i + 3] if i + 3 < len(sentences) else '。'
            result.append(merged + next_punct)
            i += 4
        else:
            result.append(sent + punct)
            i += 2
    
    # Handle remaining
    while i < len(sentences):
        result.append(sentences[i])
        i += 1
    
    new_text = ''.join(result)

    # Burstiness guard: if merging made text more uniform, revert
    if burst_before is not None:
        burst_after = _compute_burstiness(new_text)
        if burst_after is not None and burst_after < burst_before * 0.8:
            return text  # revert — merging reduced burstiness too much

    return new_text

def split_long_sentences(text, max_len=80):
    """Split overly long sentences at natural breakpoints, with burstiness guard."""
    burst_before = _compute_burstiness(text)

    sentences = re.split(r'([。！？])', text)
    result = []
    
    for i in range(0, len(sentences) - 1, 2):
        sent = sentences[i]
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''
        
        chinese_len = len(re.findall(r'[\u4e00-\u9fff]', sent))
        
        if chinese_len > max_len:
            # Find natural split points: 但是/不过/然而/同时/而且
            split_points = [
                (m.start(), m.group()) for m in
                re.finditer(r'[，,](但是|不过|然而|同时|而且|所以|因此|另外)', sent)
            ]
            
            if split_points:
                # Split at the most central point
                mid = len(sent) // 2
                best = min(split_points, key=lambda x: abs(x[0] - mid))
                part1 = sent[:best[0]]
                part2 = sent[best[0]+1:]  # Skip the comma
                result.append(part1 + '。' + part2 + punct)
            else:
                # Split at a comma near the middle
                commas = [m.start() for m in re.finditer(r'[，,]', sent)]
                if commas:
                    mid = len(sent) // 2
                    best_comma = min(commas, key=lambda x: abs(x - mid))
                    part1 = sent[:best_comma]
                    part2 = sent[best_comma+1:]
                    result.append(part1 + '。' + part2 + punct)
                else:
                    result.append(sent + punct)
        else:
            result.append(sent + punct)
    
    # Handle remaining
    if len(sentences) % 2 == 1 and sentences[-1].strip():
        result.append(sentences[-1])
    
    new_text = ''.join(result)

    # Burstiness guard: if splitting made text more uniform, revert
    if burst_before is not None:
        burst_after = _compute_burstiness(new_text)
        if burst_after is not None and burst_after < burst_before * 0.8:
            return text

    return new_text

def vary_paragraph_rhythm(text):
    """Break uniform paragraph lengths by merging or splitting"""
    paragraphs = text.split('\n\n')
    if len(paragraphs) < 3:
        return text
    
    lengths = [len(p) for p in paragraphs]
    avg_len = sum(lengths) / len(lengths) if lengths else 100
    
    result = []
    i = 0
    while i < len(paragraphs):
        para = paragraphs[i]
        
        # Randomly merge short adjacent paragraphs
        if (i + 1 < len(paragraphs) and
            len(para) < avg_len * 0.6 and
            len(paragraphs[i + 1]) < avg_len * 0.6 and
            random.random() < 0.4):
            merged = para + '\n' + paragraphs[i + 1]
            result.append(merged)
            i += 2
            continue
        
        # Split long paragraphs
        if len(para) > avg_len * 1.5:
            sentences = re.split(r'([。！？])', para)
            mid = len(sentences) // 2
            # Ensure we split at a sentence boundary (every other element is punctuation)
            if mid % 2 == 1:
                mid -= 1
            part1 = ''.join(sentences[:mid])
            part2 = ''.join(sentences[mid:])
            if part1.strip() and part2.strip():
                result.append(part1.strip())
                result.append(part2.strip())
                i += 1
                continue
        
        result.append(para)
        i += 1
    
    return '\n\n'.join(result)

def reduce_punctuation(text):
    """Reduce excessive punctuation intelligently"""
    # Replace some semicolons with commas or periods
    parts = text.split('；')
    if len(parts) > 3:
        result_parts = [parts[0]]
        for i, part in enumerate(parts[1:], 1):
            # Alternate between comma and period
            if i % 2 == 0:
                result_parts.append('。' + part.lstrip())
            else:
                result_parts.append('，' + part)
        text = ''.join(result_parts)
    
    # Limit consecutive em dashes
    text = re.sub(r'——', '—', text)
    
    return text

def add_casual_expressions(text, casualness=0.3):
    """Inject casual/human expressions"""
    if casualness < 0.2:
        return text
    
    casual_openers = ['说实话', '其实', '确实', '讲真', '坦白说']
    casual_transitions = ['话说回来', '说到这个', '不过呢', '但是吧']
    casual_endings = ['就是这么回事', '差不多就这样', '大概就这些']
    
    sentences = re.split(r'([。！？])', text)
    result = []
    added = 0
    total = len(sentences) // 2
    max_additions = max(1, int(total * casualness * 0.3))
    
    for i in range(0, len(sentences) - 1, 2):
        sent = sentences[i]
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''
        
        if added < max_additions and random.random() < casualness * 0.2:
            if i == 0:
                opener = random.choice(casual_openers)
                sent = opener + '，' + sent
            elif i > total:
                transition = random.choice(casual_transitions)
                sent = transition + '，' + sent
            added += 1
        
        result.append(sent + punct)
    
    if len(sentences) % 2 == 1 and sentences[-1].strip():
        result.append(sentences[-1])
    
    return ''.join(result)

def shorten_paragraphs(text, max_length=150):
    """Break long paragraphs for social/chat scenes"""
    paragraphs = text.split('\n\n')
    result = []
    
    for para in paragraphs:
        if len(para) > max_length:
            sentences = re.split(r'([。！？])', para)
            chunks = []
            current = ''
            
            for i in range(0, len(sentences) - 1, 2):
                sent = sentences[i] + (sentences[i + 1] if i + 1 < len(sentences) else '')
                if len(current) + len(sent) > max_length and current:
                    chunks.append(current.strip())
                    current = sent
                else:
                    current += sent
            
            if current.strip():
                chunks.append(current.strip())
            
            result.extend(chunks)
        else:
            result.append(para)
    
    return '\n\n'.join(result)

def diversify_vocabulary(text):
    """Reduce word repetition by using synonyms"""
    # Common overused words and their alternatives
    diversity_map = {
        '进行': ['做', '开展', '实施', '推进'],
        '实现': ['达到', '做到', '完成'],
        '提供': ['给出', '带来', '拿出'],
        '具有': ['有', '拥有', '带有'],
        '进一步': ['更', '再', '深入'],
        '不断': ['持续', '一直', '始终'],
        '有效': ['管用', '见效', '奏效'],
        '积极': ['主动', '热心'],
        '促进': ['推动', '带动'],
        '加强': ['强化', '增强'],
        '提高': ['提升', '增加'],
        '重要': ['关键', '核心'],
    }
    
    for word, alts in diversity_map.items():
        count = text.count(word)
        if count > 2:
            # Keep first occurrence, replace subsequent
            first = True
            parts = text.split(word)
            result = [parts[0]]
            for part in parts[1:]:
                if first:
                    result.append(word)
                    first = False
                else:
                    result.append(random.choice(alts))
                result.append(part)
            text = ''.join(result)
    
    return text

# ─── Main Humanize Pipeline ───

def humanize(text, scene='general', aggressive=False, seed=None):
    """Apply all humanization transformations in order"""
    if seed is not None:
        random.seed(seed)
    
    config = SCENES.get(scene, SCENES['general'])
    casualness = config.get('casualness', 0.3)
    if aggressive:
        casualness = min(1.0, casualness + 0.3)
    
    # Pass 1: Structure cleanup
    text = remove_three_part_structure(text)
    text = replace_phrases(text, casualness)
    
    # Pass 2: Sentence restructuring
    if config.get('merge_short', False):
        text = merge_short_sentences(text)
    if config.get('split_long', False):
        text = split_long_sentences(text)
    
    # Pass 3: Rhythm and variety
    text = reduce_punctuation(text)
    text = diversify_vocabulary(text)
    
    if config.get('rhythm_variation', False):
        text = vary_paragraph_rhythm(text)
    
    # Pass 4: Scene-specific
    if config.get('add_casual', False) or aggressive:
        text = add_casual_expressions(text, casualness)
    
    if config.get('shorten_paragraphs', False):
        text = shorten_paragraphs(text)
    
    # ── NEW: Three perplexity-boosting strategies ──
    
    # Strategy 1: Low-frequency bigram injection (always active)
    bigram_strength = 0.5 if aggressive else 0.3
    text = reduce_high_freq_bigrams(text, strength=bigram_strength)
    
    # Strategy 2 & 3: Noise injection (skipped with --no-noise)
    if _USE_NOISE:
        # Strategy 3: Noise expression injection
        noise_density = 0.25 if aggressive else 0.15
        text = inject_noise_expressions(text, density=noise_density, style='general')
        
        # Strategy 2: Sentence length randomization
        text = randomize_sentence_lengths(text, aggressive=aggressive, seed=seed)
    
    # Clean up artifacts
    text = re.sub(r'[，,]{2,}', '，', text)  # Remove double commas
    text = re.sub(r'[。]{2,}', '。', text)    # Remove double periods
    text = re.sub(r'\n{3,}', '\n\n', text)    # Normalize newlines
    text = re.sub(r'，。', '。', text)          # Remove comma before period
    text = re.sub(r'。，', '。', text)          # Remove period before comma
    
    # ── Final verification loop (stats-optimized) ──
    # If perplexity is still too low, do a targeted second pass on worst sentences
    if _USE_STATS and ngram_analyze:
        stats = ngram_analyze(text)
        ppl = stats.get('perplexity', 0)
        # Threshold: if perplexity is in the "too smooth" zone, try to improve
        if 0 < ppl < 200 and len(text) >= 100:
            sentences = re.split(r'([。！？])', text)
            # Score each sentence
            sent_scores = []
            for i in range(0, len(sentences) - 1, 2):
                s = sentences[i]
                if len(s.strip()) < 5:
                    continue
                s_stats = ngram_analyze(s)
                sent_scores.append((i, s_stats.get('perplexity', 0)))
            
            if sent_scores:
                # Sort by perplexity ascending (worst = most predictable first)
                sent_scores.sort(key=lambda x: x[1])
                # Try to improve the worst 20% (at most 5 sentences)
                n_fix = min(5, max(1, len(sent_scores) // 5))
                
                # Use a different random seed for the second pass
                if seed is not None:
                    random.seed(seed + 1)
                
                for idx, _ in sent_scores[:n_fix]:
                    sent = sentences[idx]
                    # Try each replacement on this sentence
                    sorted_phrases = sorted(PLAIN_REPLACEMENTS.keys(), key=len, reverse=True)
                    for phrase in sorted_phrases:
                        if phrase in sent:
                            alternatives = PLAIN_REPLACEMENTS[phrase]
                            if isinstance(alternatives, str):
                                alternatives = [alternatives]
                            best = pick_best_replacement(sent, phrase, alternatives)
                            sentences[idx] = sent.replace(phrase, best, 1)
                            break  # one fix per sentence to avoid over-rewriting
                
                text = ''.join(sentences)
    
    return text.strip()

# ─── Main ───

def main():
    parser = argparse.ArgumentParser(description='中文 AI 文本人性化 v2.0')
    parser.add_argument('file', nargs='?', help='输入文件路径')
    parser.add_argument('-o', '--output', help='输出文件路径')
    parser.add_argument('--scene', default='general',
                       choices=['general', 'social', 'tech', 'formal', 'chat'],
                       help='场景 (default: general)')
    parser.add_argument('--style', help='写作风格 (调用 style_cn.py)')
    parser.add_argument('-a', '--aggressive', action='store_true', help='激进模式')
    parser.add_argument('--seed', type=int, help='随机种子（可复现）')
    parser.add_argument('--no-stats', action='store_true',
                       help='跳过统计优化（困惑度反馈），回退到纯规则替换')
    parser.add_argument('--no-noise', action='store_true',
                       help='跳过噪声策略（句长随机化 + 噪声表达插入）')
    
    args = parser.parse_args()
    
    # Toggle stats optimization
    global _USE_STATS
    _USE_STATS = not args.no_stats
    
    # Toggle noise strategies
    global _USE_NOISE
    _USE_NOISE = not args.no_noise
    
    # Read input
    if args.file:
        try:
            with open(args.file, 'r', encoding='utf-8') as f:
                text = f.read()
        except FileNotFoundError:
            print(f'错误: 文件未找到 {args.file}', file=sys.stderr)
            sys.exit(1)
    else:
        text = sys.stdin.read()
    
    if not text.strip():
        print('错误: 输入为空', file=sys.stderr)
        sys.exit(1)
    
    # Humanize
    result = humanize(text, args.scene, args.aggressive, args.seed)
    
    # Apply style if specified
    if args.style:
        import subprocess
        style_script = os.path.join(SCRIPT_DIR, 'style_cn.py')
        
        if os.path.exists(style_script):
            import tempfile
            with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False, encoding='utf-8') as tmp:
                tmp.write(result)
                tmp_path = tmp.name
            
            try:
                proc = subprocess.run(
                    ['python3', style_script, tmp_path, '--style', args.style],
                    capture_output=True, text=True, encoding='utf-8'
                )
                if proc.returncode == 0 and proc.stdout.strip():
                    result = proc.stdout
                else:
                    print(f'警告: 风格转换失败: {proc.stderr}', file=sys.stderr)
            finally:
                os.unlink(tmp_path)
        else:
            print(f'警告: 未找到风格转换脚本', file=sys.stderr)
    
    # Output
    if args.output:
        with open(args.output, 'w', encoding='utf-8') as f:
            f.write(result)
        style_info = f' (风格: {args.style})' if args.style else ''
        scene_info = f' (场景: {args.scene})'
        print(f'✓ 已保存到 {args.output}{scene_info}{style_info}')
    else:
        print(result)

if __name__ == '__main__':
    main()


path: scripts/ngram_model.py
content: #!/usr/bin/env python3
"""
Chinese N-gram Language Model for AI Text Detection
Character-level bigram + trigram perplexity, burstiness, and entropy analysis.
Pure Python, no external dependencies.

Key insight: AI-generated Chinese text tends to have:
  - Lower perplexity (more predictable character sequences)
  - Lower burstiness (uniform complexity throughout)
  - More uniform entropy across paragraphs
"""

import json
import os
import re
from math import log2, exp

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
FREQ_FILE = os.path.join(SCRIPT_DIR, 'ngram_freq_cn.json')

# ─── Frequency Table Loading ───

_FREQ_CACHE = None

def _load_freq():
    """Load n-gram frequency table (cached)."""
    global _FREQ_CACHE
    if _FREQ_CACHE is not None:
        return _FREQ_CACHE

    if not os.path.exists(FREQ_FILE):
        _FREQ_CACHE = {'unigrams': {}, 'bigrams': {}, 'trigrams': {},
                       'corpus_info': {'total_chars': 1}}
        return _FREQ_CACHE

    with open(FREQ_FILE, 'r', encoding='utf-8') as f:
        _FREQ_CACHE = json.load(f)

    # Convert string counts to int if needed
    for key in ('unigrams', 'bigrams', 'trigrams'):
        table = _FREQ_CACHE.get(key, {})
        _FREQ_CACHE[key] = {k: int(v) for k, v in table.items()}

    return _FREQ_CACHE


def _extract_chinese(text):
    """Extract only Chinese characters from text."""
    return re.findall(r'[\u4e00-\u9fff]', text)


# ─── Core Perplexity Computation ───

def _bigram_log_prob(c1, c2, freq):
    """
    Compute log2 probability of bigram (c1, c2) using add-k smoothing.
    P(c2|c1) ≈ (count(c1c2) + k) / (count(c1) + k * V)
    """
    unigrams = freq['unigrams']
    bigrams = freq['bigrams']
    V = max(len(unigrams), 1000)  # vocabulary size
    k = 0.01  # smoothing factor

    bigram_key = c1 + c2
    bi_count = bigrams.get(bigram_key, 0)
    uni_count = unigrams.get(c1, 0)

    prob = (bi_count + k) / (uni_count + k * V)
    return log2(prob) if prob > 0 else -20.0  # floor for unseen


def _trigram_log_prob(c1, c2, c3, freq):
    """
    Compute log2 probability of trigram using interpolation with bigrams.
    P_interp = lambda * P_tri(c3|c1c2) + (1-lambda) * P_bi(c3|c2)
    """
    bigrams = freq['bigrams']
    trigrams = freq['trigrams']
    V = max(len(freq['unigrams']), 1000)
    k = 0.01
    lam = 0.6  # trigram weight

    # Trigram probability
    tri_key = c1 + c2 + c3
    bi_context_key = c1 + c2
    tri_count = trigrams.get(tri_key, 0)
    bi_context_count = bigrams.get(bi_context_key, 0)
    p_tri = (tri_count + k) / (bi_context_count + k * V)

    # Bigram probability (backoff)
    p_bi_raw = _bigram_log_prob(c2, c3, freq)
    p_bi = 2 ** p_bi_raw  # convert back from log2

    # Interpolation in probability space
    p_interp = lam * p_tri + (1 - lam) * p_bi
    return log2(p_interp) if p_interp > 0 else -20.0


def compute_perplexity(text, window_size=0):
    """
    Compute character-level perplexity of Chinese text using interpolated trigram model.

    Args:
        text: input text string
        window_size: if > 0, compute per-window perplexities for burstiness.
                     0 means compute whole-text perplexity only.

    Returns:
        dict with:
          - perplexity: overall perplexity (float)
          - avg_log_prob: average log2 probability per character
          - window_perplexities: list of per-window perplexities (if window_size > 0)
          - char_count: number of Chinese characters used
    """
    freq = _load_freq()
    chars = _extract_chinese(text)

    if len(chars) < 5:
        return {
            'perplexity': 0.0,
            'avg_log_prob': 0.0,
            'window_perplexities': [],
            'char_count': len(chars),
        }

    # Compute per-character log probabilities using trigram model
    log_probs = []
    for i in range(2, len(chars)):
        lp = _trigram_log_prob(chars[i-2], chars[i-1], chars[i], freq)
        log_probs.append(lp)

    if not log_probs:
        return {
            'perplexity': 0.0,
            'avg_log_prob': 0.0,
            'window_perplexities': [],
            'char_count': len(chars),
        }

    # Overall perplexity: 2^(-avg_log_prob)
    avg_lp = sum(log_probs) / len(log_probs)
    perplexity = 2 ** (-avg_lp)

    # Per-window perplexities
    window_ppls = []
    if window_size > 0 and len(log_probs) >= window_size:
        for start in range(0, len(log_probs) - window_size + 1, window_size // 2):
            end = min(start + window_size, len(log_probs))
            chunk = log_probs[start:end]
            if chunk:
                w_avg = sum(chunk) / len(chunk)
                window_ppls.append(2 ** (-w_avg))

    return {
        'perplexity': perplexity,
        'avg_log_prob': avg_lp,
        'window_perplexities': window_ppls,
        'char_count': len(chars),
    }


# ─── Burstiness ───

def compute_burstiness(text, window_size=50):
    """
    Compute burstiness as coefficient of variation of windowed perplexities.

    Human text: higher burstiness (CV > 0.3, some parts simple, some complex)
    AI text: lower burstiness (CV < 0.2, uniformly smooth)

    Args:
        text: input text
        window_size: character window for perplexity segments

    Returns:
        dict with:
          - burstiness: coefficient of variation (std/mean) of window perplexities
          - mean_ppl: mean of window perplexities
          - std_ppl: standard deviation of window perplexities
          - n_windows: number of windows analyzed
    """
    result = compute_perplexity(text, window_size=window_size)
    ppls = result['window_perplexities']

    if len(ppls) < 3:
        return {
            'burstiness': 0.0,
            'mean_ppl': result['perplexity'],
            'std_ppl': 0.0,
            'n_windows': len(ppls),
        }

    mean_ppl = sum(ppls) / len(ppls)
    if mean_ppl == 0:
        return {
            'burstiness': 0.0,
            'mean_ppl': 0.0,
            'std_ppl': 0.0,
            'n_windows': len(ppls),
        }

    variance = sum((p - mean_ppl) ** 2 for p in ppls) / len(ppls)
    std_ppl = variance ** 0.5
    cv = std_ppl / mean_ppl

    return {
        'burstiness': cv,
        'mean_ppl': mean_ppl,
        'std_ppl': std_ppl,
        'n_windows': len(ppls),
    }


# ─── Paragraph Entropy Uniformity ───

def compute_entropy_uniformity(text):
    """
    Compute entropy of each paragraph and measure how uniform they are.

    AI text: paragraphs have very similar entropy (low CV)
    Human text: entropy varies more between paragraphs

    Returns:
        dict with:
          - entropy_cv: coefficient of variation of per-paragraph entropy
          - paragraph_entropies: list of (paragraph_index, entropy) tuples
          - mean_entropy: mean paragraph entropy
          - n_paragraphs: number of paragraphs analyzed
    """
    # Split into paragraphs (by double newline or single newline with enough content)
    raw_paras = re.split(r'\n\s*\n|\n', text)
    paragraphs = [p.strip() for p in raw_paras
                  if p.strip() and len(_extract_chinese(p.strip())) >= 20]

    if len(paragraphs) < 3:
        return {
            'entropy_cv': 0.0,
            'paragraph_entropies': [],
            'mean_entropy': 0.0,
            'n_paragraphs': len(paragraphs),
        }

    # Compute per-paragraph bigram entropy
    para_entropies = []
    for i, para in enumerate(paragraphs):
        chars = _extract_chinese(para)
        if len(chars) < 10:
            continue

        # Bigram frequency within paragraph
        bigrams = {}
        for j in range(len(chars) - 1):
            key = chars[j] + chars[j+1]
            bigrams[key] = bigrams.get(key, 0) + 1

        total = sum(bigrams.values())
        if total == 0:
            continue

        entropy = 0.0
        for count in bigrams.values():
            p = count / total
            if p > 0:
                entropy -= p * log2(p)

        para_entropies.append((i, entropy))

    if len(para_entropies) < 3:
        return {
            'entropy_cv': 0.0,
            'paragraph_entropies': para_entropies,
            'mean_entropy': 0.0,
            'n_paragraphs': len(para_entropies),
        }

    entropies = [e for _, e in para_entropies]
    mean_ent = sum(entropies) / len(entropies)

    if mean_ent == 0:
        return {
            'entropy_cv': 0.0,
            'paragraph_entropies': para_entropies,
            'mean_entropy': 0.0,
            'n_paragraphs': len(para_entropies),
        }

    variance = sum((e - mean_ent) ** 2 for e in entropies) / len(entropies)
    std_ent = variance ** 0.5
    cv = std_ent / mean_ent

    return {
        'entropy_cv': cv,
        'paragraph_entropies': para_entropies,
        'mean_entropy': mean_ent,
        'n_paragraphs': len(para_entropies),
    }


# ─── Combined Analysis ───

def analyze_text(text):
    """
    Run full statistical analysis on text.

    Returns dict with all metrics and AI likelihood indicators:
      - perplexity: overall perplexity
      - burstiness: CV of windowed perplexity
      - entropy_cv: CV of paragraph entropy
      - indicators: dict of boolean flags for AI-like patterns
    """
    chars = _extract_chinese(text)
    char_count = len(chars)

    if char_count < 30:
        return {
            'perplexity': 0.0,
            'burstiness': 0.0,
            'entropy_cv': 0.0,
            'char_count': char_count,
            'indicators': {
                'low_perplexity': False,
                'low_burstiness': False,
                'uniform_entropy': False,
            },
            'details': {},
        }

    # Perplexity
    ppl_result = compute_perplexity(text, window_size=50)

    # Burstiness
    burst_result = compute_burstiness(text, window_size=50)

    # Entropy uniformity
    ent_result = compute_entropy_uniformity(text)

    # Thresholds — conservative, designed for character-level n-gram model.
    #
    # With a small corpus-based model, perplexity direction depends on text style.
    # We use RELATIVE signals rather than absolute thresholds:
    #   - Perplexity: AI formal text often has moderate-high ppl from this model
    #     (formal vocab not well covered), BUT the KEY signal is the range 100-400
    #     which is typical of formulaic AI text using semi-common formal patterns.
    #   - Burstiness: how much perplexity varies across windows.
    #     Very low values (< 0.12) = uniform complexity = AI-like.
    #     BUT short texts have unreliable burstiness, so require enough windows.
    #   - Entropy CV: how uniform paragraph entropy is.
    #     Very low values (< 0.05) = uniform paragraph structure = AI-like.
    #
    # These are intentionally conservative to avoid false positives.
    # With longer texts (1000+ chars) and better frequency data, accuracy improves.

    ppl = ppl_result['perplexity']
    burst = burst_result['burstiness']
    ent_cv = ent_result['entropy_cv']
    n_windows = burst_result['n_windows']
    n_paras = ent_result['n_paragraphs']

    indicators = {
        # Perplexity in the "formulaic formal" range (100-500) with enough text
        'low_perplexity': 50 < ppl < 500 and char_count >= 200,
        # Very low burstiness with enough data points
        'low_burstiness': burst < 0.12 and n_windows >= 6,
        # Very uniform paragraph entropy with enough paragraphs
        'uniform_entropy': ent_cv < 0.05 and n_paras >= 3,
    }

    return {
        'perplexity': ppl,
        'burstiness': burst,
        'entropy_cv': ent_cv,
        'char_count': char_count,
        'indicators': indicators,
        'details': {
            'perplexity_result': {
                'perplexity': ppl_result['perplexity'],
                'avg_log_prob': ppl_result['avg_log_prob'],
                'n_windows': len(ppl_result['window_perplexities']),
            },
            'burstiness_result': {
                'burstiness': burst_result['burstiness'],
                'mean_ppl': burst_result['mean_ppl'],
                'std_ppl': burst_result['std_ppl'],
                'n_windows': burst_result['n_windows'],
            },
            'entropy_result': {
                'entropy_cv': ent_result['entropy_cv'],
                'mean_entropy': ent_result['mean_entropy'],
                'n_paragraphs': ent_result['n_paragraphs'],
            },
        },
    }


# ─── CLI ───

def main():
    import argparse

    parser = argparse.ArgumentParser(description='中文文本 N-gram 统计分析 — 困惑度/突发度/熵')
    parser.add_argument('file', nargs='?', help='输入文件路径（不指定则从 stdin 读取）')
    parser.add_argument('-j', '--json', action='store_true', help='JSON 输出')
    parser.add_argument('-v', '--verbose', action='store_true', help='详细模式')
    args = parser.parse_args()

    import sys
    if args.file:
        try:
            with open(args.file, 'r', encoding='utf-8') as f:
                text = f.read()
        except FileNotFoundError:
            print(f'错误: 文件未找到 {args.file}', file=sys.stderr)
            sys.exit(1)
    else:
        text = sys.stdin.read()

    if not text.strip():
        print('错误: 输入为空', file=sys.stderr)
        sys.exit(1)

    result = analyze_text(text)

    if args.json:
        print(json.dumps(result, ensure_ascii=False, indent=2))
        return

    # Human-readable output
    ppl = result['perplexity']
    burst = result['burstiness']
    ent_cv = result['entropy_cv']
    indicators = result['indicators']

    print(f'字符数: {result["char_count"]}')
    print(f'困惑度 (perplexity): {ppl:.1f}')
    print(f'突发度 (burstiness):  {burst:.3f}')
    print(f'段落熵变异 (entropy CV): {ent_cv:.3f}')
    print()

    print('── AI 特征判断 ──')
    for key, desc in [
        ('low_perplexity', '困惑度异常低（过于流畅/可预测）'),
        ('low_burstiness', '困惑度变化过于均匀（缺少起伏）'),
        ('uniform_entropy', '段落间熵值分布过于均匀'),
    ]:
        flag = '⚠️  是' if indicators[key] else '✅ 否'
        print(f'  {flag} — {desc}')

    if args.verbose and result['details']:
        print()
        print('── 详细数据 ──')
        d = result['details']
        print(f'  平均 log2 概率: {d["perplexity_result"]["avg_log_prob"]:.3f}')
        print(f'  窗口数: {d["burstiness_result"]["n_windows"]}')
        print(f'  窗口平均困惑度: {d["burstiness_result"]["mean_ppl"]:.1f}')
        print(f'  窗口困惑度标准差: {d["burstiness_result"]["std_ppl"]:.1f}')
        print(f'  段落数: {d["entropy_result"]["n_paragraphs"]}')
        print(f'  段落平均熵: {d["entropy_result"]["mean_entropy"]:.3f}')


if __name__ == '__main__':
    main()


path: scripts/patterns_cn.json
content: {
  "version": "2.0.0",
  "critical_patterns": {
    "three_part_structure": {
      "weight": 15,
      "regex": [
        "首先[，,].*?其次[，,].*?最后",
        "一方面[，,].*?另一方面",
        "第一[，,].*?第二[，,].*?第三",
        "第一点.*?第二点.*?第三点",
        "其一[，,].*?其二[，,].*?其三"
      ]
    },
    "mechanical_connectors": {
      "weight": 10,
      "phrases": [
        "值得注意的是",
        "综上所述",
        "不难发现",
        "总而言之",
        "与此同时",
        "在此基础上",
        "由此可见",
        "不仅如此",
        "换句话说",
        "更重要的是",
        "需要强调的是",
        "不可否认",
        "事实上",
        "毋庸置疑",
        "显而易见",
        "不言而喻",
        "正如我们所知",
        "如前所述",
        "由此可以看出",
        "从某种意义上说",
        "在一定程度上",
        "归根结底"
      ]
    },
    "empty_grand_words": {
      "weight": 12,
      "phrases": [
        "赋能",
        "闭环",
        "智慧时代",
        "数字化转型",
        "生态",
        "愿景",
        "力量",
        "未来展望",
        "战略高度",
        "顶层设计",
        "协同增效",
        "降本增效",
        "打通壁垒",
        "深度融合",
        "创新驱动",
        "持续赋能",
        "全面升级",
        "高质量发展",
        "新质生产力",
        "提质增效",
        "双向奔赴",
        "全方位",
        "多维度",
        "立体化",
        "系统性"
      ]
    }
  },
  "high_signal_patterns": {
    "ai_high_freq_words": {
      "weight": 6,
      "phrases": [
        "助力",
        "彰显",
        "凸显",
        "焕发",
        "深度剖析",
        "加持",
        "赛道",
        "破圈",
        "出圈",
        "颠覆",
        "革新",
        "底层逻辑",
        "抓手",
        "链路",
        "触达",
        "心智",
        "沉淀",
        "对齐",
        "拉通",
        "复盘",
        "迭代",
        "卷",
        "躬身入局",
        "长期主义",
        "确定性"
      ]
    },
    "balanced_arguments": {
      "weight": 8,
      "description": "AI 特别喜欢两面论述，过度平衡",
      "regex": [
        "虽然.*?但是.*?同时",
        "一方面.*?另一方面.*?总的来说",
        "既有.*?也有.*?更有",
        "不仅.*?而且.*?更",
        "优点.*?缺点.*?总体"
      ]
    },
    "template_sentences": {
      "weight": 7,
      "description": "AI 常见句式模板",
      "regex": [
        "随着.*?的(不断)?发展",
        "在.*?的背景下",
        "在当今.*?时代",
        "作为.*?的重要(组成部分|环节|手段)",
        "对于.*?而言[，,].*?至关重要",
        "这不仅.*?更是",
        "从.*?角度(来看|来说|而言)",
        "无论是.*?还是.*?都",
        "可以说[，,]",
        "总的来说[，,]"
      ]
    },
    "filler_phrases": {
      "weight": 5,
      "phrases": [
        "值得一提的是",
        "需要指出的是",
        "不得不说",
        "毫无疑问",
        "众所周知",
        "正因如此",
        "具体来说",
        "简而言之",
        "换言之"
      ]
    }
  },
  "medium_signal_patterns": {
    "hedging_language": {
      "weight": 3,
      "description": "过度谨慎用语（AI倾向不下结论）",
      "phrases": [
        "在一定程度上",
        "或许",
        "可能",
        "某种程度上",
        "相对而言",
        "总体来说",
        "一般来说",
        "通常情况下"
      ],
      "threshold": 5
    },
    "list_addiction": {
      "weight": 4,
      "description": "AI 喜欢列举",
      "regex": [
        "(?:(?:①|②|③|④|⑤|1\\.|2\\.|3\\.|4\\.|5\\.).*?\n){3,}"
      ]
    }
  },
  "style_signals": {
    "emotional_words": [
      "愤怒",
      "高兴",
      "难过",
      "失望",
      "惊讶",
      "担心",
      "开心",
      "郁闷",
      "焦虑",
      "兴奋",
      "害怕",
      "感动",
      "烦躁",
      "痛苦",
      "崩溃",
      "无奈",
      "委屈",
      "舒服",
      "爽",
      "恶心",
      "气死",
      "乐死",
      "笑死"
    ],
    "personal_markers": [
      "我觉得",
      "我认为",
      "我个人",
      "在我看来",
      "说实话",
      "坦白讲",
      "老实说",
      "不瞒你说",
      "我的感受是"
    ],
    "internet_slang": [
      "yyds",
      "绝绝子",
      "破防",
      "emo",
      "cpu",
      "拿捏",
      "整活",
      "有梗",
      "无语",
      "6",
      "离谱",
      "逆天",
      "炸裂",
      "社死",
      "摆烂"
    ]
  },
  "replacements": {
    "值得注意的是": [
      "注意",
      "要提醒的是",
      "特别说一下",
      "有个点"
    ],
    "综上所述": [
      "总之",
      "说到底",
      "简单讲",
      "归结起来"
    ],
    "不难发现": [
      "可以看到",
      "很明显",
      "你会发现"
    ],
    "总而言之": [
      "总之",
      "总的来说",
      "一句话"
    ],
    "与此同时": [
      "同时",
      "这时候",
      "另一边"
    ],
    "在此基础上": [
      "基于这个",
      "在这上面",
      "以此为基础"
    ],
    "由此可见": [
      "所以",
      "可见",
      "看得出"
    ],
    "此外": [
      "另外",
      "还有",
      "再说"
    ],
    "需要指出的是": [
      "要说的是",
      "需要注意",
      "提一嘴"
    ],
    "值得一提的是": [
      "值得说的是",
      "还有一点",
      "顺带一提"
    ],
    "赋能": [
      "升级",
      "改进",
      "提升"
    ],
    "闭环": [
      "完整流程",
      "全链路",
      "一条龙"
    ],
    "深度剖析": [
      "深入分析",
      "仔细看看",
      "拆解一下"
    ],
    "深度融合": [
      "结合",
      "融入",
      "渗透到"
    ],
    "全面升级": [
      "升级",
      "进步"
    ],
    "实现高质量发展": [
      "发展得更好"
    ],
    "高质量发展": [
      "更好地发展"
    ],
    "从某种意义上说": [
      "可以说",
      "某种程度上"
    ],
    "毫无疑问": [
      "肯定的是",
      "可以肯定"
    ],
    "贡献力量": [
      "出一份力",
      "做点贡献"
    ],
    "前所未有的": [
      "很大的",
      "巨大的"
    ],
    "不可替代的": [
      "关键的",
      "重要的"
    ],
    "一方面": [
      "一来"
    ],
    "另一方面": [
      "二来"
    ],
    "在智慧时代中": [
      "在这个时代"
    ],
    "智慧时代": [
      "这个时代",
      "现在"
    ],
    "数字化转型": [
      "数字化",
      "转型升级"
    ],
    "助力": [
      "帮助",
      "支持",
      "推动"
    ],
    "彰显": [
      "展现",
      "体现",
      "表现出"
    ],
    "凸显": [
      "突出",
      "显示",
      "暴露出"
    ],
    "焕发": [
      "展现",
      "释放",
      "激发"
    ],
    "不可否认的是": [
      "确实",
      "不得不承认"
    ],
    "不可否认": [
      "确实",
      "没错"
    ],
    "毋庸置疑": [
      "毫无疑问",
      "肯定的是"
    ],
    "显而易见": [
      "明摆着",
      "很明显"
    ],
    "不言而喻": [
      "不用说",
      "大家都懂"
    ],
    "底层逻辑": [
      "根本原因",
      "核心逻辑",
      "本质"
    ],
    "抓手": [
      "切入点",
      "突破口",
      "着力点"
    ],
    "触达": [
      "接触到",
      "覆盖",
      "达到"
    ],
    "心智": [
      "认知",
      "印象",
      "想法"
    ],
    "沉淀": [
      "积累",
      "总结",
      "留下"
    ],
    "对齐": [
      "统一",
      "同步",
      "拉齐"
    ],
    "拉通": [
      "打通",
      "贯穿",
      "串联"
    ],
    "复盘": [
      "回顾",
      "总结",
      "反思"
    ],
    "随着.*?的(不断)?发展[，,]\\s*在当今.*?时代[，,]": [
      "如今，",
      "现在，",
      "眼下，"
    ],
    "随着.*?的(不断)?发展[，,]": [
      "如今，",
      "现在，",
      "眼下，"
    ],
    "在当今.*?时代[，,]": [
      "现在，",
      "当下，"
    ],
    "在.*?的背景下[，,]": [
      "考虑到这些，",
      "面对这个局面，"
    ]
  },
  "punctuation_thresholds": {
    "em_dash_density": 1.0,
    "semicolon_density": 0.5,
    "ellipsis_density": 1.5,
    "exclamation_density": 2.0
  },
  "scoring": {
    "description": "加权评分系统，满分100",
    "critical_multiplier": 3,
    "high_multiplier": 2,
    "medium_multiplier": 1,
    "style_multiplier": 1,
    "levels": {
      "low": [
        0,
        25
      ],
      "medium": [
        25,
        50
      ],
      "high": [
        50,
        75
      ],
      "very_high": [
        75,
        100
      ]
    }
  },
  "academic_patterns": {
    "description": "学术论文 AIGC 检测专用模式 — 针对知网/维普 AIGC 检测标准",
    "version": "1.0.0",
    "ai_academic_phrases": {
      "weight": 8,
      "description": "AI 生成学术论文的典型措辞",
      "phrases": [
        "本文旨在",
        "研究表明",
        "具有重要意义",
        "进行了深入分析",
        "具有重要的理论意义和实践价值",
        "为此本文",
        "鉴于此",
        "对此进行了系统的",
        "进行了全面的分析",
        "进行了系统的研究",
        "取得了显著的成效",
        "具有广阔的应用前景",
        "发挥着重要作用",
        "引起了广泛关注",
        "得到了广泛的应用",
        "提供了有力的支撑",
        "提出了切实可行的",
        "进行了有益的探索",
        "奠定了坚实的基础",
        "提供了重要的参考价值",
        "提供了新的思路",
        "开辟了新的途径",
        "进行了详细的阐述",
        "做出了重要贡献",
        "具有一定的参考价值",
        "受到了学术界的广泛关注",
        "成为学术界研究的热点",
        "具有十分重要的现实意义",
        "有着不可忽视的作用",
        "对其进行深入研究",
        "有待进一步研究"
      ]
    },
    "passive_patterns": {
      "weight": 5,
      "description": "学术论文中过度使用的被动句式",
      "regex": [
        "被广泛(应用|使用|采用|运用|关注|认可|接受)",
        "被认为是",
        "被(视为|看作|称为|誉为)",
        "被(普遍|学界|学术界|研究者)(认为|接受|采纳)",
        "被(充分|有效|合理)(利用|运用|开发)",
        "被(证明|证实|验证)(为|是)",
        "受到了?(广泛|高度|普遍|学界的?)(关注|重视|认可)"
      ]
    },
    "logic_connectors": {
      "weight": 7,
      "description": "逻辑连接词（密度过高是 AI 特征）",
      "phrases": [
        "因此",
        "所以",
        "由此可见",
        "综上所述",
        "总而言之",
        "此外",
        "另外",
        "与此同时",
        "不仅如此",
        "更重要的是",
        "换言之",
        "也就是说",
        "具体而言",
        "进而",
        "从而",
        "然而",
        "但是",
        "不过",
        "尽管如此",
        "相比之下",
        "一方面",
        "另一方面",
        "首先",
        "其次",
        "最后",
        "事实上",
        "实际上",
        "显然",
        "毫无疑问",
        "值得注意的是",
        "需要指出的是",
        "值得一提的是",
        "由此",
        "基于此"
      ],
      "density_threshold": 3.0
    },
    "citation_templates": {
      "weight": 4,
      "description": "模板化的引用整合方式",
      "regex": [
        "[（(]\\d{4}[）)]指出",
        "[（(]\\d{4}[）)]认为",
        "[（(]\\d{4}[）)]的研究(表明|发现|指出|证实)",
        "根据\\S{1,6}[（(]\\d{4}[）)]的研究",
        "正如\\S{1,6}[（(]\\d{4}[）)]所(指出|述|言)",
        "\\S{1,6}等[（(]\\d{4}[）)](的研究)?(表明|发现|认为)"
      ]
    },
    "data_templates": {
      "weight": 5,
      "description": "数据论述的模板化表达",
      "phrases": [
        "数据显示",
        "数据表明",
        "从表中可以看出",
        "如表所示",
        "从图中可以看出",
        "如图所示",
        "由表可知",
        "由图可知",
        "根据表中数据",
        "从上表可以看出",
        "通过数据分析可知",
        "统计结果表明",
        "分析结果显示",
        "实验结果表明",
        "调查结果显示",
        "从数据中不难发现",
        "数据分析结果表明",
        "表中数据表明",
        "从图表中可以清晰地看出"
      ]
    },
    "certainty_markers": {
      "weight": 7,
      "description": "过于确定的语气（缺少学术留白）",
      "phrases": [
        "必然",
        "一定会",
        "毫无疑问",
        "无疑",
        "显然",
        "毋庸置疑",
        "不可否认",
        "确定无疑",
        "势必",
        "必将",
        "注定",
        "无一例外",
        "绝对"
      ]
    },
    "perfect_conclusion": {
      "weight": 6,
      "description": "过于圆满的结论表述",
      "phrases": [
        "取得了显著成效",
        "取得了丰硕成果",
        "取得了重大突破",
        "圆满完成",
        "成效显著",
        "效果明显",
        "成果丰硕",
        "为.*?提供了有力支撑",
        "为.*?奠定了坚实基础",
        "具有重要的理论意义和实践价值",
        "前景广阔"
      ]
    },
    "replacements": {
      "description": "学术场景专用替换词库（120+ 条）",
      "本文旨在": [
        "本研究聚焦于",
        "本文着重",
        "本文尝试"
      ],
      "研究表明": [
        "已有文献指出",
        "前人研究发现",
        "相关研究揭示"
      ],
      "具有重要意义": [
        "不容忽视",
        "值得关注",
        "有其独特价值"
      ],
      "具有重要的理论意义和实践价值": [
        "兼具理论探索与实践参考的双重价值",
        "在学理与应用两个层面均有所推进"
      ],
      "进行了深入分析": [
        "作了较为细致的考察",
        "做了逐层剖析",
        "展开了多角度的审视"
      ],
      "进行了全面的分析": [
        "从多个维度加以考察",
        "较为系统地做了梳理"
      ],
      "进行了系统的研究": [
        "围绕该问题做了较为完整的探索",
        "对此做了逐步推进的研究"
      ],
      "取得了显著的成效": [
        "收到了一定的效果",
        "初步达到了预期目标"
      ],
      "具有广阔的应用前景": [
        "在应用层面仍有可拓展的空间",
        "其实践潜力有待进一步挖掘"
      ],
      "发挥着重要作用": [
        "扮演着不可或缺的角色",
        "在其中起到了关键的支撑"
      ],
      "引起了广泛关注": [
        "逐渐进入研究者的视野",
        "日益受到学界的重视"
      ],
      "得到了广泛的应用": [
        "已在多个领域落地",
        "实际应用范围持续扩大"
      ],
      "提供了有力的支撑": [
        "为后续工作铺垫了基础",
        "在技术层面给予了支持"
      ],
      "提出了切实可行的": [
        "给出了具有操作性的",
        "拟定了较为务实的"
      ],
      "进行了有益的探索": [
        "做了初步的尝试",
        "在这一方向上迈出了一步"
      ],
      "奠定了坚实的基础": [
        "打下了一定的基础",
        "在基础层面做了铺垫"
      ],
      "提供了重要的参考价值": [
        "对相关研究有所启发",
        "可为后续研究提供借鉴"
      ],
      "提供了新的思路": [
        "打开了一个新的切入角度",
        "在思路上有所创新"
      ],
      "开辟了新的途径": [
        "探索出一条可能的路径",
        "尝试了一种新的进路"
      ],
      "进行了详细的阐述": [
        "做了较为充分的说明",
        "逐一加以论述"
      ],
      "做出了重要贡献": [
        "在该领域有所推进",
        "对这一问题的认识起到了补充作用"
      ],
      "具有一定的参考价值": [
        "可供后来者参考",
        "对同领域研究有所帮助"
      ],
      "受到了学术界的广泛关注": [
        "近年来已成为学界讨论的议题之一"
      ],
      "成为学术界研究的热点": [
        "在学界引发了持续的讨论"
      ],
      "具有十分重要的现实意义": [
        "对当下的实际问题有着直接的观照意义"
      ],
      "有着不可忽视的作用": [
        "其作用不宜低估",
        "在其中发挥着不小的影响"
      ],
      "对其进行深入研究": [
        "对此做进一步的追问",
        "就此展开更细致的考察"
      ],
      "有待进一步研究": [
        "仍有继续追问的空间",
        "后续尚可深入"
      ],
      "鉴于此": [
        "基于上述考虑",
        "考虑到这一背景"
      ],
      "为此本文": [
        "在此基础上，本文",
        "因此本文尝试"
      ],
      "被广泛应用": [
        "已在多个场景中得到使用",
        "使用范围较广"
      ],
      "被广泛使用": [
        "使用颇为普遍",
        "在实际中频繁出现"
      ],
      "被广泛采用": [
        "已为多数研究者所采纳",
        "采用率较高"
      ],
      "被广泛关注": [
        "持续吸引着研究者的目光",
        "关注度逐年上升"
      ],
      "被认为是": [
        "一般被视为",
        "通常归入",
        "多被看作"
      ],
      "被视为": [
        "常被看作",
        "往往归类为"
      ],
      "被看作": [
        "通常被理解为",
        "多半被归为"
      ],
      "被称为": [
        "学界习惯称之为",
        "一般称其为"
      ],
      "被普遍认为": [
        "多数学者倾向于认为",
        "主流观点认为"
      ],
      "被证明": [
        "经验证",
        "已有证据支持"
      ],
      "被证实": [
        "得到了实证支持",
        "验证结果支持"
      ],
      "受到了广泛关注": [
        "引发了较多讨论",
        "获得了较高的关注度"
      ],
      "受到了高度重视": [
        "得到了相当程度的重视"
      ],
      "数据显示": [
        "从数据来看",
        "数据层面反映出",
        "统计数字指向"
      ],
      "数据表明": [
        "数据所呈现的趋势是",
        "对数据的解读提示"
      ],
      "从表中可以看出": [
        "观察该表",
        "表中所列数据反映出",
        "对比表中各列"
      ],
      "如表所示": [
        "参见下表",
        "该表汇总了",
        "以下表格列出了"
      ],
      "从图中可以看出": [
        "对照该图",
        "图中曲线显示",
        "观察图形走势"
      ],
      "如图所示": [
        "参见下图",
        "图中呈现的规律是"
      ],
      "由表可知": [
        "该表揭示了",
        "梳理表中数据后发现"
      ],
      "由图可知": [
        "图形走势反映出",
        "从该图可以初步判断"
      ],
      "根据表中数据": [
        "依据表内数据",
        "结合表中各项指标"
      ],
      "从上表可以看出": [
        "上表所列数据提示",
        "回顾上表"
      ],
      "通过数据分析可知": [
        "经过数据梳理后发现",
        "对数据做初步分析后"
      ],
      "统计结果表明": [
        "统计分析给出的结论是",
        "数据的统计特征指向"
      ],
      "分析结果显示": [
        "分析结果提示",
        "根据分析输出"
      ],
      "实验结果表明": [
        "实验所得结果指向",
        "从实验数据来看"
      ],
      "调查结果显示": [
        "问卷反馈汇总后发现",
        "受访者的回答反映出"
      ],
      "从数据中不难发现": [
        "稍加留意便可发现",
        "数据中隐含着"
      ],
      "数据分析结果表明": [
        "对数据做深入分析后的结论是"
      ],
      "从图表中可以清晰地看出": [
        "图表较为直观地呈现了"
      ],
      "综上所述": [
        "回顾以上讨论",
        "梳理上文的论述"
      ],
      "总而言之": [
        "概括地说",
        "归结起来"
      ],
      "由此可见": [
        "从这一点延伸来看",
        "这也提示我们"
      ],
      "不难发现": [
        "稍加留意便可发现",
        "细看之下"
      ],
      "值得注意的是": [
        "需要留意的一点是",
        "这里有一个值得关注的细节"
      ],
      "需要指出的是": [
        "这里需要做一个说明",
        "有必要补充的是"
      ],
      "值得一提的是": [
        "此外还有一个值得提及的细节",
        "顺便一提"
      ],
      "不仅如此": [
        "除此之外",
        "在此基础上还"
      ],
      "与此同时": [
        "同一时期",
        "在此过程中"
      ],
      "在此基础上": [
        "以此为起点",
        "在上述结论的延长线上"
      ],
      "毫无疑问": [
        "应当承认",
        "较为确定的是"
      ],
      "事实上": [
        "实际情况是",
        "就实际而言"
      ],
      "显而易见": [
        "较为明显的是",
        "比较直观的一点是"
      ],
      "不可否认": [
        "应当承认",
        "确实"
      ],
      "更重要的是": [
        "更关键的一层是",
        "进一步说"
      ],
      "换言之": [
        "换一种表述",
        "用另一种方式理解"
      ],
      "具体而言": [
        "展开来说",
        "逐一来看"
      ],
      "此外": [
        "另外还有",
        "除上述之外"
      ],
      "进而": [
        "由此推进",
        "在此基础上进一步"
      ],
      "从而": [
        "进而使得",
        "以此达到"
      ],
      "基于此": [
        "出于这一考虑",
        "以此为出发点"
      ],
      "总的来说": [
        "整体看来",
        "从宏观上概括"
      ],
      "必然": [
        "大概率",
        "在多数情况下会"
      ],
      "一定会": [
        "很可能会",
        "大概率将"
      ],
      "无疑": [
        "在很大程度上",
        "应当说"
      ],
      "毋庸置疑": [
        "较为明确的是",
        "可以说"
      ],
      "确定无疑": [
        "在现有证据下可以认为",
        "目前的结论倾向于"
      ],
      "势必": [
        "很可能",
        "有较大概率"
      ],
      "必将": [
        "预计将",
        "有望"
      ],
      "注定": [
        "大概率会",
        "从趋势看可能"
      ],
      "无一例外": [
        "在绝大多数情况下",
        "几乎所有案例中"
      ],
      "绝对": [
        "在很大程度上",
        "基本上"
      ],
      "显然": [
        "从现有资料来看",
        "比较明确的是"
      ],
      "随着社会的不断发展": [
        "近年来社会变迁加速",
        "在社会持续演进的过程中"
      ],
      "在当今社会": [
        "当前",
        "在眼下的社会语境中"
      ],
      "在当今时代": [
        "当下",
        "就当前阶段而言"
      ],
      "具有重要的现实意义": [
        "对当下的问题有着切实的回应",
        "与当前的现实需求直接相关"
      ],
      "本文主要研究": [
        "本文聚焦的问题是",
        "本文所考察的对象是"
      ],
      "本文的研究目的": [
        "本文试图解答的问题",
        "本文关心的核心议题"
      ],
      "被广泛应用于": [
        "已广泛运用于",
        "广泛用于"
      ],
      "必然会": [
        "很可能会",
        "大概率会"
      ]
    }
  }
}

path: scripts/style_cn.py
content: #!/usr/bin/env python3
"""
Chinese Text Style Transformer v2.0
将文本转换为不同的中文写作风格
支持：口语化、知乎、小红书、公众号、学术、文艺、微博
"""

import sys
import re
import random
import argparse

# ─── 风格配置 ───

STYLES = {
    'casual': {
        'name': '口语化风格',
        'description': '像朋友聊天，适合社交媒体',
    },
    'zhihu': {
        'name': '知乎风格',
        'description': '理性、有深度、带个人观点',
    },
    'xiaohongshu': {
        'name': '小红书风格',
        'description': '活泼、emoji多、种草感',
    },
    'wechat': {
        'name': '公众号风格',
        'description': '有故事感、引人入胜',
    },
    'academic': {
        'name': '学术风格',
        'description': '严谨但不死板',
    },
    'literary': {
        'name': '文艺风格',
        'description': '有文学感、意境',
    },
    'weibo': {
        'name': '微博风格',
        'description': '简短、有态度、适合传播',
    },
}

# ─── 通用工具 ───

FORMAL_TO_CASUAL = {
    '首先': '',
    '其次': '再说',
    '最后': '最后',
    '值得注意的是': '注意',
    '综上所述': '总之',
    '不难发现': '可以看到',
    '总而言之': '总的来说',
    '与此同时': '同时',
    '在此基础上': '基于这个',
    '由此可见': '所以',
    '此外': '另外',
    '然而': '但是',
    '因此': '所以',
    '并且': '而且',
    '不可否认': '确实',
    '显而易见': '明摆着',
    '不言而喻': '不用说',
    '毋庸置疑': '肯定的是',
}

EMOJIS = {
    'positive': ['😊', '👍', '❤️', '🎉', '✨', '💪', '🔥', '👏', '💯', '🌟'],
    'thinking': ['🤔', '💭', '💡', '🧐', '👀'],
    'warning': ['⚠️', '❗', '⚡'],
    'casual': ['😂', '😅', '🙃', '😎', '🥲'],
    'xhs': ['✨', '💕', '🌸', '💗', '🎀', '💫', '🌈', '💖', '🧸', '🍃'],
}

def remove_formal_structure(text):
    """移除正式三段式结构"""
    text = re.sub(r'首先[，,]\s*', '', text)
    text = re.sub(r'其次[，,]\s*', '', text)
    text = re.sub(r'最后[，,]\s*', '', text)
    text = re.sub(r'第[一二三四五六][，,、]\s*', '', text)
    text = re.sub(r'其[一二三][，,、]\s*', '', text)
    return text

def replace_formal_words(text):
    """替换正式用词为口语化"""
    for formal, casual in FORMAL_TO_CASUAL.items():
        if formal in text:
            if casual:
                text = text.replace(formal, casual)
            else:
                text = text.replace(formal + '，', '')
                text = text.replace(formal, '')
    return text

def add_emojis(text, category='positive', density=0.2):
    """在句尾添加 emoji"""
    pool = EMOJIS.get(category, EMOJIS['positive'])
    sentences = re.split(r'([。！？])', text)
    result = []
    
    for i in range(0, len(sentences) - 1, 2):
        sent = sentences[i]
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''
        
        if random.random() < density and sent.strip():
            emoji = random.choice(pool)
            result.append(sent + emoji + punct)
        else:
            result.append(sent + punct)
    
    if len(sentences) % 2 == 1 and sentences[-1].strip():
        result.append(sentences[-1])
    
    return ''.join(result)

def shorten_paragraphs(text, max_length=120):
    """缩短段落"""
    paragraphs = text.split('\n\n')
    result = []
    
    for para in paragraphs:
        if len(para) > max_length:
            sentences = re.split(r'([。！？])', para)
            chunks = []
            current = ''
            
            for i in range(0, len(sentences) - 1, 2):
                s = sentences[i] + (sentences[i + 1] if i + 1 < len(sentences) else '')
                if len(current) + len(s) > max_length and current:
                    chunks.append(current.strip())
                    current = s
                else:
                    current += s
            if current.strip():
                chunks.append(current.strip())
            result.extend(chunks)
        else:
            result.append(para)
    
    return '\n\n'.join(result)

def strip_emojis(text):
    """Remove all emoji characters"""
    # Comprehensive emoji regex
    emoji_pattern = re.compile(
        "["
        "\U0001F600-\U0001F64F"  # emoticons
        "\U0001F300-\U0001F5FF"  # symbols & pictographs
        "\U0001F680-\U0001F6FF"  # transport & map
        "\U0001F1E0-\U0001F1FF"  # flags
        "\U00002702-\U000027B0"  # dingbats
        "\U000024C2-\U0001F251"  # enclosed characters
        "\U0001F900-\U0001F9FF"  # supplemental symbols
        "\U0001FA00-\U0001FA6F"  # chess symbols
        "\U0001FA70-\U0001FAFF"  # symbols extended
        "\U00002600-\U000026FF"  # misc symbols
        "\U0000FE00-\U0000FE0F"  # variation selectors
        "\U0000200D"             # ZWJ
        "\U00002B50"             # star
        "\U00002764"             # heart
        "]+",
        flags=re.UNICODE
    )
    return emoji_pattern.sub('', text)

# ─── 风格转换函数 ───

def transform_casual(text):
    """口语化风格"""
    text = remove_formal_structure(text)
    text = replace_formal_words(text)
    
    sentences = re.split(r'([。！？])', text)
    result = []
    
    openers_used = False
    for i in range(0, len(sentences) - 1, 2):
        sent = sentences[i].strip()
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''
        
        if not sent:
            continue
        
        # 开头加口语化引导
        if not openers_used and i == 0:
            opener = random.choice(['你看', '说起来', '其实', '讲真'])
            sent = opener + '，' + sent
            openers_used = True
        
        # 随机加语气词
        if random.random() < 0.15:
            particle = random.choice(['吧', '呢', '啊', '嘛'])
            sent = sent + particle
        
        # 随机加口语连接
        if i > 0 and random.random() < 0.15:
            connector = random.choice(['不过', '话说', '说实话', '确实'])
            sent = connector + '，' + sent
        
        result.append(sent + punct)
    
    if len(sentences) % 2 == 1 and sentences[-1].strip():
        result.append(sentences[-1])
    
    text = ''.join(result)
    text = add_emojis(text, 'casual', density=0.1)
    return text

def transform_zhihu(text):
    """知乎风格"""
    text = remove_formal_structure(text)
    text = replace_formal_words(text)
    
    sentences = re.split(r'([。！？])', text)
    result = []
    total = len(sentences) // 2
    
    # 添加个人视角
    opinion_added = False
    example_added = False
    
    for i in range(0, len(sentences) - 1, 2):
        sent = sentences[i].strip()
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''
        
        if not sent:
            continue
        
        # 第一句加个人标记
        if i == 0 and not opinion_added:
            marker = random.choice(['从我的经验来看', '说点个人看法', '这个问题我来聊聊'])
            sent = marker + '，' + sent
            opinion_added = True
        
        # 中间加举例
        if not example_added and i >= total // 2 and random.random() < 0.3:
            sent = '举个例子，' + sent
            example_added = True
        
        # 在某些句子前加数据/经验支撑（避免和已有连接词叠加）
        connector_words = ['总之', '另外', '还有', '所以', '但是', '不过', '同时']
        has_connector = any(sent.startswith(w) for w in connector_words)
        if not has_connector and random.random() < 0.1:
            support = random.choice(['实际上', '从数据来看', '根据实际经验'])
            sent = support + '，' + sent
        
        result.append(sent + punct)
    
    if len(sentences) % 2 == 1 and sentences[-1].strip():
        result.append(sentences[-1])
    
    text = ''.join(result)
    
    # 知乎结尾
    if random.random() < 0.3:
        endings = [
            '\n\n以上，希望对你有帮助。',
            '\n\n欢迎讨论。',
            '\n\n个人观点，仅供参考。',
        ]
        text += random.choice(endings)
    
    return text

def transform_xiaohongshu(text):
    """小红书风格"""
    text = remove_formal_structure(text)
    text = replace_formal_words(text)
    
    # 开头
    openers = [
        '姐妹们！今天必须分享一下',
        '天哪！终于忍不住要说了',
        '家人们谁懂啊',
        '绝了绝了！分享一个',
        '姐妹们冲！',
    ]
    
    # 处理句子
    sentences = re.split(r'([。！？])', text)
    result = []
    
    for i in range(0, len(sentences) - 1, 2):
        sent = sentences[i].strip()
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''
        
        if not sent:
            continue
        
        # 替换中性词为种草用语
        xhs_replacements = {
            '好': random.choice(['绝了', '太好了', '爱了']),
            '推荐': random.choice(['强推', '安利', '必入']),
            '不错': random.choice(['真的绝', '超赞', 'yyds']),
        }
        for old, new in xhs_replacements.items():
            if old in sent and random.random() < 0.3:
                sent = sent.replace(old, new, 1)
        
        # 加感叹号比例
        if random.random() < 0.3:
            punct = '！'
        
        result.append(sent + punct)
    
    if len(sentences) % 2 == 1 and sentences[-1].strip():
        result.append(sentences[-1])
    
    text = random.choice(openers) + '～\n\n' + ''.join(result)
    
    # 加 emoji（高密度）
    text = add_emojis(text, 'xhs', density=0.4)
    
    # 短段落
    text = shorten_paragraphs(text, max_length=80)
    
    # 加话题标签
    tags = random.sample([
        '#好物分享', '#种草', '#实用推荐', '#干货分享',
        '#生活分享', '#经验分享', '#必看', '#宝藏',
    ], 3)
    text += '\n\n' + ' '.join(tags)
    
    return text

def transform_wechat(text):
    """公众号风格"""
    text = remove_formal_structure(text)
    
    # 故事化开头
    openings = [
        '你有没有想过这样一个问题——',
        '最近发生了一件有意思的事。',
        '很多人都经历过这样的困境。',
        '说一个真实的故事。',
        '前几天有个朋友问我：',
    ]
    
    if not any(text.startswith(op[:3]) for op in openings):
        text = random.choice(openings) + '\n\n' + text
    
    # 在中间插入互动性问句
    sentences = re.split(r'([。！？])', text)
    result = []
    total = len(sentences) // 2
    question_added = False
    
    for i in range(0, len(sentences) - 1, 2):
        sent = sentences[i].strip()
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''
        
        if not sent:
            continue
        
        # 中间插反问
        if not question_added and i >= total // 3 and i <= total * 2 // 3 and random.random() < 0.4:
            questions = ['你可能会问', '为什么呢', '这意味着什么']
            result.append('\n\n' + random.choice(questions) + '？\n\n')
            question_added = True
        
        result.append(sent + punct)
    
    if len(sentences) % 2 == 1 and sentences[-1].strip():
        result.append(sentences[-1])
    
    text = ''.join(result)
    
    # 公众号结尾
    if random.random() < 0.4:
        endings = [
            '\n\n**如果觉得有用，欢迎点赞收藏。**',
            '\n\n你怎么看？评论区聊聊。',
            '\n\n希望这篇对你有启发。',
        ]
        text += random.choice(endings)
    
    return text

def transform_academic(text):
    """学术风格"""
    # 口语化 → 学术化
    casual_to_formal = {
        '很': '较为',
        '特别': '尤其',
        '挺': '相对',
        '蛮': '较为',
        '说实话': '',
        '确实': '',
        '讲真': '',
        '没想到': '',
        '说起来': '',
        '你看': '',
    }
    
    for casual, formal in casual_to_formal.items():
        if casual in text:
            if formal:
                text = text.replace(casual, formal)
            else:
                text = text.replace(casual + '，', '')
                text = text.replace(casual, '')
    
    # 移除 emoji
    text = strip_emojis(text)
    
    # 学术化连接词
    text = text.replace('所以，', '因此，')
    text = text.replace('但是，', '然而，')
    text = text.replace('而且，', '此外，')
    
    # 移除口语化语气词
    text = re.sub(r'[吧呢啊嘛哦哈]([。！？，])', r'\1', text)
    
    # 把 ！ 换成 。 （学术文章少用感叹号）
    exclamation_count = text.count('！')
    if exclamation_count > 2:
        # Keep at most 1 exclamation
        parts = text.split('！')
        text = '。'.join(parts[:-1]) + '。' + parts[-1] if len(parts) > 1 else text
    
    return text

def transform_literary(text):
    """文艺风格"""
    text = remove_formal_structure(text)
    text = replace_formal_words(text)
    
    # 意象词库
    imagery = {
        '开始': ['如同晨曦初现', '宛若春风拂面'],
        '结束': ['终如落日沉入海平线', '在最后的余晖中'],
        '变化': ['恰似流水无声地改变河床', '如同季节的更迭'],
        '重要': ['像一颗定海神针', '如同根基之于大厦'],
        '困难': ['像是在荆棘中寻路', '仿佛逆流而上的鱼'],
        '美好': ['如诗如画', '像一首未完的歌'],
    }
    
    # 随机替换（克制地）
    replacements_made = 0
    for literal, metaphors in imagery.items():
        if literal in text and replacements_made < 2 and random.random() < 0.3:
            text = text.replace(literal, random.choice(metaphors), 1)
            replacements_made += 1
    
    # 添加感官描写
    sensory_inserts = [
        '空气中弥漫着', '光线透过', '远处传来',
        '指尖触碰到', '目光所及之处',
    ]
    
    sentences = re.split(r'([。！？])', text)
    result = []
    sensory_added = False
    
    for i in range(0, len(sentences) - 1, 2):
        sent = sentences[i].strip()
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''
        
        if not sent:
            continue
        
        # 偶尔添加感官描写开头
        if not sensory_added and random.random() < 0.15 and i > 0:
            sensory = random.choice(sensory_inserts)
            sent = sensory + '——' + sent
            sensory_added = True
        
        result.append(sent + punct)
    
    if len(sentences) % 2 == 1 and sentences[-1].strip():
        result.append(sentences[-1])
    
    return ''.join(result)

def transform_weibo(text):
    """微博风格"""
    text = remove_formal_structure(text)
    text = replace_formal_words(text)
    
    # 微博要短，有态度
    sentences = re.split(r'([。！？])', text)
    result = []
    
    # 只保留前几句精华
    sent_count = 0
    max_sents = 5
    
    for i in range(0, len(sentences) - 1, 2):
        sent = sentences[i].strip()
        punct = sentences[i + 1] if i + 1 < len(sentences) else ''
        
        if not sent:
            continue
        
        sent_count += 1
        if sent_count > max_sents:
            break
        
        # 加态度
        if sent_count == 1 and random.random() < 0.4:
            attitude = random.choice(['说真的，', '讲道理，', ''])
            sent = attitude + sent
        
        result.append(sent + punct)
    
    text = ''.join(result)
    
    # 加话题
    text = add_emojis(text, 'casual', density=0.15)
    
    return text

# ─── 风格路由 ───

TRANSFORM_MAP = {
    'casual': transform_casual,
    'zhihu': transform_zhihu,
    'xiaohongshu': transform_xiaohongshu,
    'wechat': transform_wechat,
    'academic': transform_academic,
    'literary': transform_literary,
    'weibo': transform_weibo,
}

def apply_style(text, style_name):
    """应用指定风格"""
    if style_name not in STYLES:
        print(f'错误: 不支持的风格 "{style_name}"', file=sys.stderr)
        print(f'支持的风格: {", ".join(STYLES.keys())}', file=sys.stderr)
        sys.exit(1)
    
    transform_fn = TRANSFORM_MAP.get(style_name)
    if transform_fn:
        return transform_fn(text)
    return text

def list_styles():
    """列出所有可用风格"""
    print('可用的文本风格：\n')
    for style_id, config in STYLES.items():
        print(f'  {style_id:15s} {config["name"]:10s}  {config["description"]}')
    print(f'\n使用: python style_cn.py input.txt --style <风格> [-o output.txt]')

# ─── Main ───

def main():
    parser = argparse.ArgumentParser(description='中文文本风格转换 v2.0')
    parser.add_argument('file', nargs='?', help='输入文件路径')
    parser.add_argument('--style', help='目标风格')
    parser.add_argument('-o', '--output', help='输出文件路径')
    parser.add_argument('--list', action='store_true', help='列出所有风格')
    parser.add_argument('--seed', type=int, help='随机种子')
    
    args = parser.parse_args()
    
    if args.list:
        list_styles()
        sys.exit(0)
    
    if not args.style:
        print('错误: 必须指定 --style 参数', file=sys.stderr)
        list_styles()
        sys.exit(1)
    
    if args.seed is not None:
        random.seed(args.seed)
    
    # Read input
    if args.file:
        try:
            with open(args.file, 'r', encoding='utf-8') as f:
                text = f.read()
        except FileNotFoundError:
            print(f'错误: 文件未找到 {args.file}', file=sys.stderr)
            sys.exit(1)
    else:
        text = sys.stdin.read()
    
    if not text.strip():
        print('错误: 输入为空', file=sys.stderr)
        sys.exit(1)
    
    result = apply_style(text, args.style)
    
    if args.output:
        with open(args.output, 'w', encoding='utf-8') as f:
            f.write(result)
        print(f'✓ 已保存到 {args.output} (风格: {STYLES[args.style]["name"]})')
    else:
        print(result)

if __name__ == '__main__':
    main()


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)