Skill: edwardwason/web-to-fim (version 2.2.0)

--- name: web-to-FIM label: 网页内容转 Markdown/飞书/IMA description: > 将任意网页链接或本地文件一键转为结构化 Markdown，并保存到 Obsidian Vault、飞书云文档或腾讯 IMA 笔记。 支持的信源：(1) X/Twitter 推文、长文 Article、Thread 线程；(2) 微信公众号文章； (3) 小红书笔记；(4) 微博；(5) YouTube 视频；(6) 任意 HTML 网页； (7) 本地文件：PDF、Word、PPT、Excel、图片、音频等。 工作流：自动识别 URL/文件类型 → 路由到最佳抓取工具 → 结构化 Markdown → 选择性保存到 Obsidian/飞书/IMA。 触发词：转文档、抓网页存飞书、网页转文档、web to feishu、url转文档、文件转飞书、存到ima、存到obsidian、web to fim。 当用户提供任意 URL 或本地文件并要求转存为文档时触发。 --- # Web-to-FIM | 网页内容转 Markdown/飞书/IMA ## 🤖 AI 时代必备的信息库基础技能 在 AI 时代，无论是使用 **OpenClaw**、**Hermes Agent**，还是实践 **Obsidian + LLM** 的信息管理方法论，**一键入库、人机共用**的 AI 信息库搭建都是必备的基础设施。 **Web-to-FIM** 就是这样一个基础技能：它将任意网络内容一键转换为结构化 Markdown，并同步到： - 📝 **Obsidian Vault** - 本地个人知识库，带 frontmatter - 📚 **飞书云文档** - 云端团队协作 - 🧠 **腾讯 IMA 笔记** - AI 原生知识库 将任意网页链接或本地文件一键转为结构化 Markdown，并保存到 Obsidian Vault、飞书云文档或腾讯 IMA 笔记。 ## 支持的信源 | 信源 | URL 特征 | 抓取方式 | |------|---------|---------| | X/Twitter | `x.com` / `twitter.com` | x-tweet-fetcher | | 微信公众号 | `mp.weixin.qq.com` | markitdown-plus | | 小红书 | `xiaohongshu.com` / `xhslink.com` | markitdown-plus | | 微博 | `weibo.com` | markitdown-plus | | YouTube | `youtube.com` / `youtu.be` | markitdown-plus | | 任意网页 | 其他 `http(s)://` 链接 | markitdown-plus | ### 本地文件支持 | 类型 | 扩展名 | |------|--------| | PDF | `.pdf` | | Word | `.docx` / `.doc` | | PowerPoint | `.pptx` / `.ppt` | | Excel | `.xlsx` / `.xls` | | 图片 | `.png` `.jpg` `.jpeg` `.gif` `.webp` | | 音频 | `.mp3` `.wav` `.m4a` `.flac` | | 数据 | `.csv` `.json` `.xml` | ## 输出目的地 | 目的地 | 环境变量 | 说明 | |--------|---------|------| | Obsidian Vault | `OBSIDIAN_VAULT_PATH` | 本地保存到指定目录，带 frontmatter（默认：`E:\Obsidian\md\inbox`） | | 飞书云文档 | `FEISHU_APP_ID` + `FEISHU_APP_SECRET` | 云端服务，参考 `references/feishu-setup.md` | | 腾讯 IMA | `IMA_CLIENT_ID` + `IMA_API_KEY` | **云端 API**，无需本地客户端，参考 `references/ima-setup.md` | ### Obsidian Vault 路径配置 跨平台支持，通过环境变量 `OBSIDIAN_VAULT_PATH` 配置： ```bash # Windows (PowerShell) $env:OBSIDIAN_VAULT_PATH = "C:\Users\YourName\Obsidian\Vault\inbox" # Windows (CMD) set OBSIDIAN_VAULT_PATH=C:\Users\YourName\Obsidian\Vault\inbox # macOS/Linux export OBSIDIAN_VAULT_PATH=~/Obsidian/Vault/inbox ``` 如果未设置，默认使用： - Windows: `E:\Obsidian\md\inbox` - macOS/Linux: `~/Obsidian/inbox` ## 一键保存到所有目的地 使用 `web_to_all.py` 一键转换并保存到 Obsidian/飞书/IMA： ```bash python3 scripts/web_to_all.py --url "<url_or_path>" python3 scripts/web_to_all.py --url "<url>" --title "自定义标题" python3 scripts/web_to_all.py --url "<url>" --no-feishu --no-ima # 仅保存 Obsidian ``` ## 安全配置 ⚠️ **凭证必须通过环境变量配置**，禁止硬编码： ### 飞书配置 ```bash # 设置环境变量 $env:FEISHU_APP_ID = "your_app_id" $env:FEISHU_APP_SECRET = "your_app_secret" ``` 参考 [references/feishu-setup.md](references/feishu-setup.md) 获取凭证。 ### ima 配置 ```bash # 设置环境变量 $env:IMA_CLIENT_ID = "your_client_id" $env:IMA_API_KEY = "your_api_key" ``` 参考 [references/ima-setup.md](references/ima-setup.md) 获取凭证。 ## 工作流 ### 步骤 1：转换为 Markdown ```bash python3 scripts/web_to_md.py --url "<url_or_path>" --output <output.md> ``` 路由逻辑： - **X/Twitter** → x-tweet-fetcher 抓取 JSON → tweet_to_md.py 结构化转换 - **其他所有** → markitdown-plus 直接转换 ### 步骤 2：存入目的地 #### Obsidian Vault ```python from scripts.web_to_all import save_to_obsidian filepath = save_to_obsidian(markdown_content, title, url) ``` #### 飞书云文档 ```python from scripts.feishu_client import FeishuClient client = FeishuClient() result = client.create_document(title="文档标题", content_md=markdown_content) print(f"文档 URL: {result['url']}") ``` #### 腾讯 ima ```python from scripts.ima_client import IMAClient client = IMAClient() result = client.create_note(title="笔记标题", content=markdown_content) print(f"笔记 URL: {result['url']}") ``` ### 验证连接 ```bash # 验证飞书 python scripts/feishu_client.py --action test # 验证 ima python scripts/ima_client.py --action test ``` ## 故障处理 | 问题 | 解决方案 | |------|---------| | x.com SSL 超时 | x-tweet-fetcher 使用 FxTwitter API 中转 | | markitdown 模块丢失 | `pip install markitdown` | | 微信反爬拦截 | markitdown 自动用移动端 UA 重试 | | 飞书凭证无效 | 检查 `FEISHU_APP_ID` 和 `FEISHU_APP_SECRET` | | ima 凭证无效 | 检查 `IMA_CLIENT_ID` 和 `IMA_API_KEY` | ## 依赖与安装 ### 完整依赖列表 | 依赖 | 安装命令 | 说明 | |------|---------|------| | markitdown | `pip install markitdown` | 网页转 Markdown 核心库 | | requests | `pip install requests` | HTTP 请求 | | python-dotenv | `pip install python-dotenv` | 环境变量管理 | | x-tweet-fetcher | 克隆到 `~/.aily/workspace/skills/x-tweet-fetcher` | X/Twitter 专用抓取（可选） | ### 一键安装所有依赖 ```bash # 安装 Python 依赖 pip install markitdown requests python-dotenv ``` ### x-tweet-fetcher（可选，仅用于 X/Twitter） ```bash # 克隆到技能目录 cd ~/.aily/workspace/skills git clone https://github.com/EdwardWason/x-tweet-fetcher.git ``` 如果不安装 x-tweet-fetcher，X/Twitter 链接将使用 markitdown 直接处理。 --- ## 安全声明 ### 🔒 安全设计原则 1. **凭证安全 - 所有 API 凭证通过环境变量配置，**无硬编码** - 支持 `FEISHU_APP_ID, FEISHU_APP_SECRET, IMA_CLIENT_ID, IMA_API_KEY 2. **文件操作 - 仅在用户明确指定时写入文件 - Obsidian Vault 路径完全可控 - 支持通过 `--no-feishu` / --no-ima` 选择性禁用功能 3. **隐私保护 - 不读取任何未经授权的用户配置文件 - 所有网络请求仅针对用户提供的 URL - 本地文件仅用于转换，不主动扫描 4. **权限透明 - 文件写入范围：Obsidian Vault 目录、临时 Markdown 文件 - 文件读取范围：用户指定的 URL、本地输入文件

path: references/feishu-setup.md
content: # 飞书云文档 API 配置指南

## 获取凭证

### 步骤 1: 创建飞书应用

1. 访问 [飞书开放平台](https://open.feishu.cn/)
2. 登录后进入「开发者后台」
3. 点击「创建应用」，填写应用名称和描述
4. 获取 **App ID** 和 **App Secret**

### 步骤 2: 配置权限

在应用后台开启以下权限：

| 权限名称 | 权限说明 | 必要性 |
|---------|---------|--------|
| `docx.document:readonly` | 读取云文档 | 可选 |
| `docx.document:write` | 创建/编辑云文档 | 必须 |
| `drive:drive:folder` | 云文档文件夹操作 | 必须 |
| `sheets:Spreadsheet:readonly` | 读取表格 | 可选 |

### 步骤 3: 发布应用

1. 在「版本管理与发布」中创建版本
2. 选择「发布」并等待审核（或自建企业直接通过）

## 本地配置

### 设置环境变量

```bash
# Windows PowerShell
$env:FEISHU_APP_ID="your_app_id"
$env:FEISHU_APP_SECRET="your_app_secret"

# Linux/macOS
export FEISHU_APP_ID="your_app_id"
export FEISHU_APP_SECRET="your_app_secret"
```

### 验证配置

```python
from feishu_client import FeishuClient

client = FeishuClient()
if client.test_connection():
    print("✅ 飞书连接成功")
else:
    print("❌ 飞书连接失败，请检查凭证")
```

## 安全注意事项

⚠️ **重要提示**：
- **永远不要**将 `.env` 文件或包含真实凭证的文件提交到 Git
- 已在 `.gitignore` 中添加 `.env` 规则
- 生产环境建议使用密钥管理服务（如 AWS Secrets Manager）

## 故障排查

| 问题 | 解决方案 |
|------|---------|
| `invalid_app_id` | 检查 App ID 是否正确 |
| `app_secret` 错误 | 在飞书开放平台重置 App Secret |
| 权限不足 | 在应用后台添加所需权限并重新发布 |
| 签名校验失败 | 确保使用 HTTPS，检查时间戳是否正确 |


path: references/ima-setup.md
content: # 腾讯 ima 笔记 API 配置指南

> ⚠️ **重要说明**：ima 是腾讯提供的**云端笔记服务**，通过 API 接口创建和管理笔记，**不需要安装本地客户端**。只需获取 API 凭证即可使用。

## 获取凭证

### 步骤 1: 访问 ima 开放平台

打开 [ima.qq.com/agent-interface](https://ima.qq.com/agent-interface)

### 步骤 2: 登录并获取凭证

1. 使用 QQ 或微信扫码登录
2. 在「Agent 接口」页面获取凭证：
   - **Client ID**: 应用标识
   - **Api Key**: API 密钥

### 步骤 3: 复制凭证

登录后页面会显示你的凭证，格式类似：

```text
Client ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Api Key: ima_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```

## 本地配置

### 设置环境变量

```bash
# Windows PowerShell
$env:IMA_CLIENT_ID="your_client_id"
$env:IMA_API_KEY="your_api_key"

# Linux/macOS
export IMA_CLIENT_ID="your_client_id"
export IMA_API_KEY="your_api_key"
```

### 验证配置

```python
from scripts.ima_client import IMAClient

client = IMAClient()
if client.test_connection():
    print("✅ ima 连接成功")
else:
    print("❌ ima 连接失败，请检查凭证")
```

## ima API 功能概览

| 功能 | API 端点 | 说明 |
|------|----------|------|
| 测试连接 | `GET /v1/notebook/list` | 验证凭证是否有效 |
| 创建笔记 | `POST /v1/note/create` | 在默认笔记本创建新笔记 |
| 更新笔记 | `POST /v1/note/update` | 向现有笔记追加内容 |
| 读取笔记 | `GET /v1/note/get` | 获取笔记详情 |
| 搜索笔记 | `POST /v1/note/search` | 按关键词搜索笔记 |

## API 基础信息

- **API 地址**: `https://ima.im.qq.com/openapi`
- **认证方式**: Client-ID + Api-Key 请求头
- **数据格式**: JSON
- **无需本地客户端**: 所有操作通过云端 API 完成

## 安全注意事项

⚠️ **重要提示**：
- **Api Key 等同于密码**，不要泄露给他人
- 永远不要将真实凭证提交到 Git
- 已在 `.gitignore` 中添加 `.env` 规则
- 定期轮换 Api Key

## 故障排查

| 问题 | 解决方案 |
|------|---------|
| `401 Unauthorized` | 检查 Client ID 和 Api Key 是否正确 |
| `403 Forbidden` | 确保 ima 开发者权限已开通 |
| `rate_limit_exceeded` | API 有调用频率限制，等待后重试 |
| 网络超时 | 检查网络连接，或使用代理 |


path: requirements.txt
content: requests
python-dotenv


path: scripts/feishu_client.py
content: #!/usr/bin/env python3
"""
Feishu (Lark) Cloud Document API Client

安全说明:
- 所有凭证通过环境变量读取，不硬编码
- 支持 .env 文件加载（需 python-dotenv）
- 凭证不写入日志或任何输出

Usage:
    from feishu_client import FeishuClient
    client = FeishuClient()
    doc_id = client.create_document(title="我的笔记", content_md="# Hello")
"""

import os
import json
import re
import time
from typing import Optional, Dict, Any, List
from urllib.parse import urlencode

try:
    import requests
except ImportError:
    print("❌ requests 库未安装: pip install requests")
    raise


class FeishuClient:
    BASE_URL = "https://open.feishu.cn/open-apis"

    def __init__(self, app_id: str = None, app_secret: str = None):
        self.app_id = app_id or os.environ.get("FEISHU_APP_ID")
        self.app_secret = app_secret or os.environ.get("FEISHU_APP_SECRET")
        self._tenant_access_token = None
        self._token_expires_at = 0

        if not self.app_id or not self.app_secret:
            raise ValueError(
                "缺少飞书凭证。请设置环境变量 FEISHU_APP_ID 和 FEISHU_APP_SECRET，"
                "或参考 references/feishu-setup.md 获取凭证"
            )

    def _get_token(self) -> str:
        """获取 tenant_access_token，带缓存"""
        if self._tenant_access_token and time.time() < self._token_expires_at:
            return self._tenant_access_token

        url = f"{self.BASE_URL}/auth/v3/tenant_access_token/internal"
        payload = {
            "app_id": self.app_id,
            "app_secret": self.app_secret
        }

        resp = requests.post(url, json=payload, timeout=30)
        resp.raise_for_status()
        data = resp.json()

        if data.get("code") != 0:
            raise Exception(f"获取飞书 Token 失败: {data.get('msg')}")

        self._tenant_access_token = data["tenant_access_token"]
        self._token_expires_at = time.time() + data.get("expire", 7200) - 60
        return self._tenant_access_token

    def _headers(self) -> Dict[str, str]:
        """生成认证头"""
        return {
            "Authorization": f"Bearer {self._get_token()}",
            "Content-Type": "application/json"
        }

    def test_connection(self) -> bool:
        """验证飞书连接"""
        try:
            self._get_token()
            return True
        except Exception:
            return False

    def create_document(self, title: str, content_md: str = "") -> Dict[str, Any]:
        """
        创建飞书云文档

        Args:
            title: 文档标题
            content_md: Markdown 格式的文档内容

        Returns:
            包含文档信息的字典，包括 document_id
        """
        url = f"{self.BASE_URL}/docx/v1/documents"
        payload = {"title": title}
        resp = requests.post(url, headers=self._headers(), json=payload, timeout=30)
        resp.raise_for_status()
        data = resp.json()

        if data.get("code") != 0:
            raise Exception(f"创建飞书文档失败: {data.get('msg')}")

        doc = data["data"]["document"]
        document_id = doc["document_id"]

        if content_md:
            self._insert_blocks(document_id, content_md)

        return {
            "document_id": document_id,
            "title": title,
            "url": f"https://.feishu.cn/docx/{document_id}"
        }

    def _insert_blocks(self, document_id: str, content_md: str) -> None:
        """将 Markdown 内容插入文档"""
        blocks = self._md_to_blocks(content_md)

        url = f"{self.BASE_URL}/docx/v1/documents/{document_id}/blocks/{document_id}/children"
        payload = {"children": blocks, "index": -1}

        resp = requests.post(url, headers=self._headers(), json=payload, timeout=30)
        resp.raise_for_status()

    def _md_to_blocks(self, md: str) -> List[Dict]:
        """将 Markdown 转换为飞书文档块格式"""
        blocks = []
        lines = md.split("\n")
        in_code_block = False
        code_content = []

        for line in lines:
            stripped = line.strip()

            if stripped.startswith("```"):
                if not in_code_block:
                    in_code_block = True
                    code_content = []
                else:
                    blocks.append({
                        "block_type": 2,
                        "code": {
                            "elements": [{"type": "text_run", "text_run": {"content": "\n".join(code_content)}}],
                            "language": 1
                        }
                    })
                    in_code_block = False
                    code_content = []
                continue

            if in_code_block:
                code_content.append(line)
                continue

            if stripped.startswith("# "):
                blocks.append({
                    "block_type": 3,
                    "heading1": {
                        "elements": [{"type": "text_run", "text_run": {"content": stripped[2:]}}],
                        "style": {}
                    }
                })
            elif stripped.startswith("## "):
                blocks.append({
                    "block_type": 4,
                    "heading2": {
                        "elements": [{"type": "text_run", "text_run": {"content": stripped[3:]}}],
                        "style": {}
                    }
                })
            elif stripped.startswith("### "):
                blocks.append({
                    "block_type": 5,
                    "heading3": {
                        "elements": [{"type": "text_run", "text_run": {"content": stripped[4:]}}],
                        "style": {}
                    }
                })
            elif stripped.startswith("- ") or stripped.startswith("* "):
                blocks.append({
                    "block_type": 12,
                    "bullet": {
                        "elements": [{"type": "text_run", "text_run": {"content": stripped[2:]}}],
                        "style": {}
                    }
                })
            elif stripped.startswith("> "):
                blocks.append({
                    "block_type": 11,
                    "quote": {
                        "elements": [{"type": "text_run", "text_run": {"content": stripped[2:]}}],
                        "style": {}
                    }
                })
            elif stripped.startswith("!["):
                match = re.match(r'!\[(.*?)\]\((.*?)\)', stripped)
                if match:
                    blocks.append({
                        "block_type": 10,
                        "image": {
                            "uri": match.group(2)
                        }
                    })
            elif stripped:
                blocks.append({
                    "block_type": 2,
                    "text": {
                        "elements": [{"type": "text_run", "text_run": {"content": stripped}}],
                        "style": {}
                    }
                })
            else:
                blocks.append({
                    "block_type": 2,
                    "text": {
                        "elements": [{"type": "text_run", "text_run": {"content": " "}}],
                        "style": {}
                    }
                })

        return blocks

    def get_document_info(self, document_id: str) -> Dict[str, Any]:
        """获取文档信息"""
        url = f"{self.BASE_URL}/docx/v1/documents/{document_id}"
        resp = requests.get(url, headers=self._headers(), timeout=30)
        resp.raise_for_status()
        return resp.json()


def main():
    import argparse

    parser = argparse.ArgumentParser(description="飞书云文档 API 客户端")
    parser.add_argument("--action", choices=["test", "create"], default="test", help="操作类型")
    parser.add_argument("--title", help="文档标题")
    parser.add_argument("--content", help="Markdown 内容文件路径")
    args = parser.parse_args()

    try:
        client = FeishuClient()

        if args.action == "test":
            if client.test_connection():
                print("✅ 飞书连接成功")
            else:
                print("❌ 飞书连接失败")

        elif args.action == "create":
            if not args.title:
                print("❌ 需要指定 --title")
                return

            content = ""
            if args.content:
                with open(args.content, "r", encoding="utf-8") as f:
                    content = f.read()

            result = client.create_document(args.title, content)
            print(f"✅ 文档创建成功:")
            print(f"   ID: {result['document_id']}")
            print(f"   URL: {result['url']}")

    except ValueError as e:
        print(f"❌ 配置错误: {e}")
    except Exception as e:
        print(f"❌ 错误: {e}")


if __name__ == "__main__":
    main()


path: scripts/ima_client.py
content: #!/usr/bin/env python3
"""
Tencent IMA Notes API Client

云端 API 服务，无需本地客户端。

API 信息:
- 端点: https://ima.qq.com/openapi/note/v1
- 认证: ima-openapi-clientid + ima-openapi-apikey

Usage:
    from scripts.ima_client import IMAClient
    client = IMAClient()
    note = client.create_note(title="我的笔记", content="# Hello")
"""

import os
import json
from typing import Optional, Dict, Any, List

try:
    import requests
except ImportError:
    print("❌ requests 库未安装: pip install requests")
    raise


class IMAClient:
    BASE_URL = "https://ima.qq.com/openapi/note/v1"

    def __init__(self, client_id: str = None, api_key: str = None):
        self.client_id = client_id or os.environ.get("IMA_CLIENT_ID")
        self.[REDACTED_SECRET] or os.environ.get("IMA_API_KEY")

        if not self.client_id or not self.api_key:
            raise ValueError(
                "缺少 ima 凭证。请设置环境变量 IMA_CLIENT_ID 和 IMA_API_KEY，"
                "或参考 references/ima-setup.md 获取凭证"
            )

    def _headers(self) -> Dict[str, str]:
        return {
            "ima-openapi-clientid": self.client_id,
            "ima-openapi-apikey": self.api_key,
            "Content-Type": "application/json"
        }

    def test_connection(self) -> bool:
        try:
            url = f"{self.BASE_URL}/note/list"
            resp = requests.get(url, headers=self._headers(), timeout=30)
            if resp.status_code == 200:
                data = resp.json()
                return data.get("code") == 0
            return False
        except Exception:
            return False

    def list_notes(self, limit: int = 20) -> List[Dict[str, Any]]:
        url = f"{self.BASE_URL}/note/list"
        payload = {"limit": limit}
        resp = requests.post(url, headers=self._headers(), json=payload, timeout=30)
        resp.raise_for_status()
        data = resp.json()
        if data.get("code") != 0:
            raise Exception(f"获取笔记列表失败: {data.get('msg')}")
        return data.get("data", {}).get("notes", [])

    def create_note(self, title: str, content: str = "", folder_id: str = None) -> Dict[str, Any]:
        url = f"{self.BASE_URL}/note/create"
        payload = {
            "title": title,
            "content": content,
            "format": "markdown"
        }
        if folder_id:
            payload["folder_id"] = folder_id

        resp = requests.post(url, headers=self._headers(), json=payload, timeout=30)
        resp.raise_for_status()
        data = resp.json()

        if data.get("code") != 0:
            raise Exception(f"创建 ima 笔记失败: {data.get('msg')}")

        note_data = data.get("data", {}).get("note", {})
        return {
            "note_id": note_data.get("note_id"),
            "title": title,
            "url": note_data.get("url", f"https://ima.qq.com/note/{note_data.get('note_id')}")
        }

    def update_note(self, note_id: str, content: str, append: bool = True) -> bool:
        url = f"{self.BASE_URL}/note/update"
        payload = {
            "note_id": note_id,
            "content": content,
            "append": append
        }

        resp = requests.post(url, headers=self._headers(), json=payload, timeout=30)
        resp.raise_for_status()
        data = resp.json()
        return data.get("code") == 0

    def get_note(self, note_id: str) -> Dict[str, Any]:
        url = f"{self.BASE_URL}/note/get"
        payload = {"note_id": note_id}
        resp = requests.get(url, headers=self._headers(), params=payload, timeout=30)
        resp.raise_for_status()
        data = resp.json()
        if data.get("code") != 0:
            raise Exception(f"获取笔记失败: {data.get('msg')}")
        return data.get("data", {}).get("note", {})

    def search_notes(self, keyword: str, limit: int = 10) -> List[Dict[str, Any]]:
        url = f"{self.BASE_URL}/note/search"
        payload = {
            "query": keyword,
            "limit": limit
        }
        resp = requests.post(url, headers=self._headers(), json=payload, timeout=30)
        resp.raise_for_status()
        data = resp.json()
        if data.get("code") != 0:
            raise Exception(f"搜索笔记失败: {data.get('msg')}")
        return data.get("data", {}).get("notes", [])


def main():
    import argparse

    parser = argparse.ArgumentParser(description="腾讯 ima 笔记 API 客户端")
    parser.add_argument("--action", choices=["test", "list", "create", "search"], default="test", help="操作类型")
    parser.add_argument("--title", help="笔记标题")
    parser.add_argument("--content", help="笔记内容或内容文件路径")
    parser.add_argument("--note-id", help="笔记 ID")
    parser.add_argument("--keyword", help="搜索关键词")
    parser.add_argument("--limit", type=int, default=20, help="返回数量限制")
    args = parser.parse_args()

    try:
        client = IMAClient()

        if args.action == "test":
            if client.test_connection():
                print("✅ ima 连接成功")
            else:
                print("❌ ima 连接失败")

        elif args.action == "list":
            notes = client.list_notes(limit=args.limit)
            print(f"📚 笔记列表 ({len(notes)} 个):")
            for note in notes:
                print(f"   - {note.get('title', '未命名')} (ID: {note.get('note_id')})")

        elif args.action == "create":
            if not args.title:
                print("❌ 需要指定 --title")
                return

            content = ""
            if args.content:
                if os.path.isfile(args.content):
                    with open(args.content, "r", encoding="utf-8") as f:
                        content = f.read()
                else:
                    content = args.content

            result = client.create_note(title=args.title, content=content)
            print(f"✅ 笔记创建成功!")
            print(f"   ID: {result['note_id']}")
            print(f"   URL: {result['url']}")

        elif args.action == "search":
            if not args.keyword:
                print("❌ 需要指定 --keyword")
                return

            notes = client.search_notes(keyword=args.keyword, limit=args.limit)
            print(f"🔍 搜索结果 ({len(notes)} 个):")
            for note in notes:
                print(f"   - {note.get('title', '未命名')} (ID: {note.get('note_id')})")

    except Exception as e:
        print(f"❌ 错误: {e}")


if __name__ == "__main__":
    main()


path: scripts/tweet_to_md.py
content: #!/usr/bin/env python3
"""
Tweet JSON → Structured Markdown Converter

Usage:
  python3 tweet_to_md.py --input <tweet.json> --output <output.md>
  python3 tweet_to_md.py --input <tweet.json>  (prints to stdout)

Input: JSON file from x-tweet-fetcher (fetch_tweet.py)
Output: Well-structured Markdown with metadata header, section headers, and images.
"""

import json
import re
import argparse
import sys
from pathlib import Path


_SECTION_HEADERS = [
    "所有人都在追模型，但跑道选错了",
    "一个被忽视的行业真相",
    "不是「接入」，是「深度适配」",
    "真人短剧赛道，被悄悄解锁了",
    "从抽卡时代到工厂时代",
    "两种模式，两种人。",
    "商业价值",
    "案例背景",
    "技术突破点",
]

_SUBSECTION_PREFIXES = ["1.", "2.", "3.", "4.", "5."]
_EMOJI_HEADERS = ["🎁", "💡", "📊", "🎬", "✅", "⚠️", "🔑", "🎯"]


def _is_section_header(line: str) -> bool:
    stripped = line.strip()
    if stripped in _SECTION_HEADERS:
        return True
    if any(stripped.startswith(p) for p in _SUBSECTION_PREFIXES) and len(stripped) < 80:
        if re.match(r'^[1-5]\.\s+\S', stripped):
            return True
    if any(stripped.startswith(e) for e in _EMOJI_HEADERS) and len(stripped) < 60:
        return True
    return False


def _is_subsection_header(line: str) -> bool:
    stripped = line.strip()
    key_phrases = [
        "更深层的矛盾", "镜头组叙事", "纳米空间引擎", "脚本智能体",
        "视觉导演智能体", "宫格生视频模式", "多参生视频模式",
    ]
    for phrase in key_phrases:
        if phrase in stripped and len(stripped) < 80:
            return True
    return False


def convert_tweet_to_md(data: dict) -> str:
    tweet = data.get("tweet", {})
    article = tweet.get("article", {})
    url = data.get("url", "")
    username = data.get("username", "")

    md_parts = []

    if article.get("title"):
        title = article["title"]
    elif tweet.get("text"):
        title = tweet["text"][:80].split("\n")[0]
    else:
        title = f"Tweet by @{username}"

    md_parts.append(f"# {title}\n")

    author = tweet.get("author", "")
    screen_name = tweet.get("screen_name", username)
    created_at = tweet.get("created_at", "")
    likes = tweet.get("likes", 0)
    retweets = tweet.get("retweets", 0)
    bookmarks = tweet.get("bookmarks", 0)
    views = tweet.get("views", 0)
    replies_count = tweet.get("replies_count", 0)
    is_article = tweet.get("is_article", False)

    md_parts.append(f"> **作者**: {author} (@{screen_name})  ")
    if created_at:
        md_parts.append(f"> **发布时间**: {created_at}  ")
    md_parts.append(f"> **数据**: {likes} 赞 / {retweets} 转发 / {bookmarks} 收藏 / {views} 浏览 / {replies_count} 评论")
    if is_article:
        md_parts.append(f"> **类型**: X 长文（Article）")
    md_parts.append("")

    if article.get("preview_text"):
        md_parts.append(f"> {article['preview_text']}\n")

    md_parts.append("---\n")

    if is_article and article.get("full_text"):
        full_text = article["full_text"]
        lines = full_text.split("\n")

        for line in lines:
            stripped = line.strip()

            if stripped.startswith("![") and "](" in stripped:
                md_parts.append(f"\n{stripped}\n")
                continue

            if not stripped:
                md_parts.append("")
                continue

            if _is_section_header(stripped):
                md_parts.append(f"\n## {stripped}\n")
                continue

            if _is_subsection_header(stripped):
                md_parts.append(f"\n### {stripped}\n")
                continue

            md_parts.append(line)
    else:
        text = tweet.get("text", "")
        if text:
            md_parts.append(text)

    images = article.get("images", [])
    if images:
        md_parts.append("\n---\n")
        md_parts.append("## 附图\n")
        for img in images:
            img_url = img.get("url", "")
            img_type = img.get("type", "image")
            if img_url:
                label = "封面" if img_type == "cover" else "图片"
                md_parts.append(f"![{label}]({img_url})\n")

    replies = data.get("replies", [])
    if replies:
        md_parts.append("\n---\n")
        md_parts.append("## 评论\n")
        for reply in replies:
            reply_author = reply.get("author", "")
            reply_text = reply.get("text", "")
            reply_likes = reply.get("likes", 0)
            md_parts.append(f"**{reply_author}** (♥ {reply_likes}): {reply_text}\n")
            for nested in reply.get("thread_replies", []):
                nested_text = nested.get("text", "")
                nested_likes = nested.get("likes", 0)
                md_parts.append(f"  ↳ {nested_text} (♥ {nested_likes})\n")

    md_parts.append("\n---\n")
    md_parts.append(f"*来源: [{url}]({url})*\n")

    return "\n".join(md_parts)


def main():
    parser = argparse.ArgumentParser(description="Convert tweet JSON to Markdown")
    parser.add_argument("--input", required=True, help="Input JSON file from x-tweet-fetcher")
    parser.add_argument("--output", help="Output Markdown file (default: stdout)")
    args = parser.parse_args()

    input_path = Path(args.input)
    if not input_path.exists():
        print(f"Error: Input file not found: {input_path}", file=sys.stderr)
        sys.exit(1)

    with open(input_path, "r", encoding="utf-8") as f:
        data = json.load(f)

    if "error" in data:
        print(f"Error in input data: {data['error']}", file=sys.stderr)
        sys.exit(1)

    markdown = convert_tweet_to_md(data)

    if args.output:
        output_path = Path(args.output)
        output_path.parent.mkdir(parents=True, exist_ok=True)
        with open(output_path, "w", encoding="utf-8") as f:
            f.write(markdown)
        print(f"✅ Markdown saved: {output_path} ({len(markdown)} chars)", file=sys.stderr)
    else:
        print(markdown)


if __name__ == "__main__":
    main()


path: scripts/web_to_all.py
content: #!/usr/bin/env python3
"""
Web Content → Markdown → Obsidian/Feishu/IMA

Convert any web URL or local file to Markdown, then:
1. Save to Obsidian Vault (configurable via OBSIDIAN_VAULT_PATH)
2. Save to Feishu Cloud Document (optional)
3. Save to Tencent IMA Note (optional)

Usage:
  python3 web_to_all.py --url <url_or_path>
  python3 web_to_all.py --url <url_or_path> --title "Custom Title"
  python3 web_to_all.py --url <url_or_path> --no-feishu --no-ima
"""

import argparse
import os
import sys
from datetime import datetime
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))

from web_to_md import convert as convert_to_md
from feishu_client import FeishuClient
from ima_client import IMAClient


def get_obsidian_vault_path() -> Path:
    """
    Get Obsidian Vault path from environment variable or use default.
    
    Environment variable: OBSIDIAN_VAULT_PATH
    
    Defaults:
    - Windows: E:\Obsidian\md\inbox
    - macOS/Linux: ~/Obsidian/inbox
    """
    env_path = os.environ.get("OBSIDIAN_VAULT_PATH")
    if env_path:
        return Path(env_path).expanduser()
    
    # Default paths by OS
    if sys.platform.startswith("win"):
        return Path(r"E:\Obsidian\md\inbox")
    else:
        return Path.home() / "Obsidian" / "inbox"


OBSIDIAN_VAULT = get_obsidian_vault_path()


def save_to_obsidian(content: str, title: str, url: str = None) -> str:
    """Save Markdown to Obsidian Vault"""
    vault_path = Path(OBSIDIAN_VAULT)
    vault_path.mkdir(parents=True, exist_ok=True)

    safe_title = "".join(c if c.isalnum() or c in " -_()[]" else "_" for c in title)
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
    filename = f"{safe_title[:50]}_{timestamp}.md"
    filepath = vault_path / filename

    frontmatter = []
    frontmatter.append("---")
    frontmatter.append(f"title: {title}")
    frontmatter.append(f"date: {datetime.now().isoformat()}")
    if url:
        frontmatter.append(f"source: {url}")
    frontmatter.append("---")
    frontmatter_str = "\n".join(frontmatter)

    full_content = f"{frontmatter_str}\n\n{content}"

    with open(filepath, "w", encoding="utf-8") as f:
        f.write(full_content)

    print(f"✅ Obsidian: {filepath}")
    return str(filepath)


def save_to_feishu(content: str, title: str) -> dict:
    """Save to Feishu Cloud Document"""
    try:
        client = FeishuClient()
        result = client.create_document(title=title, content_md=content)
        print(f"✅ Feishu: {result.get('url', 'N/A')}")
        return result
    except Exception as e:
        print(f"⚠️ Feishu failed: {e}", file=sys.stderr)
        return None


def save_to_ima(content: str, title: str) -> dict:
    """Save to Tencent IMA Note"""
    try:
        client = IMAClient()
        result = client.create_note(title=title, content=content)
        print(f"✅ IMA: {result.get('url', 'N/A')}")
        return result
    except Exception as e:
        print(f"⚠️ IMA failed: {e}", file=sys.stderr)
        return None


def main():
    parser = argparse.ArgumentParser(
        description="Web Content → Markdown → Obsidian/Feishu/IMA"
    )
    parser.add_argument("--url", required=True, help="URL or local file path to convert")
    parser.add_argument("--title", help="Custom title for the document")
    parser.add_argument("--no-feishu", action="store_true", help="Skip Feishu")
    parser.add_argument("--no-ima", action="store_true", help="Skip IMA")
    parser.add_argument("--no-obsidian", action="store_true", help="Skip Obsidian")
    args = parser.parse_args()

    url = args.url

    print(f"🔍 Converting: {url}")
    try:
        markdown = convert_to_md(url)
    except Exception as e:
        print(f"❌ Conversion failed: {e}", file=sys.stderr)
        sys.exit(1)

    title = args.title
    if not title:
        first_line = markdown.strip().split("\n", 1)[0]
        if first_line.startswith("# "):
            title = first_line[2:].strip()
        else:
            title = f"Untitled - {datetime.now().strftime('%Y-%m-%d')}"

    print(f"\n📄 Title: {title}")
    print(f"📝 Markdown length: {len(markdown)} chars\n")

    results = {}

    if not args.no_obsidian:
        results["obsidian"] = save_to_obsidian(markdown, title, url)

    if not args.no_feishu:
        results["feishu"] = save_to_feishu(markdown, title)

    if not args.no_ima:
        results["ima"] = save_to_ima(markdown, title)

    print(f"\n✅ Done!")
    return 0


if __name__ == "__main__":
    sys.exit(main())


path: scripts/web_to_md.py
content: #!/usr/bin/env python3
"""
Web Content → Structured Markdown Converter (Unified Entry)

Auto-detects URL type and routes to the appropriate converter:
  - x.com / twitter.com → x-tweet-fetcher + tweet_to_md.py
  - mp.weixin.qq.com    → markitdown (WeChat plugin)
  - xiaohongshu.com     → markitdown (Xiaohongshu plugin)
  - weibo.com           → markitdown (generic)
  - youtube.com         → markitdown (YouTube support)
  - Other URLs          → markitdown (generic HTML)
  - Local files         → markitdown (PDF/DOCX/PPTX/XLSX/Images/Audio/etc.)

Usage:
  python3 web_to_md.py --url <url_or_path> --output <output.md>
  python3 web_to_md.py --url <url_or_path>  (prints to stdout)
  python3 web_to_md.py --url <tweet_url> --replies  (fetch thread replies)
"""

import argparse
import os
import subprocess
import sys
import tempfile
from pathlib import Path

SKILLS_DIR = os.path.expanduser("~/.aily/workspace/skills")
TWEET_FETCHER = os.path.join(SKILLS_DIR, "x-tweet-fetcher/scripts/fetch_tweet.py")
SCRIPT_DIR = Path(__file__).parent
TWEET_TO_MD = str(SCRIPT_DIR / "tweet_to_md.py")


def _run(cmd, check=True):
    result = subprocess.run(cmd, capture_output=True, text=True, check=check)
    return result.stdout.strip()


def _detect_source(url_or_path: str) -> str:
    lower = url_or_path.lower()

    if "x.com" in lower or "twitter.com" in lower:
        return "twitter"
    if "mp.weixin.qq.com" in lower:
        return "wechat"
    if "weibo.com" in lower:
        return "weibo"
    if "xiaohongshu.com" in lower or "xhslink.com" in lower:
        return "xiaohongshu"
    if "youtube.com" in lower or "youtu.be" in lower:
        return "youtube"
    if lower.startswith("http://") or lower.startswith("https://"):
        return "html"

    ext = Path(lower).suffix
    file_map = {
        ".pdf": "pdf", ".docx": "docx", ".doc": "docx",
        ".pptx": "pptx", ".ppt": "pptx",
        ".xlsx": "xlsx", ".xls": "xlsx",
        ".png": "image", ".jpg": "image", ".jpeg": "image",
        ".gif": "image", ".bmp": "image", ".webp": "image", ".tiff": "image",
        ".mp3": "audio", ".wav": "audio", ".m4a": "audio", ".flac": "audio",
        ".csv": "csv", ".json": "json", ".xml": "xml",
        ".zip": "zip", ".epub": "epub",
        ".md": "markdown", ".txt": "text", ".html": "html_file", ".htm": "html_file",
    }
    return file_map.get(ext, "unknown")


def _convert_twitter(url: str, output_path: str = None, replies: bool = False) -> str:
    fetch_cmd = [sys.executable, TWEET_FETCHER, "--url", url, "--pretty"]
    if replies:
        fetch_cmd.append("--replies")

    try:
        json_str = _run(fetch_cmd)
    except subprocess.CalledProcessError as e:
        print(f"❌ Failed to fetch tweet: {e.stderr}", file=sys.stderr)
        sys.exit(1)

    with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as tmp:
        tmp.write(json_str)
        tmp_json = tmp.name

    convert_cmd = [sys.executable, TWEET_TO_MD, "--input", tmp_json]
    if output_path:
        convert_cmd.extend(["--output", output_path])

    try:
        result = _run(convert_cmd)
    finally:
        if os.path.exists(tmp_json):
            os.unlink(tmp_json)

    if output_path and os.path.exists(output_path):
        with open(output_path, 'r', encoding='utf-8') as f:
            return f.read()
    return result


def _convert_markitdown(url_or_path: str, output_path: str = None) -> str:
    try:
        from markitdown import MarkItDown
    except ImportError:
        print("❌ markitdown not installed. Run: pip install markitdown", file=sys.stderr)
        sys.exit(1)

    md = MarkItDown()
    result = md.convert(url_or_path)
    content = result.markdown

    if output_path:
        out = Path(output_path)
        out.parent.mkdir(parents=True, exist_ok=True)
        with open(out, 'w', encoding='utf-8') as f:
            f.write(content)
        print(f"✅ Markdown saved: {out} ({len(content)} chars)", file=sys.stderr)

    return content


def convert(url_or_path: str, output_path: str = None, replies: bool = False) -> str:
    source_type = _detect_source(url_or_path)

    print(f"🔍 Detected source type: {source_type}", file=sys.stderr)

    if source_type == "twitter":
        return _convert_twitter(url_or_path, output_path, replies)
    else:
        return _convert_markitdown(url_or_path, output_path)


def main():
    parser = argparse.ArgumentParser(description="Web Content → Structured Markdown Converter")
    parser.add_argument("--url", required=True, help="URL or local file path to convert")
    parser.add_argument("--output", help="Output Markdown file path (default: stdout)")
    parser.add_argument("--replies", action="store_true", help="Fetch tweet replies (Twitter only)")
    args = parser.parse_args()

    content = convert(args.url, args.output, args.replies)

    if not args.output:
        print(content)


if __name__ == "__main__":
    main()


path: _meta.json
content: {
  "name": "web-to-FIM",
  "version": "2.2.0",
  "description": "将任意网页链接或本地文件一键转为结构化 Markdown，并保存到 Obsidian Vault、飞书云文档或腾讯 IMA 笔记。支持 OpenClaw 和 Hermes Agent 双平台。",
  "author": "EdwardWason",
  "homepage": "https://github.com/EdwardWason/web-to-FIM",
  "tags": ["productivity", "ai-information-management", "markdown", "feishu", "ima", "obsidian", "twitter", "wechat", "weibo", "xiaohongshu", "youtube"],
  "platforms": ["OpenClaw", "Hermes Agent"],
  "requires": {
    "binaries": ["python3"],
    "pip_packages": ["markitdown", "requests", "python-dotenv"],
    "env_vars": ["FEISHU_APP_ID", "FEISHU_APP_SECRET", "IMA_CLIENT_ID", "IMA_API_KEY", "OBSIDIAN_VAULT_PATH"]
  },
  "file_operations": {
    "writes": ["Obsidian vault directory (configurable)", "temporary markdown files"],
    "reads": ["input URLs", "local files for conversion"]
  },
  "security_notes": [
    "所有 API 凭证通过环境变量配置，无硬编码",
    "仅在用户明确指定时写入文件",
    "支持选择性禁用飞书/IMA 功能",
    "Obsidian Vault 路径完全由用户控制",
    "不读取任何未经授权的用户配置文件"
  ]
}


Security scan verdicts — ClawScan: clean | static analysis: suspicious | VirusTotal: clean | SkillSpector: suspicious (HIGH)