Skill: kylinr/sheet-data-enrichment (version 1.0.0)

--- name: sheet-data-enrichment description: "Enrich spreadsheet data by fetching external sources (URLs, APIs) to fill missing columns, then aggregate results into summary sheets. Use when: (1) a spreadsheet has URLs/links in one column and you need to extract specific info (author, title, date, etc.) into another column, (2) batch-processing rows that require visiting web pages to scrape/extract data, (3) creating pivot/summary tables from enriched data (group-by, sum, count), (4) user says fill in, complete the table, extract from links, summarize by, aggregate, enrich spreadsheet, 补全表格, 汇总统计, 信息补齐. Works with Feishu Sheets, Google Sheets, or local CSV/Excel files." --- # Sheet Data Enrichment & Summarization ## Workflow ### Phase 1: Reconnaissance 1. Read the full sheet (or specify range) to understand the schema 2. Identify the **source column** (e.g., URLs/links) and **target column** (data to fill) 3. Count empty rows in the target column — these are the work items 4. Confirm the plan with the user before proceeding ### Phase 2: Extraction Process rows in batches. For each row with an empty target column: 1. **Classify the source** before fetching: - Regular article URL → `web_fetch` - SPA / JS-rendered page → browser automation - WeChat article (mp.weixin.qq.com) → browser (captcha may block `web_fetch`) - Social media (Weibo, Douyin, etc.) → browser or note as "no extractable data" - Dead link / paywall → flag for user 2. **Extract the target data** from the fetched content: - Search for the data in multiple locations (top, bottom, metadata) - Use targeted patterns (see [references/extraction-patterns.md](references/extraction-patterns.md)) - If ambiguous, flag for manual review rather than guessing 3. **Verify before writing** (每篇先核实): - Confirm the extracted value is correct before writing to the sheet - Cross-reference with known mappings if available - When batch-processing, verify a sample first, then apply patterns ### Phase 3: Write-back 1. Write confirmed values to the target column 2. **Always verify row alignment** — read the row before writing to confirm the target cell matches the expected entry 3. Use single-cell writes (`G5:G5`) not bare references (`G5`) 4. For Feishu Sheets: range format must be `sheetId!G5:G5` ### Phase 4: Summarization When the user requests aggregation: 1. Re-read all enriched sheets to capture any manual corrections 2. Group by the requested dimensions (e.g., assignee → contributor → sum) 3. Sort groups by total descending for readability 4. Output to a new sheet/spreadsheet with headers + subtotals + grand total ## Key Lessons (Extraction) ### Where to find data on a web page Data placement varies by source. Always check multiple locations: | Position | Example | |---|---| | Below title | `作者：张三` / `By Jane Doe` | | End of article | `（记者 李四）` / `Reporter: John` | | Before timestamp | `王五 2026-03-18 14:00` | | Metadata line | `文｜赵六` / `Author: Sarah` | | Combined format | `采写：记者 孙七 编辑：周八` | **Never conclude "no data" after checking only the top of the page.** Always check the end too. ### When to use browser vs web_fetch | Signal | Tool | |---|---| | Static HTML, server-rendered | `web_fetch` (fast, cheap) | | Returns empty/minimal content | Switch to browser | | URL contains `mp.weixin.qq.com` | Browser (WeChat captcha) | | SPA framework (React/Vue/Angular) | Browser | | Baidu mini-program (`smartapps.cn`) | Browser or find alternate URL | | Social media embeds (Weibo, Douyin) | Browser, but often no structured data | ### Reusable mappings When the same source consistently maps to the same value across rows/sheets: 1. Build a mapping table after confirming 2+ instances 2. Apply mappings to future rows without re-fetching 3. Always note the mapping source for audit ### Common "no data" patterns These formats typically have no individual attribution: - Flash news / wire alerts (e.g., "财联社电", "每经AI快讯") - Press releases / corporate announcements - Social media reposts without original attribution - Aggregated roundup articles Mark these clearly (leave blank or use a placeholder like "/" per user preference). ## Row Alignment Safety **Critical**: Off-by-one errors are the #1 failure mode when writing to spreadsheets. Before every write: 1. Read the target row to verify the adjacent cells match expectations 2. If processing multiple rows, re-read periodically to catch drift 3. Never assume row N in your data maps to row N in the sheet — always verify ## Feishu Sheets Specifics - Range format: `sheetId!A1:B2` (not sheet name) - Single-cell write: `sheetId!G5:G5` (not `sheetId!G5`) - Get sheet IDs via `feishu_sheet` action `info` - Write returns `revision` number — useful for tracking changes - Cannot create new sheet tabs in existing spreadsheet via API; create a new spreadsheet for summaries ## Output Format Summary sheets should include: - **Headers**: Group dimension, detail dimension, count, total - **Subtotals**: Per group - **Grand total**: Final row - **Sorting**: By total descending within each group, groups ordered by subtotal descending

path: references/extraction-patterns.md
content: # Extraction Patterns

Common patterns for extracting structured data from web pages, organized by data type.

## Author / Reporter Bylines

### Chinese Media

| Pattern | Example | Regex |
|---|---|---|
| 记者+名字 | `记者 张三` | `记者\s*[：:]?\s*([\u4e00-\u9fff]{2,4})` |
| 文/作者 | `文｜李四` / `文/王五` | `文[｜\|/]\s*([\u4e00-\u9fff\w]+)` |
| 作者行 | `作者：赵六` / `作者丨孙七` | `作者[：:\|丨]\s*([\u4e00-\u9fff\w]+)` |
| 括号署名 | `（周八）` / `(吴九)` | `[（(]([\u4e00-\u9fff]{2,4})[）)]` |
| 采写行 | `采写：记者 郑十` | `采写[：:]\s*(?:记者\s*)?([\u4e00-\u9fff]{2,4})` |
| 来源+记者 | `来源：新华社 记者张三` | `来源[：:].*?记者\s*([\u4e00-\u9fff]{2,4})` |

### English Media

| Pattern | Example | Regex |
|---|---|---|
| By line | `By Jane Doe` | `[Bb]y\s+([A-Z][a-z]+(?:\s+[A-Z][a-z]+)+)` |
| Author field | `Author: John Smith` | `[Aa]uthor[：:]\s*(.+?)(?:\n\|$)` |
| Reporting | `Reporting by Sarah Lee` | `[Rr]eporting by\s+(.+?)(?:\n\|;)` |

## Browser Extraction Snippets

### Quick full-text grab
```javascript
document.body.innerText.substring(0, 2000)
```

### Head + tail (find bylines at article end)
```javascript
(() => {
  const t = document.body.innerText;
  return t.substring(0, 500) + '\n---END---\n' + t.substring(Math.max(0, t.length - 300));
})()
```

### Keyword search (author markers)
```javascript
(() => {
  const t = document.body.innerText;
  const markers = ['记者', '作者', '文｜', '编辑', 'By ', 'Author'];
  for (const k of markers) {
    const idx = t.indexOf(k);
    if (idx >= 0) return `${k} @ ${idx}: ...${t.substring(Math.max(0,idx-30), idx+80)}...`;
  }
  return 'No author markers found';
})()
```

## Site-Specific Notes

| Domain | Rendering | Author Location |
|---|---|---|
| thepaper.cn (澎湃) | SPA, needs browser | Below title or end of article |
| stcn.com (证券时报) | Static | End of article `（记者 XXX）` |
| yicai.com (第一财经) | SPA | End of article in parentheses |
| nbd.com.cn (每经) | Static | Below title `记者 XXX 编辑 XXX` |
| 21jingji.com (21世纪) | Static | Below title |
| lanjinger.com (蓝鲸) | SPA | Below title `记者 XXX` |
| mp.weixin.qq.com | Needs browser (captcha) | Varies by account |
| cls.cn (财联社) | Static/SPA | End of flash news `（记者 XXX）` |
| cnfic.com.cn (新华财经) | SPA, sometimes Baidu mini-program | `记者XXX` in opening line |
| zhitongcaijing.com (智通) | SPA | Below title `作者名\n日期` |
| weibo.com | Browser only | Usually no individual reporter |


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)