Skill: ramboxie/nonferrous-daily (version 1.3.7)

--- name: metal-price description: Daily non-ferrous metals briefing for AI agents. Collects real-time base metals prices (Cu/Zn/Ni/Co/Mg/Bi) from Yahoo Finance, CCMN 長江有色, SMM, and Westmetall, then delivers a six-section professional investment research report via Telegram at 14:00 CST. Zero paid APIs required. Use when you need automated metals market monitoring, LME price tracking, or professional trading briefings with macro/inventory/futures-structure/sentiment cross-analysis. --- # 有色小鑽風 · Metal Price Daily 🦞📊 > AI-driven non-ferrous metals daily briefing — six-section investment research report via Telegram. 每日 14:00 CST（上午盤收盤後）自動採集有色金屬行情，生成六板塊專業投研報告並推送到 Telegram。**零付費 API，開箱即用。** ## Features - 📊 **多源價格聚合** — Yahoo Finance (USD)、CCMN 長江有色 (CNY)、SMM 上海有色、Westmetall (LME Cash) - 📰 **新聞日期過濾** — Google News RSS（中英文，嚴格 48h 時效過濾，不顯示過期條目） - 🏦 **機構觀點提煉** — 自動識別高盛/摩根大通/花旗報告，提煉中文結論，不堆原始鏈接 - 📈 **技術面分析** — 遠期曲線（spot/+2M/+6M）、Backwardation/Contango 結構判斷、各品種支撐阻力位 - 🔮 **四維交叉推理** — 宏觀（A/H 分化）× 庫存 × 期貨結構 × 情緒，段落式分析 - 🎯 **操作參考** — 六個品種各有具體價位建議（支撐/阻力/止損），非泛泛而談 - 🚫 **零付費 API** — 全部免費數據源 ## Metals Covered（目標品種） | Metal | USD | CNY | |-------|-----|-----| | Copper (Cu) | Yahoo HG=F ✅ + COMEX 遠期 ✅ | CCMN ✅ + SMM ✅（交叉驗證）| | Zinc (Zn) | Westmetall LME Cash ✅ | CCMN ✅ + SMM ✅（交叉驗證）| | Nickel (Ni) | Westmetall LME Cash ✅ | CCMN ✅ + SMM ✅（交叉驗證）| | Cobalt (Co) | TradingEconomics ✅ | CCMN ✅ | | Bismuth (Bi) | SMM CIF USD/kg ✅ | SMM 精鉍 ✅ | | Magnesium (Mg) | — | CCMN 1#鎂 ✅ | > 鋁（Al）不在本 Skill 追蹤範圍內。 ## Report Format（六板塊） ``` 一、行情快照 — 實時價格 + 漲跌 + 數據源 二、行業指數 — XME / COPX / 申萬有色，A/H 分化信號 三、技術面 — 各品種支撐/阻力/趨勢判斷 四、基本面 — LME 庫存（Cu/Zn/Ni）+ 去庫/累庫信號 五、市場情緒 — 機構觀點（中文提煉）+ 市場情緒分析 六、四維推理 — 宏觀/庫存/結構/情緒交叉推理 + 操作參考 ``` ## Quick Start ```bash git clone https://github.com/RAMBOXIE/metal-price.git cd metal-price cp .env.example .env # 填入 TELEGRAM_BOT_TOKEN + TELEGRAM_CHAT_ID node scripts/fetch-all-data.mjs # 採集數據（~5s） node scripts/daily-report.mjs # 採集 + 生成 + 發送完整日報 ``` ## Environment Variables ```env TELEGRAM_BOT_TOKEN= # 必填：Telegram Bot Token TELEGRAM_CHAT_ID= # 必填：目標群組/頻道 ID ``` ## Key Scripts | Script | Description | |--------|-------------| | `scripts/fetch-all-data.mjs` | 主數據採集腳本，~5s 完成，輸出完整 JSON（價格/庫存/新聞/情緒/指數）| | `scripts/daily-report.mjs` | 六板塊日報（調用 fetch-all-data → 生成分析 → Telegram 發送）| | `scripts/fetch-news.mjs` | 獨立新聞抓取，含 48h 日期過濾 | | `scripts/send-telegram.mjs` | Telegram 發送工具 | ## Agent Integration (OpenClaw Cron) 在 OpenClaw 中設置每日 14:00 定時任務： ```json { "schedule": { "kind": "cron", "expr": "0 14 * * *", "tz": "Asia/Shanghai" }, "payload": { "kind": "agentTurn", "message": "Run node D:\\Projects\\metal-price\\scripts\\daily-report.mjs and confirm delivery.", "timeoutSeconds": 90 } } ``` ## Data Sources | Source | Metals | Status | |--------|--------|--------| | Yahoo Finance (HG=F) | Cu USD | ✅ Free | | CCMN 長江有色 | Cu/Zn/Ni/Co/Mg CNY | ✅ Free | | SMM hq.smm.cn/h5 | Bi/Pb/Sn CNY+USD | ✅ Free, no login | | Westmetall | Zn/Ni USD + LME庫存 | ✅ Free | | TradingEconomics | Co USD | ✅ Free (scrape) | | OmetalCN | Cu/Zn/Ni/Sn CNY (備用) | ✅ Free | | Reddit r/Commodities | 情緒 | ✅ JSON API | | Google News RSS | 中英文新聞 | ✅ Free | | LME official | 庫存 | ❌ Cloudflare封鎖（Westmetall替代）| ## Changelog - **v1.3.7** — 报告重构为“1)行业指数 2)品种数据 3)交易要点 4)关键风险”固定目录；预警改为低/中/高；强化短中期趋势判断（趋势阶段/方向/置信度/动量状态）；行业指数新增中文名称与周环比（较约5个交易日前）；发送改为纯文本避免 Telegram Markdown 实体报错。 - **v1.3.6** — 日报升级为“结论→论据→执行→失效条件”逐标的结构（Cu/Zn/Ni/Co/Bi/Mg）；新增“证据总览表”；新增 K 线趋势判断（Cu: HG=F 5/10/20MA + 斜率，Al 预留接口）；新增可复盘记忆系统 `memory/signal-history.jsonl`（按日记录 date/cny/usd/lmeInv/cnyChange/alertLevel/trendTag/keyEvidence）；对无连续K线品种改用近N日现货序列趋势；缺失数据强制输出“缺失与替代依据”。 - **v1.3.5** — 日報框架升級：新增「全標的加速預警（Cu/Zn/Ni/Co/Bi/Mg）」模塊（0-3 級+觸發因子+可交易建議）；第六部分新增「關鍵數值依據」行（庫存/基差/進口盈虧/需求均值）；第七部分保持「結論→證據→影響」；操作參考為六品種新增狀態標籤（趨勢/加速/震盪/觀望）+一句執行建議；全報告維持 CNY/噸主展示、USD 僅括號輔助。 - **v1.3.4** — 全報告價格主展示統一為 CNY/噸（含技術面/結構維度/操作參考）；USD 僅保留為括號輔助資訊，避免非人民幣主展示。 - **v1.3.3** → 第七部分重寫為「結論→證據→影響」簡報研報格式（去標題黨/去省略號）；CCMN 故障時 Cu/Zn/Ni 自動切 SMM 備援；USD/CNY 新增多源備援（Yahoo + exchangerate.host + frankfurter）；外盤主顯示統一換算為 CNY/噸（保留原始 USD 於括號） - **v1.3.2** — 第七部分改為「今日新增」模式：機構觀點按與昨日差異輸出（附日期），情緒模塊無新增時明示「第七部分無新增」，避免連續重複文案；新增報告狀態快照（memory/daily-report-state.json） - **v1.3.1** — 机构觀點/市場情緒全中文提煉，去除英文原文；48h 時效過濾，無匹配直接跳過；無新增時明示，防止幻覺；報告前自檢（缺字段/無新訊提示），DRY_RUN 可安全預覽 - **v1.3.0** — 庫存三件套（交易所/保稅/社會佔位）、進口盈虧/到岸成本（Cu/Zn/Ni）、信號摘要四維打分（庫存/基差/進口/需求）、宏觀風險溫度計（DXY/VIX/CRB佔位/10Y）、DRY_RUN 安全開關 - **v1.2.0** — 新增鎂（Mg）品種；移除鋁（Al）；六板塊投研格式重寫；新聞 48h 時效過濾；操作建議含具體價位 - **v1.1.2** — OmetalCN 備用源；Westmetall 超時重試 - **v1.1.1** — SMM 交叉驗證（Cu/Zn/Ni）；Reddit 異動偵測 - **v1.1.0** — LME 庫存（Westmetall）；期貨遠期曲線；行業指數（XME/COPX/申萬） ## 声明（ClawHub 发布要求） - 本 Skill 仅提供公开市场数据整理与研究框架，不构成投资建议或收益承诺。 - 数据来自第三方公开源（Yahoo/CCMN/SMM/Westmetall/Reddit/Google News），可能存在延迟、修订或缺失，使用前请二次核验。 - 报告中的趋势/预警为模型化信号，不应替代风控、仓位管理与独立判断。 - 本 Skill 不包含或分发任何受保护的私有数据、付费接口凭据、破解逻辑或侵权内容。 ## License MIT · [GitHub](https://github.com/RAMBOXIE/metal-price)

path: package.json
content: {
  "name": "metal-price",
  "version": "1.3.7",
  "type": "module",
  "scripts": {
    "start": "node scripts/daily-report.mjs",
    "backfill": "node scripts/backfill-history.mjs",
    "prices": "node scripts/fetch-prices.mjs",
    "news": "node scripts/fetch-news.mjs"
  }
}


path: README.md
content: # metal-price 🦞

> AI-powered non-ferrous metals daily briefing — data collection + professional analyst report via Telegram.

A lightweight Node.js system that collects real-time base metals prices from multiple free sources, aggregates market news and forum sentiment, then delivers a professional trading-style analysis to Telegram every day at 14:00 CST (after China's morning session closes).

## Features

- 📊 **Multi-source price aggregation** — Yahoo Finance (USD), CCMN 長江有色 (CNY), SMM/Westmetall cross-checks
- 📰 **News & sentiment** — Google News (CN+EN) with 48h filter, SMM 快訊, Reddit r/Commodities 異動偵測
- 🏦 **Investment bank signals** — 自動抽取高盛/摩根大通/花旗的基本金屬觀點
- 📈 **Technical** — forward curve (spot/+2M/+6M), basis, contango/backwardation detection
- 📦 **庫存三件套** — 交易所 / 保稅 / 社會庫存（佔位兜底），周環比箭頭
- 🚢 **進口盈虧/到岸成本** — Cu/Zn/Ni 匯率+外盤→內盤，盈虧/壓力標註
- 📊 **信號摘要** — 庫存 / 基差 / 進口盈虧 / 需求 四維 +/0/- 打分
- 🌡️ **宏觀溫度計** — DXY / VIX / CRB（佔位）/ 10Y，提示風險開關
- 🔮 **Cross reasoning** — 宏觀 × 庫存 × 結構 × 情緒，段落式分析 + 操作參考
- ⏰ **14:00 CST timing** — after China morning session + LME overnight data
- 🚫 **Zero paid APIs** — all free data sources; no API key required

## Metals Covered

| Metal | USD Source | CNY Source |
|-------|------------|------------|
| Copper (Cu) | Yahoo `HG=F` + COMEX forwards | CCMN 長江有色 + SMM 交叉驗證 |
| Zinc (Zn)   | Westmetall LME Cash | CCMN 長江有色 + SMM 交叉驗證 |
| Nickel (Ni) | Westmetall LME Cash | CCMN 長江有色 + SMM 交叉驗證 |
| Cobalt (Co) | TradingEconomics (USD) | CCMN 長江有色 |
| Bismuth (Bi)| SMM CIF USD/kg | SMM 精鉍 |
| Magnesium (Mg) | — | CCMN 1#鎂 |

## Architecture

```
fetch-all-data.mjs          ← Master data script (≤3s, runs in parallel)
  ├── Yahoo Finance           USD spot + forward contracts
  ├── CCMN 長江有色 API        CNY spot prices (Cu/Zn/Ni/Co + 30 others)
  ├── Stooq                   Bismuth USD/t
  ├── LME inventory           3 methods, all Cloudflare-blocked (returns null)
  ├── Google News RSS (CN)    Chinese metals news
  ├── Google News RSS (EN)    Investment bank base metals analysis
  ├── SMM 上海有色網           Flash news headlines
  └── Reddit r/Commodities    Top (weekly) + Hot (realtime) with metal keyword filter
                              + surging post detection (hot but not in top)

send-telegram.mjs            ← Utility: pipe JSON or arg to Telegram Markdown message
```

The actual analysis/briefing is written by an AI agent (Claude) that runs `fetch-all-data.mjs`, reads the JSON, fetches 2 news articles, and composes a professional trading brief.

## Quick Start

### 1. Clone & install

```bash
git clone https://github.com/RAMBOXIE/metal-price.git
cd metal-price
# No npm install needed — zero external dependencies
```

### 2. Configure environment

```bash
cp .env.example .env
# Edit .env with your Telegram bot token and chat ID
```

### 3. Run data collection

```bash
node scripts/fetch-all-data.mjs
```

Output is a JSON object with prices, forwards, inventory, news, ibNews, and forumSentiment.

### 4. Send a message

```bash
# Send a string
node scripts/send-telegram.mjs "Hello from metal-price 🦞"

# Pipe JSON summary
node scripts/fetch-all-data.mjs | node scripts/send-telegram.mjs
```

## Environment Variables

| Variable | Description | Required |
|----------|-------------|----------|
| `TELEGRAM_BOT_TOKEN` | Your Telegram bot token (from @BotFather) | ✅ |
| `TELEGRAM_CHAT_ID` | Target chat/channel ID | ✅ |
| `METAL_PRICE_API_KEY` | metalpriceapi.com key (free tier = precious metals only, not used) | ❌ |
| `ALPHA_VANTAGE_KEY` | Alpha Vantage key (monthly copper only, not used) | ❌ |

## Output JSON Structure

```json
{
  "date": "2026-03-15",
  "dataDate": "2026-03-13",
  "isMarketOpen": false,
  "marketNote": "休市：數據截至 2026/03/13（上個交易日）",
  "changeNote": "所有漲跌均為日環比（vs 前一交易日收盤）",
  "prices": {
    "copper":  { "usd": 5.757, "usdChangePct": 0.75, "usdUnit": "USD/lb", "cny": 100690, "cnyChange": -330 },
    "zinc":    { "usd": null,  "usdChangePct": null,  "usdUnit": "USD/t",  "cny": 24130,  "cnyChange": -220 },
    "aluminum":{ "usd": 3423,  "usdChangePct": 1.18,  "usdUnit": "USD/t",  "cny": null,   "cnyChange": null },
    "nickel":  { "usd": null,  "cny": 141750, "cnyChange": 600 },
    "cobalt":  { "usd": null,  "cny": 432000, "cnyChange": 0 },
    "bismuth": { "usd": 2272,  "usdUnit": "USD/t", "source": "Stooq/BI.F", "reliabilityNote": "..." }
  },
  "forwards": {
    "copper": {
      "spot": { "price": 5.757, "symbol": "HG=F",      "expiry": "2026-03" },
      "near": { "price": 5.757, "symbol": "HGK26.CMX", "expiry": "2026-05" },
      "far":  { "price": 5.873, "symbol": "HGU26.CMX", "expiry": "2026-09" }
    }
  },
  "inventory": { "copper": null, "note": "LME blocked by Cloudflare (403)" },
  "news": [ { "title": "...", "url": "..." } ],
  "ibNews": [ { "title": "Goldman Sachs expects copper...", "url": "..." } ],
  "forumSentiment": {
    "smmHighlights": "【SMM快訊】...",
    "redditSummary": "[31↑] Copper supply concerns...",
    "redditSurging": "[🔥] Sudden copper mine shutdown...",
    "xueqiuSummary": null
  }
}
```

## Data Sources & Status

| Source | Status | Data |
|--------|--------|------|
| Yahoo Finance (HG=F) | ✅ Free | Copper USD spot + forward contracts |
| Yahoo Finance (ALI=F) | ✅ Free | Aluminum USD spot |
| Yahoo Finance (ZNC=F) | ❌ Disabled | Stale prevClose (2019), changePct unreliable |
| CCMN 長江有色 | ✅ Free | Cu/Zn/Ni/Co/Pb/Sn + 30 metals CNY |
| Stooq (BI.F) | ⚠️ Unreliable | Bismuth USD/t, price validity unconfirmed |
| LME official | ❌ Cloudflare 403 | All 3 fetch methods blocked |
| SMM 上海有色 | ✅ Headlines free | Flash news, prices require login |
| Reddit r/Commodities | ✅ JSON API | Top + Hot + surging detection |
| Google News RSS | ✅ Free | CN metals news + EN IB analysis |
| 雪球 Xueqiu | 🔒 Login required | Not implemented |

## Known Limitations

- **LME inventory**: All fetch methods return HTTP 403 (Cloudflare). Reported as `null` in output.
- **Bismuth (Bi)**: Stooq `BI.F` price ($2,272/t) is below market average ($6,600–$13,200/t). Reliability unconfirmed — use with caution.
- **Zinc USD**: Yahoo `ZNC=F` has a stale `prevClose` from 2019, making `changePct` meaningless. Disabled; CNY via CCMN still works.
- **Nickel/Cobalt USD**: Yahoo Finance has no LME Ni/Co contracts. CNY via CCMN only.

## 声明（ClawHub 发布要求）

- 本项目仅提供公开数据的整理与研究，不构成投资建议或收益承诺。
- 所有行情与资讯来自第三方公开来源，可能存在延迟、缺失或口径差异，请在交易前二次核验。
- 趋势与预警信号用于辅助决策，不替代仓位管理、风控纪律与独立判断。
- 项目不包含付费接口密钥、侵权数据或绕过访问控制的实现。

## License

MIT


path: references/ANALYSIS_DAILY_TEMPLATE.md
content: # Daily Analysis Template (Per Metal)

## [Metal]
- 状态：趋势 / 加速 / 震荡 / 观望
- 预警等级：L0-L3
- 后验判断：偏多 xx% / 中性 xx% / 偏空 xx%

### 结论
一句话结论（可执行）。

### 证据
- 数值证据 1：...
- 数值证据 2：...
- 事件/情绪证据：...

### 执行
- 动作：追随 / 回踩 / 区间 / 观望
- 仓位建议：轻 / 中 / 重

### 失效条件
- 若出现 XXX（价位/库存/情绪反转），则结论失效，切换为 XXX。

path: references/ANALYSIS_PLAYBOOK.md
content: # Metal Daily Analysis Playbook (v1.0)

目标：从“信息收集”升级为“可前瞻、可执行、可复盘”的分析系统。

## 1) 决策框架：OODA × Bayesian × Scenario

### OODA（Observe-Orient-Decide-Act）
- **Observe**：价格、库存、基差、进口盈亏、新闻/情绪。
- **Orient**：判断当前所处市场状态（趋势/加速/震荡/反转）。
- **Decide**：给出方向 + 仓位建议 + 风险条件。
- **Act**：输出可执行策略（追/等回踩/区间/观望）。

### Bayesian（证据更新）
每个标的维护先验与后验：
- 先验：昨日偏多/中性/偏空概率
- 新证据：库存、价差、情绪、事件
- 后验：更新后的偏多/中性/偏空概率

### Scenario（情景推演）
每个标的每天给 3 档情景：
1. 基准情景（概率最高）
2. 上行情景（触发条件）
3. 下行情景（触发条件）

---

## 2) 六标的统一评分体系（0-100）

每个标的：
- 供给压力/扰动（0-25）
- 库存与结构（0-20）
- 需求与现货（0-20）
- 跨市价差与进口盈亏（0-15）
- 资金与情绪（0-20）

输出：
- 0-39：弱势/观望
- 40-59：震荡/区间
- 60-74：偏强/趋势
- 75-100：加速/高波动

---

## 3) 加速预警（0-3）统一规则

- **L0**：无加速迹象
- **L1**：单因子触发（如单日涨幅异常）
- **L2**：双因子共振（价格+库存/价差/情绪其一）
- **L3**：三因子以上共振（高风险加速段）

触发因子（示例）：
- 日内涨跌幅/涨跌额超阈值
- 去库/累库超阈值
- 近远月结构异常（Backwardation/Contango 快速切换）
- 关键词命中（squeeze, shortage, export ban, policy shock）

执行规则：
- L0-L1：正常策略
- L2：不追涨，回踩确认
- L3：禁止追高，轻仓/保护性止盈

---

## 4) 每日报告最小必填项（防“空话”）

每个标的必须包含：
1. 状态标签（趋势/加速/震荡/观望）
2. 预警等级（0-3）
3. 2-3 条证据（至少 1 条数值证据）
4. 执行动作（一句话）
5. 风险触发器（失效条件）

---

## 5) 反思闭环（持续改进）

每天收盘后记录：
- 当日判断：方向/节奏/风险
- 实际结果：是否触发预案
- 偏差来源：信号滞后/权重偏差/样本噪声
- 次日调整：阈值、权重、文案

每周一次：
- 统计预测命中率（方向、节奏、风险三维）
- 调整权重，不改核心框架

---

## 6) 原则

- 先给结论，再给证据，再给动作。
- 不输出“仅情绪描述”结论。
- 不做确定性表述，统一概率语言。
- 任何“强判断”必须配失效条件。

path: scripts/backfill-history.mjs
content: import { readFileSync, writeFileSync, existsSync, readdirSync } from 'fs';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';

const __dirname = dirname(fileURLToPath(import.meta.url));
const PROJECT_ROOT = join(__dirname, '..');
const SIGNAL_HISTORY_PATH = join(PROJECT_ROOT, 'memory', 'signal-history.jsonl');
const MEMORY_DIR = join(PROJECT_ROOT, 'memory');

const METALS = [
  { code: 'Cu', key: 'copper', unit: 'USD/lb', yahoo: 'HG=F', westmetallField: 'LME_Cu_stock', westmetallUnit: 'USD/t' },
  { code: 'Zn', key: 'zinc', unit: 'USD/t', yahoo: 'ZNC=F', westmetallField: 'LME_Zn_stock', westmetallUnit: 'USD/t' },
  { code: 'Ni', key: 'nickel', unit: 'USD/t', yahoo: null, westmetallField: 'LME_Ni_stock', westmetallUnit: 'USD/t' },
  { code: 'Co', key: 'cobalt', unit: 'USD/t', yahoo: null, westmetallField: null },
  { code: 'Bi', key: 'bismuth', unit: 'USD/t', yahoo: null, westmetallField: null },
  { code: 'Mg', key: 'magnesium', unit: 'USD/t', yahoo: null, westmetallField: null },
];

const METAL_CODE_SET = new Set(METALS.map(m => m.code));

function usdToCnyPerTon(usd, unit, fxRate) {
  if (usd == null || fxRate == null) return null;
  const usdPerTon = unit === 'USD/lb' ? usd * 2204.62 : usd;
  return usdPerTon * fxRate;
}

function parseWestmetallDate(str) {
  // e.g. 24. April 2026
  const m = str.match(/^(\d{1,2})\.\s+([A-Za-z]+)\s+(\d{4})$/);
  if (!m) return null;
  const months = {
    January: 1, February: 2, March: 3, April: 4, May: 5, June: 6,
    July: 7, August: 8, September: 9, October: 10, November: 11, December: 12,
  };
  const d = Number(m[1]);
  const mo = months[m[2]];
  const y = Number(m[3]);
  if (!mo) return null;
  return `${y}-${String(mo).padStart(2, '0')}-${String(d).padStart(2, '0')}`;
}

function parseNum(s) {
  if (s == null) return null;
  const cleaned = String(s).replace(/[,$\s]/g, '');
  const n = Number(cleaned);
  return Number.isFinite(n) ? n : null;
}

function loadJsonl(filePath) {
  if (!existsSync(filePath)) return [];
  return readFileSync(filePath, 'utf-8')
    .split('\n')
    .map(x => x.trim())
    .filter(Boolean)
    .map(line => {
      try { return JSON.parse(line); } catch { return null; }
    })
    .filter(Boolean);
}

async function fetchYahooChart(symbol, range = '6mo') {
  const url = `https://query1.finance.yahoo.com/v8/finance/chart/${encodeURIComponent(symbol)}?interval=1d&range=${range}`;
  const res = await fetch(url, {
    headers: { 'User-Agent': 'Mozilla/5.0', 'Accept': 'application/json' },
    signal: AbortSignal.timeout(15000),
  });
  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  const data = await res.json();
  const result = data?.chart?.result?.[0];
  const ts = result?.timestamp || [];
  const close = result?.indicators?.quote?.[0]?.close || [];
  const out = [];
  for (let i = 0; i < ts.length; i++) {
    if (close[i] == null) continue;
    const date = new Date(ts[i] * 1000).toISOString().slice(0, 10);
    out.push({ date, close: Number(close[i]) });
  }
  return out;
}

async function fetchWestmetallHistory(field) {
  const url = `https://www.westmetall.com/en/markdaten.php?action=table&field=${encodeURIComponent(field)}`;
  const res = await fetch(url, {
    headers: { 'User-Agent': 'Mozilla/5.0', 'Accept': 'text/html' },
    signal: AbortSignal.timeout(20000),
  });
  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  const html = await res.text();
  const rows = [...html.matchAll(/<tr[^>]*>\s*<td[^>]*>([^<]+)<\/td>\s*<td[^>]*>([^<]+)<\/td>\s*<td[^>]*>([^<]+)<\/td>/g)];
  return rows.map(m => {
    const date = parseWestmetallDate(m[1].trim());
    const lmeInv = parseNum(m[2]);
    const usd = parseNum(m[3]);
    if (!date || usd == null) return null;
    return { date, usd, lmeInv };
  }).filter(Boolean);
}

function indexFxByDate(rows) {
  const m = new Map();
  for (const r of rows) m.set(r.date, r.close);
  return m;
}

function nearestFx(date, fxMap) {
  if (fxMap.has(date)) return fxMap.get(date);
  const keys = [...fxMap.keys()].sort();
  let prev = null;
  for (const k of keys) {
    if (k > date) break;
    prev = k;
  }
  return prev ? fxMap.get(prev) : null;
}

function parseLocalMarkdownHistory() {
  if (!existsSync(MEMORY_DIR)) return [];
  const mdFiles = readdirSync(MEMORY_DIR).filter(n => /^\d{4}-\d{2}-\d{2}\.md$/.test(n));
  const out = [];
  for (const f of mdFiles) {
    const date = f.replace('.md', '');
    const text = readFileSync(join(MEMORY_DIR, f), 'utf-8');
    const lines = text.split(/\r?\n/);
    for (const line of lines) {
      // 解析诸如 Cu=311600t 或 Co 415000 这类简易线索，保守处理
      const pairs = [
        { code: 'Cu', re: /(?:Cu|铜)[^\d]{0,8}(\d{4,7})/i },
        { code: 'Zn', re: /(?:Zn|锌)[^\d]{0,8}(\d{4,7})/i },
        { code: 'Ni', re: /(?:Ni|镍)[^\d]{0,8}(\d{4,7})/i },
        { code: 'Co', re: /(?:Co|钴)[^\d]{0,8}(\d{4,7})/i },
        { code: 'Bi', re: /(?:Bi|铋)[^\d]{0,8}(\d{4,7})/i },
        { code: 'Mg', re: /(?:Mg|镁)[^\d]{0,8}(\d{4,6})/i },
      ];
      for (const p of pairs) {
        const m = line.match(p.re);
        if (!m) continue;
        const cny = Number(m[1]);
        if (!Number.isFinite(cny)) continue;
        out.push({ date, metal: p.code, cny, source: 'local-memory-md' });
      }
    }
  }
  return out;
}

async function fetchCurrentSnapshot() {
  try {
    const scriptPath = join(__dirname, 'fetch-all-data.mjs');
    const cp = await import('child_process');
    const { promisify } = await import('util');
    const execFileAsync = promisify(cp.execFile);
    const { stdout } = await execFileAsync(process.execPath, [scriptPath], { timeout: 90000, maxBuffer: 8 * 1024 * 1024 });
    const d = JSON.parse(stdout);
    const rows = [];
    for (const m of METALS) {
      const item = d?.prices?.[m.key] || {};
      if (item.cny == null && item.usd == null) continue;
      rows.push({
        date: d.date,
        metal: m.code,
        cny: item.cny ?? null,
        usd: item.usd ?? null,
        lmeInv: null,
        cnyChange: item.cnyChange ?? null,
        alertLevel: 0,
        trendTag: '震荡',
        keyEvidence: '回填快照：fetch-all-data',
        source: 'fetch-all-data',
      });
    }
    return rows;
  } catch {
    return [];
  }
}

function dedupeAndSort(records) {
  const map = new Map();
  for (const r of records) {
    if (!r?.date || !r?.metal || !METAL_CODE_SET.has(r.metal)) continue;
    const k = `${r.date}|${r.metal}`;
    const prev = map.get(k);
    if (!prev) map.set(k, r);
    else {
      const prevScore = (prev.cny != null ? 2 : 0) + (prev.usd != null ? 1 : 0);
      const currScore = (r.cny != null ? 2 : 0) + (r.usd != null ? 1 : 0);
      if (currScore >= prevScore) map.set(k, { ...prev, ...r });
    }
  }
  return [...map.values()].sort((a, b) => a.date === b.date ? a.metal.localeCompare(b.metal) : a.date.localeCompare(b.date));
}

async function main() {
  const existing = loadJsonl(SIGNAL_HISTORY_PATH).filter(r => METAL_CODE_SET.has(r.metal));
  const localMd = parseLocalMarkdownHistory();

  let fxMap = new Map();
  try {
    const fxRows = await fetchYahooChart('USDCNY=X', '6mo');
    fxMap = indexFxByDate(fxRows);
  } catch {}

  const fetched = [];

  // Source A: Yahoo chart
  for (const m of METALS.filter(x => x.yahoo)) {
    try {
      const rows = await fetchYahooChart(m.yahoo, '6mo');
      for (const r of rows) {
        const fx = nearestFx(r.date, fxMap);
        fetched.push({
          date: r.date,
          metal: m.code,
          usd: r.close,
          cny: usdToCnyPerTon(r.close, m.unit, fx),
          lmeInv: null,
          cnyChange: null,
          alertLevel: 0,
          trendTag: '震荡',
          keyEvidence: `backfill: Yahoo ${m.yahoo}`,
          source: `yahoo:${m.yahoo}`,
        });
      }
    } catch {}
  }

  // Source A/B supplement: Westmetall history (Cu/Zn/Ni)
  for (const m of METALS.filter(x => x.westmetallField)) {
    try {
      const rows = await fetchWestmetallHistory(m.westmetallField);
      for (const r of rows) {
        const fx = nearestFx(r.date, fxMap);
        fetched.push({
          date: r.date,
          metal: m.code,
          usd: r.usd,
          cny: usdToCnyPerTon(r.usd, m.westmetallUnit || m.unit, fx),
          lmeInv: r.lmeInv ?? null,
          cnyChange: null,
          alertLevel: 0,
          trendTag: '震荡',
          keyEvidence: `backfill: Westmetall ${m.westmetallField}`,
          source: `westmetall:${m.westmetallField}`,
        });
      }
    } catch {}
  }

  // Source B: local structured history + markdown notes
  const currentSnapshot = await fetchCurrentSnapshot();

  const merged = dedupeAndSort([...existing, ...localMd, ...fetched, ...currentSnapshot]);
  const beforeKeys = new Set(existing.map(r => `${r.date}|${r.metal}`));
  const added = merged.filter(r => !beforeKeys.has(`${r.date}|${r.metal}`));

  writeFileSync(SIGNAL_HISTORY_PATH, merged.map(r => JSON.stringify(r)).join('\n') + '\n', 'utf-8');

  const stats = {};
  for (const m of METALS) {
    const allCount = merged.filter(r => r.metal === m.code).length;
    const addCount = added.filter(r => r.metal === m.code).length;
    stats[m.code] = { total: allCount, added: addCount };
  }

  console.log(JSON.stringify({
    ok: true,
    output: SIGNAL_HISTORY_PATH,
    added: added.length,
    total: merged.length,
    stats,
  }, null, 2));
}

main().catch(err => {
  console.error(err);
  process.exit(1);
});


path: scripts/daily-report.mjs
content: import { readFileSync, existsSync, mkdirSync, writeFileSync, appendFileSync } from 'fs';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';
import { execFile } from 'child_process';
import { promisify } from 'util';

const __dirname = dirname(fileURLToPath(import.meta.url));
const PROJECT_ROOT = join(__dirname, '..');
const REPORT_CACHE_PATH = join(PROJECT_ROOT, 'memory', 'daily-report-state.json');
const SIGNAL_HISTORY_PATH = join(PROJECT_ROOT, 'memory', 'signal-history.jsonl');
const execFileAsync = promisify(execFile);

function loadEnv() {
  const envPath = join(PROJECT_ROOT, '.env');
  const env = {};
  try {
    const content = readFileSync(envPath, 'utf-8');
    for (const line of content.split('\n')) {
      const trimmed = line.trim();
      if (!trimmed || trimmed.startsWith('#')) continue;
      const eqIdx = trimmed.indexOf('=');
      if (eqIdx === -1) continue;
      const key = trimmed.slice(0, eqIdx).trim();
      const value = trimmed.slice(eqIdx + 1).trim().replace(/^["']|["']$/g, '');
      env[key] = value;
    }
  } catch {}
  return env;
}

async function runScript(scriptName) {
  const scriptPath = join(__dirname, scriptName);
  const { stdout, stderr } = await execFileAsync(process.execPath, [scriptPath], {
    timeout: 70000,
    maxBuffer: 8 * 1024 * 1024,
  });
  if (stderr) process.stderr.write(stderr);
  return JSON.parse(stdout);
}

function fmtNum(n, decimals = 0) {
  if (n == null || Number.isNaN(Number(n))) return '—';
  const parts = Number(n).toFixed(decimals).split('.');
  parts[0] = parts[0].replace(/\B(?=(\d{3})+(?!\d))/g, ',');
  return decimals > 0 ? parts.join('.') : parts[0];
}

function fmtPct(n, decimals = 2) {
  if (n == null || Number.isNaN(Number(n))) return '—';
  return `${n >= 0 ? '+' : ''}${Number(n).toFixed(decimals)}%`;
}

function usdToCnyPerTon(usd, unit, fxRate) {
  if (usd == null || fxRate == null) return null;
  const usdPerTon = unit === 'USD/lb' ? usd * 2204.62 : usd;
  return usdPerTon * fxRate;
}

function loadSignalHistory() {
  if (!existsSync(SIGNAL_HISTORY_PATH)) return [];
  const raw = readFileSync(SIGNAL_HISTORY_PATH, 'utf-8').split('\n').filter(Boolean);
  const rows = [];
  for (const l of raw) {
    try { rows.push(JSON.parse(l)); } catch {}
  }
  return rows;
}

function appendSignalHistory(records) {
  const dir = join(PROJECT_ROOT, 'memory');
  if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
  const existing = loadSignalHistory();
  const existingKey = new Set(existing.map(r => `${r.date}|${r.metal}`));
  const toAppend = records.filter(r => !existingKey.has(`${r.date}|${r.metal}`));
  if (!toAppend.length) return 0;
  appendFileSync(SIGNAL_HISTORY_PATH, toAppend.map(r => JSON.stringify(r)).join('\n') + '\n', 'utf-8');
  return toAppend.length;
}

function recentSeries(history, metal, n = 20) {
  return history.filter(r => r.metal === metal && r.cny != null).slice(-n);
}

function toArrayMaybe(v) {
  return Array.isArray(v) ? v : [];
}

function normalizeSymbol(symbol) {
  return String(symbol || '').toUpperCase().replace(/\s+/g, '');
}

function pickIndustryRaw(data = {}) {
  return [
    ...toArrayMaybe(data.industryIndices),
    ...toArrayMaybe(data.indices),
    ...toArrayMaybe(data.metalIndices),
    ...toArrayMaybe(data.industryIndex),
  ];
}

function ensureIndustryIndices(data = {}) {
  const industryRaw = pickIndustryRaw(data);
  const targets = [
    { symbol: 'XME', name: 'SPDR S&P Metals & Mining ETF', cnName: '标普金属与采矿ETF', aliases: ['XME'] },
    { symbol: 'COPX', name: 'Global X Copper Miners ETF', cnName: '全球X铜矿ETF', aliases: ['COPX'] },
    { symbol: '000812.SS', name: '上证有色金属指数', cnName: '上证有色金属指数', aliases: ['000812.SS', '000812.SH', '000812', 'SH000812'] },
  ];

  return targets.map(t => {
    const hit = industryRaw.find(x => {
      const s = normalizeSymbol(x?.symbol ?? x?.code ?? x?.ticker ?? x?.name);
      return t.aliases.some(a => normalizeSymbol(a) === s);
    });

    const price = hit?.price ?? hit?.last ?? hit?.value ?? hit?.close ?? null;
    const changePct = hit?.changePct ?? hit?.pct ?? hit?.change_percent ?? hit?.percent ?? null;
    const wowPct = hit?.wowPct ?? hit?.weekOverWeekPct ?? hit?.weeklyChangePct ?? null;

    if (!hit || price == null) {
      return {
        symbol: t.symbol,
        name: t.name,
        cnName: t.cnName,
        missing: true,
        reason: `缺失：${t.symbol} 暂无可用行情；替代：以六品种趋势一致性与预警共振做判断。`,
      };
    }

    return {
      symbol: t.symbol,
      name: t.name,
      cnName: t.cnName,
      missing: false,
      price,
      changePct,
      wowPct,
    };
  });
}

function keywordEvidence(text, keywords) {
  const t = (text || '').toLowerCase();
  const hits = keywords.filter(k => t.includes(k.toLowerCase()));
  return { count: hits.length, hits: hits.slice(0, 4) };
}

function hardAlertLevel({ cny, cnyChange, invChange, sentimentCount, hasInventory }) {
  const prev = (cny != null && cnyChange != null) ? (cny - cnyChange) : null;
  const volPct = (prev && prev !== 0) ? Math.abs((cnyChange / prev) * 100) : 0;

  const volScore = volPct >= 2.0 ? 2 : volPct >= 1.0 ? 1 : 0;
  const invScore = hasInventory
    ? (Math.abs(invChange ?? 0) >= 8000 ? 2 : Math.abs(invChange ?? 0) >= 3000 ? 1 : 0)
    : 0;
  const sentScore = sentimentCount >= 2 ? 1 : 0;

  return Math.max(0, Math.min(3, volScore + (invScore > 0 ? 1 : 0) + sentScore));
}

function alertLevelLabel(level) {
  if (level >= 3) return '高';
  if (level >= 2) return '中';
  return '低';
}

function calcSlopePct(series, window = 3) {
  if (!series?.length || series.length < window) return null;
  const s = series.slice(-window);
  const first = s[0]?.cny;
  const last = s[s.length - 1]?.cny;
  if (first == null || last == null || first === 0) return null;
  return ((last - first) / first) * 100;
}

function spreadStatus(cny, extCny) {
  if (cny == null || extCny == null) return { label: '缺失', basis: '内外盘价差缺失' };
  const diff = cny - extCny;
  const pct = extCny !== 0 ? (diff / extCny) * 100 : 0;
  if (pct >= 0.8) return { label: '内强', basis: `内外价差${fmtNum(diff, 0)}/吨（${fmtPct(pct, 2)}）` };
  if (pct <= -0.8) return { label: '外强', basis: `内外价差${fmtNum(diff, 0)}/吨（${fmtPct(pct, 2)}）` };
  return { label: '均衡', basis: `内外价差${fmtNum(diff, 0)}/吨（${fmtPct(pct, 2)}）` };
}

function inventoryDirection(invChange, hasInventory) {
  if (!hasInventory) return '未知';
  if (invChange == null) return '缺失';
  if (invChange > 0) return '累库';
  if (invChange < 0) return '去库';
  return '平稳';
}

function trendDirection(slope5) {
  if (slope5 == null) return '震荡';
  if (slope5 >= 0.8) return '上行';
  if (slope5 <= -0.8) return '下行';
  return '震荡';
}

function trendMomentum({ slope3, slope5, invDir, direction }) {
  if (slope3 == null || slope5 == null) {
    return { status: '不明', basis: '样本不足，无法比较近3日与近5日斜率' };
  }

  const sameSign = Math.sign(slope3) === Math.sign(slope5) || slope3 === 0 || slope5 === 0;
  const faster = Math.abs(slope3) >= Math.abs(slope5) * 1.15;
  const slower = Math.abs(slope3) <= Math.abs(slope5) * 0.75;
  const invAligned = direction === '上行'
    ? invDir === '去库'
    : direction === '下行'
      ? invDir === '累库'
      : false;

  if (!sameSign) {
    return { status: '衰减', basis: `近3日斜率${fmtPct(slope3)}与近5日斜率${fmtPct(slope5)}方向分歧` };
  }
  if (faster && (invAligned || invDir === '未知' || invDir === '缺失')) {
    return { status: '增强', basis: `近3日斜率${fmtPct(slope3)}高于近5日${fmtPct(slope5)}${invAligned ? '且库存配合' : ''}` };
  }
  if (slower) {
    return { status: '衰减', basis: `近3日斜率${fmtPct(slope3)}低于近5日${fmtPct(slope5)}，趋势斜率放缓` };
  }
  return { status: '维持', basis: `近3日斜率${fmtPct(slope3)}与近5日${fmtPct(slope5)}同向，节奏稳定` };
}

function trendStage({ direction, momentum, slope3, slope5 }) {
  if (slope3 != null && slope5 != null && Math.sign(slope3) !== Math.sign(slope5) && slope3 !== 0 && slope5 !== 0) {
    return '转折观察';
  }
  if (direction === '震荡') return '转折观察';
  if (momentum === '增强') return '启动期';
  if (momentum === '维持') return '延续期';
  if (momentum === '衰减') return '钝化期';
  return '转折观察';
}

function trendConfidence({ direction, slope3, slope5, invDir, spread, sentimentCount }) {
  const checks = [];
  if (slope3 != null && slope5 != null) checks.push(Math.sign(slope3) === Math.sign(slope5));

  if (direction === '上行') {
    if (invDir !== '未知' && invDir !== '缺失') checks.push(invDir === '去库' || invDir === '平稳');
    if (spread !== '缺失') checks.push(spread !== '外强');
  } else if (direction === '下行') {
    if (invDir !== '未知' && invDir !== '缺失') checks.push(invDir === '累库' || invDir === '平稳');
    if (spread !== '缺失') checks.push(spread !== '内强');
  } else {
    checks.push(true);
  }

  checks.push(sentimentCount >= 2);

  const agree = checks.filter(Boolean).length;
  if (agree >= 4) return '高';
  if (agree >= 2) return '中';
  return '低';
}

function oneLineOverview(items) {
  const up = items.filter(x => x.trendDirection === '上行').length;
  const down = items.filter(x => x.trendDirection === '下行').length;
  const highAlert = items.filter(x => x.alertLevel >= 2).map(x => x.code);
  if (up > down) return `主线：有色处于偏多趋势，优先跟踪 ${highAlert.length ? highAlert.join('/') : '核心品种'} 的回踩建仓机会。`;
  if (down > up) return `主线：有色偏弱，维持防守仓位，优先处理 ${highAlert.length ? highAlert.join('/') : '高波动品种'} 的回撤风险。`;
  return `主线：板块震荡分化，仓位以均衡配置为主，盯住 ${highAlert.length ? highAlert.join('/') : '关键品种'} 趋势确认信号。`;
}

function buildTradePoints(items) {
  const up = items.filter(x => x.trendDirection === '上行').map(x => x.code);
  const down = items.filter(x => x.trendDirection === '下行').map(x => x.code);
  const highConf = items.filter(x => x.trendConfidence === '高').map(x => x.code);
  const momentumDecay = items.filter(x => x.momentum.status === '衰减').map(x => x.code);

  const trendLine = up.length > down.length
    ? `主趋势线（未来1-3周）：偏上行，重点跟踪 ${up.slice(0, 4).join('/')} 的延续与回踩确认。`
    : down.length > up.length
      ? `主趋势线（未来1-3周）：偏下行，重点防守 ${down.slice(0, 4).join('/')} 的反弹失败风险。`
      : '主趋势线（未来1-3周）：震荡分化，等待趋势置信度升至“高”的品种成为主线。';

  const config = up.length > down.length
    ? `配置建议：进攻/防守≈6:4，进攻端优先 ${highConf.length ? highConf.join('/') : '趋势置信度较高品种'}，防守端保留现金与低波动品种。`
    : down.length > up.length
      ? '配置建议：进攻/防守≈3:7，防守端以现金和低波动品种为主，仅对高置信度反弹做小比例试仓。'
      : '配置建议：进攻/防守≈5:5，采用均衡配置，等待趋势共振后再偏向单侧。';

  const trigger = momentumDecay.length
    ? `触发器：若 ${momentumDecay.join('/')} 持续两日“动量衰减”且斜率转负，减仓10%-20%；若转为“增强”并伴随库存配合，再恢复仓位。`
    : '触发器：当任一主线品种由“维持”切换为“增强”并保持两日，分批加仓；若转为“衰减”，先降仓再观察。';

  return { trendLine, config, trigger };
}

function buildRisks(items) {
  const reversal = items.filter(x => x.trendStage === '转折观察' || x.momentum.status === '衰减').map(x => x.code);
  return [
    `趋势反转风险：${reversal.length ? reversal.join('/') : '当前主线品种'} 出现斜率背离或动量衰减时，按失效条件先减仓后验证。`,
    '宏观/汇率风险：美元与风险偏好共振时，内外盘价差可能快速重估；USD/CNY波动放大阶段降低进攻仓位。',
    '数据失真或时滞风险：库存与资讯存在发布时滞，若价格与数据连续两日背离，降低数据权重并以价格趋势优先。',
  ];
}

function buildStrategy(x) {
  if (x.trendDirection === '上行') {
    return '策略：建仓区=近5日均线下方0%-1.5%回踩区；加仓条件=动量“增强”且库存去库/平稳，减仓条件=动量转“衰减”，失效条件=收盘跌破近5日低点。';
  }
  if (x.trendDirection === '下行') {
    return '策略：建仓区=仅保留防守底仓并等待企稳；加仓条件=由“衰减”转“维持”并连续两日，减仓条件=反弹至近5日高位受阻，失效条件=趋势方向转“上行”且置信度升至中高。';
  }
  return '策略：建仓区=区间下沿分批试仓；加仓条件=趋势方向明确并置信度升至“中”以上，减仓条件=区间中上沿动量衰减，失效条件=区间破位并延续两日。';
}

function buildReport(d, histRows) {
  const fx = d.fxRates?.usdCny?.price ?? null;
  const p = d.prices || {};
  const inv = d.inventory || {};
  const forum = d.forumSentiment || {};
  const newsText = [
    ...(d.news || []).map(x => x.title || ''),
    ...(d.ibNews || []).map(x => x.title || ''),
    forum.smmHighlights || '',
    forum.redditSurging || '',
    forum.redditSummary || '',
  ].join(' | ');

  const defs = [
    { key: 'copper', code: 'Cu', name: '铜', unit: 'USD/lb', invKey: 'copper', kws: ['copper', '铜', 'comex', 'tc'] },
    { key: 'zinc', code: 'Zn', name: '锌', unit: 'USD/t', invKey: 'zinc', kws: ['zinc', '锌', 'galvanized', '镀锌'] },
    { key: 'nickel', code: 'Ni', name: '镍', unit: 'USD/t', invKey: 'nickel', kws: ['nickel', '镍', 'npi', '不锈钢'] },
    { key: 'cobalt', code: 'Co', name: '钴', unit: 'USD/t', invKey: null, kws: ['cobalt', '钴', 'battery', '新能源'] },
    { key: 'bismuth', code: 'Bi', name: '铋', unit: 'USD/t', invKey: null, kws: ['bismuth', '铋', '半导体', '医药'] },
    { key: 'magnesium', code: 'Mg', name: '镁', unit: 'USD/t', invKey: null, kws: ['magnesium', '镁', '轻量化', '煤'] },
  ];

  const items = [];
  const records = [];

  for (const def of defs) {
    const item = p[def.key] || {};
    const invRow = def.invKey ? inv?.[def.invKey] : null;
    const kw = keywordEvidence(newsText, def.kws);

    const extCny = item.usd != null && fx != null ? usdToCnyPerTon(item.usd, def.unit, fx) : null;
    const spread = spreadStatus(item.cny ?? null, extCny);
    const invDir = inventoryDirection(invRow?.change ?? null, Boolean(def.invKey));

    const hist = recentSeries(histRows, def.code, 20);
    const series = [...hist];
    if (item.cny != null) {
      const lastDate = series[series.length - 1]?.date;
      if (lastDate !== d.date) series.push({ date: d.date, cny: item.cny });
    }

    const slope3 = calcSlopePct(series, 3);
    const slope5 = calcSlopePct(series, 5);
    const direction = trendDirection(slope5);
    const momentum = trendMomentum({ slope3, slope5, invDir, direction });
    const stage = trendStage({ direction, momentum: momentum.status, slope3, slope5 });
    const confidence = trendConfidence({
      direction,
      slope3,
      slope5,
      invDir,
      spread: spread.label,
      sentimentCount: kw.count,
    });

    const alertLevel = hardAlertLevel({
      cny: item.cny ?? null,
      cnyChange: item.cnyChange ?? 0,
      invChange: invRow?.change ?? null,
      sentimentCount: kw.count,
      hasInventory: Boolean(def.invKey),
    });

    const inventoryText = def.invKey
      ? (invRow?.tonnes != null
          ? `${fmtNum(invRow.tonnes)}t（${(invRow.change ?? 0) >= 0 ? '+' : ''}${fmtNum(invRow.change ?? 0)}t，${invDir}）`
          : '缺失与替代依据：库存缺失，改用近3日/5日斜率与价差判断')
      : '缺失与替代依据：该品种无稳定库存口径，改用斜率+价差+情绪辅助';

    items.push({
      code: def.code,
      name: def.name,
      trendStage: stage,
      trendDirection: direction,
      trendConfidence: confidence,
      observeCycle: '短中期（5-20交易日）',
      slope3,
      slope5,
      momentum,
      spread,
      invDir,
      inventoryText,
      sentimentHits: kw.hits,
      sentimentCount: kw.count,
      cny: item.cny ?? null,
      usd: item.usd ?? null,
      cnyChange: item.cnyChange ?? null,
      alertLevel,
    });

    records.push({
      date: d.date,
      metal: def.code,
      cny: item.cny ?? null,
      usd: item.usd ?? null,
      lmeInv: invRow?.tonnes ?? null,
      cnyChange: item.cnyChange ?? null,
      alertLevel,
      trendTag: direction,
      trendStage: stage,
      trendConfidence: confidence,
      keyEvidence: `slope3=${slope3 ?? 'NA'} slope5=${slope5 ?? 'NA'} inv=${invDir} spread=${spread.label} sentiment=${kw.count}`,
    });
  }

  const industry = ensureIndustryIndices(d);
  const overview = oneLineOverview(items);
  const points = buildTradePoints(items);
  const risks = buildRisks(items);

  const lines = [];
  lines.push(`有色金属趋势研报 | ${d.date}`);
  lines.push('');
  lines.push(`一句话总览：${overview}`);
  lines.push('');

  lines.push('1) 行业指数');
  for (const idx of industry) {
    const label = `${idx.symbol}（${idx.cnName || idx.name || idx.symbol}）`;
    if (idx.missing) {
      lines.push(`- ${label}：${idx.reason}`);
    } else {
      const wowText = idx.wowPct == null
        ? '周环比缺失'
        : `周环比 ${(idx.wowPct >= 0 ? '+' : '') + fmtNum(idx.wowPct, 2)}%`;
      lines.push(`- ${label}：${fmtNum(idx.price, 3)}（日变 ${(idx.changePct ?? 0) >= 0 ? '+' : ''}${fmtNum(idx.changePct ?? 0, 2)}%｜${wowText}）`);
    }
  }
  lines.push('');

  lines.push('2) 品种数据');
  for (const x of items) {
    const extText = (x.sentimentHits?.length ?? 0) > 0
      ? `${x.sentimentCount}（${x.sentimentHits.join('、')}）`
      : `${x.sentimentCount}`;

    lines.push(`*${x.name}（${x.code}）*`);
    lines.push(`趋势阶段：${x.trendStage}｜趋势方向：${x.trendDirection}｜趋势置信度：${x.trendConfidence}｜观察周期：${x.observeCycle}｜预警：${alertLevelLabel(x.alertLevel)}`);
    lines.push(`依据：近3日斜率${fmtPct(x.slope3)}｜近5日斜率${fmtPct(x.slope5)}｜库存${x.inventoryText}｜内外盘价差${x.spread.basis}｜情绪命中${extText}（辅助）`);
    lines.push(`趋势动量状态：${x.momentum.status}（${x.momentum.basis}）`);
    lines.push(buildStrategy(x));
    lines.push('');
  }

  lines.push('3) 交易要点（技术面/信号摘要）');
  lines.push(`- ${points.trendLine}`);
  lines.push(`- ${points.config}`);
  lines.push(`- ${points.trigger}`);
  lines.push('');

  lines.push('4) 关键风险');
  for (const r of risks) lines.push(`- ${r}`);
  lines.push('');

  lines.push(`数据源与时间戳：Yahoo/CCMN/SMM/Westmetall/GoogleNews/Reddit；生成时间 ${new Date().toISOString()}`);

  return { message: lines.join('\n'), records };
}

async function sendTelegram(token, chatId, text) {
  const url = `https://api.telegram.org/bot${token}/sendMessage`;
  const res = await fetch(url, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json; charset=utf-8' },
    body: JSON.stringify({ chat_id: chatId, text }),
    signal: AbortSignal.timeout(15000),
  });
  const data = await res.json();
  if (!data.ok) throw new Error(`Telegram API error: ${JSON.stringify(data)}`);
  return data;
}

function saveReportState(state) {
  try {
    const dir = join(PROJECT_ROOT, 'memory');
    if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
    writeFileSync(REPORT_CACHE_PATH, JSON.stringify(state, null, 2), 'utf-8');
  } catch {}
}

async function main() {
  const data = await runScript('fetch-all-data.mjs');
  const history = loadSignalHistory();
  const { message, records } = buildReport(data, history);

  console.log('\n─── 报告预览 ───');
  console.log(message);
  console.log('────────────────\n');

  const env = loadEnv();
  const [REDACTED_SECRET];
  const chatId = env.TELEGRAM_CHAT_ID;
  const dryRun = process.env.DRY_RUN === '1' || process.env.DRY_RUN === 'true';

  if (!dryRun) {
    appendSignalHistory(records);
    saveReportState({ generatedAt: new Date().toISOString(), date: data.date, recordsCount: records.length });
  }

  if (!token) {
    process.stderr.write('[daily-report] 未配置 TELEGRAM_BOT_TOKEN，跳过发送\n');
    return;
  }
  if (dryRun) {
    process.stderr.write('[daily-report] DRY_RUN=1，仅预览，不发送\n');
    return;
  }

  try {
    await sendTelegram(token, chatId, message);
    process.stderr.write('[daily-report] ✅ 发送成功\n');
  } catch (err) {
    process.stderr.write(`[daily-report] ❌ 发送失败：${err.message}\n`);
    process.stderr.write('[daily-report] 以下为可复制全文：\n');
    process.stderr.write(message + '\n');
    throw err;
  }
}

main().catch(err => {
  console.error('Fatal error:', err.message);
  process.exit(1);
});


path: scripts/fetch-all-data.mjs
content: /**
 * fetch-all-data.mjs
 * 收集所有有色金屬原始數據，輸出完整 JSON 到 stdout
 * 目標：≤15 秒完成
 *
 * 輸出結構：
 * { date, dataDate, isMarketOpen, marketNote, changeNote, bismuthNote, prices, forwards, inventory, news, ibNews, forumSentiment }
 *
 * v4 新增：
 * - fetchSmmNews(): SMM上海有色網新聞（免費，HTTP可達）
 * - fetchRedditCommodities(): Reddit r/Commodities 最新帖子
 * - forumSentiment: { redditSummary, smmHighlights } 市場情緒字段
 */

// ────────────────────────────────────────────
// 工具函數
// ────────────────────────────────────────────

function today() {
  return new Date().toLocaleDateString('sv-SE', { timeZone: 'Asia/Shanghai' });
}

// ────────────────────────────────────────────
// 1. CCMN 長江有色現貨（CNY）
// ────────────────────────────────────────────

async function fetchCcmnPrices() {
  const url = 'https://m.ccmn.cn/mhangqing/getCorpStmarketPriceList?marketVmid=40288092327140f601327141c0560001';
  try {
    const res = await fetch(url, {
      headers: {
        'Referer': 'https://m.ccmn.cn/mhangqing/mcjxh/',
        'X-Requested-With': 'XMLHttpRequest',
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
      },
      signal: AbortSignal.timeout(12000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const data = await res.json();
    if (!data.success) throw new Error(data.msg || 'API error');

    const list = data.body?.priceList;
    if (!Array.isArray(list)) throw new Error('No priceList');

    const nameMap = {
      '1#铜': 'copper',
      '0#锌': 'zinc',
      '1#镍': 'nickel',
      '1#钴': 'cobalt',
      'A00铝': 'aluminum',  // v8 新增：嘗試從 CCMN 獲取鋁價
      '1#铝': 'aluminum',   // 備用牌號
      '1#镁': 'magnesium',  // v12 新增：鎂
    };

    // 升級一：提取 dataDate 與 isMarketOpen
    const rawDate = list[0]?.publishDate ?? null;
    // publishDate 可能是 "2026-03-13" 或 "2026/03/13"，統一轉為 "YYYY-MM-DD"
    const dataDate = rawDate ? rawDate.replace(/\//g, '-').slice(0, 10) : null;
    const todaySH = today();
    const isMarketOpen = dataDate ? (dataDate === todaySH) : null;

    const result = { copper: null, zinc: null, nickel: null, cobalt: null, aluminum: null, magnesium: null, dataDate, isMarketOpen };
    for (const item of list) {
      const key = nameMap[item.productSortName];
      if (key) {
        const price = parseFloat(item.avgPrice);
        const updown = parseFloat(item.highsLowsAmount);
        result[key] = {
          price: isNaN(price) ? null : price,
          updown: isNaN(updown) ? null : updown,
        };
      }
    }
    return result;
  } catch (err) {
    process.stderr.write(`[fetch-all-data] CCMN 錯誤: ${err.message}\n`);
    return null;
  }
}

// ────────────────────────────────────────────
// 1b. OmetalCN 長江現貨（GBK 頁面，備用源）
// URL: http://app.ometal.cn/data/mlist.asp
// 品種：Cu / Al / Pb / Zn / Ni / Sn + 升貼水
// 優勢：有 A00鋁（CCMN 常缺）、解析穩定、響應快
// ────────────────────────────────────────────

async function fetchOmetal() {
  try {
    const res = await fetch('http://app.ometal.cn/data/mlist.asp', {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept': 'text/html',
        'Referer': 'http://app.ometal.cn/',
      },
      signal: AbortSignal.timeout(10000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const buf = await res.arrayBuffer();
    const text = new TextDecoder('gbk').decode(buf);

    // 提取純文字（去除HTML標籤）
    const plain = text.replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ');

    // 解析價格表格：格式為「品名 高-低 均價 ↑↓變動 更多」
    function parseRow(name) {
      // 匹配：品名後面跟數字區間、均價、漲跌
      const re = new RegExp(
        name.replace(/[#()]/g, c => '\\' + c) +
        '\\s+([\\d,]+-[\\d,-]+)\\s+([\\d,]+)\\s+[↑↓]([\\-\\d,]+)'
      );
      const m = plain.match(re);
      if (!m) return null;
      const range = m[1].split('-');
      const avg = parseFloat(m[2].replace(/,/g, ''));
      const change = parseFloat(m[3].replace(/,/g, ''));
      // range: 有時是「22850-22950」也有負數「-130--90」
      let low = null, high = null;
      if (range.length === 2) {
        low  = parseFloat(range[0].replace(/,/g, ''));
        high = parseFloat(range[1].replace(/,/g, ''));
      }
      if (isNaN(avg)) return null;
      return { price: avg, high: isNaN(high) ? null : high, low: isNaN(low) ? null : low, change: isNaN(change) ? null : change };
    }

    // 提取日期（格式：日期: 3/26）
    const dateMatch = plain.match(/日期[：:]\s*(\d+)\/(\d+)/);
    let dataDate = null;
    if (dateMatch) {
      const year = new Date().getFullYear();
      dataDate = `${year}-${String(parseInt(dateMatch[1])).padStart(2,'0')}-${String(parseInt(dateMatch[2])).padStart(2,'0')}`;
    }

    const result = {
      copper:          parseRow('1#铜'),
      aluminum:        parseRow('A00铝'),
      lead:            parseRow('1#铅'),
      zinc:            parseRow('0#锌'),
      nickel:          parseRow('1#镍板'),
      tin:             parseRow('1#锡'),
      copperPremium:   parseRow('铜升贴水'),
      aluminumPremium: parseRow('铝升贴水'),
      dataDate,
      source: 'OmetalCN/app.ometal.cn',
    };

    process.stderr.write(
      `[fetch-all-data] OmetalCN: Cu=¥${result.copper?.price} Al=¥${result.aluminum?.price} Zn=¥${result.zinc?.price} Ni=¥${result.nickel?.price} Sn=¥${result.tin?.price}\n`
    );
    return result;
  } catch (err) {
    process.stderr.write(`[fetch-all-data] OmetalCN 失敗: ${err.message}\n`);
    return null;
  }
}

// ────────────────────────────────────────────
// 2. Yahoo Finance v8（USD 現貨 / 遠期合約）
// ────────────────────────────────────────────

async function fetchYahoo(symbol) {
  const url = `https://query1.finance.yahoo.com/v8/finance/chart/${encodeURIComponent(symbol)}?interval=1d&range=2d`;
  try {
    const res = await fetch(url, {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept': 'application/json',
      },
      signal: AbortSignal.timeout(8000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const data = await res.json();
    const result = data?.chart?.result?.[0];
    if (!result) throw new Error('No result');

    const meta = result.meta;
    const price = meta.regularMarketPrice ?? null;
    const prevClose = meta.chartPreviousClose ?? meta.previousClose ?? null;
    let changePct = null;
    if (price != null && prevClose != null) {
      changePct = +((price - prevClose) / prevClose * 100).toFixed(2);
    }

    // 驗證數據是否過期（超過 30 天視為無效）
    const tradingDate = new Date(meta.regularMarketTime * 1000).toISOString().slice(0, 10);
    const daysDiff = (Date.now() - meta.regularMarketTime * 1000) / 86400000;
    if (daysDiff > 30) {
      return { symbol, ok: false, price: null, changePct: null, expiry: null, error: `Stale data: last traded ${tradingDate} (${Math.floor(daysDiff)}d ago)` };
    }

    // 合約到期月份（從 symbol 推算）
    let expiry = null;
    if (symbol !== 'HG=F' && symbol !== 'ZNC=F' && symbol !== 'ALI=F') {
      const m = symbol.match(/HG([FGHJKMNQUVXZ])(\d{2})\.CMX/);
      if (m) {
        const monthCodeMap = { F:1,G:2,H:3,J:4,K:5,M:6,N:7,Q:8,U:9,V:10,X:11,Z:12 };
        const mNum = String(monthCodeMap[m[1]]).padStart(2, '0');
        const yr = 2000 + parseInt(m[2]);
        expiry = `${yr}-${mNum}`;
      }
    } else {
      // 現貨合約用當前月份
      const now = new Date();
      expiry = `${now.getFullYear()}-${String(now.getMonth()+1).padStart(2,'0')}`;
    }

    return { symbol, ok: true, price, changePct, expiry };
  } catch (err) {
    process.stderr.write(`[fetch-all-data] Yahoo ${symbol} 錯誤: ${err.message}\n`);
    return { symbol, ok: false, price: null, changePct: null, expiry: null };
  }
}

// 周环比（较约5个交易日前收盘）
async function fetchYahooWoW(symbol) {
  const url = `https://query1.finance.yahoo.com/v8/finance/chart/${encodeURIComponent(symbol)}?interval=1d&range=1mo`;
  try {
    const res = await fetch(url, {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept': 'application/json',
      },
      signal: AbortSignal.timeout(8000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const data = await res.json();
    const result = data?.chart?.result?.[0];
    const closes = result?.indicators?.quote?.[0]?.close?.filter(v => v != null) || [];
    if (closes.length < 6) return null;
    const last = closes[closes.length - 1];
    const prevWeek = closes[closes.length - 6]; // 约5个交易日前
    if (last == null || prevWeek == null || prevWeek === 0) return null;
    return +(((last - prevWeek) / prevWeek) * 100).toFixed(2);
  } catch (err) {
    process.stderr.write(`[fetch-all-data] Yahoo WoW ${symbol} 錯誤: ${err.message}\n`);
    return null;
  }
}

// USD/CNY 匯率（Yahoo Finance + 備援）
async function fetchUsdcny() {
  // A) Yahoo（主來源）
  const fx = await fetchYahoo('USDCNY=X');
  if (fx.ok && fx.price != null) {
    return { price: fx.price, changePct: fx.changePct, source: 'Yahoo/USDCNY=X' };
  }

  // B) exchangerate.host（免費備援）
  try {
    const res = await fetch('https://api.exchangerate.host/convert?from=USD&to=CNY&amount=1', {
      headers: { 'User-Agent': 'Mozilla/5.0', 'Accept': 'application/json' },
      signal: AbortSignal.timeout(8000),
    });
    if (res.ok) {
      const data = await res.json();
      const p = Number(data?.result ?? data?.info?.rate);
      if (Number.isFinite(p) && p > 0) {
        return { price: +p.toFixed(4), changePct: null, source: 'exchangerate.host' };
      }
    }
  } catch (err) {
    process.stderr.write(`[fetch-all-data] FX 備援 exchangerate.host 錯誤: ${err.message}\n`);
  }

  // C) frankfurter.app（免費備援）
  try {
    const res = await fetch('https://api.frankfurter.app/latest?from=USD&to=CNY', {
      headers: { 'User-Agent': 'Mozilla/5.0', 'Accept': 'application/json' },
      signal: AbortSignal.timeout(8000),
    });
    if (res.ok) {
      const data = await res.json();
      const p = Number(data?.rates?.CNY);
      if (Number.isFinite(p) && p > 0) {
        return { price: +p.toFixed(4), changePct: null, source: 'frankfurter.app' };
      }
    }
  } catch (err) {
    process.stderr.write(`[fetch-all-data] FX 備援 frankfurter.app 錯誤: ${err.message}\n`);
  }

  return null;
}

// ────────────────────────────────────────────
// v7 新增：有色金屬行業指數（Yahoo Finance）
// 測試結果（2026-03-15）：
//   ^LMEX ⚠️ 過舊（2019）| JJM ⚠️ 過舊（2023）
//   XME ✅ | COPX ✅ | PICK ✅ | 000812.SS ✅
//   512400 ❌404 | 159163 ❌404
// 選定：XME（廣泛礦業） + COPX（銅礦股） + 000812.SS（申萬A股）
// ────────────────────────────────────────────

async function fetchMetalIndices() {
  const symbols = [
    { symbol: 'XME',       name: 'SPDR S&P Metals & Mining ETF',   market: 'US', currency: 'USD' },
    { symbol: 'COPX',      name: 'Global X Copper Miners ETF',      market: 'US', currency: 'USD' },
    { symbol: '000812.SS', name: '申萬有色金屬指數',                 market: 'CN', currency: 'CNY' },
  ];

  const results = await Promise.all(symbols.map(async ({ symbol, name, market, currency }) => {
    const [data, wowPct] = await Promise.all([
      fetchYahoo(symbol),
      fetchYahooWoW(symbol),
    ]);
    if (!data.ok || data.price === null) return null;
    const changeAbs = (data.price != null && data.changePct != null)
      ? +(data.price * data.changePct / (100 + data.changePct)).toFixed(3)
      : null;
    return {
      symbol,
      name,
      market,
      currency,
      price: data.price,
      changePct: data.changePct,
      changeAbs,
      wowPct,
    };
  }));

  return results.filter(Boolean);
}

// 宏觀風險指標：DXY / VIX / CRB / 美債10Y
async function fetchMacroIndicators() {
  const symbols = [
    { symbol: '^DXY', name: '美元指數', unit: 'pts' },
    { symbol: '^VIX', name: 'VIX恐慌指數', unit: 'pts' },
    { symbol: 'CRY',  name: 'CRB商品指數', unit: 'pts' },
    { symbol: '^TNX', name: '美債10Y收益率', unit: '%' },
  ];

  const results = await Promise.all(symbols.map(async (cfg) => {
    let data = await fetchYahoo(cfg.symbol);
    if ((!data.ok || data.price == null) && cfg.symbol === 'CRY') {
      // CRB 指數備用 symbol：TRJEFFCR
      data = await fetchYahoo('TRJEFFCR');
      cfg = { ...cfg, symbol: 'TRJEFFCR' };
    }
    if ((!data.ok || data.price == null) && cfg.symbol === '^DXY') {
      // DXY 備用 symbol：DX-Y.NYB
      data = await fetchYahoo('DX-Y.NYB');
      cfg = { ...cfg, symbol: 'DX-Y.NYB' };
    }
    if (!data.ok || data.price == null) return null;
    // ^TNX 報價通常為收益率×10（若>20則縮放），否則直接使用
    const rawPrice = data.price;
    const price = cfg.symbol === '^TNX' && rawPrice > 20 ? +(rawPrice / 10).toFixed(3) : rawPrice;
    return {
      ...cfg,
      price,
      changePct: data.changePct,
      source: 'Yahoo',
    };
  }));

  return results.filter(Boolean);
}

// ────────────────────────────────────────────
// 3. 鉍（Bi）— SMM 上海有色網 h5 頁面（免費，__NEXT_DATA__ 嵌入）
// ────────────────────────────────────────────
// URL: https://hq.smm.cn/h5/bismuth-price
// 數據：精鉍價格(CNY/t) + 精鉍CIF(USD/kg) + 4N/5N三氧化二鉍(CNY/t)
// 欄位：high / low / average / vchange(日變動絕對值) / vchange_rate(%) / renew_date
// 無需登錄，__NEXT_DATA__ 直接嵌入完整 JSON
// ────────────────────────────────────────────

async function fetchSmmBismuth() {
  try {
    const res = await fetch('https://hq.smm.cn/h5/bismuth-price', {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept-Language': 'zh-CN,zh;q=0.9',
        'Referer': 'https://www.smm.cn/',
      },
      signal: AbortSignal.timeout(10000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const html = await res.text();

    // 解析 __NEXT_DATA__
    const nd = html.match(/<script id="__NEXT_DATA__"[^>]*>([\s\S]*?)<\/script>/);
    if (!nd) throw new Error('__NEXT_DATA__ not found');
    const json = JSON.parse(nd[1]);

    const sections = json?.props?.pageProps?.datas?.BIP01?.data;
    if (!Array.isArray(sections)) throw new Error('BIP01 data not found');

    const result = { cny: null, usd: null, source: 'SMM/hq.smm.cn' };

    for (const section of sections) {
      for (const item of (section.data || [])) {
        const name = item.product_name || '';
        // 铋 = U+94CB
        // 精铋价格 (CNY/t)：name 含「铋」且不含 CIF，unit 含「元」
        // ⚠️ SMM vchange_rate 為小數格式（如 -0.0033 = -0.33%），需 ×100 轉為百分比
        if (name.includes('\u94cb') && !name.includes('CIF') && item.unit && item.unit.includes('\u5143') && !name.includes('N\u4e09')) {
          result.cny = {
            average: item.average,
            high: item.high,
            low: item.low,
            change: item.vchange,
            changePct: item.vchange_rate != null ? +(item.vchange_rate * 100).toFixed(4) : null,
            unit: item.unit,
            dataDate: item.renew_date,
          };
        } else if (name.includes('\u94cb') && name.includes('CIF')) {
          // 精铋CIF价格 (USD/kg → 換算 USD/t)
          const avgUsdPerKg = item.average;
          result.usd = {
            averagePerKg: avgUsdPerKg,
            average: avgUsdPerKg != null ? +(avgUsdPerKg * 1000).toFixed(0) : null, // USD/t
            high: item.high != null ? item.high * 1000 : null,
            low: item.low != null ? item.low * 1000 : null,
            change: item.vchange != null ? +(item.vchange * 1000).toFixed(0) : null,
            changePct: item.vchange_rate != null ? +(item.vchange_rate * 100).toFixed(4) : null,
            unit: 'USD/t',
            dataDate: item.renew_date,
          };
        }
      }
    }

    if (!result.cny && !result.usd) throw new Error('No bismuth prices found in page');

    const cnyAvg = result.cny?.average;
    const usdAvg = result.usd?.average;
    process.stderr.write(`[fetch-all-data] SMM 鉍價格：¥${cnyAvg}/t（日變動 ${result.cny?.change}）/ $${usdAvg}/t CIF\n`);
    return result;
  } catch (err) {
    process.stderr.write(`[fetch-all-data] SMM 鉍抓取失敗: ${err.message}\n`);
    return null;
  }
}

// ────────────────────────────────────────────
// 3b. SMM 長江現貨交叉驗證（Cu / Zn / Ni）
// ────────────────────────────────────────────
// 數據源調研結論（2026-03）：
//   ✅ cu-price: 長江現貨銅價 / ✅ zn-price: 長江現貨鋅錠 / ✅ ni-price: 長江鎳價
//   ❌ al-price: 404（鋁無 h5 頁面）/ ❌ cobalt-price/co-price: 404（鈷無 h5 頁面）
//   ❌ 所有金屬均無 LME USD 報價（Zn/Ni/Co USD 無免費數據源）
// ⚠️ vchange_rate 為小數格式，×100 轉百分比

async function fetchSmmCrossCheck() {
  const SMM_HEADERS = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
    'Accept-Language': 'zh-CN,zh;q=0.9',
    'Referer': 'https://www.smm.cn/',
  };

  // 遞歸提取所有含 product_name 的條目
  function walkItems(obj) {
    const items = [];
    function walk(o) {
      if (!o || typeof o !== 'object') return;
      if (o.product_name !== undefined && o.average != null) {
        items.push(o); return;
      }
      if (Array.isArray(o)) { o.forEach(walk); return; }
      Object.values(o).forEach(walk);
    }
    walk(obj);
    return items;
  }

  async function fetchSmm(slug, targetName, attempt = 1) {
    try {
      const res = await fetch(`https://hq.smm.cn/h5/${slug}`, {
        headers: SMM_HEADERS, signal: AbortSignal.timeout(12000),
      });
      if (!res.ok) throw new Error(`HTTP ${res.status}`);
      const html = await res.text();
      const nd = html.match(/<script id="__NEXT_DATA__"[^>]*>([\s\S]*?)<\/script>/);
      if (!nd) throw new Error('no __NEXT_DATA__');
      const json = JSON.parse(nd[1]);
      const items = walkItems(json?.props?.pageProps?.datas);
      const match = items.find(i => i.product_name && i.product_name.includes(targetName));
      if (!match) throw new Error(`"${targetName}" not found in ${items.length} items`);
      return {
        average: match.average,
        high: match.high,
        low: match.low,
        change: match.vchange,
        changePct: match.vchange_rate != null ? +(match.vchange_rate * 100).toFixed(4) : null,
        unit: match.unit,
        dataDate: match.renew_date,
        productName: match.product_name,
        source: `SMM/${slug}`,
      };
    } catch(err) {
      if (attempt === 1) {
        process.stderr.write(`[fetch-all-data] SMM cross-check ${slug} 第一次失敗 (${err.message})，1.5秒後重試...\n`);
        await new Promise(r => setTimeout(r, 1500));
        return fetchSmm(slug, targetName, 2);
      }
      process.stderr.write(`[fetch-all-data] SMM cross-check ${slug} 最終失敗: ${err.message}\n`);
      return null;
    }
  }

  const [cu, zn, ni] = await Promise.all([
    fetchSmm('cu-price', '\u957f\u6c5f\u73b0\u8d27\u94dc\u4ef7'),       // 长江现货铜价  铜=94DC
    fetchSmm('zn-price', '\u4e0a\u6d77\u73b0\u8d27\u950c\u952d\u4ef7\u683c0#'), // 上海现货锌锭价格0#  锌=950C 锭=952D
    fetchSmm('ni-price', '\u957f\u6c5f\u954d\u4ef7\u683c'),             // 长江镍价格  镍=954D
  ]);

  process.stderr.write(`[fetch-all-data] SMM交叉驗證: Cu=${cu?.average} Zn=${zn?.average} Ni=${ni?.average}\n`);
  return { copper: cu, zinc: zn, nickel: ni };
}

// ────────────────────────────────────────────
// 3c. 通用 fetchSmmMetal（v8 新增）
// 適用於任何有 __NEXT_DATA__ 的 SMM h5 頁面
// ────────────────────────────────────────────

async function fetchSmmMetal(slug, targetName) {
  const headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
    'Accept-Language': 'zh-CN,zh;q=0.9',
    'Referer': 'https://www.smm.cn/',
  };
  try {
    const res = await fetch(`https://hq.smm.cn/h5/${slug}`, {
      headers, signal: AbortSignal.timeout(12000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const html = await res.text();
    const nd = html.match(/<script id="__NEXT_DATA__"[^>]*>([\s\S]*?)<\/script>/);
    if (!nd) throw new Error('no __NEXT_DATA__');
    const json = JSON.parse(nd[1]);

    // 展開所有 data 條目
    const items = [];
    function walk(o) {
      if (!o || typeof o !== 'object') return;
      if (o.product_name !== undefined && o.average != null) { items.push(o); return; }
      if (Array.isArray(o)) { o.forEach(walk); return; }
      Object.values(o).forEach(walk);
    }
    walk(json?.props?.pageProps?.datas);

    const match = items.find(i => i.product_name && i.product_name.includes(targetName));
    if (!match) throw new Error(`"${targetName}" not found in ${items.length} items`);

    const changePct = match.vchange_rate != null ? +(match.vchange_rate * 100).toFixed(3) : null;
    return {
      average: match.average,
      high: match.high,
      low: match.low,
      change: match.vchange,
      changePct,
      unit: match.unit,
      dataDate: match.renew_date,
      productName: match.product_name,
      source: `SMM/hq.smm.cn/${slug}`,
    };
  } catch (err) {
    process.stderr.write(`[fetch-all-data] fetchSmmMetal(${slug}) 失敗: ${err.message}\n`);
    return null;
  }
}

// 交叉驗證說明生成器
function buildCrossCheckNote(ccmnPrice, smmPrice, smmLabel) {
  if (!ccmnPrice || !smmPrice) return null;
  const diffPct = +((smmPrice - ccmnPrice) / ccmnPrice * 100).toFixed(2);
  const sign = diffPct >= 0 ? '+' : '';
  const absDiff = Math.abs(diffPct);
  if (absDiff < 0.5) return `雙源一致：CCMN ¥${ccmnPrice} vs ${smmLabel} ¥${smmPrice} (${sign}${diffPct}%)`;
  if (absDiff > 1)   return `差異>1%：CCMN ¥${ccmnPrice} vs ${smmLabel} ¥${smmPrice} (${sign}${diffPct}%)`;
  return `差異<1%：CCMN ¥${ccmnPrice} vs ${smmLabel} ¥${smmPrice} (${sign}${diffPct}%)`;
}

// ────────────────────────────────────────────
// 4a. Westmetall.com — LME 庫存 + USD 現貨價格
// ────────────────────────────────────────────
// URL: https://www.westmetall.com/en/markdaten.php?action=table&field=LME_XX_stock
// 返回: { tonnes, change, cashUsd, threeMonthUsd, dataDate }
// 覆蓋品種: Cu / Zn / Ni（Westmetall 不提供 Co 數據）
// ────────────────────────────────────────────

const WESTMETALL_MONTHS = {
  January:1, February:2, March:3, April:4, May:5, June:6,
  July:7, August:8, September:9, October:10, November:11, December:12
};

function parseWestmetallDate(str) {
  const m = str.match(/(\d{1,2})\.\s+(\w+)\s+(\d{4})/);
  if (!m) return null;
  const d = String(parseInt(m[1])).padStart(2, '0');
  const mon = String(WESTMETALL_MONTHS[m[2]] || 0).padStart(2, '0');
  return `${m[3]}-${mon}-${d}`;
}

async function fetchWestmetallMetal(fieldName, attempt = 1) {
  // fieldName: 'LME_Cu_stock' | 'LME_Zn_stock' | 'LME_Ni_stock'
  const url = `https://www.westmetall.com/en/markdaten.php?action=table&field=${fieldName}`;
  try {
    const res = await fetch(url, {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept': 'text/html',
        'Referer': 'https://www.westmetall.com/en/markdaten.php',
      },
      signal: AbortSignal.timeout(15000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const html = await res.text();

    // 取 <tbody> 第一個 <tr>（最新一行）
    const tbodyMatch = html.match(/<tbody>([\s\S]*?)<\/tbody>/);
    if (!tbodyMatch) throw new Error('No <tbody> found');
    const tbody = tbodyMatch[1];
    const trMatch = tbody.match(/<tr>([\s\S]*?)<\/tr>/);
    if (!trMatch) throw new Error('No <tr> in tbody');
    const tr = trMatch[1];

    // 提取所有 <td> 文本
    const tdValues = [...tr.matchAll(/<td[^>]*>([\s\S]*?)<\/td>/g)]
      .map(m => m[1].replace(/<[^>]+>/g, '').trim());

    if (tdValues.length < 4) throw new Error(`Expected ≥4 td, got ${tdValues.length}`);

    const [dateStr, stockStr, cashStr, threeMonthStr] = tdValues;
    const dataDate = parseWestmetallDate(dateStr);

    // 解析下一行的庫存變化
    const rows = [...tbody.matchAll(/<tr>([\s\S]*?)<\/tr>/g)];
    let change = null;
    if (rows.length >= 2) {
      const prevTd = [...rows[1][1].matchAll(/<td[^>]*>([\s\S]*?)<\/td>/g)]
        .map(m => m[1].replace(/<[^>]+>/g, '').trim());
      if (prevTd.length >= 2) {
        const curr = parseFloat(stockStr.replace(/,/g, ''));
        const prev = parseFloat(prevTd[1].replace(/,/g, ''));
        if (!isNaN(curr) && !isNaN(prev)) change = curr - prev;
      }
    }

    return {
      tonnes: parseFloat(stockStr.replace(/,/g, '')) || null,
      change,
      cashUsd: parseFloat(cashStr.replace(/,/g, '')) || null,
      threeMonthUsd: parseFloat(threeMonthStr.replace(/,/g, '')) || null,
      dataDate,
      source: `Westmetall/${fieldName}`,
    };
  } catch (err) {
    if (attempt === 1) {
      process.stderr.write(`[fetch-all-data] Westmetall ${fieldName} 第一次失敗 (${err.message})，2秒後重試...\n`);
      await new Promise(r => setTimeout(r, 2000));
      return fetchWestmetallMetal(fieldName, 2);
    }
    process.stderr.write(`[fetch-all-data] Westmetall ${fieldName} 最終失敗: ${err.message}\n`);
    return null;
  }
}

async function fetchWestmetallAll() {
  const [cu, zn, ni] = await Promise.all([
    fetchWestmetallMetal('LME_Cu_stock'),
    fetchWestmetallMetal('LME_Zn_stock'),
    fetchWestmetallMetal('LME_Ni_stock'),
  ]);
  process.stderr.write(`[fetch-all-data] Westmetall: Cu=${cu?.cashUsd} Zn=${zn?.cashUsd} Ni=${ni?.cashUsd} | stocks: Cu=${cu?.tonnes} Zn=${zn?.tonnes} Ni=${ni?.tonnes}\n`);
  return { copper: cu, zinc: zn, nickel: ni };
}

// ────────────────────────────────────────────
// 4. LME 庫存（主函數 — 優先用 Westmetall）
// ────────────────────────────────────────────

async function fetchLmeInventory() {
  const errors = [];

  // 方案A：Westmetall.com（Cu/Zn/Ni LME 庫存，可靠免費源）
  // 同時緩存 cashUsd 供 main() 直接使用，避免重複請求
  try {
    const wm = await fetchWestmetallAll();
    const result = { copper: null, zinc: null, nickel: null, cobalt: null, note: null, _wmPrices: wm };
    for (const [metal, data] of [['copper', wm.copper], ['zinc', wm.zinc], ['nickel', wm.nickel]]) {
      if (data && data.tonnes != null) {
        result[metal] = {
          tonnes: data.tonnes,
          change: data.change,
          cashUsd: data.cashUsd,
          source: data.source,
          unit: 'tonnes',
          dataDate: data.dataDate,
        };
      }
    }
    const hasData = Object.entries(result).some(([k, v]) => k !== 'note' && k !== 'cobalt' && k !== '_wmPrices' && v != null);
    if (!hasData) throw new Error('Westmetall 返回空數據');
    process.stderr.write('[fetch-all-data] LME 方案A (Westmetall) 成功\n');
    return result;
  } catch (err) {
    errors.push(`方案A: ${err.message}`);
    process.stderr.write(`[fetch-all-data] LME 方案A (Westmetall) 失敗: ${err.message}\n`);
  }

  // 方案B：LME 倉庫統計頁面 HTML
  try {
    const res = await fetch(
      'https://www.lme.com/Market-Data/Reports-and-data/Warehouse-Stock-Statistics',
      {
        headers: {
          'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
          'Accept': 'text/html,application/xhtml+xml',
        },
        signal: AbortSignal.timeout(5000),
      }
    );
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const html = await res.text();
    if (html.includes('Just a moment') || html.includes('Cloudflare')) throw new Error('Cloudflare 攔截');

    const result = { copper: null, zinc: null, nickel: null, cobalt: null, note: null };
    const metalPatterns = [
      { key: 'copper', regex: /[Cc]opper[^0-9]*?([\d,]+)\s*tonnes?/i },
      { key: 'nickel', regex: /[Nn]ickel[^0-9]*?([\d,]+)\s*tonnes?/i },
      { key: 'zinc',   regex: /[Zz]inc[^0-9]*?([\d,]+)\s*tonnes?/i },
    ];
    for (const { key, regex } of metalPatterns) {
      const m = html.match(regex);
      if (m) {
        const tonnes = parseInt(m[1].replace(/,/g, ''), 10);
        if (!isNaN(tonnes)) {
          result[key] = { tonnes, change: null, source: 'LME', unit: 'tonnes' };
        }
      }
    }
    const hasData = Object.values(result).some(v => v && v.tonnes != null);
    if (!hasData) throw new Error('頁面無可解析數據');
    process.stderr.write('[fetch-all-data] LME 方案B 成功\n');
    return result;
  } catch (err) {
    errors.push(`方案B: ${err.message}`);
    process.stderr.write(`[fetch-all-data] LME 方案B 失敗: ${err.message}\n`);
  }

  // 方案C：Investing.com metals data
  try {
    const res = await fetch(
      'https://api.investing.com/api/financialdata/assets/equitiesByType?country=&type=metals&page=0&pageSize=20',
      {
        headers: {
          'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
          'Accept': 'application/json',
          'domain-id': 'www',
        },
        signal: AbortSignal.timeout(10000),
      }
    );
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const data = await res.json();
    if (!data) throw new Error('Empty response');
    // Investing.com 不提供庫存數據，此方案會失敗
    throw new Error('Investing.com 不提供 LME 庫存數據');
  } catch (err) {
    errors.push(`方案C: ${err.message}`);
    process.stderr.write(`[fetch-all-data] LME 方案C 失敗: ${err.message}\n`);
  }

  // 所有方案失敗
  const note = `LME 庫存獲取失敗: ${errors.join(' | ')}`;
  process.stderr.write(`[fetch-all-data] 所有 LME 方案失敗，返回 null\n`);
  return {
    copper: null,
    zinc: null,
    nickel: null,
    cobalt: null,
    note,
  };
}

// ────────────────────────────────────────────
// 5. Google News RSS 新聞
// ────────────────────────────────────────────

function parseRssItems(xml) {
  const items = [];
  const itemRegex = /<item>([\s\S]*?)<\/item>/gi;
  let match;
  while ((match = itemRegex.exec(xml)) !== null) {
    const block = match[1];
    const titleMatch = block.match(/<title><!\[CDATA\[([\s\S]*?)\]\]><\/title>/) ||
                       block.match(/<title>([\s\S]*?)<\/title>/);
    const linkMatch = block.match(/<link>([\s\S]*?)<\/link>/) ||
                      block.match(/<guid[^>]*>(https?:\/\/[^\s<]+)<\/guid>/);
    const title = titleMatch ? titleMatch[1].trim() : '';
    const url = linkMatch ? linkMatch[1].trim() : '';
    if (title) items.push({ title, url });
  }
  return items;
}

async function fetchNews() {
  const rssUrl = 'https://news.google.com/rss/search?q=%E6%9C%89%E8%89%B2%E9%87%91%E5%B1%9E+%E4%BB%B7%E6%A0%BC&hl=zh-CN&gl=CN&ceid=CN:zh-Hans';
  try {
    const res = await fetch(rssUrl, {
      headers: {
        'User-Agent': 'Mozilla/5.0 (compatible; MetalPriceBot/1.0)',
        'Accept': 'application/rss+xml, application/xml, text/xml, */*',
      },
      signal: AbortSignal.timeout(12000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const xml = await res.text();
    const items = parseRssItems(xml);
    return items.slice(0, 5);
  } catch (err) {
    process.stderr.write(`[fetch-all-data] 新聞抓取失敗: ${err.message}\n`);
    return [];
  }
}

// ────────────────────────────────────────────
// 6. 投行分析新聞（ibNews）
// v5: 改為基本金屬雙重過濾（投行名字 AND 基本金屬關鍵詞）
// ────────────────────────────────────────────

async function fetchIbNews() {
  const queries = [
    'Goldman+Sachs+JPMorgan+Citi+copper+nickel+zinc+outlook',
    'copper+nickel+zinc+cobalt+forecast+bank+2026',
    'base+metals+copper+nickel+Goldman+JPMorgan+forecast',
  ];

  const ibKeywords = ['Goldman', 'JPMorgan', 'Citi', 'Morgan Stanley', 'Bank of America', 'UBS', 'HSBC', 'Barclays', 'BNP', 'Deutsche'];
  const metalKeywords = ['copper', 'nickel', 'zinc', 'cobalt', 'alumin', 'base metal', 'industrial metal'];

  const allItems = [];

  for (const q of queries) {
    try {
      const url = `https://news.google.com/rss/search?q=${q}&hl=en-US&gl=US&ceid=US:en`;
      const res = await fetch(url, {
        headers: {
          'User-Agent': 'Mozilla/5.0 (compatible; MetalPriceBot/1.0)',
          'Accept': 'application/rss+xml, */*',
        },
        signal: AbortSignal.timeout(10000),
      });
      if (!res.ok) continue;
      const xml = await res.text();
      const items = parseRssItems(xml);

      // 雙重過濾：同時包含投行名字 AND 基本金屬關鍵詞
      const doubleFiltered = items.filter(i =>
        ibKeywords.some(k => i.title.toLowerCase().includes(k.toLowerCase())) &&
        metalKeywords.some(m => i.title.toLowerCase().includes(m.toLowerCase()))
      );

      if (doubleFiltered.length > 0) {
        process.stderr.write(`[fetch-all-data] ibNews 雙重過濾找到 ${doubleFiltered.length} 條基本金屬投行新聞 (query: ${q})\n`);
        // 合併去重
        for (const item of doubleFiltered) {
          if (!allItems.some(x => x.title === item.title)) allItems.push(item);
        }
      } else {
        process.stderr.write(`[fetch-all-data] ibNews 雙重過濾無結果 (query: ${q})，嘗試只過濾金屬關鍵詞\n`);
        // fallback：只過濾金屬關鍵詞
        const metalOnly = items.filter(i =>
          metalKeywords.some(m => i.title.toLowerCase().includes(m.toLowerCase()))
        );
        for (const item of metalOnly) {
          if (!allItems.some(x => x.title === item.title)) {
            allItems.push({ ...item, source: 'industry_news' });
          }
        }
      }
    } catch (err) {
      process.stderr.write(`[fetch-all-data] IB news fetch failed: ${err.message}\n`);
    }
  }

  if (allItems.length > 0) {
    process.stderr.write(`[fetch-all-data] ibNews 最終 ${allItems.length} 條\n`);
    return allItems.slice(0, 4);
  }
  process.stderr.write('[fetch-all-data] ibNews 未找到相關新聞\n');
  return [];
}

// ────────────────────────────────────────────
// v4 新增：7. SMM上海有色網新聞（免費公開）
// 狀態：✅ 200 OK，無需登錄即可抓取新聞標題
// ────────────────────────────────────────────

async function fetchSmmNews() {
  try {
    const res = await fetch('https://www.smm.cn/', {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept': 'text/html,application/xhtml+xml,*/*',
        'Accept-Language': 'zh-CN,zh;q=0.9',
      },
      signal: AbortSignal.timeout(8000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const html = await res.text();

    // 提取新聞標題（SMM 主頁快訊格式：包含日期時間和標題）
    const items = [];

    // 方式1：匹配 SMM快讯 格式
    const flashRegex = /【SMM[^】]*】([^<\n]{10,100})/g;
    let m;
    while ((m = flashRegex.exec(html)) !== null && items.length < 8) {
      const title = m[0].replace(/<[^>]+>/g, '').trim();
      if (title.length > 10) items.push({ title, source: 'SMM' });
    }

    // 方式2：匹配帶有日期的新聞標題區塊
    if (items.length < 3) {
      const titleRegex = /<a[^>]+href="https:\/\/news\.smm\.cn\/news\/[^"]+">([^<]{10,120})<\/a>/g;
      while ((m = titleRegex.exec(html)) !== null && items.length < 8) {
        const title = m[1].trim();
        if (title.length > 8 && !items.some(i => i.title === title)) {
          items.push({ title, source: 'SMM' });
        }
      }
    }

    process.stderr.write(`[fetch-all-data] SMM新聞: 提取 ${items.length} 條\n`);
    return items.slice(0, 5);
  } catch (err) {
    process.stderr.write(`[fetch-all-data] SMM新聞抓取失敗: ${err.message}\n`);
    return [];
  }
}

// ────────────────────────────────────────────
// v5 更新：8. Reddit 有色金屬相關討論
// 改為 r/Economics 搜索 copper+metals（v5測試結果：3條有效帖）
// 測試結果：r/mining(1帖), r/metallurgy(2帖), r/investing(0帖),
//           r/Economics search(3帖✅), global search(3帖但多為遊戲/藝術)
// ────────────────────────────────────────────

async function fetchRedditCommodities() {
  const metalKw = ['copper', 'nickel', 'zinc', 'cobalt', 'alumin', 'lead', 'tin',
                   'base metal', 'industrial metal', 'non-ferrous', 'lme', 'comex',
                   'mining', 'ore', 'smelter', 'refinery'];

  try {
    // 並行抓取 top（本週）和 hot（當前）
    const [topRes, hotRes] = await Promise.all([
      fetch('https://www.reddit.com/r/Commodities/top.json?t=week&limit=25', {
        headers: { 'User-Agent': 'MetalPriceBot/5.0 (non-ferrous metals research)' },
        signal: AbortSignal.timeout(8000),
      }),
      fetch('https://www.reddit.com/r/Commodities/hot.json?limit=25', {
        headers: { 'User-Agent': 'MetalPriceBot/5.0 (non-ferrous metals research)' },
        signal: AbortSignal.timeout(8000),
      }),
    ]);

    const topData = topRes.ok ? await topRes.json() : { data: { children: [] } };
    const hotData = hotRes.ok ? await hotRes.json() : { data: { children: [] } };

    const parsePosts = (data) => (data?.data?.children ?? [])
      .filter(p => p?.data?.title)
      .map(p => ({
        id: p.data.id,
        title: p.data.title,
        score: p.data.score || 0,
        url: `https://reddit.com${p.data.permalink}`,
      }));

    const topPosts = parsePosts(topData);
    const hotPosts = parsePosts(hotData);

    // 金屬關鍵詞過濾
    const isMetalRelated = (title) =>
      metalKw.some(k => title.toLowerCase().includes(k));

    const metalTop = topPosts.filter(p => isMetalRelated(p.title));
    const metalHot = hotPosts.filter(p => isMetalRelated(p.title));

    // 找出異動帖（在 hot 榜但不在 top 榜的 id）
    const topIds = new Set(topPosts.map(p => p.id));
    const surgingPosts = metalHot.filter(p => !topIds.has(p.id));

    // 組合輸出：金屬相關 top + 異動帖
    const combined = [
      ...metalTop.slice(0, 4).map(p => ({ ...p, tag: 'top' })),
      ...surgingPosts.slice(0, 2).map(p => ({ ...p, tag: 'surging' })),
    ];

    // 如果完全沒有金屬相關帖子，返回前3條 top 帖（帶 tag: 'general'）供參考
    const result = combined.length > 0
      ? combined
      : topPosts.slice(0, 3).map(p => ({ ...p, tag: 'general_commodities' }));

    const metalCount = metalTop.length + surgingPosts.length;
    process.stderr.write(`[fetch-all-data] Reddit r/Commodities: top=${topPosts.length}帖, hot=${hotPosts.length}帖, 金屬相關=${metalCount}帖, 異動=${surgingPosts.length}帖\n`);
    return result;
  } catch (err) {
    process.stderr.write(`[fetch-all-data] Reddit抓取失敗: ${err.message}\n`);
    return [];
  }
}

// ────────────────────────────────────────────
// v4 新增：9. 合成 forumSentiment 字段
// ────────────────────────────────────────────

function buildForumSentiment(smmItems, redditItems) {
  let smmHighlights = null;
  if (smmItems.length > 0) {
    smmHighlights = smmItems.map(i => i.title).join(' | ');
  }

  let redditSummary = null;
  let redditSurging = null;
  if (redditItems.length > 0) {
    const topItems = redditItems.filter(p => p.tag === 'top' || p.tag === 'general_commodities');
    const surgingItems = redditItems.filter(p => p.tag === 'surging');

    if (topItems.length > 0) {
      redditSummary = topItems.map(i => `[${i.score}↑] ${i.title}`).join(' | ');
    }
    if (surgingItems.length > 0) {
      redditSurging = surgingItems.map(i => `[異動🔥] ${i.title}`).join(' | ');
    }
  }

  return {
    smmHighlights,
    redditSummary,    // 金屬相關 top 帖
    redditSurging,    // 異動帖（hot but not top）
    xueqiuSummary: null,
    fetchedAt: new Date().toISOString(),
  };
}

// ────────────────────────────────────────────
// 3d. 鈷（Co）USD 現貨價格
// 主源：tradingeconomics.com（meta description 嵌入，穩定可靠）
// 備用：dailymetalprice.com（JSON 數組，USD/lb → USD/t）
// 測試日期：2026-03-17
// ────────────────────────────────────────────

async function fetchCobaltUsd() {
  // === 方案A：tradingeconomics.com meta description ===
  // 格式："Cobalt traded flat at 56,290 USD/T on March 12, 2026."
  try {
    const res = await fetch('https://tradingeconomics.com/commodity/cobalt', {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept': 'text/html,*/*',
        'Accept-Language': 'en-US,en;q=0.9',
        'Cache-Control': 'no-cache',
      },
      signal: AbortSignal.timeout(12000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const html = await res.text();

    const metaDesc = html.match(/<meta[^>]+name="description"[^>]+content="([^"]+)"/);
    if (!metaDesc) throw new Error('meta description not found');

    const priceMatch = metaDesc[1].match(/at\s+([\d,]+)\s*USD\/T/i);
    if (!priceMatch) throw new Error(`price pattern not found in: ${metaDesc[1].slice(0, 80)}`);

    const price = parseFloat(priceMatch[1].replace(/,/g, ''));
    if (isNaN(price) || price <= 0) throw new Error(`invalid price: ${priceMatch[1]}`);

    // 提取日期
    const dateMatch = metaDesc[1].match(/on\s+(\w+)\s+(\d+),\s+(\d{4})/i);
    let dataDate = null;
    if (dateMatch) {
      const monthNames = { January:1,February:2,March:3,April:4,May:5,June:6,
                           July:7,August:8,September:9,October:10,November:11,December:12 };
      const m = monthNames[dateMatch[1]];
      if (m) {
        dataDate = `${dateMatch[3]}-${String(m).padStart(2,'0')}-${String(parseInt(dateMatch[2])).padStart(2,'0')}`;
      }
    }

    process.stderr.write(`[fetch-all-data] 鈷 USD 方案A (TradingEconomics): $${price}/t, date=${dataDate}\n`);
    return { price, unit: 'USD/t', dataDate, source: 'TradingEconomics' };
  } catch(err) {
    process.stderr.write(`[fetch-all-data] 鈷 USD 方案A (TradingEconomics) 失敗: ${err.message}\n`);
  }

  // === 方案B：dailymetalprice.com JSON array（USD/lb）===
  // 格式：data = [[timestamp_ms, price_usd_per_lb], ...]
  try {
    const res = await fetch('https://www.dailymetalprice.com/metalpricecharts.php?c=co&u=usd&d=5', {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept': 'text/html,*/*',
        'Referer': 'https://www.dailymetalprice.com/',
      },
      signal: AbortSignal.timeout(10000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const text = await res.text();

    // Extract data array: [[timestamp, price], ...]
    const arrMatch = text.match(/data\s*[=:]\s*(\[\[[\s\S]*?\]\])/);
    if (!arrMatch) throw new Error('data array not found');

    const arr = JSON.parse(arrMatch[1]);
    if (!Array.isArray(arr) || arr.length === 0) throw new Error('empty data array');

    // Take the most recent entry (first = latest)
    const [ts, pricePerLb] = arr[0];
    if (typeof pricePerLb !== 'number' || pricePerLb <= 0) throw new Error(`invalid price: ${pricePerLb}`);

    // Convert USD/lb → USD/t (1 short ton = 2000 lb, but metal convention uses metric ton = 2204.623 lb)
    const pricePerTon = Math.round(pricePerLb * 2204.623);
    const dataDate = new Date(ts).toISOString().slice(0, 10);

    process.stderr.write(`[fetch-all-data] 鈷 USD 方案B (DailyMetalPrice): ${pricePerLb} USD/lb → $${pricePerTon}/t, date=${dataDate}\n`);
    return { price: pricePerTon, pricePerLb, unit: 'USD/t', dataDate, source: 'DailyMetalPrice' };
  } catch(err) {
    process.stderr.write(`[fetch-all-data] 鈷 USD 方案B (DailyMetalPrice) 失敗: ${err.message}\n`);
  }

  // === 方案C：SMM CNY ÷ USD/CNY 匯率估算 ===
  // 使用 CCMN 鈷 CNY 價格 + Yahoo Finance USD/CNY 匯率反推
  try {
    const fxRes = await fetch('https://query1.finance.yahoo.com/v8/finance/chart/USDCNY=X?interval=1d&range=2d', {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept': 'application/json',
      },
      signal: AbortSignal.timeout(8000),
    });
    if (!fxRes.ok) throw new Error(`Yahoo FX HTTP ${fxRes.status}`);
    const fxData = await fxRes.json();
    const usdcny = fxData?.chart?.result?.[0]?.meta?.regularMarketPrice;
    if (!usdcny || usdcny <= 0) throw new Error(`invalid USD/CNY: ${usdcny}`);
    // Will be combined with CCMN cobalt CNY price in main()
    process.stderr.write(`[fetch-all-data] 鈷 USD 方案C 準備: USD/CNY=${usdcny}\n`);
    return { price: null, usdcny, unit: 'USD/t', source: 'SMM-CNY/FX-estimate', needsCny: true };
  } catch(err) {
    process.stderr.write(`[fetch-all-data] 鈷 USD 方案C (SMM CNY/FX) 失敗: ${err.message}\n`);
  }

  return null;
}

// ────────────────────────────────────────────
// 主函數
// ────────────────────────────────────────────

async function main() {
  const startTime = Date.now();

  // 計算遠期合約 symbol
  const now = new Date();
  const monthCodes = ['F','G','H','J','K','M','N','Q','U','V','X','Z'];
  const curr = now.getMonth(); // 0-11
  const m2 = monthCodes[(curr + 2) % 12];
  const m6 = monthCodes[(curr + 6) % 12];
  const y2 = (curr + 2 >= 12) ? (now.getFullYear() + 1) % 100 : now.getFullYear() % 100;
  const y6 = (curr + 6 >= 12) ? (now.getFullYear() + 1) % 100 : now.getFullYear() % 100;
  const sym2 = `HG${m2}${y2 < 10 ? '0'+y2 : y2}.CMX`;
  const sym6 = `HG${m6}${y6 < 10 ? '0'+y6 : y6}.CMX`;

  process.stderr.write(`[fetch-all-data] 遠期合約: 近月=${sym2}, 遠月=${sym6}\n`);

  // 並行抓取所有數據（v10: 新增 fetchCobaltUsd）
  const [
    ccmn,
    ometal,
    copperSpot,
    zincSpot,
    alumSpot,
    fwdNear,
    fwdFar,
    bismuth,
    smmCross,
    smmLead,
    smmTin,
    inventory,
    cobaltUsdData,
    news,
    ibNews,
    smmNews,
    redditPosts,
    metalIndices,
    fxRate,
    macroIndicators,
  ] = await Promise.all([
    fetchCcmnPrices(),
    fetchOmetal(),                 // v11 新增：OmetalCN 備用源（Cu/Al/Pb/Zn/Ni/Sn）
    fetchYahoo('HG=F'),
    fetchYahoo('ZNC=F'),
    fetchYahoo('ALI=F'),          // 鋁現貨 USD/t
    fetchYahoo(sym2),
    fetchYahoo(sym6),
    fetchSmmBismuth(),             // 鉍：SMM h5 __NEXT_DATA__（含 CNY + CIF USD）
    fetchSmmCrossCheck(),          // SMM 長江報價交叉驗證（Cu/Zn/Ni）
    fetchSmmMetal('pb-price', '长江现货铅锭价格'),   // 鉛 CNY
    fetchSmmMetal('sn-price', '长江锡锭价格'),       // 錫 CNY
    fetchLmeInventory(),           // Westmetall LME 庫存 + Zn/Ni USD
    fetchCobaltUsd(),              // 鈷 USD 現貨（TradingEconomics / DailyMetalPrice）
    fetchNews(),
    fetchIbNews(),
    fetchSmmNews(),
    fetchRedditCommodities(),
    fetchMetalIndices(),
    fetchUsdcny(),                 // USD/CNY 匯率（供進口盈虧、基差）
    fetchMacroIndicators(),        // 宏觀風險指標（DXY/VIX/CRB/TNX）
  ]);
  // v9: 從 inventory._wmPrices 提取 Westmetall 現貨 USD 數據（從輸出中清理內部字段）
  const westmetall = inventory?._wmPrices ?? { copper: null, zinc: null, nickel: null };
  if (inventory) delete inventory._wmPrices;

  // 升級一：dataDate / isMarketOpen / marketNote
  const dataDate = ccmn?.dataDate ?? null;
  const isMarketOpen = ccmn?.isMarketOpen ?? null;
  const todaySH = today();
  let marketNote = null;
  if (isMarketOpen === false && dataDate) {
    const displayDate = dataDate.replace(/-/g, '/');
    marketNote = `休市：數據截至 ${displayDate}（上個交易日）`;
  }

  // 組裝 prices（v8：CCMN+SMM 交叉驗證 + 鉛/錫新增 + 鋁CNY從CCMN）
  const prices = {
    copper: {
      usd: copperSpot.price,
      usdChangePct: copperSpot.changePct,   // 日環比 %（vs 前一交易日收盤）
      usdUnit: 'USD/lb',
      cny: ccmn?.copper?.price ?? smmCross?.copper?.average ?? null,
      cnyChange: ccmn?.copper?.updown ?? smmCross?.copper?.change ?? null,  // 日環比 元/噸
      // v8 交叉驗證：SMM 長江現貨銅價
      smmCny: smmCross?.copper?.average ?? null,
      crossCheckNote: buildCrossCheckNote(ccmn?.copper?.price, smmCross?.copper?.average, 'SMM長江銅'),
    },
    zinc: {
      // v9: 從 Westmetall LME Cash-Settlement 獲取 USD 現貨
      usd: westmetall?.zinc?.cashUsd ?? null,
      usdChangePct: null,  // Westmetall 不提供日漲跌%，保持 null
      usdUnit: 'USD/t',
      cny: ccmn?.zinc?.price ?? smmCross?.zinc?.average ?? null,
      cnyChange: ccmn?.zinc?.updown ?? smmCross?.zinc?.change ?? null,
      // v8 交叉驗證：SMM 上海現貨0#鋅（與 CCMN 廣東市場報價較接近）
      smmCny: smmCross?.zinc?.average ?? null,
      crossCheckNote: buildCrossCheckNote(ccmn?.zinc?.price, smmCross?.zinc?.average, 'SMM上海0#鋅'),
    },
    aluminum: {
      usd: alumSpot.ok ? alumSpot.price : null,
      usdChangePct: alumSpot.ok ? alumSpot.changePct : null,
      usdUnit: 'USD/t',
      // v11: 優先 CCMN A00鋁 → 備用 OmetalCN A00鋁（SMM 無鋁 h5 頁面）
      cny: ccmn?.aluminum?.price ?? ometal?.aluminum?.price ?? null,
      cnyChange: ccmn?.aluminum?.updown ?? ometal?.aluminum?.change ?? null,
      cnySource: ccmn?.aluminum?.price != null ? 'CCMN' : (ometal?.aluminum?.price != null ? 'OmetalCN' : null),
    },
    nickel: {
      // v9: 從 Westmetall LME Cash-Settlement 獲取 USD 現貨
      usd: westmetall?.nickel?.cashUsd ?? null,
      usdChangePct: null,  // Westmetall 不提供日漲跌%，保持 null
      usdUnit: 'USD/t',
      cny: ccmn?.nickel?.price ?? smmCross?.nickel?.average ?? null,
      cnyChange: ccmn?.nickel?.updown ?? smmCross?.nickel?.change ?? null,
      // v8 交叉驗證：SMM 長江鎳價格（電解鎳）
      smmCny: smmCross?.nickel?.average ?? null,
      crossCheckNote: buildCrossCheckNote(ccmn?.nickel?.price, smmCross?.nickel?.average, 'SMM電解鎳'),
    },
    cobalt: (() => {
      // v10: 解析 fetchCobaltUsd() 返回值
      let cobaltUsd = null;
      let cobaltUsdSource = null;
      let cobaltUsdDate = null;
      if (cobaltUsdData) {
        if (cobaltUsdData.needsCny && cobaltUsdData.usdcny) {
          // 方案C：SMM CNY ÷ FX 估算
          const cnyCobalt = ccmn?.cobalt?.price;
          if (cnyCobalt && cnyCobalt > 0) {
            cobaltUsd = Math.round(cnyCobalt / cobaltUsdData.usdcny);
            cobaltUsdSource = 'SMM-CNY/FX-estimate';
          }
        } else if (cobaltUsdData.price) {
          cobaltUsd = cobaltUsdData.price;
          cobaltUsdSource = cobaltUsdData.source;
          cobaltUsdDate = cobaltUsdData.dataDate;
        }
      }
      return {
        usd: cobaltUsd,
        usdChangePct: null,
        usdUnit: 'USD/t',
        usdDataDate: cobaltUsdDate,
        usdSource: cobaltUsdSource,
        cny: ccmn?.cobalt?.price ?? null,
        cnyChange: ccmn?.cobalt?.updown ?? null,
      };
    })(),
    // 鉍（Bi）— SMM 上海有色網實時數據
    bismuth: bismuth ? {
      cny: bismuth.cny?.average ?? null,
      cnyHigh: bismuth.cny?.high ?? null,
      cnyLow: bismuth.cny?.low ?? null,
      cnyChange: bismuth.cny?.change ?? null,       // 日環比絕對值 元/噸
      cnyChangePct: bismuth.cny?.changePct ?? null, // 日環比 %（SMM vchange_rate×100）
      cnyUnit: '\u5143/\u5428',
      usd: bismuth.usd?.average ?? null,            // USD/t (CIF)
      usdHigh: bismuth.usd?.high ?? null,
      usdLow: bismuth.usd?.low ?? null,
      usdChange: bismuth.usd?.change ?? null,
      usdChangePct: bismuth.usd?.changePct ?? null,
      usdUnit: 'USD/t',
      dataDate: bismuth.cny?.dataDate ?? bismuth.usd?.dataDate ?? null,
      source: bismuth.source,
    } : {
      cny: null, usd: null,
      source: null,
      note: 'SMM抓取失敗，暫無鉍數據',
    },
    // v12 新增：鎂（Mg）— CCMN 1#鎂
    magnesium: ccmn?.magnesium ? {
      cny: ccmn.magnesium.price ?? null,
      cnyChange: ccmn.magnesium.updown ?? null,
      cnyUnit: '\u5143/\u5428',
      usd: null,
      source: 'CCMN',
    } : { cny: null, usd: null, source: null },
    // v8 新增：鉛（Pb）— SMM pb-price 長江現貨
    lead: smmLead ? {
      cny: smmLead.average,
      cnyHigh: smmLead.high,
      cnyLow: smmLead.low,
      cnyChange: smmLead.change,
      cnyChangePct: smmLead.changePct,
      cnyUnit: '\u5143/\u5428',
      usd: null,  // SMM pb-price 無 LME USD 頁面
      dataDate: smmLead.dataDate,
      source: smmLead.source,
    } : { cny: null, usd: null, source: null },
    // v8 新增：錫（Sn）— SMM sn-price 長江現貨
    tin: smmTin ? {
      cny: smmTin.average,
      cnyHigh: smmTin.high,
      cnyLow: smmTin.low,
      cnyChange: smmTin.change,
      cnyChangePct: smmTin.changePct,
      cnyUnit: '\u5143/\u5428',
      usd: null,  // SMM sn-price 無 LME USD 頁面
      dataDate: smmTin.dataDate,
      source: smmTin.source,
    } : { cny: null, usd: null, source: null },
  };

  // 組裝 forwards
  const spotExpiry = `${now.getFullYear()}-${String(now.getMonth()+1).padStart(2,'0')}`;
  const forwards = {
    copper: {
      spot: {
        price: copperSpot.price,
        symbol: 'HG=F',
        expiry: spotExpiry,
      },
      near: {
        price: fwdNear.price,
        symbol: sym2,
        expiry: fwdNear.expiry,
      },
      far: {
        price: fwdFar.price,
        symbol: sym6,
        expiry: fwdFar.expiry,
      },
    },
  };

  // v4 新增：組裝 forumSentiment
  const forumSentiment = buildForumSentiment(smmNews, redditPosts);

  const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
  process.stderr.write(`[fetch-all-data] 完成，耗時 ${elapsed}s\n`);

  // 計算交叉驗證差異
  const crossCheckDiff = (ccmnVal, smmVal) => {
    if (ccmnVal == null || smmVal == null) return null;
    return +((Math.abs(ccmnVal - smmVal) / ccmnVal * 100).toFixed(3));
  };

  const output = {
    date: todaySH,
    dataDate,
    isMarketOpen,
    marketNote,
    changeNote: '所有漲跌均為日環比（vs 前一交易日收盤）',
    prices,
    forwards,
    indices: metalIndices,
    macro: macroIndicators,
    fxRates: {
      usdCny: fxRate,
    },
    inventory,
    // SMM 長江報價交叉驗證（與 CCMN 對比，差異 <1% 為正常市場誤差）
    smmCrossCheck: {
      copper: smmCross?.copper ? {
        smmAvg: smmCross.copper.average,
        smmChange: smmCross.copper.change,
        smmChangePct: smmCross.copper.changePct,
        ccmnAvg: ccmn?.copper?.price ?? null,
        diffPct: crossCheckDiff(ccmn?.copper?.price, smmCross.copper.average),
        consistent: crossCheckDiff(ccmn?.copper?.price, smmCross.copper.average) != null
          ? crossCheckDiff(ccmn?.copper?.price, smmCross.copper.average) < 1
          : null,
        note: smmCross.copper.productName,
      } : null,
      zinc: smmCross?.zinc ? {
        smmAvg: smmCross.zinc.average,
        smmChange: smmCross.zinc.change,
        smmChangePct: smmCross.zinc.changePct,
        ccmnAvg: ccmn?.zinc?.price ?? null,
        diffPct: crossCheckDiff(ccmn?.zinc?.price, smmCross.zinc.average),
        consistent: crossCheckDiff(ccmn?.zinc?.price, smmCross.zinc.average) != null
          ? crossCheckDiff(ccmn?.zinc?.price, smmCross.zinc.average) < 1
          : null,
        note: smmCross.zinc.productName,
      } : null,
      nickel: smmCross?.nickel ? {
        smmAvg: smmCross.nickel.average,
        smmChange: smmCross.nickel.change,
        smmChangePct: smmCross.nickel.changePct,
        ccmnAvg: ccmn?.nickel?.price ?? null,
        diffPct: crossCheckDiff(ccmn?.nickel?.price, smmCross.nickel.average),
        consistent: crossCheckDiff(ccmn?.nickel?.price, smmCross.nickel.average) != null
          ? crossCheckDiff(ccmn?.nickel?.price, smmCross.nickel.average) < 1
          : null,
        note: smmCross.nickel.productName,
      } : null,
    },
    // 數據可用性說明（v9 更新：新增 Westmetall LME庫存+Zn/Ni USD現貨）
    dataAvailability: {
      copper:  { usd: 'Yahoo HG=F ✅', cny: ccmn?.copper?.price ? 'CCMN ✅（主）/ SMM長江✅（校驗）' : (smmCross?.copper?.average ? 'SMM長江 ✅（CCMN故障自動切備援）' : '❌ CNY源缺失') },
      zinc:    { usd: westmetall?.zinc?.cashUsd ? `Westmetall LME Cash ✅ $${westmetall.zinc.cashUsd}/t` : '❌ Westmetall抓取失敗', cny: ccmn?.zinc?.price ? 'CCMN ✅（主）/ SMM上海0#✅（校驗）' : (smmCross?.zinc?.average ? 'SMM上海0# ✅（CCMN故障自動切備援）' : '❌ CNY源缺失') },
      aluminum:{ usd: 'Yahoo ALI=F ✅', cny: ccmn?.aluminum?.price ? 'CCMN A00鋁 ✅' : (ometal?.aluminum?.price ? 'OmetalCN A00鋁 ✅（備用）' : '❌ 無鋁CNY數據') },
      nickel:  { usd: westmetall?.nickel?.cashUsd ? `Westmetall LME Cash ✅ $${westmetall.nickel.cashUsd}/t` : '❌ Westmetall抓取失敗', cny: ccmn?.nickel?.price ? 'CCMN ✅（主）/ SMM電解鎳✅（校驗）' : (smmCross?.nickel?.average ? 'SMM電解鎳 ✅（CCMN故障自動切備援）' : '❌ CNY源缺失') },
      cobalt:  {
        usd: cobaltUsdData?.price
          ? `${cobaltUsdData.source} ✅ $${cobaltUsdData.price}/t (${cobaltUsdData.dataDate})`
          : cobaltUsdData?.needsCny
            ? `SMM-CNY/FX-estimate ✅（估算）`
            : '❌ 所有源失敗',
        cny: 'CCMN ✅',
      },
      bismuth:   { usd: 'SMM CIF ✅（精鉍USD/kg×1000）', cny: 'SMM精鉍 ✅' },
      magnesium: { usd: '❌ 無免費USD源', cny: ccmn?.magnesium?.price ? 'CCMN 1#鎂 ✅（v12新增）' : '❌ CCMN無鎂數據' },
      lead:    { usd: '❌ 無免費源', cny: smmLead ? 'SMM長江鉛錠 ✅（v8新增）' : '❌ SMM抓取失敗' },
      tin:     { usd: '❌ 無免費源', cny: smmTin  ? 'SMM長江錫錠 ✅（v8新增）' : '❌ SMM抓取失敗' },
      lmeInventory: westmetall?.copper?.tonnes ? `Westmetall ✅（Cu=${westmetall.copper.tonnes}t, Zn=${westmetall.zinc?.tonnes}t, Ni=${westmetall.nickel?.tonnes}t）` : '❌ Westmetall抓取失敗',
    },
    news,
    ibNews,
    forumSentiment,
  };

  console.log(JSON.stringify(output, null, 2));
}

main().catch(err => {
  process.stderr.write(`[fetch-all-data] 致命錯誤: ${err.message}\n`);
  // 即使崩潰也輸出合法 JSON
  console.log(JSON.stringify({
    date: today(),
    dataDate: null,
    isMarketOpen: null,
    marketNote: null,
    changeNote: '所有漲跌均為日環比（vs 前一交易日收盤）',
    prices: { copper: null, zinc: null, aluminum: null, nickel: null, cobalt: null, bismuth: null, magnesium: null, lead: null, tin: null },
    forwards: { copper: null },
    indices: [],
    macro: [],
    fxRates: { usdCny: null },
    inventory: { copper: null, zinc: null, nickel: null, cobalt: null, note: err.message },
    news: [],
    ibNews: [],
    forumSentiment: { smmHighlights: null, redditSummary: null, redditSurging: null, xueqiuSummary: null, fetchedAt: new Date().toISOString() },
    error: err.message,
  }, null, 2));
  process.exit(1);
});


path: scripts/fetch-news.mjs
content: /**
 * fetch-news.mjs
 * 抓取有色金屬相關新聞（RSS 解析，無需外部庫）
 * 數據源優先級：Google News RSS → Reuters RSS → Yahoo Finance RSS
 */

const METAL_KEYWORDS = ['copper', 'zinc', 'nickel', 'cobalt', 'metal', '铜', '锌', '镍', '钴', '有色'];

// ────────────────────────────────────────────
// RSS XML 手動解析
// ────────────────────────────────────────────
function parseRSS(xml) {
  const items = [];
  const itemRegex = /<item>([\s\S]*?)<\/item>/gi;
  let match;

  while ((match = itemRegex.exec(xml)) !== null) {
    const block = match[1];

    // 標題
    const titleMatch = block.match(/<title><!\[CDATA\[([\s\S]*?)\]\]><\/title>/) ||
                       block.match(/<title>([\s\S]*?)<\/title>/);
    const title = titleMatch ? titleMatch[1].trim() : '';

    // URL
    const linkMatch = block.match(/<link>([\s\S]*?)<\/link>/) ||
                      block.match(/<guid[^>]*>(https?:\/\/[^\s<]+)<\/guid>/);
    const url = linkMatch ? linkMatch[1].trim() : '';

    // 發布時間
    const pubDateMatch = block.match(/<pubDate>([\s\S]*?)<\/pubDate>/);
    const publishedAt = pubDateMatch ? pubDateMatch[1].trim() : '';

    if (title) {
      items.push({ title, url, publishedAt });
    }
  }
  return items;
}

// ────────────────────────────────────────────
// 過濾金屬相關新聞（關鍵詞匹配）
// ────────────────────────────────────────────
function filterMetalNews(items) {
  return items.filter(item => {
    const text = (item.title + ' ' + (item.description || '')).toLowerCase();
    return METAL_KEYWORDS.some(kw => text.includes(kw.toLowerCase()));
  });
}

// ────────────────────────────────────────────
// 單個 RSS 源抓取
// ────────────────────────────────────────────
// ────────────────────────────────────────────
// 日期過濾（所有源通用，只保留 36h 內）
// ────────────────────────────────────────────
function filterByDate(items) {
  const cutoff = Date.now() - 36 * 60 * 60 * 1000;
  return items.filter(item => {
    if (!item.publishedAt) return true; // 無日期字段不過濾
    const ts = Date.parse(item.publishedAt);
    return isNaN(ts) || ts >= cutoff;
  });
}

async function fetchRSS(url, needFilter = false) {
  const res = await fetch(url, {
    headers: {
      'User-Agent': 'Mozilla/5.0 (compatible; MetalPriceBot/1.0)',
      'Accept': 'application/rss+xml, application/xml, text/xml, */*',
    },
    signal: AbortSignal.timeout(12000),
  });
  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  const xml = await res.text();
  let items = parseRSS(xml);
  if (needFilter) items = filterMetalNews(items);
  items = filterByDate(items); // 所有源都過濾日期
  return items;
}

// ────────────────────────────────────────────
// 主函數：按優先級嘗試數據源
// ────────────────────────────────────────────
async function main() {
  const sources = [
    {
      name: 'Google News RSS',
      url: 'https://news.google.com/rss/search?q=%E6%9C%89%E8%89%B2%E9%87%91%E5%B1%9E+%E4%BB%B7%E6%A0%BC&hl=zh-CN&gl=CN&ceid=CN:zh-Hans',
      needFilter: false,
    },
    {
      name: 'Reuters RSS',
      url: 'https://feeds.reuters.com/reuters/UKBusinessNews',
      needFilter: true,
    },
    {
      name: 'Yahoo Finance RSS',
      url: 'https://finance.yahoo.com/rss/topstories',
      needFilter: true,
    },
  ];

  let news = [];
  let usedSource = null;

  for (const source of sources) {
    try {
      process.stderr.write(`[fetch-news] 嘗試 ${source.name}...\n`);
      const items = await fetchRSS(source.url, source.needFilter);
      if (items.length > 0) {
        news = items.slice(0, 5);
        usedSource = source.name;
        process.stderr.write(`[fetch-news] ✅ ${source.name} 成功，獲取 ${news.length} 條\n`);
        break;
      } else {
        process.stderr.write(`[fetch-news] ⚠️ ${source.name} 返回 0 條，繼續下一個\n`);
      }
    } catch (err) {
      process.stderr.write(`[fetch-news] ❌ ${source.name} 失敗: ${err.message}\n`);
    }
  }

  if (news.length === 0) {
    process.stderr.write('[fetch-news] 所有數據源均失敗或無相關新聞\n');
  }

  const output = {
    source: usedSource,
    count: news.length,
    items: news,
  };

  console.log(JSON.stringify(output, null, 2));
}

main().catch(err => {
  console.error('Fatal error:', err.message);
  process.exit(1);
});


path: scripts/fetch-prices.mjs
content: /**
 * fetch-prices.mjs
 * 抓取有色金属现货/期货价格
 * 数据源：Yahoo Finance v8 API + 长江有色(CCMN) + Stooq
 */

import { readFileSync } from 'fs';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';

const __dirname = dirname(fileURLToPath(import.meta.url));
const PROJECT_ROOT = join(__dirname, '..');

// ────────────────────────────────────────────
// 读取 .env
// ────────────────────────────────────────────
function loadEnv() {
  const envPath = join(PROJECT_ROOT, '.env');
  const env = {};
  try {
    const content = readFileSync(envPath, 'utf-8');
    for (const line of content.split('\n')) {
      const trimmed = line.trim();
      if (!trimmed || trimmed.startsWith('#')) continue;
      const eqIdx = trimmed.indexOf('=');
      if (eqIdx === -1) continue;
      const key = trimmed.slice(0, eqIdx).trim();
      const value = trimmed.slice(eqIdx + 1).trim().replace(/^["']|["']$/g, '');
      env[key] = value;
    }
  } catch { /* ignore */ }
  return env;
}

// ────────────────────────────────────────────
// Yahoo Finance v8 API
// ────────────────────────────────────────────
async function fetchYahooPrice(symbol, name, unit, exchange) {
  const url = `https://query1.finance.yahoo.com/v8/finance/chart/${encodeURIComponent(symbol)}?interval=1d&range=2d`;
  try {
    const res = await fetch(url, {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept': 'application/json',
      },
      signal: AbortSignal.timeout(10000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const data = await res.json();
    const result = data?.chart?.result?.[0];
    if (!result) throw new Error('No result in response');

    const meta = result.meta;
    const currentPrice = meta.regularMarketPrice;
    const prevClose = meta.chartPreviousClose ?? meta.previousClose;

    let changePct = null;
    if (prevClose && currentPrice) {
      changePct = +((currentPrice - prevClose) / prevClose * 100).toFixed(2);
    }

    return {
      name,
      price: currentPrice ?? null,
      changePct,
      unit,
      exchange,
      source: 'Yahoo Finance',
      ccmnPrice: null,
      ccmnUpdown: null,
    };
  } catch (err) {
    process.stderr.write(`[fetch-prices] Yahoo Finance ${symbol} error: ${err.message}\n`);
    return {
      name,
      price: null,
      changePct: null,
      unit,
      exchange,
      source: 'Yahoo Finance',
      ccmnPrice: null,
      ccmnUpdown: null,
    };
  }
}

// ────────────────────────────────────────────
// Stooq CSV API（镍 NI.F 等，备用）
// 价格单位：美分/磅（cents/lb），需转换为 USD/t
// ────────────────────────────────────────────
async function fetchStooqPrice(symbol, name, unit, exchange) {
  const url = `https://stooq.com/q/l/?s=${encodeURIComponent(symbol)}&f=sd2t2ohlcv&h&e=csv`;
  try {
    const res = await fetch(url, {
      headers: { 'User-Agent': 'Mozilla/5.0' },
      signal: AbortSignal.timeout(10000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const text = await res.text();
    const lines = text.trim().split('\n');
    if (lines.length < 2) throw new Error('Empty CSV');
    const headers = lines[0].split(',').map(h => h.trim());
    const values = lines[1].split(',').map(v => v.trim());

    const obj = {};
    headers.forEach((h, i) => { obj[h] = values[i]; });

    if (obj['Close'] === 'N/D' || !obj['Close']) throw new Error('No data (N/D)');

    const rawPrice = parseFloat(obj['Close']);
    if (isNaN(rawPrice)) throw new Error('Invalid price');

    let price = rawPrice;
    if (unit === 'USD/t') {
      price = +(rawPrice / 100 * 2204.62).toFixed(2);
    }

    return {
      name,
      price,
      changePct: null,
      unit,
      exchange,
      source: 'Stooq',
      ccmnPrice: null,
      ccmnUpdown: null,
    };
  } catch (err) {
    process.stderr.write(`[fetch-prices] Stooq ${symbol} error: ${err.message}\n`);
    return {
      name,
      price: null,
      changePct: null,
      unit,
      exchange,
      source: 'Stooq',
      ccmnPrice: null,
      ccmnUpdown: null,
    };
  }
}

// ────────────────────────────────────────────
// 长江有色 CCMN API
// 提供 Cu/Zn/Ni/Co 人民币现货价 + 涨跌额
// ────────────────────────────────────────────
async function fetchCcmnPrices() {
  const url = 'https://m.ccmn.cn/mhangqing/getCorpStmarketPriceList?marketVmid=40288092327140f601327141c0560001';
  try {
    const res = await fetch(url, {
      headers: {
        'Referer': 'https://m.ccmn.cn/mhangqing/mcjxh/',
        'X-Requested-With': 'XMLHttpRequest',
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
      },
      signal: AbortSignal.timeout(8000),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const data = await res.json();
    if (!data.success) throw new Error(data.msg || 'API error');

    const list = data.body?.priceList;
    if (!Array.isArray(list)) throw new Error('No priceList in response');

    // productSortName -> key 映射
    const nameMap = {
      '1#铜': 'copper',
      '0#锌': 'zinc',
      '1#镍': 'nickel',
      '1#钴': 'cobalt',
    };

    const result = { copper: null, zinc: null, nickel: null, cobalt: null };

    for (const item of list) {
      const key = nameMap[item.productSortName];
      if (key) {
        const price = parseFloat(item.avgPrice);
        const updown = parseFloat(item.highsLowsAmount);
        result[key] = {
          price: isNaN(price) ? null : price,
          updown: isNaN(updown) ? null : updown,
        };
      }
    }

    return result;
  } catch (err) {
    process.stderr.write(`[fetch-prices] CCMN API error: ${err.message}\n`);
    return null;
  }
}

// ────────────────────────────────────────────
// 主函数
// ────────────────────────────────────────────
async function main() {
  // 1. 并行：Yahoo Finance (Cu/Zn USD) + CCMN (Cu/Zn/Ni/Co CNY)
  const [copperRaw, zincRaw, ccmn] = await Promise.all([
    fetchYahooPrice('HG=F', '铜', 'USD/lb', 'COMEX'),
    fetchYahooPrice('ZNC=F', '锌', 'USD/t',  'LME'),
    fetchCcmnPrices(),
  ]);

  // 2. Cu：Yahoo Finance 为主，CCMN 补充人民币数据
  const copper = {
    ...copperRaw,
    ccmnPrice: ccmn?.copper?.price ?? null,
    ccmnUpdown: ccmn?.copper?.updown ?? null,
  };

  // 3. Zn：Yahoo Finance 为主，CCMN 补充人民币数据
  const zinc = {
    ...zincRaw,
    ccmnPrice: ccmn?.zinc?.price ?? null,
    ccmnUpdown: ccmn?.zinc?.updown ?? null,
  };

  // 4. Ni：优先 CCMN（有涨跌数据），Stooq 作备用
  let nickel;
  if (ccmn?.nickel?.price != null) {
    nickel = {
      name: '镍',
      price: null,
      changePct: null,
      unit: 'CNY/t',
      exchange: '長江現貨',
      source: 'ccmn',
      ccmnPrice: ccmn.nickel.price,
      ccmnUpdown: ccmn.nickel.updown,
    };
  } else {
    process.stderr.write('[fetch-prices] CCMN 镍数据不可用，回退到 Stooq\n');
    const stooqNi = await fetchStooqPrice('NI.F', '镍', 'USD/t', 'LME');
    nickel = {
      ...stooqNi,
      ccmnPrice: null,
      ccmnUpdown: null,
    };
  }

  // 5. Co：优先 CCMN（唯一免费来源），失败则 null
  const cobalt = {
    name: '钴',
    price: null,
    changePct: null,
    unit: 'CNY/t',
    exchange: '長江現貨',
    source: ccmn?.cobalt?.price != null ? 'ccmn' : 'none',
    ccmnPrice: ccmn?.cobalt?.price ?? null,
    ccmnUpdown: ccmn?.cobalt?.updown ?? null,
  };

  // 6. Bi：暂无免费数据源
  const bismuth = {
    name: '铋',
    price: null,
    changePct: null,
    unit: 'USD/t',
    exchange: 'N/A',
    source: 'none',
    ccmnPrice: null,
    ccmnUpdown: null,
  };

  const results = [copper, zinc, nickel, cobalt, bismuth];
  console.log(JSON.stringify(results));
}

main().catch(err => {
  console.error('Fatal error:', err.message);
  process.exit(1);
});


path: scripts/lib/market-data-utils.mjs
content: function toNumber(value) {
  if (value == null) return null;
  const normalized = String(value).replace(/,/g, '').trim();
  if (!normalized) return null;
  const number = Number.parseFloat(normalized);
  return Number.isFinite(number) ? number : null;
}

function stripHtml(html) {
  return html
    .replace(/<script[\s\S]*?<\/script>/gi, ' ')
    .replace(/<style[\s\S]*?<\/style>/gi, ' ')
    .replace(/<[^>]+>/g, '\n')
    .replace(/&nbsp;/gi, ' ')
    .replace(/\r/g, '')
    .replace(/\u00a0/g, ' ')
    .replace(/[ \t]+/g, ' ')
    .replace(/\n+/g, '\n');
}

export function parseShfeInventoryHtml(html, metal) {
  const text = stripHtml(html);
  const latestMatch = text.match(/最新数据[\s\S]*?(\d{4}\s*W\d{2})[\s\S]*?([\d,]+(?:\.\d+)?)\s*[\r\n]+([\d,]+(?:\.\d+)?)/);
  if (!latestMatch) {
    throw new Error(`No weekly inventory block found for ${metal}`);
  }

  const weekLabel = latestMatch[1].replace(/\s+/g, ' ').trim();
  const tonnes = toNumber(latestMatch[2]);
  const previous = toNumber(latestMatch[3]);
  if (tonnes == null || previous == null) {
    throw new Error(`Invalid weekly inventory values for ${metal}`);
  }

  return {
    tonnes: Math.round(tonnes),
    change: Math.round(tonnes - previous),
    unit: 'tonnes',
    dataDate: weekLabel,
    weekLabel,
    source: 'SHFE/MacroMicro',
  };
}

export function buildInventorySnapshot(lmeInventory, shfeInventory) {
  return {
    copper: lmeInventory?.copper ?? null,
    zinc: lmeInventory?.zinc ?? null,
    nickel: lmeInventory?.nickel ?? null,
    cobalt: lmeInventory?.cobalt ?? null,
    note: lmeInventory?.note ?? null,
    lme: {
      copper: lmeInventory?.copper ?? null,
      zinc: lmeInventory?.zinc ?? null,
      nickel: lmeInventory?.nickel ?? null,
    },
    shfe: {
      copper: shfeInventory?.copper ?? null,
      zinc: shfeInventory?.zinc ?? null,
      nickel: shfeInventory?.nickel ?? null,
    },
  };
}

export function buildMagnesiumPrice(ccmnData, usdCny) {
  const magnesium = ccmnData?.magnesium;
  if (!magnesium) {
    return { cny: null, usdEstimate: null, source: null };
  }

  const fx = usdCny?.price;
  const usdEstimate = fx && fx > 0 ? Math.round(magnesium.price / fx) : null;

  return {
    cny: magnesium.price ?? null,
    cnyChange: magnesium.updown ?? null,
    cnyUnit: '元/吨',
    usdEstimate,
    usdUnit: 'USD/t',
    usdSource: usdEstimate != null ? 'CCMN/FX-estimate' : null,
    dataDate: ccmnData?.dataDate ?? null,
    source: 'CCMN',
  };
}


path: scripts/send-telegram.mjs
content: /**
 * send-telegram.mjs
 * 發送消息到 Telegram
 *
 * 用法：
 *   node send-telegram.mjs "消息文本"
 *   echo "消息文本" | node send-telegram.mjs
 *   node send-telegram.mjs < message.txt
 *
 * 從 .env 讀取 TELEGRAM_BOT_TOKEN 和 TELEGRAM_CHAT_ID
 */

import { readFileSync } from 'fs';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';
import { createInterface } from 'readline';

const __dirname = dirname(fileURLToPath(import.meta.url));
const PROJECT_ROOT = join(__dirname, '..');

// 讀取 .env
function loadEnv() {
  const envPath = join(PROJECT_ROOT, '.env');
  const env = {};
  try {
    const content = readFileSync(envPath, 'utf-8');
    for (const line of content.split('\n')) {
      const trimmed = line.trim();
      if (!trimmed || trimmed.startsWith('#')) continue;
      const eqIdx = trimmed.indexOf('=');
      if (eqIdx === -1) continue;
      const key = trimmed.slice(0, eqIdx).trim();
      const value = trimmed.slice(eqIdx + 1).trim().replace(/^["']|["']$/g, '');
      env[key] = value;
    }
  } catch (err) {
    process.stderr.write(`[send-telegram] 讀取 .env 失敗: ${err.message}\n`);
  }
  return env;
}

// 從 stdin 讀取所有輸入
async function readStdin() {
  return new Promise((resolve) => {
    const rl = createInterface({ input: process.stdin });
    const lines = [];
    rl.on('line', (line) => lines.push(line));
    rl.on('close', () => resolve(lines.join('\n')));
    // 如果 stdin 是 TTY（沒有 pipe），馬上 resolve 空字串
    if (process.stdin.isTTY) {
      rl.close();
    }
  });
}

// 發送 Telegram 消息
async function sendMessage(token, chatId, text) {
  const url = `https://api.telegram.org/bot${token}/sendMessage`;
  const res = await fetch(url, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      chat_id: chatId,
      text,
      parse_mode: 'Markdown',
    }),
    signal: AbortSignal.timeout(15000),
  });

  const data = await res.json();
  if (!res.ok || !data.ok) {
    throw new Error(data.description || `HTTP ${res.status}`);
  }
  return data;
}

async function main() {
  const env = loadEnv();
  const [REDACTED_SECRET];
  const chatId = env.TELEGRAM_CHAT_ID;

  if (!token || !chatId) {
    console.error('❌ 失敗: .env 缺少 TELEGRAM_BOT_TOKEN 或 TELEGRAM_CHAT_ID');
    process.exit(1);
  }

  // 優先從命令行參數讀取，否則從 stdin 讀取
  let message = process.argv[2] || '';
  if (!message) {
    message = await readStdin();
  }

  message = message.trim();
  if (!message) {
    console.error('❌ 失敗: 消息內容為空（請通過 argv[2] 或 stdin 提供）');
    process.exit(1);
  }

  try {
    await sendMessage(token, chatId, message);
    console.log('✅ 發送成功');
  } catch (err) {
    console.error(`❌ 失敗: ${err.message}`);
    process.exit(1);
  }
}

main().catch(err => {
  console.error(`❌ 失敗: ${err.message}`);
  process.exit(1);
});


path: scripts/test-sources.mjs
content: // Test alternative LME inventory data sources

async function testShfe() {
  const urls = [
    'https://www.shfe.com.cn/data/dailydata/WarehouseReceipt20260317.dat',
    'https://www.shfe.com.cn/data/dailydata/wr/wr20260317.dat',
    'https://datacenter.shfe.com.cn/statement/datatype/WareHouseReceipt//otc',
  ];
  for (const url of urls) {
    try {
      const r = await fetch(url, {
        headers: { 'User-Agent': 'Mozilla/5.0', 'Referer': 'https://www.shfe.com.cn/' },
        signal: AbortSignal.timeout(6000)
      });
      const body = (await r.text()).slice(0, 300);
      console.log('SHFE', url.split('/').pop().slice(0,40), ':', r.status, '|', body.slice(0,200));
    } catch(e) { console.log('SHFE err:', e.message); }
  }
}

async function testMacrotrends() {
  const r = await fetch('https://www.macrotrends.net/assets/php/fund_and_commodity_chart_data_download.php?t=HG00&type=price', {
    headers: { 'User-Agent': 'Mozilla/5.0', 'Referer': 'https://www.macrotrends.net/' },
    signal: AbortSignal.timeout(8000)
  });
  console.log('macrotrends:', r.status, (await r.text()).slice(0, 300));
}

async function testSmmInv() {
  const slugs = ['copper-stocks', 'lme-stocks', 'warehouse', 'cu-stocks', 'shfe-warehouse'];
  for (const slug of slugs) {
    try {
      const r = await fetch('https://hq.smm.cn/h5/' + slug, {
        headers: { 'User-Agent': 'Mozilla/5.0', 'Accept-Language': 'zh-CN,zh;q=0.9' },
        signal: AbortSignal.timeout(4000)
      });
      console.log('SMM slug', slug, ':', r.status);
    } catch(e) { console.log('SMM slug err', slug, ':', e.message); }
  }
}

// Try LME with more browser-like headers
async function testLmeDirect() {
  try {
    const r = await fetch('https://www.lme.com/api/Reports/WarehouseStockByMetalReportDownload?fileName=&isInternal=false', {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36',
        'Accept': 'application/json, text/plain, */*',
        'Accept-Language': 'en-US,en;q=0.9',
        'Accept-Encoding': 'gzip, deflate, br',
        'Referer': 'https://www.lme.com/Market-Data/Reports-and-data/Warehouse-Stock-Statistics',
        'sec-ch-ua': '"Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"',
        'sec-ch-ua-mobile': '?0',
        'sec-ch-ua-platform': '"Windows"',
        'Sec-Fetch-Dest': 'empty',
        'Sec-Fetch-Mode': 'cors',
        'Sec-Fetch-Site': 'same-origin',
      },
      signal: AbortSignal.timeout(8000)
    });
    const ct = r.headers.get('content-type') || '';
    console.log('LME direct:', r.status, ct, r.headers.get('cf-ray') ? '(Cloudflare)' : '');
    if (r.status === 200) {
      const body = await r.text();
      console.log('LME body:', body.slice(0, 400));
    }
  } catch(e) { console.log('LME err:', e.message); }
}

// Test worldbank commodities
async function testWorldBank() {
  const r = await fetch('https://api.worldbank.org/v2/en/indicator/PCOPP.USD?downloadformat=json&mrv=5', {
    headers: { 'User-Agent': 'Mozilla/5.0' },
    signal: AbortSignal.timeout(8000)
  });
  const ct = r.headers.get('content-type') || '';
  const body = ct.includes('json') ? JSON.stringify(await r.json()).slice(0, 400) : (await r.text()).slice(0, 200);
  console.log('WorldBank copper:', r.status, body.slice(0, 300));
}

await Promise.all([testShfe(), testMacrotrends(), testSmmInv(), testLmeDirect(), testWorldBank()]);
console.log('Done');


path: scripts/test-sources2.mjs
content: // Test Chinese financial data aggregators for LME inventory

// 1. 金十数据 (jin10.com) - often has LME inventory
async function testJin10() {
  const urls = [
    'https://rong360.jin10.com/api/flash_newest?category=0&channel=-1&vip=0',
    'https://flash-api.jin10.com/get_flash_by_category?category=15&count=20&vip=0',
    'https://datacenter.jin10.com/reportType/dc_lme_inventory',
    'https://datacenter.jin10.com/reportType/dc_copper_inventory',
  ];
  for (const url of urls) {
    try {
      const r = await fetch(url, {
        headers: { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36', 'Accept': 'application/json', 'Referer': 'https://www.jin10.com/' },
        signal: AbortSignal.timeout(6000)
      });
      const ct = r.headers.get('content-type') || '';
      const body = ct.includes('json') ? JSON.stringify(await r.json()).slice(0, 400) : (await r.text()).slice(0, 200);
      console.log('Jin10', url.split('/').pop(), ':', r.status, '|', body.slice(0, 200));
    } catch(e) { console.log('Jin10 err', url.split('/').pop(), ':', e.message); }
  }
}

// 2. 东方财富 (eastmoney.com) - comprehensive financial data
async function testEastmoney() {
  const urls = [
    'https://datacenter-web.eastmoney.com/api/data/v1/get?reportName=RPT_FUTU_LME_INVENTORY&columns=ALL&pageSize=10&sortColumns=UPDATE_DATE&sortTypes=-1',
    'https://datacenter-web.eastmoney.com/api/data/v1/get?reportName=RPT_FUTU_METAL_INVENTORY&columns=ALL&pageSize=10',
  ];
  for (const url of urls) {
    try {
      const r = await fetch(url, {
        headers: { 'User-Agent': 'Mozilla/5.0', 'Referer': 'https://data.eastmoney.com/', 'Accept': 'application/json' },
        signal: AbortSignal.timeout(8000)
      });
      const ct = r.headers.get('content-type') || '';
      const body = ct.includes('json') ? JSON.stringify(await r.json()).slice(0, 500) : (await r.text()).slice(0, 200);
      console.log('Eastmoney', url.split('reportName=')[1]?.split('&')[0] || url.split('/').pop(), ':', r.status, '|', body.slice(0, 300));
    } catch(e) { console.log('Eastmoney err:', e.message); }
  }
}

// 3. 同花顺 iFinD
async function testThs() {
  const urls = [
    'https://d.10jqka.com.cn/v2/future/hs_lme_inventory/block/json',
    'https://data.10jqka.com.cn/futures/lme_inventory/',
    'https://d.10jqka.com.cn/v2/report/hs_lme_copper/json',
  ];
  for (const url of urls) {
    try {
      const r = await fetch(url, {
        headers: { 'User-Agent': 'Mozilla/5.0', 'Referer': 'https://www.10jqka.com.cn/' },
        signal: AbortSignal.timeout(6000)
      });
      const ct = r.headers.get('content-type') || '';
      const body = ct.includes('json') ? JSON.stringify(await r.json()).slice(0, 300) : (await r.text()).slice(0, 200);
      console.log('THS', url.split('/').pop(), ':', r.status, '|', body.slice(0, 200));
    } catch(e) { console.log('THS err', url.split('/').pop(), ':', e.message); }
  }
}

// 4. CME Group COMEX copper inventory
async function testCme() {
  const urls = [
    'https://www.cmegroup.com/CmeWS/mvc/Settlements/futures/options/tradeDate/20260314/productCode/HG/type/ALL/code/ALL',
    'https://www.cmegroup.com/CmeWS/mvc/Volume/getCombinedVolumeDownloadDetails/tradeDate/20260314/asset/copper.csv',
    'https://www.cmegroup.com/CmeWS/mvc/Warehouse/getCopperWarehouseStocks.json',
    'https://www.cmegroup.com/market-data/reports/warehouse-stock-reports.html',
  ];
  for (const url of urls) {
    try {
      const r = await fetch(url, {
        headers: { 'User-Agent': 'Mozilla/5.0', 'Accept': '*/*' },
        signal: AbortSignal.timeout(6000)
      });
      const ct = r.headers.get('content-type') || '';
      const body = ct.includes('json') ? JSON.stringify(await r.json()).slice(0, 300) : (await r.text()).slice(0, 200);
      console.log('CME', url.split('/').pop(), ':', r.status, '|', body.slice(0, 150));
    } catch(e) { console.log('CME err', url.split('/').pop(), ':', e.message); }
  }
}

// 5. Try westmetall.com (German metals data site)
async function testWestmetall() {
  try {
    const r = await fetch('https://www.westmetall.com/en/markdaten.php?action=table&field=LME_Cu_cash', {
      headers: { 'User-Agent': 'Mozilla/5.0', 'Accept': 'text/html' },
      signal: AbortSignal.timeout(8000)
    });
    const html = await r.text();
    // Look for inventory/stock keywords
    const hasStock = html.toLowerCase().includes('stock') || html.toLowerCase().includes('inventory') || html.toLowerCase().includes('tonne');
    console.log('Westmetall:', r.status, 'hasStock:', hasStock, html.slice(0, 200));
  } catch(e) { console.log('Westmetall err:', e.message); }
}

await Promise.all([testJin10(), testEastmoney(), testThs(), testCme(), testWestmetall()]);
console.log('Done');


path: scripts/test-sources3.mjs
content: // Deep investigation of promising sources

// 1. Westmetall - returns 200 with stock keywords
async function testWestmetallDeep() {
  try {
    const r = await fetch('https://www.westmetall.com/en/markdaten.php?action=table&field=LME_Cu_cash', {
      headers: { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36', 'Accept': 'text/html,*/*', 'Accept-Language': 'en-US,en;q=0.9' },
      signal: AbortSignal.timeout(10000)
    });
    const html = await r.text();
    console.log('=== WESTMETALL full HTML length:', html.length);
    // Find all "stock" occurrences
    const lower = html.toLowerCase();
    let pos = 0;
    while ((pos = lower.indexOf('stock', pos)) !== -1) {
      console.log('  "stock" at', pos, ':', html.slice(Math.max(0, pos-50), pos+100));
      pos += 5;
    }
    // Find tonnage patterns
    const tonneMatches = html.match(/[\d,]+\s*(tonne|ton|mt)/gi);
    if (tonneMatches) console.log('  Tonne patterns:', tonneMatches.slice(0, 5));
    // Show first table or data block
    const tableIdx = html.indexOf('<table');
    if (tableIdx > -1) console.log('  First table:', html.slice(tableIdx, tableIdx + 500));
  } catch(e) { console.log('Westmetall deep err:', e.message); }
}

// 2. Jin10 datacenter LME inventory
async function testJin10Deep() {
  const urls = [
    'https://datacenter.jin10.com/reportType/dc_lme_inventory',
    'https://datacenter.jin10.com/reportType/dc_copper_inventory',
    'https://datacenter.jin10.com/v2/lme/inventory/latest',
    'https://datacenter.jin10.com/v3/lme/inventory',
  ];
  for (const url of urls) {
    try {
      const r = await fetch(url, {
        headers: {
          'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
          'Referer': 'https://datacenter.jin10.com/',
          'Accept': 'application/json, text/plain, */*',
          'Accept-Language': 'zh-CN,zh;q=0.9',
          'x-app-id': 'rU6QIu7JHe2gOUeR',
          'x-version': '1.0.0',
        },
        signal: AbortSignal.timeout(8000)
      });
      const ct = r.headers.get('content-type') || '';
      const body = ct.includes('json') ? JSON.stringify(await r.json()).slice(0, 500) : (await r.text()).slice(0, 500);
      console.log('Jin10 deep', url.split('/').pop(), ':', r.status, ct.slice(0,30), '|', body.slice(0, 300));
    } catch(e) { console.log('Jin10 deep err:', url.split('/').pop(), e.message); }
  }
}

// 3. 东方财富 - try correct report name for LME inventory
async function testEastmoneyInv() {
  // Try to discover actual report names by browsing the futures data center
  const urls = [
    'https://datacenter-web.eastmoney.com/api/data/v1/get?reportName=RPT_LME_INVENTORY&columns=ALL&pageSize=5',
    'https://datacenter-web.eastmoney.com/api/data/v1/get?reportName=RPT_FUTURES_LME_INVENTORY&columns=ALL&pageSize=5',
    'https://futurold.eastmoney.com/web/api/lme/inventory?page=1&pagesize=5',
    // Try futures page
    'https://datacenter-web.eastmoney.com/api/data/v1/get?reportName=RPT_FUTU_POSITIONS&columns=ALL&pageSize=5&sortColumns=DATE&sortTypes=-1',
  ];
  for (const url of urls) {
    try {
      const r = await fetch(url, {
        headers: { 'User-Agent': 'Mozilla/5.0', 'Referer': 'https://data.eastmoney.com/futures/', 'Accept': 'application/json' },
        signal: AbortSignal.timeout(8000)
      });
      const ct = r.headers.get('content-type') || '';
      const body = ct.includes('json') ? JSON.stringify(await r.json()).slice(0, 400) : (await r.text()).slice(0, 200);
      console.log('EM', url.split('reportName=')[1]?.split('&')[0] || url.split('/').pop(), ':', r.status, '|', body.slice(0, 250));
    } catch(e) { console.log('EM err:', e.message); }
  }
}

// 4. Try the actual LME warehouse stats page HTML for warehouse data scraping
async function testLmeHtml() {
  // The LME publishes CSV files too - maybe those aren't CF-protected
  const urls = [
    'https://www.lme.com/api/Graphs/LMEStockData',
    'https://api.lme.com/warehouse/stock',
    'https://www.lme.com/en-GB/Trading/Physical-market/Warehousing/LME-stocks',
  ];
  for (const url of urls) {
    try {
      const r = await fetch(url, {
        headers: {
          'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
          'Accept': '*/*',
          'Referer': 'https://www.lme.com/',
        },
        signal: AbortSignal.timeout(6000)
      });
      const ct = r.headers.get('content-type') || '';
      const cfRay = r.headers.get('cf-ray');
      console.log('LME', url.split('/').pop(), ':', r.status, ct.slice(0,30), cfRay ? '(CF)' : '');
      if (r.status === 200) {
        console.log('  Body:', (await r.text()).slice(0, 400));
      }
    } catch(e) { console.log('LME err:', url.split('/').pop(), e.message); }
  }
}

await Promise.all([testWestmetallDeep(), testJin10Deep(), testEastmoneyInv(), testLmeHtml()]);
console.log('Done');


path: scripts/test-westmetall.mjs
content: // Test Westmetall for all metals LME inventory
// Confirmed: LME_Cu_cash gives copper stock data

const metals = [
  { field: 'LME_Cu_cash', name: 'copper' },
  { field: 'LME_Al_cash', name: 'aluminum' },
  { field: 'LME_Ni_cash', name: 'nickel' },
  { field: 'LME_Zn_cash', name: 'zinc' },
  { field: 'LME_Pb_cash', name: 'lead' },
  { field: 'LME_Sn_cash', name: 'tin' },
  { field: 'LME_Co_cash', name: 'cobalt' },
];

async function fetchWestmetall(field, name) {
  const url = `https://www.westmetall.com/en/markdaten.php?action=table&field=${field}`;
  try {
    const r = await fetch(url, {
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36',
        'Accept': 'text/html,application/xhtml+xml,*/*',
        'Accept-Language': 'en-US,en;q=0.9',
        'Referer': 'https://www.westmetall.com/en/markdaten.php',
      },
      signal: AbortSignal.timeout(10000)
    });
    
    if (!r.ok) throw new Error(`HTTP ${r.status}`);
    const html = await r.text();
    
    // Check if page has stock column
    if (!html.includes('stock') && !html.includes('Stock')) {
      console.log(`${name}: no stock column`);
      return null;
    }
    
    // Parse first data row: looking for pattern
    // <td >16. March 2026</td><td >xxx</td><td >xxx</td><td class="last">311,600</td>
    const firstRowMatch = html.match(/<tbody>\s*<tr>\s*<td[^>]*>([^<]+)<\/td>((?:<td[^>]*>[^<]*<\/td>)*)<td[^>]*class="[^"]*last[^"]*"[^>]*>([^<]+)<\/td>/);
    if (firstRowMatch) {
      const date = firstRowMatch[1].trim();
      const stock = parseInt(firstRowMatch[3].replace(/[,\s]/g, ''), 10);
      console.log(`${name} (${field}): date=${date}, stock=${isNaN(stock) ? 'N/A' : stock + ' tonnes'}`);
      
      // Show the header to understand columns
      const headerMatch = html.match(/<thead>([\s\S]*?)<\/thead>/);
      if (headerMatch) {
        const headers = headerMatch[1].match(/<th[^>]*>([^<]+)<\/th>/g)?.map(h => h.replace(/<[^>]+>/g, '').trim());
        console.log(`  Headers:`, headers);
      }
      return { date, stock: isNaN(stock) ? null : stock };
    } else {
      // Try simpler pattern
      const lastTdMatch = html.match(/<td class="last">([^<]+)<\/td>/);
      if (lastTdMatch) {
        const stock = parseInt(lastTdMatch[1].replace(/[,\s]/g, ''), 10);
        console.log(`${name}: stock (simple)=${isNaN(stock) ? 'N/A' : stock}`);
      } else {
        console.log(`${name}: could not parse`, html.slice(html.indexOf('<tbody>'), html.indexOf('<tbody>') + 300));
      }
    }
    return null;
  } catch(e) {
    console.log(`${name} error:`, e.message);
    return null;
  }
}

for (const { field, name } of metals) {
  await fetchWestmetall(field, name);
}
console.log('Done');


path: tests/inventory-and-report.test.mjs
content: import test from 'node:test';
import assert from 'node:assert/strict';

import {
  buildInventorySnapshot,
  buildMagnesiumPrice,
  parseShfeInventoryHtml,
} from '../scripts/lib/market-data-utils.mjs';
import { buildReport } from '../scripts/lib/report-builder.mjs';

test('parseShfeInventoryHtml extracts latest weekly copper inventory from MacroMicro HTML', () => {
  const html = `
    <html><body>
      <h1>SHFE-铜库存量</h1>
      <div>#### 最新数据</div>
      <div>SHFE-铜库存量</div>
      <div>2026 W06</div>
      <div>248,911.00</div>
      <div>233,004.00</div>
      <div>资料来源</div>
    </body></html>
  `;

  assert.deepEqual(parseShfeInventoryHtml(html, 'copper'), {
    tonnes: 248911,
    change: 15907,
    unit: 'tonnes',
    dataDate: '2026 W06',
    weekLabel: '2026 W06',
    source: 'SHFE/MacroMicro',
  });
});

test('buildInventorySnapshot preserves legacy LME fields and adds SHFE branch', () => {
  const lme = {
    copper: { tonnes: 123000, change: -1200, unit: 'tonnes', dataDate: '2026-04-03', source: 'Westmetall/LME_Cu_stock' },
    zinc: null,
    nickel: { tonnes: 45600, change: 100, unit: 'tonnes', dataDate: '2026-04-03', source: 'Westmetall/LME_Ni_stock' },
    cobalt: null,
    note: null,
  };
  const shfe = {
    copper: { tonnes: 248911, change: 15907, unit: 'tonnes', dataDate: '2026 W06', weekLabel: '2026 W06', source: 'SHFE/MacroMicro' },
    zinc: null,
    nickel: { tonnes: 57457, change: 2061, unit: 'tonnes', dataDate: '2026 W06', weekLabel: '2026 W06', source: 'SHFE/MacroMicro' },
  };

  const snapshot = buildInventorySnapshot(lme, shfe);

  assert.equal(snapshot.copper.tonnes, 123000);
  assert.equal(snapshot.nickel.tonnes, 45600);
  assert.deepEqual(snapshot.lme.copper, lme.copper);
  assert.deepEqual(snapshot.shfe.copper, shfe.copper);
  assert.equal(snapshot.shfe.zinc, null);
});

test('buildMagnesiumPrice adds USD estimate only when CCMN and FX data exist', () => {
  assert.deepEqual(
    buildMagnesiumPrice(
      { magnesium: { price: 15400, updown: 150 }, dataDate: '2026-04-07' },
      { price: 7.2, source: 'Yahoo Finance', symbol: 'USDCNY=X' },
    ),
    {
      cny: 15400,
      cnyChange: 150,
      cnyUnit: '元/吨',
      usdEstimate: 2139,
      usdUnit: 'USD/t',
      usdSource: 'CCMN/FX-estimate',
      dataDate: '2026-04-07',
      source: 'CCMN',
    },
  );

  assert.deepEqual(
    buildMagnesiumPrice(
      { magnesium: { price: 15400, updown: 150 }, dataDate: '2026-04-07' },
      null,
    ),
    {
      cny: 15400,
      cnyChange: 150,
      cnyUnit: '元/吨',
      usdEstimate: null,
      usdUnit: 'USD/t',
      usdSource: null,
      dataDate: '2026-04-07',
      source: 'CCMN',
    },
  );
});

test('buildReport labels LME and SHFE inventory separately and shows magnesium USD estimate', () => {
  const report = buildReport({
    date: '2026-04-07',
    prices: {
      copper: { usd: 4.2, usdChangePct: 0.5, cny: 76000, cnyChange: 100 },
      zinc: { usd: 2500, usdChangePct: null, cny: 22000, cnyChange: -50 },
      nickel: { usd: 16000, usdChangePct: null, cny: 126000, cnyChange: 200 },
      cobalt: { usd: null, cny: 220000, cnyChange: 0 },
      bismuth: { usd: 15600, cny: 163000, cnyChange: 0, cnyChangePct: 0 },
      magnesium: {
        cny: 15400,
        cnyChange: 150,
        cnyUnit: '元/吨',
        usdEstimate: 2139,
        usdUnit: 'USD/t',
        usdSource: 'CCMN/FX-estimate',
        dataDate: '2026-04-07',
        source: 'CCMN',
      },
    },
    forwards: { copper: null },
    indices: [],
    inventory: {
      copper: { tonnes: 123000, change: -1200, unit: 'tonnes', dataDate: '2026-04-03', source: 'Westmetall/LME_Cu_stock' },
      zinc: null,
      nickel: null,
      cobalt: null,
      note: null,
      lme: {
        copper: { tonnes: 123000, change: -1200, unit: 'tonnes', dataDate: '2026-04-03', source: 'Westmetall/LME_Cu_stock' },
        zinc: null,
        nickel: null,
      },
      shfe: {
        copper: { tonnes: 248911, change: 15907, unit: 'tonnes', dataDate: '2026 W06', weekLabel: '2026 W06', source: 'SHFE/MacroMicro' },
        zinc: null,
        nickel: null,
      },
    },
    news: [],
    ibNews: [],
    forumSentiment: {},
  });

  assert.match(report, /SHFE 周庫存/);
  assert.match(report, /LME：123,000 t/);
  assert.match(report, /SHFE：248,911 t/);
  assert.match(report, /USD est\. 2,139\/t \[CCMN\/FX-estimate\]/);
});


Security scan verdicts — ClawScan: clean | static analysis: suspicious | VirusTotal: clean | SkillSpector: clean (LOW)