Initial commit

.gitattributes

@@ -1,35 +1,2 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text

README.md

@@ -9,4 +9,150 @@ base_model:
 pipeline_tag: text-classification
 tags:
 - agent
----
+---
+
+# vad-macbert
+
+Chinese VAD (valence/arousal/dominance) regression based on `hfl/chinese-macbert-base`.
+The model predicts 3 continuous values aligned to the VAD scale produced by
+`RobroKools/vad-bert` (teacher model).
+
+## Quickstart
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+
+model_path = "Pectics/vad-macbert"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForSequenceClassification.from_pretrained(model_path)
+model.eval()
+
+text = "这部电影让我很感动。"
+inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=128)
+with torch.no_grad():
+    outputs = model(**inputs)
+vad = outputs.logits.squeeze().tolist()
+print("VAD:", vad)
+```
+
+## Model Details
+
+- Base model: `hfl/chinese-macbert-base`
+- Task: VAD regression (3 outputs: valence, arousal, dominance)
+- Head: `AutoModelForSequenceClassification` with `num_labels=3`, `problem_type=regression`
+
+## Data Sources & Labeling
+
+### en-zh_cn_vad_clean.csv
+- Source: OpenSubtitles EN-ZH parallel corpus.
+- Labeling: English side fed into `RobroKools/vad-bert` to obtain VAD values,
+  then assigned to the paired Chinese text.
+
+### en-zh_cn_vad_long.csv
+- Derived from `en-zh_cn_vad_clean.csv` by filtering for longer texts using a
+  length threshold (original threshold was not recorded).
+- Inferred from statistics: minimum length is 32 characters, so the filter
+  likely kept samples with length >= 32 chars.
+
+### en-zh_cn_vad_long_clean.csv
+- Cleaned from `en-zh_cn_vad_long.csv` by removing subtitle formatting noise:
+  - ASS/SSA tag blocks like `{\\fs..\\pos(..)}` (including broken `{` blocks)
+  - HTML-like tags (e.g. `<i>...</i>`)
+  - Escape codes like `\\N`, `\\n`, `\\h`, `\\t`
+  - Extra whitespace normalization
+- Non-CJK rows were dropped.
+
+### en-zh_cn_vad_mix.csv
+- Mixed dataset created for replay training:
+  - 200k samples from `en-zh_cn_vad_clean.csv`
+  - 200k samples from `en-zh_cn_vad_long_clean.csv`
+  - Shuffled after sampling
+
+## Training Summary
+
+The final model (`vad-macbert-mix/best`) was obtained in three stages:
+
+1. **Base training** on `en-zh_cn_vad_clean.csv`
+2. **Long-text adaptation** on `en-zh_cn_vad_long_clean.csv`
+3. **Replay mix** on `en-zh_cn_vad_mix.csv` (resume from stage 2)
+
+### Final-stage Command (Replay Mix)
+
+```
+--model_name hfl/chinese-macbert-base
+--output_dir train/vad-macbert-mix
+--data_path train/en-zh_cn_vad_mix.csv
+--epochs 4
+--batch_size 32
+--grad_accum_steps 4
+--learning_rate 0.00001
+--weight_decay 0.01
+--warmup_ratio 0.1
+--warmup_steps 0
+--max_length 512
+--eval_ratio 0.01
+--eval_every 100
+--eval_batches 200
+--loss huber
+--huber_delta 1.0
+--shuffle_buffer 4096
+--min_chars 2
+--save_every 100
+--log_every 1
+--max_steps 5000
+--seed 42
+--dtype fp16
+--num_rows 400000
+--resume_from train/vad-macbert-long/best
+--encoding utf-8
+```
+
+Training environment (conda `llm`):
+
+- Python 3.10.19
+- torch 2.9.1+cu130
+- transformers 4.57.6
+
+## Evaluation
+
+Benchmark script: `train/vad_benchmark.py`
+
+- Evaluation uses a fixed stride derived from `eval_ratio=0.01`
+  (roughly 1 out of 100 samples).
+- Length buckets by character count: 0–20, 20–40, 40–80, 80–120, 120–200,
+  200–400, 400+
+
+### Results (vad-macbert-mix/best)
+
+**en-zh_cn_vad_clean.csv**
+
+- mse_mean=0.043734
+- mae_mean=0.149322
+- pearson_mean=0.7335
+
+**en-zh_cn_vad_long_clean.csv**
+
+- mse_mean=0.031895
+- mae_mean=0.131320
+- pearson_mean=0.7565
+
+Notes:
+- `400+` bucket Pearson is unstable due to small sample size; interpret with care.
+
+## Limitations
+
+- Labels are derived from an English VAD teacher and transferred via parallel
+  alignment, so they reflect the teacher’s bias and may not match human Chinese
+  annotations.
+- Subtitle corpora include translation artifacts and formatting noise; cleaned
+  versions mitigate but do not fully remove this.
+- Extreme-length sentences are under-represented; performance on 400+ chars
+  is not reliable.
+
+## Files in This Repo
+
+- `config.json`
+- `model.safetensors`
+- `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`, `vocab.txt`
+- `training_args.json`

README_zh.md

@@ -0,0 +1,135 @@
+# vad-macbert
+
+基于 `hfl/chinese-macbert-base` 的中文 VAD（valence/arousal/dominance）回归模型。
+输出 3 个连续值，目标对齐到教师模型 `RobroKools/vad-bert` 的 VAD 空间。
+
+## 快速上手
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+
+model_path = "Pectics/vad-macbert"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForSequenceClassification.from_pretrained(model_path)
+model.eval()
+
+text = "这部电影让我很感动。"
+inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=128)
+with torch.no_grad():
+    outputs = model(**inputs)
+vad = outputs.logits.squeeze().tolist()
+print("VAD:", vad)
+```
+
+## 模型信息
+
+- 基座模型：`hfl/chinese-macbert-base`
+- 任务：VAD 回归（3 维输出：valence, arousal, dominance）
+- 头部：`AutoModelForSequenceClassification`，`num_labels=3`，`problem_type=regression`
+
+## 数据来源与标注方式
+
+### en-zh_cn_vad_clean.csv
+- 来源：OpenSubtitles 英中平行语料。
+- 标注：将英文句子输入 `RobroKools/vad-bert` 获取 VAD，再把该 VAD 赋给对应中文句子。
+
+### en-zh_cn_vad_long.csv
+- 由 `en-zh_cn_vad_clean.csv` 过滤长句得到（原始阈值未记录）。
+- 根据长度统计推断最小长度为 32 字符，推测当时过滤条件为 `len >= 32`。
+
+### en-zh_cn_vad_long_clean.csv
+- 从 `en-zh_cn_vad_long.csv` 清洗得到，去掉字幕样式噪声：
+  - ASS/SSA 标签块（如 `{\\fs..\\pos(..)}`，含不完整 `{`）
+  - HTML 类标签（如 `<i>...</i>`）
+  - 转义标记（`\\N`、`\\n`、`\\h`、`\\t`）
+  - 多余空白归一化
+- 非 CJK 内容已过滤。
+
+### en-zh_cn_vad_mix.csv
+- 回放混合数据：
+  - `en-zh_cn_vad_clean.csv` 抽样 200k
+  - `en-zh_cn_vad_long_clean.csv` 抽样 200k
+  - 合并后再随机打乱
+
+## 训练过程
+
+最终模型 `vad-macbert-mix/best` 由三阶段训练获得：
+
+1. **基础训练**：`en-zh_cn_vad_clean.csv`
+2. **长句适配**：`en-zh_cn_vad_long_clean.csv`
+3. **回放混合**：`en-zh_cn_vad_mix.csv`（从阶段 2 继续训练）
+
+### 最终阶段命令（回放混合）
+
+```
+--model_name hfl/chinese-macbert-base
+--output_dir train/vad-macbert-mix
+--data_path train/en-zh_cn_vad_mix.csv
+--epochs 4
+--batch_size 32
+--grad_accum_steps 4
+--learning_rate 0.00001
+--weight_decay 0.01
+--warmup_ratio 0.1
+--warmup_steps 0
+--max_length 512
+--eval_ratio 0.01
+--eval_every 100
+--eval_batches 200
+--loss huber
+--huber_delta 1.0
+--shuffle_buffer 4096
+--min_chars 2
+--save_every 100
+--log_every 1
+--max_steps 5000
+--seed 42
+--dtype fp16
+--num_rows 400000
+--resume_from train/vad-macbert-long/best
+--encoding utf-8
+```
+
+训练环境（conda `llm`）：
+
+- Python 3.10.19
+- torch 2.9.1+cu130
+- transformers 4.57.6
+
+## 评测
+
+基准脚本：`train/vad_benchmark.py`
+
+- 使用 `eval_ratio=0.01`（约 1/100 抽样）。
+- 长度分桶（字符数）：0–20、20–40、40–80、80–120、120–200、200–400、400+
+
+### 结果（vad-macbert-mix/best）
+
+**en-zh_cn_vad_clean.csv**
+
+- mse_mean=0.043734
+- mae_mean=0.149322
+- pearson_mean=0.7335
+
+**en-zh_cn_vad_long_clean.csv**
+
+- mse_mean=0.031895
+- mae_mean=0.131320
+- pearson_mean=0.7565
+
+备注：
+- `400+` 分桶样本量很少，Pearson 不稳定，仅供参考。
+
+## 限制与注意事项
+
+- VAD 标签来自英文教师模型并通过平行语料对齐，可能带有教师偏差，不等同于人工中文标注。
+- 字幕语料存在翻译误差和格式噪声，清洗后仍可能残留。
+- 超长句样本较少，`400+` 的表现不稳定。
+
+## 目录文件
+
+- `config.json`
+- `model.safetensors`
+- `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`, `vocab.txt`
+- `training_args.json`

config.json

@@ -0,0 +1,42 @@
+{
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "directionality": "bidi",
+  "dtype": "float32",
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_2": 2
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "pooler_fc_size": 768,
+  "pooler_num_attention_heads": 12,
+  "pooler_num_fc_layers": 3,
+  "pooler_size_per_head": 128,
+  "pooler_type": "first_token_transform",
+  "position_embedding_type": "absolute",
+  "problem_type": "regression",
+  "transformers_version": "4.57.6",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 21128
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da4ec84f28cf329c6779d008fd023df99d08fe24c9340c1bb16229d7fb0fe9a0
+size 409103316

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,56 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

training_args.json

@@ -0,0 +1,32 @@
+{
+  "batch_size": 32,
+  "data_path": "train/en-zh_cn_vad_mix.csv",
+  "dtype": "fp16",
+  "encoding": "utf-8",
+  "epochs": 4,
+  "errors": "ignore",
+  "eval_batch_size": 0,
+  "eval_batches": 200,
+  "eval_every": 100,
+  "eval_ratio": 0.01,
+  "grad_accum_steps": 4,
+  "huber_delta": 1.0,
+  "learning_rate": 1e-05,
+  "log_every": 1,
+  "loss": "huber",
+  "max_length": 512,
+  "max_rows": null,
+  "max_steps": 5000,
+  "min_chars": 2,
+  "model_name": "hfl/chinese-macbert-base",
+  "num_labels": 3,
+  "num_rows": 400000,
+  "output_dir": "train/vad-macbert-mix",
+  "resume_from": "train/vad-macbert-long/best",
+  "save_every": 100,
+  "seed": 42,
+  "shuffle_buffer": 4096,
+  "warmup_ratio": 0.1,
+  "warmup_steps": 0,
+  "weight_decay": 0.01
+}