S1-mini-Chinese-1106-BF16
ไธ้จไธบ Qwen3-ASR ไธญๆๅไธญ่ฑๆททๅ่ฝฌๅไผๅ็ๆๆฌๅๅค็ๆจกๅใ
ๆฌ็ๆฌๅจไธญๆๆฐๆฎๅๅๆๅฏน็
งๆฐๆฎไธ็ปง็ปญๅ
จ้ๅพฎ่ฐ๏ผ้็นๆนๅ BF16 ไธ็ๆฐๅญ่ฝฌๅ็จณๅฎๆงใ
ๅบไบ S1-mini by Superwhisper๏ผไฟ็ๅๅ็งฐใ็ฝฒๅใLICENSE ๅ NOTICEใ
่ฟๆฏๆฐ็ stability-margin-r2 ๆ้๏ผไธๆฏๆญคๅ S1-mini-Chinese-1106๏ผnative-v10๏ผ็ๆ ผๅผๆนๅใ
Chinese-data continued full fine-tuning of S1-mini by Superwhisper, optimized for Qwen3-ASR Chinese and Chinese/English mixed transcripts. This BF16 stability release is a new checkpoint. It processes text, not audio, and is not a general-purpose assistant.
ๆนๅจ
- ๅขๅ ๆฐๅผ็ปๅใๅ ้จ้ถใ่ฟไฝๅๆถ้ดๅๆณ็่ฎญ็ปใ
- ๅขๅ ๆฐๅญๅๆผ่ฏปๅญ็ฌฆ็็ฎๆ ๅๆฐ้ด้่ฎญ็ป๏ผ้ไฝๅ้ๅๆฐ่ฟ่ฟๆถ็้่ฏฏใ
- ไฟ็ๆจกๅ่ช่บซ็ๅปๅฃๆฐด่ฏใๅป้ๅคใๆ็กฎๆนๅฃใไนฆ้ขๅคงๅฐๅใไธญๆไบบๅใ็ผๅทๅ่กจๅๆข่ก่ฝๅใ
- ๅบ็กๆฐๅญฆไฝฟ็จๆฎ้ๆๆฌ
+ - * / ^ =๏ผไธไธปๅจ่ฎก็ฎๆชๅฃ่ฟฐ็ญๆก๏ผไนไธ่ฆๆฑ LaTeXใ - ไธ้ๅธฆๆผ่ฏปๅ็ฝฎ่งๅๆๅ ๅฎนไฟๆค่ฟๆปคๅจใๆจกๅ่พๅบ็ดๆฅไบค็ป่ฐ็จๆนใ
่ฟไบๆฏ่ฎญ็ป็ฎๆ ๏ผไธๆฏไปปๆ่พๅ ฅไธ็ๆญฃ็กฎๆงไฟ่ฏใ่ฐ็จ็จๅบ่ด่ดฃ่ฏทๆฑๅผๅธธใๆชๆญๆฃๆตๅๅฎๅ จๆไบคๆข่กใ
่ฎญ็ป
ๅ จ้่ฎญ็ป 751,632,384 ไธชๅๆฐ๏ผไธๆฏ LoRA๏ผไนไธๆฏไป ่ฎญ็ปๆซๅฑใ
- ๅจ native-v10 ไธไฝฟ็จ 5,109 ๆกๅๆพไธๅฏน็ งๆฐๆฎ๏ผ320 ๆญฅใๅญฆไน ็ 5e-7ใ
- ๅๆฐๆฎ/็งๅญ/ๆญฅๆฐๅฆๅ FP32 ๆๅธ่ธ้ฆๅฏน็ งใไธค็ปๆฐๅผๅ้ๅๅ๏ผๆฎ้็็ฃ็ปๅ ฌๅผๅๅฝๆดๅฅฝ๏ผๅ ๆญค้ๆฎ้็็ฃ็ปใ
- ๅๅ ๅ ฅ 2,048 ๆก้็จๆฐๅญ็ปๅไธ่ฟไฝๅฏน็ ง๏ผๅ ฑ 7,157 ๆก๏ผ448 ๆญฅใๅญฆไน ็ 1e-6ใๆๅคฑไธบ็ฎๆ ๆๆฌ CEใๅฏไฟกๆๅธ KL ๅๅ ณ้ฎๅญ็ฌฆ็ margin lossใ
ๆ็ป้ถๆฎต้็จ effective batch16ใFP32 ไธปๅๆฐ/ๆๅธใBF16 student autocast๏ผๅ ณ้ญ TF32ใไป ็็ฃ็ฎๆ ไธ EOS๏ผๅฑ่ฝๆ็คบๅๅกซๅ ใๆๅธไธไธ token ๅฟ ้กปไธๆ ๆณจไธ่ดไธๆฆ็่ณๅฐ0.9๏ผๅ ณ้ฎๆฐๅญ/ASCII token ็็ฎๆ logit ้ด้ไธบ1๏ผmargin ๆ้0.2ใKL ๆ้0.1ใๅฏผๅบๆ้ไธบ BF16ใ
ๅๆๆฐๅญ็ป่ฟ็ฌ็ซ่งฃๆๆฃๆฅใ่ฎญ็ปๆฑ ๅ ๅซๅ่ฝๅๅๆพไธ็ฉบ่พๅบ็็ฃใๆฅๆบๅซ CS2WใDisfl-QAใ่ชๅถๅๆจกๅ่พ ๅฉๅๆๆฐๆฎ๏ผๅๆๆ ๆณจไธ็ญไบ็ฌ็ซไบบๅทฅๆ ๆณจใ่ง training-provenance.json ๅ data-provenance.jsonใไธๅๅธ่ฎญ็ปๆญฃๆใ็งๆๅฝ้ณๆ็จๆท่พๅ ฅๅๅฒใ
ๅๅ็ซฏๆต่ฏ
ๅ
ผๅฎนไฟฎ่ฎข๏ผ้
็ฝฎๅๆถๆพๅผๅฃฐๆๆงๅญๆฎต rope_theta=1000000 ไธๆฐๅญๆฎต rope_parameters.rope_theta=1000000ใๆง็ Transformers4 ๅจๅชๆๆฐๅญๆฎตๆถไผๅ้ๅฐ10000๏ผๅฏผ่ด่ฎญ็ปไธๆจ็็ไฝ็ฝฎ็ผ็ ไธไธ่ดใไปฅไธๆ็ปฉๅจไฟฎๆญฃๅ้ๆต๏ผๆฉๆๅๅธ่ฎฐๅฝไธญ็ๆงๆ็ปฉไธไปฃ่กจๅฝๅ้
็ฝฎใๆจกๅๆ้ๆชๆนๅใ
ไธ่กจไธค็ปๅไธบ vLLM0.17ใBlackwell SM120ใBF16ใgreedyใmax-num-seqs8ใeagerใprefix cacheๅ ณ้ญใๆๅคง็ๆ256 tokensใๆฏๆน64ๆกๆไบคใRoPE theta=1000000ใไธ่ฝไธๅ ถไปๅ็ซฏ็็ปๆ็ดๆฅๆทท็จใ
| ๆต่ฏ | ๅ native-v10 | ๆฌ็ๆฌ |
|---|---|---|
| ๆ ธๅฟๆผ่ฏป๏ผไธฅๆ ผๅน้ | 8/8 | 8/8 |
| ๅบ็ก่ฝๅ๏ผ่ฏๆณๅน้ | 46/48 | 46/48 |
| ๆ ผๅผไธๆฐๅญ๏ผ่ฏๆณๅน้ | 20/20 | 20/20 |
| ไธญๆไบบๅ๏ผ่ฏๆณๅน้ | 60/60 | 60/60 |
| ๆผ่ฏป่ฏญๅข๏ผไธฅๆ ผๅน้ | 318/326 | 318/326 |
| ็้ ่ฏๆผ่ฏป๏ผไธฅๆ ผๅน้ | 197/200 | 196/200 |
| ๅบ็กๆฐๅญฆ๏ผ่ฏๆณๅน้ | 235/236 | 235/236 |
| ๆฐๅญ/ๆถ้ด่ฏญๅขA๏ผไธฅๆ ผๅน้ | 367/432 | 414/432 |
| ๆฐๅญ/ๆถ้ด่ฏญๅขB๏ผไธฅๆ ผๅน้ | 388/432 | 430/432 |
| ๅ ฌๅผๆๆฌ๏ผ่ฏๆณๅน้ | 234/400 | 234/400 |
| ๅ ฌๅผๆๆฌ่ฏๆณ CER๏ผ่ถไฝ่ถๅฅฝ | 9.3136% | 9.2367% |
ๆฌๆฌก2,167ๆกๅฎ้ vLLM ็ๆๆ ๆชๆญใๆ ผๅผๆฃๆฅ้ขๅคๆ ธ้ชๆ้ๆข่ก๏ผๅฆๅๅ่ฏทๆฑ5ๆกๅ ณ้ฎๆฃๆฅ๏ผ5/5ไธฅๆ ผๅน้ ใ
ๆถ็ไธป่ฆๅจๆฐๅญ่ฏญๅข๏ผ็้ ่ฏๆผ่ฏปๅฐๅน้ 1ๆก๏ผไธๆฏๆๆ่ฝๅๅๆๅใ ๆดๅฅๅน้ ๅ ้ค้ๅญๆฏๆฐๅญๅญ็ฌฆๆถ็งฐโ่ฏๆณๅน้ โ๏ผไฟ็ๅคงๅฐๅไธๆฐๅญ๏ผไธฅๆ ผๅน้ ไฟ็ๆ ็นๅๅ ้จ็ฉบ็ฝใๅฎไปฌไธๆฏไบบๅทฅ่ฏญไนๆญฃ็กฎ็๏ผไนไธๆฏไธๆธธๆ็งฐ็94.8% token accuracyใ
่ฟไบ้ๅ็ป่ฟๅผๅไธญ็ๅๅคไฝฟ็จ๏ผๅๅบ่งไธบๅผๅ/ๅๅฝ่ฏๆฎใ่ฏญๅขBๆๅๅจ้ๆฉ็ฌฌไธ่ฝฎๆจกๅๅๆต่ฏ๏ผไฝๅ็ปญไฟฎ่ฎขๅทฒๅ่ๅ ถ้่ฏฏ็ฑปๅ๏ผๅ ๆญคๆ็ป็ๆฌไธๅ็งฐๅ ถไธบ็ฌ็ซ็ฒๆตใๆฐๅญๅผไธ่ฏญๆณๅฎถๆไนๅฏ่ฝๅทฒๅจๆดๆฉ่ฎญ็ปไธญๅบ็ฐใๅฎๆด็ปๆ่ง evaluation-summary.jsonใ
้ๅถ
- ๆฐๅญไปๅฏ่ฝ่ฝฌ้ๆฐๅผ๏ผ่ฏญๅขBๅฉ2ๆกๅ่ๅทฎๅผ๏ผ5001่ฝฌๆ5010๏ผไปฅๅ101ไฟ็ไธบไธญๆๅๆณใ
- ๆผ่ฏปใไธๆ่ฏใๅฆๅฎใๆนๅฃๅ้ฟๆฎต่ฝไปๅฏ่ฝๆด็้่ฏฏใ
- ้จๅๆฉๆๅๆๅไฝๆฐ็็ฅไบ่ง่ๅฃ่ฟฐไธญ็โ้ถโ๏ผๆฐๆฎๅๅธไธไปฃ่กจ่ช็ถ ASRใ
- ๆตฎ็น่ๅ ฅใๆนๆฌกใๅ็ซฏๅ็กฌไปถไปๅฏ่ฝๆนๅ่พๅบใ่ฎญ็ป้ไฝ้ฃ้ฉ๏ผไธๆไพไปปๆ้ ็ฝฎไธ้ไฝไธ่ด็ไฟ่ฏใ
- ๅฝๅ้ช่ฏๆฏๆฌๅฐๆๆฌๆจ็๏ผไธๆฏ่ช็ถๅฝ้ณใๅฎไฝๆ้ฎๆๆๆ่พๅ ฅๆก็้ชๆถใ
- ๅๅธๆฃๆฅๆฒกๆๆฟๆข็ฐ็จ่ฏญ้ณๆๅกใๅ ถไป็กฌไปถไธ Orin ็ๆฌๆฌกๅฎๆดๅๅฝๆช้ช่ฏใ
ไฝฟ็จ
ไฝฟ็จ้ๅธฆ tokenizer/template๏ผๅ
ณ้ญ thinkingใๆฎ้ๆๆฌ้ๆฉ Structure: prose๏ผๆ็กฎๅ่กจ้ๆฉ Structure: listsใ
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "WIlfLin/S1-mini-Chinese-1106-BF16"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id, dtype=torch.bfloat16, device_map="auto"
).eval()
messages = [
{"role": "system", "content": (
"You are a text normalizer for speech-to-text transcripts. The input begins "
"with a control line specifying the styling, structure, and context settings; "
"clean the transcript to match those settings and output only the cleaned text."
)},
{"role": "user", "content": (
"[Styling: semi-formal] [Structure: prose] [Context: general]\n"
"ๅฏ๏ผ่ฟไปฝๆฅๅ่ฟไปฝๆฅๅๆๅคฉๅไบคใ"
)},
]
text = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.inference_mode():
output = model.generate(
**inputs, do_sample=False, max_new_tokens=256,
pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(
output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True
).strip())
่ฎธๅฏ่ฏไธๆฅๆบ็ฝฒๅ
ๆฌๆจกๅๆฏ S1-mini by Superwhisper ็ไฟฎๆน็ๆฌ๏ผๅบๅฑๆถๆไธบ Qwen3-0.6Bใ ไฟ็ๅๅง LICENSE ๅ NOTICE๏ผๅ ๆฌไฟ็ๅๆจกๅๅ็งฐๅไฝ่ ็ฝฒๅ็้ๅ ๆกๆฌพใ CS2W ๅฃฐๆ CC-BY-SA-4.0๏ผDisfl-QA ๅฃฐๆ CC-BY-4.0๏ผไธๆๆฐๆฎๆกๆฌพๆน็งฐไธบ็บฏ Apache-2.0ใ
ๆ้ SHA256๏ผc0d2cf58468359e0a44970083f9ce38078208c291430d20a5c685c5ce6759312ใ
- Downloads last month
- 786
Model tree for WIlfLin/S1-mini-Chinese-1106-BF16
Base model
Qwen/Qwen3-0.6B-Base