JobBERT-zh 1M

This repository is the 1M JobBERT-zh dump, not the 3M dump.

The 3M paper-main encoder (ckpt65000 + V4 CRF, typed exact 0.4331) lives at AlfredJames/jobbert-zh. Do not treat files in this repo as that 3M checkpoint. model.safetensors and crf/best.pt here are the 1M DAPT encoder and the 1M V4 CRF head.

JobBERT-zh 1M is a Chinese job-domain encoder and CRF span head for Chinese-SkillSpan (competency span extraction from Chinese job advertisements). It follows the JobBERT / DaJobBERT domain-adaptive pre-training setup of Zhang et al., using a Chinese RoBERTa-wwm backbone, continued on 1 million Chinese job-ad sentences (3 MLM epochs).

This 1M model: https://huggingface.co/AlfredJames/jobbert-zh-1m
3M companion (different weights): https://huggingface.co/AlfredJames/jobbert-zh
Code and data: https://github.com/AlfredJamesLi/chinese-skillspan-benchmark
Archive (v0.1.1): https://doi.org/10.5281/zenodo.22288338

This is not English jjzha/jobbert-base-cased and not TechWolf JobBERT-v3.

The Hub tree may show a single “finetuned from hfl/chinese-roberta-wwm-ext” hop. Training actually has two stages (below). There is no AutoModelForTokenClassification export and no hosted Inference Provider. The sidebar widget is disabled on purpose.

What this repository contains

Stage	Role	Files in this repo
1. Public backbone	Chinese RoBERTa-wwm	not redistributed; load `hfl/chinese-roberta-wwm-ext` if needed
2. Domain-adaptive MLM	Pretrained JobBERT-zh 1M encoder (1M job-ad sentences, 3 epochs)	`config.json`, `model.safetensors`, `tokenizer.json`, `tokenizer_config.json`
3. Task CRF	Fine-tuned 1M V4 span head on V4 silver LSKT, seed 42	`crf/best.pt`

Paper 1M typed exact F1 0.427162 uses stage 2 + stage 3 and jieba span snap against the V4 hybrid 2601 gold. Loading only model.safetensors is not enough to reproduce that number.

config.json reports BertForMaskedLM (the DAPT dump used as --model_dir when the CRF was trained). AutoModel loads the BERT encoder. Hidden size 768, 12 layers, vocabulary 21,128. This is not the 3M Hub extract (BertModel, step 65000) in AlfredJames/jobbert-zh.

Sentence dumps (data/jobbert_*_sents.jsonl) and last.ckpt are not published.

File hashes (DS209213 source)

File	Bytes	SHA-256
`config.json`	890	`e7b10fb0074d03e1ee607853ce8ded56ca684f3968e3560ae16b89736b0b963c`
`model.safetensors`	474,090,208	`7fd83053d61a4b5e5e9dc964e2ca2e9bb6eada05c21fbae5edf343b15e5bed57`
`tokenizer.json`	439,125	`48cea5d44424912a6fd1ea647bf4fe50b55ab8b1e5879c3275f80e339e8fae26`
`tokenizer_config.json`	350	`a690ace137dc901c4c2025eddc805e5587d0ec8ee52dce5c3436ec88cb40c84f`
`crf/best.pt`	409,169,002	`380969306d50cd94a888592a2bbd2933dcca0f3a01ab7f52510bfa930edbe0b6`

Intended uses

Research on Chinese competency / skill-span extraction
Fine-tuning or evaluation on Chinese-SkillSpan (LSKT)
Reproducing the paper 1M encoder row after jieba alignment

Out-of-scope uses

Applicant screening, hiring automation, or profiling of individuals
Inferring protected attributes
Claiming ESCO concept-ID prediction (this model emits LSKT spans only)
Nested or overlapping NER
Treating V4 hybrid scores as fully human gold
Production HR systems without human review
Hub Inference Providers / the default token-classification widget
Substituting this 1M dump for the 3M AlfredJames/jobbert-zh weights

Architecture

Encoder: AutoModel from this repository (continued MLM from hfl/chinese-roberta-wwm-ext on 1M job-ad sentences).
Head: linear emissions + linear-chain CRF (torchcrf, batch_first=True), 9 BIO labels
O, B-L, I-L, B-K, I-K, B-S, I-S, B-T, I-T.
Default fine-tune recipe (scripts/train_cn_roberta_crf.py): seed 42, 6 epochs, patience 2, batch size 16, max length 256, learning rate 2e-5.
Tokenizer: AutoTokenizer with is_split_into_words=True on character tokens.

Training data

Backbone: hfl/chinese-roberta-wwm-ext. Hugging Face card metadata lists Apache-2.0 for that checkpoint.
MLM: 1,000,000 Chinese job-advertisement sentences, 3 epochs. Sentence dumps are not published with this model.
CRF: train_lskt_v4_silver.jsonl / dev_lskt_v4_silver.jsonl (SOP v4 silver, not human Doccano Gold)

This repository’s licence remains other until job-advertisement text rights are confirmed. Compatibility with Apache-2.0 of the backbone is required before any more permissive SPDX id is chosen.

Loading

import torch
from torch import nn
from torchcrf import CRF
from huggingface_hub import hf_hub_download
from transformers import AutoModel, AutoTokenizer

REPO = "AlfredJames/jobbert-zh-1m"

class BertCRF(nn.Module):
    def __init__(self, model_dir: str, n_labels: int = 9, dropout: float = 0.1):
        super().__init__()
        self.encoder = AutoModel.from_pretrained(model_dir)
        self.dropout = nn.Dropout(dropout)
        self.emissions = nn.Linear(self.encoder.config.hidden_size, n_labels)
        self.crf = CRF(n_labels, batch_first=True)

tok = AutoTokenizer.from_pretrained(REPO)
model = BertCRF(REPO)
crf_path = hf_hub_download(REPO, "crf/best.pt")
missing, unexpected = model.load_state_dict(
    torch.load(crf_path, map_location="cpu"), strict=False
)
assert missing == [] and unexpected == []

The paper repository class BertCRF in scripts/train_cn_roberta_crf.py is the implementation used for the published scores. After decoding tags, jieba-snap predictions and run scorer/score_lskt.py --align-mode official.

Fine-tuning

python3 scripts/train_cn_roberta_crf.py \
  --seed 42 \
  --model_dir AlfredJames/jobbert-zh-1m \
  --train data/train_lskt_v4_silver.jsonl \
  --dev data/dev_lskt_v4_silver.jsonl \
  --test data/corpus_splits/test.json \
  --gold data/gold_canonical_v2.jsonl \
  --out_dir path/to/crf_run \
  --epochs 6 --patience 2 --batch_size 16 --max_len 256 --lr 2e-5

train_cn_roberta_crf.py currently sets local_files_only=True; point --model_dir at a local snapshot if the script is used unchanged.

Evaluation

Task: typed LSKT span extraction
Scorer: cnss-lskt-1.2.0, official alignment, jieba snap
Paper gold for this row: 2,601 IDs, V4 hybrid (derived; not human Doccano Gold)
Reproduced on DS209213 from crf/best.pt preds + jieba (tables/hybrid_cws_simhuman980_all_models.csv / this upload check):

System	Typed exact F1	Typed relaxed F1
JobBERT-zh 1M + V4 CRF (this repo)	0.427162	0.595170
JobBERT-zh 3M + V4 CRF (`AlfredJames/jobbert-zh`, not these files)	0.433118	0.587322

Do not rank these figures against Gold v2 ChatGPT 0.6365 in one sentence.

Limitations

Silver CRF labels are not fully human-adjudicated. Jieba snap changes exact-match F1 substantially. Public-institution ads are difficult under Gold v2 notes. This 1M dump is below the 3M paper-main exact F1 (0.4272 vs 0.4331).

Ethics

Job advertisements may contain employer names and workplace locations. Do not re-identify people, scrape extra ads without rights, or use scores as the sole hiring signal.

Funding: National Social Science Fund of China, Grant No. 21BGL142.

Authors

Guojing Li (Renmin University of China; City University of Hong Kong) and Zichuan Fu (City University of Hong Kong) contributed equally. Junyi Li, Wenlin Zhang, Kaifeng Guo, Jinning Yang, Jingtong Gao, and Xiangyu Zhao are with City University of Hong Kong. Corresponding author: Xiangyu Zhao (xianzhao@cityu.edu.hk).

Licence

license: other

The backbone hfl/chinese-roberta-wwm-ext is listed as Apache-2.0 on Hugging Face. This checkpoint is trained further on job-advertisement text whose redistribution rights are not confirmed in the paper repository. Do not treat JobBERT-zh 1M as Apache-2.0 until that confirmation exists.

Links

Resource	URL
This 1M model	https://huggingface.co/AlfredJames/jobbert-zh-1m
3M companion	https://huggingface.co/AlfredJames/jobbert-zh
Code and data	https://github.com/AlfredJamesLi/chinese-skillspan-benchmark
Zenodo version DOI (`v0.1.1`)	https://doi.org/10.5281/zenodo.22288338
Zenodo concept DOI	https://doi.org/10.5281/zenodo.22288337

Downloads last month: 17

Safetensors

Model size

0.1B params

Tensor type

F32

Model tree for AlfredJames/jobbert-zh-1m

Base model

hfl/chinese-roberta-wwm-ext

Finetuned

(72)

this model