BeautyPackOCR-KO

🇰🇷 Korean card: MODEL_CARD.ko.md

Line-level OCR for photos of Korean cosmetics and pharmaceutical packages: a DBNet (ResNet-50) text detector and a CRNN-CTC recognizer, exported to ONNX and run on CPU with onnxruntime. It returns every detected text line with its box, text and confidence, plus normalized date strings found in those lines. It does not extract product names itself and does not tell manufacture dates from expiry dates. Product-name candidates from its output are a separate model: ThakiCloud/BeautyProductName-KO (see §3).

Scope in one line: trained and evaluated only on indoor studio-style captures of 633-category packages. Real phone photos (glare, curved surfaces, blur, several products in one frame) were not evaluated. Treat every output as a draft for a person to check.

0. How to use

pip install onnxruntime opencv-python-headless pyclipper pillow numpy huggingface_hub
hf download ThakiCloud/BeautyPackOCR-KO --local-dir BeautyPackOCR-KO      # huggingface_hub >= 0.34 (older: huggingface-cli download)
cd BeautyPackOCR-KO
python beautypackocr.py --selftest          # synthetic image, checks both ONNX graphs load and run
python beautypackocr.py your_photo.jpg --threads 4

from beautypackocr import BeautyPackOCR
ocr = BeautyPackOCR(".", threads=4)          # verifies file sha256 against config.json
out = ocr(open("your_photo.jpg", "rb").read())
for line in out["lines"]:
    print(line["box_xyxy"], line["text"], line["confidence"], line["review"])
print(out["dates"])                          # [{"value": "20270512", "granularity": "day", "date_type": "UNCLASSIFIED", ...}]

Fixed settings (the ones every number below was measured with): long side resized to 1280, DB box threshold 0.3, binarization threshold 0.3, greedy CTC decoding. Line order is detector order, not reading order. confidence = min(detector score, mean max-softmax of emitted characters) and is not a calibrated probability.

1. Files

File	What it is	sha256
det_db_resnet50.onnx	DBNet ResNet-50 text detector (ONNX, input input [1,3,H,W])	adb85ddc75bae817…
crnn_633.onnx	CRNN-CTC line recognizer (ONNX, input x [B,1,32,W])	8bf171bd02d69d4c…
charset.json	recognizer output alphabet: a JSON list of 1,578 single characters (index + 2)	afd7032e4b133abf…
checkpoints/det_db_resnet50_best.pt	detector training checkpoint (state_dict, loads with weights_only=True, needs python-doctr)	a1bda5c5dcc0db7c…
checkpoints/crnn_633_best.pt	recognizer training checkpoint (state_dict + charset)	9a29f7749ef53b8d…
beautypackocr.py	complete CPU inference pipeline (pre/post-processing, decoding, date rules)
crnn_torch.py	PyTorch CRNN definition, checkpoint loaders, torch-vs-ONNX parity check
config.json	thresholds, normalization, full sha256 of every weight file

charset.json is part of the model (the recognizer's output vocabulary, one character per entry, no duplicates), not a data record. No images, crops, labels, text transcripts or sample predictions from the training data are in this repository.

2. Results

All numbers come from our own runs, recorded in an internal experiment ledger. Nothing is quoted from other papers.

End-to-end (detection + recognition), dev-B split

dev-B = 4,024 photos from 1,173 product groups that do not appear in training. Single evaluation run.

Metric	Value
Detection H-mean	0.868
End-to-end line F1	0.809 (95% CI 0.799–0.818)
End-to-end CER (matched lines)	0.117
Date field F1	0.848

Near-duplicate caveat. An audit found that 33.9% of dev-B photos are visual near-duplicates of training photos (same product, nearly the same shot). Removing them lowers the numbers:

dev-B subset	Photos	E2E F1
full	4,024	0.809
minus visual near-duplicates of train	2,659	0.792
minus any near-duplicate signal (visual, text, hash)	1,260	0.775

So the honest range for unseen products in the same capture setting is about 0.775–0.792, not 0.809. dev-B was also used to pick between candidate configurations, so even 0.775 carries some selection bias.

Recognizer alone (ground-truth line crops)

Metric	Value
Test CER, all test crops	0.0158
Exact-match rate (full test set)	0.971

This isolates the recognizer: boxes come from the labels, not from the detector.

Latency (per photo, excluding image decode unless noted)

Setting	p50	p95
CPU, onnxruntime 1.31.1, 4 intra-op threads (AMD EPYC 9355 host)	547 ms	1,349 ms
CPU, same, including JPEG decode	597 ms	1,381 ms
GPU, H200 NVL, PyTorch FP32 (not ONNX)	44.7 ms	91.0 ms

Measured on 95 dev-B photos after 5 warm-up runs. Peak RSS on CPU 4.12 GiB. The CPU figure is from 4 threads on a GPU server's CPU, not a measurement on customer on-premise hardware. ONNX vs PyTorch end-to-end F1 differed by −0.0005 on the benchmark rows.

3. Limitations

• One capture condition. All training and evaluation photos are indoor captures from one dataset; every image has exactly one product. Real photos with glare, curved bottles, motion blur or several products were not evaluated, and the dataset has no labels for those conditions.
• Near-duplicates and selection bias in dev-B, described above.
• No product name, no date type. Dates are returned as normalized strings (YYYYMM / YYYYMMDD) with date_type: UNCLASSIFIED. A missing date or an empty result is not proof that the package has none.
• Product names are handled by a separate model. ThakiCloud/BeautyProductName-KO is now public. It ranks this model's output lines as product-name candidates for a person to confirm. What was published and what was not:
  • Published: the two variants that need no data-derived table. ocr_conf (default) reaches line-level F1 0.605 (95% CI 0.599–0.612) end-to-end on 21,044 photos (this OCR → name model). On the same photos it drops to 0.521 with near-duplicates of training photos removed, and to 0.441 on products not seen in training that are also not near-duplicates. no_lex reaches F1 0.692 on ground-truth boxes and text (1,505 photos), so that number has no OCR errors in it. The 0.605 / 0.521 / 0.441 numbers are all for ocr_conf.
  • Not published: the F1 0.844 model. That number is on ground-truth boxes and text, not on photos run through OCR. It depends on a lexicon: a frequency table of 255,150 label strings counted from the AI-Hub training labels (how often each string was a product name, and how often it appeared). The table is derived from the dataset, so it cannot be published, and the model does not run without it. 0.844 is therefore not the score of a model you can download. It is a research reference that includes memorized training strings.
  • The end-to-end numbers were measured on photos from the same dataset, mostly showing products already seen in training. They were never tested on customer photos. See that model's card for the protocol and limits.
• Characters outside the 1,578-entry alphabet cannot be produced (they map to <unk> and are dropped).
• EXIF orientation is not applied; rotate images upright before calling.

4. Training data

Trained on AI 허브(AI-Hub) — 화장품·의약품 패키지 OCR(633) (Korean cosmetics & pharmaceutical package OCR dataset, AI-Hub dataset 633).

데이터 재배포 불가, 신청자 본인 내부 이용. The dataset is not redistributable: it was used only by the applicant, inside the applicant's own environment. This repository contains trained weights only; it contains no images, crops, label files, transcripts or derived subsets of the dataset. To retrain, apply for the dataset on AI-Hub with your own account.

Initialization: the detector started from doctr's public db_resnet50 pretrained weights (Apache-2.0) and was fine-tuned for 3 epochs; the recognizer was trained from scratch on line crops.

5. License

Apache-2.0 for the weights and code. See NOTICE for the doctr-derived post-processing code and the required dataset attribution. All runtime dependencies are permissively licensed (MIT / BSD / Apache-2.0).