westhnu/articulation-therapy-lipreading

Korean Lipreading Model — ResNet-18 + Bi-GRU

웹캠 입모양으로 한국어 117개 문장을 인식해 조음치료 보조에 사용되는 모델.

• 학습 데이터: AI-Hub 립리딩 데이터셋 (561 videos → 9,047 augmented sequences)
• 백본: ResNet-18 (no pretrain) + 2-layer bidirectional GRU (256 hidden)
• 입력: 30 frames × 112×112 RGB lip crop
• 출력: 117 sentence classes (top-1 + confidence)

Files

file	description
best_model.pt	trained weights (52 MB)
label_classes.npy	class label array (numpy, allow_pickle=True)

Usage

from huggingface_hub import hf_hub_download
import torch, numpy as np

weights = hf_hub_download("westhnu/articulation-therapy-lipreading", "best_model.pt")
labels  = np.load(hf_hub_download("westhnu/articulation-therapy-lipreading", "label_classes.npy"), allow_pickle=True)

실시간 데모는 GitHub: westhnu/articulation_therapy_app 의 realtime_lipreading_demo.py 참고.

Citation

학부 컴퓨터비전 프로젝트 (2025, 인공지능학과) — 서현우 (westhnu), 최윤형, 백혜경.