CS221DoAn/Do_an_group_BiLSTM_CRF

BiLSTM-CRF-FoodNER: Deep Sequence Labeling Baseline for Vietnamese Food Orders

Table of contents

1. Introduction
2. Dataset Overview
3. Entity Label Space
4. Empirical Evaluation
5. Architecture Characteristics
6. Using BiLSTM-CRF-FoodNER with PyTorch
7. Authors & Citation

---

1. Introduction

CS221DoAn/Do_an_group_BiLSTM_CRF is a deep learning model based on the Bidirectional LSTM + Conditional Random Field (BiLSTM-CRF) architecture. It is specifically trained for Named Entity Recognition (NER) on domain-specific Vietnamese unstructured text: Online Food Delivery Orders and Messages.

In our research project for the CS221.Q21 course (Natural Language Processing), this model serves as a deep-learning sequence baseline. It was developed to overcome the limitations of traditional CRF (which relies heavily on manual feature engineering) by automating non-linear feature extraction through recurrent neural networks.

2. Dataset Overview

The model was trained on a custom, manually annotated dataset consisting of 2,325 real-world food ordering messages extracted from Facebook comments (specifically from Sơn Nguyễn vegetarian restaurant).

• Train Set: 1,860 samples (80%)
• Validation Set: 232 samples (10%)
• Test Set: 233 samples (10%)

3. Entity Label Space

The model utilizes the standard BIO (Begin - Inside - Outside) tagging scheme across 15 distinct entity labels, mapping to 7 core information classes:

Tag	Entity Class	Description	Examples
B-FOOD, I-FOOD	FOOD	Names of dishes, drinks, toppings	cơm sườn, trà sữa, trân châu
B-QUANTITY, I-QUANTITY	QUANTITY	Portions, servings, item counts	1p, 2 ly, một hộp, 3 suất
B-NOTE, I-NOTE	NOTE	Special requests, flavor modifications	không, ít ngọt, nhiều
B-PLACE, I-PLACE	PLACE	Delivery location, addresses	Ký túc xá khu A, tòa D6, rào b4
B-PHONE, I-PHONE	PHONE	Receiver's contact number	0794987xxx, 0903123xxx
B-TIME, I-TIME	TIME	Expected/requested delivery time	lúc 11h30, trưa nay, 18h
B-PRICE, I-PRICE	PRICE	Monetary cost, item prices	35k, 40000, 25 ngàn
O	OUTSIDE	Non-entity words, syntax connecting words	cho em, giao qua, với, ạ, nhé

4. Empirical Evaluation

The model was evaluated on an unseen Test Set using strict Entity-level F1-Scores via the seqeval framework.

• BiLSTM-CRF Macro F1-Score: 0.9771
• Comparison: While the BiLSTM-CRF automates feature extraction, it achieved the lowest Macro F1-score (0.9771) among our tested baselines (CRF: 0.9889, mBERT: 0.9880, PhoBERT: 0.9913).

5. Architecture Characteristics

The data flows through 3 main layers in this architecture:

1. Embedding Layer: Converts numerical vocabulary identifiers into continuous mathematical vectors.
2. BiLSTM Layer: Processes the sequence in both directions (left-to-right and right-to-left). At each time step $t$, the hidden state is a concatenation of both directions: $h_{t}=[\vec{h}{t};\vec{h{t}}]$.
3. CRF Layer: Receives emission scores from the BiLSTM and applies a transition matrix to find the most valid sequence of tags via Viterbi Decoding.

Limitation: Our analysis showed that BiLSTM-CRF suffers from Information Decay when dealing with long food orders where related entities are far apart (e.g., QUANTITY at the beginning and FOOD at the end). The Self-Attention mechanism in Transformer models (like PhoBERT) resolves this limitation efficiently.

6. Using BiLSTM-CRF-FoodNER with PyTorch

Because this is a custom PyTorch model, you must define the model architecture class in your local environment before loading the .pth weights.

Installation

pip install torch huggingface_hub

Example usage (Inference Pipeline)

⚠️ IMPORTANT: You MUST replace the BiLSTM_CRF class placeholder below with the exact PyTorch class definition you used during the training phase.

import torch
import torch.nn as nn
from huggingface_hub import hf_hub_download

# 1. DEFINE YOUR MODEL ARCHITECTURE (MUST MATCH TRAINING CODE)
class BiLSTM_CRF(nn.Module):
    def __init__(self, vocab_size, tagset_size, embedding_dim, hidden_dim):
        super(BiLSTM_CRF, self).__init__()
        # --- PASTE YOUR ACTUAL PYTORCH INIT CODE HERE ---
        pass
        
    def forward(self, sentence):
        # --- PASTE YOUR ACTUAL FORWARD PASS HERE ---
        pass
        
    def decode(self, sentence):
        # --- PASTE YOUR VITERBI DECODING HERE ---
        pass

# Initialize the model with your original hyperparameters
# model = BiLSTM_CRF(vocab_size=..., tagset_size=15, embedding_dim=..., hidden_dim=...)

# 2. DOWNLOAD AND LOAD WEIGHTS
REPO_ID = "CS221DoAn/Do_an_group_BiLSTM_CRF"
FILENAME = "bilstm_crf.pth"

print("Downloading BiLSTM-CRF weights...")
model_path = hf_hub_download(repo_id=REPO_ID, filename=FILENAME)

# Load state dict
# model.load_state_dict(torch.load(model_path, map_location=torch.device('cpu')))
# model.eval()

print("Model loaded successfully! Ready for inference.")

# 3. PREDICT (Replace with your actual text processing pipeline)
def predict_food_order(raw_text):
    print(f"\nInput: {raw_text}")
    # 1. Preprocess & Tokenize text
    # 2. Convert to Tensor
    # 3. Pass through model.decode()
    # 4. Map ID to Label
    pass

7. Authors & Citation

This project was developed for the Natural Language Processing (CS221.Q21) course at the University of Information Technology (UIT) - VNU-HCM.

• Students: Võ Thành Lộc (24520989), Nguyễn Anh Nguyên (24521185)

• Instructor: Ph.D. Nguyễn Trọng Chỉnh

• Date: July 2026

If you use this model in your academic research or projects, please cite our project:

@misc{cs221_food_order_ner_bilstm_crf,
  author = {Vo Thanh Loc and Nguyen Anh Nguyen},
  title = {Food Order Extraction: BiLSTM-CRF Baseline for Vietnamese NER},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = https://huggingface.co/CS221DoAn/Do_an_group_BiLSTM_CRF
}