## 5 Conclusions and Future Work  
This paper proposed a simple but effective approach for the domain task where training annotated data is limited or not available. We apply domain adaptation technique to further pre-train the BERT language model on domain-specific raw text (unlabeled review dataset from YELP) for contextual learning. As proof of the effectiveness of our NER approach, we performed experiments on the food menu named-entity recognition (MenuNER) task on annotated dataset. We manually labeled menu entities from the user review dataset provided by YELP. We first further pre-trained the off-the-shelf BERT-base model on a food domain corpus to obtain a domain-adapted BERT language model, BERT-domain. Next, we fine-tuned the downstream MenuNER task using the Bi-LSTM+CRF model with a concatenated feature vector consisting of word embedding from BERT-domain, character embedding, and POS tag features. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \# & **Review Text** & **BERT-Base** & **BERT-Domain** \\ \hline
1 & I tried their _wazeri platter_ which is _grilled_ & wazeri platter, chicken, beef kafta & wazeri platter, grilled chicken, \\  & _chicken, beef kafta kabub_ and _rice_ & kabab, rice & beef kafta kabab, rice \\  & I never finish the _rice_ and always ask for a & rice, naan, leftover rice & rice, naan, rice \\
2 & takeaway container for the _nann_ + leftover & rice, naan, leftover rice & rice, naan, rice \\  & & _rice_. & & \\
3 & We both ordered _green tea_ and it had a very & tea & green tea \\  & unique taste. & & & \\
4 & The _red spicy tomato-based suce_ on the side & sauce & red spicy tomato-based suce \\  & & & & \\
5 & I would say it’s like the size of 2 medium & chicken & chicken breasts \\  & & & & \\  & & & & \\  & & & & \\
6 & Also check out _Firni_ (a _sweet custard_) for & sweet custard & Firni, sweet custard \\  & & desert & & \\
7 & _Chicken_ is well cooked tooOmg, super & chicken, tooOmg, kebabs & Chicken, kebabs \\  & & & & \\
8 & It’s called _Double Ka Meeta_ and we loved it! & & Double Ka Meeta \\  & & & & \\
9 & Dessert options were _kesari, gajar halwa,_ & kesari, gajar halwa, strawberry & kesari, gajar halwa, \\  & & & & \\  & & & & \\  & & & & \\  & & & & \\ \end{tabular}
\end{table} 
Table 4: Menu entities’ prediction from applying BERT-base features vs. BERT-domain features to the deep learning model. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{\#} & \multicolumn{2}{c}{**Word Embedding**} & \multicolumn{2}{c}{**BERT-Base Embedding**} & \multicolumn{2}{c}{**BERT-Domain Embedding**} \\ \cline{2-7}  & **NER Network** & **P** & **R** & **F1** & **P** & **R** & **F1** \\ \hline
1 & SoftMax & 88.7 & 91.7 & 90.2 & 91.7 & 92.1 & 91.9 \\
2 & Bi-LSTM-CRF & 90.3 & 91.3 & 91.8 & **92.7** & **92.3** & **92.5** \\
3 & Bi-LSTM-Attn-CRF & 92.2 & 90.5 & 91.4 & 91.0 & 92.1 & 91.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Summary of experimental results for MenuNER with (BERT-base-cased) and target domain-adapted version (BERT-domain) on NER task model. We utilize feature-based BERT model with extended features for NER network 2 and 3 while fine-tuning BERT model on downstream dataset for network 1. We investigated the effect of applying fine-tuned domain-specific BERT language model on down-stream task performance. We found that the off-the-shelf BERT model, when further pre-trained on domain-specific corpus, can learn new context and perform even better and can be used for \"feature extraction\" or \"fine-tuning\" downstream tasks. Entity extraction is the first and essential step in the information extraction process. Our work extensively contributes to the field of food domain and can be extended to solve many problems in the food domain. As a future direction, our work could lead to extending many IE and entity-linking tasks including creation of food knowledge graph, menu entity-level sentiment analysis from text and menu-based text summarization from raw data, etc. Conceptualization, M.H.S. ; methodology, M.H.S. ; software, M.H.S.