|
|
<!DOCTYPE html> |
|
|
<html lang="en"> |
|
|
<head> |
|
|
<meta charset="UTF-8"> |
|
|
<title>Vietnamese NLP Tasks – Benchmark Overview</title> |
|
|
<meta name="viewport" content="width=device-width,initial-scale=1"> |
|
|
<style> |
|
|
body { font-family: 'Segoe UI', Arial, sans-serif; background: #f7faff; color: #263347; margin: 0; } |
|
|
.container { max-width: 1000px; margin: 36px auto; background: #fff; padding: 28px 20px 38px 20px; border-radius: 14px; box-shadow: 0 2px 14px #0002;} |
|
|
h1, h2, h3, h4 { margin-top: 1.7em; margin-bottom: 0.5em; } |
|
|
h1 { color: #12469b; font-size: 2.2em; margin-top: 0; } |
|
|
h2 { color: #188754; border-left: 6px solid #a3d7ff; padding-left: 12px;} |
|
|
h3, h4 { color: #1a324b;} |
|
|
table { width: 100%; border-collapse: collapse; margin: 1.1em 0 1.8em 0;} |
|
|
th, td { padding: 8px 10px; border-bottom: 1px solid #eee;} |
|
|
th { background: #eaf3ff; } |
|
|
tr:hover { background: #f6fbff;} |
|
|
a { color: #2369cb; text-decoration: none;} |
|
|
a:hover { text-decoration: underline;} |
|
|
ul, ol { margin-top: 0.5em; margin-bottom: 1.1em; } |
|
|
.dataset { background: #f1f8fc; border-left: 5px solid #97d3f6; padding: 8px 18px; margin: 12px 0 18px 0;} |
|
|
.note { color: #555; background: #f6fcf7; border-left: 5px solid #7de59d; padding: 6px 16px; margin: 14px 0 20px 0;} |
|
|
.icon { font-size: 1.1em; margin-right: 6px;} |
|
|
.footer { text-align: center; font-size: 0.96em; color: #999; margin-top: 36px; } |
|
|
@media (max-width: 700px) { |
|
|
.container { padding: 6px; } |
|
|
table, th, td { font-size: 14px;} |
|
|
} |
|
|
</style> |
|
|
</head> |
|
|
<body> |
|
|
<div class="container"> |
|
|
|
|
|
<h1>🇻🇳 Vietnamese NLP Tasks <span style="font-size:0.8em; color:#555;">— Benchmark & SOTA Overview</span></h1> |
|
|
<div style="margin-bottom:1.2em; color:#537fc2;"> |
|
|
<span class="icon">📈</span> |
|
|
<b>This page tracks major Vietnamese NLP datasets and models for <u>Dependency Parsing</u>, <u>Intent Detection</u>, <u>Machine Translation</u>, <u>NER</u>, <u>POS Tagging</u>, <u>Semantic Parsing</u>, and <u>Word Segmentation</u>.</b> |
|
|
</div> |
|
|
|
|
|
|
|
|
<h2>Dependency Parsing</h2> |
|
|
<div class="dataset"> |
|
|
<span class="icon">🗂️</span> |
|
|
<b>VnDT v1.1/v1.0</b>: Benchmark treebank >10K sentences. <br> |
|
|
<b>Test:</b> 1,020 (v1.1), Dev: 200, Rest: Train. |
|
|
</div> |
|
|
|
|
|
<h3>VnDT v1.1</h3> |
|
|
<table> |
|
|
<tr> |
|
|
<th>Model</th> |
|
|
<th>LAS</th> |
|
|
<th>UAS</th> |
|
|
<th>Paper</th> |
|
|
<th>Code</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>PhoNLP (2021)</td><td>79.11</td><td>85.47</td> |
|
|
<td><a href="https://aclanthology.org/2021.naacl-demos.1.pdf">PhoNLP</a></td> |
|
|
<td><a href="https://github.com/VinAIResearch/PhoNLP">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>PhoBERT-base (2020)</td><td>78.77</td><td>85.22</td> |
|
|
<td><a href="https://arxiv.org/abs/2003.00744">PhoBERT</a></td> |
|
|
<td><a href="https://github.com/VinAIResearch/PhoBERT">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Biaffine (2017)</td><td>74.99</td><td>81.19</td> |
|
|
<td><a href="https://arxiv.org/abs/1611.01734">Biaffine Parsing</a></td> |
|
|
<td></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>VnCoreNLP (2018)</td><td>71.38</td><td>77.35</td> |
|
|
<td><a href="http://aclweb.org/anthology/N18-5012">VnCoreNLP</a></td> |
|
|
<td><a href="https://github.com/vncorenlp/VnCoreNLP">Official</a></td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
<h3>VnDT v1.0 (Gold POS)</h3> |
|
|
<table> |
|
|
<tr> |
|
|
<th>Model</th> |
|
|
<th>LAS</th> |
|
|
<th>UAS</th> |
|
|
<th>Paper</th> |
|
|
<th>Code</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>VnCoreNLP (2018)</td><td>73.39</td><td>79.02</td> |
|
|
<td><a href="http://aclweb.org/anthology/N18-5012">VnCoreNLP</a></td> |
|
|
<td><a href="https://github.com/vncorenlp/VnCoreNLP">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>BIST BiLSTM graph (2016)</td><td>73.17</td><td>79.39</td> |
|
|
<td><a href="https://aclweb.org/anthology/Q16-1023">BIST Parser</a></td> |
|
|
<td><a href="https://github.com/elikip/bist-parser/tree/master/bmstparser/src">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>MSTparser (2006)</td><td>70.29</td><td>76.47</td> |
|
|
<td><a href="http://www.aclweb.org/anthology/P05-1012">MSTparser</a></td> |
|
|
<td></td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
|
|
|
<h2>Intent Detection & Slot Filling</h2> |
|
|
<div class="dataset"> |
|
|
<span class="icon">🛫</span> |
|
|
<b>PhoATIS Dataset</b> (flight booking domain): Train: 4,478, Dev: 500, Test: 893 |
|
|
</div> |
|
|
<table> |
|
|
<tr> |
|
|
<th>Model</th><th>Intent Acc.</th><th>Slot F1</th><th>Sent. Acc.</th><th>Paper</th><th>Code</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>JointIDSF (2021)</td><td>97.62</td><td>94.98</td><td>86.25</td> |
|
|
<td><a href="https://arxiv.org/abs/2104.02021">JointIDSF</a></td> |
|
|
<td><a href="https://github.com/VinAIResearch/JointIDSF">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>JointBERT+PhoBERT</td><td>97.40</td><td>94.75</td><td>85.55</td> |
|
|
<td><a href="https://arxiv.org/abs/2104.02021">JointIDSF</a></td> |
|
|
<td><a href="https://github.com/VinAIResearch/JointIDSF">Official</a></td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
|
|
|
<h2>Machine Translation</h2> |
|
|
<div class="dataset"> |
|
|
<span class="icon">🌐</span> |
|
|
<b>PhoMT Dataset</b>: 3.02M sentence pairs | 6 domains (TED, WikiHow, MediaWiki, OpenSubtitles, News, Blog) |
|
|
</div> |
|
|
<table> |
|
|
<tr> |
|
|
<th>Model</th><th>EN→VI (BLEU)</th><th>VI→EN (BLEU)</th><th>Paper</th><th>Code</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>mBART (2020)</td><td>43.46</td><td>39.78</td> |
|
|
<td><a href="https://arxiv.org/abs/2001.08210">mBART</a></td> |
|
|
<td><a href="https://github.com/pytorch/fairseq/tree/main/examples/mbart">Link</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Transformer-big</td><td>42.94</td><td>37.83</td> |
|
|
<td><a href="https://arxiv.org/abs/1706.03762">Transformer</a></td> |
|
|
<td><a href="https://github.com/pytorch/fairseq/tree/main/examples/translation">Link</a></td> |
|
|
</tr> |
|
|
</table> |
|
|
<div class="dataset"> |
|
|
<span class="icon">📋</span> |
|
|
<b>IWSLT2015</b>: 150K sentence pairs (EN↔VI) | <a href="https://github.com/tensorflow/nmt">Data & Scripts</a> |
|
|
</div> |
|
|
<table> |
|
|
<tr> |
|
|
<th>Model</th><th>BLEU</th><th>Paper</th><th>Code</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Nguyen & Salazar (2019)</td><td>32.8</td> |
|
|
<td><a href="https://arxiv.org/abs/1910.05895">Transformers w/o Tears</a></td> |
|
|
<td><a href="https://github.com/tnq177/transformers_without_tears">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Provilkov et al. (2019)</td><td>33.27 (uncased)</td> |
|
|
<td><a href="https://arxiv.org/abs/1910.13267">BPE-Dropout</a></td> |
|
|
<td></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Xu et al. (2019)</td><td>31.4</td> |
|
|
<td><a href="https://papers.nips.cc/paper/8689-understanding-and-improving-layer-normalization.pdf">Layer Norm</a></td> |
|
|
<td><a href="https://github.com/lancopku/AdaNorm">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Transformer (2017)</td><td>28.9</td> |
|
|
<td><a href="http://papers.nips.cc/paper/7181-attention-is-all-you-need">Transformer</a></td> |
|
|
<td><a href="https://github.com/duyvuleo/Transformer-DyNet">Link</a></td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
|
|
|
<h2>Named Entity Recognition (NER)</h2> |
|
|
<div class="dataset"> |
|
|
<span class="icon">🩺</span> |
|
|
<b>PhoNER_COVID19</b>: 10 types, 34,984 entities, 10,027 sentences |
|
|
</div> |
|
|
<table> |
|
|
<tr> |
|
|
<th>Model</th><th>F1</th><th>Paper</th><th>Code</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>PhoBERT-large</td><td>94.5</td> |
|
|
<td><a href="https://arxiv.org/abs/2003.00744">PhoBERT</a></td> |
|
|
<td><a href="https://github.com/VinAIResearch/PhoBERT">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>XLM-R-large</td><td>93.8</td> |
|
|
<td><a href="https://aclanthology.org/2020.acl-main.747/">XLM-R</a></td> |
|
|
<td><a href="https://github.com/facebookresearch/XLM">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>BiLSTM-CRF + CNN-char</td><td>91.0</td> |
|
|
<td><a href="http://www.aclweb.org/anthology/P16-1101">BiLSTM-CRF</a></td> |
|
|
<td><a href="https://github.com/UKPLab/emnlp2017-bilstm-cnn-crf/">Link</a></td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
<div class="dataset"> |
|
|
<span class="icon">📄</span> |
|
|
<b>VLSP 2016 NER</b>: 16,861 train/dev, 2,831 test sentences. |
|
|
</div> |
|
|
<table> |
|
|
<tr> |
|
|
<th>Model</th><th>F1</th><th>Paper</th><th>Code</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>PhoBERT-large</td><td>94.7</td> |
|
|
<td><a href="https://arxiv.org/abs/2003.00744">PhoBERT</a></td> |
|
|
<td><a href="https://github.com/VinAIResearch/PhoBERT">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>PhoNLP</td><td>94.41</td> |
|
|
<td><a href="https://aclanthology.org/2021.naacl-demos.1.pdf">PhoNLP</a></td> |
|
|
<td><a href="https://github.com/VinAIResearch/PhoNLP">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>vELECTRA</td><td>94.07</td> |
|
|
<td><a href="https://arxiv.org/abs/2006.15994">vELECTRA</a></td> |
|
|
<td><a href="https://github.com/fpt-corp/viBERT">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>VnCoreNLP</td><td>91.30</td> |
|
|
<td><a href="http://aclweb.org/anthology/N18-5012">VnCoreNLP</a></td> |
|
|
<td><a href="https://github.com/vncorenlp/VnCoreNLP">Official</a></td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
|
|
|
<h2>Part-of-Speech Tagging</h2> |
|
|
<div class="dataset"> |
|
|
<span class="icon">🔤</span> |
|
|
<b>VLSP 2013</b>: 27,870 train/dev, 2,120 test |
|
|
</div> |
|
|
<table> |
|
|
<tr> |
|
|
<th>Model</th><th>Accuracy</th><th>Paper</th><th>Code</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>PhoBERT-large</td><td>96.8</td> |
|
|
<td><a href="https://arxiv.org/abs/2003.00744">PhoBERT</a></td> |
|
|
<td><a href="https://github.com/VinAIResearch/PhoBERT">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>vELECTRA</td><td>96.77</td> |
|
|
<td><a href="https://arxiv.org/abs/2006.15994">vELECTRA</a></td> |
|
|
<td><a href="https://github.com/fpt-corp/viBERT">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>PhoNLP</td><td>96.76</td> |
|
|
<td><a href="https://aclanthology.org/2021.naacl-demos.1.pdf">PhoNLP</a></td> |
|
|
<td><a href="https://github.com/VinAIResearch/PhoNLP">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>PhoBERT-base</td><td>96.7</td> |
|
|
<td><a href="https://arxiv.org/abs/2003.00744">PhoBERT</a></td> |
|
|
<td><a href="https://github.com/VinAIResearch/PhoBERT">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>VnCoreNLP-VnMarMoT</td><td>95.88</td> |
|
|
<td><a href="http://aclweb.org/anthology/U17-1013">VnMarMoT</a></td> |
|
|
<td><a href="https://github.com/datquocnguyen/vnmarmot">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>BiLSTM-CRF + CNN-char</td><td>95.40</td> |
|
|
<td><a href="http://www.aclweb.org/anthology/P16-1101">BiLSTM-CRF</a></td> |
|
|
<td><a href="https://github.com/XuezheMax/LasagneNLP">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>RDRPOSTagger</td><td>95.11</td> |
|
|
<td><a href="http://www.aclweb.org/anthology/E14-2005">RDRPOSTagger</a></td> |
|
|
<td><a href="https://github.com/datquocnguyen/rdrpostagger">Official</a></td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
|
|
|
<h2>Semantic Parsing</h2> |
|
|
<div class="dataset"> |
|
|
<span class="icon">🗃️</span> |
|
|
<b>ViText2SQL</b>: 10K question/SQL pairs, the first public Text-to-SQL dataset for Vietnamese. |
|
|
</div> |
|
|
<table> |
|
|
<tr> |
|
|
<th>Model</th><th>Exact Match Acc.</th><th>Paper</th><th>Code</th><th>Note</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>IRNet (2019)</td><td>53.2</td> |
|
|
<td><a href="https://aclanthology.org/2020.findings-emnlp.364/">ViText2SQL</a></td> |
|
|
<td><a href="https://github.com/microsoft/IRNet">Link</a></td> |
|
|
<td>Using PhoBERT encoder</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>EditSQL (2019)</td><td>52.6</td> |
|
|
<td><a href="https://aclanthology.org/2020.findings-emnlp.364/">ViText2SQL</a></td> |
|
|
<td><a href="https://github.com/ryanzhumich/editsql">Link</a></td> |
|
|
<td>Using PhoBERT encoder</td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
|
|
|
<h2>Word Segmentation</h2> |
|
|
<div class="dataset"> |
|
|
<span class="icon">✂️</span> |
|
|
<b>VLSP 2013</b>: 75k train, 2,120 test sentences (manually word-segmented) |
|
|
</div> |
|
|
<table> |
|
|
<tr> |
|
|
<th>Model</th><th>F1</th><th>Paper</th><th>Code</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>UITws-v1 (2019)</td><td>98.06</td> |
|
|
<td><a href="https://arxiv.org/abs/2006.07804">UITws-v1</a></td> |
|
|
<td><a href="https://github.com/ngannlt/UITws-v1">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>VnCoreNLP-RDRsegmenter (2018)</td><td>97.90</td> |
|
|
<td><a href="http://www.lrec-conf.org/proceedings/lrec2018/pdf/55.pdf">VnCoreNLP</a></td> |
|
|
<td><a href="https://github.com/datquocnguyen/RDRsegmenter">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>UETsegmenter (2016)</td><td>97.87</td> |
|
|
<td><a href="http://doi.org/10.1109/RIVF.2016.7800279">UETsegmenter</a></td> |
|
|
<td><a href="https://github.com/phongnt570/UETsegmenter">Official</a></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>vnTokenizer (2008)</td><td>97.33</td> |
|
|
<td><a href="https://link.springer.com/chapter/10.1007/978-3-540-88282-4_23">vnTokenizer</a></td> |
|
|
<td></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>JVnSegmenter (2006)</td><td>97.06</td> |
|
|
<td><a href="http://www.aclweb.org/anthology/Y06-1028">JVnSegmenter</a></td> |
|
|
<td></td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
<div class="footer"> |
|
|
NLP Progress – Benchmarks collected by the open-source community.<br> |
|
|
<span style="color:#ccc;">Style inspired by <a href="https://github.com/sebastianruder/NLP-progress" target="_blank">NLP-progress</a></span> |
|
|
</div> |
|
|
</div> |
|
|
</body> |
|
|
</html> |
|
|
|
