  In Named Entity Recognition (NER), pre-trained language models have been
overestimated by focusing on dataset biases to solve current benchmark
datasets. However, these biases hinder generalizability which is necessary to
address real-world situations such as weak name regularity and plenty of unseen
mentions. To alleviate the use of dataset biases and make the models fully
exploit data, we propose a debiasing method that our bias-only model can be
replaced with a Pointwise Mutual Information (PMI) to enhance generalization
ability while outperforming an in-domain performance. Our approach enables to
debias highly correlated word and labels in the benchmark datasets; reflect
informative statistics via subword frequency; alleviates a class imbalance
between positive and negative examples. For long-named and complex-structure
entities, our method can predict these entities through debiasing on
conjunction or special characters. Extensive experiments on both general and
biomedical domains demonstrate the effectiveness and generalization
capabilities of the PMI.
