  Conditional Random Field (CRF) based neural models are among the most
performant methods for solving sequence labeling problems. Despite its great
success, CRF has the shortcoming of occasionally generating illegal sequences
of tags, e.g. sequences containing an "I-" tag immediately after an "O" tag,
which is forbidden by the underlying BIO tagging scheme. In this work, we
propose Masked Conditional Random Field (MCRF), an easy to implement variant of
CRF that impose restrictions on candidate paths during both training and
decoding phases. We show that the proposed method thoroughly resolves this
issue and brings consistent improvement over existing CRF-based models with
near zero additional cost.
