Metric: seqeval
Update on GitHub


seqeval is a Python framework for sequence labeling evaluation. seqeval can evaluate the performance of chunking tasks such as named-entity recognition, part-of-speech tagging, semantic role labeling and so on. This is well-tested by using the Perl script conlleval, which can be used for measuring the performance of a system that has processed the CoNLL-2000 shared task data. seqeval supports following formats: IOB1 IOB2 IOE1 IOE2 IOBES See the [] file at for more information.

How to load this metric directly with the datasets library:

from datasets import load_metric
metric = load_metric("seqeval")


    title = "Text Chunking using Transformation-Based Learning",
    author = "Ramshaw, Lance  and
      Marcus, Mitch",
    booktitle = "Third Workshop on Very Large Corpora",
    year = "1995",
    url = "",
  title={{seqeval}: A Python framework for sequence labeling evaluation},
  note={Software available from},
  author={Hiroki Nakayama},