This library includes processors for several traditional tasks. These processors can be used to process a dataset into examples that can be fed to a model.


All processors follow the same architecture which is that of the DataProcessor. The processor returns a list of InputExample. These InputExample can be converted to InputFeatures in order to be fed to the model.


General Language Understanding Evaluation (GLUE) is a benchmark that evaluates the performance of models across a diverse set of existing NLU tasks. It was released together with the paper GLUE: A multi-task benchmark and analysis platform for natural language understanding

This library hosts a total of 10 processors for the following tasks: MRPC, MNLI, MNLI (mismatched), CoLA, SST2, STSB, QQP, QNLI, RTE and WNLI.

Those processors are:
  • MrpcProcessor

  • MnliProcessor

  • MnliMismatchedProcessor

  • Sst2Processor

  • StsbProcessor

  • QqpProcessor

  • QnliProcessor

  • RteProcessor

  • WnliProcessor

Additionally, the following method can be used to load values from a data file and convert them to a list of InputExample.

Example usage

An example using these processors is given in the script.