Named Entity Recognition — Simple Transformers —Flask REST API
Tags: NLP, Deep Learning, Bert, Python, Named Entity Recognition

Named Entity Recognition — Simple Transformers —Flask REST API Soumibardhan Follow Jun 26 · 4 min read

Image by author

Named-entity recognition (NER) is a subtask of information extraction that seeks to locate and classify named entities mentioned in unstructured text into pre-defined categories such as person names, organizations, locations, medical codes.

I wanted to start with NER, I looked at SpaCy and nltk. SpaCy already has a pre trained NER model, which can be custom trained. Started with that approach. Then I decided to explore transformers in NLP.

I looked for something which can be implemented fast and found out the amazing Simple Transformers library created by Thilina Rajapakse.

Simple Transformers lets you quickly train and evaluate Transformer models.The Simple Transformers library is built on top of the Transformers library by Hugging Face.

The process of Named Entity Recognition has four steps:

Obtaining training and testing data

Preprocessing the data into dataframes required for the model inputs

Training the model

Testing the model

Tuning the model further for better accuracy

Creating an interface for the model

Obtaining training and testing data

Regarding obtaining the data, the coNLL data set can be used to start with. I was given the data from a company for a project, so the data isn’t publicly available.

So here is the format we need the data in: A list of lists with three columns, sentence number, word and the tag. Same for the test data as well.

Image by author

I tried it in google colab, and connected drive for getting the data. Here is the format my dataset was in, which if a real life dataset is constructed manually will look like.

My raw data was in this format :

Text : sentence

Rest unnamed columns :tags corresponding to each tag in the sentence

Preprocessing the data into dataframes required for the model inputs

To parse the data in the format required by simple transformers, I divided the data in 85:15 ratio and wrote a parser.

df: dataframe loads the data CSV file into a pandas dataframe. Sentences : List to store the Text column of the dataframe df['Labels'] : New column in dataframe to add the strings of all tags in the unnamed columns of the dataframe Labels : List to store the Labels column of the dataframe traindata : List of lists with each list: [sentence number,word,tag]

from 0th sentence to 94th sentence. testdata : List of lists with each list: [sentence number,word,tag]

from 95th sentence to 106th sentence.

Training the model

Training the model is real easy, though it may take some time.

The first trial gave very low f1 score, that was because by default the number of epochs was set to one.

Here I used the bert-base-cased model to train on as my data was cased. Model is saved to MODEL folder so that we do not need to train the model everytime to test it. We can load the trained model directly from the folder.



train_batch_size : Batch size, preferably a power of 2 gives good results



Testing the model

Now check the outputs by passing any arbitary sentence :

In place of ‘tax rate’, any arbitrary sentence can be entered to check output tags.

Tuning the model further for better accuracy

Now lets try tuning the Hyper Parameters (args). These are the args that are available :

Here is the link to all the hyperparameters.

Increasing the number of epochs and max_seq_length helped in getting the accuracy to 86%. By using a large model (roBERTa large), I was able to achieve 92.6% accuracy on the test data.

Outputs:

Image by author

INFO:simpletransformers.ner.ner_model:{'eval_loss': 0.6981344372034073, 'precision': 0.9069767441860465, 'recall': 0.9512195121951219, 'f1_score': 0.9285714285714286}

There are several pretrained models that can be used. You can choose one depending on whether your data is in a specific language, or if your data is cased. My training data was financial and in English. So BERT and roBERTa were my best options. I did try distilBERT as well.

Supported model types for Named Entity Recognition in Simple Transformers:

BERT

CamemBERT

DistilBERT

ELECTRA

RoBERTa

XLM-RoBERTa

Creating an interface for the model

I worked on creating an interface for the model. Using Flask and Rest API, I created a web page with two buttons, one to display the results of the sentence entered (GET NER TAGS).

Here is the Flask app.py code :

Make sure you use the same args to load the model as used to train the model.

When you click on the (GET CSV) button, a csv of the words in the sentence and the respective tags gets downloaded automatically.

Image by author

Structure of tags.csv:

Image by author

Simple transformers is a really awesome tool!