FLAIR - A Framework for NLP - GeeksforGeeks

26 Nov, 2020
What is FLAIR?
It is a simple framework for state-of-the-art NLP. It is a very powerful library which is developed by Zalando Research. The Flair framework is built on top of PyTorch.
What are the Features available in Flair?
Flair supports a number of word embeddings used to perform NLP tasks such as FastText, ELMo, GloVe, BERT and its variants, XLM, and Byte Pair Embeddings including Flair Embedding.The Flair Embedding is based on the concept of contextual string embeddings which is used for Sequence Labelling.Using Flair you can also combine different word embeddings together to get better results.Flair supports a number of languages.
Flair supports a number of word embeddings used to perform NLP tasks such as FastText, ELMo, GloVe, BERT and its variants, XLM, and Byte Pair Embeddings including Flair Embedding.
The Flair Embedding is based on the concept of contextual string embeddings which is used for Sequence Labelling.
Using Flair you can also combine different word embeddings together to get better results.
Flair supports a number of languages.
Contextual String Embeddings:
In this word embedding each of the letters in the words are sent to the Character Language Model and then the input representation is taken out from the forward and backward LSTMs.
The input representation for the word ‘Washington’ is been considered based on the context before the word ‘Washington’. The first and last character states of each word is taken in order to generate the word embeddings.
You can see that for the word ‘Washington’ the red mark is the forward LSTM output and the blue mark is the backward LSTM output. Both forward and backward contexts are concatenated to obtain the input representation of the word ‘Washington’.
After getting the input representation it is fed to the forward and backward LSTM to get the particular task that you are dealing with. In the diagram mentioned we are trying to get the NER.
Installation of Flair:
You should have PyTorch >=1.1 and Python >=3.6 installed. To install PyTorch on anaconda run the below command-
conda install -c pytorch pytorch

To install flair, run –
pip install flair

Working of Flair
1) Flair Datatypes:
Flair offers two types of objects. They are:
SentenceTokens
Sentence
Tokens
To get the number of tokens in a sentence:
Python3
import flairfrom flair.data import Sentence  # take a sentences= Sentence('GeeksforGeeks is Awesome.')print(s)
Output:
2) NER Tags:
To predict tags for a given sentence we will use a pre-trained model as shown below:
Python3
import flairfrom flair.data import Sentencefrom flair.models import SequenceTagger  # input a sentences = Sentence('GeeksforGeeks is Awesome.')  # loading NER taggertagger_NER= SequenceTagger.load('ner')  # run NER over sentencetagger_NER.predict(s)print(s)print('The following NER tags are found:\n')  # iterate and printfor entity in s.get_spans('ner'):    print(entity)
Output:
3) Word Embeddings:
Word embeddings give embeddings for each word of the text. As discussed earlier Flair supports many word embeddings including its own Flair Embeddings. Here we will see how to implement some of them.
A) Classic Word Embeddings – This class of word embeddings are static. In this, each distinct word is given only one pre-computed embedding. Most of the common word embeddings lie in this category including the GloVe embedding.
Python3
import flairfrom flair.data import Sentencefrom flair.embeddings import WordEmbeddings  # using glove embeddingGloVe_embedding = WordEmbeddings('glove')  # input a  sentences = Sentence('Geeks for Geeks helps me study.')  # embed the sentenceGloVe_embedding.embed(s)  # print the embedded tokensfor token in s:    print(token)    print(token.embedding)
Output:
Note: You can see here that the embeddings for the word ‘Geeks‘ are the same for both the occurrences.
B) Flair Embedding – This works on the concept of contextual string embeddings. It captures latent syntactic-semantic information. The word embeddings are contextualized by their surrounding words. It thus gives different embeddings for the same word depending on it’s surrounding text.
Python3
import flairfrom flair.data import Sentencefrom flair.embeddings import FlairEmbeddings  # using forward flair embeddingembeddingforward_flair_embedding= FlairEmbeddings('news-forward-fast')  # input the sentences = Sentence('Geeks for Geeks helps me study.')  # embed words in the input sentenceforward_flair_embedding.embed(s)  # print the embedded tokensfor token in s:    print(token)    print(token.embedding)
Output:
Note: Here we see that the embeddings for the word ‘Geeks’ are different for both the occurrences depending on the contextual information around them.
C) Stacked Embeddings – Using these embeddings you can combine different embeddings together. Let’s see how to combine GloVe, forward and backward Flair embeddings:
Python3
import flairfrom flair.data import Sentencefrom flair.embeddings import FlairEmbeddings, WordEmbeddingsfrom flair.embeddings import StackedEmbeddings# flair embeddingsforward_flair_embedding= FlairEmbeddings('news-forward-fast')backward_flair_embedding= FlairEmbeddings('news-backward-fast')  # glove embeddingGloVe_embedding = WordEmbeddings('glove')  # create a object which combines the two embeddingsstacked_embeddings = StackedEmbeddings([forward_flair_embedding,                                        backward_flair_embedding,                                        GloVe_embedding,])  # input the sentences = Sentence('Geeks for Geeks helps me study.')                                          # embed the input sentence with the stacked embeddingstacked_embeddings.embed(s)  # print the embedded tokensfor token in s:    print(token)    print(token.embedding)
Output:
4) Document Embeddings:
, Unlike word embeddings, document embeddings give a single embedding for the entire text. The document embeddings offered in Flair are:
A) Transformer Document Embeddings
B) Sentence Transformer Document Embeddings
C) Document RNN Embeddings
D) Document Pool Embeddings
Let’s have a look at how the Document Pool Embeddings work-
Document Pool Embeddings —  It is a very simple document embedding and it pooled over all the word embeddings and returns the average of all of them.
Python3
import flairfrom flair.data import Sentencefrom flair.embeddings import WordEmbeddings, DocumentPoolEmbeddings  # init the glove word embeddingGloVe_embedding = WordEmbeddings('glove')  # init the document embeddingdoc_embeddings = DocumentPoolEmbeddings([GloVe_embedding])  # input the sentences = Sentence('Geeks for Geeks helps me study.')                                          #embed the input sentence with the document embeddingdoc_embeddings.embed(s)  # print the embedded tokensprint(s.embedding)
Output:
Similarly, you can use other Document embeddings as well.
5) Training a Text Classification Model using Flair:
We are going to use the ‘TREC_6’ dataset available in Flair. You can also use your own datasets as well. To train our model we will be using the Document RNN Embeddings which trains an RNN over all the word embeddings in a sentence. The word embeddings which we will be using are the GloVe and the forward flair embedding.
Python3
from flair.data import Corpusfrom flair.datasets import TREC_6from flair.embeddings import WordEmbeddings, FlairEmbeddings, DocumentRNNEmbeddingsfrom flair.models import TextClassifierfrom flair.trainers import ModelTrainer  # load the corpuscorpus = TREC_6()  # create a label dictionarylabel_Dictionary = corpus.make_label_dictionary()  # list of word embeddings to be usedword_embeddings = [WordEmbeddings('glove'),FlairEmbeddings('news-forward-fast')]  # init document embeddings and pass the word embeddings listdoc_embeddings = DocumentRNNEmbeddings(word_embeddings,hidden_size = 250)  # creating the text classifiertext_classifier = TextClassifier(doc_embeddings,label_dictionary = label_Dictionary)  # init the text classifier trainermodel_trainer = ModelTrainer(text_classifier,corpus)  # train your modelmodel_trainer.train('resources/taggers/trec',learning_rate=0.1,mini_batch_size=40,anneal_factor=0.5,patience=5,max_epochs=200)
Results of training:
The accuracy of the model is around 95%.
Predictions: Now we can load the model and make predictions-
Python3
from flair.data import Sentencefrom flair.models import TextClassifierc = TextClassifier.load('resources/taggers/trec/final-model.pt')  # input example sentences = Sentence('Who is the President of India ?')  # predict class and printc.predict(s)  # print the labelsprint(s.labels)
Output:
[HUM (1.0)]
Now you would have got a rough idea of how to use the Flair library.
Natural-language-processing
Machine Learning
Python
Machine Learning
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Support Vector Machine Algorithm
Intuition of Adam Optimizer
Introduction to Recurrent Neural Network
CNN | Introduction to Pooling Layer
Singular Value Decomposition (SVD)
Read JSON file using Python
Adding new column to existing DataFrame in Pandas
Python map() function
How to get column names in Pandas dataframe