Chunking in NLP: decoded. When I started learning text processing... | by Nikita Bachani | Towards Data Science

When I started learning text processing, the one topic on which I stuck for long is chunking. (I know, it’s strange to believe🙆)Usually, we can find many articles on the web from easy to hard topic, but when it comes to this particular topic I felt there’s no single article concludes overall understanding about chunking, however below piece of writing is an amalgamation of all articles or videos I have studied so far, related to the topic.
So here’s my understanding about chunking.
Chunking is a process of extracting phrases from unstructured text, which means analyzing a sentence to identify the constituents(Noun Groups, Verbs, verb groups, etc.) However, it does not specify their internal structure, nor their role in the main sentence.
It works on top of POS tagging. It uses POS-tags as input and provides chunks as output.
In short, Chunking means grouping of words/tokens into chunks
I used to think that normally text processing is done by simply breaking the sentence into words before I was introduced to such topics further. so simply breaking into words isn’t very helpful. It’s very crucial to know that sentence involves a person, a date, places, etc.. (different entities). so they alone are of no use.
Chunking can break sentences into phrases that are more useful than individual words and yield meaningful results.
Chunking is very important when you want to extract information from text such as locations, person names. (entity extraction)
Let’s understand it from scratch.
A sentence typically follows a hierarchical structure consisting of the following components.
Group of words make up phrases and there are five major categories.
Noun Phrase (NP)
Verb phrase (VP)
Adjective phrase (ADJP)
Adverb phrase (ADVP)
Prepositional phrase (PP)
Phrase structure rules:
S -> NP VP
NP -> {Det N,Pro,PN}
VP -> V (NP) (PP) (Adv)
PP -> P NP
AP -> A (PP)
Before diving deeper into chunking, It’s good to have a brief knowledge about syntax tree and grammar rules.
As we can see, Here the whole sentence is divided into two different chunks which are NP(noun phrase).
Now lets under understand this concept with python experiment.
Regex based chunking
Regex based chunking
Code snippet for chunker based on regex pattern
Here, we have introduced a grammar.in which NP (Noun phrase) is combined byDT? → one or zero determinerJJ* → zero or more adjectivesNN → Noun
and we parse this grammar by NLTK defined regular expression parser. As we can see, Whole sentence S is divided into chunks and represented in tree-like structures. Based on defined grammar, an internally tree-like structure is created. So you can define your grammar, based on that sentence will be chunked.
2. Training tagger based chunker
I’ve used the `conll2000` corpus for training chunker. conll2000 corpus defines the chunks using IOB tags.
It specifies where the chunk begins and ends, along with its types.A POS tagger can be trained on these IOB tags
The chunk tags use the IOB format.IOB : Inside,Outside,BeginningB- prefix before a tag indicates, it’s the beginning of a chunkI- prefix indicates that it’s inside a chunkO- tag indicates the token doesn’t belong to any chunk
#Here conll2000 corpus for training shallow parser modelnltk.download('conll2000')from nltk.corpus import conll2000data= conll2000.chunked_sents()train_data=data[:10900]test_data=data[10900:]print(len(train_data),len(test_data))print(train_data[1])
tree2conlltags,conlltags2tree are chunking utility functions.
→ `tree2conlltags`, to get triples of (word, tag, chunk tags for each token). These tuples are then finally used to train a tagger and it learns IOB tags for POS tags.
→ `conlltags2tree` to generate a parse tree from these token triplesConlltags2tree() is reversal of tree2conlltags().We’ll be using these functions to train our parser
from nltk.chunk.util import tree2conlltags,conlltags2treewtc=tree2conlltags(train_data[1])wtc
tree=conlltags2tree(wtc)print(tree)
def conll_tag_chunks(chunk_sents):    tagged_sents = [tree2conlltags(tree) for tree in chunk_sents]    return [[(t, c) for (w, t, c) in sent] for sent in tagged_sents]def combined_tagger(train_data, taggers, backoff=None):    for tagger in taggers:        backoff = tagger(train_data, backoff=backoff)    return backoff
It reads text and assigns a POS tag to each word. (word, tag)
Unigram tagger: For determining the POS, It only uses a single word. (single word context-based tagger)
The UnigramTagger, BigramTagger, and TrigramTagger are classes that inherit from the base class NGramTagger, which itself inherits from the ContextTagger class, which inherits from the SequentialBackoffTagger class
We will now define a class NGramTagChunker that will take in tagged sentences as training input, get their (word, POS tag, chunk tag)WTC triples and train a BigramTagger with a UnigramTagger as the backoff tagger.
We will also define a parse() function to perform shallow parsing on a new sentence.
from nltk.tag import UnigramTagger, BigramTaggerfrom nltk.chunk import ChunkParserI#Define the chunker classclass NGramTagChunker(ChunkParserI):  def __init__(self,train_sentences,tagger_classes=[UnigramTagger,BigramTagger]):    train_sent_tags=conll_tag_chunks(train_sentences)    self.chunk_tagger=combined_tagger(train_sent_tags,tagger_classes)def parse(self,tagged_sentence):    if not tagged_sentence:      return None    pos_tags=[tag for word, tag in tagged_sentence]    chunk_pos_tags=self.chunk_tagger.tag(pos_tags)    chunk_tags=[chunk_tag for (pos_tag,chunk_tag) in chunk_pos_tags]    wpc_tags=[(word,pos_tag,chunk_tag) for ((word,pos_tag),chunk_tag) in zip(tagged_sentence,chunk_tags)]    return conlltags2tree(wpc_tags)#train chunker modelntc=NGramTagChunker(train_data)#evaluate chunker model performanceprint(ntc.evaluate(test_data))
Now we’ll leverage this model to shallow parse and chunk our sample news article headline.
import pandas as pdsentence='No new emoji may be released in 2021 due to COVID-19 pandemic word'nltk_pos_tagged=nltk.pos_tag(sentence.split())pd.DataFrame(nltk_pos_tagged,columns=['word','POS tag'])
chunk_tree=ntc.parse(nltk_pos_tagged)print(chunk_tree)
chunk_tree
You can also define classifier based chunker according to need. You can read more about it here.
https://www.geeksforgeeks.org/nlp-classifier-based-chunking-set-1/?ref=rp
We create a sequence to tokens, which are not included in the chunk. So it’s finding insights or context. (I am not covering in this section)
Thank you for reading.🙏
I’ve tried to cover this topic in maximum depth. Suggestions are welcome.
As an honorable mention, I would like to thank Dipanjan (DJ) Sarkar.I’ve been following his tutorial to learn NLP from scratch.
P.S This is my first technical writeup. Hopefully I’ll continue to write as I progress in my learning.