Linguistic Knowledge in Natural Language Processing
By: Hafidz Zulkifli
Tags: Naturallanguageprocessing, Linguistics, NLP, Data Science, Machine Learning

Ever since diving into Natural Language Processing (NLP), I’ve always wanted to write something rather introductory about it at a high level, to provide some structure in my understanding, and to give another perspective of the area — in contrast to the popularity of doing NLP using Deep Learning.

Stages of analysis in NLP

NLP Pyramid — Coursera Natural Language Processing

Given a sentence, traditionally the following are the different stages on how a sentence would be analyzed to gain deeper insights.

1. Morphology

At this stage we care about the words that make up the sentence, how they are formed, and how do they change depending on their context. Some examples of these include:

Prefixes/suffixes

Singularization/pluralization

Gender detection

Word inflection (modification of word to express different grammatical categories such tenses, case, voice etc..). Other forms of inflection includes conjugation (inflection of verbs) and declension (inflection of nouns, adjectives, adverbs etc…).

Lemmatization (the base form of the word, or the reverse of inflection)

Spell checking

Morphological analysis of a sentence — Spacy.io

2. Syntax (Parsing)

In this stage, we focus more on the relationship of the words within a sentence — how a sentence is constructed.

In a way, syntax is what we usually refer to as grammar — NLP For Hackers

To derive this understanding, syntactical analysis is usually done at a sentence-level, where as for morphology the analysis is done at word level. When we’re building dependency trees or processing parts-of-speech — we’re basically analyzing the syntax of the sentence.

Dependency tree of a sentence — Spacy.io

3. Semantics

Once we’ve understood the syntactic structures, we are more prepared to get into the “meaning” of the sentence (for a fun read on what meaning can actually mean in NLP — head over here to dive into a Twitter discussion on the subject ).

Some example of tasks performed at this stage include:

Named Entity Recognition (NER)

Relationship Extraction

Named entity recognition of a sentence — Spacy.io

In the above example from Spacy for example; Google, Apple and Amazon has been tagged as an organization based on it’s NER annotator. While Google and Amazon is a straightforward case, Apple is rather different since it could either be a fruit or a the company — Apple. In this case, the result given is based off of a prediction based on statistical analysis from trained dataset.

4. Pragmatics

At this level, we try to understand the text as a whole. Popular problems that we’re trying to solve at this stage are:

Topic modelling Coreference/Anaphora Summarization Question & Answering

Summary

NLP as an area of study isn’t something that’s actually new. And if one is already well versed in Deep Learning, one could also be forgiven to think that traditional NLP methods (ie. breaking texts into smaller structures, parsing, annotate meanings) are no longer needed.

However in cases where labelled data are hard to come by, and high precision is needed (for example when one is hard pressed for time with a tight deadline and the results really need to be accurate else the product wouldn’t be launched) — is where traditional NLP really shines.

Understanding the above topics aren’t really something that is needed for one to be able to solve NLP-related tasks. For me, it’s more of a way to put into perspective where does the different problems that I’ve solved previously belong to and what are the other gaps or areas where I could improve on.

Disclaimer: In writing this I did a lot of read up on Coursera’s NLP course, Wikipedia, and my latest favourite website for NLP materials — NLP for Hackers. Most likely some of the stuff you see here are a mishmash between those sources, so be forewarned that I might forget to properly reference their content where needed.

References:

[1] Natural Language Processing | Coursera

[2] Wikipedia, the free encyclopedia

[3] Natural Language Processing — Introduction — NLP-FOR-HACKERS

[4] Linguistic Features, Spacy.io