So... What’s spaCy?. And why it’s important in digital... | by Admond Lee | Towards Data Science

If a normal data analysis tool in Python for tabular and structured data has Pandas, then the data analysis tool in Natural Language Processing (NLP) for text and unstructured data has spaCy.
You see.
When you’re first starting out as a data scientist, chances are you’ll be dealing with structured data which doesn’t necessarily require spaCy to handle complicated text data, depending on your needs.
In most cases — at least in my opinion — Pandas will fit your usage as it’s powerful enough to do most of the data cleaning and analysis for structured data.
Once you start dealing with unstructured text data — basically NLP stuff — where this can no longer be handled by Pandas, this is when spaCy comes in with tons of in-built capabilities to process, analyze and even understand these data through sophisticated and efficient NLP techniques.
NLP is a subfield of Artificial Intelligence and is concerned with interactions between computers and human languages.
NLP is the process of analyzing, understanding, and deriving meaning from human languages for computers.
In fact, some time back when I was working with one of my clients to understand what words mean in context and how texts are related to one another.
That was when I chanced upon spaCy and I never looked back ever since.
By the end of this article, I hope you’ll understand more about spaCy and how you could leverage this powerful tool in your digital marketing space as well as other areas.
Let’s get started!
Maybe you’re a data scientist wanting to know more about spaCy to use in your projects.
Maybe you’re a company planning to leverage NLP techniques in your day-to-day work, especially in digital marketing space.
Or maybe you’re just someone who’s interested in the popular libraries available in the current state-of-the-art NLP.
I’d highly recommend you to check out spaCy as you can analyze large volumes of text and derive insights from unstructured data through statistical modeling and advanced deep learning approach.
It’s a free and open-source Python library designed specifically for production use in NLP applications. Its easy-to-use features and intuitive APIs are what make it highly popular and attractive.
Perhaps now you may be asking, “So what exactly can spaCy do for me?”
My answer is, “It depends on what you need.”
There are too many functionalities that spaCy has and I’ll only list some of the key features for your reference:
Tokenization is always the first step before we can do any text data processing. What this means is that spaCy will segment sentences into words, punctuations, symbols and others by applying specific rules to each language.
import spacynlp = spacy.load("en_core_web_sm")doc = nlp("Apple is looking at buying U.K. startup for $1 billion")for token in doc:    print(token.text)
Once we have tokenized words, the typical next step would be to perform Part-of-speech (POS) tagging to understand the grammatical properties (noun, verb, adjective etc.) of each word.
import spacynlp= spacy.load("en_core_web_sm")doc= nlp("She ate the pizza")for token in doc:    print(token.text, token.pos_)
One of the main reasons of doing POS tagging is to identify similar syntactic structure for rule-based processes, which I’ll explain later.
For now, just take POS tagging as a way to understand words in the context better.
Named entities are “real world objects” that are assigned a name — for example, a person, an organization or a country.
In other words, we want to categorize words into certain generic names for further processes. For example:
import spacynlp= spacy.load("en_core_web_sm")doc= nlp(u"Apple is looking at buying U.K. startup for $1 billion")for ent in doc.ents:    print(ent.text, ent.label_)
In this sentence, “Apple is looking at buying U.K. startup for $1 billion”, we want to assign name entities to certain words. With the pre-trained model’s prediction:
“Apple” is recognized as “Companies, agencies, institutions, etc.”
“U.K.” is recognized as “Countries, cities, states”
“$1” is recognized as “Monetary values, including unit”
However, the prediction output by the model doesn’t work perfectly as the model was trained on certain specific datasets and therefore further model training/tuning is needed, depending on your use cases.
Similarity can be determine between words, sentences, paragraphs or even whole documents/articles.
It’s computed using word vectors, which are essentially multi-dimensional representations of meanings of words in the format of vectors/matrices.
By default, the similarity returned by spaCy is the cosine similarity between two vectors — but this can be adjusted if necessary.
import spacy# Load a larger model with vectorsnlp = spacy.load('en_core_web_md')# Compare two documentsdoc1 = nlp("I like fast food")doc2 = nlp("I like pizza")print(doc1.similarity(doc2))
Using spaCy, we can calculate the similarity score with just one line of code and the score for the two sentences above is 86%.
In this case, the similarity score kind of makes sense as the two sentences are both talking about food.
In fact, predicting similarity can be useful for many types of applications. For example, to recommend a user similar articles based on the ones they have read. It can also be helpful to flag duplicate or similar content on social media, like posts on Facebook and Twitter.
Believe it or not, traditional marketing will be giving way to data-driven marketing with the rise of data science and big data ecosystem, sooner or later.
With billions of text data generated every day, more advanced tools are needed to analyze these big data with the unprecedented speed to generate actionable insights for companies and agencies.
SpaCy is precisely built to handle huge volumes of data at ease in production.
Once you know how to leverage this tool’s functionalities, the possibilities are limitless, including sentiment analysis and building recommendation systems at large scale.
One of the many examples — I’m pretty sure you’ve encountered this before — would be to suggest users similar content to what they’re currently looking at.
If you’re currently scrolling Facebook or Medium, the recommendation system is constantly running behind the scene to show you the most relevant and similar posts or articles that you’ll be most interested in.
People like personalization.
If you can leverage spaCy (or any other tools) to understand how your customers are feeling towards your services and push relevant content for them to consume, chances are they’d be more satisfied and happy with your services.
And this is just one of the many cool things that spaCy is capable of.
Thank you for reading.
If you’ve made it until here, I hope by now you’ve learned more about spaCy and more features would be included in future as it’s still under active development contributed by many brilliant developers around the world.
At the end of the day, spaCy is just one of the many tools being used by many companies and agencies to analyze text data and leverage advanced NLP for prediction.
NLP is no longer just the future. It’s already here.
And it’s important to understand how you can use that to your projects and work, especially when lots of text data are involved.
As always, if you have any questions or comments feel free to leave your feedback below or you can always reach me on my website or LinkedIn. Till then, see you in the next post! 😄
Admond Lee is now in the mission of making data science accessible to everyone. He is helping companies and digital marketing agencies achieve marketing ROI with actionable insights through NLP and innovative data-driven approach.
With his expertise in advanced social analytics and machine learning, Admond aims to bridge the gaps between digital marketing and data science.
Check out his website if you want to understand more about Admond’s story, data science services, and how he can help you in marketing space through data-driven approach.
You can connect with him on LinkedIn, Medium, Twitter, and Facebook.