Why Data is represented as a ‘Vector’ in Data Science Problems?
By: Gaurav Goel
Tags: Machine Learning, Naturallanguageprocessing, Vector Algebra, Data Science, Text Mining

Why Data is represented as a ‘Vector’ in Data Science Problems?

Applications of Vector Algebra in Data Science

Photo by Antoine Dautry on Unsplash

Science and Maths are used to describe the world around us. There are many quantities which require only 1 measurement to describe them. e.g Length of a string, or area of any shape or temperature of any surface. Such quantities are called scalars. Any quantity which can be represented as a number (positive or negative) is called scalar. This value is known as magnitude.

On the other hand, there are quantities which require at least 2 measurements to describe them. Along with the magnitude, they have a “direction” associated e.g velocity or force. These quantities are known as “Vectors”.

When we say that a person ran for 2 Kms, its a scalar but when we say that a person ran for 2 Kms, North-east from his initial position, its a vector.

Let's say you are collecting some data about a group of students in a class. You are measuring the height and weight of each student and the data collected for 5 students is as follows:

Each individual measurement here is a scalar quantity. So height or weight viewed stand-alone are scalars.

However, when you look at the observation about each student as a whole i.e height and weight together for every student, you can think of it as a vector.

In this representation, each student is a vector. It is described by 2 characteristics: height and weight. Now, there is no real “direction” here in this vector as per the standard definition. But when we represent quantities in more than one dimension (in this case, it's 2 — height and weight), there is a sense of orientation with respect to each other. So, when we are looking at 5 students (5 vectors), they have a magnitude and an idea of “direction” with respect to each other.

Since most useful datasets always have more than 1 attribute, they all can be represented as “Vectors”. Every observation in a given data set can be thought of as a vector. All possible observations of a data set constitute a “vector space”. Its a fancy way of saying that there is a space out there and every vector has its location within that space.

The benefit of representing data as vectors is — we can leverage vector algebra to find patterns or relationships within our data.

How can vectors help?

Photo by Sharon McCutcheon on Unsplash

Looking at the vectors in vector space, you can quickly compare them to check if there is a relationship. For example, you deduce that similar vectors will have smaller angle between them i.e their orientation will be close to each other. In our sample data, the students (5.4,54) and (5, 50) are quite similar.

The angle between the vectors indicates “similarity” between them. The vectors in the same direction (close to 0 degrees angle) are similar while vectors in the opposite direction (close to 180 degrees angle) are dissimilar. Theoretically. if the vectors are at 90 degrees to each other (orthogonal), then there is no relationship between them. Is there a mathematical function which can capture this relationship?

Think of the trigonometric function — cosine.

The above description may sound a little arcane. So, don’t forget that we are talking here about “representation” of data in such a way that it helps us find a relationship or pattern. Vector form is one such representation.

Cosine Similarity

Cosine Similarity is a metric that gives the cosine of the angle between vectors. It can be calculated using the “dot product” of 2 vectors. Mathematically,

So now whenever you hear about vectors in Data Science, think about the dot product and cosine of the angle between them. You will be looking at how similar or dissimilar those vectors are.

Text Vectorization

Photo by Kyle Glenn on Unsplash

The process of converting or transforming a data set into a set of Vectors is called Vectorization. It’s easier to represent data set as vectors where attributes are already numeric. What about textual data?

“Word Embedding” is the process of representing words or text as Vectors

There are a number of techniques of converting/representing text as Vectors. One of the simplest methods is Count Vectorizer. Below is a snippet of the first few lines of text from the book “A Tale of Two Cities” by Charles Dickens:

It was the best of times,

it was the worst of times,

it was the age of wisdom,

it was the age of foolishness

How can the above 4 sentences be converted into vectors:

Step 1: Get unique words from the collection of your text. The total text you have is called “corpus”

The unique words here (ignoring case and punctuation) are:

['age',

'best',

'foolishness',

'it',

'of',

'the',

'times',

'was',

'wisdom',

'worst']

This is a vocabulary of 10 words from a corpus containing 24 words.

Step 2: For each sentence, create a list of 10 zeroes

Step 3: For each sentence, start reading the word one by one. For each word, count total occurrence in the sentence. Now identify the position of the word in vocabulary list above and replace the zero with this count at that position.

For our corpus, the vectors we got are:

“It was the best of times“= [0 1 0 1 1 1 1 1 0 0]

“it was the worst of times” = [0 0 0 1 1 1 1 1 0 1]

“it was the age of wisdom” = [1 0 0 1 1 1 0 1 1 0]

“it was the age of foolishness” = [1 0 1 1 1 1 0 1 0 0]

Code in Python:

from sklearn.feature_extraction.text import CountVectorizer documents = [‘it was the best of times’, ‘it was the worst of times’,’it was the age of wisdom’,’it was the age of foolishness’] vectorizer = CountVectorizer()

sentence_vectors = vectorizer.fit_transform(documents) print(sentence_vectors.toarray())

The output:

A pure Python code (without using any in-built library) to calculate cosine similarity between 2 sentences:

import math

import re

from collections import Counter WORD = re.compile(r”\w+”) def cosine(vector1, vector2):

intersect = set(vector1.keys()) & set(vector2.keys())

numerator = sum([vector1[x] * vector2[x] for x in intersect]) sum1 = sum([vector1[x] ** 2 for x in list(vector1.keys())])

sum2 = sum([vector2[x] ** 2 for x in list(vector2.keys())])

denominator = math.sqrt(sum1) * math.sqrt(sum2) if not denominator:

return 0.0

else:

return float(numerator) / denominator def text_to_vector(text):

words = WORD.findall(text)

return Counter(words) sentence1 = “it was the age of foolishness”

sentence2 = “it was the age of wisdom” vector1 = text_to_vector(sentence1)

vector2 = text_to_vector(sentence2) cosine = cosine(vector1, vector2) print(“Cosine Similarity:”, cosine)

The cosine similarity between ’it was the age of wisdom’ and ’it was the age of foolishness’ is

The cosine similarity will be a number between 0 and 1. The more the result is close to 1, the greater the similarity.

0.83 value suggests that the two sentences in this example are pretty similar. This is because the words - ‘it’, ‘was’, ‘the’, ‘age’, ‘of’ are repeating in both sentences. However, the last word ‘wisdom’ and ‘foolishness’ completely changes the meaning of sentences and they are pretty dissimilar in this respect. What is the problem? The problem lies in the “vectorization” method used. To capture the semantic meaning of the sentences, we need to use more advanced transformation techniques of words into vectors. I will cover another vectorization technique in the next article which handles this scenario.

Conclusion

The purpose of this article is to provide the intuition behind the use of vectors to represent data. The motive of Data Science or Machine Learning is to find out the patterns and relationship between observations. Visualizing data as vectors and using vector algebra to manipulate them, solves a lot of data problems esp in the field of Natural language Processing, Text Classification and Text Analysis.

Ending this on a philosophical note -

“We all are vectors. Many of us are still in search of direction”

You may also like below articles: