Let’s Understand the Vector Space Model in Machine Learning by Modelling Cars | by GreekDataGuy | Towards Data Science

According to Wikipedia.
Vector space model or term vector model is an algebraic model for representing text documents (and any objects, in general) as vectors of identifiers, such as, for example, index terms.
Translation: We represent each example in our dataset as a list of features.
The model is used to represent documents in an n-dimensional space. But a “document” can mean any object you’re trying to model.
Whether you explicitly understand this or not, you’ve used it in your machine learning projects.
Just a feature of the objects we’re modelling. For example.
Houses could have dimensions: number_of_rooms, sale_price, construction_date, latitude and longitude.
People could have dimensions: age, weight, height, hair_color_is_blue and hair_color_is_other.
Sentences could have a dimension for each possible word/token.
Cars could have dimensions: max_speed, acceleration_time and price.
In the end, features/dimensions are whatever you decide on during the feature selection process.
Our dimensions will be:1. max_speed (maximum speed in kmph)2. acceleration_time (number seconds to reach 100 kmph)3. price (in USD)
I’ve picked 3 cars and collected their information (disclaimer: I know nothing about cars...).
2020 Porsche Cayennemax_speed: 304 kmphacceleration_time: 4.4sprice: $99,200
2018 Tesla Model Smax_speed: 250 kmphacceleration_time: 3.8sprice: $79,990
2020 BMW i3smax_speed: 160 kmphacceleration_time: 7sprice: $60,000
So what do the vectors look like?porsche = (304, 4.4, 99200)tesla = (250, 3.8, 79990)bmw = (160, 7, 60000)
Let’s plot these.
from matplotlib import pyplotfrom mpl_toolkits.mplot3d import Axes3Dfrom numpy.random import randfrom pylab import figure# featuresX = np.array([    [304, 4.4, 99200],    [250, 3.8, 79990],    [160, 7, 60000]])# labelsy = ['porsche', 'tesla', 'bmw']# setup our chartfig = figure()ax = Axes3D(fig)# iterate on the examples and plot themfor i in range(len(X)):     ax.scatter(        X[i,0],        X[i,1],        X[i,2],        color='b')             ax.text(        X[i,0],        X[i,1],        X[i,2],        '%s' % (str(y[i])),         size=20,         zorder=1,        color='k')# label our chartax.set_xlabel('max_speed')ax.set_ylabel('acceleration_time')ax.set_zlabel('price')pyplot.show()
Cool.
It’s a little difficult to see the cars’ orientation to each other, but if we were more ambitious we’d use matplotlib’s “animation” module to view the chart from different angles.
I chose 3 continuous features because its easy to plot. In reality we could have thousands of features, In NLP where I work, this is typically the case.
But there are some mental tricks use can use to visualize more than 3 features. Like imagining each on its own bar scale. So if we had 5 features instead of 3...
It makes it easy to compute similarity between 2 objects, or relevance between a search query and an object.
How similar are our cars from above?
This similarity calculation is going to ignore some key factors (ie. the importance of each feature).
First we’ll scale our data, then calculate cosine similarity, one of the most popular similarity algorithms.
from sklearn.metrics.pairwise import cosine_similarityfrom sklearn.preprocessing import StandardScaler# scale unit variancescaler = StandardScaler()scaler.fit(X)X_scaled = scaler.transform(X)X_scaled# calculate cosine_similaritysimilarities = cosine_similarity(X_scaled,X_scaled)print(similarities)#=>[[ 1.          0.42538808 -0.9284599 ] [ 0.42538808  1.         -0.73110643] [-0.9284599  -0.73110643  1.        ]]
Alright! We can see that the Porsche and Tesla are more similar to each other than to the BMW.
That’s it folks.
This was a quick introduction to the vector space model and what it’s used for. While you’ll touch this with just about any machine learning project, it’s not something you need to consciously think about. And if nothing else, it’s an interesting mental model through which to view the world.