Summarize the following : 
"""
A little personal context

This is my renewed self-torturing attempt at learning machine learning. A couple of months ago, I earned a Machine Learning Engineer Nanodegree from the online nanodegree factory udacity. Nano probably denotes that it is infinitesimally small and insignificant.

Not sure.

Money-wise, it is about 1000 dollars; effort-wise, I put in significant work in 6 months. I sometimes had my CPU work overtime and run full capacity; I had had some great satisfaction to see that my own learning projects bear fruits: identify dog breeds, classify pneumonia, enable a virtual quadcopter hover at about 10 pixels above the take-off spot (All that glory). I now identify myself as sub-literate in the realm of machine learning. Terms like CNN will no longer only stand for that “fake” news work; I can start sounding intelligent to a non-ML person about terms like RNNs, LSTM, GANs, reinforcement learning, etc. But otherwise, my nano-degree changes nothing for me.

Still, in an age of artificial intelligence, not learning is not an option. So I decided to gather myself and carry on, tumble and stumble in the torrent of machine learning theories, technologies, breakthroughs and new frontiers.

Learn for the sake of learning.

I “discovered” fastText through googling of “fun machine learning projects”. Found the article 30 Amazing Machine Learning Projects for the Past Year (v.2018). fastText is the number 1. It has 16, 185 stars Github stars (that is tens of thousands of developers’ vote of confidence) and it is still ascending in its trajectory. Dazzling.

So I must learn.

fastText is a library for efficient learning of word representations and sentence classification.

Word Vectors

Word representations and sentence classification are fundamental to the field of Natural Language Processing (NLP). NLP is to artificial intelligence is language to human civilization. This is the field that allows your Siri, Alexa, Google assistant, the gaggle of voice-enabled assistants to respond to your misspelt, wrongly worded, incomplete, potentially embarrassing questions with clever and informative answers (hopefully). This is what allows Google or Facebook follow you with “intelligent” ads whenever you read a few articles about cooking, fitness and everything else.

Simplistic word representation treats a corpus of text as an indiscriminate bag of words, aka, BoW. BoW cares about only 2 things: a) the list of known words; b) tally of word frequency. It does not care about the syntactic (structure) and semantic (meaning) relationships among the words in the sentence, sentences in the text. For some purposes, this treatment is good enough, for example, spam detection. For others, BoW falls short. Instead, word vectors becomes the state-of-the-art form of word representation.

Word vectors represent words as multidimensional continuous floating point numbers where semantically similar words are mapped to proximate points in geometric space. In simpler terms, a word vector is a row of real valued numbers (as opposed to dummy numbers) where each point captures a dimension of the word’s meaning and where semantically similar words have similar vectors.

It is a mouthful of math terms representing a mathematical universe of infinite dimensions.

When words are vectorized, they can be added or subtracted.

wife + husband + kids = family (hopefully)

Not sure the following math is true:

Mother * Mother = Grandmother?

Anyway, in a word vector, `smart` and intelligent are two words sitting next to each other, cats and dogs occupy the same animal sphere, go, went, gone are parallel to eat , ate and eaten


Word vector (From Introduction to Word Vectors)
There are two main architectures /models used in training large corpus of literature into word vectors: a Continuous Bag-of-Words model (cbow) and a Continuous Skip-gram model. There are many articles and blog posts that offer details on the technical implementation and explanation, such as The amazing power of word vectors, Word2Vec Tutorial — The Skip-Gram, Word2Vec Tutorial Part II: The Continuous Bag-of-Words Model.

In an oversimplified and naive version, given some sample sentences: the farmer takes the wife, the wife takes the kids, the kids take the cats (taken from a little kids song). The skip-gram uses one random word as input and output the probability of every other word in the word list of being the “nearby word” for that chosen word; in contrast, cbow model uses a sequence of words (context) as input and outputs probability of every other word as the target word.

So put the word take in the skip-gram machine, it will churn out the probabilities for farmer , wife , kids , cats and the ; for cbow , you need to feed it the , farmer , wife, it will try to guess the word take .

FastText supports both continuous CBOW and skip-gram models.

Text Classification

Text classification is as fundamental as it is self-explanatory. Applications of text classification range from spam filtering, sentiment analysis, content tagging/classification. It is also central to complex systems such as searching and ranking. Classification and categorization is the first step for humans and machines to process texts in increasing volumes and complexity. In a vast universe of digital information, this is what allows you to locate your subject of interest, follow the echo chamber in which you’d like to occupy, etc.

Traditionally, text classification uses a purely mathematical / statistical approach. Various feature extraction techniques are developed such as Principal Component Analysis (PDA).

Then statistical approaches become inadequate, too slow and too little. So arises machine learning which can better make sense of millions of terabytes of information on the internet. Text classification in machine learning can be divided three categories: supervised text classification, unsupervised text classification, and semi-supervised text classification based on the learning principle followed by the data model (Korde & Mahender, 2012).

FastText uses a simple and efficient baseline for sentence classification( represent sentences as bag of words (BoW) and train a linear classifier). It uses negative sampling, hierarchical softmax and N-gram features to reduce computational cost and improve efficiency.

Have to say, all of the terms made my head spin.

Implementation

The following are my learning and trying out fastText following the examples given in various places (e.g. here).

Training corpus is the book One Hundred Years of Solitude by Garcia Marquez

Training word vectors

using skipgram

./fasttext skipgram -input file.txt -output model

using cbow

./fasttext cbow -input file.txt -output model

With either skipgram or cbow , a .vec and .bin file will be generated. .vec file contains the word vector, one word per line, that is for our human’s inspection, .bin is for machines to consume.

We can peek ahead to see what the vector outputs:

head -n 4 model.vec


Sample output of fastText vector
There are a number of parameters can be adjusted to fine tune the word vector, most importantly, the dimension (dim) and the range of size for the subwords (minn, maxn).

./fasttext skipgram -input file.txt -output model -minn 3 -maxn 5 -dim 100

Reading a word vector is as fun and illuminating as reading binary 0s and 1s. So it is a relief I can directly output the fancy words I care.

echo “imagination miracles magic desperation” | ./fasttext print-word-vectors model.bin


Word vector of a few choice words
Still not quite fun. All that numbers only make me dizzy.

FastText allows you to query the nearest neighbors, which allows us to see if the word vector correctly capture the semantic relation among words.

To do that, we run the nn command.

./fasttext nn model.bin


Nearest Neighbors for the word “imagination”
My pool of training vocabulary is probably too small, I am not sure why extermination ranks as the closest neighbor of imagination . Not by any stretch of imagination. Then I do not often understand the inner working of machines.

Text Classification

For text classification, I used the publicly available Yahoo Answers data (niderhoff/nlp-datasets has a comprehensive list of public domain datasets with text data for use in Natural Language Processing).

Before plugging my data into fastText’s training machine, I did some light processing, so the format fits the requirement (Each line of the text file contains a list of labels, followed by the corresponding document. All the labels start by the __label__ prefix).

Now training time.
"""