Python | Word Embedding using Word2Vec

16 May, 2022
Word Embedding is a language modeling technique used for mapping words to vectors of real numbers. It represents words or phrases in vector space with several dimensions. Word embeddings can be generated using various methods like neural networks, co-occurrence matrix, probabilistic models, etc. Word2Vec consists of models for generating word embedding. These models are shallow two-layer neural networks having one input layer, one hidden layer, and one output layer. Word2Vec utilizes two architectures :
CBOW (Continuous Bag of Words): CBOW model predicts the current word given context words within a specific window. The input layer contains the context words and the output layer contains the current word. The hidden layer contains the number of dimensions in which we want to represent the current word present at the output layer. 
Skip Gram : Skip gram predicts the surrounding context words
 within specific window given current word. The input layer contains the current word and the output layer contains the context words. The hidden layer contains the number of dimensions in which we want to represent current word present at the input layer. 
The basic idea of word embedding is words that occur in similar context tend to be closer to each other in vector space. For generating word vectors in Python, modules needed are nltk and gensim. Run these commands in terminal to install nltk and gensim :
pip install nltk
pip install gensim
  Download the text file used for generating word vectors from here . Below is the implementation : 
Python
# Python program to generate word vectors using Word2Vec # importing all necessary modulesfrom nltk.tokenize import sent_tokenize, word_tokenizeimport warnings warnings.filterwarnings(action = 'ignore') import gensimfrom gensim.models import Word2Vec #  Reads ‘alice.txt’ filesample = open("C:\\Users\\Admin\\Desktop\\alice.txt", "utf8")s = sample.read() # Replaces escape character with spacef = s.replace("\n", " ") data = [] # iterate through each sentence in the filefor i in sent_tokenize(f):    temp = []         # tokenize the sentence into words    for j in word_tokenize(i):        temp.append(j.lower())     data.append(temp) # Create CBOW modelmodel1 = gensim.models.Word2Vec(data, min_count = 1,                              vector_size = 100, window = 5) # Print resultsprint("Cosine similarity between 'alice' " +               "and 'wonderland' - CBOW : ",    model1.wv.similarity('alice', 'wonderland'))     print("Cosine similarity between 'alice' " +                 "and 'machines' - CBOW : ",      model1.wv.similarity('alice', 'machines')) # Create Skip Gram modelmodel2 = gensim.models.Word2Vec(data, min_count = 1, vector_size = 100,                                             window = 5, sg = 1) # Print resultsprint("Cosine similarity between 'alice' " +          "and 'wonderland' - Skip Gram : ",    model2.wv.similarity('alice', 'wonderland'))     print("Cosine similarity between 'alice' " +            "and 'machines' - Skip Gram : ",      model2.wv.similarity('alice', 'machines'))
Output :
Cosine similarity between 'alice' and 'wonderland' - CBOW :  0.999249298413
Cosine similarity between 'alice' and 'machines' - CBOW :  0.974911910445
Cosine similarity between 'alice' and 'wonderland' - Skip Gram :  0.885471373104
Cosine similarity between 'alice' and 'machines' - Skip Gram :  0.856892599521
Output indicates the cosine similarities between word vectors ‘alice’, ‘wonderland’ and ‘machines’ for different models. One interesting task might be to change the parameter values of ‘size’ and ‘window’ to observe the variations in the cosine similarities.  
Applications of Word Embedding :

>> Sentiment Analysis
>> Speech Recognition
>> Information Retrieval
>> Question Answering
References :
https://en.wikipedia.org/wiki/Word_embedding
https://en.wikipedia.org/wiki/Word2vec
crigsby
Python-Miscellaneous
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Python Dictionary
Different ways to create Pandas Dataframe
Enumerate() in Python
Python String | replace()
How to Install PIP on Windows ?
*args and **kwargs in Python
Python Classes and Objects
Python OOPs Concepts
Convert integer to string in Python
Introduction To PYTHON