Chat generator
By: Baban Deep Singh
Tags: Machine Learning, Artificial Intelligence, Deep Learning, Neural Networks

Photo by Austin Distel on Unsplash

Is Artificial Intelligence(AI) making us lazy or efficient?

I think it’s making us efficient. Due to COVID-19, people are more often found interacting with their peers via social media and text messages. For instance, my push notifications are up by 37%, and positively enough I have reconnected with my school friends, old friends per se. However, this arose a problem of constantly sticking to my phone and suffering from Nomophobia and Phantom vibration syndrome.

Nomophobia — a term describing a growing fear in today’s world — the fear of being without a mobile device, or beyond mobile phone contact. The Post Office commissioned YouGov, a research organization, to look at anxieties suffered by mobile phone users. The study found that about 58 percent of men and 47 percent of women suffer from the phobia, and an additional 9 percent feel stressed when their mobile phones are off. The study sampled 2,163 people. Read more here. Phantom vibration syndrome — where you think your phone is vibrating but it’s not — has been around only since the mobile age. Nearly 90 percent of college undergrads in a 2012 study said they felt phantom vibrations. Get more insights here.

I was indeed almost always on the phone, and even while sleeping, I used to wake up hastily to check my phone as well, and being an introvert I love to sleep. So, I decided to make AI work for me.

With Recurrent Neural Networks (RNN), I decided to train my machine to generate automatic replies, trained based on my personal chats/replies/forwards, etc.

On the corollary, there are many fledgling chatbots trained on the humongous text. However, they lack the human touch and the word and sentence formations one uses while texting. While sending short messages, for instance — fewer people write “See you Later” and my personal network uses “c u l8r”- both of which convey the same message, but with different semantic and syntactic structuring.

Dataset :

I have 881 text messages which are basically interactions between 11 different participants from India(most of them), Germany, and the USA. Due to this time difference, not all are active at once. Few are more gregarious, few more tacit. So this data is a perfect mix of human interactions — sarcastic and sassy — replies, which are more prominent in taking.

The main reason for training on this data is — it’s the most active group in my network and more close to me as I do not want to sound like a bot when I am “replying”.

LETS TALK

# importing necessary libraries

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

import nltk

import string

import unidecode

import random

import torch

After importing we need to have a GPU as RNN or any deep learning neural network requires heavy computing and takes a long time on CPU. additionally, GPUs have additional advantages over CPUs, these include having more computational units and having a higher bandwidth to retrieve from memory.

train_on_gpu = torch.cuda.is_available()

if(train_on_gpu):

print('Training on GPU!')

else:

print('No GPU available, training on CPU; consider making n_epochs very small.')

This code will tell you if you have a GPU or not. even if you don't have one, it is just going to take a longer, but still gives you results.

train_df = pd.read_csv("WhatsappChat.csv")

author = train_df["Content"]

This is how the data frame looks like, I have worked on some data processing and Exploratory Data Analysis to bring it in this formation. The code to change WhatsApp chat into a similar pandas data frame visit here. As I am training it on the content of the chats, we will just be working on that column.

text = list(author)

def joinStrings(text):

return ' '.join(string for string in text)

text = joinStrings(text)

# text = [item for sublist in author[:5].values for item in sublist]

len(text.split()) test_sentence = text.lower().split() trigrams = [([test_sentence[i], test_sentence[i + 1]], test_sentence[i + 2])

for i in range(len(test_sentence) - 2)]

chunk_len=len(trigrams)

print(trigrams[:3])

after joining and making the content as a huge text data, I am training the data based on tri-gram as most of the replies — at least in my network — are sized at 3 words in reply.

Since to Train an RNN, I need a vocabulary size, so that my replies don't go out of bounds.

vocab = set(test_sentence)

voc_len=len(vocab)

word_to_ix = {word: i for i, word in enumerate(vocab)} # making input and their respective replies

inp=[]

tar=[]

for context, target in trigrams:

context_idxs = torch.tensor([word_to_ix[w] for w in context], dtype=torch.long)

inp.append(context_idxs)

targ = torch.tensor([word_to_ix[target]], dtype=torch.long)

tar.append(targ)

RNN

It’s time we define our neural network class and see what it can do for us.

class RNN(nn.Module):

def __init__(self, input_size, hidden_size, output_size, n_layers=1):

super(RNN, self).__init__()

self.input_size = input_size

self.hidden_size = hidden_size

self.output_size = output_size

self.n_layers = n_layers



self.encoder = nn.Embedding(input_size, hidden_size)

self.gru = nn.GRU(hidden_size*2, hidden_size, n_layers,batch_first=True,

bidirectional=False)

self.decoder = nn.Linear(hidden_size, output_size)



def forward(self, input, hidden):

input = self.encoder(input.view(1, -1))

output, hidden = self.gru(input.view(1, 1, -1), hidden)

output = self.decoder(output.view(1, -1))

return output, hidden def init_hidden(self):

return Variable(torch.zeros(self.n_layers, 1, self.hidden_size))

Here is a class RNN, a general object-oriented programming approach to instantiate objects and their respective methods or functions for faster execution.

The def forward() function is our forward pass or feed-forward network and connection of the neural network. And the def init_hidden() is a variable instantiate for hidden layers.

def train(inp, target):

hidden = decoder.init_hidden().cuda()

decoder.zero_grad()

loss = 0



for c in range(chunk_len):

output, hidden = decoder(inp[c].cuda(), hidden)

loss += criterion(output, target[c].cuda()) loss.backward()

decoder_optimizer.step() return loss.data.item() / chunk_len

Now we need to reduce loss to get the optimized reply and check the accuracy of the model. The above code gives us data loss in whole backpropagation.

import time, math def time_since(since):

s = time.time() - since

m = math.floor(s / 60)

s -= m * 60

return '%dm %ds' % (m, s)

This is a simple function to check how much time does it take to run the program or time taken to train the model.

n_epochs = 50

print_every = 10

plot_every = 10

hidden_size = 100

n_layers = 1

lr = 0.015 decoder = RNN(voc_len, hidden_size, voc_len, n_layers)

decoder_optimizer = torch.optim.Adam(decoder.parameters(), lr=lr)

criterion = nn.CrossEntropyLoss() start = time.time()

all_losses = []

loss_avg = 0

if(train_on_gpu):

decoder.cuda()

for epoch in range(1, n_epochs + 1):

loss = train(inp,tar)

loss_avg += loss if epoch % print_every == 0:

print('[%s (%d %d%%) %.4f]' % (time_since(start), epoch, epoch / n_epochs * 50, loss))

# print(evaluate('ge', 200), '

') if epoch % plot_every == 0:

all_losses.append(loss_avg / plot_every)

loss_avg = 0

This is where the magic happens — training the chat and learning what to reply for 50 times I let the machine read the chat and let me know what is the best reply to the message. Also, prints the loss incurred after 10 epochs and time took to execute.

import matplotlib.pyplot as plt

import matplotlib.ticker as ticker

%matplotlib inline plt.figure()

plt.plot(all_losses)

This plots the losses for those who — like me — appreciate plots and likes visual representations then numbers.

def evaluate(prime_str='this process', predict_len=100, temperature=0.8):

hidden = decoder.init_hidden().cuda() for p in range(predict_len):



prime_input = torch.tensor([word_to_ix[w] for w in prime_str.split()], dtype=torch.long).cuda()

inp = prime_input[-2:] #last two words as input

output, hidden = decoder(inp, hidden)



# Sample from the network as a multinomial distribution

output_dist = output.data.view(-1).div(temperature).exp()

top_i = torch.multinomial(output_dist, 1)[0]



# Add predicted word to string and use as next input

predicted_word = list(word_to_ix.keys())[list(word_to_ix.values()).index(top_i)]

prime_str += " " + predicted_word

# inp = torch.tensor(word_to_ix[predicted_word], dtype=torch.long) return prime_str

We need to define an evaluation function to check if we are getting any tangible replies generated. It takes the prime string, length of the sentences, and temperature which takes care of the missing words if any new message comes.

print(evaluate('trip pe',11, temperature=1)) # output

trip pe shuru ? dekh na😅. bumble to sanky bhi use kar sakta

Voila! There is a message generated and it makes less sense, but tangible words. Interestingly, it learned the smileys as well. And we use a huge amount of emoticons in our chats.

Future work

Now, all I need is work on APIs to embed this code in the WhatsApp chat, let it train in a span of a month, and generate the messages — then I don’t look at my phone. This will cure my sleep cycle and leverage me in interacting with people around me than on my phone. Hopefully with increased epochs, say 100 and more data over time this will give fewer errors and more personalized replies which will trick my friends into wondering whether I’m a BOT or replying with my conscience.

If you are interested, you can get the code here.

Do let me know if you think this method lacks some ideas, or how I can optimize it further to get to being a humanlike BOT and let this AI take over my communication.