Using a Markov chain sentence generator in Python to generate ‘real fake news’. | by Mathias Schläffer | Towards Data Science

Let’s be honest: today’s news headlines are crazy. They are especially crazy if they concern Donald Trump. So crazy sometimes that you ask yourself if they are even real! Is what you read something that actually happened or just a random collection of words made into a news headline? Need an example?
Donald Trump ‘Banned’ Unicorns ‘to North Korea’
Crazy, right? Granted, this headline is based on something the small kid of actress Alyssa Milano said. No grown up would come up with something so ridiculous...
“Donald Trump Is “A Unicorn, Riding A Unicorn Over A Rainbow”, According To Sean Spicer
Oh boy.To me, that sounds like it was generated by a random sentence generator. While it sadly wasn’t, I wondered how much (or if) randomly generated news headlines could get crazier than that. So I decided to build a Python script and see for myself.
The script consists of a quick web scraper to get as many news headlines as possible and use them in a Markov model sentence generator to create my very own ‘real fake news’ headline. Even though the Python dictionary data type seemed like an initial good fit for a Markov model, I wanted to use data frames and pandas for it. Simply because I wanted to practice some pandas functionalities and I am not ultimately interested in the most “elegant” way to do it in this case, but I wanted a quick result and learn something on the way.First, I set up a script which is accessing a popular news site using the keywords ‘Donald Trump’. I then isolated the text containing the headlines and wrote them into a text file. I found this website extremely helpful to set up my little piece of code:
https://realpython.com/python-web-scraping-practical-introduction/ On any day, this would give me about 100–120 news headlines. Not bad, but of course doing it once gives very little variety of words to work with and my Markov chain is running the risk of falling to often into a unique chain of words — and in the worst case - simply repeating and already existing news headline. Thankfully it is easy to set up a task scheduler in Windows which would execute my file every day in the background and let my text file grow continuously. All I had to do now is be patient and wait until a significant amount of headlines was accumulated. After about 1 1/2 weeks I had more than 1000 headlines. I decided that that is sufficient for such a highly unscientific and nonsensical project. After loading my text file and doing some coarse cleanup in eliminating some recurring news outlet signatures and double punctuation etc., I could finally start working the text into my Markov model. I decided to use a prefix or ‘lead’ of one word. I don’t want to get into details on Markov chains, but in very short I would summarize it like this: For each word we collect each following word in any sentence. Based on the frequency of a following word compared to its ‘rivals’, we will assign the likelihood of a word following a certain lead word. We can then start with a random word from the entire list of words in our text and then randomly select the next word in our sentence with probabilities based on the likelihoods we assigned. Word for word we will create a new sentence that is based on the properties of the text we used as input. Real news in → real fake news out.
After splitting the text file into individual words and storing them in a variable ‘words’, I defined a pandas data frame with three columns: one column ‘lead’ defining the leading word. For each leading word, there is a word that ‘follows’ and be captured in our second column ‘follow’. The third column -‘freq’- will display how many times we can observe a certain combination of a ‘lead’ and a ‘follow’ in our text. What will we fill these columns with? Well, the first column is easy — simply every word in our text will be written in a row in this column. For every word with index i in our text, the following word is then the word with index i + 1. We simply start by filling the column ‘follow’ with our text again, but this time starting with the second word in the text. For a text of initially n rows, this fills n-1 “follow” words. The very last word in our text does not automatically have a follower, so we will use a synthetic string variable to fill the gap. I called it the ‘EndWord’ (please don’t confuse it with a similar sounding thing the president allegedly said on some mysterious apprentice tape).
import pandas as pddict_df = pd.DataFrame(columns = [‘lead’, ‘follow’, ‘freq’])dict_df['lead']=wordsfollow = words[1:]follow.append('EndWord')
Before I move on, I’ll create a separate array of words containing all the very last words of each sentence. In my later code, I will use these end-words to “naturally” end my randomly generated sentence. This has the nice side effect that I don’t have to worry about my Markov chain running ‘across’ headlines, meaning that the last word of one headline should not be considered a lead for the first word of the following headline. After all I am not dealing with one continuous text, but with individual and independent sentences. By cutting of my sentence after my sentence generator gets to such an end_word, this should not be a concern.
end_words = []for word in words:    if word[-1] in ['.','!','?'] and word != '.':        end_words.append(word)print(end_words)
So far so good. Now I count the occurrence of each combination of lead- and follow word and assign the result to the frequency column using group_by and transform.
dict_df['freq']= dict_df.groupby(by=['lead','follow'])['lead','follow'].transform('count').copy()
The rows are still not unique, so I delete duplicate rows before using the pivot functionality in Python to create one big matrix with every lead word as the row-index and every follow word as a column. The frequency of the combination of lead word in row i and following word in column j is then element a_ij in this matrix.
dict_df = dict_df.drop_duplicates()pivot_df = dict_df.pivot(index = 'lead', columns= 'follow', values='freq')
For each row, I then summed up all frequencies and divided each element in row i by the total of row i. This leads to percentages summing up to 1 that my sentence generator will use as a probability distribution when selecting the follow word for a certain lead word. E.g. if after the word ‘fake’ we observe 7 times the follow word ‘news’ and 3 times the word ‘tan’, the resulting probabilities to be selected would be 0.7 and 0.3 respectively.
sum_words = pivot_df.sum(axis=1)pivot_df = pivot_df.apply(lambda x: x/sum_words)
With all this in place, I can define my function for the sentence generator. I take one start word to initialize the sentence generator. This word could be a randomly chosen word from the set of all words. But since my text consists of only headlines of Donald Trump, I think it makes sense to make sure his name appears in our fake news headline as well. In my examples, I therefore start with ‘Donald’. Starting with this word, the generator selects a follow word based on the probability matrix we set up. It will continue by using the selected follow word as the new leading word to append word after word to our sentence. If the sentence generator gets to a word which is in our array of end-words, the sentence will take this word as the very last and return our sentence. If the generator gets to our synthetic ‘EndWord’ we defined earlier, it will simply draw a new follow word for the lead word that brought us down this dead-end road. The same applies if we get to an end word and our sentence is just 2 words long (I had a few example sentences that read only “Donald Trump.”. Like in “Donald Trump.” — nuff said).
from numpy.random import choicedef make_a_sentence(start):    word= start    sentence=[word]    while len(sentence) < 30:        next_word = choice(a = list(pivot_df.columns), p = (pivot_df.iloc[pivot_df.index ==word].fillna(0).values)[0])        if next_word == 'EndWord':                continue        elif next_word in end_words:            if len(sentence) > 2:                    sentence.append(next_word)                break            else :                continue        else :            sentence.append(next_word)        word=next_word    sentence = ' '.join(sentence)    return sentencesentence = make_a_sentence('Donald')
And that’s it.
I certainly won’t win a (Trump hosted) beauty pageant with this code, but it is quick, dirty, Python-noob friendly and does the job. Granted, the bigger the text, the slower the entire process of assigning probabilities will become, but as for my sample, it still took only a few seconds so I don’t see the need for time consuming optimization just yet. Let’s have a look at some of the ‘real fake news’ headlines instead:
Donald Trumps Criticism Of Trump Obsessed With Star Wars Burn. - (Sounds plausible)
Donald Trump broke the real Donald Trump loyalty test.- (How cool would it be if that happened?)
Donald Trump’s Team Files Complaint Against Whites.- (Yeah.... probably not)
Donald Trumps best people boast.- (His worst too)
Donald Trumps aides feel his presidency this fall.- (We all do... we all do)
Donald Trump, Omarosa, and other individuals, including the 45th president is Junior Mint.-(Fresh?)
Donald Trump, you are so far.-(Maybe a bit too far for my taste)
Of course these are some of the successful examples. A lot of sentences come out as absolute gibberish (not covfefe worthy, but still gibberish):
Donald Trump Know Who Aretha Franklin Work for Utahs former aide Omarosa Manigault Newman.-(???)
So what did we learn through this? You as a reader, probably not much. But I had a few good laughs with my code and got to work on data in a structure I usually wouldn’t encounter. If you have feedback or suggestion, please feel free to reach out. Thanks!
P.S.:
If you want a more sophisticated version of a Trump Markov chain, check out this random Trump Tweet generator by someone smarter: