Generating Startup names with Markov Chains | by Gustavo Zomer | Towards Data Science

The most interesting applications of Machine Learning are, without a doubt, the generative models.
The idea of finding patterns in data and generating new content that at the same time is similar to your data, but unique in its own way, has always fascinated me.
So I have decided to develop a simple text generator to create Startup names using Markov Chains.
But first of all, a short introduction to Markov Chains 🔗
A Markov chain is a model of some random process that happens over time.
Markov chains are called this way because they follow a rule called the Markov property. The Markov property says that whatever happens next in a process only depends on how it is right now (the state).
For instance, consider the example of predicting the weather for the next day, using only the information about the current weather. By analysing some real data, we may find these conditions:
Given that today is sunny, tomorrow will also be sunny 90% of the time
Given that today is sunny, tomorrow will be rainy 10% of the time
Given that today is rainy, tomorrow will also be rainy 50% of the time
Given that today is rainy, tomorrow will be sunny 50% of the time
This is modelled via the Markov Chain below, where each circle is a state and the numbers in the arrows represent the probabilities of changing from the current state.
So if we want to make a prediction for tomorrow, we just need to verify the current state we are in (sunny or rainy) and use the transition probabilities to calculate which next state is more likely.
An in-depth explanation of Markov Chains, with some cool animations, can be found here: http://setosa.io/ev/markov-chains/
So, how can we apply this idea to generate text? Well, it's quite simple actually.
Names or sentences are basically a sequence of characters and those sequences follow some patterns.
For instance, if I asked you to give words that started with whe__, you would quickly come up with the words when, where, whenever, etc. While wheeziness is a perfectly valid word, it's less frequent considering the initial whe_. In other words, given the state whe, we will most likely change to the states when, where or whenever than to the state wheeziness.
So how can we build a model that capture these probabilities given our data?
For this, I'll show you how to build a simple Markov Chain using the words startup, statistic and artist. First of all, we will list all the states transitions for each tuple of 3 characters:
Startupsta -> tartar -> artart -> rturtu -> tupStatisticsta -> tattat -> atiati -> tistis -> istist -> stisti -> ticArtistart -> rtirti -> tisist -> ist
Now, if you pay close attention to the states, you will notice that some of them are shared among different tuples. To better visualize that, let's create a dictionary where each entry is a state, and the values are the next states and its weights.
{   "sta": {      "tar": 1,      "tat": 1   },   "tar": {      "art": 1   },   "art": {      "rtu": 1,      "rti": 1   },   "rtu": {      "tup": 1   },   "tat": {      "ati": 1   },   "ati": {      "tis": 1   },   "tis": {      "ist": 2   },   "ist": {      "sti": 1   },   "sti": {      "tic": 1   },   "rti": {      "tis": 1   }}
Ta-da! That's our Markov Chain. Simple, isn't it? (PS: Technically, Markov Chains are defined with a transition matrix with probabilities, not weights. We could easily transform into a transition matrix, but for the problem we have in mind, this is a better way of visualizing it).
Now, how can we generate new data with this?
There are basically four steps, let's go through each one of them:
Step 1: Start with some initial random state
You could select any of the states as a starting position, however, you will most likely generate text that doesn't make any sense. For instance, rtu is a valid initial state, but you won't find a word in real life that starts with those letters (none that I can't think of it, at least)
A better approach is to keep track of the starting states in another dictionary, and selecting the first state from there. In our case, the possible initial states are sta(as both startup and statistic start with sta) and art. For our example, let's select sta.
Step 2: Select randomly one of its transition states, considering its weights
For the tuple sta, you can go to tar or tat, both of them with the same probability (same weight). In a real case scenario, they would have different weights considering the distribution found in your dataset, but as we have just used three words, they have equal weights. Let's "randomly" select the tuple tar.
Step 3: Append the new state to your generated text
So far, we have started with the state sta and transitioned to the state tar. So our current generated word is star.
Step 4: Repeat step one using the new state, until a stop character is found, or until you are happy with your result
Now, for our current state tar, the only possible state is art, so our generated word become start.
Now let's continue the algorithm in a faster way. From art, we can go to rti or rtu. Let's select rti. If you continue to apply the algorithm, you will quickly generate our new word: Startist, which is a mix of startup and artist.
Even though the example is quite simple, it demonstrates the potential of Markov Chains.
Now that we have "implemented" by hand a Markov Chain, let's do in Python using real data where you can get actual useful results.
Let's start by importing some modules. We will only need two: pandas, to read CSV data, and random to (unsurprisingly) generate random numbers.
import pandas as pdimport random
As a dataset for our Startup name generator, we are going to use a 2015 dump from CrunchBase with around 18k companies.
The dataset is not that big, but you will see that Markov Chains work pretty well even with a database much smaller than this.
Reading our companies data is pretty straightforward: pandas read_csv function accepts a URL as a parameter and returns a data frame. We have also removed symbols and transformed the names to lower case.
companies = pd.read_csv('https://github.com/notpeter/crunchbase-data/blob/master/companies.csv?raw=true')companies['name'] = companies['name'].replace({r'[^\w ]':''}, regex=True)companies['name'] = companies['name'].apply(lambda n:str(n).lower())
As we have discussed previously, the simplest way to model the data structure for a Markov Chain is a dictionary containing the states and transitions weights.
chain = build_markov_chain(companies['name'].tolist(), 3)print(chain['sta'])
If you run the above code, you will get this result:
{   'tar':290,   'tat':151,   'ta.':52,   'ta ':35,   'tac':55,   'tag':43,   'tal':46,   'tay':34,   'tau':22,   'tad':14,   'tam':19,   'tas':19,   'taq':5,   'tan':92,   'tab':23,   'tap':6,   'tak':8,   'tai':22,   'taf':16,   'tax':5,   'taTM':1,   'tah':2,   'tav':5,   'tae':1,   'taj':1,   'taw':1,   'taa':2,   'taz':1}
What does this mean? These are the list of next states and weights on the Markov Chain, considering that the current state is the tuple sta. The higher the next state weight, more likely its transition to it.
For instance, if you take the first state tar, it has the largest weight on this state list. Intuitively, it makes sense, as it is probably capturing the occurrence of the word startup.
Now we need to build a function that returns a random tuple from the chain considering its weights.
Finally, here is where the magic happens: let's generate some new words.
Let's go step by step in our generate function.
tuple = select_random_item(chain['_initial'])    result = [tuple]
Remember that we mentioned that is better to keep track of the initial tuples and selecting one of those as the initial state? That's exactly what we are doing here.
while True:            tuple = select_random_item(chain[tuple])            last_character = tuple[-1]            if last_character == '.':                 break     result.append(last_character)
This is where we are navigating through our Markov Chain, considering its probabilities. We are selecting a random weighted next state and appending the last character of this state to our result string. However, if the last character is a period, we stop our generation, as this is the ending of our chain.
We could add additional rules such as generating words given a minimum or maximum length, but let's keep it simple for now.
generated = ''.join(result)    if generated not in chain['_names']:     return generated    else:            return generate(chain)
Finally, we join all of the generated characters together, and we do the last verification. As nothing prevents the Markov Chain from generating an already existing name, and as we are interested in creating new names, we will simply generate a new one if the generated name is already in our database.
Here is a couple of examples from our Startup Name GeneratorTM®.
Domos
Hup Online
Vubanky
Acara
Ignaly
iFly
Pretty cool, huh? 😎
One final idea, what if we generated Startup names specific for each industry? It would be awesome, don't you think?
Let's do it, it will be extremely easy 🙃
The only thing we have to do is to build our Markov Chain considering only examples from the industry we are interested in.
And here are our results:
print(generate_amount_by_category('Travel',5))
Pango
Movology
Nextrive
Triptel
Stingi
print(generate_amount_by_category('Technology',5))
Naco Innovation
Kicksense
NetWatch
Chony
Datars
You can try yourself using this Google Colab link or downloading the source code directly from my GitHub.
colab.research.google.com
What are your thoughts? Any suggestion for new content? Feedbacks? Let me know in the comments.
Hope you have enjoyed it :)