Generate Your Sample Dataset — A Must Have Skill For Data Scientists. | by flo.tausend | Towards Data Science

It is one thing to create powerpoint slides and talk theoretically about what you will do with data. But it is another one to create a sample dataset and present a dashboard, visualisation or data model that is already working. While many sample datasets can be found on the internet, they quite often do not fit your exact needs. Therefore, at the end of this article, you will know how to create a beautiful sample CSV dataset for your specific purposes. Once the code is setup in a Jupyter notebook, you can use it over and over again.
As final outcome a sample CSV dataset with various variables such as numbers, currencies, dates, names and strings and is desired. These variables have different types and are independent or related to each other. To get started, it is crucial to understand how we can use basic “random” functions to generate our sample dataset. Afterwards we will combine the variables in one data frame. For our purpose the data frame will finally be exported as CSV. CSV is a format quite easy to connect with various tools and sufficient for the purpose of sampling.
The good thing is that we mainly need access to python and use some of its libraries and tools. First of all we use Jupyter Notebook, an open-source application for live coding and it allows us to tell a story with the code. In our case it makes it way simpler to adapt and only use few code elements later on. Furthermore we use NumPy, and especially the random function of NumPy to generate our variables. While this is one option, a very convenient library called pydbgen helps us to accelerate our sample dataset generation. Additionally we import Pandas, which puts our data in an easy-to-use structure for data analysis and data transformation.
“random.” moduleThe most used module in order to create random numbers with Python is probably the random module with the random.random() function. When importing the module and calling the function, a float between 0.0 and 1.0 will be generated as seen in the code below.
“random.” moduleThe most used module in order to create random numbers with Python is probably the random module with the random.random() function. When importing the module and calling the function, a float between 0.0 and 1.0 will be generated as seen in the code below.
>>>import random>>>random.random()0.18215964678315466
Additionally there is the option to set a seed. This allows us to reproduce the random numbers generated in the code.
>>> random.seed(123)>>> random.random()0.052363598850944326>>> random.seed(123)>>> random.random()0.052363598850944326
If there is a specific range you want to consider for your floats or integers, you can define the functions’ parameters as done below.
>>> random.randint(10,100)21>>> random.uniform(10,100)79.20607497283636
2. DatesWhile being familiar with the random method, we can now apply the knowledge to create random dates in our sample dataset.
>>> import numpy as np>>> monthly_days = np.arange(0, 30)>>> base_date = np.datetime64('2020-01-01')>>> random_date = base_date + np.random.choice(monthly_days)>>> print(random_date)2020-01-07
There are endless variations to adjust this piece of code. Change the monthly days range to bimonthly or yearly. And of course the base date should be fitted to your requirements.
3. StringsWhile having random numbers and dates in place for our sample dataset, it is about time to look at strings. When looking at the randomString function below, the variable “letters” contains the whole alphabet in lowercase. With the help of the “random” method, a random number is created and assigned to letter. Easy as that!
import randomimport stringdef randomString(stringLength=15):    letters = string.ascii_lowercase    return ''.join(random.choice(letters) for i in range(stringLength))print ("The generated string is", randomString())The generated string is celwcujpdcdppzx
4. NamesWhile our sample dataset was previously based on the random method, it is not possible to generate expressions that make sense by just randomly putting things together. Therefor we use the names package available on PyPi. The package is really convenient and gives various options to draw from as shown in the snippet below.
>>> import names>>> names.get_full_name()‘Margaret Wigfield’>>> names.get_first_name(gender='female')'Patricia'
5. Locations, Phone Numbers, Address or TextAfter a short introduction to the principles of randomisation with “random.” as basic method, it is probably noteworthy that there are complete packages that cover most of your needs. Installation is simple via pip and you can find all infos need on the Github repository. There are two very useful ones definitely worth mentioning:
Faker
pydbgen (Partially uses Faker for some data types)
6. What about sample data for Machine LearningWhen it comes to machine learning problems, the job is not done generating by generating any random data. In most cases you want some correlation or relation between the variables / features of your sample dataset. Scikit let’s you create such datasets in seconds. Have a look at the sample code below:
import pandas as pdfrom sklearn.datasets import make_regression# Generate fetures, outputs, and true coefficient of 100 samplesfeatures, output, coef = make_regression(n_samples = 100,                                         # three features                                         n_features = 3,                                         # two features are useful,                                         n_informative = 2,                                         # one target                                         n_targets = 1,                                         # 0.0 standard deviation                                         noise = 0.0,                                         # show the true coefficient                                          coef = True)
# View the features of the first 10 rowspd.DataFrame(features, columns=['Customer A', 'Customer B', 'Customer C']).head(10)# View the output of the first 10 rowspd.DataFrame(output, columns=['Sales']).head(10)# View the actual, true coefficients used to generate the datapd.DataFrame(coef, columns=['Coefficient Values'])
After exploring, playing around and generating our sample datasets, we finally want to export and make use of them. While there are no limitations in the file format, as we do not a huge amount of data and want to plug the dataset in various tools, a simple CSV files is quite convenient for that use. But first a data frame has to be created.
import pandas as pdimport numpy as npimport random#using numpy's randintdf = pd.DataFrame(np.random.randint(0,100,size=(15, 4)), columns=list('ABCD'))# create a new column df['AminusB'] = df['A'] - (0.1 * df['B'])df['names'] = 'Laura'df.head(5)
While we have generated some random related and unrelated numbers, the name column is the same in each row. To get random names we have to iterate over the rows and use our random name generator.
import namesfor index, row in df.iterrows():      df.at[index,'names'] = names.get_first_name(gender='female')    df.head(5)
Note that instead of names you could use any other package or method to create your sample CSV dataset. There are no limitations. Now let’s export it as CSV and we are good to go.
#Export to csvexport_csv = df.to_csv (r'/Users/SampleDataset.csv', header=True) #Don't forget to add '.csv' at the end of the path
You made it — Congrats ! Now the more exiting part begins and you can start working with your generated CSV sample dataset. Good Luck.
**************************************************************
Articles you may enjoy reading as well:
Predictive Maintenance: Machine Learning vs Rule Based
Hands-on: Customer Segmentation
Setup Your Data Environment
Misleading with Data & Statistics
Learn How To Predict Customer Churn