Ultimate Pandas Guide — Creating a DataFrame
By: Skyler Dale
Tags: Pandas, Coding, Python, Data Science, Dataframes

Photo by Laura Woodbury from Pexels

Ultimate Pandas Guide — Creating a DataFrame

Build your data from scratch in Python

The first and perhaps most important step of any data analytics work is to acquire your raw ingredients; your data.

Depending on the maturity of your project, this stage can be very straightforward — requesting a csv from a colleague, querying a well-structured database, etc. — or significantly more involved, like a building a custom web scraper.

But not all the data you need for your projects will come from external sources. Sometimes you’ll need to cook it up yourself.

In this post, I’ll walk through the process of creating a DataFrame from scratch.

Understanding the “DataFrame” method

The easiest way to create a new DataFrame is to use the “DataFrame” method. If you are familiar with object oriented programming, you’ll notice that this is actually a constructor call, which instantiates a new DataFrame object.

All arguments are optional, which means you can create an empty DataFrame by passing in… nothing:

import pandas as pd

empty_df = pd.DataFrame()

This can be helpful if you want an empty DataFrame to populate later with data. For example, you could store summary results for several machine learning models by starting with an empty DataFrame and then writing a loop to populate rows for each model.

In most cases, however, you’ll want to fill your DataFrame with data from the start. Luckily, Pandas is very flexible, allowing programmers to convert a variety of data structures into DataFrames. Below is a comprehensive guide:

Creating a DataFrame from a list of lists

Pass a list of lists to the first argument of the DataFrame method, where each inner list will be a future row. You’ll typically also want to include a “columns” argument, which is a simple list of the column names you want to use.

data = [[‘panda’, ‘bamboo’, 100],

[‘koala’, ‘eucalyptus’, 100]] pd.DataFrame(data, columns = [‘animal’, ‘favorite_food’, ‘cuteness_level’])

Creating a DataFrame from a dictionary — Method #1

Pass a dictionary as the first argument. The keys in the dictionary will become the columns in the DataFrame. The values in the dictionary should be lists that will populate the columns:

data = {‘animal’: [‘panda’, ‘koala’],

‘favorite_food’: [‘bamboo’, ‘eucalyptus’],

‘cuteness_level’: [100, 100]} pd.DataFrame(data)

Creating a DataFrame from a dictionary — Method #2

You can also create a DataFrame from a dictionary using the from_dict method, which is a class method of the DataFrame class.

The advantage here is that you can set the “orient” argument to “index”, which changes how the dictionary populates the DataFrame. In this case, the dictionary’s keys will become the row index, and its values will become the rows:

data = {‘animal’: [‘panda’, ‘koala’],

‘favorite_food’: [‘bamboo’, ‘eucalyptus’],

‘cuteness_level’: [100, 100]} pd.DataFrame.from_dict(data, orient = ‘index’, columns = [‘first_animal’, ‘second_animal’])

Creating a DataFrame from a list of dictionaries

Pass a list of dictionaries to the first argument. For each dictionary, the keys correspond to the column names and the values correspond to the values in each row of the DataFrame:

data = [{'animal': 'panda',

'favorite_food': 'bamboo',

'cuteness_level': 100}, {'animal': 'koala',

'favorite_food': 'eucalyptus',

'cuteness_level': 100}] pd.DataFrame(data)

Creating a DataFrame from a numpy array

Pass your numpy array directly to the DataFrame method:

data = np.array([(‘panda’, ‘bamboo’, 100),

(‘koala’, ‘eucalyptus’, 100)],

dtype=[(‘animal’, ‘U6’),

(‘favorite_food’, ‘U6’),

(‘cuteness_level’, int)]) pd.DataFrame(data)

My Recommendation

In this post, we went over a variety of methods to create a new DataFrame.

When I’m building a table from scratch, I prefer to use the list of lists method because it has the most intuitive translation. Each inner list is rendered horizontally and actually looks like the row it becomes in the DataFrame. Happy coding!