A Python Pandas Introduction to Excel Users | by Frank Andrade | Towards Data Science

Pandas is probably the best tool to do real-world data analysis in Python. It allows us to clean data, wrangle data, make visualizations, and more.
You can think of Pandas as a supercharged Microsoft Excel. Most of the tasks you can do in Excel can be done in Pandas too and vice versa. That said, there are many areas where Pandas outperforms Excel.
In this introduction to Pandas, we will compare Pandas dataframes and Excel Spreadsheet, learn different ways to create a dataframe, and how to make pivot tables.
Note: Before learning Pandas, you should know at least know the basics of Python. If you’re new to Python, check this guide I made to get started with Python in no time.
Table of Contents1. Why Excel Users Should Learn Python/Pandas2. Pandas DataFrames & Excel Spreadsheets3. How to Create a Dataframe - Creating a dataframe by reading an Excel/CSV file - Creating a dataframe with arrays - Creating a dataframe with a dictionary
Tasks such as data cleaning, data normalization, visualization, and statistical analysis can be performed on both Excel and Pandas. That said, Pandas has some major benefits over Excel. To name a few:
Limitation by size: Excel can handle around 1 million rows, while Python can handle millions and millions of rows (the limitation is on PC computing power)
Complex data transformation: Memory-intensive computations in Excel can crash a workbook. Python can handle complex computations without any major problem.
Automation: Excel was not designed to automate tasks. You can create a macro or use VBA to simplify some tasks, but that’s the limit. However, Python can go beyond that with its hundreds of free libraries available.
Cross-platform capabilities: On Excel, you might find some incompatibilities between formulas in Windows and macOS. This also happens when sharing Excel files with people that don’t have English as the default language on their version of Microsoft Excel. In contrast, Python code remains the same regardless of the operating system or language set on a computer.
Before showing you how to replace Excel with Pandas (creating pivot tables, visualizations, etc), I will explain to you the core concepts of Pandas. Throughout this guide, I will mention Excel to provide some examples and compare it with Pandas. Excel is the most popular spreadsheet, so some analogies will help you grasp Pandas concepts fast.
Let’s get started!
The two main data structures in Pandas are series and dataframe. The first is a 1-dimensional array, while the second is a 2-dimensional array.
In Pandas, we mainly work with dataframes. A Pandas dataframe is the equivalent of an Excel spreadsheet. Pandas dataframes — just like Excel spreadsheets — have 2 dimensions or axes.
A dataframe has rows and columns (also known as series). On top of a dataframe, you will see the name of the columns and on the left side, there’s the index. By default index in Pandas start with 0.
The intersection of a row and a column is called a data value or simply data. We can store different types of data such as integers, strings, boolean, and so on.
Here’s a picture of a dataframe that shows US states ranked by population. I’m going to show you the code to create a dataframe like this later, but now let’s analyze this dataframe.
The column names (states, population, and postal) are also known as features, while each row value is known as observation. We can say that there are 3 features and 4 observations.
Keep in mind that a single column should have the same type of data. In our example, the states and postal columns only contain strings, while the population column only contains integers. We might get errors when trying to insert different data types into a column, so avoid doing this.
To sum it up, this is the terminology translation between Pandas and Excel.
We haven’t talked so much about empty cells so far, but it’s good for you to know that in Python missing data is represented with NaN, which stands for “Not a Number”. Whenever there’s an empty value in a dataframe, you’ll see an NaN.
There are multiple ways to create a dataframe. We can create a dataframe by reading an Excel/CSV file, using arrays, and also with dictionaries.
But before creating a dataframe, make sure you have Python and Pandas installed. In case you’re new to Python, you can install it by watching this video.
To install Pandas, run the command pip install pandas on the terminal or command prompt. In case you have Jupyter Notebook, you can install it from a code cell by running !pip install pandas
This is without a doubt the easiest way to create a dataframe in Pandas. We only need to import pandas, use the read_csv() method and write the name of the Excel/CSV file within parentheses.
Let’s read a CSV file that contains data about the GDP of countries around the world (You can find this file on this link)
import pandas as pddf_gdp = pd.read_csv('gdp.csv')df_gdp
In the code above, we renamed pandas as “pd.” This is only a convention to name pandas. After running the code above, you will see the following dataframe.
That’s all you need to create a dataframe! Now you can start creating a pivot table from this dataframe by watching my video tutorial below.
To create a dataframe with arrays we need to import Numpy first. Let’s import this library and create an array with random numbers.
import numpy as npdata = np.array([[1, 4], [2, 5], [3, 6]])
That said, if you don’t feel comfortable working with Numpy yet, you can create the same arrays using a list notation. The result will be the same when creating a dataframe in the next step.
data = [[1, 4], [2, 5], [3, 6]]
To create a dataframe, we import pandas and use the .Dataframe() method. Inside parentheses, we can add the index and columns arguments to assign names.
import pandas as pddf = pd.DataFrame(data, index=['row1', 'row2', 'row3'],                 columns=['col1', 'col2'])
df is the standard name for a dataframe. If we print this df object, we obtain the following:
We can also create a dataframe from a dictionary. This is my favorite choice because a dictionary allows us to organize data properly. As you might remember, a dictionary is made of a key and a value. We can set lists as data values and the key as the name of a column.
Let’s create the population dataframe we’ve seen before using a dictionary. First, we create 2 lists: states, and population.
# data used for the example (stored in lists)states = ["California", "Texas", "Florida", "New York"]population = [39613493, 29730311, 21944577, 19299981]
Now we create a dictionary setting the stringsStates and Population as keys and the lists created before as data values.
# Storing lists within a dictionarydict_states = {'States': states, 'Population': population}
Once the dictionary is created, making a dataframe is as easy as using the .Dataframe() method, but this time we don’t need to specify the column argument because it’s already specified in the keys.
df_population = pd.DataFrame(dict_states)
If you print df_population, you will get this dataframe:
Join my email list with 3k+ people to get my Python for Data Science Cheat Sheet I use in all my tutorials (Free PDF)
If you enjoy reading stories like these and want to support me as a writer, consider signing up to become a Medium member. It’s $5 a month, giving you unlimited access to thousands of Python guides and Data science articles. If you sign up using my link, I’ll earn a small commission with no extra cost to you.