How to Write All of Your SQL Queries in Pandas | by Terence Shin | Towards Data Science

What makes SQL so amazing is that it’s so easy to learn — the reason why it’s so easy to learn is that the code syntax is so intuitive.
Pandas on the other hand isn’t so intuitive, especially if you started out with SQL first like I did.
Personally, what I found really helpful was thinking about how I would manipulate my data in SQL, and then replicate it in Pandas. And so, if you’re trying to be more proficient in Pandas, I highly recommend that you take this approach as well.
Thus, this article serves as a cheatsheet, dictionary, a guide, whatever you want to call it so that you can refer to this when using Pandas.
And with that said, let’s dive into it!
Selecting RowsCombining tablesFiltering tablesSorting valuesAggregate functions
Selecting Rows
Combining tables
Filtering tables
Sorting values
Aggregate functions
If you want to select an entire table, simply call the name of the table:
# SQLSELECT * FROM table_df# Pandastable_df
If you want to select specific columns from a table, list the columns that you want in double brackets:
# SQLSELECT column_a, column_b FROM table_df# Pandastable_df[['column_a', 'column_b']]
Simply use .drop_duplicates() to get distinct values:
# SQLSELECT DISTINCT column_a FROM table_df# Pandastable_df['column_a'].drop_duplicates()
If you want to rename a column, use .rename():
# SQLSELECT column_a as Apple, column_b as Banana FROM table_df# Pandastable_df[['column_a', 'column_b']].rename(columns={'column_a':'Apple', 'column_b':'Banana'})
For the equivalent of SELECT CASE WHEN, you can use np.select() where you first specify your choices and the values for each choice.
If you want to include multiple conditions per choice, check out this article.
# SQLSELECT CASE WHEN column_a > 30 THEN "Large"            WHEN column_a <= 30 THEN "Small"            END AS SizeFROM table_df# Pandasconditions = [table_df['column_a']>30, table_df['column_b']<=30]choices = ['Large', 'Small']table_df['Size'] = np.select(conditions, choices)
Simply use .merge() to join tables, You can specify whether you want it to be a LEFT, RIGHT, INNER, or OUTER join using the “how” parameter.
# SQLSELECT * FROM table_1 t1         LEFT JOIN table_2 t1 on t1.lkey = t2.rkey # Pandastable_1.merge(table_2, left_on='lkey', right_on='rkey', how='left')
Simply use pd.concat():
# SQLSELECT * FROM table_1UNION ALLSELECT * FROM table_2# Pandasfinal_table = pd.concat([table_1, table_2])
When filtering a DataFrame the same way you would use a WHERE clause in SQL, you simply need to define the criteria within square brackets:
# SQLSELECT * FROM table_df WHERE column_a = 1# Pandastable_df[table_df['column_a'] == 1]
When you want to select a certain column from a table and filter it with a different column, follow the format below (if you want to read more about it, check out this link):
# SQLSELECT column_a FROM table_df WHERE column_b = 1# Pandastable_df[table_df['column_b']==1]['column_a']
If you want to filter by multiple criteria, simply wrap each condition in parenthesis and separate each condition using ‘&’. For more variations of this, check out this link.
# SQLSELECT * FROM table_df WHERE column_a = 1 AND column_b = 2# Pandastable_df[(table_df['column_a']==1) & (table_df['column_b']==2)]
The equivalent of LIKE in SQL is .str.contains(). If you want to apply case insensitivity, simply add case=False in the parameters (see here).
# SQLSELECT * FROM table_df WHERE column_a LIKE '%ball%'# Pandastable_df[table_df['column_a'].str.contains('ball')]
The equivalent of IN() in SQL is .isin().
# SQLSELECT * FROM table_df WHERE column_a IN('Canada', 'USA')# Pandastable_df[table_df['column_a'].isin(['Canada', 'USA'])]
The equivalent of ORDER BY in SQL is .sort_values(). Use the ‘ascending’ parameter to specify whether you want to sort the values in ascending or descending order — the default is ascending just like SQL.
# SQLSELECT * FROM table_df ORDER BY column_a DESC# Pandastable_df.sort_values('column_a', ascending=False)
If you want to ORDER BY multiple columns, list the columns in brackets, and specify the direction of ordering in the ‘ascending’ parameter in brackets as well. Make sure that you follow the respective order of the columns that you list.
# SQLSELECT * FROM table_df ORDER BY column_a DESC, column_b ASC# Pandastable_df.sort_values(['column_a', 'column_b'], ascending=[False, True])
You’ll notice a common pattern for aggregate functions.
To replicate COUNT DISTINCT, simply use .groupby() and .nunique(). See here for more info:
# SQLSELECT column_a, COUNT DISTINCT(ID) FROM table_dfGROUP BY column_a# Pandastable_df.groupby('column_a')['ID'].nunique()
# SQLSELECT column_a, SUM(revenue) FROM table_dfGROUP BY column_a # Pandastable_df.groupby(['column_a', 'revenue']).sum()
# SQLSELECT column_a, AVG(revenue) FROM table_dfGROUP BY column_a# Pandastable_df.groupby('column_a')['revenue'].mean()
Hopefully, this serves as a useful guide when manipulating data with Pandas. Don’t feel like you have to memorize all of this either! This is something that I constantly come back to when I work with Pandas.
Eventually, with enough practice, you’ll feel more comfortable with Pandas and will understand the underlying mechanics enough where you won’t need to rely on cheat sheets like this one.
As always, I wish you the best in your endeavors! :)
Not sure what to read next? I’ve picked another article for you:
towardsdatascience.com
If you enjoyed this, follow me on Medium for more
Follow me on Kaggle for more content!
Let’s connect on LinkedIn
Interested in collaborating? Check out my website.