SQL vs. Pandas — Which one to choose in 2020? | by Dario Radečić | Towards Data Science

SQL has been around for decades now and is a quite popular option because almost everybody knows it, and it’s easy to learn and understand. A lot of the time later came Pandas — and most of us fall in love ever since. Today we’ll start comparing these two, from basics to more advanced stuff — analytics-wise.
SQL and Python are two languages that are almost indispensable for data analysts and data scientists in the current times. When anyone starts their data analytics journey, they usually start with SQL as their first step and then slowly move towards R and Python.
In order to make this transition a lot easier, we will make an attempt to compare some very essential commands in SQL with those in Pandas. For the uninitiated, SQL is a language used for storing, manipulating, and retrieving data in relational databases. Pandas is a library in python used for data analysis and manipulation.
This is a part one of the series, and covers:
Importing data
Select
Where
Insert
Update
options in both languages. The next part covers more advanced topics, like joins, unions, and groupings.
For the purpose of this article, we have used the Fertility and Contraception dataset. Once we have downloaded the dataset, let’s start with importing dataset files in SQL and Pandas.
I personally feel that importing data files in SQL is a bit tedious task when compared to Pandas. In Pandas, we can simply use pandas.read_csv() and voila — our file is imported and a data frame is ready for us to explore in just one command. Isn’t that great?
But in SQL, things are a bit different. We have to first create a table using the CREATE TABLE statement with fields similar to those mentioned in the file and then copy data from the file. You may refer to the code given below.
CREATE TABLE fert_data (    country character varying(30),    region character varying(50),    tfr numeric,    contraceptors integer);
The query returned successfully and a new table called fert_data has been created. Now, in order to import records from the CSV file, we have to use the COPY command as shown below.
copy fert_data from ‘Path\to\file’ with delimiter ‘,’ csv header encoding ‘windows-1251’;
Now compare the above work with this code in Pandas for importing files.
df = pd.read_csv(’Path\to\your\dataset’)
And that’s all you have to do.
Once we have imported the dataset, we might want to have a glimpse at the records in it to understand the dataset better. In SQL, we use a SELECT statement to fetch records from a table. This statement would look something as follows :
SELECT column_name1, column_name2,...FROM table_name;
What if you don’t want to see all the records? Yes, you can use a TOP or LIMIT keyword to restrict the number of rows.
In pandas, we use head() and tail() to fetch the first 5 and last 5 rows respectively. This is equivalent to SELECT * FROM table LIMIT 5 in SQL. If a specific column has to be selected from the data frame, then the syntax would look something as follows:
df[[‘column_name1’, ‘column_name2’]]
Here is an example to illustrate this better. We will follow a procedure for examples here on, and will first write the SQL code and then write the corresponding Pandas code.
SELECT *FROM fert_dataLIMIT 5;
df.head()
In SQL, a WHERE clause is used to filter records based on a condition. The clause is usually written in conjugation with conditional and logical operators for forming filtering conditions. The syntax for a WHERE clause would be something as follows :
SELECT *FROM table_nameWHERE condition_expression;
In Pandas, there is no single command for filtering records like SQL. We can use boolean indexing and positional indexing. In Pandas indexing, boolean masks are computed through logical conditions to filter the data. The syntax of each of the following is as follows :
Boolean indexing:
df[df[‘column_name’] == value]
Positional indexing:
df[df.loc[‘column_name’] == value]
In both SQL and Pandas, we can make complex conditional expressions using comparison (>, <, >=, <=, !=, ==) and logical operators(‘and’, ’or’, ’not’ or symbols such as ‘|’, ‘&’, ‘~’).
Here’s a quick example — we want to find records that are not from the African region but have TFR between 5.5 and 7:
SELECT country, tfrFROM fert_dataWHERE tfr BETWEEN 5.5 AND 7 AND region != ‘Africa’;
df.loc[((df.tfr >= 5.5) & (df.tfr <= 7)) & (df.region !=’Africa’)]
INSERT statement is used in SQL to insert rows in a database table. The syntax for inserting new records looks something as follows :
INSERT INTO table_name(column_name_1, column_name_2,...) VALUES (value1, value2,...);
In Pandas, there is no specific function to insert new rows in the data frame. We can use tools such as concat() or append() to perform this task. But the most popularly used one is append().
df = df.append(new_row_to_be_inserted, ignore_index=True)
For further clarity, let’s insert a new row that contains details pertaining to Japan.
INSERT INTO fert_data(country, region, tfr, contraceptors)VALUES (‘Japan’,’Asia’,1.73,73);
data = {    ‘country’: ‘Japan’,     ‘region’: ‘Asia’,     ‘tfr’:1.73,     ‘contraceptors’ : 73}df = df.append(data, ignore_index=True)
You can observe from the image above that a new row at the 50th index has been created.
In SQL, UPDATE statement is used to update or modify the values of an existing record. The syntax for the update statement is as follows:
UPDATE table_nameSET column_name = ‘modified_value’WHERE condition;
In Pandas, there is no specific function to update an existing row. A common procedure is to locate the place of modification using positional indexing (as explained in the filtering part) and then assigning it a new value. It’s almost like a variable assignment.
Let’s try an example to understand it in detail — update the number of contraceptors to ‘65’ for Mauitius.
UPDATE fert_dataSET contraceptors = ‘65’WHERE country = ‘Mauitius’;
The query returned successfully and you can see the value has been updated.
df.loc[df[‘country’] == ‘Mauitius’, ‘contraceptors’] = 65
First, we have located the row where the country is ‘Mauitius’ and then set the value of the ‘contraceptors’ field to 65.
As you can see, both SQL and Pandas are better at some points and there’s no clear winner. For some tasks, like inserting and updating, SQL is a clear winner — due to more readable and easier-to-understand code, On the other hand, Pandas doesn’t require you to create a data structure for the data beforehand, which can be a huge time saver.
Just imagine creating a table with 100+ columns — or organizing that data into multiple tables and using joins. Pain in the bottom.
We’ll cover more advanced topics like grouping, union, and joins in the next article, so make sure to stay tuned.
Thanks for reading.
Loved the article? Become a Medium member to continue learning without limits. I’ll receive a portion of your membership fee if you use the following link, with no extra cost to you.
medium.com
Join my private email list for more helpful insights.