The Complete Guide of SQL For Data Scientists | by Jun Wu | Towards Data Science

SQL used to rule in the land of Enterprise Data. Before the NoSQL movement, SQL was the main query language used to retrieve data from relational databases. According to a survey done by High Scalability asking IT leaders at DeveloperWeek about the trend for database usage in 2019, SQL is still used more than 60% of the time. Multiple database strategy (SQL+NoSQL) is used more than 75% of the time. This is no surprise. Different types of databases are used for different purposes. (If you are interested, you can read more about that here.)
Out of all the relational databases, MySQL remains the most popular database for organizations.
To aspiring Data Scientists who have aced machine learning algorithms, participated in Kaggle competitions, and manipulated data heavily using Pandas, walking into an interview with questions about SQL may come as a shock. But, it happens. You want to be prepared.
In the real world, relational databases are used heavily to house all types of Enterprise data. Data is stored for easy retrieval from all types of data sources. Knowing SQL can lead to easy data retrieval and manipulation. Before the whole Data Science “hype”, Data Science was done by Statisticians in large organizations. Data Mining was the intersection of statistics and computer science that these Statisticians operated in. Data mining is simply the process that uses Machine Learning and Statistics to discover patterns in large sets of data.
Data Mining eliminates the need to transfer data or merge data before analysis. Analysis can be done across many data stores on the fly.
For Data Mining, in Enterprise Technology, large datasets are often processed using SAS. SAS enables Data Scientists to easily perform statistical analysis on a large amount of data without having to write too much code. Built-in functions in SAS are powerful for all types of data analysis. SQL can easily be wrapped up to run in SAS. The syntax of SAS commands are also similar to SQL.
DataMiner from Oracle is also another popular Database Mining software that’s used in Enterprise Technology for Data Science. It contains a host of data analysis algorithms such as classification, prediction, regression, association, feature extraction, etc.. that can be performed on large sets of data housed in relational databases.
Before we dive into SQL, there are a few relational database (RDBMS) concepts that will set the background for you.
The database schema refers to the organization of data. It is a blueprint of how the database is constructed. In RDBMS, the database model is implemented by the schema. It is the Entity-relationship model.
Here’s a good blog post describing this relationship.
One of the main units of the schema is the table. In RDBMS, tables are laid out and associated with each other in different types of relationships (one to one, one to many, many to many) relationships.
The vertical partitions of tables are called columns. In RDBMS, a column is often also called an attribute.
The horizontal partitions of tables are called rows. In RDBMS, a row is often also called a tuple.
In RDBMS, the index is a data structure that improves the operation of the table by quickly locating the data. Depending on the type of index, by creating an index, you can create copies of the data ordered in a certain way located on the server for easier retrieval. Sometimes, instead of copies of the data, the data can be ordered in a particular way by the index when it’s created for optimized retrieval.
There are two main types of indexes: Non-clustered and clustered.
Non-clustered index is when data is stored in an arbitrary order but the logical order is specified by the index.
Clustered index is when the data block where data is stored is altered to match a distinct order by the index.
Here’s a blog post describing the difference between a Non-clustered index and a Clustered index.
SQL is the Structured Query Language for RDBMS databases. When you retrieve data, you typically use SQL statements to retrieve the data. You can join tables together to select out just the right types of data from different purposes using SQL statements.
Rather than executing your SQL statements on the fly, you may want to create a set of programs called “stored procedures”. You can pass arguments to them just like you would for a function.
A query plan is an ordered set of steps to access the data. The query plan is determined by the query optimizer located inside the database. Your SQL statements and stored procedure merely state your intentions of how you want to retrieve the data. The query plan specified by the query optimizer shows the exact steps sequentially of how your SQL statements are executed by the database. Analyzing the query plan is one way to optimize and tune your stored procedures and SQL statements.
Constraints enforce data integrity inside an RDBMS database. You can configure constraints to enforce limits on the data and type of data. Constraints can be set either at the column level or at the table level. A few different types of constraints could be: NOT NULL, UNIQUE, PRIMARY KEY, FOREIGN KEY, CHECK, INDEX.
Primary Key — It is the constraint that uniquely identifies each record in the table. It is usually the unique record id.
Foreign Key — It is the constraint that allows a parent-child relationship to be established in a table. Usually, the table with the foreign key is the child table while the table without the foreign is the parent table.
Here’s a good blog post describing the relationship of Foreign keys and Primary keys in the MySQL database.
Normalization is the process of efficiently organizing the data in your RDBMS database. The idea is to eliminate redundant data. For instance, if you have a table of ClientInfo, your client has three different addresses, to normalize the data, you want to store the client information in a table called ClientInfo indexed by a ClientId. Then, you want to store the three different addresses in a ClientAddress table with the ClientId. After that, you can relate the two tables together by the ClientId. By ensuring “unique” data in each table, you can saving storage space and optimize for performance. (That is the idea, anyway.)
Setting up the MySQL database is very easy. You simply download the executable to your Mac or PC and run it to install the executable.
Then, you connect to the server using the MySQL client.
Here are the instructions that you can follow.
The following query creates the table with a primary key.
CREATE TABLE ClientInfo (  ClientId INT NOT NULL  ClientName VARCHAR(255)  PRIMARY KEY (ClientId));
The following query retrieves data from one table.
SELECT * FROM ClientInfo
The following query retrieves data from two tables by inner join:
The output will be all the records that represent the overlap of both ClientInfo and ClientAddress table.
SELECT * from ClientInfoINNER JOIN ClientAddressONClientInfo.ClientId = ClientAddress.ClientId
The following query retrieves data from two tables by implicit “inner” join notation:
The output will be the same as the output from before.
SELECT * from ClientInfo, ClientAddressWHEREClientInfo.ClientId = ClientAddress.ClientId
The following query retrieves data from two tables by outer join:
The output will be all the records of ClientInfo + the overlap of ClientInfo and ClientAddress table.
SELECT * from ClientInfoLEFT OUTER JOIN ClientAddressONClientInfo.ClientId = ClientAddress.ClientId
The following query creates a unique index on a table:
CREATE UNIQUE INDEX ClientId ON ClientInfo(ClientId)
Now that you have an idea of how to interact with the MySQL tables and retrieve information from the tables, here are the next steps.
Create a simple project working with the data in your database. The easiest way is to load a set of “dirty data” into the database. Then, go through data cleaning steps using SQL instead of using pandas.After that you can delve a little deeper into the following references:
Create a simple project working with the data in your database. The easiest way is to load a set of “dirty data” into the database. Then, go through data cleaning steps using SQL instead of using pandas.
After that you can delve a little deeper into the following references:
Different types of SQL joins
Commonly used SQL functions
Different types of SQL indexes
Different types of SQL datatypes
SQL Query Performance and Tuning
Now that you have studied the basic concepts, worked through your first project and checked out some references, it’s time to test your knowledge with these interview questions.
www.interviewbit.com
www.softwaretestinghelp.com
About the Author
Jun Wu is a Content Writer for Technology, AI, Data Science, Psychology, and Parenting. She has a background in programming and statistics. On her spare time, she writes poetry and blogs on her website.
Subscribe to my weekly newsletter to stay connected