A beginner’s guide to joining data | by Skyler Dale | Towards Data Science

Joining data is one of the fundamental skills of data analysis. Unfortunately, it can also be a bit confusing when you’re just getting started, or when you’re making the transition from excel to SQL or Python.
In this article, I’ll walk through an intuitive explanation of what it means to join data and how to do it effectively.
The best way to understand joins is to work through a simple example.
Let’s say you work for a company that sells T-shirts on a website.
You have one dataset that tells you details about your website users. For simplicity’s sake, let’s say this table contains just a unique client ID, client name, and country of origin:
You have another dataset that tells you all of the purchases that those website users have made:
Your boss walks into the room and demands that you tell her which country has the most website sales.
To answer her question, you need to bring the data that tells you where your clients are located together with the data that tells you what they’ve purchased. In other words, for each transaction, you need to figure out where the client was located.
What makes this possible is the fact that there is some commonality between the two datasets. In this case, it’s the client ID.
To “join” is simply to combine data based on a common data point. Fittingly, that common data point is called a “join key.”
For this example, we can join the user data to the purchase data with the SQL query below, which uses “client ID” as the join key:
select purchases.client_id, purchases.sale, users.country from purchasesleft join userson purchases.client_id = users.client_id;
And voila:
Now we can see that Mexico has the most sales. (Note: if we had more than a few rows, we would want to use a “groupby” and a “sum” function. But since we don’t have much data in the above example, we can easily answer this question on sight and leave those topics for another day.)
If you’re new to data analysis, you may find yourself wondering, why do we need to join data in the first place? Why can’t the data just live in one table?
There are two main reasons for this. First, data often gets acquired from different sources. Your process for getting the users data and the transaction data might be separate.
If you’re a power user of excel, think about all the times you’ve had to do a vlookup to connect two data sources. In essence, that was a join.
The second reason has to do with speed. Generally, the more data you have in one table, the longer it will take to retrieve what you need from it.
Great, now we have the intuition behind what joins are and why we need them. Next, we’ll need to learn about the different types of joins.
This is an area where people often get tripped up, so let’s try to make this part very clear.
There is one reason why we need multiple types of joins. If you can remember the reason, then you will be able to remember the types.
Here’s the reason: When the values in your join key do not exist in both datasets that you are joining, the computer needs to know what to return.
Now let’s look at an example based on our dataset from earlier.
The client_id of “8367” is in the users table:
But its not in the transactions table:
Thus the “type” of join simply tells the computer what to do in this scenario.
As the proud author of the query — you have control over what you want the computer to return.
A “left join” tells the computer: “give me everything in the left table, even if it’s not in the right table.”
Remember from the example earlier — we used a left join and got only the client_ids from the left table:
A “full outer join” tells the computer, “give me all the records in both tables.”
As a result, we get the client ids from both tables, including 1902 and 8367 — which are only in the users table — as well as 1000, which is only in the purchases table.
Notice that the some of the cells in the table above have “null” (ie. missing) values. This makes sense because this data does not actually exist in the original tables.
We know client id 1000 made a $10 purchase, but don’t know where this client is from. If it was missing in the original data, it will be missing after the join.
Finally, an “inner join” tells the computer — “give me only the records whose join keys are in both tables.”
The above query returns:
Note that all of the non-overlapping client IDs — 1902, 8367, and 1000 — are missing from the result set.
A few things to consider as you learn more about joining data:
Right Joins: A “right join” tells the computer “give me everything in the right table even if it’s not in the left table”. I’ve never used a right join. Why? Because anytime you want to do a right join you can just flip the order of the tables and do a left join. One less thing to remember.Multi-key joins: You can join on more than one column value. (ie. left join on table1.id1 = table2.id1 AND table1.id2 = table2.id2)Duplicates: Always pay attention to the number of records in your table before and after a join. A common beginner mistake is to execute a join that creates unwanted duplicates. If you run into this scenario, you will have to think critically about how you can rewrite the join to give you what you truly want.
Right Joins: A “right join” tells the computer “give me everything in the right table even if it’s not in the left table”. I’ve never used a right join. Why? Because anytime you want to do a right join you can just flip the order of the tables and do a left join. One less thing to remember.
Multi-key joins: You can join on more than one column value. (ie. left join on table1.id1 = table2.id1 AND table1.id2 = table2.id2)
Duplicates: Always pay attention to the number of records in your table before and after a join. A common beginner mistake is to execute a join that creates unwanted duplicates. If you run into this scenario, you will have to think critically about how you can rewrite the join to give you what you truly want.