How to Merge Multiple Data Sets in R With Binds and Joins
By: Tyler Harris
Tags: R, Data Science, How To, Data Wrangling, R Programming

Rename Some Columns

We can do this a lot of ways, but this is the simplest illustration I could think to use. At the end of the day, we need columns that mean the same thing to have the exact same name.

Here’s the code:

# I am going to change df1's names to be the same as df2. It's definitely possible to go the other way, but I think this looks better.

# If you are using RStudio, you can click on the object name in the Environment tab or do "view(object)" to see the data # Change df1's names to match df2

names(df1)[1] <- "Price" # Column 1 is price, but we want it to be Price like df2

names(df1)[2] <- "Item" # same as Price

names(df1)[3] <- "Retailer" # same deal again # Check the names worked

head(df1) # There we go! Now df1 and df2 have the same names but in a different order.

Here’s the output:

Let’s revisit rbind()

Now that df1 and df2 have the same column names, let’s revisit our old friend rbind()!

Here’s the code:

# Let's try rbind again to try to merge df1 and df2. They have the same column names, so this should be great, right? rbind_test_2 <- rbind(df1, df2) # Check result

rbind_test_2 # Look at that! We got our Price, Item, and Retailer columns combined!

# rbind will "bind" the rows from the second data set to the first. df2 "binds" to df1.

# Now let's see what happens with adding df3 into the equation

Here’s the output:

rbind() Function Works When Column Names Are Exactly the Same with R

Incorporate df3

With a successful binding of df1 and df2, R automatically knows to bind the right columns together. Our next task is to get df3 to cooperate. Let’s take a look at some common errors after we rename the “Price” and “Item” columns to be in line with our previous schema.

Here’s the code:

# We know we need to change the column names, but df3 has different parameters. What's going to happen when we try to bind them together? # Let's change the names we know we need to change

names(df3)[2] <- "Price"

names(df3)[3] <- "Item" # use rbind

rbind_test_3 <- rbind(rbind_test_2, df3) # check data

rbind_test_3 # Oh no! number of columns do not match. How can we resolve this?

# We could try taking out columns, but the data would not match up. We would only have two out of four columns be the same anyway. That doesn't really get us that far. We'll probably need some kind of join to accomplish this

Here’s the output:

rbind() Function Error with R

The rbind() function does not want to cooperate because it is trying to put three columns with five, which it does not understand. We need a different tactic.

Try Inner Join Again

Inner join kind of worked last time. Maybe we can try that again? Let’s see!

Here’s the code:

# Inner Join

IJtest <- inner_join(rbind_test_2, df3)

IJtest # 0 observations of 5 variables? Why? # What if we swap which is the right and which is the left table?

IJtest2 <- inner_join(df3, rbind_test_2)

IJtest2 # Still 0 observations of 5 rows # When we inner join, we are looking for where ALL the rows in the first table have matching values in the second table. Since we do not have all matches, we are not actually joining anything. The only step that is completed is R figuing out we're trying to get 5 columns.

Here’s the output:

inner_join() Function Kind of Working with R

When we inner join, we are looking for where ALL the rows in the first table have matching values in the second table. Since we do not have all matches, we are not actually joining anything. The only step that is completed is R figuring out we’re trying to get 5 columns.

Dang. Let’s try something else.

Try Left Join

Left join should help us out and get the right number of columns. It merges by columns that overlap, so maybe this will get our result.

Here’s the code:

# Left Join

LJtest <- left_join(rbind_test_2, df3)

LJtest # Alright, well we got some of the data. However, the only thing we accomplished is adding the two extra columns to the picture. Left joining is still just looking for matches that aren't there

Here’s the output:

left_join() Function Kind of Works with R

We only have 9 rows, when we should have 15. We’re getting closer!

Try Right Join

The right_join() function does a similar thing to left join, but it has a different order of operations and how the tables get combined. Let’s take a look.

Here’s the code:

# Right Join

RJtest <- right_join(rbind_test_2, df3)

RJtest # Right join is interesting because we get the five columns, but only the six rows of df3. This is because we return the rows from the second table (right table) and all rows from the first table (left table) that have matches. The only thing that matches these conditions is just the data in df3. Kind of weird, but that's how it works.

Here’s the output:

right_join() Function Kind of Working with R

Not quite what we wanted, but we can’t be that far away from getting it!

Try Full Join

Using full_join() should combine all of our rows and columns the right way, at least in theory. Let’s test it out!

Here’s the code:

# Full Join

FJtest <- full_join(rbind_test_2, df3)

FJtest # Success! We got the right table. We have 5 columns with 15 rows with all the data. If you want a slightly more appealing looking table, we can switch the tables around in the full_join() function.

# According to the documentation: full_join will "return all rows and all columns from both x and y. Where there are not matching values, returns NA for the one missing." That is exactly what we did here. The next cell shows what we could have done from the beginning to simplify our efforts.

Here’s the output:

Successful full_join() Function Part 1

Successful full_join() Function Part 2

We did it! We got all three tables to cooperate with a full_join() function! Since I did this in an R Markdown file, it breaks up the tables like this, so I left it like this and took another screenshot. Depending how you do it, it may be a single table, broken up like mine, or printed to the console.

Another Way We Could Have Done This

Just like most problems in R, there is more than one way to solve the problem. Here is an alternative solution or two that can help you get a better understanding of the magic of the full_join() function!

Here’s the code:

# We could have done all the name changes at one time. I copy/pasted from above for consistency and simplicity.

# Change df1's names to match df2

names(df1)[1] <- "Price" # Column 1 is price, but we want it to be Price like df2

names(df1)[2] <- "Item" # same as Price

names(df1)[3] <- "Retailer" # same deal again # Change df3's names to match df2

names(df3)[2] <- "Price"

names(df3)[3] <- "Item" # rbind df1 and df2

FJintermediate <- full_join(df1, df2)

# RBfinal <- rbind(df1, df2) # gives the same result # Full Join both sets together

FJfinal <- full_join(df3, FJintermediate)

# FJfinal2 <- full_join(df3, RBfinal) # gives the same result # Check final data

FJfinal

# FJfinal2 # gives the same result # Note that there is a way to do the reassignment of column names within full_join(), but it is a little dense and harder to explicitly see what is happening with the code. The documentation shows how to do this.

Here’s the output:

Final Success Image with full_join() in R

Note: The last five rows are on the other table like we just saw above. This is long enough so I didn’t include it for brevity.