11 Dplyr Functions to Start Using Right Now in R | by Amanda West | Towards Data Science

As a freshman in college, when my life goal was still to become a hot-shot diplomat or United Nations ambassador, I took an Uber back to my university with a techie guy wearing our school colors that happened to live a few blocks from my dorm. Trying to make small talk the best I could, I asked him about his life and found out that he was a recent engineering grad turned data scientist at Domino’s.
“Oh, that’s cool...what, uh, packages do you use the most?” I said, proud of myself just for remembering what “packages” were in code-speak.
“Mostly Dplyr.”
“Sounds...interesting.” I said, nodding and looking down at my hands. I tried to think of something to ask him about pizza after that, like if he got free pizza for life and if he believed it was possible to be addicted to cheese, but instead just looked out the window and continued played with my hands for thirty minutes. In the end, I gave him my number so he could Venmo request my half for the ride, but just like the deafening silence of that rideshare, I never got a follow-up text from him again (let’s hope he doesn’t read TDS at work, lol).
So now here I am, still cringing from a car ride I took with a nameless stranger four years ago. But maybe, just maybe, if you get stuck in a car with a stranger who knows Dplyr, you can use the contents of this article to have a conversation. And then you, my dear reader, will not spend four years of your life wondering what Dplyr is, realize your old dreams are no longer, teach yourself how to code, graduate in a global pandemic, explore an entirely new field in graduate school, and eventually become a data scientist yourself and revisit that initial awkward conversation in the form of a TDS article.
OK, but in all seriousness, Dplyr is pretty dang cool (another grand invention by the one and only Hadley Wickham). In terms of scope, Dplyr is to R what Pandas is to Python, but both of these languages also contain very strong similarities to SQL as well. If you find yourself relying on base R a lot, there are a number of everyday data wrangling tasks that Dplyr’s %>% can probably solve more efficiently and intuitively for you.
Before I jump into the top functions that you need to start implementing right now, there’s an elephant in the room that we should first discuss...
Alternative name for this article: What the $@&% is the %>% operator?
The %>% (read: “pipe”) is an operator in R that feeds what is before it into what comes after. This is important, because the pipe operator can be used sequentially to chain together many layers of commands. Because of this, one always starts with the dataset they wish to modify, followed by another pipe, and then the modifier. There can be an unlimited number of modifiers added but they must be broken up by a pipe between each to work. For me, I like to think of the %>% in Dplyr working similarly to how a . is utilized in Pandas. Alternatively, if x represents the base dataset, then one %>% is like putting x into the function f(x), two layers is g(f(x)), three layers is h(g(f(x))), so on and so forth.
Below is an example Pandas vs. Dplyr to rename two columns. In my humble opinion, Dplyr actually comes out a little ahead in terms of readability:
# pandasdf.rename(columns={'old_name1': 'new_name1',                    'old_name2': 'new_name2'}, inplace=True)--------------------------------------------------------------------# dplyr df %>%  rename(new_name1 = old_name1,          new_name2 = old_name2)
And here are some examples of chaining together multiple pipes, which can be done an unlimited number of times:
# in order to follow along, run these: library(tidyverse) library(gapminder)# one pipe gapminder %>%  select(country, continent, year, pop)# two pipesgapminder %>%  select(country, continent, year, pop) %>%  rename(population = pop)# three pipesgapminder %>%  select(country, continent, year, pop) %>%  rename(population = pop) %>%  filter(year > 1985)# four pipesgapminder %>%  select(country, continent, year, pop) %>%  rename(population = pop) %>%  filter(year > 1985) %>%  arrange(desc(population)) # for the last, notice that we say "arrange(desc(population))" because in step two it was renamed from "pop". 
Now that we’ve gotten that out of the way we can begin to dive into the functions themselves!
glimpse(): Returns 1) # rows, 2) # cols, 3) datatype of each column, and 4) the first ~10 or so entries (depending on what the space on your screen). The corresponding Pandas function I would equate this to is df.describe().
library(tidyverse) library(gapminder)gapminder %>%  glimpse()
arrange(): Having a set of eyes on the data can sometimes be incredibly beneficial, so we use arrange() to shuffle various columns in ascending or descending order. This is most similar to the ORDER BY clause in SQL or df.sort_values() in Pandas.
# sort ascending (default)gapminder %>%   arrange(lifeExp)# sort descending gapminder %>%   arrange(desc(lifeExp))# sort on multiple columnsgapminder %>%   arrange(lifeExp, pop, country)
select(), transmute(): The select() function takes a snapshot of the original dataset with only the columns that you specify. Similarly, transmute() works in the same way with the additional ability to create new columns simultaneously. Both of these functions work extremely similarly to the SELECT clause in SQL.
If you are simply wishing to pull untouched columns from the dataframe, I’d stick with select, while if you are going to be making transformations, transmute is the way to go. In the example below, we have the variables gdpPercap and pop and we are making a new gdp column.
gapminder %>%  select(country, continent, year, pop)gapminder %>%  transmute(country, continent, gdp = gdpPercap * pop)
There are also some more advanced options with select to help you get exactly the columns you want, including a string of connected columns or all columns that contain, start, or end with certain characters:
# advanced select() tools# 1. rename variables with select() gapminder %>%  select(country, continent, population = pop)# 2. Include a string of columns at once with ':'gapminder %>%  select(country, year, lifeExp:gdpPercap)# 3. Include any column with a given string gapminder %>%  select(country, year, contains("gdp"))# 4. Include any column starting with a given string gapminder %>%  select(country, year, starts_with("co"))# 5. Include any column ending with a given stringgapminder %>%  select(country, year, ends_with("Exp"))# Keep the last column, by any namegapminder %>%  select(country, year, last_col())
rename(), mutate(): These functions modify one or more columns and saves them back to the file. As the name suggests, rename() is for renaming while mutate() creates a new function entirely from a transformation.
With both rename() and mutate(), we can use a comma to denote multiple modifications at once. Renaming in Dplyr is similar to using df.rename() and mutating is similar to df[new_col] = df[old_col1] * df[old_col2] used in Python.
# rename a column to be called something elsegapminder %>%  rename(population = pop,       #ex. 'pop' turns into 'population'         countries  = country,          life_expectancy = lifeExp)# add a new calculated fieldgapminder %>%  mutate(gdp = gdpPercap * pop)  #ex. 'gdp' is created as a new var
filter(): Similar to the WHERE clause in SQL or a conditional expression in Python (df[df['country'] == 'France']). Akin to other Dplyr functions, one can specify multiple filters by breaking up each argument with a ,, &, or | sign. For strings, using %in% is suggested over using == .
# filter for Francegapminder %>%   filter(country %in% "France")# filter for a subset of countries (c stands for 'concat')gapminder %>%   select(pop, country, year) %>%        filter(country %in% c('France','Spain','Portugal'))# "or" operator functiongapminder %>%   select(pop, country, year) %>%          filter(pop > 50000 | pop < 100000)# "and" operator function 1 gapminder %>%   select(pop, country, year) %>%          filter(pop > 50000, pop < 100000)# "and" operator function 2gapminder %>%   select(pop, country, year) %>%          filter(pop > 50000 & pop < 100000)
count(), summarize(), and top_n(k, )
The count() function goes through the dataset and counts the number of times a value appears in a given column/columns. I most often use it to look at one column at a time, such as for understanding how many countries are in each continent that I’m looking at or how many people in a city take a car, bike, or bus to work.
gapminder %>%  count(continent)
With summarize we can look at aggregate functions such as the sum, median, mean, standard deviation, variance, min, and max of a column and give it a name. When mixed with functions like group_by() , these functions can become incredibly useful in understanding subpopulations in the dataset.
# summarize # sum(), mean(), median(), min(), max(), sd(), var()gapminder %>%  summarize(total_gdp = sum(gdpPercap))gapminder %>%  summarize(avg_gdp = mean(gdpPercap))gapminder %>%  summarize(gdp_standard_dev = sd(gdpPercap))gapminder %>%  summarize(gdp_variance = var(gdpPercap))# we can also create many columns at oncegapminder %>%  summarize(max_gdp = max(gdpPercap),            total_pop = sum(pop),             min_lifeExp = min(lifeExp))# I use __ = __ in order to give a name to the new summary columns, # but it is not required.
Finally, we can use the top_n(k, ) to only keep the top k values in a dataframe or group. For example, we can look at the country in each continent with the highest GDP per capita below. If we had not included the group_by() function first, then result would be the single country in the world with the highest GDP per capita, i.e. Kuwait. Similarly, if we had specified k=4 then it would have shown us the top four countries with highest life expectancies in each continent.
# top_n(k, )gapminder %>%   group_by(continent) %>%   top_n(1, gdpPercap)
group_by() & ungroup(): As with all programming languages, where would we be without a good group by function? Used in combination with the other functions listed, we can draw out interesting conclusions about the subpopulations present in the dataset. This function works the same as in Pandas df.group_by() or SQL’s GROUP BY, with the exception being that these groups will stay put until you manually uncouple them with ungroup() again.
# Using group_bygapminder %>%        group_by(year) %>%                      mutate(year_total = sum(pop))# Grouping and then ungrouping again with ungroup()gapminder %>%        group_by(year) %>%                      mutate(year_total = sum(pop)) %>%        ungroup() %>%                            mutate(fraction = pop / year_total)
All in all, I hope that this introduction gets across to you that Dplyr has strong similarities to both SQL and Pandas. If something about a data transformation is tripping you up, Dplyr is strong on efficiency and readability — and there’s plenty of cheat sheets and tutorials floating around for it on the interwebs!
If you liked this article on data wrangling with Dplyr please let me know with a clap or comment— if there’s enough interest then I will do a part 2 where I go over the more complex commands lying within Dplyr, and perhaps find more awkward stories of my past to expose in the process. 🙃
And to Hadley Wickham, the maker of all things R....seriously, when do we just make him our supreme overlord already?
**Edit: Base R is getting a pipe, after all! Link: https://www.youtube.com/watch?v=wmvjxb6Gdm8