Identify and Remove Duplicate Data in R - GeeksforGeeks

26 Mar, 2021
A dataset can have duplicate values and to keep it redundancy-free and accurate, duplicate rows need to be identified and removed. In this article, we are going to see how to identify and remove duplicate data in R. First we will check if duplicate data is present in our data, if yes then, we will remove it.
Data in use:
For identification, we will use duplicated() function which returns the count of duplicate rows.
Syntax:
duplicated(dataframe)
Approach:
Create data frame
Pass it to duplicated() function
This function returns the rows which are duplicated in forms of boolean values
Apply sum function to get the number
Example:
R
# Creating a sample data frame of students # and their marks in respective subjects.student_result=data.frame(name=c("Ram","Geeta","John","Paul",                                 "Cassie","Geeta","Paul"),                          maths=c(7,8,8,9,10,8,9),                          science=c(5,7,6,8,9,7,8),                          history=c(7,7,7,7,7,7,7))  # Printing datastudent_resultduplicated(student_result)sum(duplicated(student_result))
Output:
> duplicated(student_result)
[1] FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE
> sum(duplicated(student_result))
[1] 2
Approach
Create data frame
Select rows which are unique
Retrieve those rows
Display result
Method 1: Using unique()
We use unique() to get rows having unique values in our data.
Syntax:
unique(dataframe)
Example:
R
# Creating a sample data frame of students # and their marks in respective subjects.student_result=data.frame(name=c("Ram","Geeta","John","Paul",                                 "Cassie","Geeta","Paul"),                          maths=c(7,8,8,9,10,8,9),                          science=c(5,7,6,8,9,7,8),                          history=c(7,7,7,7,7,7,7))  # Printing datastudent_resultunique(student_result)
Output:
Method 2: Using distinct()
 Package “tidyverse” should be installed and  “dplyr” library should be loaded to use distinct(). We use distinct() to get rows having distinct values in our data.
Syntax:
distinct(dataframe,keepall)
Parameter:
dataframe: data in use
keepall: decides which variables to keep
Example:
R
# Creating a sample data frame of students and # their marks in respective subjects.student_result=data.frame(name=c("Ram","Geeta","John","Paul",                                 "Cassie","Geeta","Paul"),                          maths=c(7,8,8,9,10,8,9),                          science=c(5,7,6,8,9,7,8),                          history=c(7,7,7,7,7,7,7))  # Printing datastudent_resultdistinct(student_result)
Output:
Example 2: Printing unique rows in terms of maths column
R
# Creating a sample data frame of students and# their marks in respective subjects.student_result=data.frame(name=c("Ram","Geeta","John","Paul",                                 "Cassie","Geeta","Paul"),                          maths=c(7,8,8,9,10,8,9),                          science=c(5,7,6,8,9,7,8),                          history=c(7,7,7,7,7,7,7))  # Printing datastudent_resultdistinct(student_result,maths,.keep_all = TRUE)
Output:
Picked
R-Functions
R Language
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Comments
Old Comments
Change Color of Bars in Barchart using ggplot2 in R
How to Change Axis Scales in R Plots?
Group by function in R using Dplyr
How to Split Column Into Multiple Columns in R DataFrame?
Replace Specific Characters in String in R
How to filter R DataFrame by values in a column?
How to filter R dataframe by multiple conditions?
R - if statement
How to import an Excel File into R ?
Time Series Analysis in R