Exploratory Data Analysis on Google Play Store Apps
By: Manish Kumar Shah
Tags: Pandas, Visualization, Matplotlib, Data Science, Data Analysis

Data Preparation and Cleaning

Data preparation is the process of cleaning and transforming raw data prior to processing and analysis. It is an important step prior to processing and often involves reformatting data, making corrections to data, and the combining of data sets to enrich data. Data cleansing or data cleaning is the process of detecting and correcting (or removing) corrupt or inaccurate records from a recordset, table, or database and refers to identifying incomplete, incorrect, inaccurate, or irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data.

We saw that the dataset contains many Null or missing values. The column Rating , Type , Content Rating , Current Ver , and Android Ver contains 1474, 1, 1, 8, and 3 missing values respectively.

Will it not be better if we can define a function to get more useful information about the different attributes of the dataset, also there is one more valid point in defining a function which it will be reusable, and we are going to utilize our defined function several times in future.

Let’s call the function and see what it returns:

We have some useful information about the dataset. i.e., we can now see the missing number of values of any attribute, its unique count, and its respective data types.

Now we can start the process of data cleaning, lets start with the column Type :-

Since there is only one missing value in this column, So, let’s fill the missing value. After cross-checking in the play store the missing value is found to be Free, So now we can fill the missing value with free .

After filling the value we can check and see if that has been correctly placed.

Now, we can move on to the column Content Rating :

By, looking only at these rows it is not easy to say what's actually missing in this row. let us have a look at all of its near rows data. For this purpose, we have iloc and loc function.

We can clearly see that row 10472 has missing data for the Category column and all the prevailing column values are being replaced with its previous column. A better idea will be to drop this row from our data frame.

We are having some of the unwanted columns which will be of not much use in the analysis process. So let’s drop those columns.

Now, we can fix the Rating column which contains a total of 1474 of missing values. Replacing the missing values with the Mode value of that entire column.

Finally, after fixing all the missing values, we should have a look at our data frame, We defined a function as printinfo() . So, it’s time to use that function.

All the columns have the null_count as zero, which indicates that now the data frame doesn’t contain any missing values.

Now we are done with the data cleansing part and in a state to start the work for data preparation

Columns like Reviews , Size , Installs , & price should have an int or float datatype, But here we can see of object type, So let’s convert them to their respective correct type.

Starting with the column Reviews , converting its type to int .

We can see that the changes have taken its effect or not by calling our printinfo() function.

Now, the reviews column has been converted to int type, so now we can move to the Column: Size

Converting the Size Column from object to integer, but this column contains some of the special characters like , , + , M , K & also it has a some of the value as Varies with device . We need to remove all of these and then convert it to int or float .

Removing the + Symbol:

Removing the , symbol:

Replacing the M symbol by multiplying the value with 1000000:

Replacing the k by multiplying the value with 1000:

Replacing the Varies with device value with Nan :

Now, finally converting all these values to numeric type:

So, after performing all of these operations, we should have a detailed look at that column, so yes again we will call our useful function which we defined. i.e., printinfo()

Since we converted the Varies with device value to Nan , so we have to do something with those set of Nan values data. It will be a better idea to drop the Rows of the column Size having Nan values because it will be not an efficient idea to replace those values with mean or mode since the size of some apps would be too large and some of them too small.

Column: Installs :

To convert this column from object to integer type. First of all, we will need to remove the + symbol from these values.

and then let’s remove the , symbol from the numbers.

Lastly, we can now convert it from string type to numeric type, and then have a look at our dataset.

So, now we are only left with the Price column.

Column: Price :

Converting this column from object to Numeric type.

The values contain a special symbol $ which can be removed and then converted to the numeric type.

After fixing all the issues, we should have a final look at the data frame.