How your smartphone tells your story: A dive into Android activity data
By: Atharva Jakkanwar
Tags: Android, Digital, Data Science, Data Visualization, Pandas

| This article was written in October 2018, but in published in June 2019|

I was really excited when Google announced their Digital Wellbeing program, back in May 2018, especially Dashboard. It tracks all your app interactions on the phone and even helps you to limit app usage by setting time restrictions on different apps. But as of October 2018, Google still hasn’t rolled out that feature to all Android P users and is in beta even for Pixel users. So I decided to check out my own statistics with the data available at hand.

Google: My Activity

So how did I get my app interaction history? Well, Google has to have all of it. I’m an Android user. By default, the usage history for your Android device activity is turned on in your Google activity settings. It keeps a log of all the apps you open along with a timestamp. Unfortunately, it doesn’t store the duration you spent using the app. But fortunately enough, Google allows this data to be downloaded (and even be erased), the treasure trove lies at takeout.google.com in the My Activity section.

Scraping the Soup

While I was expecting all of this data to be either in JSON or CSV format, I got happy too soon. Some of the data was available in these formats, but not the Android app activity, the entire activity set can be downloaded only in HTML format. Really?! HTML?. Beautiful Soup to the rescue. Its a really popular python library for web scraping.

UPDATE: The newer version as of May 2019 has JSON format data but about a random tenth of the records from the over 2 years back are missing. Strange how logs disappear!

The only 2 important pieces of information in this file were the app name and the time at which it was opened. Here is div associated with one use of an app from the HTML file:

I parsed the HTML using Beautiful Soup, extracted the app names and time stamps and dumped them into a Pandas data frame.

Over to cleaning the data.

Skadoosh!

Panda (by Mélody P on Unsplash)

With about 80000 data points over a period of 3.5 years from 474 unique apps (When did I install these many?!) it was pretty evident something was wrong. A peek into the data showed that there were many system apps included in the list, like the home launcher, face-unlock, system clock, etc; about 102 of these. Here I also realized that the data was from 3 devices, a Xiaomi Mi 4, Moto G5 and a OnePlus 6. After removing all the entries associated with the system apps, I was left with 372 apps, 67236 app interactions over a period of 1134 days.

Here is a word cloud generated from the different words in the names of the apps:

Of the 372 apps, I was pretty sure that I did not have most of them installed for long periods of time. Some apps I installed only to try them out.

Only 21 apps had over 500 interactions:

['Wear OS by Google Smartwatch (was Android Wear)', 'Instagram', 'Google', 'Twitter', 'Google Chrome: Fast & Secure', 'Gmail', 'WhatsApp Messenger', 'Android', 'Google Photos', 'Splitwise', 'YouTube', 'Messenger – Text and Video Chat for Free', 'Contacts', 'Google Calendar', 'Home', 'Moto Camera', 'Facebook', 'Truecaller: Caller ID, spam blocking & Call Record', 'Calorie Counter - MyFitnessPal', 'com.android.camera', 'Inshorts - 60 words News summary']

MyFitnesPal and Inshorts surprised me, maybe they had the most interactions because of their constant push notifications. Similarly Truecaller also spun up with each call and hence made it to the list.

8 Google Apps, 3 Social Media Apps, 2 Messaging Apps, 2 camera Apps.

Also, the number of Google Apps was about a tenth of all the other apps.

For further analysis, I only considered apps that had more than 100 interactions (those that I used about once every week on an average). This brought the unique app-count down to 59. Here are the interaction counts for these 59 apps:

Also, when I say ‘interactions’, it only refers to the number of times I opened the app and no way indicates the duration I spent on it.

History and Lore

Here is a plot of my daily app interactions since the time this data was being collected over a period of 3.5 years: