Communicating data using visualizations
By: Emmanuel Sibanda
Tags: Data Science, Python, Data Visualization

Communicating data using visualizations

Communicating data using visualizations

I downloaded my Facebook data analysing my online behaviour and updating frequency and general engagement over the last decade.

This blog post starts the series of social media analysis blogs off with an analysis of my Facebook updates. The intention of this first analysis was to get an understanding of: i.) how often I have posted on my Facebook account between 2009 and 2018 and breaking these updates according to types of updates i.e posting a status update, a post (this refers to either sharing a picture, news article or resharing a Facebook post with some personal commentary attached to it), sharing something on a friend's timeline, sharing an article and posting a link to an article, ii.) Yearly percentage likelihood to post particular types of updates by year and iii.) my propensity to post by weekday, hour and month measured by the weighted average of posting frequency over the last 10 years. This will serve as a segue into a more detailed analysis about the types of content I have been posting over the last decade, leaving out the obviously cringeworthy posts younger me may have made.

import pandas as pd

import datetime

import matplotlib.pyplot as plt

import numpy as np

from google.colab import drive

drive.mount('/content/gdrive')

fb_posts = pd.read_csv(‘/***/myposts.csv’)

I saved the Facebook data on my Google Drive; in order to interact with this data on Google Colab I had to use the google.colab package to mount the GDrive and open the folder and the csv relevant to my analysis.

Reading the file

#I need to convert this into a readable datetime

#iterate through timestamps and convert individual rows

fb_posts[‘timestamp’] = fb_posts[‘timestamp’].apply(lambda x: datetime.datetime.fromtimestamp(int(x)))

#create month, day, year and time columns

fb_posts[‘year’] = fb_posts.timestamp.dt.year

fb_posts[‘month’] = fb_posts.timestamp.dt.month

fb_posts[‘day’] = fb_posts.timestamp.dt.day

fb_posts[‘time’] = fb_posts.timestamp.dt.time #making the assumption that if 95% of the rows are NaN values, the column doesn’t contain valuable info for the purposes of our analysis

#use dropna and specify threshhold of na values

fb_posts = fb_posts.dropna(thresh=len(fb_posts)*0.05, axis=1) #use list(fb_posts) to id the remaining columns

fb_posts = fb_posts.drop(fb_posts.iloc[:,[1,4]], axis=1)

The first thing you notice about the data you import from facebook is that the timestamp of your updates is imported in unix timestamp format (a number that represents the seconds that have occurred since the first of January 1970) because it represents all timezones at once. In order to convert this timestamp into readable time I apply a lambda function on each row in the timestamp column. So I iterate through each row and apply a lambda function that converts the timestamp into a datetime format.

I then move on to create year, month, day and time columns using .year, .month, .day, and .time respectively. I need to get these columns to answer the questions I set out to answer in the opening paragraphs.

Currently the dataframe has approximately 126 columns, most of which are not useful to our analysis. To filter out the non-essential columns I drop the columns were NaN values constitute 95% and more of the values in the rows. I do this to reduce the number of columns so I can zero in on columns that I actually want to use for my analysis (and possibly finding other columns with interesting information).

Once I have fewer columns, I then limit the columns according to the ones that are most relevant to our analysis.

Data Cleanup

#rename column

fb_posts.rename(columns = {'data/0/post':'posts', 'attachments/0/data/0/external_context/url':'links'}, inplace=True)

fb_posts['title'] = fb_posts['title'].astype(str)

for x in ['link', 'post', 'status', 'timeline','recommend','books','reviewed','playlist','feeling','event','likes','listened','played','published','shared','was','followed','commented','song','responded','wants','invested','photo']:

fb_posts.loc[fb_posts['title'].str.contains(x), 'title'] = x

#remove chart junk

removal = ['recommend','books','photo','reviewed','playlist','nan','feeling','event','likes','listened','played','published','was','followed','commented','song','responded','wants','invested']

fb_posts2= fb_posts[~fb_posts['title'].isin(removal)]

Now here comes the fun part: the clean up. I start off by renaming the columns to names that are easier to refer to: then proceed to convert all items in the title column to a string format. I do this in order to help us classify the type of post each row represents using a one word reference (e.g status = status update).

By converting all items in the title column to string, I can run through each string in the title column and check whether it contains any of the words that appear in the list of words defined as the 'x' list in the above code block. If they do I change the value of the row to the relevant reference text in the 'x' list.

I then iterate through this list and remove values that refer to the reference words specified in the removal list. I do this because I only want to deal with status, post, link, share and timeline updates. I assume these make the bulk of my Facebook activity based on an understanding of the content I have posted throughout my Facebook history.

Plotting Frequency of updates by year and type

fb_posts2.groupby([‘year’,’title’]).size().unstack().plot.bar(stacked=True)

Frequency of Facebook updates

In order to visualise our chart, I group by year and count the occurrence of each type of update (title). Interestingly, the frequency kind of resembles the gaussian distribution, frequency peaks in the middle between 2012 and 2015, with the lowest number of total updates being at the tails 2009 and 2018. This isn't necessarily an indicator of me being less active on social media, it is a case of me increasingly migrating from Facebook to Twitter. This is primarily because of the learning value that can potentially be derived from that platform and how it lends itself to far more interesting conversations than people sharing cat pictures and other random things on Facebook.

Updates by Percentage

The next trend I want to visualise is understanding the types of updates made, by percentage, during any given year between 2009 and 2018.

#we want to find the count of posts, status updates and link shares by year

nine = fb_posts2[fb_posts2['year']==2009].groupby('title').size()

ten = fb_posts2[fb_posts2['year']==2010].groupby('title').size()

eleven = fb_posts2[fb_posts2['year']==2011].groupby('title').size()

twelve = fb_posts2[fb_posts2['year']==2012].groupby('title').size()

thirteen = fb_posts2[fb_posts2['year']==2013].groupby('title').size()

fourteen = fb_posts2[fb_posts2['year']==2014].groupby('title').size()

fifteen = fb_posts2[fb_posts2['year']==2015].groupby('title').size()

sixteen = fb_posts2[fb_posts2['year']==2016].groupby('title').size()

seventeen = fb_posts2[fb_posts2['year']==2017].groupby('title').size()

eighteen = fb_posts2[fb_posts2['year']==2018].groupby('title').size()

a = [x/sum(x)*100 for x in [nine,ten,eleven,twelve,thirteen,fourteen,fifteen,sixteen,seventeen,eighteen]]

df = pd.DataFrame(a, index= my_names)

#remove all NaN values and replace with 0

df['post'].fillna(0, inplace=True)

df['shared'].fillna(0, inplace=True)

df['timeline'].fillna(0, inplace=True)

In this block of code I get the percentage of each type of update based on the total update made in that particular year. I then proceed to drop any NaN values resulting from certain types of updates not occurring in some years and I replace that NaN value with a 0.