Can you infer who to follow on Twitter?
Tags: Graph Modeling, Social Media Analytics, Assignment 3, Data Science, Machine Learning

Model Validation Process

Because each prediction is dependent on the nodes, both rows and columns selected. We need to create a clean workflow (Figure 4) that will be used to sample, train, and validate the model.

First, we will select what nodes and columns are going to be selected for prediction. This process will be done at random from nodes that are at least followed by 3 or more users in our database.

Then for each subset of prediction labels, we will select the closest accounts in terms of cosine similarity and the most similar friends. We will then train the model with the subset of rows and columns and validate against the test label.

Figure 4. Depiction of the 4 step workflow

We will do this repeatedly to estimate how accurate is our prediction model of ‘follows’ and node ‘types’ (which is categorization of ‘News’ vs ‘Influencer’ vs ‘Friends’). For this exercise we cannot create a static train-test split since the number of users is very limited.

The Data Set

We reused our dataset from ‘Twitter Tribes’ but this time we added a few more users by leapfrogging to more friends and friend of friends. We then filtered the accounts these users follow but limiting to accounts that are followed by at least 3 or more users (Figure 5). We can also see the users more clearly and they relationships in Figure 6, and the accounts they follow in Figure 7.

Figure 5. Shows the users in blue and the accounts they follow in red

Figure 6. Users and their relationships with each other

Figure 7. Accounts that have 3 or more edges from users

Model Results

Our model shows a prediction accuracy of 0.66 after 1000 samples in terms of edges correctly predicted which is ok for such a small model. However the recall of 0.38 seems pretty low for actual links. Which means the model does not find links as much as we would like. A summary of the key statistics can be found in Table 1.

Table1 precision recall f1-score support



no-edge 0.73 0.79 0.76 683

edge 0.46 0.38 0.42 317



accuracy 0.66 1000

macro avg 0.60 0.59 0.59 1000

weighted avg 0.65 0.66 0.65 1000

If we jump into some of the examples of the predictions and we compare to my personal opinion on whether the prediction makes sense from a recommendation perspective. Remember that

User Account Predicted Actual Pred makes sense?

MiguelYuste8 DANIROVIRA No-edge Edge unsure

Sai00079520 gandyno No-edge Edge No

osuna_vasco policia No-edge No-edge No

gandyno guardiacivil Edge Edge Yes

osuna_vasco perezreverte Edge Edge Yes

alvarofm84 osuna_vasco No-edge No-edge Yes

osuna_vasco JotDownSpain No-edge No-edge Yes

gontz87 DANIROVIRA No-Edge Edge unsure

analyticsrl norcoreano No-Edge No-edge Yes

gandyno InesArrimadas No-Edge Edge Yes

In terms of the type of account. We are predicting the type label based on the accounts that are closest in similarity. The model will predict via proximity if these accounts are considered Influencers, News or Regular account. We can also compare the results with the arbitrary label we created to see how did we do.

Based on the results below we see that because the majority of nodes are Influencers in the majority of prediciton our model predicted Influencers. When we compare this to the arbitrary label we created (based on followers and friends) we see that we have a pretty good precision and recall for Influencer but not so much for all the other types. This basically means we can’t really infer account type only using the amount of followers and friends an account has.

Table2 precision recall f1-score support



Influencer 0.53 1.00 0.69 522

News 1.00 0.05 0.10 316

Regular 0.00 0.00 0.00 162



accuracy 0.54 1000

macro avg 0.51 0.35 0.26 1000

weighted avg 0.59 0.54 0.39 1000

Ethical Considerations

In this model we have shown a prediction model from the perspective of connections among connected individuals in a network. However, this approach could be used to infer any other kind of attribute from the set, as long as we have a big enough sample of attributes to ‘learn from’. This opens the door to all kinds of issues in terms of the attributes that some users might not want to disclose directly or inferred via predictions. Even if the attribute is not directly protected by the user, the individual running predictions should be mindful of attributes that could cause discrimination in terms of sex, race, age, disability, color, creed, national origin, religion or genetic information among others and in full compliance with US and international laws.

The attributes we used in our forecast cannot be considered a ‘protected’ attribute, since it could hardly cause discrimination given the application we presented today. However, some applications that could cause Unintentional discrimination could be that this model will more likely recommend accounts that already have widespread popularity in your network. Applying this recommendation repeatedly will increase the disparity between popular mainstream groups and marginalized accounts that could indirectly be suppressed by recommendation engines that follow similar approaches.

The attribute categorizations in terms of followers and followed accounts could be mislabeling the type of account on Twitter. In most cases, many people would disagree with the labeling we picked given that the thresholds we picked in the first place are somewhat arbitrary between regular ‘friend’ and a ‘’influencer’. Even the category ‘News’ might not be adequate to define all accounts, but because this is a simplified assumption for educative purposes, this might be not be as big of an issue.