How has terrorism changed over the years?
By: Humza Ikram
Tags: Terrorism

What have been the motives for terrorism over the years?

We can also look at the summaries for the incidents over the years and try to find repeated words in order to glimpse what were possible motivations for the attacks. We can then compare them.

We can generate a wordcloud using the wordcloud package for python

We have stopped a lot of words from showing up that would just fill up the data.

WordCloud for 1972

WordCloud for 2017

As we can see (to an extent), 1972 was relatively tamer. The major means of terrorism were letter bombs and most of them seem to have been discovered. It also seems like there was some racial violence involved. Meanwhile, 2017 is a lot more chaotic. Casualties are a lot more common, the Islamic state seems to have a role and explosives seem to be a very common as well (this is supported by the two above graphs that show the trend of attack types and targeted nationalities). It does seem to show a trend towards more desperate attacks and the Islamic state seems to be, in part, involved in a lot of the attacks.

We can also track a word over the years and see how often it appears in the summaries for the incident. For example, if we track the word “cash” over the years, it seems to go downwards in trend as shown.

If we do the same for something like “religious”, we get this.

Or if we track a major terrorist organization, we get this.

This does seem to suggest that religious terrorism is on the rise compared to terrorism for the sake of financial gain.

Does education have a role in terrorism?

Exploring the education dataset.

The global education dataset is provided by World Bank and conveniently provides a Dataset that has education statistics on various countries from 1970 onwards as well. We can try to find some basic correlation (which is definitely not the same thing as causation). Lets see the trend for education in regions over the years.

Well, that’s not very interesting.

Maybe a bar graph would be more interesting?

Well, there is a minor positive trend in each region? That is good, right? Let us make a correlation matrix for literacy in various regions and the number of incidents in these regions.

Well, it doesn’t really show anything.

It seems like education doesn’t seem to have any correlation with terrorism that we can prove with our exploratory analysis.

Can we try to predict which group is responsible for an attack?

We already have the name of the group itself in the ‘gname’ column of the dataset. If we know where the attack occured, what type of attack it was, how many people died, what weapons were used and who it was conducted against, we should be able to predict this group by modelling it using the rest of the data. This way, we should be able to predict the responsible group from the characteristics of an attack alone in the future.

First, lets try to clean the data and only get the appropriate columns and clean out any rows we don’t have complete information about.

There is a problem with this. Group name has over 3500 different groups which have conducted different attacks over the years. That is a lot but most of the recurring groups have conducted thousands of attacks while some of the relatively minor groups have only conducted attacks less than a dozen times.

We can classify all these minor groups as a group ‘Other’ so that actual machine learning is possible and we can predict the group as 3500 different categories is simply too much for any model to classify data into. We did the same with the nationality of the target, picking only the top 50 commonly attacked nationalities and classifying the rest as ‘Other’.

Most of the popular terrorist groups weren’t around in the 70s.

With the top 50 most responsible groups shown below.

The 50 most responsible terrorist groups that have conducted the most attacks.

Speaking of models, we will be using a multiclass logistical regression model. The output variable for this will be the name of the group and we will try to predict using the other columns as features (shown in the dataset above).

Of course, most regression models can only accept numerical inputs. As such, we need to convert the data above into numerical data using ‘One hot encoding’. We then just plug it into a logistical regression model and train it.

DictVectorizer() is responsible for One Hot Encoding of the data.

We have roughly 160,000 values so we take the first 150,000 to train the model on and the remaining we use for testing the accuracy of the data. This means that we will be using data up to 2016 and use that to predict the group responsible for attacks in 2017.

The above graph is the loss function, it quickly descends to a minimum constant value. The accuracy is at the bottom as a fraction.

As can be seen, the accuracy is 0.43 or 43%. This means that the model predicted correctly 43% of the time. This might not seem a lot but if you consider that there are more than 50 different categories to choose from, it does narrow it down a fair bit. We can compare this to what happens when you just randomly choose a group.

We choose the right group only 2% of the time. So 43% is a definite step up.

Conclusion

The EDA does show that the amount of global terrorism is increasing as the years go on but there is a hopeful decrease in the very short term history of attacks. The attacks have gotten more centered in the Asia and Africa over time. It also seems like global terrorism has shifted from letter bombs and has taken more religious tones recently.

It is highly likely that education does have a role in global terrorism but from our limited dataset alone, that role seems to be minimal.

The predictor model seems to be fairly simple (its a logistical regression model with minimal feature engineering) but it is still performing decently well. We could perhaps improve the accuracy of our predictions by using a deep learning based neural net which does not have much of a problem when there are plenty of input categories. In any case, the model is performing well enough compared to randomly selecting.