Investigate Anomalies in Temporal Data With Machine Learning
By: Conner Brew
Tags: Machine Learning, Data Visualization, Data Science, Time, Dashboard

Investigate Anomalies in Temporal Data With Machine Learning

Use Python and Machine Learning to analyze time-series data and visualize it interactively.

Here’s what you’ll be making:

Overview

In this notebook, we will be exploring Facebook’s open-source SARIMA model Prophet to identify outliers in time-series data. Here’s what you’ll learn:

What is time-series data?

What is SARIMA?

Build a simple optimizer to auto-optimize the Prophet model

How to use Prophet to forecast data

Use Prophet and Bokeh to identify outliers in an interactive dashboard

Use Bokeh to create a time-series dashboard layout for investigation

What is Temporal Data?

In my last article, we worked with spatial data — that is, data with coordinates that exist somewhere in space. Temporal data is often a companion to spatial data, and it is any data with a time or date associated with it. Temporal data is critical for overall understanding of your dataset, and being able to understand statistics associated with time — trend, seasonality, and outliers — will significantly increase your overall capabilities when it comes to working with business data.

In the field of national security and defense that is my specialty, there may be no more important data type than temporal data. Temporal data helps us see patterns through time, and make predictions about the future. That’s what you’ll learn to do, too, in this article.

Our Data

We will be using data from the Armed Conflict Location Event Database, specifically narrowed to Iraq. The data can be downloaded from github here, or from the ACLED website. We will be analyzing the trend and seasonality of conflict event data in Iraq from 2016–2020, forecasting that data, and identifying outliers. We’ll then create a companion line chart that will allow us to drill down on specific event categories, so we can investigate outliers once they’re identified. Let’s go!

Load Libraries

First, let’s import our necessary libraries. We’ll be using all of Bokeh to create our interactive graphics, fbprophet for the Prophet pre-built model, sklearn for model optimization, and pandas for some data manipulation.

Load the Data

You can find the specific data I used in this notebook on Github. However, I strongly urge you to try using your own time-series data in this exercise!

In this code block, we’ll be using the value_counts() method to get the data we need for our line chart. We want to visualize the amount of events on each day, so we apply value_counts() to our date group.

Throughout the article, you’ll see a couple practices that I like to apply whenever I deal with date-time data. Firstly, I re-sort my dataframe every time I do an operation against it — this helps reduce error later on when we train our model. I also always like to have one column in my dataframe that is in datetime format and another column in string format. I use the datetime column for analysis, and the string column for labels, legends, and tooltips.

Extract the Categorical Data

Next, we want to get the value_counts() for each event type. The final product will be a visualization that allows users to select a time window in the line chart, then display relevant event types in a second partner visualization. We’ll follow the same steps as above, but now for each event type in the event_type field. We will do this using the groupby() method.

Clean Up

Next, simply drop duplicates and re-sort our values.

Why Prophet?

Prophet is an open-source SARIMA model developed by facebook. SARIMA is a time-series regression (read: prediction) technique that takes into account various statistical properties of a time-series dataset including moving average, seasonality, and trend. Trend is a property of time-series data that has to do with the overall positive and negative growth in the dataset. Seasonality has to do with the cyclical changes in the dataset, such as regular decreases in activity on a weekend or increases on a daily basis every afternoon.

Prophet was originally developed to service Facebook’s forecasting needs, as the facebook office relies heavily on time-series forecasting for setting goals and measuring progress. You can learn more about Prophet and its origins by watching this video of a presentation given by Prophet’s developer.

Prophet requires two data sources to function: ‘ds’, which stands for datestamp, and ‘y’, the target variable. If you’re following along with this notebook, then we’ve already done this — if not, then keep this in mind for your own data.

Model Optimization

Model optimization is a key element of any machine learning project. All machine learning models have one or two key parameters that need to be optimized for best performance of the model. In the case of Prophet, that parameter is changepoint_prior_scale. This parameter defines how flexibly the model responds to changepoints in the data set, and needs to be optimized to prevent over and under-fitting.

To optimize this, we’ll use a very simple optimization model. It simply iterates through a user-defined epoch, tests each parameter, and selects the parameter that returned the lowest error. For this purpose, we’ll use the Mean Squared Error statistic.

For more information on model optimization using mean squared error, see this article about mean squared error and linear regression.

Train the Model

Now that the model has been optimized, we train it on our data. You’ll see that, with the data I provided, the MSE (mean squared error) is around 38. This may seem high, but is perfectly fine for time-series forecasting — usually an MSE below 50 is good for these purposes.

If you want to further reduce the error rate, a common technique is to apply exponential smoothing to the data before training the model. Think of it like scaling and normalization for time-series data — you can learn more about exponential smoothing of time-series data here.

Plot the Data

We will be using Bokeh to plot our data. Interactivity is a great way to make your data visualizations stand out and provide more use and insight to your customers. Although there are other libraries and methods for creating interactive visualizations, Bokeh is my preferred library. However, this is not a Bokeh tutorial. If you are unfamiliar with Bokeh, try their getting started guide.

Basically, we will plot the original data as a line glyph inside the Bokeh plot and plot the Uncertainty Bounds, represented in the Prophet dataframe as yhat_upper and yhat_lower as an area plot.

Analyzing Trend and Seasonality

As discussed above, Trend and Seasonality are two of the key statistical properties of time-series data. Understanding these factors is critical for understanding the data itself. Prophet makes this easy, with the plot_components method.

In the plots below, you can see that Iraq conflict events have had an overall negative trend over the past several years (as ISIS has been steadily defeated). However, you can see that there has been a slight upward trend throughout 2020 — this is due to a variety of factors including the coronavirus pandemic and US withdrawal from multiple bases in the area, which has led to an increase in ISIS activity.

In regards to seasonality, you can see some interesting patterns — activity is generally low on Fridays, which is the Muslim sabbath. You can see a hump in activity in March — June, which is well known as the common western desert fighting season due to the warmer temperatures that dry out the wadis and make the terrain easier to pass; this also typically coincides with the Muslim holy month of Ramadan. Interestingly, the data also shows high activity in October-November, very likely due to cooler weather following the hot summer months. In the case of recent years, this has also been an annual period rife with political conflict and protesting.

Plot the Outliers

Next, we will identify our outliers in the time-series data. These are events that rise above the trend and seasonality identified by the Prophet model. This is critical for national security and defense data. In Iraq, for example, we know that there are seasonal rises and falls for terrorist activity. When those seasonal rises come, how can we know whether it is worse than usual? Outlier detection helps identify possible resurgences of terrorist activity, allowing us to be better prepared.

We will use a simple methodology — we look for data points that rise above the upper uncertainty bound predicted by Prophet, then plot that point. In the resulting plot, we can see that nearly all of 2016 was rife with higher-than-expected terrorist activity and that the same is true of late 2019 and early 2020 — this is due to the political instability following the transition of power from Iraqi PM Abdul-Mahdi to the new PM Kadhimi, significant protest and riot activity beginning in October 2019, as well as above-mentioned factors like US drawdown and the coronavirus pandemic.

Plotting Categorical Data

I want to investigate the outlier activity more thoroughly. I want to know, in times of increased activity, what types of events make up that data — IEDs, conventional warfare battles, etc. To do this, we will make a second plot to visualize that data.

Bokeh automatically links selection between plots sharing a ColumnDataSource. Because the data for both plots is coming from the same dataframe, selections in one plot will automatically be reflected in the second plot. All we need to do is visually style the elements to draw attention to the selected elements, and make it clear what categories are prevalent in different time periods.

In the case of this data, we will use the selection_glyph and nonselection_glyph properties in Bokeh to make data invisible if it is not selected. This will allow us to easily see what types of data are prevalent for time windows that we select in the original plot.

This code snippet only shows the plotting for one layer. See the notebook for the full code.

The main plot uses a box_select tool to dynamically change the visualization. Place your mouse cursor inside the plot and drag to select a time window — watch the visualization display change!

Summary

Now you can analyze time-series data using SARIMA forecasting techniques with the Prophet machine learning model. You can automatically optimize your model, and use it to understand trend and seasonality properties inside your data. You can plot that information in interactive charts for later publishing in a dashboard or web application. Great work!