India can deploy Machine Learning to douse Ceasefire Violations
By: Jatin Kataria
Tags: Machine Learning, India, Time Series Forecasting, Defense

Here is how looking into the data will tell you that CFVs are not sporadic!

From the gallery of indopakconflictmonitor.org

When a ceasefire violation (CFV) happens along the Line of Control (LoC) and international border, such occurrences are considered merely sporadic and against the grain of a written agreement between the two countries. In our study, we aim to rectify the knowledge-gap with predictions on the number of CFVs along the India-Pakistan LoC, using python to identify the best model (SARIMA vs Prophet) and data visualisation tools like Tableau to recognise susceptible sectors. Our study could be useful for Indian military experts to be better prepared beforehand against impending CFVs. Our model can be used to devise a defence strategy.

Figure 1. This map depicts the disputed region of Jammu & Kashmir including the LoC and international border between India and Pakistan.

If you’re interested in diving into the code, visit the following link

Source of Data

The study relied on the India-Pakistan conflict monitor (indopakconflictmonitor.org) for the date-wise (specific dates) data. We ran SQL query to group the same data month-wise.

Diving into the Data

The website, indopakconflictmonitor.org, collects data from only media reporting. However, the Ministry of Defence, which is the media source, does not provide date-wise and sector-wise data as it supplies only cumulative yearly figures. Even though the study is based on limited numbers, our model will be valid for any given data. We have considered CFVs from 2013 because of the sharp increase in their total count since then.

Figure 2. The total number of ceasefire violations occurred between January 2013 and December 2019.

An unusual spike of ceasefire violations was seen in October 2016 (Figure 2). This could be in response to the surgical strike that India had carried out in September 2016. Another interesting observation is that October saw the maximum number of CFVs in the years 2013, 2014, 2016 and 2019 and a dip can be seen during the summer months (April, May, June and July). These observations led us to believe that there is a pattern to violations and they are not completely random.

Building a prediction model

Analysing a time-series is a statistical method of understanding a time-indexed data by detecting any underlying pattern and forecasting values in the time horizon based on past observations, taking into account the discovered trend. The prerequisite to forecast any time-series data is that it should be a stationary series. In a stationary time-series, parameters like mean and variance remain constant. They do not vary with time. Therefore, if the time-series is not-stationary, then it becomes an absolute necessity to convert it into a stationary one so that the model can be fitted to it. Two time-series models — Seasonal Autoregressive Integrated Moving Average (SARIMA) and Facebook’s Prophet were used to fit the CFVs data. The validation of the models was performed on the given dataset (January 2013-December 2019) to identify which of the two models fit the best. Note that the actual dataset was split in the ratio 80:20, as training and testing, respectively.

SARIMA model

SARIMA is a model used to study any time-series that has a component of seasonality in it. It overcomes the limitations of ARIMA by explicitly taking care of the additional seasonal component apart from the trend component present in the time-series data. The common notation for SARIMA models is SARIMA(p,d,q)(P,D,Q)[S], where p,d,q,P,D,Q,S are all hyperparameters of the model which can be tuned to get the optimal solution. ‘S’ corresponds to the seasonality period, ‘d and D’ corresponds to the order of differencing for trend and seasonality component respectively (used to make the time-series stationary). ‘p (autoregressive) and q (moving average)’ corresponds to the trend component’s lagged series and forecast error terms respectively. ‘P and Q’ are similar to ‘p and q’, except that they are for the seasonality component.

SARIMA modelling was done in Python programming language. Box-Jenkins methodology was adopted to construct the model which has a given procedure:

(1) Checking stationarity of time-series;

(2) Establishing a tentative model;

(3) Estimating parameters of the tentative model;

(4) Evaluating the validity of the model, otherwise, follow Step 2;

(5) Forecasting using the constructed model.

Testing for Stationarity

Stationarity check using the Dickey-Fuller Test: In this statistical test, the null hypothesis is that the series is non-stationary. For the null hypothesis to be true, signed value of test statistic has to be greater than that of critical values computed for different confidence levels.

Figure 3. Stationarity test on raw time-series.

Results of Dickey-Fuller Test:

Test Statistic -4.925127

p-value 0.000031

#Lags Used 0.000000

Number of Observations Used 83.000000

Critical Value (1%) -3.511712

Critical Value (5%) -2.897048

Critical Value (10%) -2.585713

The result summary shows that the ‘Test Statistic’ < ‘Critical Value’ (1%). We can, therefore, reject the null hypothesis and the time series can be considered stationary. Hence, no need for differencing.

Figure 4. Decomposition plot of raw time-series.

From decomposed series we can see that there is obvious seasonality of 12 months and as for trend there is no consistent positive trend and therefore differencing wasn’t required (d=0). We can see that the trend and seasonality are separated out from data and now we can check the stationarity of residuals:

Figure 5. Stationarity test of residuals.

Results of Dickey-Fuller Test:

Test Statistic -4.642600

p-value 0.000108

#Lags Used 11.000000

Number of Observations Used 60.000000

Critical Value (1%) -3.544369

Critical Value (5%) -2.911073

Critical Value (10%) -2.593190

Test statistic < 1% critical value. So, this residual series can be considered stationary.

Figure 6. SARIMA model diagnostics.

The plot of model diagnostics is indicative of the fact that the residuals are near normally distributed, which should indeed be the case after removing trend and seasonality.

Model Evaluation

For our study, a range of values were selected as the SARIMA parameters using the ACF/PACF and decomposition plots.

Figure 7. ACF/PACF plots to find SARIMA parameters.

From the decomposition plot, we can see that the trend is not exactly constant, so we could expect non-zero trend order (p,q) terms. This is also visible from the ACF and PACF plots, p=q=(0,1).

A grid search method was adopted to find the optimum parameter values and Akaike Information Criterion (AIC) was used for scrutinising the models. AIC is a way to pick the best model that balances the drawbacks of overfitting and underfitting.

After getting the optimised model, it was fitted to the training data set. Subsequently, the testing data was used for validation which was conducted using the Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) metrics.

The best parameters for the model were found to be: ([(1, 0, 1), (1, 1, 1)[12]], AIC-432.688). SARIMA (1,0,1) (1,1,1) [12], with its lowest AIC value, was chosen as the best-fit model. Using the above parameters, the model was validated on the testing data set(Figure 8).

Figure 8. This plot shows the comparison between actual ceasefire violations and the rolling prediction on the testing data to validate the SARIMA model.

Prophet model

Prophet modelling was done in Python programming language.

The Prophet uses the following equation to represent any given time-series data:

y(t)=g(t)+s(t)+h(t)+εt

In any given time-series data, this model splits it into four components:

(1) g(t) is a growth function that identifies aperiodic variations;

(2) s(t) represents seasonality (for instance, weekly, yearly, seasonal, etc.);

(3) h(t) specifies fluctuations triggered by holidays etc.;

(4) εt signifies random noise.

Model Evaluation

Once the Prophet model was set up, it was used to fit the training data. Subsequently, the testing data was used for validation (Figure 9) which was conducted using Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) metrics.

Figure 9. This plot shows the comparison between actual ceasefire violations and prediction on the testing data to validate the Prophet model (the black dots represent the actual data while the blue solid line denotes the predicted data).

Figure 10. This depicts the overall trend and seasonality (yearly) of raw time-series.

Model comparison between SARIMA and Prophet

Once the two models were used to train and test the dataset, the derived prediction values were compared with the actual values (Table 1). To select the best-fit model, error metric for training and testing data should be low, and as close as possible to each other (indicative of a generalised model that can predict well on out of sample data).

Table 1. The performance of Prophet and SARIMA models.

Hence, Table 1 indicates that the Prophet model performs better than SARIMA on our dataset. The SARIMA model’s testing errors were significantly higher than the training errors indicating over-fitting of the model.

Forecasting total CFVs using Prophet

Once the model was validated, it was used to forecast the probable occurrence of CFVs for the next 12 months (January 2020-December 2020) (Figure 11). The shaded (blue) region indicates 95% confidence intervals.

Figure 11. Ceasefire violations forecast for the next 12 months (January 2020-December 2020) using the trained Prophet model. The vertical red lines represent the forecast horizon.

Table 2. The number of total ceasefire violations likely to occur in the next 12 months (January 2020-December 2020).

From Figure 11 and Table 2, it can be observed that the likelihood of ceasefire violations is highest (a significant spike) during October.

Forecasting sector-wise CFVs using Prophet

In order to make a defence strategy we need to dig deeper and look at actual sector-wise data using Tableau (Figure 12). 11 sectors have witnessed CFVs sometime in the past, namely, Poonch, Rajouri, Jammu, Kupwara, Uri, Kathua, Akhnoor, Punjab, Gurez, Chirikot and Drass-Kargil.

Figure 12. The number of CFVs reported every month in all the sectors (January 2013 to December 2019).

From Figure 12, the top six hotspots, namely, Poonch, Rajouri, Jammu, Kupwara, Uri and Kathua sectors, are visible based on the reported ceasefire violations between January 2013 and December 2019. Poonch Sector is the most prone among all the sectors as it reports the maximum number of CFVs almost every month. Poonch is followed by Rajouri, taking the second position as a hotspot. This is a consistent feature observed over the last seven years from January 2013 and December 2019. Poonch Sector sees the highest CFVs in September and October, while Rajouri in February and March, Jammu IB in October, Kupwara in August and July, and Kathua in October.

Figure 13. The hotspots (identified in Figure 12) of ceasefire violation. Source-https://mapme.com.

Figures 12 and 13 indicate only the past sector-wise data visualisation. To strategise, the sector-wise CFVs were forecasted for the next 12 months (January 2020-December 2020). The sector-wise CFVs were modelled as multiple univariate time-series run parallelly using Prophet model.

Here is an example of Rajouri sector’s plot. To check out for other sectors, scroll down to the bottom of this article.

The forecasted values for each sector were then visualised using Tableau.

Figure 14. The number of CFVs forecasted every month in all the sectors (January 2020 to December 2020).

From Figure 14, the top six hotspots, namely, Poonch, Rajouri, Jammu, Kupwara, Uri and Kathua sectors, were identified. Note that this was a consistent feature even in the reported CFVs, where the same six sectors were the hotspots (Figure 12).

Based on the forecasted data, Poonch and Rajouri seem to be the most vulnerable of all sectors. Poonch sees the highest CFVs in October, while Rajouri in November, Jammu IB in October, Kupwara in October, Uri in August and Kathua in October. The interesting feature of this forecast seems to be the unusual spike in October in almost all top six sectors. Also, Punjab, Gurez, Chirikot and Drass Kargil sectors observe almost no CFVs throughout, for all the predicted months. Therefore, reported CFVs in Figure 12, seem isolated incidents in these sectors.

Map to Devise your own Defence Strategy

As a result of the model that we have built as well as the data analysis, we have created a map that can be used to construct an informed defence strategy for any given point of time.

Though the current study is validating only a small sample-size of ceasefire violations, the model adopted is conducive to any sample size. With several CFVs taking place regularly at the LoC, the time-series data apart from providing information based on seasonality also paints a clear picture of the predictable changes that recur year after year. Our observation points to a certain pattern to the CFVs and that they are not sporadic as is perceived and therefore we can build a defence strategy as follows:

For example, let us consider October 2020 (42 forecasted CFVs) and November 2020 (21 forecasted CFVs). If Poonch, Jammu and Rajouri sectors have shown a spike in October, November showed a surge in Rajouri and Poonch (Figure 14).

To surmount infiltration on the days of heightened CFVs, our military experts could strengthen their strategies by increased vigilance on the borders, surveillance in sectors which are considered porous and inaccessible. On the other hand, sectors that observe isolated scenarios such as Punjab, Gurez, Chirikot and Drass Kargil, man-power resources can be optimised, using drones as the main weapon for surveillance. During the CFVs’ spike days, manual patrolling would help in neutralising infiltration attempts.

Sector-wise Forecast Plots

Before You Go

Thanks for reading! If you have any difficulty or any doubts kindly comment below. Your support is always highly appreciated. If you want to get in touch with me, reach me on jatin.kataria94@gmail.com.