Forecast Anomalies in Refrigeration with PySpark & Sensor-data
Tags: Pyspark, Data Science, IoT, Forecasting, Machine Learning

Model Selection

We had selected a sample of refrigeration systems with good data quality to assess different algorithms. Forecast MAPE on an average was around 4–6%, 2.5–4% & 3.5–5% for ARIMAX, BSTS and Prophet respectively. Usually we use accuracy measures for model selection but here we needed to consider other important factors as well.

In this case, the accuracy was as important as the execution time because we were handling daily forecasts of around half a million systems across ~3600 stores. As BSTS is dependent on MCMC sampling, so the run time was much higher than the rest with little improvement in accuracy. Hence, we dropped BSTS!

Another important aspect was to be able to provide a cause of the anomaly and not just the forecast. We do get a list of significant variables based on p-values in case of any model but it is dependent on whole training data and doesn’t focus on recent changes.

Prophet has a property called change-point detection, i.e., it allows the trend to change its direction through the course of time. Prophet detects change-points by first specifying a large number of potential change-points at which the rate is allowed to change. It then puts a sparse prior on the magnitudes of the rate changes (equivalent to L1 regularization) — this essentially means that Prophet has a large number of possible places where the rate can change, but will use as few of them as possible. More details here.

Light Blue:Actual, Dark Blue:Predicted, Black:Trend, Red:Favorable Temperature Band

In the above figure, the vertical black lines are the change-points in temperature. It’s useful as it helps in notifying about any sudden change in temperature pattern. By default, Prophet specifies 25 potential change-points which are uniformly placed in the first 80% of the time series. These hyper-parameters can be changed through n_changepoints and changepoint_range.

Pseudo code for model change-point in Python:

from fbprophet import Prophet



# df : dataframe consisting 'ds'(timestamp), 'y' and other exogenous variables

# future_df : dataframe consisting lagged feature variables for forecast

# features : list of features to be included in the model after multi-collinearity checks # create an expression instead of including each regressor separately all_fea = [str(".add_regressor('") + str(features[i]) + str("')") for i in range(0, len(features))]

all_fea = ''.join(all_fea) # No need to put seasonalities separately.

# Prophet detects the frequency of the data itself

# If the frequency is less than daily, then it includes daily & weekly seasonality automatically exp = str("Prophet(changepoint_prior_scale = 0.05, changepoint_range=0.8)") + all_fea + str(".fit(df)") # Fit & Predict model_prophet = eval(exp)

fcst_all = model_prophet.predict(future_df) # It will contain yhat, trend, seasonalities

We have used the same characteristic to detect potential feature which may be the cause of this sudden change. So, each of the features separately will go through the Prophet model and search for recent change-points.

Plot of one of the feature variables — Blue:Actual, Maroon:Trend

Whenever there’s any change in temperature pattern, we look for changes in the feature variables within a neighborhood of 2–3days and provide them as potential causes. Because of this in-built godly property, we chose Prophet over ARIMAX.