A short tutorial on Fuzzy Time Series — Part II | by Petrônio Silva | Towards Data Science

In the first part of this tutorial I briefly explained time series analysis, fuzzy sets and what are the Fuzzy Time Series — FTS, with a short introduction to the pyFTS library. It was awesome! I got feedback from people around the world and I heard very good ideas coming from them. But, as occurs in every introductory study, we used very simple methods and data with the aim of to facilitate the general understanding of this approach.
But now, with the assumption that the readers already know the background needed, we can advance a little bit and play with more useful things. To start we going to see an intuitive introduction to the concept of bias/variance, then we will go learn some of the most import FTS hyperparameters and their impact on model’s accuracy. To finish we will employ some FTS models to model and predict solar radiation time series, useful on photovoltaic energy prediction.
All examples of this tutorial are available on Google Colab, on http://bit.ly/short_tutorial_colab2 . Feel free to get in touch, give ou feedback and ask questions. Now, let’s get started!
The training of machine learning models is a clash to separate the signal from the noise, the general behavior from local specificities and random errors. On the training of every estimator there is a clash between the bias and the variance, between the underfit and the overfit.
Given a numeric time series Y, its individual values y(t)∈Y, and an estimator for Y, a function ŷ(t+1) = f( y(t) ). The aim of f is to predict Y the best possible way, making the difference ε(t) between the real value y(t) and the predicted value ŷ(t) tends to zero, or in other words for ε(t) = y(t) — ŷ(t) and t → ∞, lim ε(t) = 0.
Intuitively, a biased estimator is one whose expected value of ε (t) is different from zero — E [ε (t)] ≠ 0. The bias is a systematic “deviation” of the correct value and remember that the estimator will often have deviations from the real value and this is expected, but on average these deviations should tend to zero. The bias is typical of the underfitted model, when the model was not able to learn the signal, the component of the time series that interests us.
In the other hand, the variance has to do with the power of generalization of the model, especially with data that were not presented during the training phase. The high variance is related to the overfitting, when the model start to learn specificities of the training sample that do not generalize well for the test data. In short: the model learned the noise of the data.
It is known that is not possible to completely eliminate the bias and variance side effects, and the best fit is achieved by the balance between them — this is the challenge of estimation models.
Several parameters determine the best fit of an FTS model, but the principal ones are the partitioning and the order. These two parameters account for 90% (empirical value) of the accuracy of the model.
Partitioning is composed of three parameters, listed here according to their importance:
1a) Number of partitions (or fuzzy sets)
This is simply the most influential parameter in the model accuracy. The more fuzzy sets the more precise is the capture of the characteristics of the time series. And there is a trap that lies right here:
Too few fuzzy sets generate underfitting, due to signal over simplification;
Too much fuzzy sets generates overfitting, making the model to start learning noise on data;
The number of sets is a parameter that must be benchmarked. In time series that have been differentiated 10 partitions is a good number to start. In other cases, 35 partitions is a good number.
1b) Partitioning type
There are many types of partitioning, from Grid partitioning (GridPartitioner) where all sets are evenly distributed and have the same format, going through partitioners where sets have distinct sizes — such as entropy-based and cluster-based partitioners. I will not go deep into this discussion here, but for the curious there are several examples of partitioning types in the PYFTS/notebooks repository.
Always start with Grid partitioning, then if it is the case, explore the other types.
1c) Membership functions
This is a parameter that has little real influence on the accuracy of the model, but depending on the case you may have a good reason for using a Gaussian or trapezoidal function instead of the triangular function, which is the default.
One of the justifications may be the number of parameters (the Gaussian uses 2, the triangular 3 and the trapezoidal 4), the legibility of the model or even other issues related to the nature of the process and the data.
Once again I will not delve into this discussion here, take a look at the PYFTS/notebooks repository for more details.
The order of the model is the second most important parameter of the FTS, since they are autoregressive models (use lagged values to predict the next ones). The order parameter is the memory size of the model, or how much past information is needed to describe future events.
To decide this parameter it is important to be familiar with the concept of Autocorrelation Function — ACF. The ACF is not only able to indicate the order as well as the indexes of the most important lags.
The order is the amount of lags (past values) that are used by the models. This is really very important and depends on the type of time series being modeled. The question here is: how many lags do I need for the model to learn the temporal patterns, cycles, seasonalities, etc? Look at the ACF and see how many are the significant lags.
However there is a trap here: as more lags the model uses (especially if the number of partitions is large!) and as larger the model is, slowest the learning and inference becomes.
In my experience, it takes no more than 3 lags to describe a time series behavior. But of course everything depends on the data.
By default, the most recent lags, in order, are chosen by the model. But depending on the seasonality of the time series these may not be the best lags. So look at the ACF and see which lags indexes are the most significant.
The literature on FTS methods is very diversified, but two features are extremely important:
The weights increase the accuracy of the model, balancing which sets in the rules of the model are more influential for the forecast. They slightly lessen the readability of the model, but nothing awfully. If you have to choose, always prefer the weighted models !
In the example below we can compare the model HOFTS (without weights), WHOFTS (with weights in consequent of the rules) and PWFTS (with weights in the consequent and the precedent of the rules):
Most of the time we only have time series with one variable — the endogenous variable. Other times this variable is aided by other information (the exogenous variables) from which we can take advantage.
For instance the date, usually associated with time series measurements, is a very valuable information in the case of seasonal data — such as social, environmental, etc. data.
If you have multivariate data then the first thing is to know if there is correlation between variables, so use a correlation matrix to check it. The correlation coefficient point to simple linear relationship, so it should not be the unique tool you should use, the cross-entropy is a good alternative.
Another tip: If you have a monovariate time series you can enrich your model by creating a multivariate series from it, where the other variables are transformations of the endogenous variable. For example, you can have a multivariate series with the original (endogenous) variable and the differentiated endogenous variable, providing extra information about the recent fluctuations of the values.
It is time to have some fun! Just to remember: all codes and results are available at http://bit.ly/short_tutorial_colab2. I will use the solar radiation data from SONDA — National Environmental Data Organization System from Brazil, specifically from Brasília/DF station. Radiation data is in the ‘glo_avg’ variable, sampled per minute, 24 hours per day, from 2013 to 2014 for the training set and the year 2015 for testing.
As expected, the data is a bit noisy and there is no big need for the per-minute sampling. I cleaned the data and removed the other variables (whose correlation with our main variable is not so relevant) and left only the date and the endogenous variable. The next step was to reduce the data volume and the clean up the noise, I did this by sampling the hourly average of the series. The preprocessed data is available at the address https://data.world/petroniocandido/sonda-bsb-averaged-hourly.
Intuitively the solar radiation has two main cycles, the daily cycle (the sun rotation movement) and the yearly cycle (the sun’s translation movement). The biggest uncertainty that affects this kind of forecasting is the weather and the presence of clouds in the sky.
As you already know the methods in the pyFTS library are on the models package. We will then analyze how several FTS models perform in the modeling of this time series. In the real world we should start by optimizing the hyperparameters of the models by testing numerous combinations of all the parameters mentioned above. In pyFTS, we could use the hyperparam package where you can run a distributed Grid Search, which will make a cartesian product between all possible values of each hyperparameter and test each one. You could also use a genetic algorithm (hyperparam.Evolutive) or use the hyperopt library. But let’s keep it simple, okay?
We will partition our endogenous variable using a grid scheme (GridPartitioner) with 35 partitions, separated into 5 subgroups, VL— Very Low, L— Low, M — Medium, H— High and VH— Very High, each subgroup with 7 levels. This nomenclature will make the model more interpretable by generating rules such as:
L5 , L6 → L6 , M0 , M1
This rule can be read as:
IF y(t-1) is Low (sublevel 5) AND y(t) is Low (sublevel 6) THEN y(t+1) will be Low(sublevel 6) OR Medium (sublevel 0) OR Medium (sublevel 1).
According to the chosen rules (due to the membersguo of the precedent of the rule in relation to the input values) the defuzzyfication will transform the consequent into a numerical value (a simple method is the weighted average of the sets).
We already have the partitioning scheme, so let’s explore the methods. We chose the high order methods (order> 1) with and without weights and all of them were tested with the order of 1 to 3. The methods and examples of the rules generated are listed below:
hofts.HighOrderFTS: High order weightless method
L4,VL0 → VL0,VL1L5,VL0 → VL0,VL1
hofts.WeightedHighOrderFTS: High order weighted method, where the weights are only on the consequent of each rule.
L4,VL0 → VL0 (0.5), VL1 (0.5)L5,VL0 → VL0 (0.5), VL1 (0.5)
pwfts.ProbabilisticWeightedFTS: High order weighted method, where the probabilistic weights are on the precedent and consequent of each rule.
(0.003) L0,VL1 → (0.876)VL0, (0.103)VL1, (0.015)VL2, (0.006)VL3, (0.001)VL4(0.003) L0,VL2 → (0.003)L0, (0.003)L1, (0.003)L2, (0.0)L3, (0.0)L4, (0.787)VL0, (0.164)VL1, (0.03)VL2, (0.002)VL3, (0.002)VL4, (0.005)VL5, (0.001)VL6
On pyFTS, multivariate models use Pandas Dataframes as data input (univariate models use lists or Numpy arrays) and the methods are in the models.multivariate package. Each variable has its own partitioning scheme and other parameters. You must first create an object of type Variable, stating the name of the variable, the data column in the dataframe, and the partitioner. Then the exogenous and endogenous variables are incorporated to the model using the append_variable function. The endogenous variable must be setted in the target_variable property.
from pyFTS.models.multivariate import common, variable, mvftsfrom pyFTS.models.seasonal import partitioner as seasonalfrom pyFTS.models.seasonal.common import DateTimesp = {'seasonality': DateTime.day_of_year , 'names': ['Jan','Feb','Mar','Apr','May','Jun','Jul', 'Aug','Sep','Oct','Nov','Dec']}month = variable.Variable("Month", data_label="data", partitioner=seasonal.TimeGridPartitioner, npart=12,data=train_mv, partitioner_specific=sp)radiation = variable.Variable("Radiation", data_label="glo_avg", alias='rad',partitioner=Grid.GridPartitioner, npart=35, data=train_mv)model = mvfts.MultivariateFTS()model.append_variable(month)model.append_variable(radiation)model.target_variable = radiationmodel.fit(train_mv)
Besides our endogenous variable ‘glo_avg’, we will use only the ‘date’ column and extract two exogenous variables from it: the month and the hour. The partitioning of the variables is showed below:
You may be asking yourself: Why are the sets of hours and months overlapping? After all, this is very accurate information! Answer: That is fuzzy logic guys! We assume that neighbors things influence each other and have similar behaviors, hence the rules of a specific time are also influenced by neighboring hours and months.
Let’s now take a look at the multivariate methods and see examples of the rules generated by them:
mvfts.MultivariateFTS: Weightless first order method (order = 1);
Jan,8hs,VL0 → VL1,VL2,VL3,VL4,VL5Jan,8hs,VL1 → VL1,VL2,VL3,VL4,VL5,L0,L1,L3,L4
wmvfts.WeightedMultivariatedFTS: Weighted first order method;
Jan,8hs,VL0  → VL2 (0.353), VL1 (0.253), VL4 (0.147), VL3 (0.207), VL5 (0.04)Jan,8hs,VL1  → VL2 (0.276), VL3 (0.172), VL1 (0.198), VL5 (0.083), VL4 (0.151), VL6 (0.021), L0 (0.036), L4 (0.005), L1 (0.036), L2 (0.021)
granular.GranularWMVFTS: Weighted high order method;
Jan11hsV3,Jan12hsL1  → Jan13hsVL6 (1.0)Jan12hsL1,Jan13hsVL6  → Jan15hsVL3 (1.0)
Obviously the main criterion for evaluating a predictive model is its accuracy. We will use the Root Mean Squared Error (RMSE) and the lower this value the better. But another important criterion is parsimony: simpler models will always be more desirable than complex models. So we aim to minimize RMSE and model size. Let’s see the five best-performing models:
It was expected that the multivariate models performed better then the univariate ones, after all, they had more information to exploit. But see that the most parsimonious model is the PWFTS!
The possibilities do not stop there. We could now decide to make an ensemble of the best models (models.ensemble), or use non-stationary fuzzy sets (models.nonstationary). But this is for another opportunity!
When we adopt an forecasting model we are also interested in predict not only the next value, but a sequence of these values. This is really cool because with just a few lags of information we can generate an estimate for many steps ahead! For the specific case of solar radiation forecasting we will set our prediction horizon for 48 hours.
For an monovariate model on pyFTS this is very simple, just use the steps_ahead parameter of the predict function. This parameter will feed back the output values in the input, by the next steps_ahead iterations.
forecasts = model.predict(input_data, steps_ahead=48)
For multivariate models this is a little bit tricky: we are not generating only the values of the endogenous variable but also the values of the exogenous variables. As this value generation depends on the nature of each variable, the predict function will require, in addition to the steps_ahead parameter, the generators parameter, which is a dictionary that must contain a key for each Pandas DataFrame column and whose values are lambda functions that receive the current value of the variable and return the next value. In our example this is quite simple since the values of the exogenous variables are datetime and to calculate the next value we just need to add one hour to the input value:
generator = lambda x : pd.to_datetime(x) + pd.to_timedelta(1, unit='h')forecasts = model.predict(input_data, steps_ahead=48,               generators={'date': generator})
Now you have enough knowledge on FTS and the pyFTS library to employ them on your personal projects. Off course that still remains to talk about a lot of stuff! In the next tutorial we talk about interval and probabilistic forecasting, data transformations, non-stationary and incremental models, etc.
Please don’t hesitate in get in touch if you need help, ask your questions or give feedback. See you later, guys!