A short tutorial on Fuzzy Time Series | by Petrônio Silva | Towards Data Science

Time series analysis and forecasting methods are indispensable on several fields, for instance on engineering, medicine, economy, meteorology, etc.
There are several methods of analysis and forecasting, from the traditional and consecrated statistical tools (ARMA, ARIMA, SARIMA, Holt-Winters, etc), to the new computational intelligence tools (recurrent neural networks, LSTM, GRU, etc). There is no perfect method, neither the one I going to present here. But some key features distinguish the Fuzzy Time Series e turn it on a attractive option:
Readability
Manageability
Simplicity
Scalability
Hereafter I going to assume that you don’t have a machine learning (with focus on fuzzy systems) and time series background and I will present the key concepts of these fields. Then the Fuzzy Time Series methods will be introduced with the help of the pyFTS library. Let’s go?
If you already know about Fuzzy Logic and Fuzzy Sets, you can go ahead to the next section. Here are presented just the very introductory concepts!
The logic theory and classical mathematics define a Set as a dichotomy: every element is in OR out of the set. The is no middle point! The membership of an element is a boolean value, that is, a value on set {0 , 1}, imposing to each set strong and inflexible boundaries.
This dichotomous way of thinking is uncomfortable for the human being because innumerable realities are not so. We will have difficulty when we try to classify people into categories with strict limits, for example: weight = {thin, slender, fat}, age = {child, adolescent, young, adult, elderly}, height = {low, medium, high}. If we are to describe someone using these concepts in at least one of them we will find that the person could be in a middle category between these two values. But classic / rigid sets do not give us this flexibility.
The Fuzzy Logic, proposed by Zadeh (1965), state a duality instead of this dichotomy: a certain element may belong and simultaneously do not belong to the same set at certain levels, such that the membership is a value in the interval [0, 1]. The fuzzy sets have no strict boundaries and they are usually overlapping, so — using the previous example — I can be medium high and medium, or 90% medium and 10% high for example.
Given X, a Numerical Variable, such that X ∈ R — for instance an height measure — its Universe of Discourse, abbreviated to U, is the is the range of values that this variable can assume, such that U = [ min(X), max(X) ].
A linguistic variable A is the transformation of the values of the numerical variable X into a set of words / linguistic terms (what we call fuzzification). Each word/linguistic ter is a fuzzy set ã ∈ Ã, and each fuzzy set ã is associated to a function μ (mu greek letter), such that μã: X →[0,1] (this mean that μã receives an input value from X and return an output value on interval [0,1]).
Let’s get back to the numerical variable height, measured in centimeters. We define U = [20,220] and the linguistic variable Ã as:
Ã = {“very small”, “small”, “short”, “medium”, “tall”, “very tall”}
Do you see what we’re trying to do? We want to stop working with the numerical data of X and start working with the vocabulary expressed by Ã, and for this we need to map the values of X for each set ã ∈ Ã.
We will do this by dividing U into 6 overlapping intervals, one for each set ã ∈ Ã. For each interval we associate an μã(X) function. There are several types of membership functions but, just to simplify, we will use the triangular membership function. The triangular membership function can be defined as:
def triangular(x, a, b, c):    return max( min( (x-a)/(b-a), (c-x)/(c-b) ), 0 )
Where a, b and c draws a triangle with the a on the left , b on the top and c as right point. When a given input value x is equal to b we say that it is 100% inside the fuzzy set( in other words: its membership grade is equal to 1). In other hand, the membership increase linearly from 0 to 1 on interval [a, b], and linearly decrease from 1 to 0 on interval [b,c]. If x is smaller than a, or greater than c, we say that x is completely outside of the fuzzy set (int other words: its membership grade is equal to 0).
So, what would our sets look like for the linguistic variable Ã?
Suppose a person is 163cm tall. For the variable Ã, it would be .27 median and .73 tall, that is:
μ_median(163) = triangular(163, 90, 130, 175) = 0.2666666...
μ_high(163) = triangular(163, 130, 175,220) = 0.7333...
Check out the code here!
But is is only the beginning of a huge knowledge field. If you want to know more about Fuzzy Logic and Fuzzy Systems....
But where are here to talk about time series, right? So let’s talk about time!
If you already know about time series, you can go to the next section, this one is very introductory!
Time series are sets of data representing the behavior of one (or more) random variable over time, and its main characteristic is that the successive records of this variable are not independent of each other and their analysis must take into account the order in that were collected.
According to Ehlers (2009), “the neighboring observations are dependent and we are interested in analyzing and modeling this dependence”. What does that mean? That to predict future values of the time series I use past / lagged values of the same series.
A simple example is the autoregressive AR(p) model, where p indicates the number of lagged variables used on forecasting. Given a time series X(t), where t indicates a moment in time, if we going to use only use a lag in the forecast then we have an AR(1) model, that would be something like X(t) = α·X(t-1) + ε, where X(t-1) is the lagged value, α is a coefficient adjusted by statistical methods and ε represents the noise or random error, which indicates the uncertainty associated with this forecast, ε ~N(0,1). If we want to use the last two lags on forecasting we will have an AR(2) model, which would be something like X(t) = α0·X(t-1) +α1·X(t-2) +ε.
What is the amount of lags and what lags should be used? To do this, it is necessary to study the components of the time series and analyze their properties, using graphs such as ACF and PACF. Usually time series are modeled (using an additive model) at high level as:
X(t) =C(t) + T(t) + S(t) + R(t)
Where:
t is the time index;
X(t) is a point estimate of the series at time t;
C(t) is the cyclical component, foreseeable short / very short term fluctuations;
T(t) is the trend component, which indicates the long-term behavior of the series. Generally the tendency is of growth / rise or reduction / descent, otherwise it is said that the series has no tendency;
S(t) is the seasonal component, which is periodic fluctuations of medium and long term. A good example of seasonal component are ... guess what? The seasons of the year! These are intervals of 4 months that have a very characteristic of their own and are repeated annually — which makes any variable with this seasonality much more predictable.
R(t) is the noise component, a random value with constant mean and variance. This random noise is not predictable!
Not all time series have trend or seasonality. Some series are known to be stationary, meaning that their average values are (more or less) constant. In non-stationary series the average changes over time. In the homocedastic series, the variance is constant and in the heteroskedastic series the variance changes over time. To summarize: stationary and homoskedastic time series are “well behaved” and simpler to predict, and non-stationary and heteroskedastic series are much more complex. In the latter case we can apply mathematical transformations (such as differentiation, Box-Cox, etc.) to make the series stationary and homoskedastic.
But.... Where do fuzzy sets come into this story?
The use of fuzzy sets for modeling and predicting time series arises almost intuitively, first based on the ability of fuzzy models to approximate functions, but also on the readability of rules using linguistic variables that make them more accessible to experts and non-experts analysis.
The pioneer work on fuzzy time series is Song and Chisson(1993) but here we present the evolution published by Chen(1996). The idea is to divide the Universe of Discourse from time series in intervals/partitions (the fuzzy sets), and learn how each area behaves (extracting rules through the time series patterns). The rules of these models tell how the partitions relate with themselves over the time, as values jump from one place to another. In other words: let’s create a linguistic variable to represent the numerical time series, and these areas will be the linguistic terms of our variable.
When we create a linguistic variable to represent the universe of discourse, we create a “vocabulary,” and then the fuzzyfied series is composed of words in that vocabulary. The sequence of these words — the sentences or phrases — are the patterns we need to learned.
To facilitate, I will divide the methodology of fuzzy time series into two procedures: training and forecasting.
We will use in this tutorial a very known time series from the origins of the Fuzzy Time Series: the enrollments of Alabama University (Enrollments). You can see the data below:
1. Definition of the Universe of Discourse UFirst we need to know the universe of discourse U from the training data, such as U = [min(X), max(X)]. Usually we extrapolate the upper and lower bounds by 20%, as a security margin.
2. Create the Linguistic Variable Ã (Universe of Discourse Partitioning)Now we need to split U on several overlapping intervals (a.k.a partitions) and create a fuzzy set for each one of them. The number of intervals is one of the most important parameters on Fuzzy Time Series and it will directly imply on model’s accuracy.
Beyond the number of partitions, the way we split U also have great impact on accuracy. Hereafter we adopt the simplest method for partitioning, the Grid Partitioning, where all partitions have the same length and format. To know other partitioning methods click here. For our example data we will use a 10 partitions scheme, such that the linguistic variable be Ã = { A0, A1, ..., A9 }.
3. FuzzyficationNow we can convert the numerical values of X(t) into fuzzy values of the linguistic variable Ã, giving rise to the fuzzy time series F(t). It is always good to remind that the fuzzy sets on Ã are overlapped, so for each x ∈ X(t) it is possible that it belongs to more than one fuzzy set Ai ∈ Ã. On Chen methods the things are a little bit simplier: just the maximum membership fuzzy set is choosen. However in other FTS methods all fuzzyfied values are considered.
Using the Chen’s method the fuzzyfied values for our test data will be F(t) = { A1, A2, A2, A3, A4, A4, A4, A4, A5, A5, A5, A4, A4, A4, A4, A4, A5, A7, A8, A8, A8, A7 }.
4. Creating the temporal patterns A temporal patterns indicates two fuzzy sets that appear sequentially on fuzzy time series F(t) and have the format Precedent →Consequent, where the precedent indicates a fuzzy set on time t and the consequent the fuzzy set that appears soon after on time t+1.
For the previous example, the generated temporal patterns will be: A1→A2, A2→A2, A2→A3, A3→A4, A4→A4, ..., A8→A8, A8→A7
5. Creating the rulesOur model rules also have the format Precedent →Consequent. Given the previously generated temporal patterns we will group them by its precedents. Our model will contain a rule for each distinct precedent found, and the consequent of each rule will be the union of all consequents of each temporal pattern with the same precedent.
For the previous example, the generated rules will be:A1 →A2A2 →A2,A3A3 →A4A4 →A4,A5A5 →A4,A5,A7A7 →A8A8 →A7,A8
The rule set is, in fact, out FTS model. They describe how our time series behaves and, if it was stationary enough (well behaved) we can use this model to forecast the next values on the time.
An simple and readable model like that has another advantages:
a) it is very easy to parallelize/distribute, what makes it very attractive for big data;
b) it is very easy to update, what makes it very attractive for frequently-changing data.
But how we can use this rules on forecasting?
Since we know the numerical value for time t, x(t) ∈ X(t), we now want to predict the next instant, x(t + 1).
1. Input value fuzzyficationThe input value x(t) will be converted into fuzzy values of the linguistic variable Ã, generating the value f(t). As in the training process only the most pertinent set is chosen.
For the example data, for t = 1992 the value is x(t) = 18876. Fuzzyfying x(t) the set of most pertinent is A7, so f(t) = A7.
2. Find the compatible rules Find the rule whose precedent is equal to f(t). The consequence of the rule will be the fuzzy forecast for t + 1, that is, f(t + 1).
For f(t) = A7 we have the rule A7 → A8. Then f(t + 1) = A8.
3. DefuzzyficationNow you need to convert f(t + 1) to a numeric value. For this we use the center-of-mass method, where the numerical value is equal to the mean of the centers of the fuzzy sets of f(t + 1), that is x(t+1) = n−1∑ Ai, for i = 0..n-1 and n equal to the number of sets in f(t+1).
Since f(t + 1) has only one set, then x(t + 1) = 19366.46.
This presented model is very simple, and I must say, archaic (it from 1996 !!!). But it is a good guide to understanding how fuzzy time series work. The most accurate methods now use more lags (this one use only the t-1 lag), weights in the rules (from the work of Yu (2005)), optimizers to find the best number of sets, lags, etc. In the next tutorials we will delve into more advanced models.
All above explained procedures can be reproduced on this Google Colab notebook.
Now it’s time to go play a little and get our hands dirty!!!! :-)
The pyFTS: Fuzzy Time Series for Python library is developed on MINDS — Machine Intelligence and Data Science of Federal University of Minas Gerais (UFMG) at Brazil, and is intended for students, researchers, data scientists or whose want to exploit the Fuzzy Time Series methods. PyFTS is a project in continuous development and all contributions are welcome!
Let’s know some of the main features of the library:
1. Test data
In the pyFTS.data package there are several datasets of common time series such as TAIEX, NASDAQ, S&P 500, passengers, etc. Each dataset has its own features — some are univariate, others multivariate, etc — but all of them basically have two functions:
get_data(): return the univariate time series
get_dataset(): return the multivariate time series
from pyFTS.data import Enrollmentstrain = Enrollments.get_data()
2. Data transformations
In the package pyFTS.common.Transformations several data transformations can be used for the pre-processing and/or post-processing data, which directly impacts the partitioning of the universe of discourse.
from pyFTS.common import Transformationstdiff = Transformations.Differential(1)
3. Partitioners
In the pyFTS.partitioners package are the universe of discourse partitioners. Each partitioner has the function of representing the linguistic variable, creating the partitions of the Discourse Universe and its fuzzy sets. In all of them at least two constructor parameters are required:
data: (mandatory!) the train data;
npart: (mandatory!) the minimal number of fuzzy sets to build;
mf: the membership function that will be used on fuzzy sets, which by default is triangular (trimf). The various membership functions can be found in pyFTS.common.Membership;
transformation: if any transformation is used in the series, it should be reported here.
from pyFTS.partitioners import Grid, Entropy, Util as pUtilfs = Grid.GridPartitioner(data=train, npart=20)print(fs)
We can explore several several alternatives for universe of discourse partitioners, each one of them with its own characteristics and performance.
4. Fuzzy Time Series methods
The various methods are found in the package pyFTS.models. All methods inherit from the class common.fts.FTS. For the end user there are two main methods to know:
FTS.fit(data, partitioner=fs) : Trains the model from the training data on parameter data and the linguistic variable already constructed by the fs partitioner. It is advisable to study the help of this function because here it is already possible to do the distributed training using clusters dispy. As soon as possible also a version compatible with pySpark will be realeased.
FTS.predict(data, type=’point’, steps_ahead=1): It uses the already trained model to make predictions from the lags contained in the data. There are three possible prediction types, indicated by the ‘type’ parameter: ‘point’ (default), ‘interval’ and ‘distribution’. One should be careful when choosing the method because not all of them work with all these types. Finally, the ‘steps_ahead’ parameter indicates the prediction horizon, or how many steps forward you want to predict.
from pyFTS.models import chenmodel = chen.ConventionalFTS(partitioner=fs)model.fit(train)print(model)forecasts = model.predict(test)
There are several models to be explored, so give a look on the example codes.
On the second part of this tutorial I’m talking about weighted methods, high order models, multivariate models and multi step forecasting. I also apply these models to forecast photovoltaic energy.
The third part talks about interval and probabilistic forecasting, non-stationarity, concept drift and time variant models.
Be sure to read them!
Chen Shyi-Ming. Forecasting enrollments based on fuzzy time series. Fuzzy sets and systems, vol. 81, number 3, pp. 311–319, 1996. URL: https://doi.org/10.1016/0165–0114(95)00220–0. Acess in 25/07/2018.
Ehlers, R.S. Análise de séries temporais. Departamento de Estatística, Universidade Federal do Paraná. URL: http://conteudo.icmc.usp.br/pessoas/ehlers/stemp/stemp.pdf. Acess in 25/07/2018.
Morettin, P.A. e Toloi, C.M.C. Análise de Séries Temporais. Blucher, 2004
Silva, P. C. L. et al. pyFTS: Fuzzy Time Series for Python, v4.0. URL: https://doi.org/10.5281/zenodo.597359. Acess in 25/07/2018.
Song, Qiang; Chissom, Brad S. Fuzzy time series and its models. Fuzzy sets and systems, vol. 54, number 3, pp. 269–277, 1993. URL: https://doi.org/10.1016/0165-0114(93)90372-O. Acess in 25/07/2018.
Yu, Hui-Kuang, Weighted fuzzy time series models for TAIEX forecasting. Physica A: Statistical Mechanics and its Applications, vol. 349, number 3, pp. 609–624, 2005. URL: https://doi.org/10.1016/j.physa.2004.11.006. Acess in 25/07/2018.
Zadeh, L. A. Fuzzy sets. Information and Control 8 (3) 338–353, 1965. URL: https://doi.org/10.1016/S0019-9958(65)90241-X. Acess in 25/07/2018.