A Tutorial on Artificial Neuro-Fuzzy Inference Systems in R
By: Nick Sokol
Tags: Rstats, Neural Networks, Data Science, Forecasting, Fuzzy Logic

Photo Credit: Google Images. A simple fuzzy logic model representation!

Computational methodologies are in no short supply these days. Almost every data related problem has a methodological approach that is best suited for maximum yields in knowledge discovery. Regression, clustering, neural networks, deep learning, and Bayesian methods are all commonly used in practice to create models that are reliable and precise. However, I have observed that a model type that is widely ignored outside of the engineering community, and works quite well, is fuzzy logic. Originally formulated by Lotfi Zadeh [1], fuzzy logic works by assigning predictors to fuzzy memberships rather than solid binary results that other modeling methods utilize. So instead of assigning a value of “78” and automatically classifying it as hot, a fuzzy logic model could classify that value as “moderately warm”, which would likely fit a greater range of the population. Furthermore, one important aspect of building great models, feature engineering, is handled to the extreme in fuzzy logic as a user must establish a universe of discourse (the range of values within a dataset, e.g. 0 to 200) and a number of fuzzy rules that cover all possible cases for combinations of variables within a given dataset. This allows for all types of adjustments to be made to the model to increase performance while decreasing computational costs. In short, fuzzy logic uses a combination of mathematical predictive power and human subjectivity to create the best model possible. While this would not be considered “machine learning” because of the human interactivity component, an extension of fuzzy logic has limited human subjectivity and added artificial neural network predictive power to the fuzzy logic schema. This approach is called Adaptive Neuro-Fuzzy Inference Systems (ANFIS) and has not seen as much application in the industrial realm as it has in the academic realm. This write-up will cover some of what ANFIS is capable of, and why many practitioners consider it to be superior to neural networks.

ANFIS was developed in the 1990’s [2,3] and allowed for the application of both fuzzy inference and neural networks to be applied to the same dataset. ANFIS models consist of five layers or steps, which conduct each phase of both the fuzzy logic portion of the algorithm and the neural network portion. Here I use the ANFIS package in R and solar energy production values for the United States to produce my example.

Step 1: Calculating Membership Functions

Membership functions are indicators of “likeness” or “degrees of membership” that a certain value has. These are typically represented by sinusoidal curves over a range of values (Figure 1). So if you are estimating energy consumption for a given year, you would break apart the values from least to greatest (with probably the lower and upper quantiles and the mean value) into five distinct categories with a deviation value causing a degree of spread and similarity between each of those values.

Photo credit: Author. Image showing fuzzy membership grades for each break of values.

Step 2: Firing Strength of Fuzzy Rules

The values, which were “fuzzified” in step 1 are now transported to a node layer and multiplied by the strength of an automatically generated fuzzy rule (Figure 2) [3]. Think of this as calculating a “weight” based on the automatically generated rule and the data provided to that rule.

Photo credit: From Jang, 1991 [2]. Shows the second step of the ANFIS algorithm.

Step 3: Normalize Firing Strength Calculation

The third step takes the output from step 2 and compares the value of the firing strength from the former node against the sum of all firing strengths (Figure 2)[2,3]. To simplify this description, think of the algorithm comparing the strength of an output rule of a single node, to the strength of other nodes and their underlying rules. If the strength is greater for a certain node, it is probably the “best possible” rule configuration for the dataset and is prioritized for the next step.

From Jang, 1991 [2]. Shows the third step of the ANFIS algorithm.

Step 4: Combine Premises (Independent Variables) with Consequents (Dependent Variable)

The fourth step takes the weighted values combined with the original inputs from the training data set to calculate an output based on the consequents data (Figure 2)[2,3].

From Jang, 1991 [2]. Shows the third step of the ANFIS algorithm.

Step 5: Predict & Final Output

The final step oversees the calculation of the sum of all incoming signals and applies them to a test dataset to produce a predicted value. This step also includes the de-fuzzification process for the data and translation back to meaningful values (Figure 3).

Photo Credit: Author. Shows predictions of total solar energy generation for each state for the year 2017.

As you can see, the produced output is very similar to the original data. Mean square error for our model is approximately 12% and the R2 is .91, which is very good, especially since I used such a limited training dataset. While ANFIS is certainly not the penultimate modeling technique, it has shown quite a bit of veracity in predicting values from a moderately complex dataset. One limitation of the model is a high computational cost in the form of necessary memory. When I tried running a model that had 8 independent variables, I consistently ran out of memory until I moved to another computer with 64GB of RAM. This is caused by the 5 layer approach per variable, which expands upon the demands. The range of values can also cause calculation time to increase. Despite these drawbacks, it is still possible to leverage the power of ANFIS for improved forecasting and as a comparison to other models and their output.

Some Tips:

If you are trying to apply this model to your own dataset, make sure you are calculating the universe of discourse correctly. If your values do not cover the full range of values that exist within your dataset, you will not be able to successfully train the model. In addition, if your sigma (deviation) does not accurately converge with other fuzzy memberships, you will also have difficulty running the model and getting an accurate prediction. If you have a big dataset (10GB or greater) it would be wise to calculate these descriptive statistics in Apache Hadoop and then apply them to ANFIS.

References

[1] Zadeh, L. A. (1988). Fuzzy logic. Computer, 21(4), 83–93.

[2] Jang, Jyh-Shing R (1991). Fuzzy Modeling Using Generalized Neural Networks and Kalman Filter Algorithm (PDF). Proceedings of the 9th National Conference on Artificial Intelligence, Anaheim, CA, USA, July 14–19. 2. pp. 762–767.

[3] Jang, J.-S.R. (1993). “ANFIS: adaptive-network-based fuzzy inference system”. IEEE Transactions on Systems, Man and Cybernetics. 23 (3): 665–685. doi:10.1109/21.256541.