Package: sanfis
sanfisThis is a PyTorch-based implementation of my project S-ANFIS:State-ANFIS: A Generalized Regime-Switching Model for Financial Modeling (2022). S-ANFIS is an generalization of Jang'sANFIS: adaptive-network-based fuzzy inference system (1993). The implemenation can easliy be used to fit an ANFIS network.1. What is S-ANFISS-ANFIS is a simple generalization of the ANFIS network, where the input to the premise and the consequence part of the model can be controlled separately. As general notation, I call the input the premise part "state" variablessand the input of the consequence part "input" or "explanatory" variablesx.For an in-depth explaination, check outour paper.2. InstallationThis package is intended to be installed on top of PyTorch, so you need to do that first.Step 1: Install PyTorchMake sure to consider the correct operating system: Windows, macOS (Intel / Apple Silicon) or Linux. Everything is explained on thedeveloper's website.To ensure that PyTorch was installed correctly, verify the installation by running sample PyTorch code:importtorchx=torch.rand(5,3)print(x)Step 2: Install sanfissanfis can be installed via pip:pipinstallsanfis3. Quick startFirst let's generate some data! The given example is anAR(2)-processwhoose AR-parameters depend on the regime of two independent state variables:# Load modulesimportnumpyasnpimporttorchfromsanfisimportSANFIS,plottingtoolsfromsanfis.datageneratorsimportsanfis_generator# seed for reproducibilitynp.random.seed(3)torch.manual_seed(3)## Generate Data ##S,S_train,S_valid,X,X_train,X_valid,y,y_train,y_valid,=sanfis_generator.gen_data_ts(n_obs=1000,test_size=0.33,plot_dgp=True)Set a list of membership functions for each of the state variables that enter the model:# list of membership functionsmembfuncs=[{'function':'sigmoid','n_memb':2,'params':{'c':{'value':[0.0,0.0],'trainable':True},'gamma':{'value':[-2.5,2.5],'trainable':True}}},{'function':'sigmoid','n_memb':2,'params':{'c':{'value':[0.0,0.0],'trainable':True},'gamma':{'value':[-2.5,2.5],'trainable':True}}}]The given example uses two sigmoid functions for each state variable.Now create the model, fit and evaluate:# make model / set loss function and optimizerfis=SANFIS(membfuncs=membfuncs,n_input=2,scale='Std')loss_function=torch.nn.MSELoss(reduction='mean')optimizer=torch.optim.Adam(fis.parameters(),lr=0.005)# fit modelhistory=fis.fit([S_train,X_train,y_train],[S_valid,X_valid,y_valid],optimizer,loss_function,epochs=1000)# eval modely_pred=fis.predict([S,X])plottingtools.plt_prediction(y,y_pred,save_path='img/sanfis_prediction.pdf')# plottingtools.plt_learningcurves(history)4. Features4.1 Membership functionsThe implementation allows a very flexible usage of membership functions. For each input variable that enters the premise-part of the model, the type and number of membership functions can be flexibly chosen. As of today, three possible membership functions are implemented:GaussianThe Gaussian is described by 2 parameters,mufor the location andsigmafor the wideness.# Examplegaussian_membfunc={'function':'gaussian','n_memb':3,# 3 membership functions'params':{'mu':{'value':[-2.0,0.0,1.5],'trainable':True},'sigma':{'value':[1.0,0.5,1.0],'trainable':True}}}In this example, three membership functions are considered.General bell-shapedThe general bell-shaped function is described by three parameters,a(wideness),b(shape) andc(location).bell_membfunc={'function':'bell','n_memb':2,'params':{'c':{'value':[-1.5,1.5],'trainable':True},'a':{'value':[3.0,1.0],'trainable':False},'b':{'value':[1.0,3.0],'trainable':False}}}SigmoidThe sigmoid is described by two parameters:c(location) andgamma(steepness).sigmoid_membfunc={'function':'sigmoid','n_memb':2,'params':{'c':{'value':[0.0,0.0],'trainable':True},'gamma':{'value':[-2.5,2.5],'trainable':True}}}Remember to add a list of membership functions asmembfuncargument when creating theSANFISoject, e.g.:MEMBFUNCS=[gaussian_membfunc,bell_membfunc,sigmoid_membfunc]model=SANFIS(MEMBFUNCS,n_input=2)model.plotmfs(bounds=[[-2.0,2.0],# plot bounds for first membfunc[-4.0,2.0],# plot bounds for second membfunc[-5.0,5.0]],# plot bounds fo third membfuncsave_path='img/membfuncs.pdf')4.2 TensorboardTensorboard provides visualization needed for machine learning experimentation. Further information can be foundhereStep 1: Install tensorboardpipinstalltensorboardStep 2: enable tensorboard usage during trainingTensorboard functionality can be added via arguments in thefit()function, e.g.history=model.fit(...use_tensorboard=True,logdir='logs/tb',hparams_dict={})Note thathparams_dictis an optional argument where you can store additional hyperparameters of you model, e.g.hparams_dict={'n_input':2}.Step 3: Open tensorboardtensorboard--logdir=logs/tb5. Using the plain vanilla ANFIS networkTo use the plain vanilla ANFIS network, simply remove the state variablessfrom the training (fit()). This automatically sets the same input for premise and consequence part of the model.# Set 4 input variables with 3 gaussian membership functions eachMEMBFUNCS=[{'function':'gaussian','n_memb':3,'params':{'mu':{'value':[-0.5,0.0,0.5],'trainable':True},'sigma':{'value':[1.0,1.0,1.0],'trainable':True}}},{'function':'gaussian','n_memb':3,'params':{'mu':{'value':[-0.5,0.0,0.5],'trainable':True},'sigma':{'value':[1.0,1.0,1.0],'trainable':True}}},{'function':'gaussian','n_memb':3,'params':{'mu':{'value':[-0.5,0.0,0.5],'trainable':True},'sigma':{'value':[1.0,1.0,1.0],'trainable':True}}},{'function':'gaussian','n_memb':3,'params':{'mu':{'value':[-0.5,0.0,0.5],'trainable':True},'sigma':{'value':[1.0,1.0,1.0],'trainable':True}}},]# generate some data (mackey chaotic time series)X,X_train,X_valid,y,y_train,y_valid=datagenerator.gen_data(data_id='mackey',n_obs=2080,n_input=4)# create modelmodel=SANFIS(membfuncs=MEMBFUNCS,n_input=4,scale='Std')optimizer=torch.optim.Adam(params=model.parameters())loss_functions=torch.nn.MSELoss(reduction='mean')# fit modelhistory=model.fit(train_data=[X_train,y_train],valid_data=[X_valid,y_valid],optimizer=optimizer,loss_function=loss_functions,epochs=200,)# predict datay_pred=model.predict(X)# plot learning curvesplottingtools.plt_learningcurves(history,save_path='img/learning_curves.pdf')# plot predictionplottingtools.plt_prediction(y,y_pred,save_path='img/mackey_prediction.pdf')6. Related workAnfisTensorflow2.0by mebare-bones implementation of ANFIS(manual derivatives) bytwmeggsPyTorch implementationbyJames Powersimple ANFIS based on Tensorflow 1.15.2bySantiago CuervoContactI am very thankful for feedback. Also, if you have questions, please contactgregor.lenhard92@gmail.comReferencesIf you use my work, please cite it appropriately:G. Lenhard and D. Maringer, "State-ANFIS: A Generalized Regime-Switching Model for Financial Modeling," 2022 IEEE Symposium on Computational Intelligence for Financial Engineering and Economics (CIFEr), 2022, pp. 1-8, doi: 10.1109/CIFEr52523.2022.9776208.BibTex:@INPROCEEDINGS{lenhard2022sanfis,
  author={Lenhard, Gregor and Maringer, Dietmar},
  booktitle={2022 IEEE Symposium on Computational Intelligence for Financial Engineering and Economics ({CIFEr})}, 
  title={State-{ANFIS}: A Generalized Regime-Switching Model for Financial Modeling}, 
  year={2022},
  pages={1--8},
  doi={10.1109/CIFEr52523.2022.9776208},
  organization={IEEE}
  }