In this article, we propose a novel hybridization of regress ion trees (RT) and radial basis function networks (RBFN), namely, radial basis neura l tree (RBNT) model, for waste recovery process improvement in the paper industr y. As a by-product of the paper manufacturing process, a lot of waste along with va luable ﬁbers and ﬁllers come out from the paper machine. The waste recovery process ( WRP) involves sep- arating the unwanted materials from the valuable ones so tha t the recovered ﬁbers and ﬁllers can be further reused in the production process. T his job is done by ﬁber-ﬁller recovery equipment (FFRE). The eﬃciency of FFRE depends on several crucial process parameters and monitoring them is a diﬃcult proposition. The pro- posed model can be useful to ﬁnd the essential parameters fro m the set of available data and perform prediction task to improve waste recovery p rocess eﬃciency. An idea of parameter optimization along with regularity condi tions for the universal consistency of the proposed model are given. The proposed mo del has the advan- tages of easy interpretability and excellent performance w hen applied to the FFRE eﬃciency improvement problem. Improved waste recovery wil l help the industry to become environmentally friendly with less ecological da mage apart from being cost-eﬀective. KEYWORDS: Waste recovery; Regression tree; Radial basis function net work; Radial basis neural tree 1 INTRODUCTION This work is motivated by a speciﬁc problem in a modern paper industry w hich produces papers for multiple uses. Paper machine produces papers by using pulp, ﬁbers, ﬁllers, chemical lubricants, and hu ge water resources. As a by-product of the paper manufacturing process, a lot of waste materials comes out from the paper mach ine. These waste materials contain lean water, garbages and valuable ﬁbers, ﬁllers. To save these useful materials from th e waste, an equipment called FFRE (also popularly 2 Chakraborty ET AL known as Krofta supercell) is used by many paper manufacturing industries1,2. Dissolved air ﬂotation cum sedimentation process (DAFSP) is used in FFRE to save valuable materials from waste. If FFRE b ecomes eﬃcient, it will help the company to reuse the relevant material which will be economical for the company. However, the eﬃ ciency of FFRE is not always satisfactory, and that causes a monetary loss for the company. A lot of preliminary analysis was d one to ﬁnd out a set of possible causal variables that aﬀects FFRE eﬃciency which in other words means high recovery percentage of valuab le materials. The objective of this paper is to correctly ﬁnd out the most important process parameters from the s et of all available causal variables. Also, we will try to develop a prediction model that can help the company to estimate future los es. This work aims to help the company in improving the quality and productivity of the paper manufacturing process. Furth ermore, process improvement through waste management and valuable material recovery can make the manufacturing process environmentall y friendly with very less ecological damage. The formulation of the above-stated problem along with data descriptio n is presented in Section 4. The problem can be viewed as a typical nonparametric regression problems w here one can establish a relationship between the response variable (recovery percentage of FFRE) and the major causal variables (process parameters of DAFSP) without having any prior information about the data. Regression trees (RT), support v ector regression (SVR) and artiﬁcial neural networks (ANN) have been applied for various prediction tasks in water quality imp rovement, water planning and many other related problems3,4,5,6. Even various hybrid regression models have been developed for perform ing regression task in many real-life problems like water demand forecasting and others7,8,9,10. To develop a prediction model for the waste recovery improvement that can also ﬁnd out critical parameters among the set of possible parameter s, we take recourse to nonparametric regression methodology. Tree-structured model, RT, is popular for its easy inter pretability and built-in mechanism of important variable selection11but they are sometimes unstable. On the other hand, neural networks (N N) were developed to mathematically model human intellectual abilities by biologically plausible engineering desig ns12but it may fail when limited data are available. Among the family of ANNs, radial basis function networks (RBFN) ha ve the advantages of having only one hidden layer, less time complexity, and easy interpretability. To harness their advantages as well as to overcome their drawbacks, several works of literature concentrated on the hybridization of RT and ANN models, viz. entropy nets13and neural tree14. Motivated by the above discussion, we have proposed a radial basis neur al tree (RBNT) model which utilizes the power of both the tree-based models and ANN to solve the production process eﬃciency problem. We also ﬁll the gap between theory and practice by showing the consistency of the proposed regression model. I n our model, we combine RBFN with RT to enhance the predicting accuracy for this typical regression problem. In the case of FFRE dataset, parameter optimization within the proposed RBNT framework yields the proposed model that is intermediate between R T and RBFN and outperforms RT, RBFN and other commonly used nonparametric regression models. The proposed model has the advantages of higher accuracy, converges much faster than many other complex hybrid models and easy interpretability as co mpared to many advanced “black-box-like" models. The proposed model has no assumptions on the distribution of the i nput and output variables, unlike parametric regresion models. Chakraborty ET AL 3 When applied to solve the process eﬃciency problem, we found a set of most important process parameters from the available data that controls process recovery percentage. Our proposed model is found u seful for forecasting future process eﬃciency in terms of recovery percentage till the company is able to make suggested chan ges to the process for its improvement. To summarize, the contributions are: •We propose a pipeline model, namely, RBNT model for improving waste recovery process based on decision tree and neural networks. •Optimization of model parameters along with the regularity conditions for the universal consistency of the proposed model is discussed. •The robustness of the proposed algorithm is shown through experi mental evaluation over ﬁve publicly available regression datasets from various applied ﬁelds. The paper is organized as follows. In Section 2, we describe the proposed R BNT model and its statistical properties are discussed in Section 3. Waste recovery improvement problem, model evaluati on and comparisons are presented in Section 4. To show the robustness and general applicability of the proposed model, we evaluated its performance on the publicly available regression datasets in Section 5. Finally, the concluding remarks are giv en in Section 6. 2 RADIAL BASIS NEURAL TREE (RBNT) MODEL RT uses a hierarchical segmentation of the input feature spaces whereas RBFN is a particular type of ANN model that uses a radial basis function as an activation function. In the proposed RBNT mod el, we ﬁrst split the input feature space into areas by RT algorithm (for detailed discussion see Section 3.1). Based on feature rankings provided by RT, a set of important features are chosen and extracted from the training dataset. We then build the RBF N model using the important variables obtained through RT algorithm along with the prediction results obtained fro m RT as another input information in the input layer of the network. The eﬀectiveness of the proposed classiﬁer lies in the selection o f important features and use of prediction results of RT followed by the application of the RBFN model. The inclusion of RT output as an additional input feature not only improves the model accuracy but also increases class separability. The proposed RBNT regr ession model can handle high-dimensional datasets through the implementation of RT in selecting features as well as th e incorporation of its predicted outputs tied up with one hidden layered RBFN model. This hybridization improves the perfo rmances of RT, RBFN as well as reduces the biases and variances of these individual models. The informal workﬂow of our pro posed model is as follows: 4 Chakraborty ET AL FIGURE 1 An example of RBNT with /u1D44B/u1D456;/u1D456= 1,2,3,4,5as Important features obtained by RT, /u1D43F/u1D456as leaf nodes and /u1D442/u1D443as RT output. Creation of RT (Left) and one hidden layered RBFN model (Rig ht). •Apply RT algorithm to rank the feature, extracts important features and ﬁn d the splits between diﬀerent adjacent values of the features. •Choose the features that have minimum mean squared error. We further gro w unpruned RT and record its outputs. RT has an in-built important feature selection mechanism. •Export important input variables along with additional feature (predi ction values of RT algorithm) to the RBFN model and a neural network is generated. •RBFN model uses a gaussian kernel as an activation function, and parameter opti mization is done using a gradient descent algorithm (for detail discussion see Section 3.2). Finally, we record the ﬁnal outputs. RBNT model is a two-step pipeline approach such as choosing most imp ortant features among the set of available features and ﬁnally getting an improved prediction values. Since RT is robust in handling the curse of high-dimensional datasets, thus incorporation of its predicted class levels along with important features obtained by RT as input features in RBFN will necessarily improve the performance of the single classiﬁers. A ﬂowchart of the R BNT model is presented in Figure 1. 3 THEORETICAL PROPERTIES OF THE RBNT MODEL The proposed RBNT model has the following architecture: (1) it extract s important features from the feature space using RT algorithm; (2) it builds one hidden layered RBFN model with the imp ortant features extracted using RT along with RT outputs as an additional feature. Now let us investigate the theoretical properties of the proposed model by introducing a set of regularity conditions for consistency of RT followed by the consistency results o f RBFN algorithm. Further, an idea about parameter optimization in the RBNT model is proposed in Section 3.2. Chakraborty ET AL 5 3.1 Regularity Conditions for Universal Consistency Let/u1D44Bbe the space of all possible values of /u1D451features, i.e. /u1D44B= (/u1D44B1,/u1D44B2,...,/u1D44B/u1D451)and/u1D44Cbe set of all possible values that the response variable can take and /u1D44C∈ [−/u1D43E,/u1D43E]. We wish to estimate regression function /u1D45F(/u1D465) =/u1D438(/u1D44C/uni007C.var/u1D44B=/u1D465) ∈ [−/u1D43E,/u1D43E]based on a training sample /u1D43F/u1D45B= {(/u1D44B1,/u1D44C1),(/u1D44B2,/u1D44C2),...,(/u1D44B/u1D45B,/u1D44C/u1D45B)}, where/u1D44B/u1D456= (/u1D44B/u1D4561,/u1D44B/u1D4562,...,/u1D44B/u1D456/u1D451) ∈/u1D44Bwith n observations and let Ω = {/u1D7141,/u1D7142,...,/u1D714/u1D458}be a partition of the feature space /u1D44B. We denote /uni0303.s1Ωas one such partition of Ω. Deﬁne(/u1D43F/u1D45B)/u1D714/u1D456= {(/u1D44B/u1D456,/u1D44C/u1D456) ∈ /u1D43F/u1D45B∶/u1D44B/u1D456∈/u1D714/u1D456,/u1D44C/u1D456∈ [−/u1D43E,/u1D43E]}to be the subset of /u1D43F/u1D45Binduced by/u1D714/u1D456and let(/u1D43F/u1D45B)/uni0303.s1Ωdenote the partition of /u1D43F/u1D45Binduced by/uni0303.s1Ω. The criterion used to identify the best features involves mean squared error for regression problems. We don’t make any assumption on the distribution of the pair (/u1D44B,/u1D44C) ∈ℝ/u1D451×[−/u1D43E,/u1D43E]. Mean squared error is used to partition feature space into a set/uni0303.s1Ωof nodes. There exists a partitioning regression function /u1D45D∶/uni0303.s1Ω→/u1D44Csuch that/u1D45Dis constant on every node of /uni0303.s1Ω. Now let us deﬁne/uni0302.s1/u1D43F/u1D45Bbe the space of all learning samples and /u1D53Bbe the space of all partitioning regression function then, binary parti tioning and regression tree based rule Φ ∶/uni0302.s1/u1D43F/u1D45B→/u1D53Bsuch that Φ(/u1D43F/u1D45B) = (/u1D713◦/u1D719)(/u1D43F/u1D45B), where/u1D719maps/u1D43F/u1D45Bto some induced partition (/u1D43F/u1D45B)/uni0303.s1Ω and/u1D713is an assigning rule which maps (/u1D43F/u1D45B)/uni0303.s1Ωto a partitioning regression function /u1D45Don the partition /uni0303.s1Ω. A reasonable assigning rule/u1D713is the plurality rule /u1D713/u1D45D/u1D459((/u1D43F/u1D45B)/uni0303.s1Ω) =/u1D45Dsuch that if/u1D465∈/u1D714/u1D456, then /u1D45D(/u1D465) = arg min /u1D44C/u1D456∈[−/u1D43E,/u1D43E]/uni007C.var1 /u1D45B/u1D45B/uni2211.s1 /u1D456=1(Φ(/u1D43F/u1D45B)−/u1D44C/u1D456)2/uni007C.var The stopping rule in RT is decided based on the minimum number of spli t in the posterior sample called minsplit. If minsplit ≥/u1D6FCthen/u1D714/u1D456will split into two child nodes and if minsplit </u1D6FCthen/u1D714/u1D456is a leaf node and no more split is required. Here /u1D6FCcan be determined by the user, usually it is taken as 10% of the training sample size. Now we are going to evolve a suﬃcient regularity condition for the binary partitioning and regression tree based rule t o be universally consistent. Theorem 1. Suppose(/u1D44B,/u1D44C)be a random vector in ℝ/u1D451×[−/u1D43E,/u1D43E]and/u1D43F/u1D45Bbe the training set of n outcomes of (/u1D44B,/u1D44C). Finally if for every n and /u1D464/u1D456∈̃Ω/u1D45B, the induced subset (/u1D43F/u1D45B)/u1D464/u1D456contains at least /u1D458/u1D45Bof/u1D44B, then empirically optimal regression trees strategy employing axis parallel splits are consistent when the size /u1D458/u1D45Bof the tree grows as /u1D45C(/u1D45B /u1D459/u1D45C/u1D454(/u1D45B)). Proof. The proof of the theorem can be easily derived from the proof of consisten cy of the histogram regression estimation using data-dependent partitions15. Remark 1. Regression trees are consistent when the size of the tree grows as /u1D45C(/u1D45B /u1D459/u1D45C/u1D454(/u1D45B)), where n is the number of training samples. It should be noted that the choice of important features based on RT is a g reedy algorithm and the optimality of local choices of the best feature for a node doesn’t guarantee that the constructed tree w ill be globally optimal12. Further, we build the RBFN model with RT extracted features and /u1D442/u1D443as another input feature in our model. The dimension of the input layer in the RBFN model, denoted by /u1D451/u1D45A(</u1D451)equals the number of important features obtained by RT + 1. Since 6 Chakraborty ET AL RBFN model consists of strictly one hidden layer, thus the proposed model is easily interpretable and fast in implementation. Our next objective is to discuss the suﬃcient condition for universal consistency of the RBFN model. After incorporating RT output in the feature space, we have /u1D45Btraining sequence /u1D709/u1D45B= {(/u1D44D1,/u1D44C1),...,(/u1D44D/u1D45B,/u1D44C/u1D45B)}of/u1D45Bi.i.d copies of (/u1D44D,/u1D44C)taking values fromℝ/u1D451/u1D45A×[−/u1D43E,/u1D43E], a regression estimate realized by a one-hidden layered neural network is ch osen to minimize the empirical /u1D43F2risk1 /u1D45B∑/u1D45B /u1D457=1∣/u1D453(/u1D467/u1D457)−/u1D44C/u1D457∣2. RBFN is a family of ANNs, consists of only a single hidden layer and u ses a nonlinear function called radial basis function as an activation function, unlike feed forward neural network. Figure 1 gi ves an example of RBFN with one hidden layer. RBFN is presented as a three-layered feed-forward structure where input layer di stributes inputs to the hidden layer which contains neurons with nonlinear activation function16. Gaussian functions are most frequently used in this layer can be deﬁned as f ollows: /u1D719/u1D456(/u1D467/u1D456) =/u1D719/parenleft.s1∥/u1D467/u1D456−/u1D450/u1D456∥;/u1D70E/u1D456/parenright.s1=/u1D452/u1D465/u1D45D/parenleft.s3 −∥/u1D467/u1D456−/u1D450/u1D456∥2 2/u1D70E2 /u1D456/parenright.s3 where/u1D467/u1D456is an input vector, /u1D719/u1D456is the output of /u1D456/u1D461ℎhidden neuron in the hidden layer with centers /u1D450/u1D456and/u1D70E2 /u1D456as the variance. Here we can select the center vector as cluster centers of the input data. For practical use, t he number of clusters is generally chosen to be much smaller than the number of data points resulting in RBFN of less complexity than other types of ANNs. Let us now consider a RBF network with one hidden layer (an example is given in Fi gure 2) having /u1D458nodes for a ﬁxed gaussian function is given by the equation: /u1D453(/u1D467/u1D456) =/u1D458/uni2211.s1 /u1D457=1/u1D464/u1D457/u1D719/parenleft.s1 ∥/u1D467/u1D456−/u1D450/u1D457∥/parenright.s1 +/u1D4640, where/u1D4640,/u1D4641,...,/u1D464/u1D458∈ [−/u1D44F,/u1D44F] (/u1D44F>0)and/u1D4501,/u1D4502,...,/u1D450/u1D458∈ℝ/u1D451/u1D45A. The weights /u1D464/u1D457and/u1D450/u1D457are parameters of the RBFN and /u1D719is the gaussian radial basis function with /u1D719(/u1D467)→0/u1D44E/u1D460/u1D465→∞. The next Theorem due to krzyzak et al.17gives regularity conditions for the universal consistency of RBFN model. Theorem 2. Consider a RBF network with one hidden layer having /u1D458nodes. If/u1D458→∞, /u1D44F→∞/u1D44E/u1D45B/u1D451/u1D458/u1D44F4/u1D459/u1D45C/u1D454(/u1D458/u1D44F2) /u1D45B→0as/u1D45B→∞, then RBFN model is said to be universally consistent for all distribut ion of(/u1D44D,/u1D44C). Remark 2. We can conclude that if the RBNT model satisﬁes the regularity conditions as stated in Theorem 1 and Theorem 2, then the proposed algorithm will be universally consistent. This is a fundamental property of any model for its robustness and general use. But computationally choosing parameters of RBFN by minimi zing empirical /u1D43F2risk will be very costly. In practice, parameters of the RBFN model are learned by a gradient descent algorithm as ment ioned below. Chakraborty ET AL 7 FIGURE 2 An example of Radial basis network with one hidden layer. 3.2 Optimization of Model Parameters Here we are going to discuss about the tuning parameters of the RBNT mo del. In the ﬁrst stage of the pipeline model, ‘minsplit’ function (see Section 3.1) to be chosen as 10% of the training dataset is recommended as the stopping rule in RT algori thm. Further, we grow unpruned RT and use RT suggested features along with RT output as an additional feature in the input space of RBFN. RBFN is trained using a linear combination of gaussian basis fu nctions. Therefore we need to use an optimization algorithm for empirical error (to be denoted by E in the rest of the paper) minimization on /u1D709/u1D45B18. Three important parameters to be optimized while training RBF network are: centers (/u1D450/u1D456), standard deviation (/u1D70E/u1D456)and weights (/u1D464/u1D457)of each neuron. We use a gradient descent algorithm over E to perform the optimization task in the RBFN model19, as follows: Δ/u1D450/u1D456= −/u1D70C/u1D450∇/u1D450/u1D456/u1D438, Δ/u1D70E/u1D456= −/u1D70C/u1D70E/u1D715/u1D438 /u1D715/u1D70E/u1D456, Δ/u1D464/u1D457= −/u1D70C/u1D464/u1D715/u1D438 /u1D715/u1D464/u1D457, where/u1D70C/u1D450,/u1D70C/u1D70Eand/u1D70C/u1D464are small positive constants. Using this the parameters of the gaussi an basis function will be optimized. Generalization error is usually estimated by cross-validation method, and the optimum value of /u1D458would be found by trial and error. 8 Chakraborty ET AL FIGURE 3 SIPOC diagram of ﬁber and ﬁller recovery process 4 APPLICATION TO WASTE RECOVERY PROBLEM A modern paper manufacturing company is facing a problem of ﬁber and ﬁller los ses which account for a substantial monetary loss for the company. The objective of this work is to improve the w aste recovery process by improving the percentage of recovery of the FFRE which will reduce the ﬁber and ﬁller losses. The inlet to F FRE is the waste (lean backwater combined with some valuable materials) coming out of the paper machine which undergoes frot h ﬂotation when treated with chemical and air20,21. The chemical helps to form the ﬂocks from the ﬁnes present in the lean back water and air helps to form the cake over the FFRE for recovery. The cake is then scooped out, collected and sent back to the paper mac hine process stream. Remaining water is sent to water level tank for further to clariﬁcation. To give a better idea ab out the process, we presented the SIPOC diagram of ﬁber ﬁller recovery process in Figure 3. FFRE removes solids using air ﬂoatation and sedimentation. Turbulence cau sed by water movement is an important factor in ﬂoatation and greatly reduces the eﬃciency of the other types of ﬂoatation un its. Conventionally there must always be water movement for the water to ﬂow from inlet to outlet. In FFRE, the inl et and outlet of the process are not stationary, and it rotates about the center. The rotation is synchronized so that the water in the wat er level tank achieves zero velocity during ﬂoatation22. In practical terms, this allows better clariﬁcation in smaller surface areas and a much shallower tank. The processing time of water from the inlet to outlet takes around 2-3 minutes. Air is dissolved into the water using air dissolving tube (ADT) of FFRE1, and unclariﬁed water is released through a valve. The water ﬂows in at the exact center, through a rotary joint. Coarse air is discharged through a vent pipe in the duct. The ﬂow is directed to eliminate turbu lence. Since the inlet distribution is moving forward at the same speed than the water is ﬂowing out, the water stays in one spot in th e tank without any movement during ﬂoatation. The ﬂoated material is recovered from the top surface through the FFRE spiral s coop. The scoop is designed to remove the ﬂoated material at the highest possible consistency, with a minimum surface di sturbance. The level of water determines the consistency of the ﬂoated material removed. The ﬂoatation system in FFRE removes the so lid content in the water by ﬂoating them to the Chakraborty ET AL 9 FIGURE 4 Schematic diagram of ﬁber and ﬁller recovery process TABLE 1 Sample data set Sl. No. Inlet Flow Water Pressure Air Pressure Air-Left Air-Right ADT-D ADT-D Amount of Recovery Left Right chemical Percentage 1 1448 6.4 5.8 1.0 2.1 3.2 4.0 2.0 96.80 2 1794 5.2 5.6 2.4 1.6 3.6 4.0 3.0 97.47 3 2995 6.0 6.0 1.5 4.5 4.0 4.8 4.0 28.87 4 1139 6.5 6.0 1.2 1.7 3.0 4.6 2.0 33.05 5 2899 6.2 5.7 2.0 1.2 3.1 4.0 2.0 97.91 6 1472 6.6 6.8 3.7 3.1 5.2 4.8 4.0 57.77 7 1703 6.2 6.0 2.9 1.0 3.0 4.2 2.0 26.94 8 1514 5.5 5.0 2.0 2.1 3.8 4.7 2.0 67.01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . surface for removal. Chemicals are used to increase the clariﬁer eﬃciency by ﬂocki ng out small and colloidal particles, that otherwise would not ﬂoat or settle in the clariﬁer. The schematic diag ram of the FFRE process is shown in Figure 4. 4.1 Data To identify the causal parameters aﬀecting the FFRE eﬃciency, a cause and eﬀect analys is is done for revealing the key relation- ship among causal variables with FFRE eﬃciency. A list of controllable parame ters is prepared and accordingly we collected the data for the paper tissue from the process. The dataset collected for a year from the process on the following causal variables: Inlet Flow, Water Pressure (water inlet pressure to ADT), Air Pressu re, Pressure of Air-Left, Pressure of Air-Right, Pressure of ADT-D Left, Pressure of ADT-D Right and Amount of chemical lubricants . The response variable (FFRE recovery percentage) lies between 20 to 100. Sample data set for the paper tissue is presented i n Tables 1. This dataset will be used for ﬁnding crucial process parameters and also ﬁnding a prediction model that can help the company for forecasting future recovery percentage of FFRE. 10 Chakraborty ET AL 4.2 Performance metrics We are going to use root mean square error (RMSE), mean absolute percentage error (MAPE), co-eﬃcient of multiple deter- mination (/u1D4452)and adjusted /u1D4452(/u1D434/u1D451/u1D457/u1D4452) to evaluate the performance of diﬀerent regression models: /u1D445/u1D440/u1D446/u1D438=/uni221A.s1 1 /u1D45B∑/u1D45B /u1D456=1(/u1D466/u1D456−/uni0302.s1 /u1D466/u1D456)2;/u1D440/u1D434/u1D443/u1D438=1 /u1D45B∑/u1D45B /u1D456=1/uni007C.x/uni007C.x/uni007C.x/uni007C.x/u1D466/u1D456−/uni0302.s1 /u1D466/u1D456 /u1D466/u1D456/uni007C.x/uni007C.x/uni007C.x/uni007C.x; /u1D4452= 1−/bracketleft.s3∑/u1D45B /u1D456=1(/u1D466/u1D456−/uni0302.s1 /u1D466/u1D456)2 ∑/u1D45B /u1D456=1(/u1D466/u1D456−/u1D466)2/bracketright.s3 ;/u1D434/u1D451/u1D457/u1D4452= 1−/bracketleft.s3 (1−/u1D4452)(/u1D45B−1) /u1D45B−/u1D458−1/bracketright.s3 ; where,/u1D466/u1D456,/u1D466,/uni0302.s1 /u1D466/u1D456denote the actual value, average value and predicted value of the dependent variab le, respectively for the /u1D456/u1D461ℎ instant. Here /u1D45Band/u1D458denote the number of data points and independent variables used for perfor mance evaluation, respectively. The lower the value of RMSE and MAPE and the higher the value of /u1D4452and Adj/u1D4452, the better is the model is. 4.3 Analysis of Results Here we elaborate about the experimental evaluation of the proposed RBNT model while applied to the waste recovery process (WRP) dataset. We ﬁrst shuﬄed the observations of the WRP dataset rand omly and split them into training and testing data in a ratio of 60 ∶ 40 . Each experiments are repeated 10 times with diﬀerent randomly assigned trai ning and test sets. We have considered here six well known machine learning models for comparison pu rpose. All the competing methods are implemented using standards libraries in R on a PC with 2.1GHz processor and 8 GB memory. These algorithms have been executed with their default parameters. The performance of each model was evaluated with ﬁve statistical measures (viz. MAE, RMSE, MAPE, /u1D4452and Adj(/u1D4452)) based on the prediction values relative to the observed values. The FFRE eﬃciency corresponding to a tissue depends on several process featu res such as Water Pressure, Air Pressure, Inlet Flow, AIR-Left, AIR-Right, amount of chemical, ADT-D Left and ADT-D Right. Our task is to select the important features out the above and apply the model to improve the recovery percentage. We ﬁrs t apply the RT method to select the important FFRE features and then compute the RBFN model with the selected important f eatures along with RT output as shown in Figure 1. RT is trained using rpart package implementation in R that uses gini index, and /u1D436/u1D45Dfor selection of variables to enter and leave the tree structure. To implement neural nets, we ﬁrst standardize t he datasets and run the ANN model. A transfer function (“logsig") is applied to bring back the original form at the end of model ing. We implemented the ANN model with diﬀerent combinations of hidden layers without employing any other feature select ion algorithm using neuralnet package. Due to the fact that the dataset is medium-sized, thus going beyond 2 hidden layered (2 HL) neural net will overﬁt the dataset. The number of neurons in 1st hidden layer are chosen as 2/3 the size of the input layer and the number of neurons in 2nd hidden layer are chosen as 1/3 the size of the input layer. RBFN with gaussian radial basis kernel was trained using RSNNS package and the maximum Chakraborty ET AL 11 TABLE 2 Quantitative measures of performance for diﬀerent regression models. Regression Models MAE RMSE MAPE /u1D4452Adj(/u1D4452) RT 11.691 16.927 29.010 59.028 55.304 ANN 12.334 17.073 27.564 58.310 54.529 SVR 12.460 20.362 40.010 40.174 35.325 BART 12.892 16.010 30.038 59.380 56.458 RBFN 13.926 18.757 32.48 49.689 46.335 Neural tree 10.895 16.012 24.021 65.120 62.946 RBNT 9.226 14.331 20.187 70.632 68.675 number of iterations in all these NN implementations are chosen as 100. Ex ecution time for RBFN model is much less than ANN but higher than RT. The essential features for FFRE recovery dataset are: Water Pressure, Air Pressu re, Inlet Flow, ADT-D Left and ADT-D Right. Table 2 exhibits the quantitative measures corresponding to M AE, RMSE, MAPE, /u1D4452and Adj(/u1D4452) respectively for the data set and for all the competing methods. A method in predicting th e data is said to perform well if it is minimize the MAE, RMSE, MAPE values and maximize the /u1D4452and Adj(/u1D4452) values between the predictions relative to the observations. It can be observed from Table 2 that the MAE, RMSE and MAPE values obtained throug h the proposed model is always smaller than the values obtained from all other competing methods over the WRP datase t. Further, we compare our proposed model with diﬀerent regressions models, viz. RT, ANN, RBFN, SVR, bayesian adaptiv e regression tree (BART) and neural tree. Neural tree provides the second best performance than the remaining models in term s of having low RMSE, MAE and MAPE values. Based on the recommendation of the model, we conducted design of experiment s to ﬁnd optimal level of essential parameters to improve FFRE eﬃciency. Also, the paper company can use our model for fut ure prediction of recovery percentage. 5 OTHER EXPERIMENTS We now compare the RBNT models with standard RT, standard ANN, BART, e tc. on ﬁve regression data sets from UCI Machine Learning Repository23. Nonnumerical features and samples with missing entries were systematically removed from the datasets. A brief summary of these datasets are available in Table 3. To showcase the abilities of the RBNT, we picked diverse, but mostly s mall and medium-sized regression datasets. These are the sets where RT often outperforms NN, since the latter typically requ ire plentiful training data to perform reasonably well. It is on these particular types of datasets where the beneﬁts of neural optimizati on can usually not be exploited, but with the speciﬁc inductive bias of the proposed RBNT model it becomes possible. These are such types of datasets in which prediction is not the only task but also feature selection plays a vital role. To start, we ﬁ rst shuﬄed the observations in each of the ﬁve diﬀerent 12 Chakraborty ET AL TABLE 3 Data set description: number of samples and number of features, after removi ng observations with missing information or nonnumerical input features. Data set Number of samples Number of features Auto MPG 398 7 Concrete 1030 8 Forest Fires 517 10 Housing 506 13 Wisconsin 194 32 TABLE 4 Average RMSE results for each of the models across the diﬀerent datasets Data RT ANN SVR BART RBFN Neural Tree RBNT Auto MPG 3.950 4.260 5.720 3.220 4.595 3.300 3.215 Concrete 8.700 10.180 11.588 5.540 10.210 7.420 7.063 Forest Fires 75.138 90.702 91.985 65.890 82.804 62.478 64.411 Housing 4.980 9.054 12.520 3.978 7.871 4.590 3.077 Wisconsin 41.059 34.710 41.220 32.054 38.495 40.700 23.659 datasets randomly and split them into training and test datasets in a rati o of60 ∶ 40 . We have repeated each of the experiments 10 times with diﬀerent randomly assigned training and test sets. Table 4 summarizes the results in terms of RMSE for the diﬀerent models. The ﬁrst important comment is that the proposed model improves over the original RT, ANN, RBFN, SVR models consisten tly across all data sets. Proposed RBNT model out- performs all the similar types of machine learning models for Auto MPG , Housing, and Wisconsin dataset and found competitive on Concrete and Forest Fires dataset as well. 6 CONCLUSIONS In this paper, we proposed a novel distribution-free hybrid regress ion model which is a hybridization of RT and RBFN model for improving waste recovery process in a paper company. The proposed mod el demonstrated the best performance and oﬀered a practical solution to the WRP problem of ﬁnding crucial process parameter s to improve recovery percentage of ﬁber ﬁller recovery equipment. The model can also be useful for future recovery predicti on of the process that may give the company an indication about the stability of the process. Using the recommendatio n of the model, we further designed a set of experiments to ﬁnd the optimal level of essential process parameters (as found in Sectio n 4.3) causing the FFRE eﬃciency. This resulted in improved waste recovery from an existing average recovery percentage of 60% to80% which added ﬁnancial beneﬁt to the paper company. Our proposed model takes into account the curse of high-dim ensional data and performs well on small or medium-sized datasets. Through computational experiments, we have sh own our proposed methodology performed quite well as compared to the other state-of-the-arts in the tissue paper dataset as well as ﬁve publicly available datasets. The proposed Chakraborty ET AL 13 RBNT model has the most desired statistical property, viz. universal cons istency. The usefulness and eﬀectiveness of the model lie in its robustness and easy interpretability as compared to complex “bl ack-box-like" models. An immediate future work of this paper is to investigate the applicability of the proposed model on spatio-temporal datasets involving regression tasks.