# Statistical Inference for Piecewise Affine System Identification

Authors: Hong Jianwang; Ricardo A. Ramirez-Mendoza; Xiang Yan
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

This short note studies the problem of piecewise affine system identification, being a special nonlinear system based on our previous contribution on it. Two different identification strategies are proposed to achieve our mission, such as centralized identification and distributed identification. More specifically, for centralized identification, the total observed input-output data are used to estimate all unknown parameter vectors simultaneously without any consideration on the classification process. But for distributed identification, after the whole observed input-output data are classified into their own right subregions, then part input-output data, belonging to the same subregion, are applied to estimate the unknown parameter vector. Whatever the centralized identification and distributed identification, the final decision is to determine the unknown parameter vector in one linear form, so the recursive least squares algorithm and its modified form with the dead zone are studied to deal with the statistical noise and bounded noise, respectively. Finally, one simulation example is used to compare the identification accuracy for our considered two identification strategies.


---

## Body

## 1. Introduction

Over the past few decades, the rapid evolution of computing, communication, and sensor technologies has brought about the proliferation of new dynamic systems, mostly technological and often highly complex. When to control these new dynamics systems, we must need some prior knowledge about them, i.e., their corresponding mathematical models or equations are firstly constructed for the latter controller design. The concept of the system and model is the basis of the engineering disciplines, where one is generally interested in a quantitative assessment of the behavior of a dynamical system. Therefore, it is necessary to obtain a mathematical description of it. Starting from the model, it is necessary to develop formal tools of analysis and synthesis. Basically, there are two ways of constructing mathematical models, i.e., mathematical modeling and system identification. More specifically, mathematical modeling is an analytic approach, and prior knowledge and physical insight about the considered system are used to describe the dynamic behavior of a system. System identification is an experimental approach, for which some experiments are performed on the considered system in order to learn a mathematical model from the collected input-output data by means of parameter estimation.The goal of system identification is to derive one appropriate mathematical model or equation for the considered unknown system; then, this constructed mathematical model is a reference basis for the next controller design. During these years, lots of research studies exist on linear system identification, but the linear system is one ideal case in practice, as we all know that all phenomena are nonlinear in nature. The research direction corresponds to the linear system identification; then, the obtained results for linear system identification are tried to extend to more general case-nonlinear system identification. The most difficulty for nonlinear system identification is about which nonlinear form is used to describe the considered nonlinear phenomenon. After extending the nonlinear phenomenon as one detailed expression, then all the obtained results about linear system identification are modified to be suited for nonlinear system identification. One more widely used way to extend the nonlinear phenomenon is that orthonormal function expansion or its generalized form. As the neural network can approximate the nonlinear system with low approximation error, this nice property of the neural network attracts lots of researchers in studying nonlinear system identification based on orthonormal function expansion and neural network. The detailed description based on nonlinear system identification is omitted due to space limitations. Our mission in above explanation is to show research on nonlinear system identification is worth, and it is more suited for practical phenomenon in nature.To apply the obtained or existed results on linear system identification to nonlinear system identification well, the nonlinear phenomenon is formulated as almost one special linear form, i.e., this special linear form has not only linear form but also some properties, coming from nonlinearity. This special linear form is our considered piecewise affine system here, which shows one nonlinear form in the whole region, but one linear form in each subregion. It is a trade-off between the linear system and nonlinear system. Furthermore, if we can determine which region it belongs to, then it reduces to linear system identification. It means all results on linear system identification are applied directly, such as identification algorithm, optimal input design, and model structure validation.Due to the trade-off for this piecewise affine system, it bridges the closed relation between linear system identification and nonlinear system identification, so the identification for this piecewise affine system is widely studied in these recent years. For example, one convex optimization strategy-mixed integer programming is introduced to identify the piecewise affine system in [1]. Nonlinear auto-regressive with exogenous input model [2] is an input-output description, often used in piecewise affine system identification. The current output is obtained by means of a nonlinear functional expansion of lagged inputs, which outputs elementary terms [3]. The piecewise affine system reduces the number of model terms and combinations of elementary terms, with respect to other parametric class, such as Volterra series [4]. This peculiarity is very important when the model structure is unknown since it allows to reduce the number of possible candidate structures [5], and the idea of piecewise affine means switching linear too [6]. The computational complexity on piecewise affine system identification [7] is analyzed from the point of complexity theory. Although the piecewise affine system could be in the principle done in the linear case, it is not feasible in the nonlinear one since the number of possible model terms increases rapidly with the number [8]. One recursive multiple least squares algorithm is proposed to identify piecewise affine regression [9], where multicategory discrimination is deemed for data clustering in case of stochastic noise, i.e., zero mean white Gaussian and mutually uncorrelated. Moreover, for unknown but bounded noise, existing in the piecewise affine system, a bounded error approach can achieve the identification goal in [10], where one guaranteed interval is yielded to include the parameter estimation, and the center of this guaranteed interval is chosen as the final estimation. Furthermore, other aspects are studied in detail for the piecewise affine system, such as its input-output representation [11], recursive identification algorithm [12], and statistical clustering technique [13]. The detailed review of piecewise affine system identification can be seen [14], where the similarity between our considered piecewise affine system and hybrid system is also analyzed. The classical observability and controllability of the linear system are extended for the piecewise affine system and hybrid system in [15], where observability and controllability analysis are two important factors for regulating the future state converge to its stable point. In the year, research on piecewise affine system identification are still done, for example, Federico Bianchi Valentina Breschi [16] develops a general framework to alternate between parameter estimation and sample mode assignment. In [17], given a set of input-output data, the identification of a switched nonlinear system for the underlying system involves the simultaneous identification of the mode sequence. Our previously published paper [18] proposes one zonotope parameter identification algorithm to identify the unknown parameter vector in each separated region in presence of unknown but bounded external noise. The identification idea can be combined with optimal control theory; then, the data-driven control is yielded [19]. The difference between system identification and data driven control is that the system identification is to identify the unknown plant from observed data, but the data-driven control is to apply the observed data to design the considered controller. As lots of information are included in observed data, the control task concerns on extracting these useful information from data [20]. As the number of published works on piecewise affine system identification is vast, we cannot list all of them here, but only main works in recent years. Le Quang et al. [21] investigate the identification of continuous piecewise affine systems in state space form with jointly unknown partition and subsystem matrices. A novel online fault detection and identification strategy was established for a class of continuous piecewise affine systems, namely, bimodal and trimodal piecewise affine systems in [22], where the recursive nature of the proposed scheme and the consideration of parametric uncertainties in both partitions and in subsystems parameters were proposed there.Based on abovementioned works and our previous contribution on piecewise affine system identification, this technical note continues to do it deep. As the nice property of the piecewise affine system is that the linear form holds during each subregion, which combines the whole considered region. It means the whole considered region is divided into many subregions, and in these subregions, the form is the special linear form. Based on this explanation on the piecewise affine system, the first step for piecewise affine system identification is to classify the observed input-output data into their own subregion, as the observed data are included in different subregion. After classifying all observed input-output data, then least squares algorithm can be applied to identify the unknown parameter vector in each subregion. More precisely, assume the whole considered region is divided intoM different subregions, i.e., the number of unknown parameter vectors is M. Our mission for piecewise affine system identification is to estimate these M parameter vectors by using the observed input-output data. The identification process of applying all observed input-output data to estimate all M unknown parameter vectors corresponds to previous centralized identification, which does not care any subregion without any classification. For this centralized identification, all unknown parameter vectors are obtained simultaneously. On the contrary, the other identification process of combining classification and parameter estimation is the distributed identification. For this distributed identification, firstly all observed input-output data are classified into their own subregion; then, secondly, the classified data are dealt with to estimate the unknown parameter vector in each subregion. The reason about why we study these two different identification strategy is that through comparing them, we find if the number of subregions is low, i.e., M=2or3, the previous centralized identification can give a nice identification results. But when M>3, we need the distributed identification to complete our given identification accuracy, and it means the centralized identification is useless for the case of M>3.The main contributions of this short note are listed as follows. (1) Two different identification strategies are proposed to identify the special piecewise affine system respectively, i.e., centralized identification and distributed identification. (2) Comparisons are given for these two identification strategies. (3) One novel least squares algorithm with the dead zone is proposed in the identification process. Generally, these two identification strategies and least squares algorithm with the dead zone can be widely applied in other research fields, such as signal processing, spectral estimation, and adaptive control.This short note is organized as follows. In Section2, the basic piecewise affine system is presented, and its identification problem is also given. In Section 3, the previous centralized identification is proposed to estimate all unknown parameter vectors simultaneously, through introducing one discrete variable and least squares algorithm. Furthermore, the distributed identification is given to classify the observed data and estimate the unknown parameter vectors step by step in Section 4. To complete the drawback of our previous work, one lease squares algorithm with the dead zone is proposed to consider the unknown but bounded noise. Section 5 gives one example to compare these above identification strategies and illustrate our proposed theory. Finally, conclusions and comments about next work are presented in Section 6.


## 2. Problem Statement

### 2.1. Piecewise Affine System

Consider the following linear form, existing in linear system identification many times as(1)yt=−∑i=1naaiyt−i+∑j=1nbbiut−j+et,where, in equation (1), ut and yt are the corresponding input and output signal at time instant t.Parametersaii=1na and bjj=1nb are the unknown parameters, which are needed to identify in the latter study. Two model orders na and nb are given or prior known. et is one external noise or disturb, and it may be one statistical noise or bounded noise. To apply the existing identification algorithm in estimating these two kinds of unknown parameters aii=1na and bjj=1nb, linear form (1) is always rewritten as one single linear regressor form, i.e.,(2)yt=ϕTtθ+et,where regressor vector ϕt and parameter vector θ are defined as follows:(3)ϕt=−yt−1,…,−yt−na,ut−1,…,ut−nb,θ=a1,…,ana,b1,…,bnb.In regressor vectorϕt, na and nb are also named as time delay.When to extend classical linear regressor form (2) into our considered piecewise affine system, we observe equation (2) is always time invariant within the whole region. Then, we divide the whole region into M different subregions, so in each subregion, parameter vector θ is not the same with each other. This property makes us reformulate the original linear regressor form (2) as the other piecewise affine form, i.e.,(4)yt=ϕTtθi+et,ϕt∈Ri,i=1,2,⋯M,where in equation (4), M is the number of subregions, i.e., R=R1∪R2∪⋯∪RM.Equation (4) means firstly we need to know which subregion the regressor vector ϕt belongs to; then, secondly, one approximated parameter vector θi is used to correspond it. Comparing the original linear regressor form (2) and its extended piecewise affine system (4), the computational complexity about the identification process is increased, as only one constant parameter vector θ exists in equation (2), but M different parameter vectors θii=1M are needed to be identified in equation (4).


### 2.2. Main Mission

ThoseM subregions Rii=1M constitute the whole considered region R. From a practical point of view, these M subregions Rii=1M are always assumed to be convex and connected, i.e., their closely relations are shown in Figure 1, where R=R1∪R2∪⋯∪R8 and Ri∩Rj=line  or  space. As all regions have one common equilibrium point, i.e., origin point, and this origin point means all elements in that regressor vector correspond to zero, i.e., ϕt=0, it is no useful in analyzing, so we always delete this special original point to let ϕt≠0.Figure 1
An example of separated subregions.From above description about the piecewise affine system, our mission is to estimate theseM different unknown parameter vectors θii=1M by using only the input-output data ut,ytt=1N, where N is the number of observed input-output data. Then, this short note proposes two identification strategies-centralized identification and distributed identification to combine the least squares algorithm.


## 2.1. Piecewise Affine System

Consider the following linear form, existing in linear system identification many times as(1)yt=−∑i=1naaiyt−i+∑j=1nbbiut−j+et,where, in equation (1), ut and yt are the corresponding input and output signal at time instant t.Parametersaii=1na and bjj=1nb are the unknown parameters, which are needed to identify in the latter study. Two model orders na and nb are given or prior known. et is one external noise or disturb, and it may be one statistical noise or bounded noise. To apply the existing identification algorithm in estimating these two kinds of unknown parameters aii=1na and bjj=1nb, linear form (1) is always rewritten as one single linear regressor form, i.e.,(2)yt=ϕTtθ+et,where regressor vector ϕt and parameter vector θ are defined as follows:(3)ϕt=−yt−1,…,−yt−na,ut−1,…,ut−nb,θ=a1,…,ana,b1,…,bnb.In regressor vectorϕt, na and nb are also named as time delay.When to extend classical linear regressor form (2) into our considered piecewise affine system, we observe equation (2) is always time invariant within the whole region. Then, we divide the whole region into M different subregions, so in each subregion, parameter vector θ is not the same with each other. This property makes us reformulate the original linear regressor form (2) as the other piecewise affine form, i.e.,(4)yt=ϕTtθi+et,ϕt∈Ri,i=1,2,⋯M,where in equation (4), M is the number of subregions, i.e., R=R1∪R2∪⋯∪RM.Equation (4) means firstly we need to know which subregion the regressor vector ϕt belongs to; then, secondly, one approximated parameter vector θi is used to correspond it. Comparing the original linear regressor form (2) and its extended piecewise affine system (4), the computational complexity about the identification process is increased, as only one constant parameter vector θ exists in equation (2), but M different parameter vectors θii=1M are needed to be identified in equation (4).


## 2.2. Main Mission

ThoseM subregions Rii=1M constitute the whole considered region R. From a practical point of view, these M subregions Rii=1M are always assumed to be convex and connected, i.e., their closely relations are shown in Figure 1, where R=R1∪R2∪⋯∪R8 and Ri∩Rj=line  or  space. As all regions have one common equilibrium point, i.e., origin point, and this origin point means all elements in that regressor vector correspond to zero, i.e., ϕt=0, it is no useful in analyzing, so we always delete this special original point to let ϕt≠0.Figure 1
An example of separated subregions.From above description about the piecewise affine system, our mission is to estimate theseM different unknown parameter vectors θii=1M by using only the input-output data ut,ytt=1N, where N is the number of observed input-output data. Then, this short note proposes two identification strategies-centralized identification and distributed identification to combine the least squares algorithm.


## 3. Centralized Identification

Observing that piecewise affine system (4), we must know which subregion the regressor variable belongs to; then, its corresponding unknown parameter vector is used. The above process means one data clustering process in machine learning. Here, one centralized identification is introduced without the above data clustering process, i.e., all observed input-output data are used to estimate those unknown parameter vectors θii=1M simultaneously.Introduce one discrete variableδi∈0,1, satisfying(5)δi=0,ifϕt∉Ri,1,ifϕt∈Ri,using this defined discrete variable δi∈0,1; then, equation (4) is rewritten as follows:(6)yt=∑i=1MδiϕTtθi+et=δ1ϕTtθ1+et+δ2ϕTtθ2+et+⋯+δMϕTtθM+et.Reformulating equation (6) as follows:(7)yt=δ1ϕtδ2ϕt⋯δMϕtθ1θ2⋮θM+∑i=1Mδiet=δ1δ2⋯δMϕtθ1θ2⋮θM+et=ϕ1Ttθ+et,where the new ϕ1t and θ are defined as follows:(8)ϕ1Tt=δ1δ2⋯δMϕt,θ=θ1θ2⋮θM.Based on our defined discrete variableδi∈0,1, then piecewise affine system (4) can be also reformulated as one linear regressor form (7). The unknown parameter vector θ, existing in equation (7), can be obtained easily through the classical least squares algorithm, i.e., the parameter estimation θ^ is given as that(9)θ^=∑t=1Nϕ1Ttϕ1t−1∑t=1Nϕ1Ttyt,where N denotes the number of observed input-output data. Furthermore, to increase the computational speed, its recursive least squares algorithm is that(10)P−1t=∑t=1Nϕ1Ttϕ1t,Lt=Pt−1ϕ1t1+ϕ1TtPt−1ϕ1t,Pt=Pt−1−Pt−1ϕ1Ttϕ1tPt−11+ϕ1TtPt−1ϕ1t,θ^t=θ^t−1+Ltyt−ϕ1Ttθ^t−1,where θ^t and θ^t−1 are the parameter estimations at time instant t and t−1, respectively.To start the above recursive least squares algorithm, the initial parameter valueθ^0 can be chosen as(11)θ^0=12I,where I is a column vector with element 1.From equation (8), we see that the computational complexity of this centralized identification is vast, as the dimensions of the regressor and parameter vector is Mn (n is the dimension of each parameter vector θi), so if M and n are all small, then this centralized identification holds.


## 4. Distributed Identification

Distributed identification is different from centralized identification, as the observed input-output data are used to estimate each unknown parameter vectorθi, not the total unknown parameter vector θ=θ1,θ2,…,θM. But one obvious problem appears, it is about which parameter vector θi is determined by the observed input-output data, and we must be sure that the observed input-output data, used to estimate that unknown parameter vector θi, belong to that corresponding subregion Ri, so firstly one additional process is added here.Let(12)zt=yt,utT=yt,−yt−1,…,−yt−na,ut−1,…,ut−nbT.When given one observed dataztt=1N, we need to classify it in its right subregion, through computing its projection on each subregion or set, i.e.,(13)distzt,Ri=infzitzt−zitzit∈Ri,i=1,2,⋯M,where  denotes one norm.Identify its minimal component and treat this index of this observed data as our guess from the number of the class to whichzt belongs, i.e.,(14)Ri=argminiz1t,z2t,…,zMt.Then, we regard the following fact:(15)zt∈Ri.By the way the idea case is one index, satisfyingzt=z1t, it means(16)distzt,Ri=infzitzt−zitzit∈Ri=0.After we divide all observed input-output data into their right subregions, then we describe the considered piecewise affine system as follows:(17)yt=ϕTtθ1+et,ϕt∈R1,ztt=1N1,yt=ϕTtθ2+et,ϕt∈R2,ztt=N1+1N2,⋮yt=ϕTtθM+et,ϕt∈RM,ztt=NM+1N,where number N satisfies(18)N=N1+N2+⋯+NM.Based on above clustering or classifying process, then some observed input-output data, belonging to its corresponding subregion, are applied to estimate only one corresponding unknown parameter vector. For notational convergence only, we use the observed input-output dataztt=1N1 to estimate the unknown parameter vector θ1. It is rewritten as follows:(19)yt=ϕTtθ1+et,ϕt∈R1,ztt=1N1.Least squares algorithm (9) and its recursive form (10) are applied to identify the unknown parameter vector θ1.From the existing results in linear system identification, the identification result is nice in case of noiseet. It means when external noise et is a white noise, then the parameter estimation θ^1 is unbiased and consistent. But this nice result does not hold for bounded noise. Our previously work [18] proposes zonotope parameter identification in case of bounded noise. As knowledge of zonotope is little for some readers, so here we modify the recursive least squares algorithm (10) into one new least squares algorithm with the dead zone.As that external noiseet is bounded, without loss of generality, assume that supet≤Δ, where Δ is one upper bound. This new least squares algorithm with the dead zone is listed as follows:(20)θ^1t=θ^1t−1+atϕtc+ϕTtϕtyt−ϕTtθ^1t−1,where coefficient c>0 and(21)at=1,yt−ϕTtθ^1t−1>2Δ,0,otherwise.The above iterative algorithm is terminated byat. After subtracting the true parameter vector θ1 on both sides of equation (20), we have(22)θ˜1t=θ^1t−θ1,θ˜1t=θ˜1t−1−atϕtc+ϕTtϕtϕTtθ˜1t−1+et,where θ˜1t and θ˜1t−1 are parameter errors at time instant t and t−1. Due to the definition of at, it holds that(23)θ˜1t2=θ˜1t−12−2atwt−etwtc+ϕTtϕt+a2tϕTtϕtw2tc+ϕTtϕt2≤θ˜1t−12+atc+ϕTtϕt2wtet−atw2tc+ϕTtϕt,where in equation (23), wt is one model error, i.e.,(24)wt=yt−ϕTtθ^1t−1=−ϕTtθ˜1t−1+et,From an applied point of view, for anyt, we have(25)θ˜1t2=θ˜1t−12−atw2tc+ϕTtϕt+atϕTtϕtw2tc+ϕTtϕt2w2t2+2w2t≤θ˜1t−12−12atw2tc+ϕTtϕt+2atΔ2c+ϕTtϕt.Based on our above mathematical derivation, we seeθ˜1t2 is a decreasing sequence, and it also satisfies that(26)θ^1t−θ1≤θ^1t−1−θ1≤⋯≤θ^10−θ1t≥1.It means the parameter estimationθ^1t, obtained by the new least squares algorithm with the dead zone, converges to its true parameter value θ1 with the iterative step increases. The parameter error is one decreasing sequence, not related with the upper bound Δ.


## 5. Simulation Example

In this simulation part, one example is used to prove the efficiency of our proposed theories in this short note. The piecewise affine system is defined as follows:(27)yt=ϕTtθi+et=et+0.35utut∈0,150.25utut∈15,300.18utut∈30,50.It means(28)M=3,R1=0,15,R2=15,30,R3=30,50ϕt=ut,θ1=0.35,θ2=0.25,θ3=0.18,or(29)θ=θ1,θ2,θ3=0.35,0.25,0.18.Firstly, the centralized identification strategy is studied to estimate the parameter vectorθ=0.35,0.25,0.18 simultaneously through the observed input-output data. Sinusoidal signal is used to excite the above piecewise affine system, and Figure 2 shows our chosen sinusoidal signal. We measure the corresponding output data by some physical devices or sensors; then, the observed output signal is plotted in Figure 3. Combining our observed input-output data in Figures 2 and 3, three unknown parameters θ=0.35,0.25,0.18 are identified by the recursive least squares algorithm in case of a white noise, and the corresponding identification result for the centralized identification strategy is shown in Figure 4, where the true parameters and their estimations correspond to each other. It tells us when the number of subregions is 3, the identification result for the centralized identification strategy is tolerable.Figure 2
Input signal.Figure 3
Output signal.Figure 4
Identification result for two identification strategies.Secondly, an additional subregion is added in equation (24), i.e.,(30)yt=ϕTtθi+et=et+0.35utut∈0,150.25utut∈15,300.18utut∈30,500.5utut∈50,70.The observed input-output data are also seen in Figures2 and 3, where the number of subregion is 4. To compare our mentioned two identification strategies for estimating these four unknown parameters, we simulate them in the framework of bounded noise, i.e., et≤1. For this case of bounded noise in the piecewise affine system with four subregions, the least squares algorithm for centralized identification and least squares algorithm with the dead zone for distributed identification are all considered to estimate those four unknown parameters. We do not plot the parameter estimations; just like in Figure 4, here we substitute the parameter estimations into the original piecewise affine system to get the identified model, and the original system is named as the true model. Two identification results are plotted in Figure 4, where the black dotted line and red line are for centralization identification and distributed identification, respectively. Their output responses correspond to the true output and identified output. Figure 5 and 6 show the comparisons between the true output and identified output for centralized identification and distributed identification, respectively.Figure 5
Comparisons of the output for centralized identification.Figure 6
Comparisons of the output for distributed identification.More specifically in Figure5, the black curve is the true output and red curve is the identified output from centralized identification. Then, we see the deviation error between these two curves is not accepted due to the large deviation error. But for the distributed identification in Figure 6, the blue curve denotes the identified output. Then, these two curves are very closely to each other, it means their deviation error is little, and the parameter estimations converge to their corresponding true values. Furthermore, from Figures 5 and 6, if the number of subregions does not exceed to 3, then centralized identification is efficient. But on the contrary, it is better to apply that distributed identification to complete our identification task.


## 6. Conclusion

As the piecewise affine system is one trade-off between the linear system and nonlinear system, further analysis on the piecewise affine system is considered here. From the point of knowledge about classification process or not, centralized identification and distributed identification are studied in detail. To deal with statistical noise and bounded noise, the recursive least squares algorithm and its modified form with the dead zone are applied to estimate the unknown parameter vector. As this short note is our continuous contribution on piecewise affine system identification, the detailed computational complexity of our considered two identification strategies is our next ongoing work.



---

*Source: 4618030-2021-10-05.xml*