# On the Relation between NARX Clusters and Even/Odd Nonlinearities through Frequency-Domain Analysis

Authors: Alexandro G. Brito; Elder M. Hemerly; Waldemar C. Leite Filho
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

Although polynomial NARX models have been intensively used in nonlinear system identification, few papers discussed how to relate the inner nonlinearities to specific types of clusters and regressors. The objective of this paper is to discuss this relationship for a class of systems that contain even or odd nonlinearities. This class covers block-structured models (Hammerstein, Wiener, and others) and systems with dynamic nonlinearities. To achieve the paper’s aim, a deep frequency-domain analysis is performed. For each type of nonlinearity, all the NARX clusters are investigated and the results show that each regressor type provides specific nonlinear contribution. The investigation is based on an output power spectra analysis when a specific multisinusoidal excitation is applied. According to the spectral contributions in some of the frequency lines, the nonlinearity classification is possible. By applying the same procedure to the clusters, one interprets how these clusters can (or not) contribute to explain the system nonlinearity. The paper findings have two major impacts: (i) one gains deep knowledge on how the nonlinearities are coded by the clusters, and (ii) this information can be used, for instance, to aid a structure selection procedure (ERR, term clustering, etc.) during the discarding of the clusters which are not able to explain the system nonlinear behavior. Some practical and experimental aspects are discussed, while numerical examples are presented to show the validity of the theoretical analysis.


---

## Body

## 1. Introduction

The system identification theory is really vast, including advances in time as in frequency domain [1–3]. For nonlinear systems, an important question is how to determine a proper structure to be used during the modeling procedure. Among the several model structures that can be used to represent nonlinear phenomena, the NARX/NARMAX representation [4, 5] has been intensively used due to its adequacy for a wide range of applications. However, the NARX/NARMAX model efficiency is related to a proper model size and overparametrization problems were reported [6]. Thus, a previous model structure determination plays key role for a successful identification procedure and many methodologies are available for the NARX/NARMAX case [7–9]. Nowadays, there are robust numerical methods to perform the NARX structure selection [10, 11].The technique known asterm clustering has been extensively used as an auxiliary method during the structure selection of NARX/NARMAX models, providing useful information about the group of regressors that should be included into the model. It was demonstrated that a model containing spurious clusters can present dynamic behavior considerably different to the real system [12]. Additionally, it was demonstrated that certain classes of nonlinearities can be related to specific group of regressors calledclusters in term clustering context [13–15].This paper aims to analyze some other aspects concerning the relation between the term clusters and the inner system nonlinearities. Differently from the discussions in the literature, the proposal here is to perform such analysis by using arguments from the frequency domain. Although there are some studies involving generalized frequency response analysis of nonlinear systems in the literature [16, 17], the approach discussed in this paper is sufficient to provide the necessary knowledge about the mechanisms relating the term clusters to nonlinearities by using a simple spectral analysis. This not only reinforces some conclusions already obtained in the time domain, but also brings new insights about the model structure problem. Secondly, it is demonstrated that a frequency-domain analysis can be useful to detect, classify, and relate some other nonlinearity types to specific term clusters, thereby providing some additional information during a further structure selection procedure. Similar ideas were used to develop a structure selection approach for NARMAX models [18] but new results involving the relation between term clusters and spectral analysis are presented here.The paper is organized as follows. Section2 provides the background concerning the NARX models and the term clustering. In Section 3 the mathematical concepts related to model analysis in the frequency domain are introduced, and a methodology for nonlinearity classification is discussed. These ideas are used in Section 4 where the clusters are related to each type of nonlinearity that they can represent. Finally, Section 5 discusses some numerical results that illustrate the proposed concept, while some conclusions and remarks are drawn in Section 6.


## 2. NARX Models and Term Clustering

As was discussed by Billings et. al. [4, 5], a wide class of nonlinear systems can be expressed as a nonlinear discrete polynomial related to actual and previous values from the input and output if mild assumptions are considered. In this case, a model can be written as
(1)yk=∑m=0l‍∑p=0m‍∑n1,nmny,nu‍cp,m-p(n1,…,nm)×∏i=1p‍y(k-ni)∏i=p+1m‍u(k-ni),
where
(2)∑n1,nmny,nu‍≡∑n1=1ny‍⋯∑nm=1nu‍,
which is a general formulation borrowed from [12], representing the Nonlinear AutoRegressive with eXogenous inputs (NARX) models. The most important characteristic of this formulation is that it is linear in the parameters cp,m-p.The structure selection procedure consists in choosing the most relevant terms of (1)—also denoted asregressors—to take part into the final model, so that the overall nonlinear behavior is properly represented. Then, arelevant regressor is denoted as the one which represents the output in a better way than another, based on some criterion. Several criteria can be used for this purpose and one commonly adopted is to rank a determined regressor according to its ability to “explain” the output data when compared to another one. This approach is the basis of theerror reduction ratio (ERR) [7]. This ratio is based on the full orthogonalization of the regression matrix, so that the individual importance of each regressor can be evaluated.Theterm clustering, which analyzes the identified parameters for regressors with similar format, is another auxiliary strategy for regressor set determination [12, 19]. For the purpose of presentation, suppose that the sampling time is sufficiently short so that y(k-1)≈y(k-2)≈⋯≈y(k-ny) and u(k-1)≈u(k-2)≈⋯≈u(k-nu). Then, (1) can be rewritten as
(3)yk≈∑n1,nmny,nu‍cp,m-p(n1,…,nm)×∑m=0l‍∑p=0m‍yk-1puk-1m-p.
The constants ∑n1,nmny,nu‍cp,m-p(n1,…,nm) are the coefficients of theterm clustersΩyp,um-p, which contain terms of the form y(k-i)pu(k-j)m-p for m=0,…,l and p=0,…,m. These coefficients are calledcluster coefficients, represented as Σypum-p [12]. It was demonstrated that terms of a same cluster explain the same type of nonlinearity [15]. In addition, the cluster coefficient behavior can be used to detect spurious clusters which should be discarded during the model structure selection [12, 15].


## 3. System Analysis in the Frequency Domain

The problem of nonlinearity detection has been deeply studied in the literature. In the frequency domain context, most of the methodologies are based on specific multisinusoidal excitations, due to their nice properties in relating the power spectrum to the type of nonlinearity in the system. A class of excitation commonly used for this task is theodd-odd multisine signal. More precisely, let u(k) be a discrete periodic input with fundamental frequency ω0 and a sampling frequency ωs=Nuω0, where Nu is the data length. Denote by I={l∣l=1,3,5,…,N<Nu/2} the set of all odd numbers up to N and F⊂I a subset with some of these odd numbers. Then, the odd-odd multisine signal (OOMS) is defined as
(4)u(k)=∑l∈F‍Alsin2πkωslω0+ϕl,ϕl~U-π,π,
where U(r) denotes a uniformly distributed random number within the specified range r. In the frequency domain, this signal is characterized by an impulse train at specific frequencies harmonic to the fundamental ω0. These frequencies are defined by the set I where only some odd numbers are available and the result is a signal which excites only specific odd harmonics of such fundamental frequency. Notice that u(k) is a deterministic signal. This is possible because ϕl, once chosen by a random law, is maintained constant along the experiment. This signal is an odd function. Another important fact for nonlinearity detection is that an integer number of signal periods—a function of ω0—should be used. Additional details about the signal design for application in identification and nonlinearity detection are richly discussed in the literature [3, 20, 21].The nonlinearity detection is based on the behavior of nonlinear systems when a multisinusoidal input is applied [21]. Consider an input-output system and an odd-odd multisine signal input. Denote theexcited frequency lines as ωe={lω0∣l∈F} which correspond to those harmonics effectively excited by the input. Define the set M={m∣m=N,N+2,…,2N-1} which represents the odd numbers from N to 2N-1 and denote theodd non-excited frequency lines as ωi={lω0∣l∈(F~∪M),F~∪F=I}. These points represent the odd harmonics of ω0 whichare not excited by the input. Finally, denote theeven nonexcited frequency lines as ωp={lω0∣l=0,2,…,2N} which represent the even harmonics of ω0. Then, the system output satisfies one of the following conditions:(i)
pure linear system: there exists output power only at the excited frequency lines ωe;(ii)
system containing only even nonlinearities: there exists output power at some even nonexcited frequency lines ωp and, possibly, at the excited frequency lines ωe;(iii)
system containing only odd nonlinearities: there exists output power at the excited frequency lines ωe and at some odd nonexcited frequency lines ωi;(iv)
system containing both odd and even nonlinearities: there exists output power at the excited frequency lines ωe, at some odd nonexcited frequency lines ωi, and at some even nonexcited frequency lines ωp.Based on (i)–(iv), the nonlinearity detection is performed as follows. Firstly, an odd-odd multisinusoidal input is applied and the output signal is acquired. The next step is to calculate and analyze the Fourier transform of this signal. If the system contains power spectrum only at the excited lines, it is linear. If the system also exhibits power spectra at the odd nonexcited frequency lines, then it contains some odd nonlinearity such as cubic terms and saturation. Finally, if it has power spectrum at the even nonexcited frequency lines, the system contains some even nonlinearity such as quadratic terms and absolute-value function.In Section4, these results are used to analyze and code the relation between some term clusters and system inner nonlinearities.


## 4. Cluster Interpretation

Some works were successful in relating a term cluster to the specific type of nonlinearity that it can represent [13–15]. The conclusions are helpful in choosing a group of term cluster to take part into the model according to the inner system behavior.This paper proposes a different approach. Consider a black-box nonlinear system, in which only the input and output signals are available. As already discussed, one can infer the inner nonlinearities by performing a preliminary output spectra analysis. If in addition, one analyzes the spectral content of the regressors formed through the input-output signals, it is possible to infer if such regressors can take part into a NARX model for the system. The idea is to analyze the spectral power amplitude at the excited (ωe), the odd nonexcited (ωi), and the even nonexcited (ωp) for the output signal and the regressors so that any conclusion can be drawn.This paper deals with systems which the nonlinearity can be classified as an even or an odd function (static or dynamic). Consider a nonlinear system with a specific nonlinearity type (even or odd). By performing the proposed output spectral analysis, one can know the involved nonlinearity and at which nonexcited frequency lines the power is expected to occur. Suppose a specific regressor (e.g.,u(k-1)y(k-1)) is constructed through the available input and output signals. There are three possibilities:(i)
if the spectral analysis for this regressor reveals power spectra exclusively at the same type of nonexcited frequency lines from the output analysis, then there is a possibility that such regressor takes part in the NARX model;(ii)
if the regressor spectral analysis reveals power spectra at both types of nonexcited frequency lines (ωp and ωi), nothing can be assured about its presence into NARX model;(iii)
if the regressor spectral analysis reveals power spectra exclusively at different type of nonexcited frequency lines from the output analysis, then such regressor cannot take part into the NARX model and it can be discarded. Then, the analysis uses an exclusion argument. For example, a nonlinear system with an even nonlinearity has spectral power at ωp. If a specific regressor provides power only at ωp, then such regressor is a candidate to appear into the NARX model. If the regressor provides power at ωp and ωi, nothing can be assured. The reason is that, in spite of the presence of unexpected ωi lines, the interplay with other regressors can result in a final output spectral content only at ωp. However, if the regressor analysis reveals power spectra only at ωi, such regressor is certainly spurious and it should be excluded. The reason is because it is trying to explain an inexistent system nonlinear behavior.Denote a system containing only even nonlinearities as LE (Linear + Even) and a system containing only odd nonlinearities as LO (Linear + Odd). To obtain the conclusions, each type of regressor should be analyzed considering the involved nonlinearity. This procedure is performed as follows.


### 4.1. Input Related Clusters

The input related cluster analysis is firstly performed because it does not depend on the output characteristics, and the conclusions are valid for any nonlinearity type within the system. Moreover, this case enables an easier demonstration of the mathematical tools concerning the frequency-domain analysis.Consider initially the regressoru(k-τ1)u(k-τ2) from cluster Ωu2 and yΩu2 the contribution that such cluster can provide to the model output. By applying discrete Fourier transform and its properties, one can straightforwardly derive that
(5)YΩu2ejω=Fuk-τ1uk-τ2=e-jωτ1+τ22πUejω*Uejω.
Then, |YΩu2|∝|U*U| (the Fourier transform argument ejω is dropped for notation simplification). As the input is an impulse train at the frequencies ωe, the cluster provides spectral contributions at the frequency lines given by the convolution of the input harmonics. Without loss of generality, suppose the odd signal U(k)=A, k≡ejω, k=1,5,7,9,… where U (k=3), is purposely null. The unilateral convolution U*U is expressed by
(6)YΩu2(k)=U(k)*U(k)=∑j=0k‍U(j)U(k-j).
By evaluating (6), one can easily notice that YΩu2(k)≠0 only for k=2,6,8,…. In other words, YΩu2(k)≠0 only for even values of k or at the “even frequency lines” in frequency-domain context. Observe that such even points are not excited in U(k). Hence, YΩu2(k) only have contributions at theeven nonexcited frequency lines. This proves that the cluster Ωu2 will provide spectral power only at the even (and nonexcited) frequency lines ωp. Notice that similar conclusion can be obtained for any cluster Ωum with m being an even number.Consider now the regressoru(k-τ1)u(k-τ2)u(k-τ3) from cluster Ωu3. In this case |YΩu3|∝|U*U*U|=|U*YΩu2|. Based on the input characteristics and the spectral content of YΩu2, one can conclude that YΩu3 will contain power at the excited (ωe) and some odd nonexcited (ωi) frequency lines. There is no power at ωp. To show this, let us suppose the signal U(k) above once again. By evaluating the correlation
(7)YΩu3(k)=U(k)*YΩu2(k)=∑j=0k‍U(j)YΩu2(k-j)
it is straightforward to conclude that YΩu3(k)≠0 only for k=3,5,7,…. Notice that k are only odd numbers and one of them was not excited by the original signal (k=3). This proves that the cluster Ωu3 will provide spectral power only at the odd excited ωe and nonexcited ωi frequency lines. The same conclusion can be extended for any cluster Ωum with m being an odd number.Similar analysis can be extended to any cluster type, to exhibit its spectral content. However, such procedure would be quite tedious. In Sections4.2–4.4, the spectra content of each cluster type will be summarized by following the same arguments from this section.


### 4.2. LO System

The LO system is characterized by the occurrence of excited (ωe) and odd nonexcited (ωi) frequency lines at the output, when an odd-odd multisine input is applied. This means that a polynomial NARX model should contain regressors which provide power spectrum only at these frequency lines. In other words, a regressor which provides power spectrum only at ωp could not take part into the NARX model because it explains a type of nonlinearity which does not exist in the system. Then, one can infer the unnecessary clusters through an output spectral analysis. Such analysis follows for each type of clusters.Ω
u
m. Based on the discussion in Section 4.1 and the output spectral content of a LO system, it is easy to conclude that the cluster Ωum, with m being an odd number, can appear at NARX model. In other hand, regressors from Ωum with m being an even number are not expected.Ω
y
m. Firstly, consider the regressor y(k-τ1)y(k-τ2) (Ωy2). In this case |YΩy2|∝|Y*Y|. The LO system output contains power only at the odd frequency lines. The consequence is that the convolution above would produce power spectrum only at the even nonexcited frequency lines ωp. However, these frequencies are not possible for such system. The conclusion is that the cluster Ωy2 cannot appear into the polynomial NARX model of a LO system. It is straightforward to show the same for any cluster Ωym with m being an even number. Conversely, the regressor y(k-τ1)y(k-τ2)y(k-τ3) (cluster Ωy3) is characterized by |YΩy3|∝|Y*YΩy2|, which provides spectral contribution at the odd frequency lines (ωe and ωi). Then, this cluster—and any other Ωym with m being an odd number—can take part into the NARX model.Ω
u
r
y
s. Consider the regressor u(k-τ1)y(k-τ2) (cluster Ωuy). It implies |YΩuy|∝|U*Y|. In this case, as the input and the output are odd functions, presenting spectral power only at odd frequency lines. Then, the convolution would produce contributions only at even frequency lines, but this is impossible for a LO system. Similar conclusion can be drawn for any cluster Ωurys where r+s is an even number. All these clusters can be discarded from the polynomial NARX model of a LO system. The regressor u(k-τ1)y2(k-τ2) (cluster Ωuy2) produces |YΩuy2|∝|U*YΩy2|. In this case, there is the convolution between an odd function (input) and the even function y2(k-τ2) (due to the output odd aspect). This will produce contributions only at odd frequency lines (excited and/or nonexcited). Then, such cluster is possible for a LO system. The same holds for any cluster Ωurys where r+s is an odd number.


### 4.3. LE System

The LE system is characterized by the occurrence of excited (ωe) and even nonexcited (ωp) frequency lines at the output, when an odd-odd multisine input is applied. The cluster analysis is identical to those performed in Section 4.2; however, the results are quite different.Ω
u
m. Only the cluster Ωum with m being an even number is possible to appear in the NARX model, due to the output content of a LE system. In this case, Ωum with m being an odd number is not possible because it produces contributions at the odd nonexcited frequency lines ωi.Ω
y
m
and
Ω
u
r
y
s. Nothing can be assured about these clusters. Their regressors can produce contributions not only at ωp, but also at ωi. This is because the output is a not-even-nor-odd function, and any regressor formed with the output is also a not-even-nor-odd function. The final output contribution at ωi can vanished by the interplay of different output-related regressors and, because of this, any conclusion can be drawn with basis on the spectral analysis. For instance, the regressor u(k-τ1)y2(k-τ2) (Ωuy2) produces contributions given by the convolution of an odd function (input) and y2(k-τ2). Notice that this is a not-even-nor-odd function, since the output has both odd and even frequency lines. The result is a signal which—in addition to ωp and ωe—also contains odd nonexcited frequency lines ωi. These nonexcited odd frequency lines can be—in principle—canceled by the interplay with other regressors, resulting in output power only at ωp (and, eventually, at ωe). Because of this, it is not possible to assure the presence (or not) of such regressor.


### 4.4. Block-Oriented Systems with a Single Nonlinearity Type

Consider the following block-oriented models in Figure1, f(·) being a static nonlinearity and H(z) a linear system.Block-oriented models: (a) Hammerstein system and (b) Wiener system.
(a)
(b)For the Hammerstein system (Figure1(a)), the inner signal w(k) is just the nonlinearity applied to the input. Only nonlinear regressors related to the input are possible because the nonlinearity only affects this signal. Moreover, only terms with the same lag are present (e.g., um(k-τ1)) since a static nonlinearity does not affect the input lags. A term such as um(k-τ1)un(k-τ2), τ1≠τ2 is impossible.Iff(·) is an even function, W(ejω) will have frequency lines only at ωp for an odd-odd multisine input. The same will occur for the output Y(ejω) since it is the result of w(k) applied to a linear system (H(z)). Hence, only the nonlinear cluster Ωum with m being an even number—and with terms of identical lags—can be present in NARX models for Hammerstein systems with an even nonlinearity. On the other hand, if f(·) is an odd function; Y(ejω) will have frequency lines at ωe and ωi. By following the same arguments above, the NARX model can contain only terms from cluster Ωum with m being an odd number—and with terms of identical lags. Notice that these results are in perfect accordance with those in [13], where a different eigenvalue analysis was applied to Hammerstein models. The merit here is that one can discard some clusters of type Ωum which are not able to represent the actual inner nonlinearity.For the Wiener system (Figure1(b)), the analysis is more involving. The literature already showed that NARX regressors containing terms on ym(k-τ), for all m are not possible [13]. The reason is that such regressors yield a model with output multiplicity, which is impossible for a NARX polynomial model of Wiener systems.Because the nonlinearity is located after the linear block, it is applied to bothu(k-τu) and w(k-τw) (τu and τw determined by the linear transfer function). The consequence is that some nonlinear clusters involving output terms are expected to appear in a NARX model. Technically, the input terms from Ωum can contain different lags since the linear block can introduce lagged input terms in w(k). To explore such affirmation, consider that f is a square function and
(8)H(z)=W(z)U(z)=1+a1z-1+a2z-21+b1z-1+b2z-2.
Hence, the output can be written as
(9)yk=w2k=∑l=12‍∑m=12‍blbmw(k-l)w(k-m)+∑l=12‍∑m=12‍2albmu(k-l)w(k-m)+∑l=12‍∑m=12‍alamu(k-l)u(k-m).
Notice that the last summation in (9) has the product of different lagged terms from the input. The conclusion is that the polynomial NARX model for a Wiener system can have some terms from cluster Ωum with different lags.Concerning the nonlinearity analysis, consider a Wiener system with an even static function. In this case,Y(ejω) will have frequency lines only at ωp because it is the consequence of an odd signal applied to an even nonlinearity. Similarly to the analysis of the Hammerstein case, the cluster Ωum with m being an even number is possible to occur in such Wiener system. Now, consider the cluster Ωu2y and its output contribution |YΩu2y|∝|YΩu2*Y|. Notice that both YΩu2 and Y only contain power spectrum at even nonexcited frequency lines ωp. As a consequence, YΩu2y also contain power spectrum only at ωp. Hence, the cluster Ωu2y is possible. Similar analysis shows that any cluster Ωumy with m being an even number is possible for Wiener systems with even nonlinearity, in accordance with [13]. Finally, the cluster Ωumy with m being an odd number results in power only at ωe and ωi. However, power is expected only at ωp for even Wiener systems. Hence, this cluster can be discarded.Similar analysis can be applied to Wiener systems with an odd nonlinearity. The conclusion is that the clustersΩum with m being an odd number and Ωury, r=0,2,4,… can take part into the NARX model, because they produce output power spectrum only at the odd frequency lines (ωe and ωi). Regressors from the cluster Ωum with m being an even number should be discarded. The regressors from the cluster Ωury, r=1,3,5,… can be discarded.


### 4.5. Summary

Based on specific multisine excitation and output frequency-domain analysis, several conclusions were drawn in Sections4.1–4.4 related to the clusters of a NARX model. Such results are conveniently summarized in Table 1.Table 1
Summary of the results.
Cluster
System type
LO
LE
H
W
O
E
O
E
Ω
u
m, m even
×
P
×
P
*
×
P
Ω
u
m, m odd
P
×
P
*
×
P
×
Ω
y
m, m even
×
?
×
×
×
×
Ω
y
m, m odd
P
?
×
×
×
×
Ω
u
r
y
s, r+s even
×
?
×
×
×
×
Ω
u
r
y
s, r+s odd
P
?
×
×
P
†
P
†
H—Hammerstein, W—Wiener, O—odd, and E—even. LE and LO according to Section4. P implies that such cluster can appear in a polynomial NARX model, in contrast to ×. The symbol ? denotes that nothing can be assured about the cluster. P* indicates that only terms with the same lag are possible. P† implies s≡1.Such analysis is not able to decide whether a possible cluster (marked asP in Table 1) is really useful for the model or not. For instance, the clusters Ωum with m even (Ωu2,Ωu4,…) do not present the necessary conditions to be present in a LO system model. In consequence, they should be automatically discarded from the candidate set. On the other hand, the cluster Ωu3can be present in the LO model. Further analysis is still necessary to conclude if such cluster is really necessary or not.In conclusion, the proposed methodology does not assign if a cluster marked asP in Table 1 is spurious or not. What is possible to assure is that those clusters marked as × are surely spurious and they should be discarded from the candidate set.


## 4.1. Input Related Clusters

The input related cluster analysis is firstly performed because it does not depend on the output characteristics, and the conclusions are valid for any nonlinearity type within the system. Moreover, this case enables an easier demonstration of the mathematical tools concerning the frequency-domain analysis.Consider initially the regressoru(k-τ1)u(k-τ2) from cluster Ωu2 and yΩu2 the contribution that such cluster can provide to the model output. By applying discrete Fourier transform and its properties, one can straightforwardly derive that
(5)YΩu2ejω=Fuk-τ1uk-τ2=e-jωτ1+τ22πUejω*Uejω.
Then, |YΩu2|∝|U*U| (the Fourier transform argument ejω is dropped for notation simplification). As the input is an impulse train at the frequencies ωe, the cluster provides spectral contributions at the frequency lines given by the convolution of the input harmonics. Without loss of generality, suppose the odd signal U(k)=A, k≡ejω, k=1,5,7,9,… where U (k=3), is purposely null. The unilateral convolution U*U is expressed by
(6)YΩu2(k)=U(k)*U(k)=∑j=0k‍U(j)U(k-j).
By evaluating (6), one can easily notice that YΩu2(k)≠0 only for k=2,6,8,…. In other words, YΩu2(k)≠0 only for even values of k or at the “even frequency lines” in frequency-domain context. Observe that such even points are not excited in U(k). Hence, YΩu2(k) only have contributions at theeven nonexcited frequency lines. This proves that the cluster Ωu2 will provide spectral power only at the even (and nonexcited) frequency lines ωp. Notice that similar conclusion can be obtained for any cluster Ωum with m being an even number.Consider now the regressoru(k-τ1)u(k-τ2)u(k-τ3) from cluster Ωu3. In this case |YΩu3|∝|U*U*U|=|U*YΩu2|. Based on the input characteristics and the spectral content of YΩu2, one can conclude that YΩu3 will contain power at the excited (ωe) and some odd nonexcited (ωi) frequency lines. There is no power at ωp. To show this, let us suppose the signal U(k) above once again. By evaluating the correlation
(7)YΩu3(k)=U(k)*YΩu2(k)=∑j=0k‍U(j)YΩu2(k-j)
it is straightforward to conclude that YΩu3(k)≠0 only for k=3,5,7,…. Notice that k are only odd numbers and one of them was not excited by the original signal (k=3). This proves that the cluster Ωu3 will provide spectral power only at the odd excited ωe and nonexcited ωi frequency lines. The same conclusion can be extended for any cluster Ωum with m being an odd number.Similar analysis can be extended to any cluster type, to exhibit its spectral content. However, such procedure would be quite tedious. In Sections4.2–4.4, the spectra content of each cluster type will be summarized by following the same arguments from this section.


## 4.2. LO System

The LO system is characterized by the occurrence of excited (ωe) and odd nonexcited (ωi) frequency lines at the output, when an odd-odd multisine input is applied. This means that a polynomial NARX model should contain regressors which provide power spectrum only at these frequency lines. In other words, a regressor which provides power spectrum only at ωp could not take part into the NARX model because it explains a type of nonlinearity which does not exist in the system. Then, one can infer the unnecessary clusters through an output spectral analysis. Such analysis follows for each type of clusters.Ω
u
m. Based on the discussion in Section 4.1 and the output spectral content of a LO system, it is easy to conclude that the cluster Ωum, with m being an odd number, can appear at NARX model. In other hand, regressors from Ωum with m being an even number are not expected.Ω
y
m. Firstly, consider the regressor y(k-τ1)y(k-τ2) (Ωy2). In this case |YΩy2|∝|Y*Y|. The LO system output contains power only at the odd frequency lines. The consequence is that the convolution above would produce power spectrum only at the even nonexcited frequency lines ωp. However, these frequencies are not possible for such system. The conclusion is that the cluster Ωy2 cannot appear into the polynomial NARX model of a LO system. It is straightforward to show the same for any cluster Ωym with m being an even number. Conversely, the regressor y(k-τ1)y(k-τ2)y(k-τ3) (cluster Ωy3) is characterized by |YΩy3|∝|Y*YΩy2|, which provides spectral contribution at the odd frequency lines (ωe and ωi). Then, this cluster—and any other Ωym with m being an odd number—can take part into the NARX model.Ω
u
r
y
s. Consider the regressor u(k-τ1)y(k-τ2) (cluster Ωuy). It implies |YΩuy|∝|U*Y|. In this case, as the input and the output are odd functions, presenting spectral power only at odd frequency lines. Then, the convolution would produce contributions only at even frequency lines, but this is impossible for a LO system. Similar conclusion can be drawn for any cluster Ωurys where r+s is an even number. All these clusters can be discarded from the polynomial NARX model of a LO system. The regressor u(k-τ1)y2(k-τ2) (cluster Ωuy2) produces |YΩuy2|∝|U*YΩy2|. In this case, there is the convolution between an odd function (input) and the even function y2(k-τ2) (due to the output odd aspect). This will produce contributions only at odd frequency lines (excited and/or nonexcited). Then, such cluster is possible for a LO system. The same holds for any cluster Ωurys where r+s is an odd number.


## 4.3. LE System

The LE system is characterized by the occurrence of excited (ωe) and even nonexcited (ωp) frequency lines at the output, when an odd-odd multisine input is applied. The cluster analysis is identical to those performed in Section 4.2; however, the results are quite different.Ω
u
m. Only the cluster Ωum with m being an even number is possible to appear in the NARX model, due to the output content of a LE system. In this case, Ωum with m being an odd number is not possible because it produces contributions at the odd nonexcited frequency lines ωi.Ω
y
m
and
Ω
u
r
y
s. Nothing can be assured about these clusters. Their regressors can produce contributions not only at ωp, but also at ωi. This is because the output is a not-even-nor-odd function, and any regressor formed with the output is also a not-even-nor-odd function. The final output contribution at ωi can vanished by the interplay of different output-related regressors and, because of this, any conclusion can be drawn with basis on the spectral analysis. For instance, the regressor u(k-τ1)y2(k-τ2) (Ωuy2) produces contributions given by the convolution of an odd function (input) and y2(k-τ2). Notice that this is a not-even-nor-odd function, since the output has both odd and even frequency lines. The result is a signal which—in addition to ωp and ωe—also contains odd nonexcited frequency lines ωi. These nonexcited odd frequency lines can be—in principle—canceled by the interplay with other regressors, resulting in output power only at ωp (and, eventually, at ωe). Because of this, it is not possible to assure the presence (or not) of such regressor.


## 4.4. Block-Oriented Systems with a Single Nonlinearity Type

Consider the following block-oriented models in Figure1, f(·) being a static nonlinearity and H(z) a linear system.Block-oriented models: (a) Hammerstein system and (b) Wiener system.
(a)
(b)For the Hammerstein system (Figure1(a)), the inner signal w(k) is just the nonlinearity applied to the input. Only nonlinear regressors related to the input are possible because the nonlinearity only affects this signal. Moreover, only terms with the same lag are present (e.g., um(k-τ1)) since a static nonlinearity does not affect the input lags. A term such as um(k-τ1)un(k-τ2), τ1≠τ2 is impossible.Iff(·) is an even function, W(ejω) will have frequency lines only at ωp for an odd-odd multisine input. The same will occur for the output Y(ejω) since it is the result of w(k) applied to a linear system (H(z)). Hence, only the nonlinear cluster Ωum with m being an even number—and with terms of identical lags—can be present in NARX models for Hammerstein systems with an even nonlinearity. On the other hand, if f(·) is an odd function; Y(ejω) will have frequency lines at ωe and ωi. By following the same arguments above, the NARX model can contain only terms from cluster Ωum with m being an odd number—and with terms of identical lags. Notice that these results are in perfect accordance with those in [13], where a different eigenvalue analysis was applied to Hammerstein models. The merit here is that one can discard some clusters of type Ωum which are not able to represent the actual inner nonlinearity.For the Wiener system (Figure1(b)), the analysis is more involving. The literature already showed that NARX regressors containing terms on ym(k-τ), for all m are not possible [13]. The reason is that such regressors yield a model with output multiplicity, which is impossible for a NARX polynomial model of Wiener systems.Because the nonlinearity is located after the linear block, it is applied to bothu(k-τu) and w(k-τw) (τu and τw determined by the linear transfer function). The consequence is that some nonlinear clusters involving output terms are expected to appear in a NARX model. Technically, the input terms from Ωum can contain different lags since the linear block can introduce lagged input terms in w(k). To explore such affirmation, consider that f is a square function and
(8)H(z)=W(z)U(z)=1+a1z-1+a2z-21+b1z-1+b2z-2.
Hence, the output can be written as
(9)yk=w2k=∑l=12‍∑m=12‍blbmw(k-l)w(k-m)+∑l=12‍∑m=12‍2albmu(k-l)w(k-m)+∑l=12‍∑m=12‍alamu(k-l)u(k-m).
Notice that the last summation in (9) has the product of different lagged terms from the input. The conclusion is that the polynomial NARX model for a Wiener system can have some terms from cluster Ωum with different lags.Concerning the nonlinearity analysis, consider a Wiener system with an even static function. In this case,Y(ejω) will have frequency lines only at ωp because it is the consequence of an odd signal applied to an even nonlinearity. Similarly to the analysis of the Hammerstein case, the cluster Ωum with m being an even number is possible to occur in such Wiener system. Now, consider the cluster Ωu2y and its output contribution |YΩu2y|∝|YΩu2*Y|. Notice that both YΩu2 and Y only contain power spectrum at even nonexcited frequency lines ωp. As a consequence, YΩu2y also contain power spectrum only at ωp. Hence, the cluster Ωu2y is possible. Similar analysis shows that any cluster Ωumy with m being an even number is possible for Wiener systems with even nonlinearity, in accordance with [13]. Finally, the cluster Ωumy with m being an odd number results in power only at ωe and ωi. However, power is expected only at ωp for even Wiener systems. Hence, this cluster can be discarded.Similar analysis can be applied to Wiener systems with an odd nonlinearity. The conclusion is that the clustersΩum with m being an odd number and Ωury, r=0,2,4,… can take part into the NARX model, because they produce output power spectrum only at the odd frequency lines (ωe and ωi). Regressors from the cluster Ωum with m being an even number should be discarded. The regressors from the cluster Ωury, r=1,3,5,… can be discarded.


## 4.5. Summary

Based on specific multisine excitation and output frequency-domain analysis, several conclusions were drawn in Sections4.1–4.4 related to the clusters of a NARX model. Such results are conveniently summarized in Table 1.Table 1
Summary of the results.
Cluster
System type
LO
LE
H
W
O
E
O
E
Ω
u
m, m even
×
P
×
P
*
×
P
Ω
u
m, m odd
P
×
P
*
×
P
×
Ω
y
m, m even
×
?
×
×
×
×
Ω
y
m, m odd
P
?
×
×
×
×
Ω
u
r
y
s, r+s even
×
?
×
×
×
×
Ω
u
r
y
s, r+s odd
P
?
×
×
P
†
P
†
H—Hammerstein, W—Wiener, O—odd, and E—even. LE and LO according to Section4. P implies that such cluster can appear in a polynomial NARX model, in contrast to ×. The symbol ? denotes that nothing can be assured about the cluster. P* indicates that only terms with the same lag are possible. P† implies s≡1.Such analysis is not able to decide whether a possible cluster (marked asP in Table 1) is really useful for the model or not. For instance, the clusters Ωum with m even (Ωu2,Ωu4,…) do not present the necessary conditions to be present in a LO system model. In consequence, they should be automatically discarded from the candidate set. On the other hand, the cluster Ωu3can be present in the LO model. Further analysis is still necessary to conclude if such cluster is really necessary or not.In conclusion, the proposed methodology does not assign if a cluster marked asP in Table 1 is spurious or not. What is possible to assure is that those clusters marked as × are surely spurious and they should be discarded from the candidate set.


## 5. Numerical Results

This section presents the numerical examples that show the cluster interpretation based on the proposed spectral analysis. Some practical details are also provided. All the considered systems are constructed through a linear part properly connected to nonlinear blocks. The linear part was assumed as(10)H(s)=1s+1.
Six different cases with the transfer function in (10) were considered.(I)
Linear system with a Coulomb friction term, preceded by saturation and dead zone nonlinearities (Figure2). In this case, the saturation level was set as ±2 while the dead zone length is ±0.5. The Coulomb friction has unitary offset and gain. Similar block diagram is common in mathematical models of mechanical systems.(II)
Linear system in parallel to an unsigned term (Figure3).(III)
A Hammerstein system with linear part given by (10) preceded by saturation and dead zone nonlinearities (identical to Case (I)).(IV)
A Hammerstein system with linear part given by (10) preceded by the square operation.(V)
A Wiener system with linear part given by (10) followed by saturation and dead zone nonlinearities (identical to Case (I)).(VI)
A Wiener system with linear part given by (10) followed by the square operation. Initially, a nonnoisy environment is considered. This is useful to show the main points discussed in the paper. The effects of measurement noise on the proposed methodology are presented in Section 5.1.Figure 2
Nonlinear system considered in Case (I).Figure 3
Nonlinear system considered in Case (II).A multisine input signal should be designed so that the spectral analysis can be performed. Full details about the signal design can be found in the literature [3, 21] and such design involves decisions about the signal magnitude, excited frequency lines, and so forth. Given a sampling frequency (10Hz for these examples), the fundamental frequency ω0 is chosen by establishing a compromise between a sufficient amount of available harmonics (by choosing a small ω0) and a reasonable experiment duration (by choosing higher values for ω0). Concerning the excitation frequency range, it should be sufficiently wide to cover the nonlinear phenomena to be explained. For this study, a fundamental frequency of 0.01Hz was chosen. The maximum excitation frequency was set as 1Hz, which provides 50 odd frequency harmonics, 20% of them—randomly chosen—are not effectively excited. The harmonic amplitudes were chosen so that the input RMS value is 1.8. The same input was used in all the discussed cases.In this paper, a fictitious candidate regressor set is considered for illustration purpose. This set is formed by terms from all clusters up to second degree and with single-lagged regressors. This includes the nonlinear regressorsu2(k-1), u3(k-1), y2(k-1), y3(k-1), u(k-1)y(k-1), u2(k-1)y(k-1), and u(k-1)y2(k-1).The spectral content is used to analyze the term clusters. Initially, consider Figure4(a) concerning Case (I). In all plots, the points marked as “×” represent the input excited frequency lines (ωe), while the points marked as “*” and “∘” are respectively the even and odd nonexcited frequency lines (ωp and ωi). The left bigger plot represents the output spectrum for Case (I). Notice the power dominance at the excited and odd nonexcited frequency lines. According to the exposed in Section 4, this system can be classified as “Linear + Odd” (LO) and the expected clusters which can take part into a polynomial NARX model are Ωu3, Ωy3, Ωu2y, and Ωuy2. To show this, the power spectrum of each cluster above was evaluated and presented at the smaller plots in Figure 4(a). It is important to observe that the clusters Ωu2, Ωy2, and Ωuy produce dominant power spectrum only at the even nonexcited frequency lines and then they are not useful to represent the nonlinear characteristics in Case (I). In accordance with the discussed in Section 4.2, Figure 4(a) shows that the clusters Ωu3, Ωy3, Ωu2y, and Ωuy2 produced power spectrum at the same frequency lines than the output signal. Hence, they are candidate to form the polynomial NARX model.Nonlinearity detection and cluster analysis through power spectrum. Case (I) (a) has an odd nonlinearity. Case (II) (b) has an even nonlinearity (× excited, ∘ odd,* even).
(a)
(b)Similar study was performed for Case (II) and it is summarized in Figure4(b). Notice that the output power spectrum (left) appears dominantly at excited and even nonexcited frequency lines in such case, thereby enabling its classification as a “Linear + Even” (LE) system. From the smaller plots one can observe that the nonlinear cluster Ωu2 presents power only at ωp. Hence, it is possible to be present in a NARX model. Ωu3 is not expected due to it produces power only at ωe and ωi. Concerning the other clusters, nothing can be assured because they can produce power at ωeωi and ωp.The same procedure can be applied to the other cases so that the conclusions in Section4 can be derived. However, only the output power spectrum for each case is presented in this paper for sizing questions.For Case (III) (Figure5(a)) it is possible to note the power spectral dominance at the excited and odd nonexcited frequency lines, thereby enabling its classification as a “Linear + Odd” (LO) system. This spectrum is similar to those presented in Figure 4(a) and this could induce that the clusters Ωy3 and Ωuy2 can be relevant in a NARX model. However, these clusters only occur in systems with output multiplicity, what is not possible for Hammerstein models [13]. Moreover Ωu2y is also impossible due to the Hammerstein system characteristics. Hence, only the cluster Ωu3 with single-lagged terms can be included at the polynomial NARX model.Power spectral analysis: Cases (III) (a), (IV) (b), (V) (c), and (VI) (d) (× excited, ∘ odd,* even).
(a)
(b)
(c)
(d)For Case (IV) (Figure5(b)), the output contains power spectrum only at the even nonexcited frequency lines. This is a characteristic common in block-oriented systems with even nonlinearities. Then, if a system contains an output power spectrum with such aspect, one can infer a Hammerstein and/or Wiener block connection containing even nonlinearities. In addition, by considering the discussed in Section 4.4 for Hammerstein systems, the only useful cluster for a NARX model is Ωu2 with single-lagged terms.Case (V) (Figure5(c)) can be also classified as a “Linear + Odd” (LO) system. Once again, notice the similarity with Figure 4(a). As it is a Wiener model, it can contain only the clusters Ωu3 and Ωu2y, including multilagged terms (see Section 4.4). In spite of the clusters Ωy3 and Ωuy2 are also possible from the frequency-domain point of view, they are not possible due to their output multiplicity aspect.Finally, Case (VI) (Figure5(d)) contains only the even nonexcited frequency lines—a characteristic of block-oriented systems with even nonlinearities. A Wiener model with such character can contain regressors from the cluster Ωu2, Ωu2y. The clusters Ωy2 and Ωuy2 cannot be included because—although they are possible from the frequency-domain point of view—they explain an inexistent output multiplicity behavior.


### 5.1. Noisy Environment

Noise can hamper any methodology for structure selection and this is also true for the spectral analysis. The main difficulty in this case is to decide whether the power at a specific frequency line is related to a dynamic behavior or not. When the power related to the noise is as high as the effective power provided by the system, the nonlinearity classification can become hard to perform. Hence, the main question here is how to improve the signal to noise ratio (SNR).A well-known methodology is the averaging procedure [3]. The idea is to perform M realizations of the same experiment (or, equivalently, to apply M consecutive multisine periods). This M records are then used to reduce the bias and variance of the power spectra estimation. Such procedure produces high signal to noise ratio (SNR). However, a considerably high number of measurements can be necessary.Since a precise determination of the spectral power in each frequency line is not necessary for nonlinearity detection, this paper considers an alternative procedure. The idea is to assume some assumptions for the noise and to analyze the power spectrum content. Initially, consider a sufficiently high SNR (signal-to-noise ratio) so that the system dynamic behavior can be remarkably noted from the output power spectrum—at least in some frequency range. By assuming that the noise has a white Gaussian behavior, it will present a flat power spectral contribution over the whole frequency range. Then, one can obtain an empirical “noise cutting level” through a spectrum inspection at the frequency range where the system behavior is not that clear. Those frequency lines whose power spectrum is above such cutting level are then classified as “useful” from the nonlinearity detection point of view.Figure6 shows the procedure for Case (II) when an output white Gaussian noise is added (σ2=0.01). By observing the final part of the output power spectrum, it is possible to infer that there is the dominance of noise. Then, the contribution along all the spectrum can be inferred by extrapolation of a level at this final part, here assumed as the maximum amplitude. Now, the nonlinearity classification is performed by noting the dominant power spectrum above such level. Even for a colored noise, such analysis can be useful if the SNR condition holds.Figure 6
Case (II) including a Gaussian noise withσ2=0.1 (× excited, ∘ odd, * even, — noise level).It is obvious that this is an approximate procedure and definitive information about the noise cannot be assured. However, this procedure, although it is simple, can be sufficient for nonlinearity detection and it is recommended as a first attempt, since the output power spectrum was already evaluated.


## 5.1. Noisy Environment

Noise can hamper any methodology for structure selection and this is also true for the spectral analysis. The main difficulty in this case is to decide whether the power at a specific frequency line is related to a dynamic behavior or not. When the power related to the noise is as high as the effective power provided by the system, the nonlinearity classification can become hard to perform. Hence, the main question here is how to improve the signal to noise ratio (SNR).A well-known methodology is the averaging procedure [3]. The idea is to perform M realizations of the same experiment (or, equivalently, to apply M consecutive multisine periods). This M records are then used to reduce the bias and variance of the power spectra estimation. Such procedure produces high signal to noise ratio (SNR). However, a considerably high number of measurements can be necessary.Since a precise determination of the spectral power in each frequency line is not necessary for nonlinearity detection, this paper considers an alternative procedure. The idea is to assume some assumptions for the noise and to analyze the power spectrum content. Initially, consider a sufficiently high SNR (signal-to-noise ratio) so that the system dynamic behavior can be remarkably noted from the output power spectrum—at least in some frequency range. By assuming that the noise has a white Gaussian behavior, it will present a flat power spectral contribution over the whole frequency range. Then, one can obtain an empirical “noise cutting level” through a spectrum inspection at the frequency range where the system behavior is not that clear. Those frequency lines whose power spectrum is above such cutting level are then classified as “useful” from the nonlinearity detection point of view.Figure6 shows the procedure for Case (II) when an output white Gaussian noise is added (σ2=0.01). By observing the final part of the output power spectrum, it is possible to infer that there is the dominance of noise. Then, the contribution along all the spectrum can be inferred by extrapolation of a level at this final part, here assumed as the maximum amplitude. Now, the nonlinearity classification is performed by noting the dominant power spectrum above such level. Even for a colored noise, such analysis can be useful if the SNR condition holds.Figure 6
Case (II) including a Gaussian noise withσ2=0.1 (× excited, ∘ odd, * even, — noise level).It is obvious that this is an approximate procedure and definitive information about the noise cannot be assured. However, this procedure, although it is simple, can be sufficient for nonlinearity detection and it is recommended as a first attempt, since the output power spectrum was already evaluated.


## 6. Conclusion

This paper presented a comprehensive study involving the even/odd nonlinearity mapping onto NARX models by considering arguments from the frequency domain. The idea is to apply a specific class of broadband excitation at the input and to analyze the output spectral content. This analysis contemplated two objectives: the inner nonlinearity classification and exhibiting the relation between such nonlinearity and the possible clusters which can explain it. These results only apply to input-output nonlinear systems where a specific excitation is possible. This condition is met by several real cases discussed in the literature.Important results are drawn for systems with a unique (or dominant) nonlinearity type. As shown, some regressors cannot represent an odd nonlinearity while others are not proper to explain even nonlinearities. This is an important contribution, because such conclusions are hard to obtain in the time domain. Although it is not the paper’s aim to proposing any formal methodology for model structure selection, the proposed analysis provides relevant additional information which can be used in association to techniques such as ERR and term clustering.These results show that the proposed analysis can be useful in many applications. The spectral study enables not only the nonlinearity detection—which is a great feature by itself—but also the nonlinearity codification through some clusters.



---

*Source: 650737-2014-12-07.xml*