# Most Effective Sampling Scheme for Prediction of Stationary Stochastic Processes

Authors: Mohammad Mehdi Saber; Zohreh Shishebor; M. M. Abd El Raouf; E.H. Hafez; Ramy Aldallal
Journal: Complexity
 (2022)

---

## Abstract

The problem of finding optimal sampling schemes has been resolved in two models. The novelty of this study lies in its cost efficiency, specifically, for the applied problems with expensive sampling process. In discussed models, we show that some observations counteract other ones in prediction mechanism. The autocovariance function of underlying process causes mentioned result. Our interesting result is that, although removing neutralizing observations convert sampling scheme to nonredundant case, it causes to worse prediction. A simulation study confirms this matter, too.


---

## Body

## 1. Introduction

Prediction is often the main goal in analyzing spatial processes and time series. It is widely used to making applicable decisions in a numerous scientific fields such as geology, biology, medicine, crime surveying, natural disasters, and so on (see for instance Isaaks and Srivastava [1] and Akselsson et al. [2]). The problem of finding an optimal sampling scheme is an important task of researchers, since in applied sciences often collecting of samples is a laborious and expensive process. We refer the interested readers to the works such as McBratney and Webster [3]; Zio et al. [4]; Xiao et al. [5]; and Sward et al. [6].In this work, the authors attempt to consider some theoretical points in a branch of stochastic processes. In contrast, there are deep studies in existing stochastic processes. Among them, we refer to Xie et al. [7] and Cheng et al. [8]. In the first work, a memory-based event-triggered asynchronous control was addressed for semi-Markov switching systems. In the latter, the authors focus on static output feedback quantized control for fuzzy Markovian switching singularly perturbed systems with deception attacks. For other works, one can see Zhou et al. [9], Cheng et al. [10] and Xie et al. [7].By Cressie [11], kriging is the most important prediction method for geostatistical data. For prediction in lattice data or discrete time series, we need to apply a model to data. Two predictors for a single missing value in a stationary autoregressive process of order one (AR1) model have been compared by Hamaz and Ibazizen [12] and Saadatmand et al. [13]. They considered the following model:(1)Zt=φZt−1+εt,t=1,…,n,φ<1,in which εt is a white noise process. Their comparison was done by Pitman’s measure of closeness (PMC) criterion, They showed the best predictor depend only on two nearest observations. In spatial context, Saber and Nematollahi [14] and Saber [15] studied the stationary first-order multiplicative spatial autoregressive (MSAR1) model on a m×n lattice Λ, that is,(2)Zi,j=aZi−1,j+bZi,j−1−abZi−1,j−1+εij,=1,…,m,j=1,…,n,a<1,b<1.Saber and Nematollahi [14] performed a comparison among the three predictors in the above model. They showed that predictor which uses the quarter observations is better than predictor based on observations in the first neighborhood. Also, it is better than predictor based on observations in the nearest neighborhood wherever parameters a and b are near 0. However, for most values of a and b, the predictor which uses the nearest neighborhood observations is the best among three recommended predictors. Saber [15] compared interpolation and extrapolation in MSAR1 model. Figure 1 represents schemes of used observations for these two predictors.Figure 1
Locations of observations■ and missing value △. Upper figure shows three schemes for interpolation while lower figure shows interpolation and extrapolation.In Saber and Nematollahi [14], predictors use from all observations, while two predictors in Saber [15] do not use all observations. In fact, the interpolator which has been constructed on eight observations uses only two observations. Also, the extrapolator which is based on ten observations is constructed by just one observation. There are one or two observations which play an important role in the prediction process. It can be seen that the result of the prediction based on one basic and neutralizing observation is better than prediction based on a large number of nonessential observations. We show both models (1) and (2) are in this mentioned class.So, this paper is organized as follows. In Section2, the neutralizing observation for prediction in MSAR1 model is achieved in some sampling schemes. A comparison between prediction by one neutralizing observation and prediction by eight usual observation has been performed in this section, too. Section 3 is the same as Section 2 in which the AR1 model is in the role of MSAR1 model. Finally, a discussion about redundant sampling is stated in Section 4.


## 2. Prediction in MSAR(1) Model

Suppose for any fixed values ofi and j we want to predict Zi,j by a set of other observations (Zi,j) whose indices belong to Oi,j as a subset of lattice Λ. In other words, Oi,j=k,l;Zk,l∈Zi,j and Zi,j=Zk,l;k,l∈Oi,j. Also, the predictor of Zi,j in terms of the components of set Zi,j is denoted by Z^i,j.Theorem 1.
LetZi,j satisfy (1) with EZi,j=0 and VarZi,j<∞. Then, the best linear predictor for Zi,j w.r.t mean square error (MSE) based on set Zi,j is only dependent to Zs0,t0 and is given by(3)Z^i,j=ai−s0bj−t0Zs0,t0,if there exists a location s0,t0∈Oi,j such that one of the following conditions is satisfied for all u,v∈Oi,j.(4)i>u,j>v,s0≥u,andt0≥v,(5)i>u,j>v,s0≥u,andt0≥v,(6)i>u,j>v,s0≥u,andt0≥v,(7)i>u,j>v,s0≥u,andt0≥v,(8)i>u,j>v,s0≥u,andt0≥v,(9)i<u,s0≤u,andt0=j,(10)i<u,s0≤u,andt0=j,(11)i<u,s0≤u,andt0=j.Proof.
Here, the best predictor is earned by projection theorem (see Brockwell and Davis [16] for details of this theorem). Consider this predictor in a linear form of observations Z^i,j=αZi,j where Zi,j=Zk,l;k,l∈Oi,j and α=αk,l;k,l∈Oi,j. By Saber [15], α is computed from the following equation:(12)Aα=B,in which A=varZi,j and B=covZi,j,Zi,j; the first is covariance matrix of observations and the second is covariance vector between observations and unobserved variable, respectively. Regarding Gaetan and Guyon [17], covZs+h1,t+h2Zs,t=σε2/b2−1a2−1ah1bh2 whose constants σε2/b2−1a2−1 are removed from both sides of (12). Therefore, the general components of A and B are(13)as−ubt−v,as−ibt−j,for alls,tandu,v∈Oi,j.
Equation (12) can be rewritten as follows:(14)∑k,l∈Oi,jαk,lCk,l=B,in which Ck,l are the columns of A. Since one of the columns of A is covariance between Zi,j and Zs0,t0, the latter equation can be written in the following form:(15)αs0,t0Cs0,t0+∑k,l∈Oi,j,k,l≠s0,t0αk,lCk,l=B.
On the other hand, assumptions (3)–(10) along with (13) lead to(16)covZk,l,Zi,j=covZk,l,Zs0,t0covZs0,t0,Zi,j∀k,l∈Oi,j,which gives(17)B=Cs0,t0ai−s0bj−t0.
By substituting (17) in (15), we have(18)αs0,t0Cs0,t0+∑k,l∈Oi,j,k,l≠s0,t0αk,lCk,l=ai−s0bj−t0Cs0,t0,and the solution is(19)αs0,t0=ai−s0bj−t0andαk,l=0.∀k,l≠s0,t0.
This completes the proof.
To understand the concept of equations (4)–(7), some examples are given in Figure 2. Conditions (4)–(11) are available in a sampling process. In Saber [15], Oi,j2 has been defined as Oi,j2=s,t;s−i≤2,t−j=−1or−2 which satisfies condition (11) for s0,t0=i,j−1. Therefore, we see that the predictor Z^i,j2 is only dependent on Zi,j−1 observation.Figure 2
Locations of neutralized observations△, neutralizing observation (+), and missing value ■.


### 2.1. Comparison with Unredundant Sampling

Some of the observations, which satisfy the mentioned cases, do not have an effect on prediction.Therefore, an important question is that with a fixed number of observations which sampling scheme is better: sampling with redundant observation or the other?In other words, is it logical that we remove neutralizing observations in the prediction process? In order to find answer of this question we compare these two methods in this section. To this end, two sampling schemes are considered in Figure3.Figure 3
Two schemes for redundant sampling and unredundant sampling with eight observations.(20)A=γ0,0γ0,1γ1,1γ1,0γ1,1γ2,1γ2,0γ2,1γ0,1γ0,0γ1,2γ1,1γ1,0γ2,2γ2,1γ2,0γ1,1γ1,2γ0,0γ0,1γ0,2γ1,0γ1,1γ1,2γ1,0γ1,1γ0,1γ0,0γ0,1γ1,1γ1,0γ1,1γ1,1γ1,0γ0,2γ0,1γ0,0γ1,2γ1,1γ1,0γ2,1γ2,2γ1,0γ1,1γ1,2γ0,0γ0,1γ0,2γ2,0γ2,1γ1,1γ1,0γ1,1γ0,1γ0,0γ0,1γ2,1γ2,0γ1,2γ1,1γ1,0γ0,2γ0,1γ0,0.By Theorem1, the predictor which has based on redundant sampling is(21)Z^i,jR=a2b2Zi+2,j+2,i=1,…,m−2,j=1,…,n−2.By the same method as Saber [15], the other predictor based on unredundant sampling is(22)Z^i,jIR=A−1BZi,jIR,i=1,…,m−4,j=1,…,n−4,where Zi,jIR=Zi+2,j+3,Zi+2,j+4,Zi+3,j+2,Zi+3,j+3,Zi+3,j+4,Zi+4,j+2,Zi+4,j+3,Zi+4,j+4.BT=γ2,3γ2,4γ3,2γ3,3γ3,4γ4,2γ4,3γ4,4 and γu,v=aubv. After a cumbersome computation, we have(23)Z^i,jIR=a2b3Zi+2,j+3+a3b2Zi+3,j+2−,a3b3Zi+3,j+3,i=1,…,m−4,j=1,…,n−4.A theoretical comparison ofZ^i,jR and Z^i,jIR is not possible, so to compare them, we use MSP^ZZ^=∑i,jm,nZ^i,j–Zi,j2/mn and PMC^ZZ^|Z˜=∑i,jm,nUZ˜i,j−Zi,j−Z^i,j–Zi,j/mn estimators. Here Ut is 1 for positive t and 0 otherwise.Two well-known distributions normal (N0,1) and exponential (E1) for errors have been used. By Saber and Nematollahi [14], in case of E1 errors, we apply the mean-zero variables Zi,j∗=Zi,j−1.Now, we simulate random variablesZi,j with Normal distributed errors on a 200×200 lattice and compute MSP^ZZ^R, MSP^ZZ^IR, and PMC^ZZ^R|Z^IR. All the findings are demonstrated in Figures 4 and 5, respectively. In both figures, one can see that MSP^ZZ^R≤MSP^ZZ^IR and PMC^ZZ^R|Z^IR≥0.5 for almost all values of parameters a and b. For some values of these parameters which are less than 0.25, PMC^ZZ^R|Z^IR<0.5. In these cases, PMC^ZZ^R|Z^IR>0.495, so they are negligible. In other words, approximately we have 0≤MSP^ZZ^R−MSP^ZZ^R<1.75 and 0.495≤PMC^ZZ^R|Z^IR<0.66 for all values of parameters a and b. These both criteria are increasing functions w.r.t parameters a and b. These findings state that the predictor Z^R is better than predictor Z^IR with respect to both criteria MSP and PMC. Therefore, for predicting Zi,j in model (2), using one observation Zi+2,j+2 leads to better results than using 8 observations in Zi,jIR. When parameters a and b are near 1, recommendation for using of this one observation rather than 8 observations in Zi,jIR becomes more serious.Figure 4
Comparison betweenZ^R and Z^IR for some fixed a and all b. Here, innovations have normal distribution.Figure 5
Comparison betweenZ^IR and Z^IR with normal errors for some fixed b and all a.The similar work with normal errors has been done for case of exponential distributed errors with parameter1. For exponential case, our findings demonstrate a bit difference with normal case. Results of this simulation are displayed in Figures 6 and 7. These figures show approximately 0≤MSP^ZZ^IR−MSP^ZZ^R<1.6 and 0.47≤PMC^ZZ^R|Z^IR<0.65 for all values of parameters a and b. These findings state that the predictor Z^R is better than predictor Z^IR with respect to criterion MSP. Regarding PMC criterion, we can see that the predictor Z^IR is better than predictor Z^R whenever parameters a and b are near 0.5. However, the latter result is not significant.Figure 6
Comparison betweenZ^R and Z^R with exponential errors for some fixed a and all b.Figure 7
Comparison betweenZ^R and Z^IR with exponential errors for some fixed b and all a.Finally, we conclude that removing observation which has caused other observations be redundant in prediction does not lead to better results in almost all situations.


## 2.1. Comparison with Unredundant Sampling

Some of the observations, which satisfy the mentioned cases, do not have an effect on prediction.Therefore, an important question is that with a fixed number of observations which sampling scheme is better: sampling with redundant observation or the other?In other words, is it logical that we remove neutralizing observations in the prediction process? In order to find answer of this question we compare these two methods in this section. To this end, two sampling schemes are considered in Figure3.Figure 3
Two schemes for redundant sampling and unredundant sampling with eight observations.(20)A=γ0,0γ0,1γ1,1γ1,0γ1,1γ2,1γ2,0γ2,1γ0,1γ0,0γ1,2γ1,1γ1,0γ2,2γ2,1γ2,0γ1,1γ1,2γ0,0γ0,1γ0,2γ1,0γ1,1γ1,2γ1,0γ1,1γ0,1γ0,0γ0,1γ1,1γ1,0γ1,1γ1,1γ1,0γ0,2γ0,1γ0,0γ1,2γ1,1γ1,0γ2,1γ2,2γ1,0γ1,1γ1,2γ0,0γ0,1γ0,2γ2,0γ2,1γ1,1γ1,0γ1,1γ0,1γ0,0γ0,1γ2,1γ2,0γ1,2γ1,1γ1,0γ0,2γ0,1γ0,0.By Theorem1, the predictor which has based on redundant sampling is(21)Z^i,jR=a2b2Zi+2,j+2,i=1,…,m−2,j=1,…,n−2.By the same method as Saber [15], the other predictor based on unredundant sampling is(22)Z^i,jIR=A−1BZi,jIR,i=1,…,m−4,j=1,…,n−4,where Zi,jIR=Zi+2,j+3,Zi+2,j+4,Zi+3,j+2,Zi+3,j+3,Zi+3,j+4,Zi+4,j+2,Zi+4,j+3,Zi+4,j+4.BT=γ2,3γ2,4γ3,2γ3,3γ3,4γ4,2γ4,3γ4,4 and γu,v=aubv. After a cumbersome computation, we have(23)Z^i,jIR=a2b3Zi+2,j+3+a3b2Zi+3,j+2−,a3b3Zi+3,j+3,i=1,…,m−4,j=1,…,n−4.A theoretical comparison ofZ^i,jR and Z^i,jIR is not possible, so to compare them, we use MSP^ZZ^=∑i,jm,nZ^i,j–Zi,j2/mn and PMC^ZZ^|Z˜=∑i,jm,nUZ˜i,j−Zi,j−Z^i,j–Zi,j/mn estimators. Here Ut is 1 for positive t and 0 otherwise.Two well-known distributions normal (N0,1) and exponential (E1) for errors have been used. By Saber and Nematollahi [14], in case of E1 errors, we apply the mean-zero variables Zi,j∗=Zi,j−1.Now, we simulate random variablesZi,j with Normal distributed errors on a 200×200 lattice and compute MSP^ZZ^R, MSP^ZZ^IR, and PMC^ZZ^R|Z^IR. All the findings are demonstrated in Figures 4 and 5, respectively. In both figures, one can see that MSP^ZZ^R≤MSP^ZZ^IR and PMC^ZZ^R|Z^IR≥0.5 for almost all values of parameters a and b. For some values of these parameters which are less than 0.25, PMC^ZZ^R|Z^IR<0.5. In these cases, PMC^ZZ^R|Z^IR>0.495, so they are negligible. In other words, approximately we have 0≤MSP^ZZ^R−MSP^ZZ^R<1.75 and 0.495≤PMC^ZZ^R|Z^IR<0.66 for all values of parameters a and b. These both criteria are increasing functions w.r.t parameters a and b. These findings state that the predictor Z^R is better than predictor Z^IR with respect to both criteria MSP and PMC. Therefore, for predicting Zi,j in model (2), using one observation Zi+2,j+2 leads to better results than using 8 observations in Zi,jIR. When parameters a and b are near 1, recommendation for using of this one observation rather than 8 observations in Zi,jIR becomes more serious.Figure 4
Comparison betweenZ^R and Z^IR for some fixed a and all b. Here, innovations have normal distribution.Figure 5
Comparison betweenZ^IR and Z^IR with normal errors for some fixed b and all a.The similar work with normal errors has been done for case of exponential distributed errors with parameter1. For exponential case, our findings demonstrate a bit difference with normal case. Results of this simulation are displayed in Figures 6 and 7. These figures show approximately 0≤MSP^ZZ^IR−MSP^ZZ^R<1.6 and 0.47≤PMC^ZZ^R|Z^IR<0.65 for all values of parameters a and b. These findings state that the predictor Z^R is better than predictor Z^IR with respect to criterion MSP. Regarding PMC criterion, we can see that the predictor Z^IR is better than predictor Z^R whenever parameters a and b are near 0.5. However, the latter result is not significant.Figure 6
Comparison betweenZ^R and Z^R with exponential errors for some fixed a and all b.Figure 7
Comparison betweenZ^R and Z^IR with exponential errors for some fixed b and all a.Finally, we conclude that removing observation which has caused other observations be redundant in prediction does not lead to better results in almost all situations.


## 3. Prediction inAR1 Model

In this section, we show that the best linear predictor at timet for stochastic process AR1 uses at most one observation in every side of time t. First of all, for any fixed time t, define vector ZT=Zk;k∈Ot where Ot is a subset of natural numbers (ℕ).In the following theorem, we demonstrate the best linear predictor in this model based onn samples uses at least 2 observations.Theorem 2.
LetZt come from model (1) with EZt=0 and VarZt<∞ and there exists a time t0∈Ot such that for all observed times i∈Ot,(24)i<t,andi≤t0,or(25)i>t,andi≥t0.
Then, the best linear predictor forZtw.r.t criterion MSP based on variables Zt=Zk;k∈Ot is only dependent to Zt0 and(26)Z^t=φt−t0Zt0.Proof.
The proof is similar with the proof of Theorem1 in which Z^t=αZt where α=αk;k∈Ot, A=varZt, B=covZt,Zt, and covZs+h,Zs=σε2/1−φ2φh.
This matter shows that this is not required to do a corresponding comparison with Section2.1. By Theorem 2 and above paragraph, the best linear predictors based on observations Zt and Ztnew=Zk;k∈Ot−t0 are Z^t=φt−t0Zt0 and Z^tnew=φt−t1Zt1, respectively. Here, t1 is the nearest time to time t among all times in Ot−t0. Clearly, t−t1>t−t0, and hence the predictor Z^t is better than predictor Z^tnew.
In the end, we give a theorem with which Theorems1 and 2 can be expressed as its special cases.Theorem 3.
LetZs;s∈S, S⊆Rk,k∈ℕ be a stationary stochastic process with EZs=0 and VarZs<∞. Then, the best linear predictor for Zs based on Zs=Zv;v∈Os with respect to MSE is Z^s=covZs0,ZsZs0 if there exists a point s0∈Os such that(27)covZu,Zs=covZu,Zs0covZs0,Zs∀u∈Os.The proof of theorem is not presented, since it is point to point similar to the proof of Theorem1. Notice that both Theorems 1 and 2 are special cases of Theorem 3 when k=2 and k=1, respectively.


## 4. Conclusion

In this paper, three theorems were presented which are useful for deriving optimal sampling scheme. In Theorem1, the design of an optimal sampling scheme is given for model (2). A similar result for model (1) is found in Theorem 2.In fact, we have tried to give an answer to the following question: for a fixed set of data collected through observations, if we would like to find the best linear predictor of a missing value, which of the following sampling schemes is logical?(1)
Should we use neutralizing observations?(2)
Should we remove neutralizing observations and use remaining observations?The results show that the first scenario is more efficient in prediction process. Indeed, we have shown that some samples have no effect on prediction in discussed models. The achievement of this work will be useful, since it might not be possible to access a list of elements in many practical situations and several types of populations, so the use of an element as a sampling unit is not applicable.There are other complicated situations than those described in Theorem1. They lead to redundant sampling in mechanism of prediction, too. This is left as future work. Also, in Saber and Khorshidian [18], the best predictor in stationary first-order moving average model has been represented. So, another interesting study may be searching and exploring neutralizing and neutralized samples in moving average models.



---

*Source: 4997675-2022-07-09.xml*