An over-rejection puzzle of bootstrap average tests
for the no-threshold-eﬀect hypothesis
Kaiji Motegi∗andJohn W. Dennis†

Kobe University

IDA
April 1, 2024

Abstract

When testing the null hypothesis of no threshold eﬀects based on thresholdautoregressive models, wild-bootstrap supremum, average, and exponential tests are routinely used to handle an identiﬁcation issue under the null.
In this note, we demonstrate via Monte Carlo simulations that the bootstrap average tests lose control for the type-I error rate when the thresholdvariable is persistent and the delay parameter is chosen from more than a
handful of choices. In some cases, the average tests reject the correct null
hypothesis with probability exceeding nominal size by more than 10%.
The size distortion is present even in large samples, indicating the average tests may not converge to the intended asymptotic null distribution.
Supremum and exponential tests achieve correct type-I error rates, posing
a puzzle why only the average tests suﬀer from over-rejections.
JEL codes: C12, C22.
Keywords: Asymmetry, nonlinear time series analysis, size distortion, Threshold
Autoregression (TAR), type-I error, wild bootstrap.

∗
Corresponding author. Graduate School of Economics, Kobe University. 2-1 Rokkodai-cho,
Nada, Kobe, Hyogo 657-8501 Japan. E-mail: motegi@econ.kobe-u.ac.jp†
Institute for Defense Analyses (IDA). Research results and conclusions expressed are those of
the authors and do not necessarily reﬂect the views of IDA. E-mail: jay.dennis@alumni.unc.edu

11Introduction

In the literature of nonlinear time series analysis, it is well known that economic andﬁnancial indicators sometimes exhibit threshold eﬀects: a target variable y has asymmetric structures when a threshold variable x is below versus above a threshold µ.
A variety of threshold time series models, including Tong’s (1978) Threshold Autoregression (TAR), have been proposed in the literature. Whichever threshold model isused, a major concern lies in testing the null hypothesis of no threshold eﬀects H0 .
Under H0 , several parameters such as the delay d and threshold µ are unidentiﬁed.
Hansen (1996) provides a well-known solution to this issue via wild-bootstrap tests,
where common test statistics are supremum (sup-), average (ave-), and exponential(exp-) Wald statistics as well as their Lagrange Multiplier (LM) counterparts. 1
In this note, we demonstrate via Monte Carlo simulations that the bootstrap aveWald and ave-LM tests exhibit distortions for the type-I error rate if the thresholdvariable x is persistent and the space of delay parameter d contains several choices.
In some cases, the average tests reject the correct no-threshold-eﬀect hypothesis withprobability exceeding nominal size by more than 10%. This is particularly puzzlingas x and d do not play any role under the true data generating process. The size
distortion exists even in large samples, indicating that the average tests may notconverge to the intended asymptotic null distribution. Supremum and exponential
tests do not appear to exhibit size distortions in the same scenario. This puzzle is not
documented in the existing literature, and reasons for the over-rejections are unknown.
While using sup-tests or exp-tests is a practical solution, it would be an interesting
future task to explain why only the average tests appear to produce these distortions.
The remainder of this note is organized as follows. Section 2 sets up a simulation
study. We present our simulation results in Section 3, and give some concluding
remarks in Section 4. Omitted technical details and complete Monte Carlo simulations
are collected in a separate supplemental material. We use the following notation
throughout: R is the set of real numbers, N is the set of natural numbers, ⌊a⌋ is thelargest integer not larger than a ∈ R, #A is the number of elements of set A, andA × B is the Cartesian product of sets A and B.
1

Extensive surveys on TAR are provided by Tong (2015) and Tsay and Chen (2019), among
others. Recent empirical applications of threshold time series models and the wild-bootstrap tests
include Chen, Qiao, and Zhang (2022) and Motegi and Hamori (2023); they detected signiﬁcantthreshold eﬀects of stock market realized volatilities on crude oil realized volatilities. Andrews andPloberger (1994) discuss use of the wild-bootstrap tests when a nuisance parameter is present only
under the alternative hypothesis.

22

Simulation design

Suppose that the true data generating process (DGP) is as follows.
  

  
yt
ϕ0 0
yt−1
ϵ
 =

 +  t ,
xt
0 ψ0
xt−1
νt

 
  

ϵt i.i.d.
0
1 0  ∼ N   , 
 .
νt
0
0 1

(1)

The target variable y and the threshold variable x follow mutually independent singleregime AR(1) processes. Fix the AR(1) parameter for y at ϕ0 = 0.6.2 Consider low,
medium, or high persistence in x: ψ0 ∈ {0.3, 0.6, 0.9}. The joint standard normalityof the true errors is a conventional assumption which simpliﬁes analysis. Generate
J = 1000 Monte Carlo samples of size n ∈ {125, 250, 500, 1000} from DGP (1).
For each sample generated from (1), ﬁt a two-regime TAR model with lag lengthp = 1:yt =α + ϕ y

if xt−d < µ,

α + ϕ y

if xt−d ≥ µ,1

2

1 t−1 + ut
2 t−1 + ut

(2)

where β r = (αr , ϕr )⊤ is a vector of regression parameters in regime r ∈ {1, 2}; d ∈ N
is the delay parameter; and µ ∈ R is the threshold parameter.3 Consider the null
hypothesis of no threshold eﬀects H0 : β 1 = β 2 versus a ﬁxed alternative hypothesis
H1 : β 1 ̸= β 2 . Since the DGP is given by (1), H0 is true in our experiment. Under H0 ,the nuisance parameters γ = (d, µ)⊤ are not identiﬁed. To address this identiﬁcation
problem, we perform Hansen’s (1996) wild-bootstrap tests with ave-Wald, exp-Wald,
ave-LM, and exp-LM test statistics. When computing the actual and bootstrap test
statistics, we use a simple covariance matrix which is not robust to heteroscedasticity.
This is a correct choice since the error term u in model (2) is indeed homoscedastic
given the true DGP (1). Nominal size is set to be a = 0.05, and the number ofbootstrap iterations is B = 500.
To implement the bootstrap tests, one needs to specify the choice space of nuisance¯ with the upper boundparameter γ. We set the choice space of d to be D = {1, . . . , d}being d¯ ∈ {1, 2, 4, 8}. Speciﬁcally, the choice space D is either {1}, {1, 2}, {1, 2, 3, 4},or {1, . . . , 8}. For the choice space of µ, let x[1] ≤ · · · ≤ x[n] be a sorted version of x.
Simulation results are similar for ϕ0 ∈ {0.3, 0.6, 0.9}, hence we focus on the middle value here.
Complete results are reported in the supplemental material.
3
Previous results in the literature, including Hansen (1996), ensure uniform consistency of theestimator for (β 1 , β 2 ).
2

3

In general, the space of µ is speciﬁed as{}
Xκ,n = x[⌊0.5(1−κ)n⌋] , . . . , x[⌊{1−0.5(1−κ)}n⌋] ,(3)where κ ∈ [0, 1) signiﬁes the fraction of #Xκ,n to n. We pick κ = 0.7, following a wellknown suggestion of Andrews (1993). This means that Xκ,n consists of observations
above the 15 percentile and below the 85 percentile of x. The space of γ is givenby Γκ,n = D × Xκ,n . Compute a conditional test statistic given each γ ∈ Γκ,n , andaggregate the #Γκ,n outcomes into a single test statistic by either the average orexponential transformation.4
In ﬁnite samples, there is expected to be a bias-variance trade-oﬀ on the cardinality of Γκ,n . Under H0 , the ﬁnite sample performance of the bootstrap tests is expected
to be worse due to larger variance with larger #Γκ,n . Under H1 , asymptotic bias arises
if a true value of γ is not included in Γκ,n . In our current set-up, threshold eﬀects areabsent and H0 is true given DGP (1). Hence, the empirical size of the bootstrap tests
should be closest to the nominal size 5% for d¯ = 1 and farthest for d¯ = 8, particularly
for small sample sizes. When the sample size is large enough (say n = 1000), thetype-I error rate should be suﬃciently close to 5% for any d¯ ∈ {1, 2, 4, 8}. Keeping
this conjecture in mind, we report rejection frequencies in the next section.

3

Simulation results

In Table 1, we report rejection frequencies of the bootstrap average tests for theno-threshold-eﬀect hypothesis H0 based on the TAR model. We maintain focus on
the ave-LM test, as the ave-Wald test results are similar. When the threshold variable x has low persistence (i.e., ψ0 = 0.3), the empirical size of the ave-LM test
is suﬃciently close to the nominal size a = 0.05 for any d¯ ∈ {1, 2, 4, 8}. Taking¯ n) = (0.3, 1, 1000) as an example, the empirical size of the ave-LM test is(ψ0 , d,0.055.
When x has medium persistence (i.e., ψ0 = 0.6), size distortions emerge as d¯ in¯ n) = (8, 1000) is 0.108 (Table
creases. The empirical size of the ave-LM test with (d,1). When x has high persistence (i.e., ψ0 = 0.9), the tendency to over-reject H0 becomes more salient. The average tests lose control for the type-I error rate as d¯ grows.
The empirical size of the ave-LM test with n = 1000 is {0.051, 0.086, 0.132, 0.163} for
4Complete procedures of the bootstrap tests are described in the supplemental material.

4

Table 1: Empirical size of the bootstrap average tests for the no-threshold-eﬀect
hypothesis based on the TAR model (nominal size 5%)n = 125n = 250n = 500

n = 1000ψ0d¯ Wald

LM

Wald

LM

Wald

LM

Wald

LM

0.3

1

0.057

0.041

0.085

0.071

0.055

0.041

0.058

0.055

0.3

2

0.074

0.046

0.063

0.046

0.055

0.050

0.070

0.068

0.3

4

0.073

0.043

0.081

0.049

0.072

0.058

0.070

0.062

0.3

8

0.080

0.037

0.061

0.050

0.067

0.059

0.057

0.055

0.6

1

0.075

0.045

0.047

0.038

0.050

0.045

0.049

0.049

0.6

2

0.098

0.061

0.084

0.065

0.063

0.053

0.069

0.066

0.6

4

0.104

0.067

0.087

0.068

0.083

0.074

0.097

0.092

0.6

8

0.125

0.073

0.104

0.081

0.096

0.083

0.117

0.108

0.9

1

0.075

0.049

0.071

0.062

0.059

0.050

0.051

0.051

0.9

2

0.120

0.088

0.094

0.082

0.082

0.072

0.089

0.086

0.9

4

0.152

0.128

0.142

0.122

0.124

0.115

0.136

0.132

0.9

8

0.193

0.141

0.138

0.122

0.138

0.131

0.172

0.163

i.i.d.

DGP: yt = 0.6yt−1 + ϵt , xt = ψ0 xt−1 + νt , ψ0 ∈ {0.3, 0.6, 0.9}, (ϵt , νt )⊤ ∼ N (0, I). Sample size:n ∈ {125, 250, 500, 1000}. TAR Model: yt = α1 + ϕ1 yt−1 + ut if xt−d < µ and yt = α2 + ϕ2 yt−1 + ut if
xt−d ≥ µ. We perform the bootstrap ave-Wald and ave-LM tests for the no-threshold-eﬀect hypothesis H0 : (α1 , ϕ1 ) = (α2 , ϕ2 ). The choice space of delay parameter, D, is either {1}, {1, 2}, {1, 2, 3, 4},or {1, . . . , 8} (i.e., d¯ ∈ {1, 2, 4, 8}). The choice space of µ is X0.7,n = {x[⌊0.15n⌋] , . . . , x[⌊0.85n⌋] }, wherex[1] ≤ · · · ≤ x[n] is a sorted x. The nominal size is a = 0.05, and the number of bootstrap samples is
B = 500. This table reports the empirical size of the tests across J = 1000 Monte Carlo samples.

d¯ ∈ {1, 2, 4, 8}, respectively. The excess rejection rate is 8.2% for d¯ = 4 and 11.3% ford¯ = 8, which is substantially large compared with the fact that our DGP and model
are quite simple.
Surprisingly, the size distortion of the average tests does not vanish as the sample
size grows; the type-I error rate is still above the nominal size even for the largest
sample sizes we examined. This indicates that the average tests may not converge tothe intended asymptotic distribution in the relevant scenario. We are not aware ofany existing work that documents this puzzle. In previous simulation studies such as
Hansen (1996, Table II) and Ahmad and Donayre (2016, Table 1), the average tests
with persistent x and large upper bound d¯ are not covered. This note is likely the5

ﬁrst work that inspects the relevant case.
What is even more puzzling is that the sup-Wald, exp-Wald, sup-LM, and expLM tests achieve correct type-I error rates. The rejection frequencies of the exp-Waldand exp-LM tests are shown in Table 2. For all cases considered, the empirical size
is suﬃciently close to the nominal size a = 0.05. The exp-Wald test tends to overreject correct H0 in small samples such as n = 125, but their size quickly converges¯ = (0.9, 8), in which case the averageto the nominal size as n grows. Focus on (ψ0 , d)tests suﬀer from the most serious over-rejections. The empirical size associated withn = 1000 is 0.056 for the exp-Wald test and 0.050 for the exp-LM test. The rejection
frequencies of the supremum tests are similar to those of the exponential tests, andwe relegate them to the supplemental material.
Table 2: Empirical size of the bootstrap exponential tests for the no-threshold-eﬀect
hypothesis based on the TAR model (nominal size 5%)n = 125n = 250n = 500

n = 1000ψ0d¯ Wald

LM

Wald

LM

Wald

LM

Wald

LM

0.3

1

0.062

0.033

0.073

0.048

0.059

0.049

0.055

0.053

0.3

2

0.068

0.032

0.063

0.040

0.054

0.044

0.067

0.055

0.3

4

0.068

0.023

0.064

0.044

0.059

0.049

0.058

0.056

0.3

8

0.080

0.019

0.055

0.032

0.047

0.034

0.059

0.052

0.6

1

0.079

0.036

0.051

0.037

0.057

0.050

0.049

0.046

0.6

2

0.078

0.030

0.059

0.042

0.048

0.042

0.048

0.048

0.6

4

0.074

0.033

0.062

0.043

0.061

0.044

0.069

0.064

0.6

8

0.086

0.024

0.045

0.031

0.059

0.047

0.064

0.054

0.9

1

0.085

0.045

0.076

0.056

0.057

0.051

0.060

0.056

0.9

2

0.088

0.044

0.059

0.043

0.044

0.038

0.054

0.049

0.9

4

0.076

0.039

0.065

0.048

0.048

0.037

0.057

0.052

0.9

8

0.084

0.025

0.051

0.033

0.059

0.044

0.056

0.050

i.i.d.

DGP: yt = 0.6yt−1 + ϵt , xt = ψ0 xt−1 + νt , ψ0 ∈ {0.3, 0.6, 0.9}, (ϵt , νt )⊤ ∼ N (0, I). Sample size:n ∈ {125, 250, 500, 1000}. TAR Model: yt = α1 + ϕ1 yt−1 + ut if xt−d < µ and yt = α2 + ϕ2 yt−1 + ut if
xt−d ≥ µ. We perform the bootstrap exp-Wald and exp-LM tests for the no-threshold-eﬀect hypothesis H0 : (α1 , ϕ1 ) = (α2 , ϕ2 ). The choice space of delay parameter, D, is either {1}, {1, 2}, {1, 2, 3, 4},or {1, . . . , 8} (i.e., d¯ ∈ {1, 2, 4, 8}). The choice space of µ is X0.7,n = {x[⌊0.15n⌋] , . . . , x[⌊0.85n⌋] }, wherex[1] ≤ · · · ≤ x[n] is a sorted x. The nominal size is a = 0.05, and the number of bootstrap samples is
B = 500. This table reports the empirical size of the tests across J = 1000 Monte Carlo samples.

6

Summarizing Tables 1-2, only the average tests over-reject the true no-thresholdeﬀect hypothesis H0 in large samples. Reasons for the over-rejections are unknown.
A practical solution is to use a sup-LM or exp-LM test, as these lead to accurate
empirical size in both small and large samples. It is an interesting future task, however,to explain what distorts the type-I error rate of the average tests.
To further characterize the over-rejection puzzle of the bootstrap average tests,
we report additional simulation evidence. Recall that D signiﬁes the choice spaceof the delay parameter d. In the previous simulation, we always ﬁxed the lower
bound of D at 1. Consequently, an exact role of D remains unclear. Which distorts
the size of the average tests: a large cardinality of D or a large candidate value ford? To answer this question, we consider some additional speciﬁcations for D withvarious lower and upper bounds. In view of the resulting rejection frequencies, it is
the cardinality of D, not a speciﬁc value for candidate d, that plays a key role in
the over-rejection puzzle. Excess rejection rates of the average tests increase as #D
grows. Conditional on the value of #D, we observe similar rejection rates irrespectiveof the speciﬁc candidate values contained in D. More detailed results are presented
in the supplemental material.

4

Discussion and conclusion

We have presented simulation evidence that the wild-bootstrap average tests for theno-threshold-eﬀect hypothesis H0 : β 1 = β 2 fail to control the type-I error rate when
the threshold variable x is suﬃciently persistent and the choice space of the delay
parameter d is suﬃciently large. In these cases, the type-I error rate far exceeds thenominal size of 5%. Moreover, the size distortion does not diminish as the sample
size grows, which indicates that the average tests may converge to an unintended null
distribution. The supremum and exponential tests have correct type-I error rates,
posing a puzzle why only the average tests suﬀer from over-rejections.
In additional simulations, we consider a number of alternative scenarios to conﬁrm
the robustness of our ﬁnding. First, we try larger sample sizes, such as n = 2000.
Second, the nominal size is determined from a ∈ {0.01, 0.05, 0.10}. Third, various
degrees of persistence in y are considered: ϕ0 ∈ {0.3, 0.6, 0.9}. Fourth, the choice
space of µ is more tightly restricted by choosing κ ∈ {0, 0.35, 0.7} in (3). Fifth,
the simple covariance matrix is replaced with a heteroscedasticity-robust covariancematrix when computing the actual and bootstrap test statistics. Sixth, the TAR

7model (2) is replaced with a self-exciting version (SETAR) by using y itself as a
threshold variable. Our ﬁndings are the same for all these scenarios, and most of theadditional simulation results are shown in the supplemental material.

Acknowledgements
We thank Shigeyuki Hamori, Seok Young Hong, Ekaterina Kazak, and Tatevik Sekhposyan for helpful comments and discussions. The ﬁrst author is grateful for the
ﬁnancial support of JSPS KAKENHI Grant-in-Aid for Challenging Research (Exploratory) with Grant Number 23K17555, Ishii Memorial Securities Research Promotion Foundation, Japan Securities Scholarship Foundation, Murata Science and
Education Foundation, and Nomura Foundation.

References
Ahmad, Y., and L. Donayre (2016): “Outliers and persistence in threshold autoregressive processes,” Studies in Nonlinear Dynamics & Econometrics, 20, 37–56.
Andrews, D. W. K. (1993): “Tests for Parameter Instability and Structural Changewith Unknown Change Point,” Econometrica, 61, 821–856.
Andrews, D. W. K., and W. Ploberger (1994): “Optimal tests when a nuisanceparameter is present only under the alternative,” Econometrica, 62(6), 1383–1414.
Chen, Y., G. Qiao, and F. Zhang (2022): “Oil price volatility forecasting: Threshold eﬀect from stock market volatility,” Technological Forecasting & Social Change,
180, #121704.
Hansen, B. E. (1996): “Inference when a nuisance parameter is not identiﬁed under
the null hypothesis,” Econometrica, 64(2), 413–430.
Motegi, K., and S. Hamori (2023): “Conditional threshold eﬀects of stock market
volatility on crude oil market volatility,” SSRN Working Paper #4512310.
Tong, H. (1978): “On a threshold model,” in Pattern Recognition and Signal Processing, ed. by C. H. Chen. Sijthoﬀ and Noordhoﬀ, Amsterdam.
(2015): “Threshold models in time series analysis—Some reﬂections,” Journal of Econometrics, 189, 485–491.
Tsay, R. S., and R. Chen (2019): Nonlinear Time Series Analysis. John Wiley &Sons, Inc.

8