DP2018-14

Semiparametric Bayes
Instrumental Variable Estimation
with Many Weak Instruments
Ryo KATO
Takahiro HOSHINO

May 15, 2018

Semiparametric Bayes Instrumental Variable
Estimation with Many Weak Instruments
Ryo Kato
Research Institute for Economics and Business Administration,
Kobe University, 2-1 Rokkodai-cho, Nada-ku, Kobe, Japan
and
Takahiro. Hoshinoy
Department of Economics, Keio University, 2-15-45 Mita, Minato-ku,
Tokyo, Japan /RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi,
Chuo-ku, Tokyo, Japan

Summary.We develop a new semiparametric Bayes instrumental variables
estimation method. We employ the form of the regression function of the
reduced-form equation and the disturbances are modelled nonparametricallyto achieve better preditive power of the endogenous variables, whereas we
use parametric formulation in the structural equation, which is of interest
in inference. Our simulation studies show that under small sample size theproposed method obtains more e¢ cient estimates and very precise credibleintervals compared with existing IV methods. The existing methods fail toreject the null hypothesis with higher probability, due to larger variance ofthe estimators. Moreover, the mean squared error in the proposed method
may be less than 1/30 of that in the existing procedures even in the presenceof weak instruments. We applied our proposed method to a Mendelian randomization dataset where a large number of instruments are available andsemiparametric speci…cation is appropriate. This is a weak instrument case;hence, the non-Bayesian IV approach yields ine¢ cient estimates. We obtained statistically signi…cant results that cannot be obtained by the existing
methods, including standard Bayesian IV.
Key words: Instrumental variable; Mendelian Randomization; Semiparametric Bayes model; Probit stick-breaking process mixtureyemail: kato.ryo@keio.jp
email: bayesian@jasmine.ocn.ne.jp

11. Introduction
This article presents a new semiparametric Bayes model for instumental variables problems. We treat the reduced-form equation (or the “…rst-stage”
regression model) and the joint distribution of the error terms as nonparametric and potentially changing in form, corresponding to the values of the
instrumental variables. In addition, our emphasis is on the semiparametric model formulation. Structural equation models (or the “second-stage”
regression models), which are of interest in inference, are formulated parametrically, whereas the reduced-form equation and the disturbance terms areformulated nonparametrically.
Instrumental variables (IV) methods have become increasingly importantin many empirical …elds. Initially, IV estimation and its application were restricted to empirical economics. However, IV methods have been recently
applied in other …elds, such as epidemiology for causal inference (Ramsahai
and Lauritzen, 2011, Baiocchi et al., 2014, Wang et al., 2017). In particular,
IV approaches are employed when it is not feasible to carry out randomized
controlled trials (RCT) or standard causal inference methodology, which assumes that no unobserved confounding exists. If it is possible to …nd su¢ cientIV that are predictive of endogenous variables, have no direct e¤ect on theoutcome, and are independent of the unobserved confounders, then the e¤ectof the unobserved confounders can be controlled.
However, inference based on IV is prone to be imprecise when the instruments explain only a small portion of the variation in the endogenous variable(weak instrument case). This problem is inherent in samples of small size, as
the data is insu¢ cient to identify the parameters of interest (Conley et al.,2008). In general, frequentist methods depend on asymptotics and this property can be a hindrance when the sample size is small. Therefore they are
not occasionally suitable for IV problems. By contrast, since Bayesian methods do not rely on asymptotics, applying these methods to IV problems is areasonable choice. Even though there are no direct incentives for adopting
Bayesian methods, Conley et al. (2008) showed that they incur smaller meansquared error (MSE) and provide better interval estimation compared with
non-Bayesian methods when the structural equation and the reduced-fromequation are properly speci…ed. Moreover, Bayesian methods allow for more‡exible modeling of the structural equation, since Bayesian inference depends
only on the joint model of the structural and the reduced-form equation,
whereas classical methods require the development of di¤erent estimation
procedure according to whether we have discrete, clustered, or panel data.
One of the disadvantages of Bayesian methods is that they impose strong2

distributional assumptions on the parameters. This is the case in IV problems, since Bayesian IV generally assumes that the joint distribution of
disturbances is bivariate normal. Conley et al. (2008) proposed another
Bayesian IV method that uses a Dirichlet process mixture (DPM) model forthe error terms. It moderates the assumption on the disturbances by using
DPM nonparametric speci…cation.
Our proposed procedure is a semiparametric Bayesian IV method andis more ‡exible than Conley’s method. We also assume that disturbances
have nonparametric structure. Moreover, we use nonparametric formulationin the reduced-from equation. In general, the true functional form of the
reduced-from equation is unknown, and its parameters are not of interest. In
addition, if we use many instruments, parametric modeling of a large numberof variables may result in misspeci…cation bias (Chib, 2007). The approaches
assuming that the reduced-form equation has some speci…c functional form(linear regression is often assumed), including frequentist and Conley’s IV
methods, yield unbiased estimates of the structural equation. However, they
are less e¢ cient. By contrast, if, for example, the reduced-from equation is
not a simple linear combination with additive disturbances, our semiparametric Bayes model …ts the data better and yields e¢ ciency gains comparedwith classical parametric method and Conley’s method.
Since the parameters of the structural equation are important in appliedresearch, we assume that the structural equation regression model has parametric structure. Moreover, our model is di¤erent from other frequentist
nonparametric IV approaches in that these approaches use nonparametricspeci…cation in the structural equation and parametric speci…cation in the
reduced-form equation.
We employ a probit stick-breaking process mixture (PSBPM) model proposed by Chung and Dunson (2009) to realize more ‡exible semiparametricrepresentations for IV. Nonparametrics based on Dirichlet process makes it
possible to represent a distribution by in…nite mixture of well-known “base”
distributions. Whereas the mean regression structure of the DPM is reduced
to a linear regression model, PSBPM is more ‡exible than DPM since it enables us to make a probability weight of the components change by predictors
in the regression model. Hence, we can treat reduced-form equation and thejoint distribution of error terms as potentially changing in shape as the valueof instruments vary. Even in the case that the reduced-form equation and theerror terms are truly linear and bivariate normal, respectively, our procedure
has small e¢ ciency loss, since it is not necessary to prespecify the numberof components. The optimal number of components, which is needed for a3

…nite mixture of regression models, is de…ned by the data.
We conduct a Monte Carlo simulation study in order to evaluate theperformance of the proposed method. We investigate the …nite sample performance of the estimators when the reduced-from model is not a simple
linear combination. The proposed method may incur as little as 1/30 of the
MSE incurred by existing procedures. Moreover, the coverage of nominal
95% con…dence (or credible) intervals of the proposed method is very close
to 0.95, whereas the other methods provide signi…cantly narrower or wider
interval estimates.
The proposed method is applied to a real Mendelian randomization dataset.
In general, the number of instrumental variables in Mendelian randomization(i.e. information of the genotype) is large, and correct speci…cation of the
reduced-form regression model is di¢ cult. In addition, the instruments may
not explain the endogenous variables satisfactorily, and non-Bayesian methods that rely on the asymptotic approximation yield biased results. Therefore, the proposed Bayesian nonparametric formulation for the reduced-form
equation is appropriate and results in obtaining e¢ cient endogenous (causal)
parameters. In fact, we provide statistically signi…cant results that are notobtained by the standard Bayesian IV approach.
The remainder of the paper is organized as follows. In the next section,
we propose and formulate a new semiparametric Bayes IV (SB-IV) methodthat can overcome the drawbacks of the existing IV methods. MCMC algorithm for our procedure is also presented in Section 2. The simulation
studies illustrating the performance of the proposed method compared withthe existing nonBayesian and Bayesian IV approaches are presented in Section 3. In Section 4, we apply our proposed method to the real dataset for
the Mendelian randomization. Section 5 concludes after providing a short
discussion.
2. Semiparametric Bayes estimation for instrumental variable
2.1 The linear model and the existing approaches
We consider a dataset consisting of N (i = 1; :::; N ) cases. We further
consider a case with one linear structural equation and one corresponding
reduced-form equation as follows:yi = xi + wi + 1;ixi = z i + 2;i(1)where yi is the outcome variable, xi is the endogenous regressor (E( 1;i xi ) 6=0), wi is the k-dimensional vector of exogenous regressors, and z i is the4m-dimensional vector of instruments (E( 1;i z i ) = 0) that includes wi . We
consider the situation that 1;i and 2;i are correlated, which causes the endogeneity problem. We call the …rst line in (1) structural equation and thesecond line reduced-form equation.
There is rich literature in econometrics that develops non-Bayesian andBayesian methods for IV model estimation. Two stage least squares (TSLS)(Theil, 1953, Basmann, 1957) and limited information maximum likelihood(LIML) (Anderson and Rubin, 1949, Hood and Koopmans, 1953) are themost frequently used non-Bayesian approaches to IV problems. These methods do not specify the distribution of( 1 ; 2 )t and are robust with respect
to the error distribution. However, these frequentist methods rely on asymptotics. This may be a problem when the sample size is not large enough.
Non-Bayesian methods for IV problems include nonparametric IV methods.
Newey and Powell (2003) proposed IV estimation of nonparametric models
that considers the structural equation to be the conditional expectation of
the endogenous variables. Later, Dreze (1976), and Kleibergen (1997) developed IV estimation methods from the Bayesian perspective. One of thedi¤erences between non-Bayesian and Bayesian methods for IV problems isthat non-Bayesian approaches do not rely on the assumption of some speci…c joint distribution regarding the error terms. By contrast, Bayesian approaches generally assume bivariate normal distribution (Chao and Phillips,
2011, Geweke, 1996, Kleibergen and Zivot, 2003, Rossi et al., 2005, Hoogerheide et al., 2007).
However, explicit distributional assumptions can cause misspeci…cation
due to heterogeneity or outliers of the data. To overcome this, Conley
et al. (2008) considered a more ‡exible semiparametric Bayesian model for
instrumental variables problems. They employed a Dirichlet process prior forthe error terms in the structural and reduced-form equations. More specifically, they considered i( 1;i ; 2;i )t
N (0; i ) instead of i
N (0; ),
where i are i.i.d. following a discrete random distribution G, and G ismodeled with concentration parameter and base distritbution G0 , that is,G
DP ( ; G0 ); namely, an in…nite mixture of normal distributions where
the number of components and the shape parameters are determined by the
speci…ed prior and the observed data. Their Monte Carlo simulation study
showed that when the error terms do not follow the normal distribution,
their method is more e¢ cient than existing Bayesian and non-Bayesian approaches.
Even though the Bayesian method in Conley et al. (2008) moderates theBayesian strong assumption on the distribution of the error terms, existing5

Bayesian and non-Bayesian methods, as well as Conley’s method, specify
some functional form, usually a linear combination, for the reduced-form
equation. In fact, the true relationship between endogenous variables andinstrumental variables is unknown. Therefore, existing methods impose verystrong assumptions on IV regression. In addition, we are not generally interested in the parameters of reduced-form models, since these models are just“…rst-step”regression models.
We develop a new more ‡exible semiparametric Bayes IV model where
we use parametric formulation in the structural equation, whereas we use
nonparametric formulation in the reduced-form equation and the joint distribution of the error terms. We note that we do not use nonparametricrepresentation in the structural equation, since its parameters, such as thegradient coe¢ cient of xi or wi , are usually of interest.
2.2 Semiparametric model formulation
We express the full model as the product of three submodels.
p(yi ; xi ; jwi ; z i ) / p(yi jxi ; wi ; #s ; )p(xi jz i ; #r ; )p( )where p(yi jxi ; wi ; #s ; ) represents the structural equation, p(xi jz i ; #r ; )represents the reduced form equation, and p( ) represents the joint (prior)
distribution of error terms covariance matrix. #s and #r denote the parameters of the structural equation and the reduced form equation except for, respectively. We assume the parametric model for the structural equationp(yi jxi ; wi ; #s ; ) since the researchers’concern generally lies in #s . On theother hand, we consider nonparametric structure for the reduced form equation p(xi jz i ; #r ; ) and p( ) because they are not generally interested in #r
or . Speci…cally,yi = xi + wi + 1;i; ixi = g (z i ; ) + 2;i

1;i

f

(2)

2;i

We use DPM representation for the reduced-form equation and the joint
error distribution. According to Sethuraman (1994), when the Dirichlet
process prior is assumed for a random distribution G, denoted by G
DP ( ; G0 ); G is expressed as follows:
G=

1
X

l

l

;

l

G0

l=1

where and G0 are the concentration parameter and the base distribution,
reQ
spectively. In this notation, is a point mass at and l = l h<l (1h ),6where 1 ; 2 ; ::: are independent draw from a beta distribution Be(1; ) (refer
to Walker et al. (1999) for a detailed description of DPM). These representations indicate that any distribution can be expressed as a mixture ofwell-known distributions, such as the Gaussian distribution.
Fortunately, the DPM model can be estimated with a relatively simpler
MCMC algorithm by applying blocked Gibbs sampling (Ishwaran and James,2011). Therefore, DPM modeling has been employed to avoid distributional
assumptions on the parameters within the Bayesian framework. In addition
to Conley et al. (2008), Hirano (2002) developed dynamic panel data models
where the error terms are represented by a DP prior. DPM models not only
treat disturbances of nonparametric classes but also a¤ord a good deal of‡exibility for a wide range of statistical models. Miyazaki and Hoshino (2009)
proposed a Bayesian semiparametric item response model with DP prior.
Kunihama and Dunson (2016) constructed a method for variable selection
within Bayesian nonparametric DPM. The theoretical properties of DPM
were investigated in Shen et al. (2013).
Even though DPM is often employed to avoid assumptions on parameter
distribution, it is restricted due to the fact that the probability weight l
is constant across observations (Dunson et al., 2007). If the stick-breaking
weights l are constant and independent of the predictor i ; the mean reP1
T
Tgression structurei,i =l=1 l lP1 is Treduced to be linear, namely,.
Therefore,
instead
of
DPM,
we
apply
the
PSBPM
where =
l=1 l lmodel proposed by Chung and Dunson (2009) since it can change probability weights depending on the value of predictors in the regression model.
In addition, since PSBPM uses Gaussian distribution as a base distribution,
the models a¤ords a conjugate structure and simpler posterior calculation.
PSBPM uses the probability weight l ( ) instead of l ,Y
f1( h ( ))g :(3)
l ( ) = ( l ( ))h<lwhere ( ) is the cumulative distribution function of the standard Gaussian
distribution. PSBPM allows the probability weight l ( ) to vary as thepredictors vary.
For example, Hoshino (2013) proposed a semiparametric Bayesian model
for causal inference where the irrelevant parameters are modeled using PSBPM. The statistical properties of PSBPM are described in Pati et al. (2013).
We apply PSBPM modeling to the reduced-form equation and the joint
error distribution. We rewrite the models as an in…nite mixture model ofsimultaneous equations.

7

fi =
f yei jX

where
yei =

yi
xi

fi =
;X

el =

1
X

l (z)N

l=1

fi el ;
X

xi w i 00 0 zi
1;l
2;l

; el

N (0;

l

0

; el = @
l ):

l

1

A;

We note that since we assume parametric speci…cation for the structural
equation, and are common parameters across classes, whereas l varies
with l. Since we speci…ed PSBPM formulation for the reduced-form equation,
we let the probability weight be dependent on the instruments z, l (z). Asin equation (3), l (z) is expressed by the following “stick-breaking”process.
Yf1( h (z))g :l (z) = ( l (z))
h<lWe employ the same structure of instruments depending weight l (z)as in Chung and Dunson (2009). That is, we let l (z) = l + fl (z), lN ( ; 1),N ( 0 ; 2 0 ) and we introduce the following regression function:fl (z) =mX
q=1

lq jzqlq j

PBq 1( lq ), where N ( U ; 2U )1U
with lq
N ( q ; 2 q )1R+ ; lq
q=1 Bqqb
denotes a normal distribution with mean U and variance 2U truncated tothe set U , and qb are discrete points over a reasonable range of the q-th
covariate zq .
Our semiparametric Bayes IV model in equation (2) is di¤erent fromthe recent nonparametric IV approaches that consider the nonparametricstructural equation. Such examples are Newey and Powell (2003), Hall andHorowitz (2005) and Darolles et al. (2011), which developed nonparametric IV regression models from non-Bayesian perspective, and Liao and Jiang(2011), Florens and Simoni (2012), and Kato (2013), which employed quasiBayesian approach to nonparametric Bayesian IV regression problem. On theother hand, our semiparametric approach employ nonparametric reducedform model and disturbances, which are di¢ cult to correctly prespecify, and8

parametric structural model, which is of main interest. Therefore, our semiparametric method is practical when we can use many instruments and especially interested in the parameters of the structural equation.
Some of non-Bayesian and quasi-Bayesian IV methods employ GMM(generalized method of moments) based approach, and it is well-known that
GMM attains semiparametric e¢ ciency bounds. However, this e¢ ciency is
obtained when the sample size is large enough. On the other hand, the proposed method is based on Bayesian modeling, and therefore expected to be
more e¢ cient than exisitng GMM-based IV methods relying on asymptotic
theory when the sample size is not large enough, as indicated by Conley et al.
(2008). In fact, we will show that our proposed method obtains more e¢ cient and smaller MSE (mean squared error) estimates than GMM-based IV
approach in the simulation study section when the sample size is not large.
2.3 MCMC algorithm
We apply the blocked Gibbs sampler (Ishwaran and James, 2011) tothe posterior computation of the PSBPM parameters. Ishwaran and James(2011) proved that the case with …nite number of classes l can be used toapproximate the inference that is based on in…nite classes with satisfactory
accuracy when the maximum number of classes L is large enough. Therefore, blocked Gibbs sampler considers the case for truncation of the numberof classes (e.g. L = 20), hence the simpler posterior computation. It is very
similar to the Gibbs sampler except for the assignment of samples to eachclass.
We provide the detailed posterior computation using the MCMC estimation as follows.
1. Conditional for Ki (i = 1; :::; N )Let K i be the indicator denoting where case i belongs, and K i = l if
casePi belongs to class l: To assign samples to each class, generate Ki
by Ll=1 li l ( ), where li is
li = XLf e l;l (z i )N (X ifel (z i )N (X i l ;

l=1

with

l (z i ) =( l (z i ))2. Conditional for Zil

Ql)h<l f1( h (z i ))g.

9

;
l)

The latent variable Zil is introduced in this step where Zil = 1(Zil > 0)
and
8
!
q
X
>
>
>
>
l
lq j ; 1 1R+ for l = Ki
lq jziq
< N
k=1
!
Zil
q
X
>
>
>
for l < Ki :>llq j ; 1 1R
lq jziq: N
k=1

3. Conditional for
Simulatel (l = 1; :::; Ll from the following normal distribution.

4. Conditional for

q

N

;

Nl + 1lq (l = 1; :::; L

1
Nl + 1;

1; q = 1; :::; m)P+ i:Ki l jziq
P2+
i:Ki l jziqqlq j Uilq

5. Conditional for
Simulate

i:Ki l Wil +lq from the following left-truncated normal distribution.
2

lq

P

N

l

Simulate

1)

2
lq j

lq (l = 1; :::; L

;

2

+
q

1; q = 1; :::; m)

1

P

2
lq j

i:Ki l jziq

!

1R+ ;

lq from the following probability.

Pr

lq =

1 Y
N Zil ;
Bq i:K l;1 ; 1iqb= B
Pq 1 YN Zil ;
q=1 Bq i:K l:;2 ; 1iwhere ;1 =Pqls=1;s6=klPqls jziss=1;s6=kls j

ls jzis
lq

ziq

ls j
qb

lq

ziq

qb

and

;2 =

:

6. Conditional for
SimulateNfrom the following normal distribution.
"L 1
#
X
2
(L 1 + 02 ) 1; (L 1 +
l +
0
0
l=1

10

2
0

) 1

!

:

7. Conditional for e lPe from the following multivariate
Let nl = N
li=1 1(K i = l): Simulate
normal and inverted Wishart distribution given Ki = l.
e

where
D l=

nl
X
i=1

and
l

IW

t

f
X
il

l

"n
l
X
i=1

8. Conditional for

N D ld l; D l

l

and

1f

X il

! 1

;d l =

i=1

fil e l
X

e il
y

nl
X

t

f
X
il

fil e l
X

e il
y

1
l

t

#

e il ;y; nl!

Consistent with Rossi et al. (2005), the structural model parametersand can be simulated by applying Bayesian linear regression draw
with N (0; 1) error termsyi

E [ 1;i j 2;i ]1;i j 2;i

=

xi

+

1;i j 2;i

wi

where
=
1;i j 2;i+ ui ;ui

N (0; 1)

1;i j 2;i

2
12;! i2
11;! i11;! iandE [ 1;i j 2;i ] =12;! i2;i22;! i3. Simulation study
We conduct two simulation studies in order to illustrate the performance ofthe proposed method when the reduced-form equation is not a linear combination with additive disturbance that follows Gaussian distribution. Simulation 1 considers the case where the reduced-form equation is a mixture
model of two classes (l = 2). Simulation 2 considers the case of a non-linear
reduced-form equation with skewly distributed (log-normally distributed) error terms. Each simulation is designed to cover the cases of weak instruments
and strong instruments, namely, our simulation results in a experiment withfour cells. We classify instruments as weak or strong according to the classical criteria proposed by Staiger and Stock (1997) and Stock and Yogo (2005).
11

These stipulate that when the F-statistics of the …rst stage regression is lessthan 10, we have a weak instrument case, otherwise we have a strong instrument case. We evaluate the proposed method by comparison with existing
non-Bayesian and Bayesian IV methods.
In the two simulation studies, we evaluate the …nite sample propertyof the estimators. We consider 1; 000 replications of the dataset and con…rm if the true value of endogenous variable coe¢ cient can be recovered.
Throughout the simulations, the sample size for each dataset is set to be
N = 200. We calculate standard performance measures, such as empiricalmean, standard deviation, the coverage of nominal 95% con…dence (or credible) intervals (CI) of the estimate, and the deviation (MSE) from the true
value. Moreover, we con…rm, if the point estimate of is positively statistically signi…cant at 95% con…dence level for non-Bayes approaches, andthe 95% credible interval of includes “zero”for Bayes approaches (denoted
as “% of sig.” in Table 1). Although it could be sometimes inappropriate,
we take more practical perspective to compare the Bayesian credible intervals with the classical con…dence intervals as Conley et al. (2008) did. The
detailed experimental design is described in Supplementary Materials.
For comparison, we consider …ve non-Bayesian estimators and two Bayesianestimators. The …rst is the most widely used approach, namely, two-stage
least squares (TSLS). The second alternative is limited information maximum likelihood (LIML). The third is a modi…cation of the LIML estimator,
namely, Fuller’s procedure (F-LIML) (Fuller, 1977), which obtains better estimates when the instruments are weak. We also emplyed GMM based IV
estimation (GMM-IV) as an alternative. Moreover, we compare the resultsfrom ordinary least squares (OLS). As for Bayesian alternatives, we employ
the standard Bayesian IV method, which assumes that the joint distributionof the error terms is bivariate Gaussian (BayesIV), and Conley’s semiparametric Bayesian IV method (C-BayesIV). For BayesIV and C-BayesIV, aswell as the proposed method, burn-in of 5,000 iterations followed by 10,000
iterations were used for the posterior inference. All algorithms except for the
proposed method and GMM-IV assume an ordinary linear regression modelfor the reduced-form equation, that is, xi = z i + 2;i : GMM-IV used themoment condition of E( 1;i z i ) = 0, therefore GMM-IV does not prespecify
the functional form of the reduced-form equation. We used inverse varianceof the moments as a weighting matrix, namely, the optimal weighting matrix. We calculated the intervals based on large sample approximations forthe classical estimators.
This section presents the results of simulation study 1. The results of12

simulation 2 are provided in Supplementary Materials.

Fig.1. The solid horizontal line is the “true”coe¢ cient value = 1. The boxes
span the range from the 25th to the 75th percentiles, and the whiskers extend toan area no more than 1.5 times the range from the 25th to the 75th percentiles
from the box. The circles above and below the whiskers represent outliers. OLS,
ordinary least squares; TSLS, two stage least squares; LIML, limited imformation
maximum likelihood; F-LIML, Fuller’s modiication of the LIML; BayesIV, standard Bayes IV method; C-BayesIV, Conley’s semiparametric Bayes IV method

13

Table 1 shows the results of simulation 1, including the empirical mean,
standard deviation, the coverage of nominal 95% con…dence intervals (CI) of
the estimate, the mean squared error (MSE) from the true value of , and“% of sig.”. Figure 1 shows the box plot of the estimates for obtained by
1; 000 replications.
For both the weak and the strong instrument cases, OLS obtain overestimated results, since they ignore the existence of the instrumental variable.
Therefore, the coverage of nominal 95% con…dence intervals for OLS is verypoor.
As for the weak instrument case, all estimation methods except for the
proposed and C-BayesIV seem to be biased due to the weakness of the instruments. Moreover, the standard deviations of the estimates are totallydi¤erent among the methods. The proposed method exhibits the best e¢ ciency (the smallest standard deviation). Even though GMM-IV does notprespecify the reduced-from regression model and the distribution of the error
terms, as is the case with the proposed, it shows the larger standard deviation
than our semiparametric approach. The best coverage is obtained by the proposed method, which is very close to 0:95. For other Bayes models, BayesIV
results in narrower credible intervals. Furthermore, C-BayesIV yields slightly
wider intervals. Regarding non-Bayes methods, TSLS, LIML, F-LIML, and
GMM-IV, show overly wider coverage and do not seem to obtain correctly
the con…dence intervals. These results indicate that weak instruments andmisspeci…cations of IV models provide incorrect CI. Therefore, wrong parametric assumptions on the IV equations may lead to poor prediction powerof the reduced-form equation and incorrect decision making in …elds where
results of statistical signi…cance (or hypothesis testing) are crucial. The …fth
and the sixth column shows the MSE from the true value of = 1 and the
MSE ratio when the MSE of the proposed method is set to be 1. It is seenthat the existing methods yield from 4:7 to 37:1 times larger MSE than the
proposed approach. All estimates from the proposed method do not contain
zero for its credible intervals, by contrast, “% of sig.” of the estimates fromthe other methods are only 60% to 91%.
As far as the strong instrument case is concerned, all estimation methods
except for OLS can recover the true value= 0:1, since they have consistency, even when they misspecify the reduced-form equation. However,e¢ ciency varies greatly among the methods. Hence, the MSE results are
dispersed. Speci…cally, except for OLS estimates of inconsistency, the meanof the standard deviations of the proposed method is the smallest, indicatingthat misspeci…cation of the reduced-form equation and the joint distribution

14

of the error terms results in less e¢ cient estimates. GMM-IV has the property of semiparametric e¢ ciency, but in this simulation study of …nite sample
with N = 200, GMM-based methodology seems to be less e¢ cient even in
the strong instruments case. Thus, our semiparametric speci…cation is moresuitable even in the strong instrument case. Existing methods yield at most
18:4 times larger MSE than the proposed. In addition, there is more than
92% chance that the estimates from the proposed method are “positively
signi…cant” (CIs do not cotain zero), whereas the estimates from the existing methods are signi…cant with probability less than 30%. This indicatesthat even in the case of strong instruments, the proposed semiparametricapproach can correctly detect signi…cant variables.
4. Real data analysis
Herein, we apply our methods to a real data example of Mendelian randomization. Mendelian randomization uses genotype information as an instrumental variable to correctly estimate the causal e¤ect of a biomarker to adisease. It is an outstanding IV approach in that the appropriate genetic
variant is independent of the confounders of the intermediate phenotypeoutcome association. Moreover, it can a¤ect the outcome only through thecausal intermediate phenotype as long as it is related to the intermediate
phenotype (Burgess et al., 2015).
The data used in this implementation are sourced from the Orkney Complex Disease Study (ORCADES) study, which is a genetic epidemiology study
based on an isolated population in the north of Scotland (http://www.orcades.
ed.ac.uk/orcades/). The data is open sourced and published in (http://www.
homepages.ed.ac.uk/pmckeigu/instrumental/). It is based on the dataset described in McKeigue et al. (2010).
The aim of the analysis is to uncover the causal relationship between
plasma urate level and the “metabolic syndrome”. There exist two con‡icting opinions on this issue. Feig et al. (2008) indicates that uric acid and themetabolic syndrome are positively related. Therefore, they are said to be
have positive association (Lohsoonthorn et al., 2006, Onat et al., 2006, Choi
and Ford, 2007, Hjortnaes et al., 2007, Sui et al., 2008). By contrast, there isindirect evidence in Waring et al. (2003) and Waring et al. (2006) supporting that uric acid is bene…cial in avoiding the metabolic syndrome. Using
Mendelian randomization with the dataset described in this section, McKeigue et al. (2010) showed the negative e¤ect of uric acid on the metabolic
syndrome. However, the e¤ect was statistically not signi…cant, with 95%
credible interval of the logistic regression coe¢ cient ( 2:91; 0:05).

15

In this example, plasma urate level is an endogenous variable and themetabolic syndrome is an outcome. As an instrumental variable for Mendelianrandomization, we can use six SNPs (single nucleotide polymorphisms) in
the urate transporter gene SLC2A9 which were typed using a TaqMan assay,
rs737267, rs13129697, rs1014290, rs6449213, rs13131257, and rs4447863. In
addition, the dataset include the information on age (average:53 years) andsex (40 % male). We include them in both the reduced-form and the structural equation. The plasma urate level is standardized to have mean 0 andstandard deviation 1. The age is also scaled to have mean 0.
The outcome, the metabolic syndrome, is a dichotomous variable (equals
one if the sample indicates metabolic syndrome, otherwise zero; 20 % of thesample is regarded as metabolic syndrome). Therefore, we consider the probit regression model for the structural equation. The original sample size is
1; 017, and the SNP information is available for 706 of these individuals. We
used a complete-case sample, that is, individuals for whom all variables, including the outcome (metabolic syndrome), the endogenous variable (plasma
urate level), the covariates (age and sex), and the instrumental variables (six
SNP) were observed. The sample size resulting from the complete-case analysis is 647.
For comparison, we …rst used OLS regression. The estimated coe¢ cient ofplasma urate level is 0:44 and is statistically signi…cant (p < 0:01). Moreover,
we considered the classical method for the IV probit model. In this case, theplasma urate level coe¢ cient is 0:67. However, this is not statistically signi…cant, with con…dence interval ( 1:52; 0:18). In addition, the F-statisticsof the reduced-form equation (regression model of plasma urate level against
six SNP) is 3:56. Hence, this is considered to be a weak instrument case.
Therefore, the results obtained from a non-Bayesian IV method, in which
the precision of the estimates depends on asymptotic approximation, are notreliable.
Subsequently, we applied the standard Bayesian IV method to the dataset.
We considered an ordinary linear regression model as the reduced-form modeland assumed that the distribution of the error terms is bivariate normal.
Burn-in of 5; 000 iterations followed by 10; 000 iterations were used for theposterior inference. The posterior mean of the coe¢ cient of plasma urate
level is 0:42, with credible interval of ( 0:98; 0:03).
Finally, we applied the proposed method. We considered a truncation of
the components L = 20. The prespeci…ed parameters were same as in the previous simulation study. Burn-in of 5; 000 iterations followed by 10; 000 iterations were used for the posterior inference. The posterior mean of the plasma

16urate level coe¢ cient is 0:48, with credible interval of ( 1:01; 0:02). Theinterval obtained from the proposed is shorter than the standard Bayesian
IV method and do not contain 0 in the interval. The results obtained fromthe proposed method are the opposite of OLS results, which do not consider
the presence of the instruments.
In line with McKeigue et al. (2010), this indicates that high level of uricacid is negatively related to the metabolic syndrome. However, the results
may be controversial, since they not consistent with an established theory
supported by a large number prior studies. This article focuses on statisticalmethods for IV rather than the relationship between the urate acid level
and the metabolic syndrome. The results, as well as the plausibility of theassumptions in the IV approach, should be veri…ed by additional studies.
5. Conclusion and discusions
We proposed a semiparametric Bayes instrumental variable estimation method.
Our method does not specify the distribution of the reduced-form and the
disturbances and is robust in the weak instrument case, as shown in thesimulation studies. Therefore, application in Mendelian randomization isbene…cial, since the instruments of genetic information are numerous andoccasionally explain only a small portion of the variation in the biomarker.
There are several directions for future research. We considered the caseof the structural equation is an ordinary linear regression model or probit
regression model. An extension to the logistic regression model or other
regression models is natural. As stated in Section 1, Bayesian methods areable to develop these models straightforwardly. In addition, further study
is required to improve the computational speed of the proposed algorithm.
Since our inference is based on the MCMC algorithm with nonparametric
Bayes parameters, the computation time required to obtain valid estimatesis greater than that of the existing non-Bayesian and Bayesian methods.
Supplementary Materials
1. MCMC algorithm for binary outcome caseWe can extend MCMC algorithm for binary outcome case. Consider thefollowing representation of probit model by latent variable yi :yi = xi + wi + 1;i; ixi = g (z i ; ) + 2;i
and
17

1;i
2;i

f

1; yi > 0
0
0; yi

yi =

Posterior simulation requires following 3 steps in addition to the main
MCMC computation described above, and replace yi in the algorithm bylatent variable yi .
1. Conditional for yi (i = 1; :::; N )Simulate yi from the following truncated normal distribution.
8< T N(0;1) xi + wi +

yi

where

: T N( 1;0) xi + wi +
1

l =

12;l

2. Conditional for
Simulate

12;l
2
22;l

12;l
2
22;l

[xi

12;l
2
22;l

zi l] ; 1[xizi l] ; 1

12;l (l = 1; :::; L)12;l from the following normal distribution.

N (D d ; D );Pnl
1
xi + wi )2 = 2v;l + V0;l1;
where D =
i=1 (yiPnl
d = i=1 (yi xi + wi ) (xi z i l )= 2v;l +V0;l1 0;l , and2
N ( 0;l ; V0;l ).
12;l , and the prior for 12;l
3. Conditional for

2
v;l

IG

;

:

12;l

Simulate

2
12;l
2
22;l
2
12;l
2
22;l

2
v;l =

2
22;l

2
v;l (l = 1; :::; L)2v;l from the following inverse Gamma distribution.
nl
1Xnl+ a; b 1 +
fxi
2
2 i=1

with the prior for

2
v;l

zi l

12;l (yi!

xi + wi )g ;IG(a; b).

2. Detailed simulation design and results
Herein, we describe in detail the simulation study mentioned in Section 3.
18

2.1 Experimental design
Throughout the simulation, we consider the case of linear structural equation with one endogenous variable. This experimental design is justi…ed by
the survey conducted in Chernozhukov and Hansen (2008); they noticed thatin the leading journals of economics from 1999 to 2004, 89 out of 129 articles
regarded only one explanatory variable as endogenous.
Figure 2 shows boxplots of the F-statistics distribution of simulation 1
and simulation 2 for the weak and strong instrument case. We classify instruments as weak or strong according to the classical criteria proposed by
Staiger and Stock (1997) and Stock and Yogo (2005) as noted in the main
paper.
Through the simulation studies, we adopt the same default choices for
hyperparameters as in Chung and Dunson (2009), namely, L = 20, 0 = 0,2
= 1, q = 0, 2 q = 100, and qb are 50 equally spaced grid points in0( 3:5; 3:5):

Fig.2. The boxes span the range from the 25th to the 75th percentiles, andthe whiskers extend to an area no more than 1.5 times the range from the 25th tothe 75th percentiles from the box. Sim.1 weak, simulation 1 for weak instruments;
Sim.1 strong, simulation 1 for strong instruments; Sim.2 weak, simulation 2 forweak instruments; Sim.2 strong, simulation 2 for strong instruments

2.2 Setup for simulation study 1
In this study, we evaluate the …nite sample property of the estimators
when the true reduced-form equation is a …nite mixture regression model.
We consider the case with a sample size N = 200 and a mixture model withtwo classes (l = 2). We assume 1-dimensional endogenous regressor xi , 3dimensional exogeneous regressor wi , whose …rst elements are set to be 1 (i.e.
19intercept), and 5-dimensional instruments z i , whose …rst elements are set to
be 1 and are independent of 2;i . The true relationship of the full model is
as follows.
8< yi = xi + wi + 1;i
1;i
2
P
; i
N (0; l )(.1)
1i2l z i l + 2;i: xi =2;il=1

1; 2; 2; 0; 0)t ;
1 0:5
2 1
t
= (1; 2; 3)t ; ( 1 ; 2 ) =;;
2 = ( 1; 2; 2; 0; 0) ;
0:5 1
1 2
and the interested parameter = 1: We generate the exogeneous random
0
1 0:5
variables from w 1;i M V N;; where w 1;i denote0
0:5 1:5
the components of wi with the …rst component 1 removed. As for z i , we0
1 0:5
…rst generate z 1;i M V N;; where z 1;i denotes0
0:5 1
the components of z i without the …rst component 1 and the component ofwi , and set z i to be z i = 1; z 1;i ; wi :For the case of strong instruments with small coe¢ cients, we replace lwith 1 = ( 1; 2; 2; 0; 0)t and 2 = ( 1; 1; 0; 0; 0)t and the parameter ofinterest with = 0:1. The other true parameters remain unchanged.
In this simulation setup, the mean of the F-statistics of the reducedform model for the weak and the strong instrument case is 2:40 and 29:11,
respectively.
The results are provided in the main article.
For weak instrument case, we set ( 1 ;

2 ) = (0:5; 0:5);

1 = (2.3 Setup for simulation study 2
In this study, we consider the case with a sample size N = 200, a nonlinear reduced-form equation, and skewly distributed (log-normally distributed) error terms. We assume 1-dimensional endogenous regressor xi , 3dimensional exogeneous regressor wi whose …rst elements are set to be 1
(i.e., intercept), and 4-dimensional instruments z i that are independent of
2;i . The true relationship of the model is as follows.
yi = xi + wi + 1;ixi = 2 ln z 1;i 1 + 1 sgn z 1;i

1 + 2;i; i1;i= v c; v2;i(.2)
where sgn( i ) takes the value 1; 0; 1 if i is positive, zero, and negative,
respectively. c is taken so that the mean of i is zero. In this setting,there is S-shaped relationship between xi and z 1;i . Again, z 1;i denotes20

ln N (0; )the components of z i with the …rst component 1 and the component of wi
removed. Hence, in this data generating process, z 1;i is one dimension.
1 0:5
For weak instrument case, we set = (1; 2; 3)t ; =; and
0:5 2:5
the interested parameter = 1: We generate the exogenous random variables
0
1 0:5
from wiMV N;; z 1;iN (0; 1) : For strong in0
0:5 1:5
1 0:5
strument with small coe¢ cient case, we replace with =0:5 1:25
and the interested parameter with = 0:3. The other true parameters areremained to be unchanged.
In this simulation setup, the mean of the F-statistics of the reducedform model for the weak and the strong instrument case is 5:13 and 29:85,
respectively.
2.4 Results of Simulation study 2
Table 2 shows the results of simulation 2. It is seen that for both cases,
OLS obtains overestimated results. Therefore, the coverage of nominal 95%
con…dence intervals is very poor.
Regarding the weak instrument case, all estimation methods except forthe proposed approach are unstable. Speci…cally, the standard deviationof the estimates is larger than that of the proposed approach. The bestcoverage is obtained by the proposed method. This is very close to 0:95.
As for the other Bayes models, BayesIV and C-BayesIV obtain somewhat
narrower credible intervals. By contrast, non-Bayes approaches again yieldoverly wider credible intervals. These results also indicate that weak instruments and misspeci…cations of IV models yield incorrect CI. The results on
the MSE and the MSE ratio show that existing methods yield from 5:6 to 36:7
times larger MSE than the proposed approach. In addition, “% of sig.” ofthe existing methods are only with probability 86% at minimum, whereas all
estimates of from the proposed method do not include zero for its credibleintervals.
As for the strong instrument case, all estimation methods except for OLS
can recover the true value = 0:3. However, e¢ ciency varies among themethods. More speci…cally, except for OLS, the mean of the standard deviations of the proposed model is the smallest. In the simulation 2 settings,
which consider the non-normal disturbances with …nite samples, GMM-IV
does not outperform the proposed for both weak and strong instruments
case even though it do not require the normality of the error terms. Additionally, “% of sig.” of the proposed method is 96%, whereas those of the21competitors are only 39% to 59%.

Acknowledgements
Funding: This work was supported by JSPS KAKENHI Grant Number
JP26285151.

References
Anderson, T. W. and Rubin, H. (1949). Estimation of the parameters of asingle equation in a complete system of stochastic equations. Ann. Math.
Stat. 20, 46–63.
Baiocchi, M., Cheng, J. and Small, D. S. (2014). Instrumental variable
methods for causal inference. Stat. Med. 33, 2297–2340.
Basmann, R. L. (1957). A generalized classical method of linear estimationof coe¢ cients in a structural equation. Econometrica 25, 77–83.
Burgess, S., Scott, R. A., Timpson, N. J., Smith, G. D., Thompson, S. G.
and Consortium, E.-I. (2015). Using published data in mendelian randomization: a blueprint for e¢ cient identi…cation of causal risk factors.
Eur. J. Epidemiol. 30, 543–552.
Chao, J. and Phillips, P. (2011). Posterior distributions in limited information
analysis of the simultaneous equations model using the je¤reys prior. J.
Econometrics 87, 49–86.
Chernozhukov, V. and Hansen, C. (2008). The reduced form: A simple
approach to inference with weak instruments. Econ. Lett. 100, 68–71.

22

Chib, S. (2007). Analysis of treatment response data without the joint distribution of potential outcomes. J. Econometrics 140, 401–412.
Choi, H. K. and Ford, E. S. (2007). Prevalence of the metabolic syndrome
in individuals with hyperuricemia. Am. J. Med. 120, 442–447.
Chung, Y. and Dunson, D. B. (2009). Nonparametric Bayes conditional
distribution modeling with variable selection. J. Am. Statist. Ass. 104,
1646–1660.
Conley, T. G., Hansen, C. B., McCulloch, R. E. and Rossi, P. E. (2008). A
semi-parametric bayesian approach to the instrumental variable problem.
J. Econometrics 144, 276–305.
Darolles, S., Fan, Y., Florens, J. P. and Renault, E. (2011). Nonparametric
instrumental regression. Econometrica 79, 1541–1565.
Dreze, J. H. (1976). Bayesian limited information analysis of the simultaneous
equations model. Econometrica 44, 1045–1075.
Dunson, D. B., Pillai, N. and Park, J. H. (2007). Bayesian density regression.
J. Roy. Stat. Soc. B Met. 69, 163–183.
Feig, D. I., Kang, D. H. and Johnson, R. J. (2008). Uric acid and cardiovascular risk. N. Engl. J. Med. 359, 1811–1821.
Florens, J. P. and Simoni, A. (2012). Nonparametric estimation of an instrumental regression: A quasi-bayesian approach based on regularized
posterior. J. Econometrics 170, 458–475.
Fuller, W. A. (1977). Some properties of a modi…cation of the limited information estimator. Econometrica 45, 939–953.
Geweke, J. (1996). Bayesian reduced rank regression in econometrics. J.
Econometrics 75, 121–146.
Hall, P. and Horowitz, J. L. (2005). Nonparametric methods for inference inthe presence of instrumental variables. Ann. Stat. 33, 2904–2929.
Hirano, K. (2002). Semiparametric Bayesian inference in autoregressive panel
data models. Econometrica 70, 781–799.
Hjortnaes, J., Algra, A., Olijhoek, J., Huisman, M., Jacobs, J., van derGraaf, Y. and Visseren, F. (2007). Serum uric acid levels and risk for
vascular diseases in patients with metabolic syndrome. J. Rheumatol 34,
1882–1887.
Hood, W. and Koopmans, T. C. (1953). Studies in econometric method. John
Wiley and Son, New York.
Hoogerheide, L., Kleibergen, F. and Van Dijk, H. (2007). Natural conjugate priors for the instrumental varaibles regression model applied to theangrist-krueger data. J. Econometrics 138, 63–103.
Hoshino, T. (2013). Semiparametric Bayesian estimation for marginal para-

23

metric potential outcome modeling: Application to causal inference. J.
Am. Statist. Ass. 108, 1189–1204.
Ishwaran, H. and James, L. F. (2011). Gibbs sampling methods for stickbreaking priors. J. Am. Statist. Ass. 96, 161–173.
Kato, K. (2013). Quasi-bayesian analysis of nonparametric instrumental variables models. Ann. Stat. 41, 2359–2390.
Kleibergen, F. and Zivot, E. (2003). Bayesian and classical approaches to
instrumental variable regression. J. Econometrics 114, 29–72.
Kleibergen, F. R. (1997). Equality restricted random variables: Densities
and sampling algorithms. Econometric Institute Research Papers, No. EI
9662-/A .
Kunihama, T. and Dunson, D. B. (2016). Nonparametric bayes inference on
conditional independence. Biometrika 103, 35–47.
Liao, Y. and Jiang, W. (2011). Posterior consistency of nonparametric conditional moment restricted models. Ann. Stat 39, 3003–3031.
Lohsoonthorn, V., Dhanamun, B. and Williams, M. A. (2006). Prevalenceof hyperuricemia and its relationship with metabolic syndrome in thaiadults receiving annual health exams. Arch. Med. Res. 37, 883–889.
McKeigue, P. M., Campbell, H., Wild, S., Vitart, V., Hayward, C., Rudan,
I., F., W. A. and Wilson, J. F. (2010). Bayesian methods for instrumental
variable analysis with genetic instruments (‘mendelian randomization’):example with urate transporter slc2a9 as an instrumental variable fore¤ect of urate levels on metabolic syndrome. Int. J. Epidemiol. 39, 375–
393.
Miyazaki, K. and Hoshino, T. (2009). A bayesian semiparametric item response model with dirichlet process priors. Psychometrika 74, 375–393.
Newey, W. K. and Powell, J. L. (2003). Instrumental variable estimation of
nonparametric models. Econometrica 71, 1565–1578.
Onat, A., Uyarel, H., Hergenş, G., Karabulut, A., Albayrak, S., Sar¬, I.,
Yazici, M. and Keleş, I. (2006). Serum uric acid is a determinant of
metabolic syndrome in a population-based study. Am. J. Hypertens. 19,
1055–1062.
Pati, D., Dunson, D. B. and Tokdar, S. T. (2013). Posterior consistency inconditional distribution estimation. J. Multivariate Anal. 116, 456–472.
Ramsahai, R. R. and Lauritzen, S. L. (2011). Likelihood analysis of thebinary instrumental variable model. Biometrika 98, 987–994.
Rossi, P. E., Allenby, G. M. and McCulloch, R. (2005). Bayesian statistics
and marketing. Wiley, NewYork.
Sethuraman, J. (1994). A constructive de…nition of dirichlet priors. Stat.

24

Sinica 4, 639–650.
Shen, W., Tokdar, S. T. and Ghosal, S. (2013). Adaptive Bayesian multivariate density estimation with dirichlet mixtures. Biometrika 100, 623–640.
Staiger, D. O. and Stock, J. H. (1997). Instrumental variables regression withweak instruments. Econometrica 65, 557–586.
Stock, J. and Yogo, M. (2005). Testing for weak instruments in linear iv
regression. Andrews DWK Identi…cation and Inference for Econometric
Models pages 80–108.
Sui, X., Church, T. S., Meriwether, R. A., Lobelo, F. and Blair, S. N. (2008).
Uric acid and the development of metabolic syndrome in women and men.
Metabolism 57, 845–852.
Theil, H. (1953). Estimation and simultaneous correlation in complete equation systems. North-Holland Publishing Co., Amsterdam.
Walker, S. G., Damien, P., Laud, P. W. and Smith, A. F. (1999). Bayesiannonparametric inference for random distributions and related functions.
J. Roy. Stat. Soc. B Met. 61, 485–527.
Wang, L., Robins, J. M. and Richardson, T. S. (2017). On falsi…cation of
the binary instrumental variable model. Biometrika 104, 229–236.
Waring, W. S., Convery, A., Mishra, V., Shenkin, A., Webb, D. J. andMaxwell, S. R. J. (2003). Uric acid reduces exercise-induced oxidative
stress in healthy adults. Clin. Sci. 105, 425–430.
Waring, W. S., McKnight, J. A., Webb, D. J. and Maxwell, S. R. (2006).
Uric acid restores endothelial function in patients with type 1 diabetes
and regular smokers. Diabetes 55, 3127–3132.

25