# A Measure of Uncertainty regarding the Interval Constraint of Normal Mean Elicited by Two Stages of a Prior Hierarchy

Authors: Hea-Jung Kim
Journal: The Scientific World Journal
 (2014)

---

## Abstract

This paper considers ahierarchical screened Gaussian model (HSGM) for Bayesian inference of normal models when an interval constraint in the mean parameter space needs to be incorporated in the modeling but when such a restriction is uncertain. An objective measure of the uncertainty, regarding the interval constraint, accounted for by using the HSGM is proposed for the Bayesian inference. For this purpose, we drive a maximum entropy prior of the normal mean, eliciting the uncertainty regarding the interval constraint, and then obtain the uncertainty measure by considering the relationship between the maximum entropy prior and the marginal prior of the normal mean in HSGM. Bayesian estimation procedure of HSGM is developed and two numerical illustrations pertaining to the properties of the uncertainty measure are provided.


---

## Body

## 1. Introduction

Consider the following model for normally distributed data:(1)
y
i
=
θ
+
ϵ
i
,
ϵ
i
~
N
(
0
,
τ
2
)
,
i
=
1
,
…
,
n
.
Bayesian analysis of the model (1) begins with the specification of prior distributions for unknown parameters θ and the noise variance τ
2. Specifically, we assign a normal prior distribution for θ and an inverse gamma (IG) prior for τ
2, that is, θ
~
N
(
μ
,
σ
2
) and τ
2
~
IG
(
c
,
d
), which are commonly used in a normal model as conjugate priors, where all the hyperparameters μ and σ
2,  c, and d are assumed to be known in the first place.On the other hand, when we are completely sure about a functional constraint ofθ a priori; a suitable restriction on the parameter space Θ
=
R, such as using a truncated normal distribution, is expected. However, it is often the case that the actual observations of (1) may violate the constraint on account of the measurement error or due to some other reasons. Further, the data may provide strong evidence that the constraint is inappropriate and therefore may appear to contradict the theory associated with the constraint. In this respect, it is expected that the uncertainty about the constraint is taken into account in the estimation procedure. O’Hagan and Leonard [1] indeed proposed two stages of a prior hierarchy based on the truncated prior distribution, which reflects the uncertainty about the parameter constraint. Liseo and Loperfido [2], Kim [3], and Kim and Choi [4] among others considered the Bayesian estimation of normal models with uncertain interval constraints using the idea of two stages of a prior hierarchy. In particular, Kim [3] obtained the marginal prior of θ as the normal selection distribution (e.g., [5]) and thus exploited the class of weighted normal distribution by Kim [6] for reflecting the uncertain prior belief on θ.Although the two-stage prior is applied by many investigators, there is no objective method to measure (or control) the degree of uncertainty regarding the interval constraint ofθ accounted for by using the two stages of a prior hierarchy. This is a major hindrance factor in developing the idea of the two stages of a prior hierarchy which is advocated by O’Hagan and Leonard [1]. Thus, such practical problem motivates us to develop a formal measure of uncertainty about the constraint in order to show how the uncertainty of the prior information regarding interval constrained parameter space Θ of θ can be reflected by utilizing the two stages of a prior hierarchy. This topic is tackled in this paper.To propose the uncertainty measure, we consider the Bayesian inference of the normal meanθ in (1), but subject to an uncertain interval constraint. Because the maximum entropy prior by Jaynes [7, 8] is useful for describing (or measuring) the relative levels of uncertainty about the distribution of the prior parameter, our investigation focuses on the theoretical relationship between the two stages of a prior hierarchy of θ by O’Hagan and Leonard [1] and the maximum entropy prior subject to an uncertain interval constraint. The remainder of this paper is organized as follows. In Section 2, we briefly discuss the two-stage prior of θ, which will be used for the Bayesian analysis of θ subject to uncertainty regarding the interval constraint. Accordingly, influenced by the seminal work of O’Hagan and Leonard [1], we provide a normal model based on the two-stage prior distribution of θ, referred to ashierarchical screened Gaussian model (HSGM). In Section 3, we explore the theoretical properties of the two-stage prior of θ by using Boltzmann’s maximum entropy theorem [9, 10]. Based on the properties, we propose an objective measure of uncertainty regarding the interval constraint of θ that is accounted for by the two-stage prior. In Section 4, we explore Bayesian estimation procedure by analytically deriving the posterior distribution of the unknown parameters under HSGM, and we discuss the properties of the proposed measure of uncertainty that can be explained in the context of HSGM. Finally, the concluding remarks along with a discussion are made in Section 5.


## 2. Hierarchical Screened Gaussian Model

Let us assume the normal model (1) and consider the two stages of a prior hierarchy in the following way:
(2)
y
i
=
θ
+
ϵ
i
,
ϵ
i
~
N
(
0
,
τ
2
)
,
i
=
1
,
…
,
n
,
τ
2
~
IG
(
c
,
d
)
,
θ
∣
θ
0
~
N
(
θ
0
,
σ
0
2
)
,
independent
of
(
ϵ
1
,
…
,
ϵ
n
)
⊤
,
θ
0
~
N
(
a
≤
θ
0
≤
b
)
(
μ
,
σ
1
2
)
,
where N
(
a
≤
μ
≤
b
)
(
μ
,
σ
1
2
) denotes a doubly truncated N
(
μ
,
σ
1
2
) distribution with the lower truncation point a and upper truncation point b. In practice, there are certain cases in which we have a priori information that θ is highly likely to have an interval constraint, and thus the value of θ needs to be located with uncertainty in a restricted space Θ,
(3)
Θ
=
{
θ
;
a
≤
θ
≤
b
}
or
,
simply
,
Θ
=
[
a
,
b
]
.In order to elicit the prior distribution on the uncertain interval constraint, we utilize the two-stage hierarchical model as in (2), which was initially advocated by O’Hagan and Leonard [1] in which values of θ that do not belong to Θ are penalized to a lesser extent. In this respect, the normal model structure of (2) is referred to as HSGM in the remainder of the paper, because the two stages of a prior hierarchy on θ are considered and the resulting marginal prior distribution of θ becomes the weighted normal (or interval screened normal) distribution studied by Kim [6]. This is shown as follows. Since θ
0
~
N
(
a
,
b
)
(
μ
,
σ
1
2
), the marginal prior of θ under HSGM is
(4)
p
(
θ
)
=
(
ϕ
(
θ
;
μ
,
ω
2
)
{
Φ
(
ξ
v
1
(
a
)
-
ω
2
η
(
θ
-
μ
)
)
Φ
(
ξ
v
1
(
b
)
-
η
(
θ
-
μ
)
)
v
v
v
v
v
v
v
v
v
v
v
{
Φ
(
ξ
v
1
(
b
)
-
η
(
θ
-
μ
)
)
-
Φ
(
ξ
v
1
(
a
)
-
ω
2
η
(
θ
-
μ
)
)
}
ω
2
)
ϕ
×
(
Φ
(
v
1
(
b
)
)
-
Φ
(
v
1
(
a
)
)
)
-
1
,
θ
∈
R
,
where ϕ
(
·
;
μ
,
w
2
) and Φ
(
·
), respectively, denote the density of N
(
μ
,
w
2
) and the distribution function of N
(
0,1
)
,
ω
2
=
σ
0
2
+
σ
1
2
,
v
1
(
a
)
=
(
a
-
μ
)
/
σ
1
,
v
1
(
b
)
=
(
b
-
μ
)
/
σ
1
,
ξ
=
ω
/
σ
0 and η
=
ξ
σ
1
/
ω
2.We see that (4) is the density of a weighted normal (WN) distribution by Kim [6]. This leads to the following assertion in Lemma 1.Lemma 1.
The two-stage prior ofθ in HSGM of (2), eliciting the uncertain interval constraint (3), is marginally distributed as a weighted normal distribution,
(5)
θ
~
W
N
(
a
,
b
)
(
μ
,
Ω
)
with its density (4), where
(6)
μ
=
(
μ
μ
)
,
Ω
=
(
σ
1
2
σ
1
2
σ
1
2
σ
0
2
+
σ
1
2
)
.Note thatθ
=
d
[
V
∣
a
≤
U
≤
b
], where (
U
,
V
)
⊤
~
N
2
(
μ
,
Ω
), a bivariate normal distribution. See Kim [6] for various properties of the W
N
(
a
,
b
)
(
μ
,
Ω
) distribution. According to O’Hagan and Leonald [1], the first stage variance, σ
0
2, of the mean may measure the degree of uncertainty in the constraint. However, there is no systematic method to assign the values of σ
0
2 (or σ
1
2), according to a priori specified degree (say, (
1
-
α
)
×
100
%
) of uncertainty about the interval constraint (3). This is a major hindrance factor in developing the HSGM.


## 3. The Measure of Uncertainty

### 3.1. A Maximum Entropy Prior

Sometimes we have a situation where partial prior information is available, outside of which it is desired to use a priori that is as noninformative as possible. A useful method of dealing with this problem is through the concept of entropy by Jaynes [7, 8]. As discussed by Rosenkrantz [11], entropy has a direct relationship to information theory and in a sense measures the amount of uncertainty inherent in the probability distribution.Assume now that we can specify the partial information concerning a location parameterθ (including the normal mean) with continuous space Θ of the form
(7)
E
[
t
j
(
θ
)
]
=
∫
Θ
‍
t
j
(
θ
)
π
(
θ
)
d
θ
=
t
j
,
j
=
1
,
…
,
k
,
but with nothing else about our prior distribution π
(
θ
). Then the maximum entropy prior can be obtained by choosing π
(
θ
) that maximizes the entropy
(8)
ξ
(
π
)
=
-
∫
Θ
‍
π
(
θ
)
log
⁡
π
(
θ
)
d
θ
,
in the presence of the partial information in the form of (7). A straightforward application of the calculus of variation leads us to Boltzmann’s maximum entropy theorem. This tells us that the density π
(
θ
) that maximizes ξ
(
π
), subject to the constraints E
[
t
j
(
θ
)
]
=
t
j
,
j
=
1
,
…
,
k, takes the k-parameter exponential family form
(9)
π
(
θ
)
∝
exp
⁡
{
λ
1
t
1
(
θ
)
+
λ
2
t
2
(
θ
)
+
⋯
+
λ
k
t
k
(
θ
)
}
,
θ
∈
Θ
,
where λ
1
,
λ
2
,
…
,
λ
k can be determined, via the k-constraints, in terms of t
1
,
…, t
k. See Leonard and Hsu [12] for the proof.


### 3.2.  Maximum Entropy Prior for Constrained Normal Mean

Let the location parameterθ of our interest be the normal mean in (1). Then the results of the previous subsection can be applied to the prior for the normal mean θ. This subsection considers the case where the partial priori information of θ is in the form of an interval, that is, θ
∈
Θ
,
Θ
=
[
a
,
b
] with a
<
b, and examines how the maximum entropy prior of θ, that is, π
(
θ
), has different formula according to the degree of uncertainty regarding the interval constraint.Case 1.
Θ
=
R, and θ
∈
Θ with certainty.When we have partial priori information aboutθ that we can specify values for both mean μ and variance σ
2. Then the N
(
μ
,
σ
2
) prior specification is the maximum entropy prior for θ (e.g., [12]). Thus the prior density and its entropy for the Case 1 are
(10)
π
1
(
θ
)
=
ϕ
(
θ
;
μ
,
σ
2
)
,
θ
∈
R
,
Ent
(
π
1
(
θ
)
)
=
-
∫
Θ
‍
ϕ
(
θ
;
μ
,
σ
2
)
ln
⁡
ϕ
(
θ
;
μ
,
σ
2
)
d
θ
=
1
2
(
1
+
ln
⁡
(
2
π
)
+
ln
⁡
σ
2
)
,
respectively.Case 2.
Θ
=
[
a
,
b
], and θ
∈
Θ with certainty.On the other hand, when we are completely sure about the priori interval constraint ofθ, a suitable restriction on the parameter space Θ such as using a truncated distribution is expected. This case supposes that we can specify values for both E
[
θ
]
=
μ and E
[
(
θ
-
μ
)
2
]
=
σ
2 on the space θ
∈
R by a priori information. Further suppose that we have certain prior information that the parameter θ is concentrated on the region [
a
,
b
], that is,
(11)
p
(
a
≤
θ
≤
b
)
=
1
but nothing else about our prior distribution π
(
θ
). The last condition is equivalent to E
[
I
(
a
≤
θ
≤
b
)
]
=
1. Therefore, by Bolzmann’s maximum entropy theorem, the prior density of θ for the Case 2 is
(12)
π
2
(
θ
)
∝
exp
⁡
{
λ
1
t
1
(
θ
)
+
λ
2
t
2
(
θ
)
+
λ
3
t
3
(
θ
)
}
∝
exp
⁡
⁡
{
[
θ
-
(
μ
-
λ
1
/
2
λ
2
)
]
2
λ
2
-
1
}
×
exp
⁡
⁡
{
λ
3
I
(
a
≤
θ
≤
b
)
}
by (9), because t
1
(
θ
)
=
θ
,
t
1
=
μ
,
t
2
(
θ
)
=
(
θ
-
μ
)
2
,
t
2
=
σ
2, and t
3
(
θ
)
=
I
(
a
≤
θ
≤
b
). Since t
3
=
1, the maximum entropy prior, subject to these three restrictions, is thus
(13)
π
2
(
θ
)
=
ϕ
(
θ
;
μ
,
σ
2
)
Φ
(
v
(
b
)
)
-
Φ
(
v
(
a
)
)
,
a
≤
θ
≤
b
,
provided that we choose λ
1
=
0
,
λ
2
=
-
1
/
2
σ
2
,
λ
3
=
-
ln
⁡
{
Φ
(
v
(
b
)
)
-
Φ
(
v
(
a
)
)
}, where v
(
a
)
=
(
a
-
μ
)
/
σ and v
(
b
)
=
(
b
-
μ
)
/
σ.Some algebra using the moments of theN
(
a
,
b
)
(
μ
,
σ
2
) distribution in Johnson et al. [13] yields the entropy of π
2
(
θ
) which is
(14)
Ent
(
π
2
(
θ
)
)
=
1
2
(
1
+
ln
⁡
⁡
(
2
π
σ
2
)
+
v
(
a
)
ϕ
(
v
(
a
)
)
-
v
(
b
)
ϕ
(
v
(
b
)
)
Φ
(
v
(
b
)
)
-
Φ
(
v
(
a
)
)
)
+
ln
⁡
(
Φ
(
v
(
b
)
)
-
Φ
(
v
(
a
)
)
)
.Case 3.
Θ
=
[
a
,
b
], and θ
∈
Θ with (
1
-
α
)
×
100
% uncertain.Now suppose that we have partial priori information that we can specify values for both meanμ and variance σ
2 of θ for θ
∈
R. In addition, suppose that we have priori uncertain interval constraint information that θ
∈
[
a
,
b
]. Then along with the priori moment conditions E
[
θ
]
=
μ and E
[
(
θ
-
μ
)
2
]
=
σ
2, the additional uncertain prior information about the interval constraint can be expressed by
(15)
p
(
a
≤
θ
≤
b
)
=
α
,
0
≤
α
≤
1
,
where 1
-
α (or (
1
-
α
)
×
100
%) denotes the degree of uncertainty. Thus the priori uncertain interval constraint is equivalent to
(16)
E
[
I
(
a
≤
θ
≤
b
)
]
=
α
.
Applying this partial prior information to Bolzmann’s maximum entropy theorem, we have the following lemma.Lemma 2.
Ifα
=
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
/
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
), the maximum entropy prior distribution of θ, reflecting the uncertain interval constraint in (15) is
(17)
θ
~
W
N
(
a
,
b
)
(
μ
,
Σ
)
,
where v
*
(
a
)
=
(
a
-
μ
)
/
(
σ
δ
), v
*
(
b
)
=
(
b
-
μ
)
/
(
σ
δ
), a
=
(
a
,
a
)
⊤
,
b
=
(
b
,
b
)
⊤
,
Φ
¯
2
(
(
a
,
b
)
:
μ
,
Σ
) denote a rectangle probability p
(
a
≤
X
2
≤
b
,
a
≤
X
1
≤
b
) of X
1 and X
2 whose joint distribution is a bivariate normal N
2
(
μ
,
Σ
),
(18)
μ
=
(
μ
μ
)
,
Σ
=
σ
2
(
δ
δ
δ
1
)
,
0
<
δ
<
1
.Proof.
Takingt
1
(
θ
)
=
θ
,
t
1
=
μ
,
t
2
(
θ
)
=
(
θ
-
μ
)
2
,
t
2
=
σ
2
,
t
3
(
θ
)
=
I
(
a
≤
θ
≤
b
), and t
3
=
α, and setting λ
1
=
0 and λ
2
=
-
1
/
2
σ
2, the maximum entropy prior π
(
θ
) in (9) reduces to
(19)
π
3
(
θ
)
∝
exp
⁡
{
(
θ
-
μ
)
2
2
σ
2
}
exp
⁡
{
λ
3
t
3
(
θ
)
}
,
θ
∈
R
,
by Bolzmann’s maximum entropy theorem. Now the second exponential term in the right hand side of (19) can be determined by using the condition (16) as follows. Among all the possible proper prior densities of the form (19), the choice of
(20)
λ
3
t
3
(
θ
)

=

ln
⁡
{
Φ
(
ξ
*
v
*
(
b
)

-

η
*
(
θ
-
μ
)
)

-

Φ
(
ξ
*
v
*
(
a
)

-

η
*
(
θ
-
μ
)
)
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
}
yields the proper prior density (22).Further, this choice leads to the only proper prior that satisfies
(21)
α
=
p
(
a
≤
X
2
≤
b
∣
a
≤
X
1
≤
b
)
=
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
)
in the condition (16), because (22) is the density of θ
=
d
[
X
2
∣
a
≤
X
1
≤
b
]
~
W
N
(
a
,
b
)
(
μ
,
Σ
).Thus the maximum entropy prior density for the Case3 is given by
(22)
π
3
(
θ
)
=
ϕ
(
θ
;
μ
,
σ
2
)
×
Φ
(
ξ
*
v
*
(
b
)
-
η
*
(
θ
-
μ
)
)

-

Φ
(
ξ
*
v
*
(
a
)
-
η
*
(
θ
-
μ
)
)
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
,
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
θ
∈
R
,
where ξ
*
=
(
1
-
δ
)
-
1
/
2 and η
*
=
ξ
*
δ
/
σ.Note, from Lemma2, that π
1
(
θ
)
=
π
2
(
θ
)
=
π
3
(
θ
) for a
=
-
∞ and b
=
∞. This is consistent with our partial priori information that E
[
θ
]
=
μ and Var
⁡
(
θ
)
=
σ
2 for θ
∈
R. Some algebra using the moments of the W
N
(
a
,
b
)
(
μ
,
Σ
) distribution given in Kim [6] yields the entropy of π
3
(
θ
) (i.e., Ent
(
π
3
(
θ
)
)) given by
(23)
1
2
{
1
+
ln
⁡
(
2
π
σ
2
)
+
v
*
(
a
)
ϕ
(
v
*
(
a
)
)
-
v
*
(
b
)
ϕ
(
v
*
(
b
)
)
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
δ
}
+
ln
⁡
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
)
-
∫
R
‍
ln
⁡
{
Φ
(
ξ
*
v
*
(
b
)
-
η
*
(
θ
-
μ
)
)
-
Φ
(
ξ
*
v
*
(
a
)
-
η
*
(
θ
-
μ
)
)
}
π
3
(
θ
)
d
θ
.
As seen in (23), the calculation of Ent
(
π
3
(
θ
)
) involves a complicated integration. Instead, by using a Monte Carlo integration, we may calculate it approximately. According to Kim [6], it follows that the stochastic representation of the prior distribution θ
~
W
N
(
a
,
b
)
(
μ
,
Σ
) with density π
3
(
θ
) is
(24)
θ
=
d
μ
+
σ
δ
Z
(
v
*
(
a
)
,
v
*
(
b
)
)
+
σ
(
1
-
δ
)
1
/
2
Z
,
where Z
(
v
*
(
a
)
,
v
*
(
b
)
)
~
N
(
v
*
(
a
)
,
v
*
(
b
)
)
(
0,1
) and Z
~
N
(
0,1
), and they are independent. This stochastic representation is useful for generating θ’s from the prior distribution θ, and hence implementing the Monte Carlo integration. The following corollary asserts the relationship among π
j
(
θ
)
,
j
=
1,2
,
3, in terms of δ
∈
(
0,1
).Corollary 3.
Asδ
→
1, the maximum entropy prior π
3
(
θ
) approximates to π
2
(
θ
) in (13), while π
3
(
θ
) is equivalent to π
1
(
θ
) for δ
=
0.Proof.
Note from (22) that ξ
v
*
(
b
)
-
η
(
θ
-
μ
)
→
∞ and ξ
v
*
(
a
)
-
η
(
θ
-
μ
)
→
-
∞ as δ
→
1 for θ
∈
[
a
,
b
]. Also note, from the conditional property of N
2
(
μ
,
Σ
), that α
=
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
/
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
)
→
1 is equivalent to δ
→
1 for finite vales of μ,σ,a, and b with b
>
a. Applying these two results to π
3
(
θ
), we see that it approximates to π
2
(
θ
) as δ
→
1. It is straightforward to see from Lemma 2 thatπ
3
(
θ
)
=
π
1
(
θ
)for δ
=
0, because the W
N
(
a
,
b
)
(
μ
,
Σ
) distribution is equivalent to N
(
μ
,
σ
2
) for δ
=
0.Each graph of Figure1 depicts the difference between Ent
(
π
3
(
θ
)
) and Ent
(
π
2
(
θ
)
) as a function of δ
∈
(
0,1
) for three values of σ
2, two cases of Θ
=
[
a
,
b
], and μ
=
0. In Figure 1, the difference is denoted by Dif
f
Ent. Since each graph coincides with the results of Corollary 3, we can obtain the following implications from the figure. (i) As expected, we see that Ent
(
π
1
(
θ
)
)
>
Ent
(
π
3
(
θ
)
)
>
Ent
(
π
2
(
θ
)
) for δ
∈
(
0,1
). (ii) The entropy of π
3
(
θ
) is a monotone decreasing function of δ. (iii) Each entropy of the three priors increases as σ
2 becomes large. (iv) Dif
f
Ent is closely related with degree of uncertainty (i.e,. (
1
-
α
)
×
100
%) for it is a monotone decreasing function of δ
∈
(
0,1
) and α is a function of δ. (v) Dif
f
Ent is a monotone increasing function of σ
2 for the case where a value of δ is given.Plots ofDif
f
Ent
=
Ent
(
π
3
(
θ
)
)
-
Ent
(
π
2
(
θ
)
) for different values of δ
∈
(
0,1
) and σ
2 for the cases where (a) Θ
=
[
-
1.0,0
] and (b) Θ
=
[
-
1.0,1.5
].
(a)
(b)


### 3.3. Objective Measure of Uncertainty

From a relationship between Lemmas1 and 2, we can propose an objective measure for the degree of uncertainty regarding a prior interval constraint accounted for by HSGM. The following theorem proposes the objective measure using the same notations as used in Lemmas 1 and 2.Theorem 4.
Letσ
0
2
=
(
1
-
δ
)
σ
2
,
σ
1
2
=
δ
σ
2, and δ
∈
(
0,1
). Then the two-stage prior of θ defined by H
S
G
M in (2) is equivalent to π
3
(
θ
) and the degree of uncertainty regarding the interval constraint, θ
∈
[
a
,
b
], accounted for by the H
S
G
M is
(25)
{
1
-
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
)
}
×
100
%
.Proof.
Ifσ
0
2
=
(
1
-
δ
)
σ
2
,
σ
1
2
=
δ
σ
2, and δ
∈
(
0,1
), the marginal prior distribution of θ in Lemma 1 is equivalent to the density ofW
N
(
a
,
b
)
(
μ
,
Σ
). Thus the prior density p
(
θ
) in (4) is equal to the maximum entropy prior π
3
(
θ
) in (22). Now Lemma 2 asserts that π
3
(
θ
) reflects uncertainty about the interval constraint θ
∈
[
a
,
b
] by the degree of (
1
-
α
)
×
100
% with α
=
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
/
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
).Theorem4 provides an exact measure of the uncertainty about the priori interval constraint on θ accounted for by HSGM, and it shows that the uncertainty measure is different from that of O’Hagan and Leonald [1] which mainly depends on the first stage variance, σ
0
2, of θ in HSGM. Theorem 4 also indicates that HSGM in (2) can be used to elicit the priori uncertain interval information associated with π
3
(
θ
). Further, the entropy of the two-stage prior θ defined by the HSGM (i.e., p
(
θ
) in (4)) can be calculated by using the formula of Ent
(
π
3
(
θ
)
) in (23). We can visualize the degree of uncertainty about the priori interval constraint, θ
∈
[
a
,
b
], by plotting 1
-
α for different values of δ
∈
(
0,1
) in Figure 2.Plots of(
1
-
α
) for different values of δ
∈
(
0,1
) and σ
2 for the cases where (a) Θ
=
[
-
1.0,0
] and (b) Θ
=
[
-
1.0,1.5
].
(a)
(b)From Figure2, we can see, for fixed value of σ
2, that HSGM with small values of δ tends to increase the uncertainty regarding the priori constraint. This coincides with the result which is asserted by Corollary 3. Further, we see from the figure that, for a fixed value of δ
∈
(
0,1
), we can enlarge (or reduce) the uncertainty about the priori constraint by increasing (or decreasing) the amount of σ
2 (or equivalently σ
0
2 and σ
1
2) in the two stages of a prior hierarchy of θ.When the data information isy
~
N
(
θ
,
τ
2
), it is well known that the maximum entropy priors (4) and (9) are conjugate priors for the normal mean θ when τ
2 is known. This is obtained from the following posteriors:
(26)
p
1
(
θ
∣
y
)
∝
ϕ
(
y
;
θ
,
τ
2
)
π
1
(
θ
)
∝
ϕ
(
θ
;
μ
*
,
σ
*
2
)
vvvvvvvvvvvvvvvvvvvvvvvvvvv
v
for
θ
∈
R
,
p
2
(
θ
∣
y
)
∝
ϕ
(
y
;
θ
,
τ
2
)
π
2
(
θ
)
∝
ϕ
(
θ
;
μ
*
,
σ
*
2
)
vvvvvvvvvvvvvvvvvvvvvvvvv
v
for
θ
∈
[
a
,
b
]
,
where μ
*
=
(
σ
2
y
+
τ
2
μ
)
/
(
σ
2
+
τ
2
) and σ
*
2
=
σ
2
τ
2
/
(
σ
2
+
τ
2
). Thus, we see that each prior satisfies the conjugate property that π
j
(
θ
) and p
j
(
θ
∣
y
) belong to the same family of distributions for j
=
1,2. The following corollary provides this conjugate property which also applies to π
3
(
θ
) in (22).Corollary 5.
Lety
~
N
(
θ
,
τ
2
) with known τ
2. Then the prior π
3
(
θ
) yields the posterior distribution of θ given by
(27)
(
θ
∣
y
)
~
W
N
(
a
,
b
)
(
μ
*
,
Σ
*
)
,
where
(28)
μ
*
=
(
(
1
-
δ
)
μ
+
δ
μ
*
μ
*
)
,
Σ
*
=
(
δ
(
1
-
δ
)
σ
2
+
δ
2
σ
*
2
δ
σ
*
2
δ
σ
*
2
σ
*
2
)
,
μ
*
=
(
σ
2
y
+
τ
2
μ
)
/
(
σ
2
+
τ
2
), and σ
*
2
=
σ
2
τ
2
/
(
σ
2
+
τ
2
).Proof.
Sincey
~
N
(
θ
,
τ
2
) with known τ
2, under the prior π
3
(
θ
) in (22), the posterior density of θ is given by
(29)
p
(
θ
∣
y
)
∝
ϕ
(
y
;
θ
,
τ
2
)
ϕ
(
θ
;
μ
,
σ
2
)
vvvvv
×
{
Φ
(
ξ
*
v
*
(
b
)
-
η
*
(
θ
-
μ
)
)
v
v
v
v
v
v
v
v
v
v
v
v
v
-
Φ
(
ξ
*
v
*
(
a
)
-
η
*
(
θ
-
μ
)
)
}
vvv
∝
ϕ
(
θ
;
μ
*
,
σ
*
2
)
{
Φ
(
ξ
*
γ
(
b
)
-
η
*
(
θ
-
μ
*
)
)
vvvvvvvvvvvvvv
vvvvv
-
Φ
(
ξ
*
γ
(
a
)
-
η
*
(
θ
-
μ
*
)
)
}
in that {
Φ
(
ξ
*
v
*
(
b
)
-
η
*
(
θ
-
μ
)
)
-
Φ
(
ξ
*
v
*
(
a
)
-
η
*
(
θ
-
μ
)
)
}
=
{
Φ
(
ξ
*
γ
(
b
)
-
η
*
(
θ
-
μ
*
)
)
-
Φ
(
ξ
*
γ
(
a
)
-
η
*
(
θ
-
μ
*
)
)
}, where γ
(
b
)
=
{
b
-
(
(
1
-
δ
)
μ
+
δ
μ
*
)
}
/
(
σ
δ
) and γ
(
a
)
=
{
a
-
(
(
1
-
δ
)
μ
+
δ
μ
*
)
}
/
(
σ
δ
). This is a kernel of W
N
(
a
,
b
)
(
μ
*
,
Σ
*
) density.


## 3.1. A Maximum Entropy Prior

Sometimes we have a situation where partial prior information is available, outside of which it is desired to use a priori that is as noninformative as possible. A useful method of dealing with this problem is through the concept of entropy by Jaynes [7, 8]. As discussed by Rosenkrantz [11], entropy has a direct relationship to information theory and in a sense measures the amount of uncertainty inherent in the probability distribution.Assume now that we can specify the partial information concerning a location parameterθ (including the normal mean) with continuous space Θ of the form
(7)
E
[
t
j
(
θ
)
]
=
∫
Θ
‍
t
j
(
θ
)
π
(
θ
)
d
θ
=
t
j
,
j
=
1
,
…
,
k
,
but with nothing else about our prior distribution π
(
θ
). Then the maximum entropy prior can be obtained by choosing π
(
θ
) that maximizes the entropy
(8)
ξ
(
π
)
=
-
∫
Θ
‍
π
(
θ
)
log
⁡
π
(
θ
)
d
θ
,
in the presence of the partial information in the form of (7). A straightforward application of the calculus of variation leads us to Boltzmann’s maximum entropy theorem. This tells us that the density π
(
θ
) that maximizes ξ
(
π
), subject to the constraints E
[
t
j
(
θ
)
]
=
t
j
,
j
=
1
,
…
,
k, takes the k-parameter exponential family form
(9)
π
(
θ
)
∝
exp
⁡
{
λ
1
t
1
(
θ
)
+
λ
2
t
2
(
θ
)
+
⋯
+
λ
k
t
k
(
θ
)
}
,
θ
∈
Θ
,
where λ
1
,
λ
2
,
…
,
λ
k can be determined, via the k-constraints, in terms of t
1
,
…, t
k. See Leonard and Hsu [12] for the proof.


## 3.2.  Maximum Entropy Prior for Constrained Normal Mean

Let the location parameterθ of our interest be the normal mean in (1). Then the results of the previous subsection can be applied to the prior for the normal mean θ. This subsection considers the case where the partial priori information of θ is in the form of an interval, that is, θ
∈
Θ
,
Θ
=
[
a
,
b
] with a
<
b, and examines how the maximum entropy prior of θ, that is, π
(
θ
), has different formula according to the degree of uncertainty regarding the interval constraint.Case 1.
Θ
=
R, and θ
∈
Θ with certainty.When we have partial priori information aboutθ that we can specify values for both mean μ and variance σ
2. Then the N
(
μ
,
σ
2
) prior specification is the maximum entropy prior for θ (e.g., [12]). Thus the prior density and its entropy for the Case 1 are
(10)
π
1
(
θ
)
=
ϕ
(
θ
;
μ
,
σ
2
)
,
θ
∈
R
,
Ent
(
π
1
(
θ
)
)
=
-
∫
Θ
‍
ϕ
(
θ
;
μ
,
σ
2
)
ln
⁡
ϕ
(
θ
;
μ
,
σ
2
)
d
θ
=
1
2
(
1
+
ln
⁡
(
2
π
)
+
ln
⁡
σ
2
)
,
respectively.Case 2.
Θ
=
[
a
,
b
], and θ
∈
Θ with certainty.On the other hand, when we are completely sure about the priori interval constraint ofθ, a suitable restriction on the parameter space Θ such as using a truncated distribution is expected. This case supposes that we can specify values for both E
[
θ
]
=
μ and E
[
(
θ
-
μ
)
2
]
=
σ
2 on the space θ
∈
R by a priori information. Further suppose that we have certain prior information that the parameter θ is concentrated on the region [
a
,
b
], that is,
(11)
p
(
a
≤
θ
≤
b
)
=
1
but nothing else about our prior distribution π
(
θ
). The last condition is equivalent to E
[
I
(
a
≤
θ
≤
b
)
]
=
1. Therefore, by Bolzmann’s maximum entropy theorem, the prior density of θ for the Case 2 is
(12)
π
2
(
θ
)
∝
exp
⁡
{
λ
1
t
1
(
θ
)
+
λ
2
t
2
(
θ
)
+
λ
3
t
3
(
θ
)
}
∝
exp
⁡
⁡
{
[
θ
-
(
μ
-
λ
1
/
2
λ
2
)
]
2
λ
2
-
1
}
×
exp
⁡
⁡
{
λ
3
I
(
a
≤
θ
≤
b
)
}
by (9), because t
1
(
θ
)
=
θ
,
t
1
=
μ
,
t
2
(
θ
)
=
(
θ
-
μ
)
2
,
t
2
=
σ
2, and t
3
(
θ
)
=
I
(
a
≤
θ
≤
b
). Since t
3
=
1, the maximum entropy prior, subject to these three restrictions, is thus
(13)
π
2
(
θ
)
=
ϕ
(
θ
;
μ
,
σ
2
)
Φ
(
v
(
b
)
)
-
Φ
(
v
(
a
)
)
,
a
≤
θ
≤
b
,
provided that we choose λ
1
=
0
,
λ
2
=
-
1
/
2
σ
2
,
λ
3
=
-
ln
⁡
{
Φ
(
v
(
b
)
)
-
Φ
(
v
(
a
)
)
}, where v
(
a
)
=
(
a
-
μ
)
/
σ and v
(
b
)
=
(
b
-
μ
)
/
σ.Some algebra using the moments of theN
(
a
,
b
)
(
μ
,
σ
2
) distribution in Johnson et al. [13] yields the entropy of π
2
(
θ
) which is
(14)
Ent
(
π
2
(
θ
)
)
=
1
2
(
1
+
ln
⁡
⁡
(
2
π
σ
2
)
+
v
(
a
)
ϕ
(
v
(
a
)
)
-
v
(
b
)
ϕ
(
v
(
b
)
)
Φ
(
v
(
b
)
)
-
Φ
(
v
(
a
)
)
)
+
ln
⁡
(
Φ
(
v
(
b
)
)
-
Φ
(
v
(
a
)
)
)
.Case 3.
Θ
=
[
a
,
b
], and θ
∈
Θ with (
1
-
α
)
×
100
% uncertain.Now suppose that we have partial priori information that we can specify values for both meanμ and variance σ
2 of θ for θ
∈
R. In addition, suppose that we have priori uncertain interval constraint information that θ
∈
[
a
,
b
]. Then along with the priori moment conditions E
[
θ
]
=
μ and E
[
(
θ
-
μ
)
2
]
=
σ
2, the additional uncertain prior information about the interval constraint can be expressed by
(15)
p
(
a
≤
θ
≤
b
)
=
α
,
0
≤
α
≤
1
,
where 1
-
α (or (
1
-
α
)
×
100
%) denotes the degree of uncertainty. Thus the priori uncertain interval constraint is equivalent to
(16)
E
[
I
(
a
≤
θ
≤
b
)
]
=
α
.
Applying this partial prior information to Bolzmann’s maximum entropy theorem, we have the following lemma.Lemma 2.
Ifα
=
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
/
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
), the maximum entropy prior distribution of θ, reflecting the uncertain interval constraint in (15) is
(17)
θ
~
W
N
(
a
,
b
)
(
μ
,
Σ
)
,
where v
*
(
a
)
=
(
a
-
μ
)
/
(
σ
δ
), v
*
(
b
)
=
(
b
-
μ
)
/
(
σ
δ
), a
=
(
a
,
a
)
⊤
,
b
=
(
b
,
b
)
⊤
,
Φ
¯
2
(
(
a
,
b
)
:
μ
,
Σ
) denote a rectangle probability p
(
a
≤
X
2
≤
b
,
a
≤
X
1
≤
b
) of X
1 and X
2 whose joint distribution is a bivariate normal N
2
(
μ
,
Σ
),
(18)
μ
=
(
μ
μ
)
,
Σ
=
σ
2
(
δ
δ
δ
1
)
,
0
<
δ
<
1
.Proof.
Takingt
1
(
θ
)
=
θ
,
t
1
=
μ
,
t
2
(
θ
)
=
(
θ
-
μ
)
2
,
t
2
=
σ
2
,
t
3
(
θ
)
=
I
(
a
≤
θ
≤
b
), and t
3
=
α, and setting λ
1
=
0 and λ
2
=
-
1
/
2
σ
2, the maximum entropy prior π
(
θ
) in (9) reduces to
(19)
π
3
(
θ
)
∝
exp
⁡
{
(
θ
-
μ
)
2
2
σ
2
}
exp
⁡
{
λ
3
t
3
(
θ
)
}
,
θ
∈
R
,
by Bolzmann’s maximum entropy theorem. Now the second exponential term in the right hand side of (19) can be determined by using the condition (16) as follows. Among all the possible proper prior densities of the form (19), the choice of
(20)
λ
3
t
3
(
θ
)

=

ln
⁡
{
Φ
(
ξ
*
v
*
(
b
)

-

η
*
(
θ
-
μ
)
)

-

Φ
(
ξ
*
v
*
(
a
)

-

η
*
(
θ
-
μ
)
)
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
}
yields the proper prior density (22).Further, this choice leads to the only proper prior that satisfies
(21)
α
=
p
(
a
≤
X
2
≤
b
∣
a
≤
X
1
≤
b
)
=
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
)
in the condition (16), because (22) is the density of θ
=
d
[
X
2
∣
a
≤
X
1
≤
b
]
~
W
N
(
a
,
b
)
(
μ
,
Σ
).Thus the maximum entropy prior density for the Case3 is given by
(22)
π
3
(
θ
)
=
ϕ
(
θ
;
μ
,
σ
2
)
×
Φ
(
ξ
*
v
*
(
b
)
-
η
*
(
θ
-
μ
)
)

-

Φ
(
ξ
*
v
*
(
a
)
-
η
*
(
θ
-
μ
)
)
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
,
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
θ
∈
R
,
where ξ
*
=
(
1
-
δ
)
-
1
/
2 and η
*
=
ξ
*
δ
/
σ.Note, from Lemma2, that π
1
(
θ
)
=
π
2
(
θ
)
=
π
3
(
θ
) for a
=
-
∞ and b
=
∞. This is consistent with our partial priori information that E
[
θ
]
=
μ and Var
⁡
(
θ
)
=
σ
2 for θ
∈
R. Some algebra using the moments of the W
N
(
a
,
b
)
(
μ
,
Σ
) distribution given in Kim [6] yields the entropy of π
3
(
θ
) (i.e., Ent
(
π
3
(
θ
)
)) given by
(23)
1
2
{
1
+
ln
⁡
(
2
π
σ
2
)
+
v
*
(
a
)
ϕ
(
v
*
(
a
)
)
-
v
*
(
b
)
ϕ
(
v
*
(
b
)
)
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
δ
}
+
ln
⁡
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
)
-
∫
R
‍
ln
⁡
{
Φ
(
ξ
*
v
*
(
b
)
-
η
*
(
θ
-
μ
)
)
-
Φ
(
ξ
*
v
*
(
a
)
-
η
*
(
θ
-
μ
)
)
}
π
3
(
θ
)
d
θ
.
As seen in (23), the calculation of Ent
(
π
3
(
θ
)
) involves a complicated integration. Instead, by using a Monte Carlo integration, we may calculate it approximately. According to Kim [6], it follows that the stochastic representation of the prior distribution θ
~
W
N
(
a
,
b
)
(
μ
,
Σ
) with density π
3
(
θ
) is
(24)
θ
=
d
μ
+
σ
δ
Z
(
v
*
(
a
)
,
v
*
(
b
)
)
+
σ
(
1
-
δ
)
1
/
2
Z
,
where Z
(
v
*
(
a
)
,
v
*
(
b
)
)
~
N
(
v
*
(
a
)
,
v
*
(
b
)
)
(
0,1
) and Z
~
N
(
0,1
), and they are independent. This stochastic representation is useful for generating θ’s from the prior distribution θ, and hence implementing the Monte Carlo integration. The following corollary asserts the relationship among π
j
(
θ
)
,
j
=
1,2
,
3, in terms of δ
∈
(
0,1
).Corollary 3.
Asδ
→
1, the maximum entropy prior π
3
(
θ
) approximates to π
2
(
θ
) in (13), while π
3
(
θ
) is equivalent to π
1
(
θ
) for δ
=
0.Proof.
Note from (22) that ξ
v
*
(
b
)
-
η
(
θ
-
μ
)
→
∞ and ξ
v
*
(
a
)
-
η
(
θ
-
μ
)
→
-
∞ as δ
→
1 for θ
∈
[
a
,
b
]. Also note, from the conditional property of N
2
(
μ
,
Σ
), that α
=
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
/
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
)
→
1 is equivalent to δ
→
1 for finite vales of μ,σ,a, and b with b
>
a. Applying these two results to π
3
(
θ
), we see that it approximates to π
2
(
θ
) as δ
→
1. It is straightforward to see from Lemma 2 thatπ
3
(
θ
)
=
π
1
(
θ
)for δ
=
0, because the W
N
(
a
,
b
)
(
μ
,
Σ
) distribution is equivalent to N
(
μ
,
σ
2
) for δ
=
0.Each graph of Figure1 depicts the difference between Ent
(
π
3
(
θ
)
) and Ent
(
π
2
(
θ
)
) as a function of δ
∈
(
0,1
) for three values of σ
2, two cases of Θ
=
[
a
,
b
], and μ
=
0. In Figure 1, the difference is denoted by Dif
f
Ent. Since each graph coincides with the results of Corollary 3, we can obtain the following implications from the figure. (i) As expected, we see that Ent
(
π
1
(
θ
)
)
>
Ent
(
π
3
(
θ
)
)
>
Ent
(
π
2
(
θ
)
) for δ
∈
(
0,1
). (ii) The entropy of π
3
(
θ
) is a monotone decreasing function of δ. (iii) Each entropy of the three priors increases as σ
2 becomes large. (iv) Dif
f
Ent is closely related with degree of uncertainty (i.e,. (
1
-
α
)
×
100
%) for it is a monotone decreasing function of δ
∈
(
0,1
) and α is a function of δ. (v) Dif
f
Ent is a monotone increasing function of σ
2 for the case where a value of δ is given.Plots ofDif
f
Ent
=
Ent
(
π
3
(
θ
)
)
-
Ent
(
π
2
(
θ
)
) for different values of δ
∈
(
0,1
) and σ
2 for the cases where (a) Θ
=
[
-
1.0,0
] and (b) Θ
=
[
-
1.0,1.5
].
(a)
(b)


## 3.3. Objective Measure of Uncertainty

From a relationship between Lemmas1 and 2, we can propose an objective measure for the degree of uncertainty regarding a prior interval constraint accounted for by HSGM. The following theorem proposes the objective measure using the same notations as used in Lemmas 1 and 2.Theorem 4.
Letσ
0
2
=
(
1
-
δ
)
σ
2
,
σ
1
2
=
δ
σ
2, and δ
∈
(
0,1
). Then the two-stage prior of θ defined by H
S
G
M in (2) is equivalent to π
3
(
θ
) and the degree of uncertainty regarding the interval constraint, θ
∈
[
a
,
b
], accounted for by the H
S
G
M is
(25)
{
1
-
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
)
}
×
100
%
.Proof.
Ifσ
0
2
=
(
1
-
δ
)
σ
2
,
σ
1
2
=
δ
σ
2, and δ
∈
(
0,1
), the marginal prior distribution of θ in Lemma 1 is equivalent to the density ofW
N
(
a
,
b
)
(
μ
,
Σ
). Thus the prior density p
(
θ
) in (4) is equal to the maximum entropy prior π
3
(
θ
) in (22). Now Lemma 2 asserts that π
3
(
θ
) reflects uncertainty about the interval constraint θ
∈
[
a
,
b
] by the degree of (
1
-
α
)
×
100
% with α
=
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
/
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
).Theorem4 provides an exact measure of the uncertainty about the priori interval constraint on θ accounted for by HSGM, and it shows that the uncertainty measure is different from that of O’Hagan and Leonald [1] which mainly depends on the first stage variance, σ
0
2, of θ in HSGM. Theorem 4 also indicates that HSGM in (2) can be used to elicit the priori uncertain interval information associated with π
3
(
θ
). Further, the entropy of the two-stage prior θ defined by the HSGM (i.e., p
(
θ
) in (4)) can be calculated by using the formula of Ent
(
π
3
(
θ
)
) in (23). We can visualize the degree of uncertainty about the priori interval constraint, θ
∈
[
a
,
b
], by plotting 1
-
α for different values of δ
∈
(
0,1
) in Figure 2.Plots of(
1
-
α
) for different values of δ
∈
(
0,1
) and σ
2 for the cases where (a) Θ
=
[
-
1.0,0
] and (b) Θ
=
[
-
1.0,1.5
].
(a)
(b)From Figure2, we can see, for fixed value of σ
2, that HSGM with small values of δ tends to increase the uncertainty regarding the priori constraint. This coincides with the result which is asserted by Corollary 3. Further, we see from the figure that, for a fixed value of δ
∈
(
0,1
), we can enlarge (or reduce) the uncertainty about the priori constraint by increasing (or decreasing) the amount of σ
2 (or equivalently σ
0
2 and σ
1
2) in the two stages of a prior hierarchy of θ.When the data information isy
~
N
(
θ
,
τ
2
), it is well known that the maximum entropy priors (4) and (9) are conjugate priors for the normal mean θ when τ
2 is known. This is obtained from the following posteriors:
(26)
p
1
(
θ
∣
y
)
∝
ϕ
(
y
;
θ
,
τ
2
)
π
1
(
θ
)
∝
ϕ
(
θ
;
μ
*
,
σ
*
2
)
vvvvvvvvvvvvvvvvvvvvvvvvvvv
v
for
θ
∈
R
,
p
2
(
θ
∣
y
)
∝
ϕ
(
y
;
θ
,
τ
2
)
π
2
(
θ
)
∝
ϕ
(
θ
;
μ
*
,
σ
*
2
)
vvvvvvvvvvvvvvvvvvvvvvvvv
v
for
θ
∈
[
a
,
b
]
,
where μ
*
=
(
σ
2
y
+
τ
2
μ
)
/
(
σ
2
+
τ
2
) and σ
*
2
=
σ
2
τ
2
/
(
σ
2
+
τ
2
). Thus, we see that each prior satisfies the conjugate property that π
j
(
θ
) and p
j
(
θ
∣
y
) belong to the same family of distributions for j
=
1,2. The following corollary provides this conjugate property which also applies to π
3
(
θ
) in (22).Corollary 5.
Lety
~
N
(
θ
,
τ
2
) with known τ
2. Then the prior π
3
(
θ
) yields the posterior distribution of θ given by
(27)
(
θ
∣
y
)
~
W
N
(
a
,
b
)
(
μ
*
,
Σ
*
)
,
where
(28)
μ
*
=
(
(
1
-
δ
)
μ
+
δ
μ
*
μ
*
)
,
Σ
*
=
(
δ
(
1
-
δ
)
σ
2
+
δ
2
σ
*
2
δ
σ
*
2
δ
σ
*
2
σ
*
2
)
,
μ
*
=
(
σ
2
y
+
τ
2
μ
)
/
(
σ
2
+
τ
2
), and σ
*
2
=
σ
2
τ
2
/
(
σ
2
+
τ
2
).Proof.
Sincey
~
N
(
θ
,
τ
2
) with known τ
2, under the prior π
3
(
θ
) in (22), the posterior density of θ is given by
(29)
p
(
θ
∣
y
)
∝
ϕ
(
y
;
θ
,
τ
2
)
ϕ
(
θ
;
μ
,
σ
2
)
vvvvv
×
{
Φ
(
ξ
*
v
*
(
b
)
-
η
*
(
θ
-
μ
)
)
v
v
v
v
v
v
v
v
v
v
v
v
v
-
Φ
(
ξ
*
v
*
(
a
)
-
η
*
(
θ
-
μ
)
)
}
vvv
∝
ϕ
(
θ
;
μ
*
,
σ
*
2
)
{
Φ
(
ξ
*
γ
(
b
)
-
η
*
(
θ
-
μ
*
)
)
vvvvvvvvvvvvvv
vvvvv
-
Φ
(
ξ
*
γ
(
a
)
-
η
*
(
θ
-
μ
*
)
)
}
in that {
Φ
(
ξ
*
v
*
(
b
)
-
η
*
(
θ
-
μ
)
)
-
Φ
(
ξ
*
v
*
(
a
)
-
η
*
(
θ
-
μ
)
)
}
=
{
Φ
(
ξ
*
γ
(
b
)
-
η
*
(
θ
-
μ
*
)
)
-
Φ
(
ξ
*
γ
(
a
)
-
η
*
(
θ
-
μ
*
)
)
}, where γ
(
b
)
=
{
b
-
(
(
1
-
δ
)
μ
+
δ
μ
*
)
}
/
(
σ
δ
) and γ
(
a
)
=
{
a
-
(
(
1
-
δ
)
μ
+
δ
μ
*
)
}
/
(
σ
δ
). This is a kernel of W
N
(
a
,
b
)
(
μ
*
,
Σ
*
) density.


## 4. Posterior Estimation

Let us revisit the HSGM with the following two stages of a prior hierarchy ofθ:
(30)
y
i
=
θ
+
ϵ
i
,
ϵ
i
~
N
(
0
,
τ
2
)
,
i
=
1
,
…
,
n
,
τ
2
~
IG
(
c
,
d
)
,
θ
∣
θ
0
~
N
(
θ
0
,
(
1
-
δ
)
σ
2
)
,
independent
of
(
ϵ
1
,
…
,
ϵ
n
)
⊤
,
θ
0
~
N
(
a
≤
θ
0
≤
b
)
(
μ
,
δ
σ
2
)
,
δ
∈
(
0,1
)
.
According to Theorem 4, we see that the two-stage prior of θ is essentially the same as the maximum entropy prior which properly elicits the partial priori information of an uncertain interval constraint, {
θ
;
θ
∈
[
a
,
b
]
}, with (
1
-
α
)
×
100
% degree of uncertainty, where α
=
Φ
¯
2
(
(
a
,
b
)
;
μ
,
Σ
)
/
(
Φ
(
v
*
(
b
)
)
-
Φ
(
v
*
(
a
)
)
). Here μ and σ
2 are true prior mean and variance of the parameter θ when the uncertain priori interval condition does not exist.Based on the marginal prior distribution ofθ in Lemma 1, we have the joint posterior distribution of θ and τ
2 proportional to the product of likelihood and the prior distribution,
(31)
p
(
θ
,
τ
2
∣
D
n
)
∝
∏
i
=
1
n
‍
ϕ
(
y
i
;
θ
,
τ
2
)
π
3
(
θ
)
IG
(
τ
2
;
c
,
d
)
,
where D
n
=
{
y
1
,
…
,
y
n
}
,
IG
(
·
;
c
,
d
) is the inverse-gamma density with parameters c and d, and π
3
(
θ
) is the density (22), that is, the density of θ
~
W
N
(
a
,
b
)
(
μ
,
Σ
). Note that the joint posterior is not simplified in an analytic form of the known density and thus intractable for posterior inference. Instead, we derive each of the conditional posterior distributions of θ and τ
2 in an explicit form, which will be useful for posterior inference such as Gibbs sampling (e.g., [14]).Corollary 6.
Given the joint posterior distribution (31), we have the following. (i)
The full conditional posterior distribution ofθ is given by
(32)
θ
∣
(
τ
2
,
D
n
)
~
W
N
(
a
,
b
)
(
θ
*
,
Ω
*
)
,where
(33)
θ
*
=
(
(
1
-
δ
)
μ
+
δ
θ
*
θ
*
)
,
Ω
*
=
(
δ
(
1
-
δ
)
σ
2
+
δ
2
ω
*
2
δ
ω
*
2
δ
ω
*
2
ω
*
2
)
,
θ
*
=
(
y
¯
σ
2
+
μ
τ
2
/
n
)
/
(
σ
2
+
τ
2
/
n
), and ω
*
2
=
σ
2
τ
2
/
(
n
σ
2
+
τ
2
).(ii)
The full conditional posterior distribution ofτ
2 is the inverse-Gamma distribution
(34)
τ
2
∣
(
θ
,
D
n
)
~
I
G
(
c
+
2
n
,
d
+
∑
i
=
1
n
‍
(
y
i
-
θ
)
2
)
.Proof.
(
i
) The unnormalized conditional density of θ given that τ
2 and D
n is proportional to
(35)
p
(
θ
∣
τ
2
,
D
n
)
∝
ϕ
(
y
¯
;
θ
,
τ
2
n
)
π
3
(
θ
)
,
where y
¯
=
∑
i
=
1
n
‍
y
i
/
n. Thus direct application of Corollary 5 yields the result.
(
ii
) It is straightforward to see from (31) that the full conditional posterior distribution of τ
2 is
(36)
f
(
τ
2
∣
θ
,
D
n
)
∝
(
τ
2
)
-
(
c
+
n
/
2
+
1
)
exp
⁡
{
(
d
+
∑
i
=
1
n
‍
(
y
i
-
θ
)
2
)
τ
2
}
,
which is a kernel of IG
(
c
+
2
/
n
,
d
+
∑
i
=
1
n
‍
(
y
i
-
θ
)
2
) distribution.It is not difficult to construct the Gibbs sampling scheme working with(
θ
,
τ
2
) because their full conditional distributions are given in Corollary 6. A routine Gibbs sampler would work to generate posterior samples of (
θ
,
τ
2
). The only difficulty would lie in generating random samples from WN distribution, that is, θ
∣
(
τ
2
,
D
n
)
~
W
N
(
a
,
b
)
(
μ
*
,
Ω
*
) as given in (32). In order to generate random samples from the WN distribution, we can make use of the stochastic representation given in (24). Also we can note that the stochastic representation (24) of the full conditional posterior distribution (32) provides the posterior mean and variance given by
(37)
E
[
θ
∣
(
τ
2
,
D
n
)
]
=
θ
*
-
β
1
κ
,
Var
⁡
(
θ
∣
(
τ
2
,
D
n
)
)
=
ω
*
2
-
β
2
κ
2
,
where κ
=
δ
ω
*
2
/
ω
11
*
,
-
β
1, and 1
-
β
2 are respective mean and variance of the truncated standard normal distribution, N
(
γ
*
(
a
)
,
γ
*
(
b
)
)
(
0,1
) with γ
*
(
a
)
=
(
a
-
θ
1
*
)
/
ω
11
*
,
γ
*
(
b
)
=
(
b
-
θ
1
*
)
/
ω
11
*
,
θ
1
*
=
(
1
-
δ
)
μ
+
δ
θ
*, and ω
11
*
2
=
δ
(
1
-
δ
)
σ
2
+
δ
2
ω
*
2. Johnson et al. [13] shows that they are
(38)
β
1
=
ϕ
(
γ
*
(
b
)
)
-
ϕ
(
γ
*
(
a
)
)
Φ
(
γ
*
(
b
)
)
-
Φ
(
γ
*
(
a
)
)
,
β
2
=
γ
*
(
b
)
ϕ
(
γ
*
(
b
)
)
-
γ
*
(
a
)
ϕ
(
γ
*
(
a
)
)
Φ
(
γ
*
(
b
)
)
-
Φ
(
γ
*
(
a
)
)
+
β
1
2
,
where ϕ
(
·
) denotes the density of N
(
0,1
). It is seen that each of the first terms in (37), that is, θ
* and ω
*
2, is the posterior mean and variance of θ when the prior π
1
(
θ
) is put on θ (e.g. [15, 16]), instead of the two-stage prior of Theorem 4, that is, π
3
(
θ
). That is each of the second terms in the posterior mean and covariance vanishes when θ is assumed to be N
(
μ
,
σ
2
) a priori without any interval constraint. In this sense, the second terms in (37) can be interpreted as aconstraining effect obtained by adopting the HSGM. See, for example, Kim [3], Kim and Choi [4], and H. j. Kim and H. M. Kim [17] for practical applications of HSGM methodology to get theconstraining effect in various constrained parameter problems.


## 5. Conclusion

This paper considered the normal models which include the two-stage prior of the normal mean, referred to ashierarchical screened Gaussian normal model (HSGM). The HSGMis based on the two stages of a prior hierarch advocated by O’Hagan and Leonard [1] and elicits partial priori information obtained for the case where an interval constraint of the normal mean needs to be incorporated in the modeling but such a restriction is uncertain. Then we proposed an objective method to measure (or control) the uncertainty accounted for by HSGM. For this purpose, we derived the maximum entropy prior, reflecting the uncertainty about an interval constraint on the normal mean, by using Boltzmann’s maximum entropy theorem. As a result, we found that both the maximum entropy prior and the two-stage prior in HSGM belong to the family of weighted normal distributions considered by Kim [6]. By exploring the distributional relationship between the two priors, we proposed the objective measure of uncertainty. This paper also proposed the Bayesian estimation procedure of HSGM and investigated some properties of the procedure by deriving the full conditional posterior distributions of unknown parameters under HSGM in analytic forms.



---

*Source: 676545-2014-07-06.xml*