# A Focused Bayesian Information Criterion

Authors: Georges Nguefack-Tsague; Ingo Bulla
Journal: Advances in Statistics
 (2014)

---

## Abstract

Myriads of model selection criteria (Bayesian and frequentist) have been proposed in the literature aiming at selecting a single model regardless of its intended use. An honorable exception in the frequentist perspective is the “focused information criterion” (FIC) aiming at selecting a model based on the parameter of interest (focus). This paper takes the same view in the Bayesian context; that is, a model may be good for one estimand but bad for another. The proposed method exploits the Bayesian model averaging (BMA) machinery to obtain a new criterion, thefocused Bayesian model averaging (FoBMA), for which the best model is the one whose estimate is closest to the BMA estimate. In particular, for two models, this criterion reduces to the classical Bayesian model selection scheme of choosing the model with the highest posterior probability. The new method is applied in linear regression, logistic regression, and survival analysis. This criterion is specially important in epidemiological studies in which the objective is often to determine a risk factor (focus) for a disease, adjusting for potential confounding factors.


---

## Body

## 1. Introduction

A variety of model selection criteria (Bayesian or frequentist) have been proposed in the literature; most of them aim at selecting a single model for any purposes. For an overview of model selection criteria, see the studies by Leeb and Poetscher [1] and Zucchini [2]; for inference after model selection, see the studies by Nguefack-Tsague [3], Nguefack-Tsague and Zucchini [4], Zucchini et al. [5], Behl et al. [6], and Nguefack-Tsague [7–9]. Allen [10], within the context of Mallows’ C
p [11], developed a criterion that depends on a given prediction. In a frequentist approach, Claeskens and Hjort [12] developed a focused information criterion (FIC) for model selection which, unlike common model selection criteria that lead to a single model for all purposes, selects different models for different purposes. Thus Allen’s criterion can be considered as an early precursor of FIC. So far the FIC is gaining in popularity as evidenced by its applications in various fields and specific models. Some of these applications include missing response (Sun et al. [13]), energy substitution (Behl et al. [6]), economic applications (Behl et al. [14]), Tobit model (Zhang et al. [15]), additive partial models (Zhang and Liang [16]), volatility forecasting (Brownlees and Gallo [17]), and Cox proportional hazard regression models (Hjort and Claeskens [18]). Focused information criterion and model averaging can be found in the studies by Sueishi [19] and Sun et al. [13]. A more recent development of FIC on quantile regression can be found in the studies by Du et al. [20], Behl et al. [21], and Xu et al. [22]. The motivation for the new method is based on the fact that this concept appears, up to our knowledge, to be virtually unknown and overlooked in Bayesian model selection; thus, there is a need to develop a Bayesian counterpart. The present paper is organized as follows. Section 2 presents the concept of Bayesian model averaging and model selection while Section 3 introduces the new criterion. Section 4 provides practical examples, while Section 5 provides discussions. The paper ends with concluding remarks.


## 2. Bayesian Model Selection and Model Averaging

### 2.1. Framework

Consider a situation in which some quantity of interest,μ, is to be estimated from a sample of observations that can be regarded as realizations from some unknown probability distribution and that, in order to do so, it is necessary to specify a model for the distribution. There are usually many alternative plausible models available and, in general, they each lead to different estimates of μ.Consider a sample of data,x, and a set of K models M
=
(
M
1
,
…
,
M
K
), which we will assume to contain the true model M
t. Each M
k consists of a family of distributions P
(
x
∣
η
k
,
M
k
), where η
k represents a parameter (or vector of parameters).The prior probability thatM
k is the true model is denoted by P
(
M
k
) and the prior distribution of the parameters of M
k (given that M
k is true) by P
(
η
k
∣
M
k
). Conditioning on the data x and integrating out the parameter η
k, one obtains the following posterior model probabilities:
(1)
P
(
M
k
∣
x
)
=
P
(
x
∣
M
k
)
P
(
M
k
)
∑
j
=
1
K
P
(
x
∣
M
j
)
P
(
M
j
)
,
where
(2)
P
(
x
∣
M
j
)
=
∫
‍
P
(
x
∣
η
k
,
M
k
)
P
(
η
k
∣
M
k
)
d
η
k
is the integrated likelihood under M
k. If P
(
η
k
∣
M
k
) is a discrete distribution, the integral in (2) is replaced by a sum.


### 2.2. Bayesian Model Selection

Within this framework, classical Bayesian model selection involves selecting the model with the highest posterior. Sometimes Bayes factors are used. From the Bayes factor framework, the Bayes factor (Kass and Raftery [23]) for M
i versus model M
j is defined by
(3)
B
i
j
=
P
(
x
∣
M
i
)
P
(
x
∣
M
j
)
.ModelM
i is chosen if B
i
j
>
1. Under certain assumptions and approximations (in particular the Laplace approximation) and taking all candidate models as a priori equally likely to be true, this leads to the Bayesian information criterion (BIC), also known as the Schwarz criterion [24]. More information on Bayesian model selection and applications can be found in the studies by Guan and Stephens [25], Nguefack-Tsague [26], Carvalho and Scott [27], Fridley [28], Robert [29], Liang et al. [30], and Bernado and Smith [31].


### 2.3. Focused Information Criterion (FIC)

As one can notice, in classical Bayesian model selection, a single (selected) model is used to explain all aspects of data, that is, regardless of the purpose of the selection, irrespective of the inference to follow (Claeskens and Hjort [12]). Allen [10] first developed this idea of focusing on a parameter of interest in a prediction problem in which the prediction at a particular value (target) of the regressor vector different from the values in the sample is of interest. Geisser [32] focused on prediction as a major inferential goal, rather than estimation, under many circumstances. In that method, the steps in the derivation of Mallows’ C
p are repeated for this target; thus, this criterion depends on that particular value and is therefore an early precursor to the FIC. In the FIC framework, a parameter of interest, say, μ, must have a definition making it meaningful across competing models. FIC methodology uses general parametric models and maximum likelihood as the estimation method in a general large sample theory. The FIC is derived as the result of establishing an unbiased estimation of the limiting risk of any submodel-based estimator of the parameter μ. FIC is based on the (crucial) assumption that the true data generating mechanism is contained in the largest parametric model considered. The candidate model with the smallest value of FIC is chosen. As one can see, FIC actually is based on frequentist approach; so far, up to our knowledge, Bayesian equivalent has not yet been considered.


### 2.4. Bayesian Model Averaging

Bayesian model averaging (BMA) is used to deal with the problem of model uncertainty. A discussion on the issue of model uncertainty is given in the study by Clyde and George [33]. Let μ be a quantity of interest depending on x, for example, a future observation from the same process that generated x. The idea is to use a weighted average of the estimates of μ obtained using each of the alternative models, rather than the estimate obtained using any single model. More precisely, the posterior distribution of μ is given by
(4)
P
(
μ
∣
x
)
=
∑
k
=
1
K
‍
P
(
μ
∣
x
,
M
k
)
P
(
M
k
∣
x
)
.
Note that P
(
μ
∣
x
) is a weighted average of the posterior distributions P
(
μ
∣
M
k
,
x
)
,
k
=
1
,
…
,
K, where the kth weight, P
(
M
k
∣
x
), is the posterior probability that M
k is the true model.The posterior distribution ofμ, conditioned on M
k being true, is given by
(5)
P
(
μ
∣
x
,
M
k
)
=
∫
‍
P
(
μ
∣
η
k
,
M
k
)
P
(
η
k
∣
x
,
M
k
)
d
η
k
.
The posterior mean and posterior variance are given by
(6)
μ
^
bma
=
E
bma
(
μ
∣
x
)
=
∑
k
=
1
K
‍
E
(
μ
∣
x
,
M
k
)
P
(
M
k
∣
x
)
,
Va
r
bma
(
μ
∣
x
)
=
∑
k
=
1
K
‍
[
(
E
(
μ
∣
x
,
M
k
)
-
μ
^
bma
)
2
Var
⁡
(
μ
∣
x
,
M
k
)
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
+
(
E
(
μ
∣
x
,
M
k
)
-
μ
^
bma
)
2
]
P
(
M
k
∣
x
)
.
Raftery et al. [34] call this averaging schemeBayesian model averaging. Leamer [35] and Draper [36] advocate the same idea. Madigan and Raftery [37] note that BMA provides better predictive performance than any single model if the measure of performance is Good’s [38] logarithm score rule, under the posterior distribution of μ given x.Hoeting et al. [39] give an extensive framework of BMA methodology and applications for different statistical models. Various real data and simulation studies have investigated the predictive performance of BMA (Clyde [40]; Clyde and George [33]). Nguefack-Tsague [41] uses BMA in the context of estimating a multivariate mean.


### 2.5. Challenges

Implementing BMA is demanding, especially the computation of the integrated likelihood. Software for BMA implementation, as well as some BMA papers, can be found at “http://www.research.att.com/~volinsky/bma.html”. An R [42] package for BMA is now available for computational purposes; this package provides ways for carrying out BMA for linear regression, generalized linear models, and survival analysis using Cox proportional hazard models. For computations, Monte Carlo methods, or approximating methods, are used; thus, many BMA applications are based on the BIC, an asymptotic approximation of the log posterior odds when the prior odds are all equal.Another problem is the selection of priors for both models and parameters. In most cases, a uniform prior is used for each model; that is,P
(
M
k
)
=
1
/
K, k
=
1,2
,
…
,
K. When the number of models is large, model search strategies are sometimes used to reduce the set of models, by eliminating those that seem comparatively less compatible with the data.


## 2.1. Framework

Consider a situation in which some quantity of interest,μ, is to be estimated from a sample of observations that can be regarded as realizations from some unknown probability distribution and that, in order to do so, it is necessary to specify a model for the distribution. There are usually many alternative plausible models available and, in general, they each lead to different estimates of μ.Consider a sample of data,x, and a set of K models M
=
(
M
1
,
…
,
M
K
), which we will assume to contain the true model M
t. Each M
k consists of a family of distributions P
(
x
∣
η
k
,
M
k
), where η
k represents a parameter (or vector of parameters).The prior probability thatM
k is the true model is denoted by P
(
M
k
) and the prior distribution of the parameters of M
k (given that M
k is true) by P
(
η
k
∣
M
k
). Conditioning on the data x and integrating out the parameter η
k, one obtains the following posterior model probabilities:
(1)
P
(
M
k
∣
x
)
=
P
(
x
∣
M
k
)
P
(
M
k
)
∑
j
=
1
K
P
(
x
∣
M
j
)
P
(
M
j
)
,
where
(2)
P
(
x
∣
M
j
)
=
∫
‍
P
(
x
∣
η
k
,
M
k
)
P
(
η
k
∣
M
k
)
d
η
k
is the integrated likelihood under M
k. If P
(
η
k
∣
M
k
) is a discrete distribution, the integral in (2) is replaced by a sum.


## 2.2. Bayesian Model Selection

Within this framework, classical Bayesian model selection involves selecting the model with the highest posterior. Sometimes Bayes factors are used. From the Bayes factor framework, the Bayes factor (Kass and Raftery [23]) for M
i versus model M
j is defined by
(3)
B
i
j
=
P
(
x
∣
M
i
)
P
(
x
∣
M
j
)
.ModelM
i is chosen if B
i
j
>
1. Under certain assumptions and approximations (in particular the Laplace approximation) and taking all candidate models as a priori equally likely to be true, this leads to the Bayesian information criterion (BIC), also known as the Schwarz criterion [24]. More information on Bayesian model selection and applications can be found in the studies by Guan and Stephens [25], Nguefack-Tsague [26], Carvalho and Scott [27], Fridley [28], Robert [29], Liang et al. [30], and Bernado and Smith [31].


## 2.3. Focused Information Criterion (FIC)

As one can notice, in classical Bayesian model selection, a single (selected) model is used to explain all aspects of data, that is, regardless of the purpose of the selection, irrespective of the inference to follow (Claeskens and Hjort [12]). Allen [10] first developed this idea of focusing on a parameter of interest in a prediction problem in which the prediction at a particular value (target) of the regressor vector different from the values in the sample is of interest. Geisser [32] focused on prediction as a major inferential goal, rather than estimation, under many circumstances. In that method, the steps in the derivation of Mallows’ C
p are repeated for this target; thus, this criterion depends on that particular value and is therefore an early precursor to the FIC. In the FIC framework, a parameter of interest, say, μ, must have a definition making it meaningful across competing models. FIC methodology uses general parametric models and maximum likelihood as the estimation method in a general large sample theory. The FIC is derived as the result of establishing an unbiased estimation of the limiting risk of any submodel-based estimator of the parameter μ. FIC is based on the (crucial) assumption that the true data generating mechanism is contained in the largest parametric model considered. The candidate model with the smallest value of FIC is chosen. As one can see, FIC actually is based on frequentist approach; so far, up to our knowledge, Bayesian equivalent has not yet been considered.


## 2.4. Bayesian Model Averaging

Bayesian model averaging (BMA) is used to deal with the problem of model uncertainty. A discussion on the issue of model uncertainty is given in the study by Clyde and George [33]. Let μ be a quantity of interest depending on x, for example, a future observation from the same process that generated x. The idea is to use a weighted average of the estimates of μ obtained using each of the alternative models, rather than the estimate obtained using any single model. More precisely, the posterior distribution of μ is given by
(4)
P
(
μ
∣
x
)
=
∑
k
=
1
K
‍
P
(
μ
∣
x
,
M
k
)
P
(
M
k
∣
x
)
.
Note that P
(
μ
∣
x
) is a weighted average of the posterior distributions P
(
μ
∣
M
k
,
x
)
,
k
=
1
,
…
,
K, where the kth weight, P
(
M
k
∣
x
), is the posterior probability that M
k is the true model.The posterior distribution ofμ, conditioned on M
k being true, is given by
(5)
P
(
μ
∣
x
,
M
k
)
=
∫
‍
P
(
μ
∣
η
k
,
M
k
)
P
(
η
k
∣
x
,
M
k
)
d
η
k
.
The posterior mean and posterior variance are given by
(6)
μ
^
bma
=
E
bma
(
μ
∣
x
)
=
∑
k
=
1
K
‍
E
(
μ
∣
x
,
M
k
)
P
(
M
k
∣
x
)
,
Va
r
bma
(
μ
∣
x
)
=
∑
k
=
1
K
‍
[
(
E
(
μ
∣
x
,
M
k
)
-
μ
^
bma
)
2
Var
⁡
(
μ
∣
x
,
M
k
)
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
+
(
E
(
μ
∣
x
,
M
k
)
-
μ
^
bma
)
2
]
P
(
M
k
∣
x
)
.
Raftery et al. [34] call this averaging schemeBayesian model averaging. Leamer [35] and Draper [36] advocate the same idea. Madigan and Raftery [37] note that BMA provides better predictive performance than any single model if the measure of performance is Good’s [38] logarithm score rule, under the posterior distribution of μ given x.Hoeting et al. [39] give an extensive framework of BMA methodology and applications for different statistical models. Various real data and simulation studies have investigated the predictive performance of BMA (Clyde [40]; Clyde and George [33]). Nguefack-Tsague [41] uses BMA in the context of estimating a multivariate mean.


## 2.5. Challenges

Implementing BMA is demanding, especially the computation of the integrated likelihood. Software for BMA implementation, as well as some BMA papers, can be found at “http://www.research.att.com/~volinsky/bma.html”. An R [42] package for BMA is now available for computational purposes; this package provides ways for carrying out BMA for linear regression, generalized linear models, and survival analysis using Cox proportional hazard models. For computations, Monte Carlo methods, or approximating methods, are used; thus, many BMA applications are based on the BIC, an asymptotic approximation of the log posterior odds when the prior odds are all equal.Another problem is the selection of priors for both models and parameters. In most cases, a uniform prior is used for each model; that is,P
(
M
k
)
=
1
/
K, k
=
1,2
,
…
,
K. When the number of models is large, model search strategies are sometimes used to reduce the set of models, by eliminating those that seem comparatively less compatible with the data.


## 3. Using BMA for Model Selection

The purpose of this section is to define the focused Bayesian model averaging (FoBMA).Definition 1.
For a set ofK models M
=
(
M
1
,
…
,
M
K
), M
k with focus parameter μ
k, under BMA framework, the selection criterion FoBMA consists of choosing M
j
∈
M for which its estimate μ
^
j is closest (in terms of squared error) to BMA estimate.Proposition 2.
Under the square error loss and the weighted posterior probability of (4), FoBMA is an optimal model choice.Proof.
Conditioning on all models, that is, under (4), the optimal choice M
j
∈
M is the one for which its estimate μ
^
j minimizes
(7)
∫
‍
(
μ
^
k
-
μ
)
2
P
(
μ
∣
x
)
d
μ
.
This is equivalent to minimizing
(8)
∫
(
μ
^
k
-
μ
)
2
{
∑
l
=
1
K
‍
P
(
μ
∣
x
,
M
l
)
P
(
M
l
∣
x
)
}
d
μ
=
∑
l
=
1
K
‍
P
(
M
l
∣
x
)
{
∫
(
μ
^
k
-
μ
^
l
+
μ
^
l
-
μ
)
2
P
(
μ
∣
x
,
M
l
)
d
μ
}
=
∑
l
=
1
K
‍
P
(
M
l
∣
x
)
(
μ
^
k
-
μ
^
l
)
2
︸
(
1
)
+
∑
l
=
1
K
‍
P
(
M
l
∣
x
)
Var
⁡
(
μ
∣
x
,
M
l
)
︸
(
2
)
.
The term (2) does not depend on μ
^
k and, denoting
(9)
μ
^
bma
=
∑
l
=
1
K
‍
μ
^
l
P
(
M
l
∣
x
)
,
the term (1) can be rearranged as (see also the study by Bernardo and Smith [31])
(10)
∑
l
=
1
K
‍
(
μ
^
k
-
μ
^
l
)
2
P
(
M
l
∣
x
)
=
∑
l
=
1
K
‍
(
(
μ
^
bma
-
μ
^
l
)
+
(
μ
^
k
-
μ
^
bma
)
)
2
P
(
M
l
∣
x
)
=
∑
l
=
1
K
‍
(
μ
^
bma
-
μ
^
l
)
2
P
(
M
l
∣
x
)
︸
(
1
′
)
+
∑
l
=
1
K
‍
(
μ
^
k
-
μ
^
bma
)
2
P
(
M
l
∣
x
)
︸
(
2
′
)
+
2
∑
l
=
1
K
‍
(
μ
^
bma
-
μ
^
l
)
(
μ
^
k
-
μ
^
bma
)
P
(
M
l
∣
x
)
︸
(
3
′
)
.
Since
(11)
(
2
′
)
=
(
μ
^
k
-
μ
^
bma
)
2
∑
l
=
1
K
‍
P
(
M
l
∣
x
)
=
(
μ
^
k
-
μ
^
bma
)
2
,
as well as
(12)
(
3
′
)
=
(
μ
^
k
-
μ
^
bma
)
∑
l
=
1
K
‍
(
μ
^
bma
-
μ
^
l
)
P
(
M
l
∣
x
)
=
(
μ
^
k
-
μ
^
bma
)
(
μ
^
bma
-
∑
l
=
1
K
‍
μ
^
l
P
(
M
l
∣
x
)
)
=
0
,
and since (1
′) does not depend on μ
^
k, the only term that depends on μ
^
k is (2
′). Hence, the preferred M
j is the one whose estimate μ
^
j is closest to the BMA estimate μ
^
bma.In particular, for two models, Corollary3 shows that FoBMA reduces to the classical Bayesian model selection scheme of choosing the model with the highest posterior probability.Corollary 3.
For two models, the selected model is the one with the highest posterior probability.Proof.
From Proposition2, let us find the distance between each model and BMA model. Consider
(13)
(
μ
^
1
-
μ
^
bma
)
2
=
(
μ
^
1
-
P
(
M
1
∣
x
)
μ
^
1
-
P
(
M
2
∣
x
)
μ
^
2
)
2
=
(
(
1
-
P
(
M
1
∣
x
)
)
μ
^
1
-
P
(
M
2
∣
x
)
μ
^
2
)
2
=
(
P
(
M
2
∣
x
)
μ
^
1
-
P
(
M
2
∣
x
)
μ
^
2
)
2
=
P
2
(
M
2
∣
x
)
(
μ
^
1
-
μ
^
2
)
2
.
Similarly,
(14)
(
μ
^
2
-
μ
^
bma
)
2
=
P
2
(
M
1
∣
x
)
(
μ
^
2
-
μ
^
1
)
2
.
Therefore, M
1 is selected if
(15)
P
2
(
M
2
∣
x
)
(
μ
^
1
-
μ
^
2
)
2
<
P
2
(
M
1
∣
x
)
(
μ
^
2
-
μ
^
1
)
2
,
that is, if P
(
M
1
∣
x
)
>
P
(
M
2
∣
x
). Hence, FoBMA is equivalent to selecting the model with the highest posterior probability.


## 4. Applications

In this section, we apply the methodology to three models: linear regression, logistic regression, and survival analysis. The three following examples have been widely used in Bayesian model averaging (Raftery et al. [43]) and are available in the R packages M
A
S
S andsurvival. They are also used as tutorial in the R package B
M
A. Since these are parametric models, the focus parameter, μ, is in every case the regression coefficient (β). FoBMA is compared to the classical well-known Bayesian information criterion (BIC). All computations were performed with R [42].


### 4.1. Linear Regression

In this subsection, FoBMA is applied to data of the effect of punishment regimes on crime rates (Ehrlich [44]), used in the study by Raftery [45]. It can be downloaded in the R package M
A
S
S.


#### 4.1.1. Data Description

Criminologists are interested in the effect of punishment regimes on crime rates. This is a dataset on per capita crime rates in 47 U.S. states in 1960 originally published in the study by Ehrlich [44]. The variables have been rescaled to convenient numbers.Table1 describes all variables used for linear regression. The dependent variable is the rate of crimes in a particular category per head, and 15 potential independent variables perceived to be associated with crime rates (Ehrlich [44]).Table 1
Description of variables used for linear regression (Ehrlich [44]).
Label
Description
M
Percentage of males aged 14–24
S
o
Indicator variable for a Southern state (South = 1 and others = 0)
E
d
Mean years of schooling
P
o
1
Police expenditure in 1960
P
o
2
Police expenditure in 1959
L
F
Labour force participation rate
M
.
F
Number of males per 1000 females
P
o
p
State population
N
W
Number of nonwhites per 1000 people
U
1
Unemployment rate of urban males 14–24
U
2
Unemployment rate of urban males 35–39
G
D
P
Gross domestic product per head
I
n
e
q
Income inequality
P
r
o
b
Probability of apprehension and imprisonment
T
i
m
e
Average time served in state prisons (years)
Y
Rate of crimes in a particular category per head of population


#### 4.1.2. Results

Table2 shows that the classical BIC (no focus) selects the model with variables M
,
E
d
,
P
o
1
,
L
F
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e (-BIC = 55.91). Since the number of explanatory variables in this case is 15, the model space is small enough, which allows for full enumeration of model space. If the focus was theprobability of imprisonment (the number of offenders imprisoned per offense known), the selected model is E
d
,
P
o
1
,
P
o
p
,
N
W
,
I
n
e
q
,
P
r
o
b which is ranked 25 with BIC. If the focus was theaverage time spent in state prisons, the selected model is M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e which is ranked 18 with BIC. The other focuses parameters show that there is a great discrepancy between the FoBMA and the classical BIC.Table 2
Linear regression example: selected (best) model according to the focus parameter (FoBMA) compared to its BIC rank and value, the probability that the focus is not null (100P(β
≠
0)), its BMA mean (μ
^
bma
=
E
bma
(
μ
∣
x
)), and standard deviation (S
D
bma
(
μ
∣
x
)
).
Focus
Best model (FoBMA)
-BIC
Rank
β
≠
0
Mean
SD
M
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
1
,
U
2
,
G
D
P
,
n
e
q
,
P
r
o
b
51.6
34
97.5
1.4
0.53
S
o
M
,
S
o
,
E
d
,
P
o
1
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
52.1
30
6.3
0.007
0.04
E
d
M
,
E
d
,
P
o
1
,
L
F
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.2
29
100
2.13
0.51
P
o
1
M
,
E
d
,
P
o
1
,
N
W
,
U
1
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.5
23
75.4
0.67
0.42
P
o
2
M
,
E
d
,
P
o
2
,
N
W
,
U
1
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
51.2
45
24.6
0.22
0.40
L
F
M
,
E
d
,
P
o
1
,
L
F
,
N
W
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
50.9
46
2.1
0.003
0.08
M
.
F
M
,
E
d
,
P
o
1
,
M
.
F
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.3
27
5.2
−0.07
0.45
P
o
p
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.9
18
28.9
−0.02
0.04
N
W
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
1
,
U
2
,
I
n
e
q
,
P
r
o
b
52.5
22
89.6
0.09
0.05
U
1
M
,
E
d
,
P
o
1
,
N
W
,
U
1
,
U
2
,
I
n
e
q
,
P
r
o
b
52.3
28
11.9
−0.04
0.14
U
2
M
,
E
d
,
P
o
1
,
N
W
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
54.7
3
83.9
0.27
0.19
G
D
P
M
,
E
d
,
P
o
1
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
51.4
37
33.8
0.20
0.35
I
n
e
q
M
,
E
d
,
P
o
2
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
53.1
14
100.0
1.38
0.33
P
r
o
b
E
d
,
P
o
1
,
P
o
p
,
N
W
,
I
n
e
q
,
P
r
o
b
52.4
25
98.8
−0.25
0.10
T
i
m
e
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.9
18
44.5
−0.13
0.18


### 4.2. Logistic Regression

In this subsection, FoBMA is applied to data of risk factors associated with low infants birth weights (Hosmer and Lemeshow [46]). These data were also used in the study by Raftery [45] and can be downloaded in the R package M
A
S
S. The aim was to study the risk factors associated with low infants birth weights.


#### 4.2.1. Data Description

The “birthwt” data frame has 189 rows and 10 columns. The data were collected at Baystate Medical Center, Springfield, Massachusetts, during 1986.


#### 4.2.2. Results

Table3 describes all variables used for logistic regression. The outcome variable is low (indicator of birth weight (BWT) less than 2.5 kg), and 7 potential independent variables perceived to be associated with low birth weight (Hosmer and Lemeshow [46]).Table 3
Description of variables used for logistic regression (Hosmer and Lemeshow [46]).
Label
Description
L
B
W
T
Indicator of birth weight (BWT) less than 2.5 kg
A
G
E
Mother’s age in years
L
W
T
Mother’s weight in pounds at last menstrual period
R
A
C
E
Mother’s race (1 = white, 2 = black, and 3 = other)
S
M
O
K
E
Smoking status during pregnancy
P
T
L
Number of previous premature labours
H
T
History of hypertension
U
I
Presence of uterine irritability
F
T
V
Number of physician visits during the first trimester
B
W
T
Birth weight in gramsTable4 shows that BIC selects the one with variables L
W
T
,
H
T (-BIC = 753). If the focus was the L
W
T, the selected model is L
W
T
,
S
M
O
K
E
,
U
I which is ranked 38 with BIC. If the focus was the U
I, the selected model is L
W
T
,
P
T
L
,
U
I which is ranked 39 with BIC. The other focuses parameters show that there is a great discrepancy between the FoBMA and BIC. If hypertension (HT) is the focus, the selected model is the one with variables R
A
C
E
,
S
M
O
K
E
,
H
T; but this model is ranked 20 if the focus is S
M
O
K
E. The model A
G
E
,
L
W
T
,
P
T
L
,
H
T
,
U
I is ranked 4 if the focus is A
G
E, 11 if L
W
T, 14 if P
T
L, 23 if H
T, and 5 if U
I. It is not best (number 1) to estimate any of these focuses. Other applications of FIC for logistic regression in dental restoration can be found in the study by Candolo [47].Table 4
Logistic regression example: selected (best) model according to the focus parameter (FoBMA) compared to its BIC rank and value, the probability that the focus is not null (100P(β
≠
0)), its BMA mean (μ
^
bma
=
E
bma
(
μ
∣
x
)), and standard deviation (S
D
bma
(
μ
∣
x
)
).
Focus
Best model (FoBMA)
-BIC
Rank
β
≠
0
Mean
SD
A
G
E
A
G
E
,
L
W
T
,
H
T
749.8
20
12.6
0.008
0.024
L
W
T
L
W
T
,
S
M
O
K
E
,
U
I
748.6
38
68.7
−0.012
0.010
R
A
C
E
2
R
A
C
E
,
S
M
O
K
E
,
H
T
748.1
46
9.6
0.115
0.390
R
A
C
E
3
L
W
T
,
R
A
C
E
,
H
T
748.2
44
9.6
0.092
0.320
S
M
O
K
E
L
W
T
,
S
M
O
K
E
,
P
T
L
,
H
T
,
U
I
748.5
40
33.2
0.256
0.43
P
T
L
1
L
W
T
,
S
M
O
K
E
,
P
T
L
,
H
T
,
U
I
748.5
40
35.1
0.62
0.089
P
T
L
2
L
W
T
,
S
M
O
K
E
,
P
T
L
,
H
T
,
U
I
748.5
40
35.1
0.170
0.61
P
T
L
3
A
G
E
,
P
T
L
,
H
T
749.3
29
35.1
−4.91
520
H
T
R
A
C
E
,
S
M
O
K
E
,
H
T
748.1
46
65.4
1.167
1
U
I
L
W
T
,
P
T
L
,
U
I
748.5
39
33.8
0.310
0.51
F
T
V
L
W
T
,
H
T
,
F
T
V
748.7
33
1.5
−0.001
0.024


### 4.3. Survival Analysis

In this subsection, FoBMA is applied to the Mayo Clinic Primary Biliary Cirrhosis Data (Therneau and Grambsch [48]). It was used in the study by Volinsky et al. [49] and can be downloaded in the R packagesurvival.


#### 4.3.1. Data Description

The data are from the Mayo Clinic trial on primary biliary cirrhosis (PBC) of the liver conducted between 1974 and 1984. A total of 424 PBC patients, referred to Mayo Clinic during that ten-year interval, met eligibility criteria for the randomized placebo controlled trial of the drug D-penicillamine. The first 312 cases in the dataset participated in the randomized trial and contain largely complete data. The additional 112 cases did not participate in the clinical trial but consented to have basic measurements recorded and to be followed for survival. Six of those cases were lost to follow-up shortly after diagnosis, so the data here are on an additional 106 cases as well as the 312 randomized participants.


#### 4.3.2. Results

Table5 describes all variables used for survival analysis. The dependent variable was the survival time while the 18 others were the independent variables.Table 5
Description of variables used for logistic regression (Hosmer and Lemeshow [46]).
Label
Description
A
g
e
Age of patients in years
A
l
b
u
m
i
n
Serum albumin (mg/dL)
A
l
k
.
p
h
o
s
Alkaline phosphatase (U/liter)
A
s
c
i
t
e
s
Presence of ascites
A
s
t
Aspartate aminotransferase, once called SGOT (U/mL)
B
i
l
i
Serum bilirubin (mg/dL)
C
h
o
l
Serum cholesterol (mg/dL)
C
o
p
p
e
r
Urine copper (ug/day)
E
d
e
m
a
0 no edema, 0.5 untreated or successfully treated edema, and 1 edema despite diuretic therapy
H
e
p
a
t
o
Presence of hepatomegaly or enlarged liver
P
l
a
t
e
l
e
t
Platelet count
P
r
o
t
i
m
e
Standardized blood clotting time
S
e
x
Male/female
S
p
i
d
e
r
s
Blood vessel malformations in the skin
S
t
a
g
e
Histologic stage of disease (which needs biopsy)
S
t
a
t
u
s
Status at endpoint, 0/1/2 for censored, transplant, or dead
T
i
m
e
Number of days between the date of registration and the earlier date of death or transplantation
T
r
t
1/2/NA for D-penicillamine, placebo, not randomized
T
r
i
g
Triglycerides (mg/dL)Table6 shows that BIC selects the model with variables A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u. If the focus was the B
i
l
i, the selected model is A
g
e
,
B
i
l
i
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u which is ranked 14 with BIC. If the focus was the A
l
b
u, the selected model is A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
A
s
t
,
C
o
p
p
,
A
l
b
u which is ranked 7 with BIC. The other focuses parameters show that there is a great discrepancy between the FoBMA and BIC. If the focus was on age, BIC and FoBMA select the same model.Table 6
Survival regression example: selected (best) model according to the focus parameter (FoBMA) compared to its BIC rank and value, the probability that the focus is not null (100P(β
≠
0)), its BMA mean (μ
^
bma
=
E
bma
(
μ
∣
x
)), and standard deviation (S
D
bma
(
μ
∣
x
)
).
Focus
Best model (FoBMA)
-BIC
Rank
β
≠
0
Mean
SD
A
g
e
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
174.8
1
100
0.031
0.009
A
l
k
P
A
g
e
,
A
l
k
P
,
B
i
l
i
,
E
d
e
m
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
168.9
34
7.8
0.000
0.000
A
s
c
i
A
g
e
,
A
s
c
i
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
168.9
24
1.7
0.003
0.045
B
i
l
i
A
g
e
,
B
i
l
i
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
170.8
14
100
0.78
0.13
E
d
e
m
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
l
a
t
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
170.8
15
84.7
0.74
0.43
H
e
p
a
A
g
e
,
B
i
l
i
,
E
d
e
m
,
H
e
p
a
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
170.7
16
2.8
0.006
0.05
P
l
a
t
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
l
a
t
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
168.8
39
5.4
0.000
0.000
P
r
o
t
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
t
a
g
,
A
l
b
u
171.0
13
78
2.5
1.7
S
e
x
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
e
x
,
C
o
p
p
,
A
l
b
u
170.4
18
5.6
−0.018
0.1
A
s
t
A
g
e
,
B
i
l
i
,
P
r
o
t
,
A
s
t
,
C
o
p
p
,
A
l
b
u
169.6
28
20.9
0.097
0.23
S
p
i
d
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
p
i
d
,
C
o
p
p
,
A
l
b
u
170.0
27
1.4
0.000
0.026
S
t
a
g
A
g
e
,
A
l
k
P
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
169.5
29
36.6
0.1
0.16
T
r
t
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
T
r
t
,
C
o
p
p
,
A
l
b
u
170.2
23
1.6
0.002
0.027
C
o
p
p
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
A
s
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
171.3
11
73.8
0.26
0.2
A
l
b
u
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
A
s
t
,
C
o
p
p
,
A
l
b
u
172.0
7
100
−2.8
8


## 4.1. Linear Regression

In this subsection, FoBMA is applied to data of the effect of punishment regimes on crime rates (Ehrlich [44]), used in the study by Raftery [45]. It can be downloaded in the R package M
A
S
S.


### 4.1.1. Data Description

Criminologists are interested in the effect of punishment regimes on crime rates. This is a dataset on per capita crime rates in 47 U.S. states in 1960 originally published in the study by Ehrlich [44]. The variables have been rescaled to convenient numbers.Table1 describes all variables used for linear regression. The dependent variable is the rate of crimes in a particular category per head, and 15 potential independent variables perceived to be associated with crime rates (Ehrlich [44]).Table 1
Description of variables used for linear regression (Ehrlich [44]).
Label
Description
M
Percentage of males aged 14–24
S
o
Indicator variable for a Southern state (South = 1 and others = 0)
E
d
Mean years of schooling
P
o
1
Police expenditure in 1960
P
o
2
Police expenditure in 1959
L
F
Labour force participation rate
M
.
F
Number of males per 1000 females
P
o
p
State population
N
W
Number of nonwhites per 1000 people
U
1
Unemployment rate of urban males 14–24
U
2
Unemployment rate of urban males 35–39
G
D
P
Gross domestic product per head
I
n
e
q
Income inequality
P
r
o
b
Probability of apprehension and imprisonment
T
i
m
e
Average time served in state prisons (years)
Y
Rate of crimes in a particular category per head of population


### 4.1.2. Results

Table2 shows that the classical BIC (no focus) selects the model with variables M
,
E
d
,
P
o
1
,
L
F
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e (-BIC = 55.91). Since the number of explanatory variables in this case is 15, the model space is small enough, which allows for full enumeration of model space. If the focus was theprobability of imprisonment (the number of offenders imprisoned per offense known), the selected model is E
d
,
P
o
1
,
P
o
p
,
N
W
,
I
n
e
q
,
P
r
o
b which is ranked 25 with BIC. If the focus was theaverage time spent in state prisons, the selected model is M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e which is ranked 18 with BIC. The other focuses parameters show that there is a great discrepancy between the FoBMA and the classical BIC.Table 2
Linear regression example: selected (best) model according to the focus parameter (FoBMA) compared to its BIC rank and value, the probability that the focus is not null (100P(β
≠
0)), its BMA mean (μ
^
bma
=
E
bma
(
μ
∣
x
)), and standard deviation (S
D
bma
(
μ
∣
x
)
).
Focus
Best model (FoBMA)
-BIC
Rank
β
≠
0
Mean
SD
M
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
1
,
U
2
,
G
D
P
,
n
e
q
,
P
r
o
b
51.6
34
97.5
1.4
0.53
S
o
M
,
S
o
,
E
d
,
P
o
1
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
52.1
30
6.3
0.007
0.04
E
d
M
,
E
d
,
P
o
1
,
L
F
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.2
29
100
2.13
0.51
P
o
1
M
,
E
d
,
P
o
1
,
N
W
,
U
1
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.5
23
75.4
0.67
0.42
P
o
2
M
,
E
d
,
P
o
2
,
N
W
,
U
1
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
51.2
45
24.6
0.22
0.40
L
F
M
,
E
d
,
P
o
1
,
L
F
,
N
W
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
50.9
46
2.1
0.003
0.08
M
.
F
M
,
E
d
,
P
o
1
,
M
.
F
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.3
27
5.2
−0.07
0.45
P
o
p
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.9
18
28.9
−0.02
0.04
N
W
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
1
,
U
2
,
I
n
e
q
,
P
r
o
b
52.5
22
89.6
0.09
0.05
U
1
M
,
E
d
,
P
o
1
,
N
W
,
U
1
,
U
2
,
I
n
e
q
,
P
r
o
b
52.3
28
11.9
−0.04
0.14
U
2
M
,
E
d
,
P
o
1
,
N
W
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
54.7
3
83.9
0.27
0.19
G
D
P
M
,
E
d
,
P
o
1
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
51.4
37
33.8
0.20
0.35
I
n
e
q
M
,
E
d
,
P
o
2
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
53.1
14
100.0
1.38
0.33
P
r
o
b
E
d
,
P
o
1
,
P
o
p
,
N
W
,
I
n
e
q
,
P
r
o
b
52.4
25
98.8
−0.25
0.10
T
i
m
e
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.9
18
44.5
−0.13
0.18


## 4.1.1. Data Description

Criminologists are interested in the effect of punishment regimes on crime rates. This is a dataset on per capita crime rates in 47 U.S. states in 1960 originally published in the study by Ehrlich [44]. The variables have been rescaled to convenient numbers.Table1 describes all variables used for linear regression. The dependent variable is the rate of crimes in a particular category per head, and 15 potential independent variables perceived to be associated with crime rates (Ehrlich [44]).Table 1
Description of variables used for linear regression (Ehrlich [44]).
Label
Description
M
Percentage of males aged 14–24
S
o
Indicator variable for a Southern state (South = 1 and others = 0)
E
d
Mean years of schooling
P
o
1
Police expenditure in 1960
P
o
2
Police expenditure in 1959
L
F
Labour force participation rate
M
.
F
Number of males per 1000 females
P
o
p
State population
N
W
Number of nonwhites per 1000 people
U
1
Unemployment rate of urban males 14–24
U
2
Unemployment rate of urban males 35–39
G
D
P
Gross domestic product per head
I
n
e
q
Income inequality
P
r
o
b
Probability of apprehension and imprisonment
T
i
m
e
Average time served in state prisons (years)
Y
Rate of crimes in a particular category per head of population


## 4.1.2. Results

Table2 shows that the classical BIC (no focus) selects the model with variables M
,
E
d
,
P
o
1
,
L
F
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e (-BIC = 55.91). Since the number of explanatory variables in this case is 15, the model space is small enough, which allows for full enumeration of model space. If the focus was theprobability of imprisonment (the number of offenders imprisoned per offense known), the selected model is E
d
,
P
o
1
,
P
o
p
,
N
W
,
I
n
e
q
,
P
r
o
b which is ranked 25 with BIC. If the focus was theaverage time spent in state prisons, the selected model is M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e which is ranked 18 with BIC. The other focuses parameters show that there is a great discrepancy between the FoBMA and the classical BIC.Table 2
Linear regression example: selected (best) model according to the focus parameter (FoBMA) compared to its BIC rank and value, the probability that the focus is not null (100P(β
≠
0)), its BMA mean (μ
^
bma
=
E
bma
(
μ
∣
x
)), and standard deviation (S
D
bma
(
μ
∣
x
)
).
Focus
Best model (FoBMA)
-BIC
Rank
β
≠
0
Mean
SD
M
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
1
,
U
2
,
G
D
P
,
n
e
q
,
P
r
o
b
51.6
34
97.5
1.4
0.53
S
o
M
,
S
o
,
E
d
,
P
o
1
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
52.1
30
6.3
0.007
0.04
E
d
M
,
E
d
,
P
o
1
,
L
F
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.2
29
100
2.13
0.51
P
o
1
M
,
E
d
,
P
o
1
,
N
W
,
U
1
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.5
23
75.4
0.67
0.42
P
o
2
M
,
E
d
,
P
o
2
,
N
W
,
U
1
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
51.2
45
24.6
0.22
0.40
L
F
M
,
E
d
,
P
o
1
,
L
F
,
N
W
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
50.9
46
2.1
0.003
0.08
M
.
F
M
,
E
d
,
P
o
1
,
M
.
F
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.3
27
5.2
−0.07
0.45
P
o
p
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.9
18
28.9
−0.02
0.04
N
W
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
1
,
U
2
,
I
n
e
q
,
P
r
o
b
52.5
22
89.6
0.09
0.05
U
1
M
,
E
d
,
P
o
1
,
N
W
,
U
1
,
U
2
,
I
n
e
q
,
P
r
o
b
52.3
28
11.9
−0.04
0.14
U
2
M
,
E
d
,
P
o
1
,
N
W
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
54.7
3
83.9
0.27
0.19
G
D
P
M
,
E
d
,
P
o
1
,
U
2
,
G
D
P
,
I
n
e
q
,
P
r
o
b
51.4
37
33.8
0.20
0.35
I
n
e
q
M
,
E
d
,
P
o
2
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
53.1
14
100.0
1.38
0.33
P
r
o
b
E
d
,
P
o
1
,
P
o
p
,
N
W
,
I
n
e
q
,
P
r
o
b
52.4
25
98.8
−0.25
0.10
T
i
m
e
M
,
E
d
,
P
o
1
,
P
o
p
,
N
W
,
U
2
,
I
n
e
q
,
P
r
o
b
,
T
i
m
e
52.9
18
44.5
−0.13
0.18


## 4.2. Logistic Regression

In this subsection, FoBMA is applied to data of risk factors associated with low infants birth weights (Hosmer and Lemeshow [46]). These data were also used in the study by Raftery [45] and can be downloaded in the R package M
A
S
S. The aim was to study the risk factors associated with low infants birth weights.


### 4.2.1. Data Description

The “birthwt” data frame has 189 rows and 10 columns. The data were collected at Baystate Medical Center, Springfield, Massachusetts, during 1986.


### 4.2.2. Results

Table3 describes all variables used for logistic regression. The outcome variable is low (indicator of birth weight (BWT) less than 2.5 kg), and 7 potential independent variables perceived to be associated with low birth weight (Hosmer and Lemeshow [46]).Table 3
Description of variables used for logistic regression (Hosmer and Lemeshow [46]).
Label
Description
L
B
W
T
Indicator of birth weight (BWT) less than 2.5 kg
A
G
E
Mother’s age in years
L
W
T
Mother’s weight in pounds at last menstrual period
R
A
C
E
Mother’s race (1 = white, 2 = black, and 3 = other)
S
M
O
K
E
Smoking status during pregnancy
P
T
L
Number of previous premature labours
H
T
History of hypertension
U
I
Presence of uterine irritability
F
T
V
Number of physician visits during the first trimester
B
W
T
Birth weight in gramsTable4 shows that BIC selects the one with variables L
W
T
,
H
T (-BIC = 753). If the focus was the L
W
T, the selected model is L
W
T
,
S
M
O
K
E
,
U
I which is ranked 38 with BIC. If the focus was the U
I, the selected model is L
W
T
,
P
T
L
,
U
I which is ranked 39 with BIC. The other focuses parameters show that there is a great discrepancy between the FoBMA and BIC. If hypertension (HT) is the focus, the selected model is the one with variables R
A
C
E
,
S
M
O
K
E
,
H
T; but this model is ranked 20 if the focus is S
M
O
K
E. The model A
G
E
,
L
W
T
,
P
T
L
,
H
T
,
U
I is ranked 4 if the focus is A
G
E, 11 if L
W
T, 14 if P
T
L, 23 if H
T, and 5 if U
I. It is not best (number 1) to estimate any of these focuses. Other applications of FIC for logistic regression in dental restoration can be found in the study by Candolo [47].Table 4
Logistic regression example: selected (best) model according to the focus parameter (FoBMA) compared to its BIC rank and value, the probability that the focus is not null (100P(β
≠
0)), its BMA mean (μ
^
bma
=
E
bma
(
μ
∣
x
)), and standard deviation (S
D
bma
(
μ
∣
x
)
).
Focus
Best model (FoBMA)
-BIC
Rank
β
≠
0
Mean
SD
A
G
E
A
G
E
,
L
W
T
,
H
T
749.8
20
12.6
0.008
0.024
L
W
T
L
W
T
,
S
M
O
K
E
,
U
I
748.6
38
68.7
−0.012
0.010
R
A
C
E
2
R
A
C
E
,
S
M
O
K
E
,
H
T
748.1
46
9.6
0.115
0.390
R
A
C
E
3
L
W
T
,
R
A
C
E
,
H
T
748.2
44
9.6
0.092
0.320
S
M
O
K
E
L
W
T
,
S
M
O
K
E
,
P
T
L
,
H
T
,
U
I
748.5
40
33.2
0.256
0.43
P
T
L
1
L
W
T
,
S
M
O
K
E
,
P
T
L
,
H
T
,
U
I
748.5
40
35.1
0.62
0.089
P
T
L
2
L
W
T
,
S
M
O
K
E
,
P
T
L
,
H
T
,
U
I
748.5
40
35.1
0.170
0.61
P
T
L
3
A
G
E
,
P
T
L
,
H
T
749.3
29
35.1
−4.91
520
H
T
R
A
C
E
,
S
M
O
K
E
,
H
T
748.1
46
65.4
1.167
1
U
I
L
W
T
,
P
T
L
,
U
I
748.5
39
33.8
0.310
0.51
F
T
V
L
W
T
,
H
T
,
F
T
V
748.7
33
1.5
−0.001
0.024


## 4.2.1. Data Description

The “birthwt” data frame has 189 rows and 10 columns. The data were collected at Baystate Medical Center, Springfield, Massachusetts, during 1986.


## 4.2.2. Results

Table3 describes all variables used for logistic regression. The outcome variable is low (indicator of birth weight (BWT) less than 2.5 kg), and 7 potential independent variables perceived to be associated with low birth weight (Hosmer and Lemeshow [46]).Table 3
Description of variables used for logistic regression (Hosmer and Lemeshow [46]).
Label
Description
L
B
W
T
Indicator of birth weight (BWT) less than 2.5 kg
A
G
E
Mother’s age in years
L
W
T
Mother’s weight in pounds at last menstrual period
R
A
C
E
Mother’s race (1 = white, 2 = black, and 3 = other)
S
M
O
K
E
Smoking status during pregnancy
P
T
L
Number of previous premature labours
H
T
History of hypertension
U
I
Presence of uterine irritability
F
T
V
Number of physician visits during the first trimester
B
W
T
Birth weight in gramsTable4 shows that BIC selects the one with variables L
W
T
,
H
T (-BIC = 753). If the focus was the L
W
T, the selected model is L
W
T
,
S
M
O
K
E
,
U
I which is ranked 38 with BIC. If the focus was the U
I, the selected model is L
W
T
,
P
T
L
,
U
I which is ranked 39 with BIC. The other focuses parameters show that there is a great discrepancy between the FoBMA and BIC. If hypertension (HT) is the focus, the selected model is the one with variables R
A
C
E
,
S
M
O
K
E
,
H
T; but this model is ranked 20 if the focus is S
M
O
K
E. The model A
G
E
,
L
W
T
,
P
T
L
,
H
T
,
U
I is ranked 4 if the focus is A
G
E, 11 if L
W
T, 14 if P
T
L, 23 if H
T, and 5 if U
I. It is not best (number 1) to estimate any of these focuses. Other applications of FIC for logistic regression in dental restoration can be found in the study by Candolo [47].Table 4
Logistic regression example: selected (best) model according to the focus parameter (FoBMA) compared to its BIC rank and value, the probability that the focus is not null (100P(β
≠
0)), its BMA mean (μ
^
bma
=
E
bma
(
μ
∣
x
)), and standard deviation (S
D
bma
(
μ
∣
x
)
).
Focus
Best model (FoBMA)
-BIC
Rank
β
≠
0
Mean
SD
A
G
E
A
G
E
,
L
W
T
,
H
T
749.8
20
12.6
0.008
0.024
L
W
T
L
W
T
,
S
M
O
K
E
,
U
I
748.6
38
68.7
−0.012
0.010
R
A
C
E
2
R
A
C
E
,
S
M
O
K
E
,
H
T
748.1
46
9.6
0.115
0.390
R
A
C
E
3
L
W
T
,
R
A
C
E
,
H
T
748.2
44
9.6
0.092
0.320
S
M
O
K
E
L
W
T
,
S
M
O
K
E
,
P
T
L
,
H
T
,
U
I
748.5
40
33.2
0.256
0.43
P
T
L
1
L
W
T
,
S
M
O
K
E
,
P
T
L
,
H
T
,
U
I
748.5
40
35.1
0.62
0.089
P
T
L
2
L
W
T
,
S
M
O
K
E
,
P
T
L
,
H
T
,
U
I
748.5
40
35.1
0.170
0.61
P
T
L
3
A
G
E
,
P
T
L
,
H
T
749.3
29
35.1
−4.91
520
H
T
R
A
C
E
,
S
M
O
K
E
,
H
T
748.1
46
65.4
1.167
1
U
I
L
W
T
,
P
T
L
,
U
I
748.5
39
33.8
0.310
0.51
F
T
V
L
W
T
,
H
T
,
F
T
V
748.7
33
1.5
−0.001
0.024


## 4.3. Survival Analysis

In this subsection, FoBMA is applied to the Mayo Clinic Primary Biliary Cirrhosis Data (Therneau and Grambsch [48]). It was used in the study by Volinsky et al. [49] and can be downloaded in the R packagesurvival.


### 4.3.1. Data Description

The data are from the Mayo Clinic trial on primary biliary cirrhosis (PBC) of the liver conducted between 1974 and 1984. A total of 424 PBC patients, referred to Mayo Clinic during that ten-year interval, met eligibility criteria for the randomized placebo controlled trial of the drug D-penicillamine. The first 312 cases in the dataset participated in the randomized trial and contain largely complete data. The additional 112 cases did not participate in the clinical trial but consented to have basic measurements recorded and to be followed for survival. Six of those cases were lost to follow-up shortly after diagnosis, so the data here are on an additional 106 cases as well as the 312 randomized participants.


### 4.3.2. Results

Table5 describes all variables used for survival analysis. The dependent variable was the survival time while the 18 others were the independent variables.Table 5
Description of variables used for logistic regression (Hosmer and Lemeshow [46]).
Label
Description
A
g
e
Age of patients in years
A
l
b
u
m
i
n
Serum albumin (mg/dL)
A
l
k
.
p
h
o
s
Alkaline phosphatase (U/liter)
A
s
c
i
t
e
s
Presence of ascites
A
s
t
Aspartate aminotransferase, once called SGOT (U/mL)
B
i
l
i
Serum bilirubin (mg/dL)
C
h
o
l
Serum cholesterol (mg/dL)
C
o
p
p
e
r
Urine copper (ug/day)
E
d
e
m
a
0 no edema, 0.5 untreated or successfully treated edema, and 1 edema despite diuretic therapy
H
e
p
a
t
o
Presence of hepatomegaly or enlarged liver
P
l
a
t
e
l
e
t
Platelet count
P
r
o
t
i
m
e
Standardized blood clotting time
S
e
x
Male/female
S
p
i
d
e
r
s
Blood vessel malformations in the skin
S
t
a
g
e
Histologic stage of disease (which needs biopsy)
S
t
a
t
u
s
Status at endpoint, 0/1/2 for censored, transplant, or dead
T
i
m
e
Number of days between the date of registration and the earlier date of death or transplantation
T
r
t
1/2/NA for D-penicillamine, placebo, not randomized
T
r
i
g
Triglycerides (mg/dL)Table6 shows that BIC selects the model with variables A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u. If the focus was the B
i
l
i, the selected model is A
g
e
,
B
i
l
i
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u which is ranked 14 with BIC. If the focus was the A
l
b
u, the selected model is A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
A
s
t
,
C
o
p
p
,
A
l
b
u which is ranked 7 with BIC. The other focuses parameters show that there is a great discrepancy between the FoBMA and BIC. If the focus was on age, BIC and FoBMA select the same model.Table 6
Survival regression example: selected (best) model according to the focus parameter (FoBMA) compared to its BIC rank and value, the probability that the focus is not null (100P(β
≠
0)), its BMA mean (μ
^
bma
=
E
bma
(
μ
∣
x
)), and standard deviation (S
D
bma
(
μ
∣
x
)
).
Focus
Best model (FoBMA)
-BIC
Rank
β
≠
0
Mean
SD
A
g
e
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
174.8
1
100
0.031
0.009
A
l
k
P
A
g
e
,
A
l
k
P
,
B
i
l
i
,
E
d
e
m
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
168.9
34
7.8
0.000
0.000
A
s
c
i
A
g
e
,
A
s
c
i
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
168.9
24
1.7
0.003
0.045
B
i
l
i
A
g
e
,
B
i
l
i
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
170.8
14
100
0.78
0.13
E
d
e
m
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
l
a
t
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
170.8
15
84.7
0.74
0.43
H
e
p
a
A
g
e
,
B
i
l
i
,
E
d
e
m
,
H
e
p
a
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
170.7
16
2.8
0.006
0.05
P
l
a
t
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
l
a
t
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
168.8
39
5.4
0.000
0.000
P
r
o
t
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
t
a
g
,
A
l
b
u
171.0
13
78
2.5
1.7
S
e
x
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
e
x
,
C
o
p
p
,
A
l
b
u
170.4
18
5.6
−0.018
0.1
A
s
t
A
g
e
,
B
i
l
i
,
P
r
o
t
,
A
s
t
,
C
o
p
p
,
A
l
b
u
169.6
28
20.9
0.097
0.23
S
p
i
d
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
p
i
d
,
C
o
p
p
,
A
l
b
u
170.0
27
1.4
0.000
0.026
S
t
a
g
A
g
e
,
A
l
k
P
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
169.5
29
36.6
0.1
0.16
T
r
t
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
T
r
t
,
C
o
p
p
,
A
l
b
u
170.2
23
1.6
0.002
0.027
C
o
p
p
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
A
s
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
171.3
11
73.8
0.26
0.2
A
l
b
u
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
A
s
t
,
C
o
p
p
,
A
l
b
u
172.0
7
100
−2.8
8


## 4.3.1. Data Description

The data are from the Mayo Clinic trial on primary biliary cirrhosis (PBC) of the liver conducted between 1974 and 1984. A total of 424 PBC patients, referred to Mayo Clinic during that ten-year interval, met eligibility criteria for the randomized placebo controlled trial of the drug D-penicillamine. The first 312 cases in the dataset participated in the randomized trial and contain largely complete data. The additional 112 cases did not participate in the clinical trial but consented to have basic measurements recorded and to be followed for survival. Six of those cases were lost to follow-up shortly after diagnosis, so the data here are on an additional 106 cases as well as the 312 randomized participants.


## 4.3.2. Results

Table5 describes all variables used for survival analysis. The dependent variable was the survival time while the 18 others were the independent variables.Table 5
Description of variables used for logistic regression (Hosmer and Lemeshow [46]).
Label
Description
A
g
e
Age of patients in years
A
l
b
u
m
i
n
Serum albumin (mg/dL)
A
l
k
.
p
h
o
s
Alkaline phosphatase (U/liter)
A
s
c
i
t
e
s
Presence of ascites
A
s
t
Aspartate aminotransferase, once called SGOT (U/mL)
B
i
l
i
Serum bilirubin (mg/dL)
C
h
o
l
Serum cholesterol (mg/dL)
C
o
p
p
e
r
Urine copper (ug/day)
E
d
e
m
a
0 no edema, 0.5 untreated or successfully treated edema, and 1 edema despite diuretic therapy
H
e
p
a
t
o
Presence of hepatomegaly or enlarged liver
P
l
a
t
e
l
e
t
Platelet count
P
r
o
t
i
m
e
Standardized blood clotting time
S
e
x
Male/female
S
p
i
d
e
r
s
Blood vessel malformations in the skin
S
t
a
g
e
Histologic stage of disease (which needs biopsy)
S
t
a
t
u
s
Status at endpoint, 0/1/2 for censored, transplant, or dead
T
i
m
e
Number of days between the date of registration and the earlier date of death or transplantation
T
r
t
1/2/NA for D-penicillamine, placebo, not randomized
T
r
i
g
Triglycerides (mg/dL)Table6 shows that BIC selects the model with variables A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u. If the focus was the B
i
l
i, the selected model is A
g
e
,
B
i
l
i
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u which is ranked 14 with BIC. If the focus was the A
l
b
u, the selected model is A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
A
s
t
,
C
o
p
p
,
A
l
b
u which is ranked 7 with BIC. The other focuses parameters show that there is a great discrepancy between the FoBMA and BIC. If the focus was on age, BIC and FoBMA select the same model.Table 6
Survival regression example: selected (best) model according to the focus parameter (FoBMA) compared to its BIC rank and value, the probability that the focus is not null (100P(β
≠
0)), its BMA mean (μ
^
bma
=
E
bma
(
μ
∣
x
)), and standard deviation (S
D
bma
(
μ
∣
x
)
).
Focus
Best model (FoBMA)
-BIC
Rank
β
≠
0
Mean
SD
A
g
e
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
174.8
1
100
0.031
0.009
A
l
k
P
A
g
e
,
A
l
k
P
,
B
i
l
i
,
E
d
e
m
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
168.9
34
7.8
0.000
0.000
A
s
c
i
A
g
e
,
A
s
c
i
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
168.9
24
1.7
0.003
0.045
B
i
l
i
A
g
e
,
B
i
l
i
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
170.8
14
100
0.78
0.13
E
d
e
m
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
l
a
t
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
170.8
15
84.7
0.74
0.43
H
e
p
a
A
g
e
,
B
i
l
i
,
E
d
e
m
,
H
e
p
a
,
P
r
o
t
,
C
o
p
p
,
A
l
b
u
170.7
16
2.8
0.006
0.05
P
l
a
t
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
l
a
t
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
168.8
39
5.4
0.000
0.000
P
r
o
t
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
t
a
g
,
A
l
b
u
171.0
13
78
2.5
1.7
S
e
x
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
e
x
,
C
o
p
p
,
A
l
b
u
170.4
18
5.6
−0.018
0.1
A
s
t
A
g
e
,
B
i
l
i
,
P
r
o
t
,
A
s
t
,
C
o
p
p
,
A
l
b
u
169.6
28
20.9
0.097
0.23
S
p
i
d
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
p
i
d
,
C
o
p
p
,
A
l
b
u
170.0
27
1.4
0.000
0.026
S
t
a
g
A
g
e
,
A
l
k
P
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
169.5
29
36.6
0.1
0.16
T
r
t
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
T
r
t
,
C
o
p
p
,
A
l
b
u
170.2
23
1.6
0.002
0.027
C
o
p
p
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
A
s
t
,
S
t
a
g
,
C
o
p
p
,
A
l
b
u
171.3
11
73.8
0.26
0.2
A
l
b
u
A
g
e
,
B
i
l
i
,
E
d
e
m
,
P
r
o
t
,
A
s
t
,
C
o
p
p
,
A
l
b
u
172.0
7
100
−2.8
8


## 5. Discussion

### 5.1. Comparison to Other Bayesian Approaches

As with any other Bayesian approach, parameter priors and model priors are a source of uncertainty in FoBMA. FoBMA shares the same drawbacks of BMA, namely, the computation of integrated likelihoods of various models and the choice of model space. As model selection criterion, compared to Bayesian likes, it is simpler after BMA estimates have been obtained. It was shown in Corollary3 that FoBMA is equivalent to the classical approach of selecting the model with the highest posterior model probability if there are only two competing models.


### 5.2. Interpretation of Focus

It was shown in Paragraph 4 that, in general, FoBMA is found to select a model different from the one selected without a focus. Hereby, the difference in terms of ranking is sometimes large. For example, in studying the risk factors associated with low infants birth weight, the classical Bayesian model selection approach selects the one with variablesL
W
T
,
H
T. However, if the focus was the L
W
T (mother’s weight in pounds at last menstrual period), that is, if the main objective was to find out whether L
W
T was a risk factor, adjusted for possible confounding factors, the selected model is L
W
T
,
S
M
O
K
E
,
U
I which is ranked 38 with BIC. Thus, a poor model for estimating a focus μ
1 may be the best for μ
2.


### 5.3. Heuristic Explanation of Focus

Consider a set of datax
1
,
…
x
n, each x
i with PDF f
(
x
i
∣
θ
). Mean, median, and mode measure central tendencies. Mean is obtained by maximizing the square error loss, the median by maximizing the absolute risk, and the mode by maximizing o-1 loss. In this context of model selection, the square error loss was used; then it is reasonable to find out the “mean,” which can be identified as BMA estimate. Since BMA estimate is not any of the competing models, it seems appropriate to select the closest one to it.


## 5.1. Comparison to Other Bayesian Approaches

As with any other Bayesian approach, parameter priors and model priors are a source of uncertainty in FoBMA. FoBMA shares the same drawbacks of BMA, namely, the computation of integrated likelihoods of various models and the choice of model space. As model selection criterion, compared to Bayesian likes, it is simpler after BMA estimates have been obtained. It was shown in Corollary3 that FoBMA is equivalent to the classical approach of selecting the model with the highest posterior model probability if there are only two competing models.


## 5.2. Interpretation of Focus

It was shown in Paragraph 4 that, in general, FoBMA is found to select a model different from the one selected without a focus. Hereby, the difference in terms of ranking is sometimes large. For example, in studying the risk factors associated with low infants birth weight, the classical Bayesian model selection approach selects the one with variablesL
W
T
,
H
T. However, if the focus was the L
W
T (mother’s weight in pounds at last menstrual period), that is, if the main objective was to find out whether L
W
T was a risk factor, adjusted for possible confounding factors, the selected model is L
W
T
,
S
M
O
K
E
,
U
I which is ranked 38 with BIC. Thus, a poor model for estimating a focus μ
1 may be the best for μ
2.


## 5.3. Heuristic Explanation of Focus

Consider a set of datax
1
,
…
x
n, each x
i with PDF f
(
x
i
∣
θ
). Mean, median, and mode measure central tendencies. Mean is obtained by maximizing the square error loss, the median by maximizing the absolute risk, and the mode by maximizing o-1 loss. In this context of model selection, the square error loss was used; then it is reasonable to find out the “mean,” which can be identified as BMA estimate. Since BMA estimate is not any of the competing models, it seems appropriate to select the closest one to it.


## 6. Concluding Remarks

The present paper has derived a new model selection criterion (FoBMA), which, in contrast to other Bayesian model selection criteria, focuses on the parameter singled out for interest. The methodology was applied to concrete examples. The method needs to be applied in a variety of conditions; in particular, more works need to be done to find the asymptotic properties of FoBMA. It is expected that this paper brings motivations for more researches on focus-related criteria in Bayesian model selection.



---

*Source: 504325-2014-10-14.xml*