# Two-Stage Adaptive Optimal Design with Fixed First-Stage Sample Size

Authors: Adam Lane; Nancy Flournoy
Journal: Journal of Probability and Statistics
 (2012)

---

## Abstract

In adaptive optimal procedures, the design at each stage is an estimate of the optimal design based on all previous data. Asymptotics for regular models with fixed number of stages are straightforward if one assumes the sample size of each stage goes to infinity with the overall sample size. However, it is not uncommon for a small pilot study of fixed size to be followed by a much larger experiment. We study the large sample behavior of such studies. For simplicity, we assume a nonlinear regression model with normal errors. We show that the distribution of the maximum likelihood estimates converges to a scale mixture family of normal random variables. Then, for a one parameter exponential mean function we derive the asymptotic distribution of the maximum likelihood estimate explicitly and present a simulation to compare the characteristics of this asymptotic distribution with some commonly used alternatives.


---

## Body

## 1. Introduction

Elfving [1] introduced a geometric approach for determining a c-optimal design for linear regression models. Kiefer and Wolfowitz [2] developed the celebrated equivalence theorem which provides an efficient method for verifying if a design is D-optimal, again for a linear model. These two results were generalized by Chernoff [3] and White [4] to include nonlinear models, respectively. See Bartroff [5], O’Brien and Funk [6], and references therein for extensions to the geometric and equivalence approaches. Researchers in optimal design have built an impressive body of theoretical and practical tools for linear models based on these early results. However, advances for nonlinear models have not kept pace.One reason for the prevalence of the linear assumption in optimal design is that the problem can be explicitly described. The goal of optimal design is to determine precise experiments. Define an approximate design, proposed by Kiefer and Wolfowitz [7], as ξ={xi,wi}1K, where ξ is a probability measure on 𝒳 consisting of support points xi∈𝒳 and corresponding design weights wi; wi are rational and defined on the interval [0,1] and ∑‍wi=1. Then the optimal design problem is to find the design that maximizes precision for a given experimental interest. Typically, this precision is achieved by maximizing some concave function, ϕ, of Fisher's information matrix. For example, when the estimation of all the parameters is the primary interest then the D-optimality criteria, where ϕ is equal to the determinant of the inverse of Fisher's information, are the most popular method. See Pukelsheim [8] for a detailed discussion of common optimality criteria.The basic principles for nonlinear models are the same as for linear models except Fisher's information will be a function of the model parameters. As a result, optimal designs depend on the parameters and thus are only optimal in the neighborhood of the true parameters. The term locally optimal design is commonly used for nonlinear optimal designs to reflect this dependence on the parameters of interest.To overcome this dependence Fisher [9] and Chernoff [3] suggest using expert knowledge to approximate the locally optimal design. Ford et al. [10] suggest optimal designs in nonlinear problems are to be used to provide a benchmark or to construct sequential or adaptive designs. Atkinson et al. [11] suggest using a polynomial expansion to approximate the nonlinear model with a linear one.Stein [12] provides the earliest two-stage procedure in which the information from the first stage is used to determine design features for the second stage. In this paper we examine a two-stage adaptive optimal design procedure. An adaptive optimal design uses the data from all previous stages to estimate the locally optimal design of the current stage. Many, including Box and Hunter [13], Fedorov [14], White [15], and Silvey [16], have suggested using such designs. Recently, Lane et al. [17], Dragalin et al. [18], Fedorov et al. [19], Yao and Flournoy [20], and so forth have investigated the properties and performance of these procedures.Lane et al. [17] show that the optimal stage-one sample size is of the order n, where n is the overall sample size, in a two-stage regression model. Luc Pranzato obtains this relationship for a more general model (personal communication, 2012). However, in certain experiments, for example, early phase clinical trials or bioassay studies, it is common to use designs with very small stage-one sample sizes. Current literature has characterized the adaptive optimal design procedure under the assumption that both stage-one and stage-two sample sizes are large.In this paper we characterize the asymptotic distribution of the maximum likelihood estimate (MLE) when the stage-one sample size is fixed. The distribution for a nonlinear regression model with normal errors and a one parameter exponential mean function is derived explicitly. Then for a specific numeric example the differences between the finite stage-one sample distribution are compared with other candidate approximate distributions.


## 2. Adaptive Optimal Procedure for a Two-Stage Nonlinear Regression Model with Normal Errors

### 2.1. The Model

Let{yij}1,1ni,2 be observations from a two-stage experiment, where ni is the number of observations and xi is the single-dose level used for the ith stage, i=1,2. Assume that
(2.1)yij=η(xi,θ)+εij,εij~𝒩(0,σ2),
where η(x,θ) is some nonlinear mean function. In most practical examples it is necessary to consider a bounded design space, that is, xi∈𝒳=[a,b], -∞<a<b<∞. It is assumed that yij are independent conditional on treatment xi, where x1 is fixed and x2 is selected adaptively. Denote the adaptive design by ξA={xi,wi}12, where wi=ni/n.The likelihood for model (2.1) is
(2.2)ℒn(θ∣y-1,y-2)∝exp{-n12σ2(y-1-η(x1,θ))2-n22σ2(y-2-η(x2,θ))2},
where y-i=ni-1∑1ni‍yij are the stage specific sample means, and the total score function is
(2.3)S=ddθlnℒn(θ∣y-1,y-2)=n12σ2(y-1-η(x1,θ))dη(x1,θ)dθ+n22σ2(y-2-η(x2,θ))dη(x2,θ)dθ=S1+S2,
where Si represents the score function for the ith stage.


### 2.2. The Adaptive Optimal Procedure

Fix the first stage design pointx1 and let θ~n1 represent an estimate based on the first-stage complete sufficient statistic y-1. The locally optimal design point for the second stage is
(2.4)x*=argmaxx∈𝒳Vary-2∣x(S2)=argmaxx∈𝒳(dη(x,θ)dθ)2,
which is commonly estimated by x*∣θ=θ~n1 for use in stage 2. Because the adaptive optimal design literature assumes n1 is large, the MLE of the second stage design point, x*∣θ=θ^n1, where θ^n1 is the MLE of θ based on the first stage data, is traditionally used to estimate x*.However, whenn1 is small the bias of the MLE can be considerable. Therefore, for some mean functions η using a different estimate would be beneficial. In general, the adaptively selected stage two treatment is
(2.5)x2=argmaxx∈𝒳(dη(x,θ)dθ)2|θ=θ~n1.


### 2.3. Fisher's Information

Sincex1∈𝒳=[a,b], a bounded design space, but y∈ℝ, there is a positive probability that x2 will equal a or b. Denote these probabilities as πa=P(x2=a) and πb=P(x2=b), respectively. Then the per subject information can be written as
(2.6)M(ξA,θ)=1nVar(S)=1σ2[w1(dη(x1,θ)dθ)21σ21σ21σ21σ21σ21σ21σ2+w2(πa(dη(a,θ)dθ)2+πb(dη(b,θ)dθ)21σ2w2w21σ21σ21σ21σ21σ21σ2w2+Ex2[(dη(x2,θ)dθ)2I(a<x2<b)])],
where x2 is the random variable defined by the onto transformation (2.5) of y-1.


## 2.1. The Model

Let{yij}1,1ni,2 be observations from a two-stage experiment, where ni is the number of observations and xi is the single-dose level used for the ith stage, i=1,2. Assume that
(2.1)yij=η(xi,θ)+εij,εij~𝒩(0,σ2),
where η(x,θ) is some nonlinear mean function. In most practical examples it is necessary to consider a bounded design space, that is, xi∈𝒳=[a,b], -∞<a<b<∞. It is assumed that yij are independent conditional on treatment xi, where x1 is fixed and x2 is selected adaptively. Denote the adaptive design by ξA={xi,wi}12, where wi=ni/n.The likelihood for model (2.1) is
(2.2)ℒn(θ∣y-1,y-2)∝exp{-n12σ2(y-1-η(x1,θ))2-n22σ2(y-2-η(x2,θ))2},
where y-i=ni-1∑1ni‍yij are the stage specific sample means, and the total score function is
(2.3)S=ddθlnℒn(θ∣y-1,y-2)=n12σ2(y-1-η(x1,θ))dη(x1,θ)dθ+n22σ2(y-2-η(x2,θ))dη(x2,θ)dθ=S1+S2,
where Si represents the score function for the ith stage.


## 2.2. The Adaptive Optimal Procedure

Fix the first stage design pointx1 and let θ~n1 represent an estimate based on the first-stage complete sufficient statistic y-1. The locally optimal design point for the second stage is
(2.4)x*=argmaxx∈𝒳Vary-2∣x(S2)=argmaxx∈𝒳(dη(x,θ)dθ)2,
which is commonly estimated by x*∣θ=θ~n1 for use in stage 2. Because the adaptive optimal design literature assumes n1 is large, the MLE of the second stage design point, x*∣θ=θ^n1, where θ^n1 is the MLE of θ based on the first stage data, is traditionally used to estimate x*.However, whenn1 is small the bias of the MLE can be considerable. Therefore, for some mean functions η using a different estimate would be beneficial. In general, the adaptively selected stage two treatment is
(2.5)x2=argmaxx∈𝒳(dη(x,θ)dθ)2|θ=θ~n1.


## 2.3. Fisher's Information

Sincex1∈𝒳=[a,b], a bounded design space, but y∈ℝ, there is a positive probability that x2 will equal a or b. Denote these probabilities as πa=P(x2=a) and πb=P(x2=b), respectively. Then the per subject information can be written as
(2.6)M(ξA,θ)=1nVar(S)=1σ2[w1(dη(x1,θ)dθ)21σ21σ21σ21σ21σ21σ21σ2+w2(πa(dη(a,θ)dθ)2+πb(dη(b,θ)dθ)21σ2w2w21σ21σ21σ21σ21σ21σ2w2+Ex2[(dη(x2,θ)dθ)2I(a<x2<b)])],
where x2 is the random variable defined by the onto transformation (2.5) of y-1.


## 3. Asymptotic Properties

We examine three different ways of deriving an asymptotic distribution of the final MLE which may be used for inference at the end of the study. The first is under the assumption that bothn1 and n2 are large. The second considers the data from the second stage alone. Finally, assume a fixed first-stage sample size and a large second-stage sample size.


### 3.1. Large Stage-1 and Stage-2 Sample Sizes

Ifdη(x2,θ)/dθ is bounded and continuous and provided common regularity conditions that hold,
(3.1)n(θ^n-θt)→𝒟𝒩(0,M-1(ξ*,θ)),
as n1→∞ and n2→∞, where ξ*={(x1,n1),(x*,n2)}. This result is used to justify the common practice of using x*|θ=θ^n1 to estimate x* in order to make inferences about θ. However, if dη(x2,θ)/dθ is not bounded and continuous then it is very difficult to obtain the result in (3.1) and for certain mean functions the result will not hold. In such cases the asymptotic variance in (3.1) must be replaced with limn1→∞M-1(ξA,θ). Lane et al. [17] examine using the exact Fisher's information for an adaptive design ξA, M(ξA,θ), instead of M(ξ*,θ) in (3.1) to obtain an alternative approximation of the variance of the MLE θ^n.


### 3.2. Distribution of the MLE If Only Second-Stage Data Are Considered

Often pilot data are discarded after being used to design a second experiment then the derivation of the distribution of the MLE using only the second-stage data takes ifx2 to be fixed:
(3.2)n2(θ^n2-θt)→𝒟𝒩(0,M2-1(x2,θ)),
as n2→∞, where M2(x2,θ)=σ-2(dη(x2,θ)/dθ)2. The estimate θ^n2 will likely perform poorly in comparison to θ^n if n1 and n2 are relatively of the same size but conceivably may perform quite well when n1 is much smaller than n. For this reason it represents an informative benchmark distribution.


### 3.3. Fixed First-Stage Sample Size; Large Second-Stage Sample Size

When the first-stage sample size is fixed and the second stage is large we have the following result.Theorem 3.1.
For model (2.1) with x2 as defined in (2.5) if dη/dθ≠0 for all x∈𝒳, θ∈Θ, x2 is an onto function of y-1, |dη/dθ|<∞ and provided common regularity conditions,
(3.3)n(θ^n-θt)→𝒟UQ
as n2→∞, where Q~𝒩(0,σ2) and U=((dη(x2,θ))/dθ)-1 is a random function of y-1.Proof.
As in classical large sample theory (cf. Ferguson [21] and Lehmann [22]):
(3.4)n(θ^n-θ)≈(1/n)S-(1/n)(d/dθ)S,
since S(θ^n) can be expanded around S(θt) as
(3.5)S(θ^n)=S(θt)+(θ^n-θt)ddθS(θt)+12(θ^n-θt)2d2dθ2S(θ*),
where θt is the true value of the parameter and θ*∈(θt,θ^n). Solving for n(θ^n-θt) gives
(3.6)n(θ^n-θt)=(1/n)S(θt)-(1/n)((d/dθ)S(θt)+(1/2)(θ^n-θt)(d2/dθ2)S(θ*)).
It can be shown that θ^n is consistent for θt if n2→∞ and n1/n→0 which gives the result in (3.4).
Now, decompose the right hand side of (3.4) as
(3.7)(1/n)S-(1/n)(d/dθ)S=(1/n)(S1+S2)-(1/n)((d/dθ)S1+(d/dθ)S2)=(1/n)S1-(1/n)((d/dθ)S1+(d/dθ)S2)+(1/n)S2-(1/n)((d/dθ)S1+(d/dθ)S2).
As n2→∞, S1/n→0, (n2/n)→1, and (1/n)(d/dθ)S2↛0 as n→∞. Thus, the first term in (3.7) goes to 0 as n→∞. Write the second term in (3.7) as
(3.8)(1/n)S2-(1/n)((d/dθ)S1+(d/dθ)S2)=(-(1/n)(d/dθ)S1(1/n)S2-(1/n)(d/dθ)S2(1/n)S2)-1.
Further as n2→∞, (1/n)(d/dθ)S1→0 and (1/n)S2↛0,
(3.9)(1/n)(d/dθ)S1(1/n)S2→p0,(1/n)(d/dθ)S2(1/n)S2=(1/n)(y-2-η(x2,θ))(d2η(x2,θ)/dθ2)+w2((dη(x2,θ))/dθ)2(1/n)n2(y-2-η(x2,θ))((dη(x2,θ))/dθ)=(1n)d2η(x2,θ)/dθ2dη(x2,θ)/dθ+w2(dη(x2,θ))/dθn2(y-2-η(x2,θ)).
The first term in (3.9) goes to 0. To evaluate the second term, it is important to recognize that εi2=y-2-η(x2,θ)~𝒩(0,σ2/n2) and y-1~𝒩(0,σ2/n1) are independent and thus
(3.10)y-2-η(x2,θ),dη(x2,θ)dθ
are independent. Because of this independence,
(3.11)n2(y-2-η(x2,θ))(dη(x2,θ)dθ)-1~UQ,
where U is a random function of y-1 and Q~𝒩(0,σ2) as determined by ((dη(x2,θ))/dθ)-1. Now, with w2→1 as n2→∞ the result follows from an application of Slutsky's theorem.Remark 3.2.
Provideddη(x,θ)/dθ is bounded and continuous UQ is the asymptotic distribution of n(θ^n-θt) as n→∞. The important case for this exposition is presented in Theorem 3.1. However, the two other potential cases can be shown easily.Case 1.
n
1
→
∞
,
n
2
→
∞, and n→∞. As n1→∞, x2→x* which implies that U→((d(x*,θ))/dθ)-1, a constant, and thus UQ converges to asymptotic distribution of n(θ^n-θt) given in (3.1).Case 2.
n
1
→
∞
,
n
2 fixed, and n→∞. Just as in Case 1, U→M-1(x*,θ), where M(x*,θ)=((dη(x*,θ))/dθ)-2. Note that M(x*,θ) differs from M(ξ*,θ) which depends on x1 and x*. Therefore UQ→𝒩(0,σ2M-1(x*,θ)). Look back at (3.7) in the proof, but now take n2 to be fixed; (1/n)S2→0 and (1/n)(d/dθ)S2→0 and the only term left is
(3.12)(1/n)S1-(1/n)(d/dθ)S1.
Consider the following: (1/n)S1→𝒩(0,σ2M-1(x*,θ)) and (1/n)(d/dθ)S1→M-1(x*,θ) as n→∞. Therefore, n(θ^n-θt)→𝒩(0,σ2M-1(x*,θ)) as n→∞ which is equivalent to UQ.


## 3.1. Large Stage-1 and Stage-2 Sample Sizes

Ifdη(x2,θ)/dθ is bounded and continuous and provided common regularity conditions that hold,
(3.1)n(θ^n-θt)→𝒟𝒩(0,M-1(ξ*,θ)),
as n1→∞ and n2→∞, where ξ*={(x1,n1),(x*,n2)}. This result is used to justify the common practice of using x*|θ=θ^n1 to estimate x* in order to make inferences about θ. However, if dη(x2,θ)/dθ is not bounded and continuous then it is very difficult to obtain the result in (3.1) and for certain mean functions the result will not hold. In such cases the asymptotic variance in (3.1) must be replaced with limn1→∞M-1(ξA,θ). Lane et al. [17] examine using the exact Fisher's information for an adaptive design ξA, M(ξA,θ), instead of M(ξ*,θ) in (3.1) to obtain an alternative approximation of the variance of the MLE θ^n.


## 3.2. Distribution of the MLE If Only Second-Stage Data Are Considered

Often pilot data are discarded after being used to design a second experiment then the derivation of the distribution of the MLE using only the second-stage data takes ifx2 to be fixed:
(3.2)n2(θ^n2-θt)→𝒟𝒩(0,M2-1(x2,θ)),
as n2→∞, where M2(x2,θ)=σ-2(dη(x2,θ)/dθ)2. The estimate θ^n2 will likely perform poorly in comparison to θ^n if n1 and n2 are relatively of the same size but conceivably may perform quite well when n1 is much smaller than n. For this reason it represents an informative benchmark distribution.


## 3.3. Fixed First-Stage Sample Size; Large Second-Stage Sample Size

When the first-stage sample size is fixed and the second stage is large we have the following result.Theorem 3.1.
For model (2.1) with x2 as defined in (2.5) if dη/dθ≠0 for all x∈𝒳, θ∈Θ, x2 is an onto function of y-1, |dη/dθ|<∞ and provided common regularity conditions,
(3.3)n(θ^n-θt)→𝒟UQ
as n2→∞, where Q~𝒩(0,σ2) and U=((dη(x2,θ))/dθ)-1 is a random function of y-1.Proof.
As in classical large sample theory (cf. Ferguson [21] and Lehmann [22]):
(3.4)n(θ^n-θ)≈(1/n)S-(1/n)(d/dθ)S,
since S(θ^n) can be expanded around S(θt) as
(3.5)S(θ^n)=S(θt)+(θ^n-θt)ddθS(θt)+12(θ^n-θt)2d2dθ2S(θ*),
where θt is the true value of the parameter and θ*∈(θt,θ^n). Solving for n(θ^n-θt) gives
(3.6)n(θ^n-θt)=(1/n)S(θt)-(1/n)((d/dθ)S(θt)+(1/2)(θ^n-θt)(d2/dθ2)S(θ*)).
It can be shown that θ^n is consistent for θt if n2→∞ and n1/n→0 which gives the result in (3.4).
Now, decompose the right hand side of (3.4) as
(3.7)(1/n)S-(1/n)(d/dθ)S=(1/n)(S1+S2)-(1/n)((d/dθ)S1+(d/dθ)S2)=(1/n)S1-(1/n)((d/dθ)S1+(d/dθ)S2)+(1/n)S2-(1/n)((d/dθ)S1+(d/dθ)S2).
As n2→∞, S1/n→0, (n2/n)→1, and (1/n)(d/dθ)S2↛0 as n→∞. Thus, the first term in (3.7) goes to 0 as n→∞. Write the second term in (3.7) as
(3.8)(1/n)S2-(1/n)((d/dθ)S1+(d/dθ)S2)=(-(1/n)(d/dθ)S1(1/n)S2-(1/n)(d/dθ)S2(1/n)S2)-1.
Further as n2→∞, (1/n)(d/dθ)S1→0 and (1/n)S2↛0,
(3.9)(1/n)(d/dθ)S1(1/n)S2→p0,(1/n)(d/dθ)S2(1/n)S2=(1/n)(y-2-η(x2,θ))(d2η(x2,θ)/dθ2)+w2((dη(x2,θ))/dθ)2(1/n)n2(y-2-η(x2,θ))((dη(x2,θ))/dθ)=(1n)d2η(x2,θ)/dθ2dη(x2,θ)/dθ+w2(dη(x2,θ))/dθn2(y-2-η(x2,θ)).
The first term in (3.9) goes to 0. To evaluate the second term, it is important to recognize that εi2=y-2-η(x2,θ)~𝒩(0,σ2/n2) and y-1~𝒩(0,σ2/n1) are independent and thus
(3.10)y-2-η(x2,θ),dη(x2,θ)dθ
are independent. Because of this independence,
(3.11)n2(y-2-η(x2,θ))(dη(x2,θ)dθ)-1~UQ,
where U is a random function of y-1 and Q~𝒩(0,σ2) as determined by ((dη(x2,θ))/dθ)-1. Now, with w2→1 as n2→∞ the result follows from an application of Slutsky's theorem.Remark 3.2.
Provideddη(x,θ)/dθ is bounded and continuous UQ is the asymptotic distribution of n(θ^n-θt) as n→∞. The important case for this exposition is presented in Theorem 3.1. However, the two other potential cases can be shown easily.Case 1.
n
1
→
∞
,
n
2
→
∞, and n→∞. As n1→∞, x2→x* which implies that U→((d(x*,θ))/dθ)-1, a constant, and thus UQ converges to asymptotic distribution of n(θ^n-θt) given in (3.1).Case 2.
n
1
→
∞
,
n
2 fixed, and n→∞. Just as in Case 1, U→M-1(x*,θ), where M(x*,θ)=((dη(x*,θ))/dθ)-2. Note that M(x*,θ) differs from M(ξ*,θ) which depends on x1 and x*. Therefore UQ→𝒩(0,σ2M-1(x*,θ)). Look back at (3.7) in the proof, but now take n2 to be fixed; (1/n)S2→0 and (1/n)(d/dθ)S2→0 and the only term left is
(3.12)(1/n)S1-(1/n)(d/dθ)S1.
Consider the following: (1/n)S1→𝒩(0,σ2M-1(x*,θ)) and (1/n)(d/dθ)S1→M-1(x*,θ) as n→∞. Therefore, n(θ^n-θt)→𝒩(0,σ2M-1(x*,θ)) as n→∞ which is equivalent to UQ.


## 4. Example: One Parameter Exponential Mean Function

In model (2.1) let η(x,θ)=e-θx, where x∈𝒳=[a,b], 0<a<b<∞ and θ∈(0,∞). The simplicity of the exponential mean model facilitates our illustration, but it is also important in its own right. For example, Fisher [9] used a variant of this model to examine the information in serial dilutions. Cochran [23] further elaborated on Fisher's application using the same model.For this illustration we use the MLE of the first-stage data to estimate the second-stage design point. Here,(4.1)θ^n1={-lny-1x,ify-1∈(e-θ¯x,1),0,ify-1≥1,θ¯,ify-1≤e-θ¯x.
The adaptively selected second-stage treatment as given by (2.5) is
(4.2)x2=argmaxx∈𝒳(x2e-2θx)={θ^n1-1,ify-1∈(e-a-1x1,e-b-1x1),b,ify-1≥e-b-1x1,a,ify-1≤e-a-1x1.
Thus, the exact per subject Fisher information is
(4.3)M(ξA,θ)=1nVar(S)=w1x12e-2θx1+w2πaa2e-2θa+w2πbb2e-2θb+w2Ex2[x22e-2θx2·I(a<x2<b)].
For this example M(ξA,θ)→M(ξ*,θ) as n1→∞. For more detailed information on the derivations of (4.1), (4.2), and (4.3) see Lane et al. [17].The asymptotic distributions of the MLE in Sections3.1 and 3.2 can be derived easily. For the asymptotic distribution of the MLE in Section 3.3 consider the following corollary. For details on the functions h, v1, and v2 see the proof of the corollary.Corollary 4.1.
Ifη(x,θ)=e-θx in model (2.1) then
(4.4)n(θ^n-θ)→𝒟UQ
as n→∞, where UQ is defined by
(4.5)P(UQ≤t)={P(U≥tq∣-∞<q≤0)Φ(q),ift∈(-∞,0),P(U≤t1∣0<q≤∞)(1-Φ(q)),ift∈(0,∞),
where Φ(·) is the standard normal cumulative distribution function. Let Ψ(q)=Φ(n(q-η(x,θ))/σ) and h(s)=s-1eθs. Then if h(a)<h(b),
(4.6)P(U≥tq∣-∞<q≤0)Φ(q)=Φ(tσh(1/θ))+[1-(Ψ(v2(h(a)))-Ψ(v1(h(a))))]×[Φ(tσh(a))-Φ(tσh(1/θ))]+[Ψ(v2(h(b)))-Ψ(v2(h(a)))]×[Φ(tσh(b))-Φ(tσh(a))],P(U≤tq∣0<q≤∞)(1-Φ(q))=Φ(tσh(b))+[Ψ(v2(h(a)))-Ψ(v1(h(a)))]×[Φ(tσh(1/θ))-Φ(tσh(a))]+[1-(Ψ(v2(h(b)))-Ψ(v2(h(a))))]×[Φ(tσh(a))-Φ(tσh(b))].
If h(b)<h(a), then
(4.7)P(U≥tq∣-∞<q≤0)Φ(q)=Φ(tσh(1/θ))+[1-(Ψ(v2(h(b)))-Ψ(v1(h(b))))]×[Φ(tσh(b))-Φ(tσh(1/θ))]+[Ψ(v1(h(b)))-Ψ(v1(h(a)))]×[Φ(tσh(b))-Φ(tσh(a))],P(U≤tq∣0<q≤∞)(1-Φ(q))=Φ(tσh(a))+[Ψ(v2(h(b)))-Ψ(v1(h(b)))]×[Φ(tσh(1/θ))-Φ(tσh(b))]+[1-(Ψ(v1(h(b)))-Ψ(v1(h(a))))]×[Φ(tσh(b))-Φ(tσh(a))].Proof.
First, we find the distribution ofU where U=h(z) and the random variable z is defined by
(4.8)z={-x1lny-1,ify-1∈(e-x1/a,e-x1/b),-x1lna,ify-1≤e-x1/a,-x1lnb,ify-1≥e-x1/b.
Figure 1 illustrates the map from U to z∈[a,b] where θ=1, σ=.5, a=.25, and b=4.
Lambert's product log function (cf. Corless et al. [24]) is defined as the solutions to
(4.9)wew=c
for some constant c. Denote the solutions to (4.9) by W(w). Let
(4.10)V(c)=argy-1{(-x1lny-1)-1exp{θ-x1lny-1}=c}.
Then
(4.11)V(c)=exp{θx1W(-θ/c)}.
The W function is real valued on w≥-1/e, single valued at w=-1/e, and double valued on w∈(-1/e,0). U∈{θe,max{h(a),h(b)}}, x1∈[a,b], 0<a<b<∞. Therefore V(c) is real valued for all θ∈(0,∞). For simplicity, define v1=minV(c) and v2=maxV(c) for a given c.
We present the proof for the cumulative distribution function (CDF) ofU and the CDF of UQ for the case where x*∈[a,b] and h(a)<h(b). The derivation of the distributions under alternative cases is tedious and does not differ greatly from this case.
Note in this case the domain ofU is [h(1/θ)=θe,h(b)]. If h(1/θ)<U<h(a), then
(4.12)P(U≤t1)=P(h(y-1)<t)=P(v1(t1)<y-1<v2(t1))=Ψ(v2(t1))-Ψ(v1(t1)).
If U=h(a), then
(4.13)P(U≤h(a))=Ψ(v2(h(a))).
If U∈(h(a),h(b)), then
(4.14)P(U≤t1)=P(v1(t1)<y-1<v2(t1)).
However, since t1<h(a)P(y-1<v1(t1))=0,
(4.15)P(U≤t1)=Ψ(v2(t1)).
If U≥h(b), then
(4.16)P(U≤h(b))=1.
Thus,
(4.17)P(U≤t1)={0,ift1≤h(1θ),Ψ(v2(t1))-Ψ(v2(t1)),ift1∈(h(1θ),h(a)),Ψ(e-x1/a),ift1=h(a),Ψ(v2(t1)),ift1∈(h(a),h(b)),1,ifh(b)≤t1≤∞.
Figure 2 plots the CDF of U for θ=1, x1=2, n1=5, σ=.5, a=.25, and b=4. The distribution is a piecewise function with discontinuities at the boundary points a and b.
Now consider the distribution ofUQ. Recall q~𝒩(0,σ2) and U and Q are independent. If t∈(-∞,0), then
(4.18)P(UQ≤t)=P(U≥tq∣0<tq≤h(1θ))P(0<tq≤h(1θ))+P(U≥tq∣h(1θ)<tq≤h(a))P(h(1θ)<tq≤h(a))+P(U=h(a)∣tq=h(a))P(tq=h(a))+P(U≥tq∣h(a)<tq≤h(b))P(h(a)<tq≤h(b))+P(U=h(b)∣tq=h(b))P(tq=h(b))+P(U≥tq∣h(b)<tq≤∞)P(h(b)<tq<∞).
The distribution is symmetric, thus the derivation of the CDF if t∈(0,∞) is analogous.Figure 1
Map ofz=-x1/lny-1 for θ=1, a=.25, and b=4.Figure 2
CDF ofU for θ=1, x1=2, n1=5, σ=.5, a=.25, and b=4.


### 4.1. Comparisons of Asymptotic Distributions

First, consider the distribution described in (3.1) using M(ξA,θ) in place of M(ξ*,θ) and the distribution described in (3.2). When n1 is significantly smaller than n2, M(ξA,θ) and M(x2,θ) can differ significantly as a function of y-1. This is primarily because M(x2,θ) is a function of x2, whereas M(ξA,θ) is an average over y-1. Through simulation it can be seen that a 𝒩(0,M-1(x2,θ)) is a better approximate distribution of n(θ^n-θ) than 𝒩(0,M-1(ξA,θ)) for only a small interval of x2, and this interval has a very small probability. For these reasons the distribution of the MLE using only the second stage data as described in Section 3.2 is not considered further.Now for a set of numeric examples consider three distributions: (3.1), (3.1) using M(ξA,θ) in place of M(ξ*,θ) and the distribution of UQ defined in (3.3). An asymptotic distribution can be justified in inference if it is approximately equal to the true distribution. In this case the true distribution is that of n(θ^n-θ). However, θ^n does not have a closed form and thus its distribution cannot be obtained analytically or numerically. To approximate this distribution 10,000 Monte Carlo simulations have been completed for each example to create a benchmark distribution.Figure3 plots the three different candidate approximate distributions, found exactly using numerical methods, together with the distribution of n(θ^n-θ) approximated using Monte Carlo simulations, for θ=1, x1, σ=.5, a=.25, b=4, n1=5, and n={30,1000}. Note the y-axis represents P(Ti≤t), i=1,2,3, where T1 is 𝒩(0,M-1(ξ*,θ)), T2 is 𝒩(0,M-1(ξA,θ)), and T3 is UQ. When n=30 it is difficult, graphically, to determine if T2 or T3 provides a better approximation for n(θ^n-θ). It seems that if t∈(-4,0) the distribution T3 is preferable to T2; however, when t∈(0,4) the opposite appears to be the case. It is fairly clear that for this example T1 performs poorly.In each plot the solid line represents the CDF ofn(θ^n-θ) obtained via Monte Carlo simulations. The dotted-dashed line is the P(T1≤t), where T1~𝒩(0,M-1(ξ*,θ)). The dotted line is the P(T2≤t), where T2~𝒩(0,M-1(ξA,θ)). The dashed line is the P(T3≤t), where T3~Q. Values θ=1, x1=2, n1=5, σ=.5, a=.25, and b=4 were used.
(a)
n
=
30
(b)
n
=
1000Whenn=1000, it is clear that T3 is much closer to n(θ^n-θ) than both T1 and T2. Further, comparing the two plots one can see how the distribution of n(θ^n-θ) has nearly converged to UQ but still differs from those T1 and T2 significantly, as predicted by Theorem 3.1 and Corollary 4.1.Using only graphics it is difficult to assess which ofT1, T2, and T3 is nearest n(θ^n-θ) for a variety of cases. To get a better understanding, the integrated absolute difference of the CDFs of T1, T2, and T3 versus that of n(θ^n-θ) for x1=2, σ=.5, a=.25, b=4, n={5,10,15}, and n={30,50,100,400} is presented in Table 1. First, consider the table where θ=.5. The locally optimal stage-1 design point is x1=2 when θ=.5; as a result this scenario is the most generous to distribution T1. However, even for this ideal scenario T3 outperforms T1 and T2 for all values of n1. In many cases the difference between T3 and T1 is quite severe. In this scenario T3 outperforms T2; however, the differences are not great.Table 1
Integrated absolute difference of the cumulative distributions (×100) of T1~𝒩(0,M-1(ξ*,θ)), T2~𝒩(0,M-1(ξA,θ)), and T3~UQ versus the approximate cumulative distribution of n(θ^n-θ) obtained via Monte Carlo simulations for various n1 and various moderate sizes of n. The values θ=1, x1=2, σ=.5, a=.25, and b=4 were used.
(a)
(
θ
=
.
5
)
n
1
n
=
30
n
=
50
n
=
100
n
=
400
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
5
19
24
11
16
17
8
15
16
7
14
15
5
10
11
13
8
9
12
7
9
11
6
7
12
4
15
9
10
8
8
9
6
6
9
5
4
10
3
(b)
(
θ
=
1
)
n
1
n
=
30
n
=
50
n
=
100
n
=
400
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
5
30
33
25
30
27
19
34
24
12
39
21
6
10
40
40
32
26
27
22
23
28
16
26
20
8
15
34
34
33
27
28
24
21
23
17
18
20
9
(c)
(
θ
=
1.5
)
n
1
n
=
30
n
=
50
n
=
100
n
=
400
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
5
32
33
31
39
25
25
42
21
23
42
17
21
10
34
33
22
27
25
16
32
22
10
35
19
12
15
35
35
32
26
26
21
26
22
13
28
21
7Next, examine the results forθ=1 and θ=1.5. Once again T3 outperforms T1 and T2 in all but 2 cases, where in many cases its advantage is quite significant. Also note that T2 outperforms T1 about half the time when θ=1 and the majority of the time when θ=1.5. This supports our observation that when the distance between x1 and x* increases the performance of T1 compared with T2 and T3 worsens which indicates a lack of robustness for the commonly used distribution T1. This lack of robustness is not evident for T1 and T2.One final comparison is motivated by the fact that ifn1→∞, T1, T2, and T3 have the same asymptotic distribution. Although our method is motivated by the scenario where n1 is a small pilot study, there is no theoretical reason that T3 will not perform competitively when n1 is large. Table 2 presents the integrated differences for the distributions T2 and T3 from n(θ^n-θ) for x1=2, θ=1, σ=.5, a=.25, b=4, n1={50,100,200}, and n={400,1000}. T1 is not included in the table due to the lack of robustness; it can perform better or worse than the other two distributions based on the value of θ. Even with larger values of n1, T3 performs slightly better when n1=50 and 100 and only slightly worse when n=200 indicating that using T3 is robust for moderately large n1.Table 2
Integrated absolute difference of the cumulative distributions (×100) of T1~𝒩(0,M-1(ξ*,θ)), T2~𝒩(0,M-1(ξA,θ)), and T3~UQ versus the approximate cumulative distribution of n(θ^n-θ) obtained via Monte Carlo simulations for various n1 and various large sizes of n. The values θ=1, x1=2, σ=.5, a=.25, and b=4 were used.
n
1
n
=
400
n
=
1000
T
2
T
3
T
2
T
3
50
13
9
13
5
100
10
9
8
4
200
11
14
4
7


## 4.1. Comparisons of Asymptotic Distributions

First, consider the distribution described in (3.1) using M(ξA,θ) in place of M(ξ*,θ) and the distribution described in (3.2). When n1 is significantly smaller than n2, M(ξA,θ) and M(x2,θ) can differ significantly as a function of y-1. This is primarily because M(x2,θ) is a function of x2, whereas M(ξA,θ) is an average over y-1. Through simulation it can be seen that a 𝒩(0,M-1(x2,θ)) is a better approximate distribution of n(θ^n-θ) than 𝒩(0,M-1(ξA,θ)) for only a small interval of x2, and this interval has a very small probability. For these reasons the distribution of the MLE using only the second stage data as described in Section 3.2 is not considered further.Now for a set of numeric examples consider three distributions: (3.1), (3.1) using M(ξA,θ) in place of M(ξ*,θ) and the distribution of UQ defined in (3.3). An asymptotic distribution can be justified in inference if it is approximately equal to the true distribution. In this case the true distribution is that of n(θ^n-θ). However, θ^n does not have a closed form and thus its distribution cannot be obtained analytically or numerically. To approximate this distribution 10,000 Monte Carlo simulations have been completed for each example to create a benchmark distribution.Figure3 plots the three different candidate approximate distributions, found exactly using numerical methods, together with the distribution of n(θ^n-θ) approximated using Monte Carlo simulations, for θ=1, x1, σ=.5, a=.25, b=4, n1=5, and n={30,1000}. Note the y-axis represents P(Ti≤t), i=1,2,3, where T1 is 𝒩(0,M-1(ξ*,θ)), T2 is 𝒩(0,M-1(ξA,θ)), and T3 is UQ. When n=30 it is difficult, graphically, to determine if T2 or T3 provides a better approximation for n(θ^n-θ). It seems that if t∈(-4,0) the distribution T3 is preferable to T2; however, when t∈(0,4) the opposite appears to be the case. It is fairly clear that for this example T1 performs poorly.In each plot the solid line represents the CDF ofn(θ^n-θ) obtained via Monte Carlo simulations. The dotted-dashed line is the P(T1≤t), where T1~𝒩(0,M-1(ξ*,θ)). The dotted line is the P(T2≤t), where T2~𝒩(0,M-1(ξA,θ)). The dashed line is the P(T3≤t), where T3~Q. Values θ=1, x1=2, n1=5, σ=.5, a=.25, and b=4 were used.
(a)
n
=
30
(b)
n
=
1000Whenn=1000, it is clear that T3 is much closer to n(θ^n-θ) than both T1 and T2. Further, comparing the two plots one can see how the distribution of n(θ^n-θ) has nearly converged to UQ but still differs from those T1 and T2 significantly, as predicted by Theorem 3.1 and Corollary 4.1.Using only graphics it is difficult to assess which ofT1, T2, and T3 is nearest n(θ^n-θ) for a variety of cases. To get a better understanding, the integrated absolute difference of the CDFs of T1, T2, and T3 versus that of n(θ^n-θ) for x1=2, σ=.5, a=.25, b=4, n={5,10,15}, and n={30,50,100,400} is presented in Table 1. First, consider the table where θ=.5. The locally optimal stage-1 design point is x1=2 when θ=.5; as a result this scenario is the most generous to distribution T1. However, even for this ideal scenario T3 outperforms T1 and T2 for all values of n1. In many cases the difference between T3 and T1 is quite severe. In this scenario T3 outperforms T2; however, the differences are not great.Table 1
Integrated absolute difference of the cumulative distributions (×100) of T1~𝒩(0,M-1(ξ*,θ)), T2~𝒩(0,M-1(ξA,θ)), and T3~UQ versus the approximate cumulative distribution of n(θ^n-θ) obtained via Monte Carlo simulations for various n1 and various moderate sizes of n. The values θ=1, x1=2, σ=.5, a=.25, and b=4 were used.
(a)
(
θ
=
.
5
)
n
1
n
=
30
n
=
50
n
=
100
n
=
400
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
5
19
24
11
16
17
8
15
16
7
14
15
5
10
11
13
8
9
12
7
9
11
6
7
12
4
15
9
10
8
8
9
6
6
9
5
4
10
3
(b)
(
θ
=
1
)
n
1
n
=
30
n
=
50
n
=
100
n
=
400
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
5
30
33
25
30
27
19
34
24
12
39
21
6
10
40
40
32
26
27
22
23
28
16
26
20
8
15
34
34
33
27
28
24
21
23
17
18
20
9
(c)
(
θ
=
1.5
)
n
1
n
=
30
n
=
50
n
=
100
n
=
400
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
T
1
T
2
T
3
5
32
33
31
39
25
25
42
21
23
42
17
21
10
34
33
22
27
25
16
32
22
10
35
19
12
15
35
35
32
26
26
21
26
22
13
28
21
7Next, examine the results forθ=1 and θ=1.5. Once again T3 outperforms T1 and T2 in all but 2 cases, where in many cases its advantage is quite significant. Also note that T2 outperforms T1 about half the time when θ=1 and the majority of the time when θ=1.5. This supports our observation that when the distance between x1 and x* increases the performance of T1 compared with T2 and T3 worsens which indicates a lack of robustness for the commonly used distribution T1. This lack of robustness is not evident for T1 and T2.One final comparison is motivated by the fact that ifn1→∞, T1, T2, and T3 have the same asymptotic distribution. Although our method is motivated by the scenario where n1 is a small pilot study, there is no theoretical reason that T3 will not perform competitively when n1 is large. Table 2 presents the integrated differences for the distributions T2 and T3 from n(θ^n-θ) for x1=2, θ=1, σ=.5, a=.25, b=4, n1={50,100,200}, and n={400,1000}. T1 is not included in the table due to the lack of robustness; it can perform better or worse than the other two distributions based on the value of θ. Even with larger values of n1, T3 performs slightly better when n1=50 and 100 and only slightly worse when n=200 indicating that using T3 is robust for moderately large n1.Table 2
Integrated absolute difference of the cumulative distributions (×100) of T1~𝒩(0,M-1(ξ*,θ)), T2~𝒩(0,M-1(ξA,θ)), and T3~UQ versus the approximate cumulative distribution of n(θ^n-θ) obtained via Monte Carlo simulations for various n1 and various large sizes of n. The values θ=1, x1=2, σ=.5, a=.25, and b=4 were used.
n
1
n
=
400
n
=
1000
T
2
T
3
T
2
T
3
50
13
9
13
5
100
10
9
8
4
200
11
14
4
7


## 5. Discussion

Assuming a finite first-stage sample size and a large second-stage sample size, we have shown for a general nonlinear one parameter regression model with normal errors that the asymptotic distribution of the MLE is a scale mixture distribution. We considered only one parameter for simplicity and clarity of exposition.For the one parameter exponential mean function, the distribution of the adaptively selected second-stage treatment and the asymptotic distribution of the MLE were derived assuming a finite first-stage sample size and a large second-stage sample size. Then the performance of the normalized asymptotic distribution of the MLE,UQ, was analyzed and compared to popular alternatives for a set of simulations.The distribution ofUQ was shown to represent a considerable improvement over the other proposed distributions when n1 was considerably smaller than n. This was true even when n1 is moderately large in size.Since the optimal choice ofn1 was shown to be of the order n for this model in Lane et al. [17], the usefulness of these findings could have significant implications for many combinations of n1 and n.Suppose it is desired thatP(D1≤n(θ^n-θ)≤D2=1-α), where α is the desired confidence level and θt is the true parameter. If one was to use the large sample approximate distribution given in (3.1), D1 and D2, and therefore n, cannot be determined until after stage 1. However, using (3.1) with M(ξA,θ) in place of M(ξ*,θ) or by using UQ on can compute the overall sample size necessary to solve for D1 and D2 before stage one is initiated. One could determine n initially using (3.1) with M(ξA,θ) or UQ and then update this calculation after stage-1 data is available. Such same size recalculation requires additional theoretical justification and investigation of their practical usefulness.We have not, in this paper, addressed the efficiency of the estimateθ^n. One additional way to improve inference would be to find biased adjusted estimates θ~n that are superior to θ^n for finite samples. We have not investigated the impact on inference of estimating the variances in the distributions of UQ, 𝒩(0,M-1(ξ*,θ)), 𝒩(0,M-1(ξA,θ), and 𝒩(0,M-1(x2,θ)). Instead, the distributions themselves are compared. For some details on the question of estimation and consistency see Lane et al. [17] and Yao and Flournoy [20].



---

*Source: 436239-2012-10-04.xml*