# Presentation: 661362

## Active Control Non-Superiority Trial - What It Is About*

-                            George Y.H. Chi
-  Director
-   Division of Biometrics I
-     Center for Drug Evaluation and Research
- chig@cder.fda.gov
**        *****  Presented at the 2002 ICSA Applied Statistics Symposium held at Double Tree Guest**
**            ****Suites, Plymouth Meeting, PA, June 6-8, 2002..**

## Acknowledgment

- This talk is based on the joint work with
- Dr. Mark Rothmann
- Dr. Gang Chen
- Dr. Ning Li
- Dr. Robert Temple
- ! Mr. Xeloda !

## Table of Contents

- 1.  Trials with Active Controls
- 2.  Types of Active Control Trials
- 3.  Non-Superiority Hypothesis (Fixed Margin vs Fraction Retention) - Why Fraction Retention?
- 4.  Test of the Non-Superiority Hypothesis
- 5.  Relationship between Different Procedures
- 6.  Summary

## Trials with Active Controls

-             _Some Reasons for Using Active Controls_
- Active Control Trials - For mortality or serious morbidity trials, it is unethical to give patients placebo when there are effective drugs available, or for making comparative statements.
-  Placebo Control Trials - Active controls are used as check for assay sensitivity, as in psychotropic trials, or for purpose of making comparative statements.

## Types of Active Control Trials

- Superiority Trial : New advances in treatment therapy, or effect of active control is small.
- Non-Superiority Trial :
-       Equivalence or Non-inferiority Trial: Drug in the same
-       class , the active control has moderate to significant effect,
-       or specific safety problems.
-       Efficacy Trial :  Drugs with other important benefits
-       such as, better safety profile, ease of administration, etc.,
-       that are not available in the active control.

## Active Control Superiority Trials

-  Let  T = New Drug, or Treatment
-         C = Active Control
-         P = Placebo (not really present)
-  Consider a mortality trial with T and C.
-  One may wish to show that T > C

## Active Control Superiority Trials

-  That is, in terms of the hazard ratio, HR(T/C),
-  we want to test the null hypothesis
-     Ho:   HR(T/C)  1   vs.   Ha:   HR(T/C) < 1
-   (One can define the hypotheses in terms of
-     log hazard.  What follows also applies.)

## Active Control Superiority Trial

- The above null hypothesis can be tested by the following statistic:
-                                       HR(T/C) -  1
-              **t   **    =                                                     ------ (1)
-                                  s.e.2 [HR(T/C) -1]
-     We reject the null hypothesis if  **t** < -1.96 .

## Equivalence of the Test Statistic t and the 95% Confidence Interval Testing Procedure

- The above test of the null hypothesis is equivalent to the following single 95% confidence interval testing procedure:
-     “**Reject the null hypothesis, if the 95% upper confidence**
**      ****limit of the estimate of HR(T/C) is less than 1.”**
- That is, if
**               ****HR(T/C) + 1.96 s.e. [HR(T/C)]  <  1        **** ****        ----- (2).**
**    **

## Active Control Superiority Trials

-  That is, in terms of the 95% confidence interval,
-     Ha:  HR(T/C) < 1                  Ho:    HR(T/C)  1
-                                          1                                     Hazard Ratio
-                         (                               )
-                                   HR(T/C) **+ **1.96 s.e. [HR(T/C)]

## Active Control Superiority Trials

- If  Ho is rejected, we then usually conclude
-    that T > C.
- What if C is not effective in the current trial?
-         -  Change in standard of care
-         -  Use of concomitant therapies
-         -  Change in medical practice
-         -  Different patient population

## Active Control Superiority Trials

-  **Remark 1:**
- Even in an active control superiority trial, strictly speaking, when we reject Ho, we can  conclude that T is superior to C only if we assume that ***C is effective in the current trial******.***
- If C is not effective in the current trial, then we can at most conclude that T is effective (assuming that C is not worse than placebo in
-     the current trial) - It affects interpretation.

## Active Control Non-Superiority Trials

- Let us consider the case where C is relatively
- effective.
- Since it is relatively difficult to show T > C,
- we would like to consider a non-superiority
- trial.

## Active Control Non-Superiority Trials

- Case (1): Fixed Margin 
- That is, in terms of the hazard ratio, HR(T/C), we
-  want to test the null hypothesis
-  Ho:    HR(T/C)  1+    vs.   Ha:  HR(T/C) < 1+ 
-                                                                      -----(3)                                              

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- The above null hypothesis can be tested at the boundary,  (1+ o), of the null hypothesis by the test statistic
-                                   [HR(T/C) - (1+o)]
-              **t(********o****)**    =                                                    ----- (4).
-                                  s.e.2 [HR(T/C)]
-     We reject the null hypothesis if  **t(********o****)** < -1.96 .

## Equivalence of the Test Statistic t(o) and the 95% Confidence Interval Testing Procedure

- The above test of the null hypothesis is equivalent to the following single 95% confidence interval testing procedure:
-     “**Reject the null hypothesis, if the 95% upper confidence**
**      ****limit of the estimate of HR(T/C) is less than 1 + ********o****.”**
- That is, if
**               ****HR(T/C) + 1.96 s.e. [HR(T/C)]  <  1 + ********o ****        ****----- (5).**
**    **

## Active Control Non-Superiority Trials Case (1): Fixed Margin  

-     Graphically,
-     Ha:  HR(T/C) < 1+      Ho:    HR(T/C)  1+ 
-                                       
-                                   1              1+ Hazard Ratio
-                          (                                         )
-                                         HR(T/C) + 1.96 s.e. [HR(T/C)]

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- This  is just a constant, and it has no other meaningful interpretation.
- We allow T to be worse than C by an amount 
-     We say, T is within a - margin from C.
- Otherwise, a non-superiority trial with a fixed margin  is analogous to a superiority trial.
- So, what is the problem?

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

**Remark 2:**
- The same as in active control superiority trials, we need to assume that ***the active control is effective in the current trial***.
-    If the active control is not effective in the current trial, then we can only conclude that the new drug is within a margin of o  from placebo.

## Active Control Non-inferiority TrialsCase (1): Fixed Margin

- The fixed margin, o, is just an arbitrary constant.  It has no relation to the true control effect.

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- If the fixed margin o is actually ***larger*** than the true control effect, then the rejection of the above null hypothesis ***may lead to the approval of a new treatment that may be worse than a placebo*** as illustrated in the following diagram.

## When o is Larger than the True Effect of the Active Control

- This can readily be seen from the following diagram:
-                                    Ha:  HR(T/C) < 1 +               Ho:  HR(T/C)  1 + 
-                                               ********                    (1+)
-                                       Effect of C        HR(P/C)
-                              1                                   1.10             1.20                    Hazard Ratio
-                                (                                             )
-                                                   HR(T/C) + 1.96 s.e. [HR(T/C)]

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- If the true control effect is known, or if the true control effect can be estimated relatively accurately, then we may consider an active control non-superiority trial with a fixed margin  < control effect.
- For example, in antibiotic trials.

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- In reality, we never know the true effect of the control.
- We may estimate the effect of the control and its variability
- By choosing an arbitrary fixed margin, o , there is a positive probability that o > the true control effect.  The magnitude of this probability depends on the variability of the estimate of the control effect.

## Figure 1Fixed Margin and Active Control Effect

**1**

**HR(P/C)**

**             ****(                                     )    **
**HR(P/C)-z******** s.e.HR(P/C)**

**HR(T/C)**

**      ****(                                         )**
**                           ****HR(T/C)+z********s.e.HR(T/C)**
** **

**1+********o**

******o **** = 1/2 { [****HR(P/C) - z********s.e.HR(P/C)] - 1 }**
**      ****=  Amount of control effect willing to give up**

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- If the true control effect is small, or if the estimate of the true control effect has  relatively large variability, then an active control non-superiority trial with an arbitrary fixed margin o may be problematic.
- This is because the probability of approving a drug that may be worse than a placebo can be high.

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- So in order to avoid or minimize this problem, the best way is to set the margin to be smaller than the true control effect.   That is, by setting the margin to be a certain fraction, , of the true control effect:
-                         =  [HR(P/C) - 1]            ----(6)

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- But we never know the true control effect, [HR(P/C) - 1].
- If we use the lower limit of the 95% confidence interval as an estimate of the control effect, and define  to be a certain fraction  of this estimate, then the probability of  > true control effect will be small.  That is,
-     =  {HR(P/C) - 1.96 s.e.[HR(P/C)] - 1}  ---(7)

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- In (7), if we choose = 1/2, then the test of the non-superiority hypothesis (3) with the fixed margin  given by
-  = 1/2 {HR(P/C) - 1.96 s.e.[HR(P/C)] - 1}  ---(8),
-    corresponds to the two 95% confidence interval testing procedure, also known as the CBER method.

## Active Control Non-inferiority TrialsCase (1): Fixed Margin

- With the fixed margin  defined in (8), the probability of > the true control effect, [HR(P/C) -1], will be small.  Thus, the probability of approving a new drug that is worse than a placebo will be small.
- Is there a problem with this choice of ?

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- The answer is yes.  The reason is that this choice of  is generally too conservative, because the likelihood of the true control effect being that extreme is small.
- If there is good reason to be conservative, then the non-superiority hypothesis (3) with this fixed margin  may be acceptable.  This is the case with some thrombolytic trials.

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- But if there is no justifiable reason to be conservative, then the non-superiority hypothesis (3) with the fixed margin  as defined in (8) will not be recommended.
- In that event, what is the alternative?

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- In (8), if we use simply the point estimate for HR(P/C), then the test of the non-superiority hypothesis (3) with the fixed margin  set at half of the observed active control effect given by
-  = 1/2 [HR(P/C) - 1]  ---(9).

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- In (8), if we use the lower limit of any other  100 (1-)% confidence interval of the estimate for HR(P/C), then the test of the non-superiority hypothesis (3) with the fixed margin  given by
-  = 1/2 [HR(P/C) - z(1-/2) s.e.[HR(P/C)] 1]  --- (10).

## Active Control Non-Superiority TrialsCase (1): Fixed Margin

- With the fixed margin  defined as in (10), the probability that > the true control effect, [HR(P/C) -1], will be larger, and this implies that the probability of approving a new drug that is worse than a placebo will also become larger.
- What would be an appropriate choice for ?

## Active Control Non-Superiority TrialsCase (1): Fraction Retention

- Let us revisit the non-superiority hypothesis with fixed margin o given in (3), i.e.,:
-    Ho: HR(T/C)  1+   vs.  Ha:  HR(T/C) < 1+ ---(3)
- If we simply replace the o in (3) by the  in (6), i.e.,
-   =  [HR(P/C) - 1]                       ---(6)

## Active Control Non-Superiority TrialsCase (1): Fraction Retention

- We have the following non-superiority hypotheses:
-         Ho: HR(T/C)  1+  [HR(P/C) - 1]
-                               vs.                                            ---(11)
-          Ha:  HR(T/C) < 1+  [HR(P/C) - 1]

## Active Control Non-Superiority TrialsCase (1): Fraction Retention

- It can be shown that if we define
-                        = ( 1 - o)
-    where o is the fraction of the control effect to be retained by the new drug, then the margin
-  [HR(P/C) - 1] = ( 1 - o) [HR(P/C) - 1]
-    can be interpreted as the fraction of the true control effect that we are willing to give up.

## Active Control Non-Superiority TrialsCase (1): Fraction Retention

- Our non-superiority hypotheses (11) then become
-         Ho: HR(T/C)  1 + ( 1 - o) [HR(P/C) - 1]
-                                  vs.                                          ---(12)
-          Ha: HR(T/C) < 1 +( 1 - o) [HR(P/C) - 1]
-                                                   

## Active Control Non-Superiority Trials Case (2): Fraction Retention

- The null hypothesis Ho in (12) can be tested directly by the test statistic T given by:
-                 [HR(T/C) - 1] - {(1- ) [HR(P/C) - 1]}
-      T =                                                                                   < - 1.96
-             s.e.2 [HR(T/C)-1] +  s.e.2{(1- ) [HR(P/C) - 1]}
-      Reject Ho , if  T < -1.96.

## Active Control Non-Superiority TrialsCase (2): Fraction Retention

- The rejection of the null hypothesis in (12) leads to the conclusion that the new drug loses less than (1-o) fraction of the control effect [HR(P/C) -1].
- The null hypothesis Ho in (12) can be derived more formally as follows.

## Active Control Non-Superiority TrialsCase (2): Fraction Retention

- Consider the new treatment effect as a fraction of the control effect:
-                         [HR(P/C) - 1] - [HR(T/C) - 1]
-              =
-                                      [HR(P/C) - 1]
-                             [HR(P/C)  - HR(T/C)]
-                =                                                     ,  where HR(P/C)-1 > 0
-                                      [HR(P/C) - 1]

## The Fraction Retention Hypothesis

- Our interest is to test whether the new treatment retains at least a certain fraction,  of the control effect:
-          Ho:        vs.  Ha:      >  
-     That is,
-                             [HR(P/C)  - HR(T/C)]
-          Ho:     =                                               ----(13)
-                                    [HR(P/C) - 1]

## The Fraction Retention Hypothesis

- The hypotheses in (13) can be algebraically rewritten as (12).
- Thus, testing the non-superiority hypotheses in (12) is equivalent to testing the non-superiority hypothesis in (13).  That is, the test of the hypotheses in (12) is equivalent to testing whether the new drug retains at least  fraction of the true control effect, [HR(P/C)-1].

## Active Control Non-Superiority Trials Case (2): Fraction Retention

- The two 95% confidence interval testing procedure is simply testing whether
-      {HR(T/C) + 1.96 s.e.HR(T/C) -1} - {(1- o)[HR(P/C) -1.96 s.e.HR(P/C) - 1]} < 0  ---(14).

## Active Control Non-Superiority Trials Case (2): Fraction Retention

- We can show that the two 95% confidence interval testing procedure is equivalent to testing the null hypothesis (12) of a loss of less than (1- ) of the true control effect, [HR(P/C) -1],  using the test statistic
-                 [HR(T/C)-1] - {(1- )[HR(P/C)-1]}
-  =                                                                                     < - 1.96
-             s.e.2[HR(T/C)-1] + s.e.2{(1- )[HR(P/C)-1]}

## Some Simple Geometry

- s.e.HR(P/C) = b

- a = s.e. HR(T/C)

-   a2 + b2

- b’ = (z1-/1.96) s.e.HR(P/C)

- a

-    a + b  > a2 + b2  and   0 < b’< b   a+b’= a2 + b2

## Active Control Non-Superiority Trials Case (2): Fraction Retention

- Now from the preceding geometric property, we have:
-           s.e.2[HR(T/C)-1] +  s.e.2{(1- )[HR(P/C)-1]}   >
-          s.e.2 [HR(T/C)-1] +  s.e.2{(1- )[HR(P/C)-1]}    --- (15)

## Active Control Non-Superiority Trials Case (2): Fraction Retention

-    is a statistic for testing the null hypothesis Ho of retaining at least o percent of the active control effect that corresponds to the two-95% CI testing procedure.
-     From the geometric property (15),  it follows that  T<  < -1.96.

## Active Control Non-Superiority TrialsCase (2): Fraction Retention

- This implies that relative to the non-superiority hypothesis (12) or (13), the two 95% CI testing procedure is more conservative than the test statistic T in testing the null hypothesis Ho  in (13) of retaining at least o fraction of the  control effect (Rothmann et.al.)
- When we set o = 0, then the test of the hypothesis (12) or (13) using the test statistic  is the Hasselblad-Kong method for testing 0% retention.

## Active Control Non-Superiority TrialsCase (2): Fraction Retention

- So we have argued why the non-superiority hypothesis should be defined or formulated as fraction retention (13), or as fraction loss (12).
- We have shown that relative to this fraction retention null hypothesis(13), the two 95% confidence interval testing approach is more conservative than testing the null hypothesis (12) using the statistic T.

## Active Control Non-Superiority Trials Case (2): Fraction Retention

- We can similarly show that the use of point estimate for HR(P/C) would inflate the type I error relative to this null hypothesis, because   < T <  < -1.96, where  is the test statistic corresponding to using the margin  in (9),
-                 [HR(T/C)-1] - {(1- )[HR(P/C)-1]}
-  =                                                                                    < - 1.96.
-                                  s.e.2[HR(T/C)-1]

## Active Control Non-Superiority Trials Case (2): Fraction Retention

- In fact, for a given o, any two asymmetric confidence interval testing procedure is equivalent to testing the null hypothesis of  retaining o fraction, or 100 o% of the active control effect by the following test statistic, () :
-                                      [HR(T/C)-1] - (1- )[HR(P/C)-1]
-          () =                                                                                              < -1.96
-                       s.e.[HR(T/C)-1] - (1-)[z(1-)/1.96] s.e.[HR(P/C)]-1]}
- We have,                     (1) < () < (0.05) = 

## Active Control Non-Superiority TrialsCase (2): Fraction Retention

- For a given retention fraction, o, we can show that the test statistic T for testing the non-inferiority hypothesis is *conditionally equivalent to an asymmetric (95%, 100(1-*****)%) confidence interval testing procedure, where ****** 0.05*.
- In other words, the data allows one to claim that the new treatment retains at least a fraction  of the active control effect when this control effect is estimated with greater precision than it actually is.

## Active Control Non-Superiority TrialsCase (2): Fraction Retention

- So what is the problem?
- The question is:  How do we set the level of fraction retention?

## Active Control Non-inferiority TrialsCase (2): Fraction Retention

- For demonstrating non-inferiority or equivalence, what should be the fraction of retention?
-                                   o = 0.80, 0.90?

## Active Control Efficacy TrialsCase (2): Fraction Retention

- For simply demonstrating effectiveness, what should be the fraction of retention?
- o = 0.50?
- The answers to these questions should not be based on clinical judgment alone.  We need statistical guideline also.  We are currently working on this statistical guideline.  That would be the subject of our next talk.

## Summary

- For both active control superiority and non-superiority trials, a necessary and important assumption is that the active control be effective in the current trial.
- This active control effect can be reliably estimated by historical data and other available information.

## Summary

- Assurance about the assay sensitivity of the current trial.
- Except when the true active control effect is known, or can be fairly precisely estimated, an arbitrary fixed margin active control non-superiority trial may be problematic.  It may lead to the approval of a new treatment that is worse than a placebo, when this fixed margin is > true control effect.

## Summary

- To avoid or minimize this problem of a fixed margin, it is shown that a natural approach is to define the active control non-superiority hypotheses in terms of a fixed fraction of the true control effect to be retained.
- The two 95% confidence interval testing procedure is shown to be more conservative than testing the same non-superiority hypotheses using the test statistic T.

## Summary

- Whereas the corresponding testing procedure based on the point estimate of the control effect is shown to be more liberal than the test statistic T.
- It is shown that T is conditionally equivalent to an asymmetric (95%, 100(1-*)%) two confidence interval testing procedure.

## Summary

- The objective of an active control non-superiority trial partially determines the fraction of control effect to be retained.  It also requires statistical guideline in addition to clinical considerations.
- The subject of this statistical guideline will be the content of our next talk.

## References

- Hasselblad & Kong (Drug Inf. J. 35, 435-449, 2001)
- Koch & Tangen (Drug Inf. J. 33, 1145-1159, 1999)
- Hauck & Anderson (Drug Inf. J. 33, 109-118, 1999)
- Berger & Hsu (JASA 1996)

## On the Effect of New Treatment as A Percent of Active Control Effect

- Returning to the hypothesis (6):
- Ho: HR(T/C)  >  1 + (1- ) [HR(P/C) - 1], i.e.,
- Ho: [HR(T/C) - 1] - {(1- ) [HR(P/C) - 1]} > 0
-     A test statistic of Ho is given by:
-                 [HR(T/C) - 1] - {(1- ) [HR(P/C) - 1]}
- T =                                                                                   < - 1.96
-             s.e.2 [HR(T/C)-1] +  s.e.2{(1- ) [HR(P/C) - 1]}
-      Reject Ho , if  T < -1.96.

## What the Two Confidence Interval Testing Procedure is really Testing?

- In other words, any given two confidence interval testing procedure is equivalent to testing the null hypothesis of  retaining o fraction, or 100 o% of the active control effect by the following test statistic, () :
-                                      [HR(T/C)-1] - (1- )[HR(P/C)-1]
-          () =                                                                                              < -1.96
-                       s.e.[HR(T/C)-1] - (1-)[c(1-)/1.96] s.e.[HR(P/C)]-1]}
- Let us note that if  = 0.95, then this is the CBER’s two 95% confidence interval testing procedure.  If  = 0.00, then this is equivalent to using the point estimate of the active control effect without any error.  If o=0.0 and  =0.95, then this is the Hassalblad-Kong testing procedure which tests for 0% retention, that is, comparing the new treatment to an imputed (virtual) placebo.  For Hassalblad-Kong, testing at 0% retention and =0.05 implies that the margin is just the active control effect.

## What the Two Confidence Interval Testing Procedure is really Testing?

- If we consider the margin  as a fixed constant, then the test of the null hypothesis of o fraction, or 100 o% of the active control effect will be at the =0.025 level.
- If we don’t consider the  as a fixed constant, then the two confidence interval testing procedure provides a test statistic, ().
- How do we evaluate the statistical property of the test statistic () ?
- How do we know if () inflates the type I error?
- (0) < () < (0.95)

## Active Control Superiority Trials

-  If  C is very effective, then it would be
-     difficult to demonstrate that T > C.
- If C is not so effective, then T may have a chance, if T is slightly better than C and the sample size is relatively large.

## On the Effect of New Treatment as A Percent of Active Control Effect

- Now from the geometric inequality:
-          s.e.2 [HR(T/C) -1] +   s.e.2{(1- )[HR(P/C)-1]}   >
-          s.e.2 [HR(T/C)-1] +  s.e.2{(1- )[HR(P/C)-1]}    --- (**)
-    It follows that generally, there exists a 0 <  < 1, and if
-                   [HR(T/C)-1] - {(1- )[HR(P/C)-1]}
-  =                                                                                              < - 1.96
-             s.e.2[HR(T/C)-1] +(c 1-/1.96)s.e.2{(1- )[HR(P/C)-1]}

## Some Simple Geometry

- s.e.HR(P/C) = b

- a = s.e. HR(T/C)

-   a2 + b2

- b’ = (c1-/1.96) s.e.HR(P/C)

- a

-    a + b  > a2 + b2  and   0 < b’< b   a+b’= a2 + b2

## Some Simple Geometry

- s.e. HR(P/C)  =  b

- a = s.e. HR(T/C)

-   a2 + b2

- a

-    a + b  >  a2 + b2  and   0 < b’< b   a+b’= a2 + b2

- b’ = (c1-/1.96) s.e.HR(P/C)

## Some Simple Geometry

- s.e. HR(P/C)  =  b

- a = s.e. HR(T/C)

-   a2 + b2

- a

-    a + b  >  a2 + b2  and   0 < b’< b   a+b’= a2 + b2

- b’ = (c1-/1.96) s.e.HR(P/C)

## Some Simple Geometry

- The preceding figures shows that when the s.e. HR(P/C) is very small compared to the s.e. HR(T/C), then b’ becomes very small and negligible.  This implies that when the variability of the estimate of HR(P/C) is small compared to the variability of the estimate of HR(T/C), then using the point estimate instead of the lower (1-% confidence limit for the estimate of HR(P/C) may be acceptable (for the same percent retention).

## Some Simple Geometry

- The preceding figures shows that when the s.e. HR(P/C) is very large compared to the s.e. HR(T/C), then b and b’ are not very different.  This implies that when the variability of the estimate of HR(P/C) is large compared to the variability of the estimate of HR(T/C), then CBER’s two 95% confidence interval testing procedure and testing the percent retention hypothesis are not much different (for the same percent retention).

## Testing the Non-inferiority Null Hypothesis via the Two Confidence Interval Procedure

- The basic problem with the two confidence interval testing procedure is that the active control effect,
-     [HR(P/C) -1], is unknown, not a fixed constant.
- We need to estimate this active control effect along with the variability of this estimate, that is, the distributional properties of the estimate.
- Thus, the margin, , in
-         Ho:  HR(T/C) > 1 +     vs.   Ha:  HR(T/C) < 1 + 
-     should be considered as an unknown given by
-  = (1- ) [HR(P/C) - 1]

## What the Two Confidence Interval Testing Procedure really Testing?

- The two confidence interval testing procedure is really testing the following null hypothesis of retaining at least a certain o fraction, or 100 o%, of the active control effect.
- We can show that the two confidence interval testing procedure is equivalent to a test statistic,  , for testing this null hypothesis.

## What the Two Confidence Interval Testing Procedure really Testing?

- The two confidence interval testing procedure for testing the null hypothesis of retaining at least a certain o fraction, or 100 o%, of the active control effect is given by:
-     [HR(T/C)-1] + 1.96 s.e.[HR(T/C)-1] < (1- ){[HR(P/C)-1] - c(1-)s.e.[HR(P/C)-1]}
-       which is equivalent to,
-      {HR(T/C) + 1.96 s.e.[HR(T/C)] -1} < (1- ){HR(P/C) - c(1-)s.e.[HR(P/C)] -1}

## What the Two Confidence Interval Testing Procedure really Testing?

- Which is equivalent to,
-       HR(T/C) + 1.96 s.e.[HR(T/C)]  < 1 + (1- ){HR(P/C) - c(1-)s.e.[HR(P/C)] -1]}
- Which is equivalent to,
- [HR(T/C)-1]-(1- )[HR(P/C)-1] < -1.96 s.e.[HR(T/C)]-(1-)c(1-)s.e.[HR(P/C)]-1]}
- Which is equivalent to,
-                                  [HR(T/C)-1]-(1- )[HR(P/C)-1]
-     () =                                                                                              < -1.96
-                  s.e.[HR(T/C)-1] - (1-)[c(1-)/1.96] s.e.[HR(P/C)-1]}

## What the Two Confidence Interval Testing Procedure is really Testing?

- In other words, any given two confidence interval testing procedure is equivalent to testing the null hypothesis of  retaining o fraction, or 100 o% of the active control effect by the following test statistic, () :
-                                      [HR(T/C)-1] - (1- )[HR(P/C)-1]
-          () =                                                                                              < -1.96
-                       s.e.[HR(T/C)-1] - (1-)[c(1-)/1.96] s.e.[HR(P/C)]-1]}
- Let us note that if  = 0.95, then this is the CBER’s two 95% confidence interval testing procedure.  If  = 0.00, then this is equivalent to using the point estimate of the active control effect without any error.  If o=0.0 and  =0.95, then this is the Hassalblad-Kong testing procedure which tests for 0% retention, that is, comparing the new treatment to an imputed (virtual) placebo.  For Hassalblad-Kong, testing at 0% retention and =0.05 implies that the margin is just the active control effect.

## What the Two Confidence Interval Testing Procedure is really Testing?

- In other words, any given two confidence interval testing procedure is equivalent to testing the null hypothesis of  retaining o fraction, or 100 o% of the active control effect by the following test statistic, () :
-                                      [HR(T/C)-1] - (1- )[HR(P/C)-1]
-          () =                                                                                              < -1.96
-                       s.e.[HR(T/C)-1] - (1-)[c(1-)/1.96] s.e.[HR(P/C)]-1]}
- For Hassalblad-Kong, testing at 0% retention and =0.05 implies that the margin is just the active control effect.  Moreover, the numerator reduces to [HR(T/C) - HR(P/C)], which is the effect of the new treatment relative to P.
- We have,
-                                               (0) < () < (0.95)

## What the Two Confidence Interval Testing Procedure is really Testing?

- If we consider the margin  as a fixed constant, then the test of the null hypothesis of o fraction, or 100 o% of the active control effect will be at the =0.025 level.
- If we don’t consider the  as a fixed constant, then the two confidence interval testing procedure provides a test statistic, ().
- How do we evaluate the statistical property of the test statistic () ?
- How do we know if () inflates the type I error?
- (0) < () < (0.95)

## What the Two Confidence Interval Testing Procedure really Testing?

- The two confidence interval testing procedure is really testing the following null hypothesis of retaining at least a certain o fraction, or 100 o%, of the active control effect.
- We can show that the two confidence interval testing procedure is equivalent to a test statistic,  , for testing this null hypothesis.

## What the Two Confidence Interval Testing Procedure really Testing?

- The two confidence interval testing procedure for testing the null hypothesis of retaining at least a certain o fraction, or 100 o%, of the active control effect is given by:
-     [HR(T/C)-1] + 1.96 s.e.[HR(T/C)-1] < (1- ){[HR(P/C)-1] - c(1-)s.e.[HR(P/C)-1]}
-       which is equivalent to,
-      {HR(T/C) + 1.96 s.e.[HR(T/C)] -1} < (1- ){HR(P/C) - c(1-)s.e.[HR(P/C)] -1}

## What the Two Confidence Interval Testing Procedure really Testing?

- Which is equivalent to,
-       HR(T/C) + 1.96 s.e.[HR(T/C)]  < 1 + (1- ){HR(P/C) - c(1-)s.e.[HR(P/C)] -1]}
- Which is equivalent to,
- [HR(T/C)-1]-(1- )[HR(P/C)-1] < -1.96 s.e.[HR(T/C)]-(1-)c(1-)s.e.[HR(P/C)]-1]}
- Which is equivalent to,
-                                  [HR(T/C)-1]-(1- )[HR(P/C)-1]
-     () =                                                                                              < -1.96
-                  s.e.[HR(T/C)-1] - (1-)[c(1-)/1.96] s.e.[HR(P/C)-1]}

## What the Two Confidence Interval Testing Procedure is really Testing?

- In other words, any given two confidence interval testing procedure is equivalent to testing the null hypothesis of  retaining o fraction, or 100 o% of the active control effect by the following test statistic, () :
-                                      [HR(T/C)-1] - (1- )[HR(P/C)-1]
-          () =                                                                                              < -1.96
-                       s.e.[HR(T/C)-1] - (1-)[c(1-)/1.96] s.e.[HR(P/C)]-1]}
- Let us note that if  = 0.95, then this is the CBER’s two 95% confidence interval testing procedure.  If  = 0.00, then this is equivalent to using the point estimate of the active control effect without any error.  If o=0.0 and  =0.95, then this is the Hassalblad-Kong testing procedure which tests for 0% retention, that is, comparing the new treatment to an imputed (virtual) placebo.  For Hassalblad-Kong, testing at 0% retention and =0.05 implies that the margin is just the active control effect.

## What the Two Confidence Interval Testing Procedure is really Testing?

- In other words, any given two confidence interval testing procedure is equivalent to testing the null hypothesis of  retaining o fraction, or 100 o% of the active control effect by the following test statistic, () :
-                                      [HR(T/C)-1] - (1- )[HR(P/C)-1]
-          () =                                                                                              < -1.96
-                       s.e.[HR(T/C)-1] - (1-)[c(1-)/1.96] s.e.[HR(P/C)]-1]}
- For Hassalblad-Kong, testing at 0% retention and =0.05 implies that the margin is just the active control effect.  Moreover, the numerator reduces to [HR(T/C) - HR(P/C)], which is the effect of the new treatment relative to P.
- We have,
-                                               (0) < () < (0.95)

## What the Two Confidence Interval Testing Procedure is really Testing?

- If we consider the margin  as a fixed constant, then the test of the null hypothesis of o fraction, or 100 o% of the active control effect will be at the =0.025 level.
- If we don’t consider the  as a fixed constant, then the two confidence interval testing procedure provides a test statistic, ().
- How do we evaluate the statistical property of the test statistic () ?
- How do we know if () inflates the type I error?
- (0) < () < (0.95)

## On the Effect of New Treatment as A Percent of Active Control Effect

- Actually, from the preceding slide, one can see that the original null hypothesis, Ho , could be rewritten as follows:
-                     [HR(T/C) - 1]
-                 Ho:                                    (1- ) ,
-                             [HR(P/C) - 1]
-     vs.
-                     [HR(T/C) - 1]
-                 Ha:                                    < (1- )
-                             [HR(P/C) - 1]

## On the Effect of New Treatment as A Percent of Active Control Effect

- Now returning to the hypothesis (*):
- Ho: [HR(T/C) - 1] >  (1- ) [HR(P/C) - 1], i.e.,
- Ho: [HR(T/C) - 1] - {(1- ) [HR(P/C) - 1]} > 0
-     (This is the opposite of the ordinary superiority hypothesis), a test statistic of Ho is given by:
-                 [HR(T/C) - 1] - {(1- ) [HR(P/C) - 1]}
- T =                                                                                   < - 1.96
-             s.e.2 [HR(T/C)-1] +  s.e.2{(1- ) [HR(P/C) - 1]}
-      Reject Ho , if  T < -1.96.

## The Two-95% Confidence Interval Testing Procedure - The CBER Approach

- But if in the CBER’s two 95% confidence interval testing procedure, ******(************,1,0.05)** is not considered as a constant, then the two 95% confidence interval testing procedure (16) or (17) is actually testing the null hypothesis of retaining at least a certain o fraction, or 100 o%, of the active control effect.
- We can show that the two confidence interval testing procedure is equivalent to a test statistic, ************ , for testing this null hypothesis. ************ may have slow asymptotic convergence property.

## The Two-95% Confidence Interval Testing Procedure - The CBER Approach

- This can be seen as follows:

## What the Two 95% Confidence Interval Testing Procedure is Really Testing?

- The two 95% confidence interval testing procedure for testing the null hypothesis of retaining at least a certain o fraction, or 100 o%, of the active control effect is given by:
-     [HR(T/C)-1]+1.96 s.e.[HR(T/C)-1] < (1- ){[HR(P/C)-1] - 1.96 s.e.[HR(P/C)-1]}
-                                                                                                             ----- (18)
-       which is equivalent to,
-      {HR(T/C) + 1.96 s.e.[HR(T/C)] -1} < (1- ){HR(P/C) - 1.96 s.e.[HR(P/C)] -1}
-                                                                                                             ----- (19)

## What the Two 95% Confidence Interval Testing Procedure is Really Testing?

- Which is equivalent to,
-       HR(T/C) + 1.96 s.e.[HR(T/C)]  < 1 + (1- ){HR(P/C) - 1.96 s.e.[HR(P/C)] -1]}
-                                                                                                                ----- (20)
- Which is equivalent to,
- [HR(T/C)-1]-(1- )[HR(P/C)-1] < -1.96 s.e.[HR(T/C)]- 1.96(1-) s.e.[HR(P/C)]-1]}
-                                                                                                                ----- (21)
- Which is equivalent to,
-                                  [HR(T/C)-1]  -  (1- )[HR(P/C)-1]
-     ****() =                                                                                              < -1.96  ----- (22)
-                           s.e.[HR(T/C)-1]  +  (1-) s.e.[HR(P/C)-1]}

## An Asymmtric Two Confidence Interval Testing Procedure 

- One can similarly show that any asymmetric two confidence interval testing procedure is really testing the following null hypothesis of retaining at least a certain o fraction, or 100 o%, of the active control effect.
- We can show that any asymmetric two confidence interval testing procedure is equivalent to a test statistic, ************ , for testing this null hypothesis.

## What An Asymmetric Two Confidence Interval Testing Procedure is Really Testing?

- An asymmetric two confidence interval testing procedure for testing the null hypothesis of retaining at least a certain o fraction, or 100 o%, of the active control effect is given by:
-     [HR(T/C)-1]+1.96 s.e.[HR(T/C)-1] < (1- ){[HR(P/C)-1] -  c(1-) s.e.[HR(P/C)-1]}
-                                                                                                                      ----- (23)
-       which is equivalent to,
-      {HR(T/C) + 1.96 s.e.[HR(T/C)] -1} < (1- ){HR(P/C) - c(1-) s.e.[HR(P/C)] -1}
-                                                                                                                     ----- (24)

## What An Asymmetric Two Confidence Interval Testing Procedure is Really Testing?

- Which is equivalent to,
-       HR(T/C) + 1.96 s.e.[HR(T/C)]  < 1 + (1- ){HR(P/C) - c(1-) s.e.[HR(P/C)] -1]}
-                                                                                                                      ----- (25)
- Which is equivalent to,
- [HR(T/C)-1]-(1- )[HR(P/C)-1] < -1.96 s.e.[HR(T/C)]-(1-) c(1-) s.e.[HR(P/C)]-1]}
-                                                                                                                     ----- (26)
- Which is equivalent to,
-                                        [HR(T/C)-1]  -  (1- )[HR(P/C)-1]
-     ****o =                                                                                                         < -1.96
-                           s.e.[HR(T/C)-1]  +  (1-)[c(1-)/1.96] s.e.[HR(P/C)-1]}
-                                                                                                                     ----- (27)

## What An Asymmetric Two Confidence Interval Testing Procedure is Really Testing?

- In other words, any asymmetric two confidence interval testing procedure is equivalent to testing the null hypothesis of  retaining o fraction, or 100 o% of the active control effect by the following test statistic, (o,) :
-                                      [HR(T/C)-1] - (1- )[HR(P/C)-1]
-      ****(o,) =                                                                                             < -1.96 ---- (28)
-                       s.e.[HR(T/C)-1] + (1-)[c(1-)/1.96] s.e.[HR(P/C)]-1]}
- Note that if  = 0.05, then this is the CBER’s two 95% confidence interval testing procedure.  If  = 1, then this is equivalent to using the point estimate of the active control effect without any error.  If o=0 and  =0.05, then this is the Hassalblad-Kong testing procedure that tests for 0% retention, i.e., comparing the new treatment to an imputed (virtual) placebo.  For Hassalblad-Kong, testing at 0% retention and =0.05 implies that the margin = the point estimate of the active control effect.

## What An Asymmetric Two Confidence Interval Testing Procedure is Really Testing?

- In other words, any asymmetric two confidence interval testing procedure is equivalent to testing the null hypothesis of  retaining o fraction, or 100 o% of the active control effect by the following test statistic, ****(o,) :
-                                      [HR(T/C)-1] - (1- )[HR(P/C)-1]
-       (o,) =                                                                                             < -1.96 ---(29)
-                       s.e.[HR(T/C)-1] + (1-)[c(1-)/1.96] s.e.[HR(P/C)]-1]}
- For Hassalblad-Kong, testing at 0% retention and =0.05 implies that the margin is just the point estimate of the active control effect.  Moreover, the numerator reduces to [HR(T/C) - HR(P/C)], which is the effect of the new treatment relative to P.
- We have,
-                             (o,1) < (o,) < (o,0.05) < (o,0)              -----(30)

## What An Asymmetric Two Confidence Interval Testing Procedure is Really Testing?

- The problem is, how does one choose ?
- Do we need to choose a ?
- The answer is, we do not need to choose a  ahead of time.  We will explain why later.

## What An Asymmteric Two Confidence Interval Testing Procedure is Really Testing?

- If we consider the margin  as a fixed constant, then the null hypothesis (8) can only be interpreted as non-inferiority of T to C within , where  is determined by (7).   That is all.  The use of 95% lower confidence limit in defining  is simply a way of reassuring ourselves that we are not defining a  that is larger than the active control effect.  We can not assert that we are retaining at least o fraction of the active control effect which is unknown (What is the type I error rate relative to the null hypothesis of retaining at least o fraction of the active control effect ?).  In this case of a fixed margin , the test statistic **t** always guarantees the Type I error at a one-sided =0.025 level for the fixed margin null hypothesis of (8).
- If we don’t consider the  as a fixed constant, then the two confidence interval testing procedure provides a test statistic, ****(o,), given by (29).
- How do we evaluate the statistical property of the test statistic ****(o,) ?
- How do we know if ****(o,) inflates the type I error?
- ****(o,1) < ****(o,) < ****(o,0.05)

## Fraction of the Active Control Effect to be Retained

- Note that when  is considered as a fixed constant, then **t**() is controlled at =0.05 by design.
- When  is not considered as a fixed constant,I.e., we are treating the control effect as unknown parameter, then the two 95% confidence interval testing procedure will be too conservative, while the point estimate testing procedure will be too liberal relative to the non-inferiority hypothesis of retaining a certain percent of the active control effect (as discussed on slides 33-104).
- Relative to the non-inferiority null hypothesis, ****(o) is just right.  It controls the type I error at =0.05 and protects against over conservatism as well as over liberalism, as we have demonstrated (through the equivalence of the asymmetric confidence interval approach and the test statistic ****(o).

## Active Control Non-inferiority Trials Case (2): Fraction Retention

- The two 95% confidence interval testing procedure is simply testing whether
-            {HR(T/C) + 1.96 s.e.HR(T/C) -1} - {(1- o)[HR(P/C) -1.96 s.e.HR(P/C) - 1]} < 0  ---(#).
-      Upon some algebraic manipulation, one can show that the above
-      inequality (#) is simply equivalent to testing
-                                              {HR(T/C) - 1} - {(1- o)[HR(P/C) -1]}
-                               =                                                                                   <  - 1.96.
-                                        s.e.[HR(T/C) - 1] + (1- o) s.e. {[HR(P/C) -1]}