# Presentation: 183543

**Efficient:****	**The correct answer about a treatment’s clinical
- 		benefit with as few subjects as possible as quickly
- 		as possible.
**Definitive:****	**Yields results that change clinical practice.
*         **We’re focusing today on design and analysis options. *
*                              **What about the effect size? **	** *

- Can we improve the ***efficiency*** of ***definitive*** clinical trials of
- treatments to protect beta-cells in persons with or
- at risk for T1D?

## I’ll talk about: 

-    •	The effect size in general terms.
-    •	The clinical justification for smaller definitive trials that
- 	are only powered to detect larger effects.
-    •      The disadvantages of smaller trials.
-    •      Focus will mostly be on prevention trials.

## The Effect Size

- Sample size estimates for T1D prevention trials:
- 	   Risk for diabetes in control group
- 	   Risk for diabetes in experimental group
- 	   Risk for a false-positive conclusion
- 	   Risk for a false-negative conclusion

- The effect size is the difference in risks for diabetes.
- Larger effects yield smaller sample sizes.

## Biological Versus Clinical Effects

- Biological effects:
- •  Concerned with variables bearing on inferences re: disease pathogenesis,
-     mechanism of action, pharmacokinetics/dynamics, etc.
- •  Primary focus in Phase 1/2 trials (but can be important in Phase 3 trials).
- •  Not always clinically important, at least in the short-term.

- Clinical effects:
- •  Concerned with variables that matter to patients – “live better, live longer”.
- •  Primary focus in Phase 3 (definitive) trials.

## Choosing an Effect Size

- Operational Definition:
- •   The smallest difference that would make patients and clinicians want
-      to use the new treatment or the Minimal Clinically Important Difference
-      (MCID).
- •   Other terms: MID, CMD, MDD.

- Other Inputs:
- •   Prevailing view that highly effective treatments are uncommon,
-      eg., RRR in diabetes rates that exceed 50%.
- •   Results of Phase 2 trials and/or observational studies in humans.
- •   Practical considerations: “How many subjects can I get?” instead of
-     “How many subjects do I need?”

## Choosing an Effect Size

-       pre-trial thinking on clinical issues.
- •    Larger MCIDs yield smaller sample sizes.
- •   It’s therefore important that TrialNet think carefully about
-      the MCID and, if the clinical argument is strong, consider
-      accepting larger over smaller MCIDs.

**Choosing an Effect Size**

## The Case for Choosing Large MCIDs 

**in TID Prevention Trials**

- Start at the end and work  backwards:
- •    TrialNet completes a prevention trial that is “positive”,
-       ie., reduced risk for diabetes over the trial’s duration
-       ( 5 years).
- •    Translating the results to clinical  practice will need to
-      deal with real and perceived concerns among clinicians
-      and patients to whom the results will apply.
- •    What are those concerns?

## Concern Number 1.

- For most treatments, the risk for adverse effects in the
- long-term (eg., 5 – 10 years) are unclear, esp. in kids.
- Otherwise well persons will receive a potentially toxic
- treatment for a problem (future diabetes) that, in some
- cases, they aren’t going to develop in the first place.    ** **

## Concern Number 2. 

- The efficacy-effectiveness “dilemma”:
- The benefits and risks seen in a Phase 3 trial under ideal
- (efficacy) conditions will be, respectively, over- and under-
- estimated in routine practice (effectiveness).
- This applies to most clinical trials -- it will receive greater
- emphasis in T1D prevention trials.

## Concern Number 3.

- Treatment for T1D has improved over the last 2 decades, and
- is likely to continue to improve over the next decade.
- The new treatment option of beta-cell protection, in whatever
- form it ultimately takes, will not be used in routine practice if
- there are doubts it’s not better relative to the existing
- standard of care.

##  1.   Exact level of resistance to adapt any single treatment into 

-  1.   Exact level of resistance to adapt any single treatment into
-        practice can’t be predicted.  However, in some cases it will
-        prevent translation to practice.
-  2.   All other things being equal, prevention trials showing
-        large effects will have a greater chance of being accepted
-        over trials showing small effects.
- 		   *(The MCID is bigger than we think?)*

## Numbers Needed to Treat (NNT) and Numbers Needed to Screen (NNS) 

**	****	****         ****	**** **
**Diabetes rates over 5 years:**
-      Controls 				50%	    50%	     50%	      50%
-      Experimental subjects  			33%	    25%  	     17%        12%
**Relative risk reduction****	****	****	**33%	    50%	     67%	      75%
**Total N randomized ****	****   ****	****	**256	    110	       56 	        40
**NNT to prevent 10 cases/5yrs:****	**** ****	****  **60	      40	       30	        26
-    Number who develop DM despite Rx	  20	      10	         5 	         3
-    Number who will not develop DM  but 	  30	      20           15           13
-    who still receive the Rx
**NNS to prevent 10 cases/5yrs ****	****         **13,620	 9,080	  6,810      5,900

**Numbers Needed to Treat (NNT) and Numbers Needed to Screen (NNS) **
**to prevent 10 cases of diabetes/5 years according to effect size.**

## Small trials have important 

- limitations that have to be
- accepted or accomodated for.

## We will miss treatments that are not highly effective, 

**because most treatments won’t exert large effects.****  **
- •    Negative impact on Study Group and third party payers
-      as several negative (“no effect”) trials accrue.
- •	Modestly effective treatments may prove to be clinically
- 	important.
- •  	Lost opportunity to combine 2 or more modestly effective
- 	Rxs in subsequent trials for additive/synergistic effects.
- •    Miss biological inferences based on the Rx’s mechanism
- 	of action relative to mechanisms of beta-cell destruction.

## “We will miss real effects in smaller trials, but I don’t 

***care. That’s because I think treatment effects smaller ***
***than what we’re able to detect in a small trial are not ***
***clinically or biologically important.”***

**Responses:**

- The “silver bullet” gamble may not be as risky as we think:
-     •  There are many promising-looking treatments at hand
-     •  There are precedents, egs., The CERT of Cyclosporine
- 				          The French anti-CD3?

## Greater potential impact on the trial’s conclusion because

**of imbalances in baseline variables associated with the **
**outcome.**

- Response:
- •   There are ways to deal with this, eg. stratification.
- •   Our challenge – agreeing on the variables that are
-      important enough to need  attention:
- 	•	Prevention trials: good shape.
- 	•	Intervention trials: not quite as clear?

## Reduced power to detect adverse effects, especially

**uncommon or rare serious adverse effects.**

- Response:
-    •   Randomized trials have never been a good tool to
-        determine risks for rare events.
-    •   Followup of all randomized subjects over an extended
-         time (~ 5 to 10 years) beyond completion of trial’s
-         initial efficacy phase is needed no matter how big the
-         trial was.
*(To me, this is the single most important reason against us doing small trials.)*

## Summing Up

- TrialNet should consider shifting attention toward larger effect
- sizes (MCIDs), eg., RRR ~ 67% in diabetes rates in T1D
- prevention trials, at least for the general case.
- The final choice for the MCID must still be adjusted according
- to information that is specific to the test therapy.
- Smaller trials have important disadvantages that we have to
- either accept or address during trial planning and post-trial
- followup.

##    N/group  =  (Z + Z) 2 [(Pr C)(1 – Pr C) + (Pr  E) (1 – Pr  E)] 

-    N/group  =  _(Z____ + Z____) __2__ [(Pr C)(1 – Pr C) + (Pr  E) (1 – Pr  E)]_
- 	                                (Pr C  –   Pr E) 2
- 	Z  = 1.96 (5% 2-tails), Z = 0.84 (80% power)
- 	Pr C = diabetes rate/5 yrs in controls
- 	Pr E = diabetes rate/5 yrs in experimental subjects