Study designs and statistical methods
for current observational studies
7. Feb. 2024
Kazuhiro Abe
Project Assistant Professor,
Graduate School of Medicine,
The University of Tokyo

Health Transition
1900
Disease
Structure

Acute Infection

High infant mortality

Industrial
Structure

Primary

Healthcare
Structure

Quantitative expansion

2000

Cancer/Stroke/Cardiovascular→

→

1980

1960

Chronic Infection

Population
Structure

K. Abe, 2.7.2024

1940

1920

Increased workforce

Secondary→→Technological innovation→

Tertiary→

2020

Social /Environment

Low birthrate and aging→

IoT/AI

Quality and quantity optimization2

Current features of observational research• Observational data
(i.e., administrative data, claims data, and GPS data)• Improved PC computing power• Development of research questions that we needto consider many confounders and unmeasured
confounders
K. Abe, 2.7.2024

3

Confounding
Confounders C(Patients’ background,hospitals’ factors, etc.)

Treatment X(Exposure)Intermediatevariables

Outcome Y• Confounders associate with the outcome and treatment.
They are not intermediate variables between treatment andoutcome.
• We often use the adjustment, stratification, matching, andweighting to deal with confounders.
K. Abe, 2.7.2024

4

Issues on the regression analysis• If there are many confounders and unobserved
confounders, the following issues would occur.
• Misconfiguration of the model• If too many variables need to be included in a model
relative to the number of events, the estimates fromthese models can be incorrect.
Am J Epidemiol 2003; 158. 280-7

K. Abe, 2.7.2024

5

Study designs and statistical analysis incurrent observational studies
1. Difference-in-Differences
Natural experimental
2. Instrumental Variable Analysis methods (Quasiexperimental methods)
3. Regression Discontinuity
4. Panel Data Analysis / Interrupted Time-Series
5. Propensity Score Analysis (matching, weighting,stratification, and adjustment using propensityscores)
6. Adjustment(regression), weighting, stratification,andmatching
K. Abe, 2.7.2024
6

Difference-in-Differences Analysis
Reference:Dimick, J. B. and A. M. Ryan (2014). Methods for evaluating changes in health carepolicy: the difference-in-differences approach. JAMA 312(22): 2401-2402.

K. Abe, 2.7.2024

7• Only one group: everyone gets treated• Counterfactual: the outcome would nothave changed without the intervention• Assumption: the outcome would beconstant before and after theintervention

OutcomePre-post comparison
B
B-A:

A

• “Simple differences” over time could
represent:– Actual interventional effect– Other factors that may haveaffected the group over time
– Mix of bothAbsolute changes are weak evidence forcausal effects
K. Abe, 2.7.2024

Change intreatmentgroupIntervention

TimeTreatment
BeforeA

AfterB

Change

B-A8

Key assumptions
1. Parallel trends assumption
If there is no intervention, theoutcomes of the treatment andcontrol group would be a paralleltrend.
Demonstrate that the outcomes ofthe treatment and control groups
are parallel at several time points
before the intervention.
2. Common shocks assumption
There are no other exogenous
events occurring that affect preand post-intervention outcomes, orif they occur, they affect the twogroups in the same way.
DID
= ( change in treatment ) – ( changein control )= ( TreatAfter - TreatBefore ) –After - ControlBefore )(ControlK. Abe, 2.7.2024

Outcome

Difference-in-Differences design
B

TreatmentGroup(B-A)-(D-C):
DID
B-A: change intreated group

A
DControlGroup

D-C: change incontrol group

CIntervention

Time

Treatment

Control

BeforeA

CAfterB

D

Change

B-A

D-CDID( B-A ) – ( D-C )9

DID example

FIGURE 1. Trends in hospitalizations for asthma and
COVID-19 for Japanese acute-care hospitals, 2017
through 2020.

Did the first COVID-19
policy announced by theJapanese government
affect asthma patients?
Intervention: COVID-19
policy (Week 9)
Treatment: 2020
Control: 2017-2019
DID=(78.3-140.8)-(229.9-184.9)= -107.5 per week
K. Abe, 2.7.2024

Abe K, Miyawaki A, Nakamura M, Ninomiya H, Kobayahi Y. Trends in Hospitalizations forAsthma During the COVID-19 Outbreak in Japan. The Journal of Allergy and Clinical
10
Immunology: In Practice. 2020;9(1):494-6.e1.

To be continued

K. Abe, 2.7.202411