Modeling approaches to estimate 

community annoyance due to 

sonic booms using data from 

repeated surveys

Robyn Ferg

Jean Opsomer

FCSM 2022

1
Introduction

❯ NASA is developing the X-59 aircraft, for researching the effects of lowmagnitude sonic booms (thumps)

❯ Westat is part of the project team, responsible for conducting 

community surveys to estimate community annoyance levels based on 

noise level from X-59 supersonic flights

❯ Use data from previous flight test (QSF18) to develop and apply 

candidate approaches

❯ QSF18 study used different plane and different maneuver, but methods 

still applicable to X-59

2
QSF18 Study

❯ Flights took place over a two-week period in November 2018 in 

Galveston, Texas

❯ 52 sonic thumps delivered over the test period

❯ Noise monitors used to measure sound levels across the survey area

3
QSF18 Data

❯ 5634 total observations from 373 participants

❯ Each observation has an estimated noise level (PL) based on the 

participant’s location at the time of the flight

4
QSF18 Data

❯ Annoyance is measured on a five-point scale:

• Not at all annoyed

• Slightly annoyed

• Moderately annoyed

• Very annoyed 

• Extremely annoyed

❯ Top two categories are considered ‘highly annoyed’

❯ Challenge: only 1.1% of responses were highly annoyed

5
QSF18 Data

❯ Demographic data

• Gender

• Age

• Household size

• Kids under 6

• Geographic quadrant

• Education

❯ No weights included in the data set

❯ Since X-59 study will include weights calibrated to Census Bureau totals, 

simulate weights for QSF18 data ~unif(50, 250)

6
Quadrants

7
Multilevel Logistic Regression

❯ Multilevel logistic regression model with participant-level random effects 

is a common method of creating a dose-response curve for this type of 

data

❯ Model log-odds of high annoyance (HA) based on noise level PLij and 

covariates Xj, with random effects for each participant j:

HAij~Binom 1, pij

pij = logit

−1 β0 + β1 ∗ PLij + β2Xj + uj =

1

1 + exp[− β0 + β1 ∗ PLij + β2Xj + uj]

uj~N(0, σ

2

)

❯ For demonstration, use gender and geographical quadrant as covariates

❯ Note: Unweighted, so resulting curve will not be representative of the 

population

8
Sample of 100 Participant-Level Dose-Response Curves

9
Participant-Level Intercepts

10
Proposed Two-Stage Model

❯ Break up modeling into two stages

❯ Stage 1: Model probability of ever being highly annoyed given demographics 

of participant j

• Logistic regression:

p

1 Xj = logit−1 β0 + β1Xj =

1

1 + exp − β0 + β1Xj

❯ Stage 2: Model probability of high annoyance given demographics of 

participant j and noise level PLij given participant j was ever highly annoyed

• Multilevel logistic regression:

p

2 PLij, Xj = logit−1 α0 + α1Xj + α2PLij + uj =

1

1 + exp − α0 + α1Xj + α2PLij + uj

11
2nd Stage Participant-Level Intercepts

12
Marginal Dose-Response Curve

❯ To accurately account for the participant-level random effects in 

predicting the dose-response curve for the population, we integrate over 

the distribution of random effects (Pavlou et al. 2015)

❯ For a given PL noise level, the predicted fraction of people highly 

annoyed among those ever highly annoyed with covariates X is

pƸ

2

(PL, X) = න

−∞

∞

logit

−1

( β෠

0 + β෠1 ∗ PL + β෠2X + u) f(u) du

where f(u) is a normal distribution with mean 0 and variance equal to 

the estimated random effect variance

13
2nd-Stage Marginal Poststrata Curves

14

Quadrant A Quadrants B&C Quadrant D

Male

Female
Multilevel Regression and Poststratification

❯ To generalize the dose-response curve to the population, we use 

Multilevel Regression and Poststratification (MRP) (Gelman & Little, 

1997)

❯ Using the fitted multilevel regression model, create marginal doseresponse curves for each poststratum

❯ Create a weighted average of these curves based on the proportion of 

the population within each poststratum

❯ Let X

g be the gth combination of categorical variables in the study, 

where g = 1, ... , G and N = σg=1

G Ng. Then the estimated probability of high 

annoyance for the population is

pƸ PL =

1

N

෍g=1

G

NgpƸ

1 Xg pƸ2 PL, Xg

15
Population Dose-Response Curve

16
Inference

❯ Confidence intervals and standard errors can be estimated through 

bootstrapping

❯ Take 100 samples with replacement from the data

❯ Repeat the exact same process of fitting a multilevel logistic regression 

model and use MRP to estimate the study region population-level curve 

for the subsample

❯ 100 resulting study region population-level curves

❯ At each PL level, take the standard deviation of the 100 curves

17
Population-Level Dose Response Curve with 95% Confidence Interval

18
Summary

❯ Two-Stage Model

❯ Multilevel regression and Poststratification (MRP)

❯ Bootstrap inference

❯ Not included, but possible extensions:

• Multilevel ordinal regression or monotone regression model

• Bayesian hierarchical model to account for covariate (noise level) 

uncertainty

19