# Design issues adumbrate conclusions on LED-mediated bycatch risk reduction of cetaceans and turtles in fishing nets: A comment on Bielli et al. (2020)

Matthieu Authier, Florence Caurant

<figure><img src="image_1.png" /></figure>

## To cite this version:

Matthieu Authier, Florence Caurant. Design issues adumbrate conclusions on LED-mediated bycatch risk reduction of cetaceans and turtles in fishing nets: A comment on Bielli et al. (2020). Biological Conservation, 2020, 243, pp.108488. 10.1016/j.biocon.2020.108488. hal-02734897

HAL Id: hal-02734897

https://hal.science/hal-02734897v1

Submitted on 5 Aug 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
1 Design issues adumbrate conclusions on LED-mediated bycatch risk reduction of cetaceans

2 and turtles in fishing nets: a comment on Bielli et al. (2020)

3 Author: Matthieu Authier (1, 2), Florence Caurant (1, 3)

4 Email: matthieu.authier@univ-lr.fr

5 Affiliation 1: Observatoire Pelagis, UMS 3462 CNRS - La Rochelle Université, 5 allée de

6 l'Océan, 17 000 La Rochelle, France

7 Affiliation 2: ADERA, 162 avenue Albert Schweitzer CS 60040, 33 608 Pessac Cedex, France

8 Affiliation 3: Centre d'Etudes Biologiques de Chizé, UMR 7372 CRNS - La Rochelle Université,

9 79 360 Villiers en Bois, France

10 Article type: Correspondence (main text: 785 words/800 words max.)

11 Last Updated: 2020-01-22

12 Keywords: causal analysis, design of experiments, bycatch, marine megafauna, covariate

13 overlap

14

15 Acknowledgements

16 MA thanks the French Ministry of Ecology (“Ministère de la Transition Ecologique et

17 Solidaire”) for financial support.
1 Design issues adumbrate conclusions on LED-mediated bycatch risk reduction of cetaceans

2 and turtles in fishing nets: a comment on Bielli *et al.* (2020)

3

4 Article type: Correspondence (main text: 785 words/800 words max.)

5 Last Updated: 2020-01-23
6 Worldwide, bycatch is one of the biggest threat to the long-term viability of cetaceans and 7 turtles (Reeves et al., 2013; Wallace et al., 2013), or marine megafauna in general (Lewison 8 et al., 2004). Reducing marine megafauna bycatch is urgent: Bielli et al. (2020) reported a 9 decreased risk of bycatch for cetaceans and turtles by small-scale fisheries targeting 10 elasmobranchs in Peruvian waters when green visible spectrum light emitting diodes (LEDs) 11 were added on fishing nets. To estimate the causal effect of LEDs on bycatch, some nets had 12 LEDs (treatment) and some nets had none (control). Due to intrinsic difficulties of collecting 13 relevant data in realistic settings while not overburdening fishermen with additional 14 constraints, nets with LEDs were, on average, shorter than nets without. The ratio of effort 15 (in km of net soaked per day) in treatment versus control amounts to 40%. Because of this 16 systematic difference between treatment and control in the experimental design, less 17 bycatch (in absolute numbers) is expected in treatment than in control conditions. This may 18 be taken into account in the statistical analysis of data. However, such a design results in 19 treatment and effort being intrinsically confounded: estimating the causal effect of LEDs on 20 bycatch with these data is problematic as any estimated effect will be model-dependent 21 (Rubin, 2008).

22 Two main modelling choices are possible for analysing these data with generalized linear 23 mixed models. Either use a Bernoulli likelihood (0: no bycatch; 1: at least one individual 24 bycaught) with effort as a covariate to account for the systematic difference between 25 control and treatment; or use a Poisson likelihood with effort as an offset. The two choices 26 are reasonable (since the Poisson distribution can approximate a Bernoulli when the 27 probability parameter is small), but can lead to different conclusions depending on the 28 experimental design for data collection. Simulations can illustrate this point: Poisson- 29 distributed bycatch data were generated in two conditions (treatment versus control)
assuming a rate of 0.05 event per unit effort for both. Hence, the treatment effect was
always nil in these simulations. This bycatch rate implies a probability of no bycatch (for a
unit effort) of $Pr(\text{Bycatch} = 0) = e^{-0.05} \approx 0.951$ and a probability of bycatch
$Pr(\text{Bycatch} = 1) = 1 - e^{-0.05} \approx 0.049$. Effort was assumed to follow a lognormal
distribution of mean 1 and coefficient of variation 0.5 (computed from Table S1 in Bielli et
al., 2020). 10,000 datasets with 850 observations in control and treatment conditions each
were simulated. Crucially, the ratio of effort between treatment and control conditions was
allowed to vary from 0.1 to 1.0 by increment of 0.05. For each ratio, the simulated datasets
were analysed with generalized linear models, assuming a Bernoulli or a Poisson likelihood
(Figure 1).

40 ### insert figure 1 here ###

41 Both the Bernoulli and Poisson models were able to reach the correct conclusions (in the
42 simulations) of no treatment effect when effort was comparable across treatment and
43 control (ratio close to 1). As less effort was allocated to treatment compared to control, the
44 Bernoulli model gave negatively biased estimates of treatment effect, which suggested a
45 decreased risk with the treatment. In contrast, the Poisson model did not suggest any effect.
46 These results illustrate how a design imbalance and a lack of covariate overlap (here effort)
47 prevents an unambiguous assessment of treatment effect (Rubin, 2008). When treatment is
48 confounded with effort, results can be heavily model-dependent because the data do not
49 allow to statistically disentangle a treatment effect from the systematic difference in effort.

50 Because effort was systematically lower for treatment compared to control, the estimated
efficacy of LEDs on bycatch may be very model-dependent: little or no imbalance in effort is
required for a clear assessment of the causal effect of LEDs on cetacean and turtle bycatch.
53 Currently, the effect of LEDs on cetacean and turtle bycatch in fishing nets may have been

54 overestimated. Bycatch mitigation is urgent in many fisheries and requires unambiguous

55 evidence to gain acceptance and maintain trust between scientists and fishermen.
56

**Figure caption**

Figure 1: A – Bias in estimated treatment effect when assuming a Bernoulli likelihood for bycatch data (upper panel). Data were simulated assuming no difference in bycatch between control and treatment conditions, but with a difference in effort due to study design. The bias disappears when the data are analysed with a Poisson likelihood (lower panel). Boxplots represent 10,000 estimated regression coefficients from a generalized linear model that includes both effort (as an offset with a Poisson likelihood) and treatment effects. The dotted red line shows the true effect size for treatment, and the blue dashed line the ratio of effort reported in Bielli *et al*. (2020). B – Probability of selecting with Akaike Information Criterion (AIC) a spurious treatment effect: models with and without treatment effects were compared and the model with the lowest AIC was kept for inference ($\Delta_{AIC} > 2$).

67
68 References

69 Bielli, A., Alfaro-Shigueto, J., Doherty, P. D., Godley, B. J., Ortiz, C., Pasara, A., Wang, J. H.,

70 Mangel, J. C., 2020. An Illuminating Idea to Reduce Bycatch in the Peruvian Small-Scale

71 Gillnet Fishery. Biological Conservation. https://doi.org/10.1016/j.biocon.2019.108277

72 Lewison, R. L., Crowder, L. B., Read, A. J., Freeman, S. A., 2004. Understanding Impacts of

73 Fisheries Bycatch on Marine Megafauna. Trends in Ecology and Evolution, 19 (11), 598-604.

74 https://doi.org/10.1016/j.tree.2004.09.004

75 Reeves, R. R., McClellan, K., Werner, T. B., 2013. Marine Mammal Bycatch in Gillnet and

76 Other Entangling Net Fisheries, 1990 to 2011. Endangered Species Research, 2013, 20, 71-

77 97. https://doi.org/10.3354/esr00481

78 Rubin, D., 2008 For Objective Causal Inference, Design Trumps Analysis. Annals of Applied

79 Statistics, 2, 808--840. https://doi.org/10.1214/08-AOAS187

80 Wallace, B. P., Kot, C. Y., DiMatteo, A. D., Lee, T., Crowder, L. B., Lewison, R. L., 2013. Impacts

81 of Fisheries Bycatch on Marine Turtle Populations Worldwide: Toward Conservation and

82 Research Priorities. Ecosphere 4 (3), 1-49. https://doi.org/10.1890/ES12-00388.1

83
84 Conflict of Interests

85 None

86
87 Supporting Information
88 R code to reproduce analysis and figures
89
## SCRIPT : Simulation to illustrate a design problem in causal inference
90
## Authors :
91
## Last update : 2020-01-23
92
## R version 3.6.1 (2019-07-05) -- "Action of the Toes"
93
## Copyright (C) 2019 The R Foundation for Statistical Computing
94
## Platform: x86_64-w64-mingw32/x64 (64-bit)
95
##
96
97
98 lapply(c("mvtnorm", "ggplot2", "ggthemes", "tidyverse", "cowplot"),
99 library, character_only = TRUE
100 )
101
102 rm(list = ls())
103
104 WorkDir <- getwd()
105 OutDir <- paste(WorkDir, "output", sep = "/")
106
107
### data from Bielli et al 2020: no coefficient of variation available
108
bielli <- data.frame(condition = rep(c("Control", "Treatment"), each = 4 * 3),
109 net = rep(c(14, 156, 141, 54, 84, 43, 231, 77, 1, 7, 41, 15), times = 2),
110 effort = c(1.0, 0.9, 0.8, 1.0, 0.4, 0.5, 0.9, 0.8, 3.0, 3.4, 1.7, 1.5, 0.3, 0.6, 0.5, 0.5,
111 0.2, 0.2, 0.2, 0.2, 0.6, 1.4, 0.5, 0.6),
112 cv = rep(0, 24)
113 )
114
### coefficient
115 effort <- lapply(c("Control", "Treatment"),
116 function(id) {
117 df <- subset(bielli, condition == id)
118 return(do.call('c', lapply(1:nrow(df), function(row_i) { rlnorm(df$net(row_i],
119 log(df$effort(row_i]) - 0.5 * (df$cv(row_i] * df$effort(row_i])^2, sqrt(log1p(df$cv(row_i]^2)))
120 } )))
121 }
122 )
123 ### mean effort in control condition
124 mean(effort[[1]]) # 0.88
125 ### mean effort in treatment condition
126 mean(effort[[2]]) # 0.37
127 ### ratio is
128 round(mean(effort[[2]]) / mean(effort[[1]]), 2) # 0.42
129 ### which is similar to the reported
130 310.0 / 747.7 # 0.41
131 ### coefficient of variation is
132 round(sd(effort[[1]]) / mean(effort[[1]]), 3) # 0.404
133 round(sd(effort[[2]]) / mean(effort[[2]]), 3) # 0.526
134
135 ### have a look at the overlap between effort in control and treatment condition
136 ## assuming a log-normal distribution of effort
137 cv <- 0.5
138 ratio <- 0.4
139 x <- seq(0, 2.5, 0.01)
140
141 df <- data.frame cond = rep(c("Control", "Treatment"), each = length(x)),
142 x = rep(x, 2),
143 y = c(dlnorm(x, log(1.00) - 0.5 * log1p(cv), log1p(cv)),
144 dlnorm(x, log(1.00 * ratio) - 0.5 * log1p(cv), log1p(cv))
145 )
146 )
147 theme_set theme bw())
148 ggplot data = df,
149 aes x = x, y = y, color = cond
150 ) +
151 geom line size = 2) +
152 scale y continuous name = "density") +
153 scale x continuous name = "Effort") +
154 scale color manual name = "", values = c "midnightblue", "seagreen") +
155 theme legend position = "top",
156 plot title = element text (lineheight = 0.8, face = "bold"),
157 axis text = element text size = 10),
158 panel grid minor = element blank(),
159 panel background = element blank()
160 )
161
162 ### clean-up
163 rm effort, x, df, cv, ratio
164
165 ### check the proba Pr X = n for a Poisson distribution of parameter lambda = 0.05
166 dpois x = 2, lambda = 0.05 # 0.001
167 dpois x = 1, lambda = 0.05 # 0.048
168 dpois x = 0, lambda = 0.05 # 0.951
169
170 make data <- function ratio, theta = 0.05, n tot = 850, cv = 0.45) {
171 ### data from Bielli et al. 2020 Biological Conservation
172 ### sample size is approx. 850
173 ### true rate per effort is theta set to 0.05 bycatch event per Unit Effort
174 ### rho is ratio of effort in treatment condition and in control condition
175
176 # effort
177 effort <- data.frame(Treatment = rep(c(0, 1), each = n_tot),
178 Effort = c(rlnorm(n_tot, log(1.00) - 0.5 * log1p(cv), log1p(cv)),
179 rlnorm(n_tot, log(1.00 * ratio) - 0.5 * log1p(cv), log1p(cv))
180 )
181 )
182 # response variable
183 effort$Bycatch <- rpois(2 * n_tot, effort$Effort * theta)
184 # cap at one
185 effort$Bycatch <- ifelse(effort$Bycatch > 1, 1, effort$Bycatch)
186 # wrap-up
187 return(effort)
188 }
189
190 make_analysis <- function(iter, ratio) {
191 # compute aic weights
192 delta <- function(aic) {
193 x <- aic - min(aic)
194 return(exp(-0.5 * x) / sum(exp(-0.5 * x)))
195 }
196 # generate data
197 dd <- make_data(ratio = ratio)
198 # reformat and summarize data
199 raw_data <- dd %>%
200 group_by(Treatment) %>%
201 summarize(SumEffort = sum(Effort),
202 MeanEffort = mean(Effort),
203 SdEffort = sd(Effort),
204 Bycatch_tot = sum(Bycatch)
205 ) %>%

206 as.data.frame()

207 raw_data$simul_id <- iter

208 raw_data$ratio <- ratio

209 # fit models

210 mod0 <- glm(Bycatch ~ 1 + Effort, family = binomial.link = "logit"), data = dd)

211 mod1 <- glm(Bycatch ~ 1 + Effort + Treatment, family = binomial.link = "logit"), data = dd)

212 mod2 <- glm(Bycatch ~ 1 + offset(log(Effort)), family = poisson.link = "log"), data = dd)

213 mod3 <- glm(Bycatch ~ 1 + offset(log(Effort)) + Treatment, family = poisson.link = "log"),
214 data = dd)

215 # summarize and return estimates, AIC etc.

216 return(list theta = data.frame(likelihood = c("poisson", "bernoulli"))[crep(2, 5), rep(1, 3)]),

217 param = c("intercept", "effort", "treatment") [c(1:2, 1:3, 1, 1, 3)],

218 estimate = as.numeric(coef(mod0), coef(mod1), coef(mod2),
219 coef(mod3))),

220 std_error = sqrt(diag(vcov(mod0)), diag(vcov(mod1)), diag(vcov(mod2)),
221 diag(vcov(mod3))),

222 model = c("M0", "M1") [c(1, 1, 2, 2, 2, 1, 2, 2)],

223 simul_id = rep(iter, 8),

224 ratio = rep(ratio, 8)

225 ),

226 akaike = data.frame(likelihood = rep(c("bernoulli", "poisson"), each = 2),

227 ic = c(AIC(mod0), AIC(mod1), AIC(mod2), AIC(mod3)),

228 w = c delta(c(AIC(mod0), AIC(mod1))), delta(c(AIC(mod2), AIC(mod3))),

229 omega = c delta(c(AIC(mod0), AIC(mod1), AIC(mod2), AIC(mod3))),

230 model = rep(c("M0", "M1"), 2),

231 simul_id = rep(iter, 4),

232 ratio = rep(ratio, 4)

233 ),

234 rho = data.frame(likelihood = c("poisson", "bernoulli") [c(rep(2, 4), 1)],
235 param = c("intercept-effort", "intercept-effort", "intercept-treatment",
236 "effort-treatment", "intercept-treatment"),
237 estimate = round(c(as.numeric cor(vcov(mod0)))[2],
238 as.numeric cor(vcov(mod1))[c(2, 3, 6)], as.numeric cor(vcov(mod3))[2]), 3),
239 model = c("M0", "M1":[c(1, 2, 2, 2, 2)],
240 simul_id = rep(iter, 5),
241 ratio = rep(ratio, 5)
242 ),
243 raw_data = raw_data
244 )
245 )
246 }
247
248 ### set seed for reproducibility
249 set.seed(20200107)
250
251 ### do 10,000 Monte Carlo simulations
252 n_sim <- 1e4
253
254 ### store results
255 ## this is a bit long: it took 3 days on a desk computer
256 theta <- akaike <- rho <- raw_data <- NULL
257 system.time(
258 for(j in seq(0.10, 1.0, 0.05)) {
259 for(i in 1:n_sim) {
260 dd <- make_analysis(iter = i, ratio = j)
261 theta <- rbind theta, dd$theta
262 akaike <- rbind akaike, dd$akaike
263 rho <- rbind rho, dd$rho
264 raw_data <- rbind raw_data, dd$raw_data
265 rm(dd)
266 }
267 writeLines(paste("\tPerforming simulations for ratio of ", 100 * j, " %", sep = ""))
268 }
269 )
270 rm(i, j)
271 save.image(paste(OutDir, "Simul_LED.RData", sep = "/"), safe = TRUE)
272
273 ### check that the simulated data is similar to the actual data
274 raw_data %>%
275 group_by(Treatment, ratio) %>%
276 summarize(Effort_tot = mean(SumEffort),
277 Effort_mean = mean(MeanEffort),
278 Bycatch = mean(Bycatch_tot)
279 ) %>%
280 as.data.frame()
281
282 ### some graphics
283 theme_set themeBW()
284 ggplot(data = subset theta, ratio == 0.4),
285 aes(x = param, y = estimate)) +
286 geom_violin() +
287 geom_hline(yintercept = 0.0, color = "tomato", linetype = "dotted", size = 2) +
288 facet_grid(likelihood ~ model) +
289 scale_y continuous(breaks = seq(-10, 10, 1)) +
290 theme(legend position = "top",
291 plot.title = element_text(lineheight = 0.8, face = "bold"),
292 axis.text = element_text(size = 10),
293 panel.grid.minor = element Marginal(),
294 panel-background = element-blank()
295 )
296
297 ### look specifically at treatment effect
298 theme-set theme-bw(base_size = 20))
299 a <- ggplot(data = subset theta, param == "treatment"),
300 aes(x = ratio, y = estimate, group = ratio)
301 ) +
302 geom-boxplot() +
303 geom-hline(y-intercept = 0.0, color = "tomato", linetype = "dotted", size = 2) +
304 geom-vline(x-intercept = 0.4, color = "midnightblue", linetype = "dashed", size = 2, alpha =
305 0.3) +
306 facet-grid(likelihood ~ .) +
307 scale-y-continuous(name = "Estimated\ntreatment effect", breaks = seq(-10, 10, 1)) +
308 scale-x-continuous(name = "Ratio of Effort between\nTreatment and Control",
309 breaks = seq(0.1, 1, 0.1)
310 ) +
311 coord-cartesian(ylim = c(-5, 2)) +
312 theme(legendDashPosition = "top",
313 plotTitle = element-text(lineheight = 0.8, face = "bold"),
314 axis-text.y = element-text(size = 12),
315 axis-text.x = element-text(size = 12, angle = 45),
316 panel.grid.minor = element-blank(),
317 panel-background = element-blank()
318 )
319
320 ### model selection
321 modelSel <- rbind(data.frame(ratio = rep(seq(0.1, 1.0, 0.05), each = 1e4),
322 model = ifelse(subset(subset(akaike, likelihood == "bernoulli"), model ==
323 "M0")$ic -
```r
subset(subset(akaike, likelihood == "bernoulli"), model ==
"M1")$ic >= 2, 1, 0)

) %>%

group_by(ratio) %>%

summarize(prob = mean(model)) %>% # probability to select M1

mutate(likelihood = "bernoulli") %>%

as.data.frame(),

data.frame(ratio = rep(seq(0.1, 1.0, 0.05), each = 1e4),

model = ifelse(subset(subset(akaike, likelihood == "poisson"), model ==
"M0")$ic -

subset(subset(akaike, likelihood == "poisson"), model == "M1")$ic
>= 2, 1, 0)

) %>%

group_by(ratio) %>%

summarize(prob = mean(model)) %>% # probability to select M1

mutate(likelihood = "poisson") %>%

as.data.frame()

)

theme_set themeBW(base_size = 20))

b <- ggplot(data = model_sel,

aes(x = ratio, y = prob)

) +

geom_line(size = 2) +

geom_hline(yintercept = 0.05, color = "tomato", linetype = "dotted", size = 2) +

geom_vline(xintercept = 0.4, color = "midnightblue", linetype = "dashed", size = 2, alpha =
0.3) +

facet_grid(likelihood ~ .) +

scale_y continuous(name = "Probability of selecting\na spurious Treatment effect",

breaks = c(0.05, 0.25, 0.50, 0.75, 1.00),

limits = c(0, 1)
```
355 ) +

356 scale_x continuous(name = "Ratio of Effort between
Treatment and Control",

357 breaks = seq(0.1, 1, 0.1)

358 ) +

359 theme(legendikt = "top",

360 plot. title = element_text(lineheight = 0.8, face = "bold")

361 axis. text. y = element_text(size = 12),

362 axis. text. x = element_text(size = 12, angle = 45),

363 panel. grid.minor = elementDA blank(),

364 panel. background = elementDA blank()

365 )

366 g <- cowplot::plot_grid(a, b, align = "hv", labels = c("A", "B"))

367 g

368 ggsave(paste(OutDir, "Figure1.pdf", sep = "/"), dpi = 600,

369 width = 20, height = 15, units = "cm"

370 )

371 ggsave(paste(OutDir, "Ratio.png", sep = "/"), dpi = 600,

372 width = 20, height = 15, units = "cm"

373 )

374

375 ### correlation

376 rho %>%

377 group_by(likelihood, param, model, ratio) %>%

378 summarize cor = mean(estimate)) %>%

379 as. data. frame()
A

Estimated
treatment effect

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

Ratio of Effort between
Treatment and Control

B

Probability of selecting
a spurious Treatment effect

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

Ratio of Effort between
Treatment and Control