Proceedings of the

32nd International
Workshopon
Statistical Modelling
Volume II

Groningen, Netherlands
3-7 July, 2017
Editors: Marco Grzegorczyk, Giacomo Ceoldo

iiProceedings of the 32nd International Workshop on Statistical Modelling
Volume II,
Groningen, July 3-7, 2017,
Marco Grzegorczyk, Giacomo Ceoldo (editors),
Groningen, 2017.

Editors:
Marco Grzegorczyk, m.a.grzegorczyk@rug.nl
University of Groningen
Nijenborgh 99747 AG Groningen
The Netherlands

Giacomo Ceoldo, g.ceoldo@student.rug.nl
University of Groningen
Nijenborgh 99747 AG Groningen
The Netherlands

iii

Preface
Dear Participants,
First of all, I would like to welcome you to the 32nd International Workshop on
Statistical Modelling (IWSM 2017) in the Netherlands, and I wish you a very
pleasant stay in Groningen. With around 200,000 inhabitants Groningen is the
7th largest town in the Netherlands, and Groningen has two large universities:
The Hanzehogeschool for Applied Sciences (about 20,000 students) and the Rijksuniversiteit Groningen (RUG) with about 30,000 students. The venue of the
IWSM 2017 is the Academy Building of RUG and located in the town center ofGroningen.
Before you lies one of the two proceedings volumes of IWSM 2017. It is a uniquefeature within the statistical community that all speakers at this workshop also
provide an extended abstract of their talk. This not only provides the participants
with a compact written account of interesting contributions, but it also improves
the quality of the talks.
Like every year, there was a huge amount of excellent paper submissions, andit was a really challenging task to select from 138 abstracts 56 (41%) for oralpresentations. Each paper had to be reviewed and scored by three members of
the scientific programme committee. This was a very time-consuming task for thereviewers, and for their valuable efforts I thank all members of the scientific committee: Ernst Wit (RUG), Marijtje van Duijn (RUG), Kenan Matawie (IWSM
2005), Arnošt Komárek (IWSM 2012), Vito Muggeo (IWSM 2013), Thomas
Kneib (IWSM 2014), Helga Wagner (IWSM 2015), Jean-François Dupuy (IWSM
2016), Simon Wood (IWSM 2017), Paul Eilers, John Hinde, Dirk Husmeier, Sonja
Greven, Edwin van den Heuvel, Jörg Rahnenführer and Korbinian Strimmer.
Some of the abstracts that could not be selected for oral presentation have been
given the opportunity to be presented on a poster. On Tuesday (17:15-19:30h)
there will be a poster presentation where everybody is invited to meet the researchers and to discuss their ongoing work one-on-one. It will be taken care ofdrinks and some ‘fingerfood’ in order not to be distracted by a thirsty throat
or a hungry belly. Although it would have been possible to extend the numberof presentations, it is an important feature of the IWSM workshops that there
are no parallel sessions. This means that each presentation, whether by a PhD
student or by a famous statistician, are awarded the same amount of attention.
This means that the IWSM is a very coherent meeting, whereby the emphasis lies
precisely on the word: ‘meeting’. It is a place where junior and senior researchers
mix and mingle.
Not coincidently, also this year you will get ample opportunities to meet yourfellow participants. On Sunday evening, after the excellent short course by Tom
Snijders and before the official start of the conference on Monday, there wasalready the informal welcome drink gathering in the Pool Restaurant of the Student Hotel. On Monday evening, there will be the official welcome reception in
the Spiegelzaal of the Academy Building. On Tuesday, the Pizza & Beer Poster
session in the Academia Restaurant of the Academy Building will encourage lively

iv
scientific interactions. On Wednesday, after the excursions, we will reconvene altogether in the Ni Hao restaurant (Gedempte Kattendiep 122). On Thursday
evening, if you are hungry again, there will be the official conference dinner in‘t Feithhuis (Martinikerkhof 10). Even the conference dinner will be kept quite
informal, as the emphasis should be on meeting your fellow participants. It would
be great if, long after this conference is over, you could look back on IWSM 2017
and say: ‘Groningen was the place where I met ’em all! ’.
I thank the Statistical Modelling Society for trusting in my proposal and for giving me this great opportunity to chair IWSM 2017. Many colleagues from RUG
helped me planing and realizing this workshop. I would like to thank all membersof my Local Organizing Committee, in particular, Ernst Wit, Ineke Schelhaas,
Martijn Wieling, Casper Albers, Wendy Post, Marijtje van Duijn and Hans Burgerhof for their contributions. Also Mariska Pater and Sharon de Puijselaar fromthe Groningen Congres Bureau have helped me tremendously.
My special acknowledgements go to all sponsors of the IWSM 2017. Without
those sponsorships certain things could not have been realized and the programme
would certainly have been much sparser. A list of the sponsors of IWSM 2017
can be found on the last pages of this volume.
Last but not least, I would like to thank all authors for the excellent scientificcontributions, and I hope that every participant of IWSM 2017 will have a greatand especially research-stimulating week in Groningen,
Marco Grzegorczyk
Groningen, 16 June 2017

Scientific committee• Chair: Marco Grzegorczyk (Groningen University, Netherlands)• Ernst Wit (Groningen University, Netherlands)• Marijtje van Duijn (Groningen University, Netherlands)• Arnošt Komárek (Charles University Prague, Czech Republic)• Dirk Husmeier (Glasgow University, Scotland)• Edwin van den Heuvel (Eindhoven University, Netherlands)• Helga Wagner (Linz University, Austria)• Jean-François Dupuy (INSA of Rennes, France)• John Hinde (NUI Galway, Ireland)• Jörg Rahnenfüehrer (TU Dortmund University, Germany)• Kenan Matawie (University of Western Sydney, Australia)• Korbinian Strimmer (Imperial College London, England)• Paul Eilers (Erasmus Rotterdam, Netherlands)• Simon Wood (Bath University, England)• Sonja Greven (LMU Munich, Germany)• Thomas Kneib (Goettingen University, Germany)• Vito Muggeo (Palermo University, Italy)

vi

Local Organizing Committee• Chair: Marco Grzegorczyk• Ernst Wit• Ineke Schelhaas• Martijn Wieling• Casper Albers• Wendy Post
• Marijtje van Duijn• Hans Burgerhof• Sacha la Bastide• Christine zu Eulenburg• Mark Huisman• Ruud Koning• Mariska Pater• Laura Spierdijk• Christian Steglich• Marieke Timmerman

Contents
Part I – Invited Papers
Roland Langrock: Nonparametric inference in hidden Markov and related models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Laura M. Sangalli: Functional Data Analysis, Spatial Data Analysis
and Partial Differential Equations: A fruitful union. . . . . . . . . . . . . . . . . . .

3
4Jelle Goeman, Aldo Solari: Minimal Adequate Models: Assessing Uncertainty in Variable Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

Emmanuel Lesaffre, Kris Bogaerts, Arnošt Komárek: Why bothering about Interval Censoring? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

10

R. Harald Baayen, Tino Sering, Cyrus Shaoul, Petar Milin: Language comprehension as a multiple label classification problem . . . . . . .

21Part II – Contributed Papers
Mason Chen: Predict NBA 2016-2017 Regular Season MVP Winner . . . .
Mason Chen, Charles Chen: Detect Exam Cheating Pattern on
Multiple-Choice Exams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35
39

R. Bruyndonckx, N. Hens, M. Aerts: Simulation-based evaluation of
the impact of singletons on the fixed effects in a linear mixed model . .

43Eugen Pircalabelu, Gerda Claeskens, Lourens J. Waldorp: Topdown joint graphical lasso. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47

Marcelo Bourguignon: Modelling time series of counts with deflation
or inflation of zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

51

Autcha Araveeporn: An Estimated Parameter of Local Polynomial Regression on Time Series Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

55

Artur J. Lemonte, Jorge L. Bazán: New links for binary regressions:
An application to the coca cultivation in Peru. . . . . . . . . . . . . . . . . . . . . . . .

59

Aluı́sio Pinheiro, Eufrásio Lima Neto: Wavelet regression and generalized linear models with continuous distributions . . . . . . . . . . . . . . . . . . . .

63

Rafael A. Moral, John Hinde, Clarice G. B. Demétrio: Bivariate
residual plots with simulation polygons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

67

Antonio Hermes Marques da Silva Junior: Gradient test for variancecomponent models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

71

viii

Contents

Carlo G. Camarda: Constrained Mortality Forecast . . . . . . . . . . . . . . . . . . .

75

Enrico A. Colosimo, Frederico M. Almeida, Vincius D. Mayrink:
Prior Specifications to Handle Monotone Likelihood in the Cox Regression Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

79

Jennifer Brown, Blair Robertson, Trent McDonald: Spatially Balanced Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

83

Fraser Tough: A comparison of wind power forecast models . . . . . . . . . . .

87

Luiz R. Nakamura, Pedro H.R. Cerqueira, Thiago G. Ramires, Rodrigo R. Pescim, Robert A. Rigby, Dimitrios M. Stasinopoulos:
Beyond regression: what does really affect a football team on championships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

91

Tiago M. de Carvalho , Ardo van den Hout , Jonathan Miles ,Stephen Duffy , Nora Pashayan : A Continuous Time Multi-Statemodel for Prostate Cancer Screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

95

Hildete Pinheiro, Rafael Maia, Daniel Barboza: Multinomial andDiscrete survival models to evaluate the performance of students . . . . .

99

Rosaria Simone, Gerhard Tutz, Maria Iannario: A Random EffectMixture Model for Repeated Measurements of ordinal variables . . . . . .

103

Michael Richard, Christophe Hurlin, Jérôme Collet: A Generic
Method for Density Forecast Recalibration . . . . . . . . . . . . . . . . . . . . . . . . . . .

107,

Susan Fennell, Michael Quayle, James P. Gleeson , Kevin Burke:
Analysis of social interaction data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

111

Marco Geraci, Bo Cai: Nonlinear quantile regression with clustereddata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

115

Michela Eugenia Pasetto, Dirk Husmeier, Umberto Noè, Alessandra Luati: Statistical Inference in the Duffing System with the Unscented Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

119

Liciana Vaz de Arruda Silveira, Alberto Osella, Maria del Pilar
Diaz: Effect of Health Risk Factors on All-Causes Mortality in Elderly
People: A Joined Survival Analysis using Flexible Parametric Modelsof Cohorts coming from Brazil, Argentina and Italy.. . . . . . . . . . . . . . . . . .

123

Gialuca Sottile and Giada Adelfio: A new approach for clustering ofeffects in quantile regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

127

Ivana Malá: Social and material deprivation in the population aged over
50 in the Czech Republic in 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

131

Roberto F. Manghi, Francisco J. A. Cysneiros, Gilberto A.
Paula: On generalized additive partial linear models for correlated
data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

135

Contentsix

Anne-Françoise Donneau, Cibele Russo, Nadia Dardenne,
Murielle Mauer, Corneel Coens, Andrew Bottomley, Emmanuel Lesaffre: What is the best approach to analyze longitudinalbounded scores? Application to Quality of Life data . . . . . . . . . . . . . . . . .

139

Marcos Antonio Alves Pereira, Cibele Maria Russo: Nonlinear
mixed-effects models with scale mixtures of skew-normal distributions

142

Alexander Bauer, Fabian Scheipl, Helmut Küchenhoff, AliceAgnes Gabriel: Modeling spatio-temporal earthquake dynamics using
generalized functional additive regression. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

146

Gladys D.C. Barriga, Vicente G. Cancho, Gauss M. Cordeiro,
Edwin M. M. Ortega, Michael W. Kattan : A Destructive Series
Power Cure Rate Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

150

Trung Dung Tran, Joke Duyck, Geert Verbeke, Emmanuel Lesaffre: Latent vector autoregressive models: Application to evaluate evolution of oral health and general health . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

154

Maren Vens, Jördis Stolpmann: Multivariable fractional polynomials
for correlated dependent variables using generalized estimating equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

158

Carla Azevedo, Dankmar Böhning, Mark Arnold, Antonello
Maruotti: An application of mixture models to estimate the size of
Salmonella infected flock data through the EM algorithm using validation information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

162

Cristian Villegas, Emmanuel Lesaffre, Vı́ctor Espejo: Retrospective estimation of growth of Chilean common hake based on the otolithmeasurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

166

Somsri Banditvilai, Chammaree Chubuathong: Forecasting Qualityof Hard Disk Drive with Logistic Regression and Neural Networks . . . .

169

Arbi Setiawan, Ni Lessari: Growing Bigger and More Accurate with
GSBPM (part one) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

173

Shahla Faisal, Gerhard Tutz : Imputation in High-dimensional MixedType data by Nearest Neighbors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

177

Timo Adam, Vianey Leos-Barajas, Roland Langrock, Floris M.
van Beest: Using hierarchical hidden Markov models for joint inference at multiple temporal scales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

181

Vicente Garibay Cancho, Daiane de Souza, Josemar Rodrigues,
N. Balakrishnan : Bayesian cure rate models induced by frailty insurvival analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

185

Bernardi Mauro, Ruli Erlis: Robust Generalised Autoregressive Score
Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

189

x

Contents

Claudia Di Caterina, Giuliana Cortese, Nicola Sartori: Monte
Carlo modified profile likelihood in survival models for clustered censored data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

193

Kirsten Eilertson, Matthew Ferrari, John Fricks: Using StateSpace models to track Measles Infections . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

197

M. ZAGORŠČAK, A. BLEJEC, Ž. RAMŠAK, M. PETEK, K. GRUDEN : DiNAR: web application for Differential Network Analysis andvisualisation in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

201

Euloge Clovis Kenne Pagui, Edoardo Michielon, Alessandra Salvan, Nicola Sartori : Median unbiased estimator for the twoparameter logistic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

203

Duncan McGregor, Andrew Bell, Bruce J. Worton: Modelling andcategorisation of seismic waves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

207

Abdolreza Mohammadi, Ernst C. Wit : Bayesian Modelling of theExponential Random Graph Models with Non-observed Networks . . .

211

Adrian Quintero, Emmanuel Lesaffre, Geert Verbeke, Benedict
Orindi, Luk Bruyneel: Analysis of differential effects in multi-group
regression methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

215

Victor Bernal , Corry A. Brandsma, Alen Faiz, Victor Guryev,
Wim Timens, Maarten van den Berge, Rainer Bischoff, Marco
Grzegorczyk , Peter Horvatovich : Gene regulatory network reconstruction with prior knowledge over mRNA data for COPD patientsand controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

219

Priyantha Wijayatunga: Resolving the Lord’s Paradox . . . . . . . . . . . . . . .

223

Mário F. Desousa, Helton Saulo, Manoel Santos-Neto, Vı́ctor
Leiva: On a mixture regression model for left-censored data . . . . . . . . .

227

Kouji Tahata: On asymmetry models and decompositions of symmetry
for square contingency tables with ordered categories. . . . . . . . . . . . . . . . .

231

M. Regis, N. Nooraee, A. Brini, E. R. van den Heuvel: Analysis of
the t linear mixed model and evaluation of a direct estimation method

235

Diego Ayma, Dae-Jin Lee, Marı́a Durbán: Combining information at
different scales in spatial epidemiology: a Composite Link Generalized
Additive Model approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

239

Marius Ötting, Christian Deutscher, Roland Langrock: Detecting
Match-Fixing in the Italian Serie B Using Flexible Regression . . . . . . . .

243

Lucia Benetazzo, Ernst Wit: Copula VAR models . . . . . . . . . . . . . . . . . . .

247

Kanae Takahashi, Kouji Yamamoto: An exact test for comparing two
predictive values in small-size clinical trials . . . . . . . . . . . . . . . . . . . . . . . . . .

251

Contents

xi

Takuma Ishihara, Kouji Yamamoto: Comparison of Two Testing Procedures in Clinical Trials with Multiple Binary Endpoints . . . . . . . . . . . .

254

L. Mihaela Paun, M. Umar Qureshi, Mitchel Colebank, Mansoor
A. Haider, Mette S. Olufsen, Nicholas A. Hill, Dirk Husmeier:Parameter Inference in the Pulmonary Blood Circulation of Mice . . . .

258

Toivo Glatz, Wim Tops, Natasha Maurits, Ben Maassen: Exploring brain-behaviour interactions during auditory oddball detection withgeneralized additive modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

262

Parviz Nasiri: Interval Shrinkage Estimation Parameter of Burr type III
Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

266

Adriaan van der Graaf: Fitting binomial mixtures to RNA-seq data tofind discrete gene regulation mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

270

Hung Chu : Modelling weekly temperatures in Eelde using Bayesian Structural Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

274

Thomas Nijman: A comparison between GMLVQ and Random Forests .

278

D.A. Langbroek: Polynomial and spline regression . . . . . . . . . . . . . . . . . . . . .

282

Part I – Invited Papers

Nonparametric inference in hidden Markov
and related models
Roland Langrock1

Department of Business Administration and Economics, Bielefeld University,
Germany

E-mail for correspondence: roland.langrock@uni-bielefeld.de
Abstract: Hidden Markov models (HMMs) have been successfully applied in various disciplines, including biology, speech recognition, economics/finance, climatology, psychology and medicine. They combine immense flexibility with relative
mathematical simplicity and computational tractability, and as a consequence
have become increasingly popular as general-purpose models for time series data.
In this talk, I will first introduce the basic HMM machinery and showcase thepractical application of HMMs using intuitive examples. I will then demonstrate
how the HMM machinery can be combined with penalized splines to allow for
flexible nonparametric inference in general-purpose HMM-type classes of models. The focus of the presentation will lie on practical aspects of nonparametricmodelling in these frameworks, with the methods being illustrated in economicand ecological real data examples, featuring, inter alia, the famous wild haggis
animal, blue whales and the well-known Lydia Pinkham sales data.
Keywords: Animal behaviour; Markov-switching regression; P-splines
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Functional Data Analysis, Spatial Data
Analysis and Partial Differential Equations:
A fruitful union
Laura M. Sangalli
1

MOX - Dipartimento di Matematica, Politecnico di Milano, Italy

E-mail for correspondence: laura.sangalli@polimi.it
Abstract: I will discuss an innovative class of regularized regression models
for the analysis of spatially distributed data, that merges advanced statisticaland numerical analysis techniques. Based on these regression models, I will thenpresent a principal component analysis method that can handle functional signals
distributed over complex domains.
Keywords: Penalized regression; functional principal component analysis; data
distributed over two-dimensional manifold domains; finite elements.

1

Spatial regression with differential regularization

I will present a novel class of models for the analysis of spatially (or space-time)
distributed data, based on the idea of regression with differential regularizations.
The models merge statistical methodology, specifically from functional data analysis, and advanced numerical analysis techniques. Thanks to the combination ofpotentialities from these different scientific areas, the proposed method has important advantages with respect to classical spatial data analysis techniques. Spatial
regression with differential regularizations is able to efficiently deal with data distributed over irregularly shaped domains, with complex boundaries, strong concavities and interior holes [Sangalli et al. (2013)]. Moreover, it can comply withspecific conditions at the boundaries of the problem domain [Sangalli et al. (2013),
Azzimonti et al. (2014, 2015)], which is fundamental in many applications to obtain meaningful estimates. The proposed models have the capacity to incorporate
problem-specific priori information about the spatial structure of the phenomenon
under study, formalized in terms of a governing partial differential equation [Azzimonti et al. (2014, 2015)]; this very flexible modeling of space-variation allows
to naturally account for anisotropy and non-stationarity. Space-varying covariate information is accounted for via a semiparametric framework. The models
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Sangalli

5

can also be extended to space-time data [Bernardi et al. (2017)]. Furthermore,
spatial regression with differential regularizations can deal with data scatteredover non-planar domains, specifically over two-dimensional Riemannian manifold
domains, including surface domains with non-trivial geometries [Ettinger et al.
(2016), Dassi et al. (2015), Wilhelm et al. (2016)]. This has fascinating applications in the earth-sciences, life-sciences and engineering. The use of advanced
numerical analysis techniques, and in particular of the finite element method orof isogeometric analysis, makes the models computationally very efficient. The
models are implemented in the R package fdaPDE [Lila et al. (2016)].

2

Smooth principal component analysis for functionalsignals over complex domains

Based on the regularized regression models outlined above, I will present a regularized method for principal component analysis of functional signals observed
over two-dimensional Riemannian manifold domains [Lila et al. (2016)]. This
will be illustrated with an application in the neurosciences, studying neuronalconnectivity on the cerebral cortex, starting from functional magnetic resonanceimaging scans on about 500 healthy volunteers.

FIGURE 1. Study of high-dimensional neuroimaging signals (dataavailable
from
The
Human
Connectome
Project
Consortium,
www.humanconnectomeproject.org). Left: Triangulated surface approximating the left hemisphere of a cerebral cortex. Right: functional connectivity
map obtained from fMRI signal. Figure adapted from Lila et al. (2016).

Acknowledgments: This talk is based on joint works with John A.D. Aston,
Laura Azzimonti, Mara Bernardi, Bree Ettinger, Michelle Carey, Eardi Lila, Fabio
Nobile, Simona Perotto, Jim Ramsay, Piercesare Secchi.

References
Azzimonti, L., Nobile, F., Sangalli, L.M., Secchi, P. (2014). Mixed Finite Elements
for spatial regression with PDE penalization. SIAM/ASA Journal on Uncertainty Quantification, 2, 1, pp. 305 – 335.

6

Functional Data Analysis, Spatial Data Analysis and Partial Differential Equations

Azzimonti, L., Sangalli, L.M., Secchi, P., Domanin, M., Nobile, F. (2015). Bloodflow velocity field estimation via spatial regression with PDE penalization.
Journal of the American Statistical Association, Theory and Methods, 110,
511, pp. 1057 – 1071.
Bernardi, M.S., Sangalli, L.M., Mazza, G., Ramsay, J.O. (2017). A penalized regression model for spatial functional data with application to the analysisof the production of waste in Venice province. Stochastic Environmental
Research and Risk Assessment, 31, 1, pp. 23 – 38.
Dassi, F., Ettinger, B., Perotto, S., Sangalli, L.M. (2015). A mesh simplification
strategy for a spatial regression analysis over the cortical surface of thebrain. Applied Numerical Mathematics, 90, 1, pp. 111 – 131.
Ettinger, B., Perotto, S., Sangalli, L.M. (2015). Spatial regression models overtwo-dimensional manifolds. Biometrika, 103, 1, pp. 71 – 88.
Lila, E., Aston, J.A.D., Sangalli, L.M. (2016). Smooth Principal Component Analysis over two-dimensional manifolds with an application to Neuroimaging.
Annals of Applied Statistics, 10, 4, pp. 1854 – 1879.
Lila, E., Sangalli, L.M., Ramsay, J.O., Formaggia, L. (2016). fdaPDE: functional
data analysis and Partial Differential Equations; statistical analysis of functional and spatial data, based on regression with partial differential regularizations, R package version 0.1-4,, http://CRAN.R-project.org/package=fdaPDE.
Sangalli, L.M., Ramsay, J.O., Ramsay, T.O. (2013). Spatial spline regression models. Journal of the Royal Statistical Society Ser. B, Statistical Methodology,
75, 4, pp. 681 – 703.
Wilhelm, M., Dede’, L., Sangalli, L.M., Wilhelm, P. (2016). IGS: an IsoGeometric approach for Smoothing on surfaces. Computer Methods in Applied Mechanics and Engineering, 302, pp. 70 – 89.
Wilhelm, M., Sangalli, L.M. (2016). Generalized Spatial Regression with Differential Regularization. Journal of Statistical Computation and Simulation,
86, 13, pp. 2497 – 2518.

Minimal Adequate Models: Assessing
Uncertainty in Variable Selection
Jelle Goeman1 , Aldo Solari2
1
2

Leiden University Medical Center, The Netherlands
University of Milano-Bicocca, Italy

E-mail for correspondence: j.j.goeman@lumc.nl
Abstract: We present an alternative approach to variable selection that does
not select a single “best” model but attempts to find a collection of models
that is “good enough”. We call models adequate if they are not significantly
worse than the true model. The collection of all adequate models is spanned bya smaller collection of minimal adequate models: the smallest of the adequate
models. These minimal adequate models give great insight in model selection
uncertainty as well as in collinearity, and are therefore a very practical modelbuilding tool. We illustrate the approach with several classical data sets.
Keywords: Model selection; Closed testing; Model misspecification.

1

Variable selection

The goal of variable selection methods in regression is to discard a subset of thecovariates without reducing the predictive potential of the remaining variables.
Typical variable selection methods find a single “best” model according to achosen criterion, e.g. AIC or BIC. Variable selection is done to reduce overfit but
also for reasons of interpretation. Selected variables are interpreted as important,and discarded variables as irrelevant. Different criteria and different methods,
however, can yield very different selected models. Especially when collinearity ispresent, variable selection methods tend to differ greatly both in which variables
are selected and how many.
Clearly, there is uncertainty about the selected model. If we see the single selectedmodel as a point estimate of the true model, then we can say that typical variableselection methods neglect to give standard errors or confidence intervals aroundthe statements they make. Interpreting the selected model in terms of importantand irrelevant variables is like interpreting point estimates without an associatedmeasure of uncertainty.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

8Minimal Adequate Models

We present a different approach to variable selection that, in contrast, emphasizes
the uncertainty in the variable selection process. We adopt a hypothesis testing
framework to construct a confidence interval around the true model. Thus, weselect not a single model, but a range of models. A model is in the confidence
interval if its likelihood is not significantly worse than that of the true model. By
construction, the confidence interval contains the true model with probability atleast 1 − α.
Our work builds upon work laid out by Mallows (1973), Aitkon (1974) andSpjotvoll (1977). We show that their work can be seen as a special case of closed
testing, which allows their results to be extended outside the scope of linear
models e.g. to generalized linear models.

2

Minimal adequate models

The construction of the confidence intervals will be such that if a model is in
the confidence set all supersets of the model are also in the confidence set. The
confidence set is therefore spanned by its smallest members, the minimal adequate
models.
The minimal adequate models give great insight in the reliability of inferential
statements made as a result of variable selection methods. They can be used to
distinguish between variables that must always be selected by a variable selection
method and variables that can take each other’s roles in the model because they
contain the same information, e.g. because of collinearity.
For example, two minimal adequate models can be {A, B} and {A, C, D}. In thiscase covariate A is necessary for any adequate model, but the role of B can betaken over by the combination of C and D, which together contain the same
information. A user may have a preference for model {A, B} because it is moresparse or for {A, C, D} because it may have a better fit or be more interpretable.
In either case the presence of the other minimal adequate model functions as aprotection against overinterpretation of the selected model.

3

Model misspecification

A confidence interval for the true model supposes the existence of the true model,
which in turn implies that the full model is true. Since this is quite a strong
assumption, we will investigate how to relax it. We do this by refining the null
hypothesis to be tested for each model: instead of testing whether the reduced
model is as good as the true model, we test whether it is as good as the fullmodel. We show that this hypothesis can be conservatively tested even when thefull model is not the true model.

4

Application

The use of minimal adequate models will be illustrated with several classicalregression data sets, such as Hald’s cement data and the famous prostate cancer
data set (Hastie et al. 2001).

Goeman and Solari

9

References
Aitkin, M.A. (1974). Simultaneous inference and the choice of variable subsets
in multiple regression. Technometrics, 16, 221 – 227.
Hastie, T., Tibshirani, R., and Fiedman, J. (2001). The Elements of Statistical
Learning. New York: Springer.
Mallows, E. (1973). Some comments on CP . Technometrics, 15, 661 – 675.
Spjotvoll, E. (1977). Alternatives to plotting CP in multiple regression. Biometrika,
64, 1 – 8.

Why bothering about Interval Censoring?
Emmanuel Lesaffre1 , Kris Bogaerts1 , Arnošt Komárek2
1
2

I-BioStat, KU Leuven, Leuven and UHasselt, Hasselt, Belgium
Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic

E-mail for correspondence: emmanuel.lesaffre@kuleuven.be
Abstract: We consider here methods to analyze interval-censored survival times.
Interval censoring occurs when it is only known that the event happened inbetween two examinations. Well-known examples of an interval-censored time
are the time until HIV, AIDS, the emergence of a tooth, etc. Most often interval
censoring is not appropriately addressed in a statistical analysis and dealt withby methods that handle right censoring of data, e.g. by replacing the intervalby the mid-point. Despite several published results it is still too often believedthat ignoring the interval-censored character of the data has a minimal impact
on the results and conclusions of the statistical analysis. In this contribution wesummarize the literature on interval censoring largely from a practical point ofview under the frequentist and a Bayesian paradigm. It will be also discussed
when it is important to take interval censoring into account.
Keywords: Bayesian inference; Interval censoring; Survival analysis.

1

Introduction

In survival studies, right censoring is most prevalent and generally dealt withappropriately. Occasionally also left censoring occurs, but in randomized controlled trials and epidemiological studies interval censoring occurs frequently.
Left censoring occurs, e.g., in a dental study on emergence of permanent teeth
when a tooth emerged prior to the start of the study. An emergence time is theninterval-censored when it is only known that the tooth emerged in-between two
examinations. Interval censoring also occurs often in HIV/AIDS studies, wheretime to HIV seroconversion and AIDS are usually determined at planned visits to the clinical researcher. In fact many developments on interval censoring
find their origin in HIV/AIDS research. Finally, in cancer trials progression freesurvival can only be established in the hospital at planned visits. Despite thefrequent occurrence of interval censoring, this interval censoring is often treatedThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Lesaffre et al.

11

inappropriately in practice. Note that left and right censoring can be consideredas special cases of interval censoring. Also, in practice most often a mix of thecensoring types is encountered.
Often interval censoring is bypassed with a single imputation technique, with
mid-point imputation being most popular. That is, while the interval-censoredsurvival time is replaced by the mid-point of the interval, the data are analyzed
using methods for right-censored data. The effect of inappropriately dealing withinterval censoring depends on the size of the intervals, on whether covariates
impact the size of the interval and the type of statistical analysis. In the past,absence of statistical software was the main reason for avoiding interval censoring. This is not the case anymore nowadays. This is clearly illustrated in theforthcoming book on interval censoring (Bogaerts, Komárek and Lesaffre, 2017),
hereafter referred to as BKL.
One distinguishes case I interval-censored data, also called current status data.
This occurs in practice when it is only known whether the event has happend
or not at the time of examination. We concentrate in this contribution on case
II interval censoring. Namely, we assume that an independent sample of survival
times T1 , . . . , Tn is only observed to lie in intervals bli , ui c (i = 1, . . . , n), where
bli means that either li is included or not in the interval and the same for ui . By
allowing li to be zero, interval censoring reduces to left censoring. On the other
hand right censoring is a special case of interval censoring when ui = ∞ (in practice this implies a large value). Further we assume that the censoring mechanism
is independent of the true survival times. We will return to this assumption in
Section 5.
A popular data set in the statistical literature on interval-censored data comesfrom a breast cancer study. It consists of the subset of 96 patients who weretreated at the Joint Center for Radiation Therapy in Boston between 1976 and1980. Forty-six patients were randomized to radiation therapy only regimen, while
48 patients to the radiation therapy and adjuvant chemotherapy regimen. Theintervals represent the time period during which breast retraction occurred. A
graphical representation of the data is shown in Figure 1. Illustrations will also betaken from the Signal Tandmobielr study, which is a longitudinal dental study
that examined, e.g., the emergence distributions of several permanent teeth.

2

Univariate models

2.1

Frequentist approaches

Let the true survival times T1 , . . . , Tn be i.i.d. with survival distribution S(·).
When the survival times are interval-censored with intervals bli , ui c (i = 1, . . . , n),
the likelihood to maximize is:L=nY{S(li ) − S(ui )},

(1)

i=1

with S(t) the unknown but true survival distribution. Peto (1973) was the first
to note that the nonparametric maximum likelihood solution of S results in a setof intervals {[pj , qj ]}m
j=1 with the following properties: outside these intervals, theestimated survival function is constant. Further, the mass assigned to each of theintervals is well determined but within each interval there is no information as12Interval Censoring

TABLE 1. Breast cancer study. Regions of possible support and NPMLE equivalence classes for the radiotherapy-only group.
(pj , qj ]mass

(4, 5]
0.046(6, 7]
0.033(pj , qj ]mass

(25, 26]
0(7, 8]
0.089(33, 34]
0.082(11, 12]0.071(34, 35]
0(15, 16]0(36, 37]
0(17, 18]
0(38, 40]
0.121(24, 25]0.093(40, 44]0(46, 48]
0.466

to how that mass is assigned. The intervals are called regions of possible mass or
support because the maximum likelihood procedure can only tell in which regions
there is probability of events to occur. Peto (1973) and Turnbull (1976) suggested
a simple reduction algorithm to identify the intervals of possible mass from thedata. Further, Turnbull (1976) suggested the self-consistency algorithm, a version of the EM algorithm, to determine the nonparametric maximum likelihood
estimator (NPMLE) of S. Thus, in contrast to the Kaplan-Meier estimator, the
NPMLE of the survival function for interval-censored data has no closed solution
and must be obtained by an iterative algorithm.
Two versions of the NPMLE are given in Figure 2 obtained from the patientstreated with radiotherapy alone in the breast cancer study. The left panel is the
NPMLE of the cumulative distribution function of the time to cosmetic deterioration of the breast. Fourteen regions of possible support were found but only
to eight regions mass > 0 has been attributed. In Table 1, these intervals areshown. The gray areas indicate that the distribution of probability within theregions of support is not determined. In the right panel the corresponding estimated survival distribution is given but assuming a linear behavior of Sb in theintervals.
Since the seminal papers of Peto and Turnbull, the classical significance tests inRadiotherapy onlyRadiotherapy + Chemotherapy

50

Number of patients

40

30

20

10

0
0

20

40

60

0

20

40

60

Time to breast retraction (months)

FIGURE 1. Breast cancer study. Observed intervals in months for time to breast
retraction of early breast cancer patients per treatment group.

Lesaffre et al.

13

survival analysis to compare two or more groups (logrank test, Gehan-Wilcoxon
test, Peto-Prentice-Wilcoxon test, etc.) have been extended to interval-censoredobservations.
Because for a long time Turnbull’s algorithm was not available in statistical software, it was standard to show the Kaplan-Meier estimate based on singly imputed
survival times. Alternatively and depending on the application area, also a parametric estimate was computed. In medical applications the most popular choices
are the Weibull and the log-normal distribution. Computations and inference
are simpler in the parametric case relying often on Newton-Raphson type of algorithms and standard asymptotic likelihood theory. In-between nonparametricand parametric approaches are flexible estimation methods. Numerous techniques
have been proposed that either smooth the hazard, the cumulative hazard or thesurvival distribution. Popular in this sense is spline smoothing based on cubic
splines, B-splines or penalized B-splines adapted to interval-censored data. A
smooth solution can also be obtained from, say, a mixture of Gaussian densities
for the survival density. Examples of these approaches with software applicationsin R and SAS software can be found in BKL.

2.2

Bayesian approaches

1.0
0.8
0.6
0.0

0.2

0.4

^
S(t)

0.6
0.4
0.0

0.2

^
F(t)

0.8

1.0

Parametric analysis of interval-censored observations is fairly standard in classical Bayesian statistical software, such as Win/OpenBUGS or SAS, as long as
the chosen survival distribution is supported by the package. More complicatedis to perform a Bayesian nonparametric (BNP) analysis. BNP estimation of acumulative distribution function (and thus of the survival distribution) started
with the seminal paper of Ferguson (1973), who introduced the Dirichlet process0

10 20 30 40 50
Time0

10 20 30 40 50
Time

FIGURE 2. NPMLE of the cumulative distribution function (left panel) and
NPMLE of the survival function with the additional assumption of a piecewise
linear survival curve (right panel).

14Interval Censoring(DP) prior. The DP prior D(cS ∗ ) is a prior on the survival distributions S definedaround a guess survival distribution S ∗ with variability ruled by a scalar c. Basedon a DP prior, Susarla and Van Ryzin (1976) proposed a nonparametric Bayesian
approach to estimate the survival function for right-censored survival times. Calle
and Gómez (2001) further extended the procedure to interval-censored data. Limiting cases of the posteriors are the Kaplan-Meier for right-censored observations
and Turnbull’s estimate for interval-censored observations. Until recently no generally available software was available for the BNP approach, this changed with
the R package DPpackage (Jara, 2007). In the supplementary materials of BKL,
some self-written R programs can be found for fitting survival distributions ina nonparametric way as well as illustrations of the use of DPpackage.

3

Regression models

Of more interest are survival models that allow for covariates, say X1 , . . . , Xn .
In that case the likelihood becomes:
L=nY{S(li | Xi ) − S(ui | Xi )}.

(2)

i=1

While for right-censored data the Cox proportional hazards (PH) model takes acentral position because the partial likelihood approach renders the estimation of
the baseline hazard obsolete, with interval-censored survival times the baseline
hazard/distribution needs to be estimated together with the regression coefficients. We consider here the PH model and the accelerated failure time (AFT)model for interval-censored survival times. Again a variety of approaches weresuggested from semiparametric to parametric.

3.1

Frequentist approaches

The likelihood to maximize for the PH model is given byL(β, S0 ) =

n n
Y

o
>
>
S0 (li )exp(Xi β) − S0 (ui )exp(Xi β) ,

(3)

i=1

with β a p-vector of regression parameters and S0 (t) the baseline survival function.
Finkelstein (1986) extended the nonparametric approach of Turnbull to the proportional hazards model with interval-censored data. For this she assumed themodel expressed in (3). Note that likelihood (3) depends only on the baseline hazard through its values at the different observation time points. Let s0 = 0 < s1 <. . . < sK+1 = ∞ denote the ordered distinct time points of all observed time intervals bli , ui c (i = 1, . . . , n). Further, let αij = I{sj ∈ bli , ui c} (j = 1, . . . , K +1,
i = 1, . . . , n). To remove the range restrictions on the parameters for S0 , the likelihood is parameterized by γk = log[− log S0 (sk )] (k = 1, . . . , K + 1). Note that
because S0 (s0 ) = 1 and S0 (sK+1 ) = 0, γ0 = −∞ and γK+1 = ∞. In terms of βand γ = (γ1 , . . . , γK )> the log-likelihood function `(β, S0 ) can be written as
(K+1
)
n
h
i
X
X
−ζk−1 exp(Xi> β)−ζk exp(Xi> β)`(β, γ) =
log
αik e
−e
,
(4)
i=1

k=1

Lesaffre et al.

15

1.0

^
h(t)

0.15

^
f (t)

0.20

0.25

1.5

0.30

0.35

P
where ζk = km=0 exp(γm ).
To estimate the mass of the regions of possible support and the regression parameters, Finkelstein proposed the Newton-Raphson algorithm. It turned out thatthe score equations provide a generalization of the self-consistency algorithm
suggested by Turnbull when β = 0. For the appropriateness of the asymptotic
χ2 -distribution for testing β = 0, it is assumed that K does not increase with thesample size. Farrington’s approach (1996) allows to fit the approach of Finkelsteinwith generalized linear model software. He also provided a technique to select a
subset of L < K significant time points sl (l = 1, . . . , L). Other approaches suggested for the PH model are: the piecewise exponential model (available in SAS
procedure ICPHREG) and a variety of smooth approaches some based on splinesmoothing.
Another popular semiparametric approach is to apply the partial likelihood approach on multiple imputed (MI) data sets. In the MI approach finite intervalcensored survival times are regarded as missing and replaced by a possible survival
time given an assumed model.
Standard methodology canthen be used to analyze
Density
Hazardthe (often between 3 and 10) imputed data sets of right-censored survival times.
The results of the multiple analyses are then combined. Pan (2000) proposed twosuch multiple imputation schemes assuming a particular distribution within theregions of support, but no other assumptions are made.
Finally, here again a parametric approach is easiest to handle, but could be too
restrictive in practice.
0.5

0.10

For the AFT model, the true survival times T1 , . . . , Tn are assumed to satisfy(i = 1, . . . , n),

(5)

0.0

0.00

0.05

Yi = log(Ti ) = Xi> β + εi

where εi are independent and identically distributed with density g(e). In caseof interval-censoredtimes
6 survival
78
9 10
11 12the13 likelihood
6
7 is 8given
9 10 by11 (2)
12 13but now withS(t | X) = S0 exp(−X > β)t.
Age (years)
Age (years)

0.8
0.6
0.4
0.2
0.0

Proportion not yet emerged

1.0

Survival function

6

7

8

9

10

11

12

13

Age (years)

FIGURE 3. Signal Tandmobielr study (boys). Estimated survival function compared to NPMLE (dashed line) for the time to emergence of tooth 44 estimatedusing the penalized Gaussian mixture with R package smoothSurv.

16

Interval Censoring

Several distributions have been suggested for g, but there exists no semiparametric version of the AFT model. While the parametric AFT model is the simplest tohandle it is often too restrictive. More flexible approaches are based on a smootherror density. One option is the Penalized Gaussian Mixture (PGM) model, whichassumes that the error density is a mixture of a (large) number of Gaussian densities with fixed means (knots) and with weights that are constrained by a penalty
term to produce a smooth density. This approach has been implemented in the R
package smoothSurv. In Figure 3 the solution from smoothSurv is comparedto the NPMLE for the emergence distribution of tooth 44 from boys.

3.2

Bayesian approaches

For a long time, the Bayesian PH model could only be fit parametrically tointerval-censored observations making use of the statistical packages Win/OpenBUGS
and later with the SAS procedure MCMC. Unfortunately, a pure semiparametric approach does not seem to be possible here, but recently at least two flexible
modelling approaches have been proposed and implemented in software. Wanget al. (2013) proposed a Bayesian PH model with a piecewise constant baseline
hazard via a reversible jump MCMC procedure in combination with data augmentation. Their approach fits a dynamic survival model to the data, thereby
providing a check for the PH assumption. The method is implemented in the
R package dynsurv. The recently developed R package ICBayes is based on
fitting the baseline hazard in a smooth manner using integrated I-splines, see Lin
et al. (2015). To this end the relationship of the PH model with a latent nonhomogeneous Poisson process was used in combination with data augmentation.

0.6
0.4

Boy, DMF 0
Boy, DMF 1
Girl, DMF 0
Girl, DMF 1

0.0

0.2

Survival distribution

0.8

1.0

The parametric Bayesian AFT model can be fitted with BUGS-like and SAS software in very much the same manner as the PH model. We are only aware of

6

7

8

9

10

11

12

13

Age at emergence tooth 44 (years)

FIGURE 4. Signal Tandmobielr study. Survival functions for emergence of permanent tooth 44 in gender by DMF groups based on a semiparametric Bayesianmodel with MDP priors, obtained with R function DPsurvint.

Lesaffre et al.

17

the R package bayesSurv to fit a smooth AFT model in a Bayesian way. The
package is based on the reversible jump MCMC technique, but also a PGM as
an error distribution can be fitted. With the package DPpackage practitioners
have several programs at their disposal for fitting Bayesian AFT models in asemiparametric manner. In the package a Mixture of Dirichlet process prior isused to fit an AFT model for interval-censored observations, which is the basis
for the R function DPsurvint. This function was applied to examine the dependence of the emergence distribution of a permanent tooth (tooth 44) on genderof a child and the history of caries status of the predecessor deciduous tooth
84 expressed by its dichotomized DMF score (DMF=1, caries on the deciduous
tooth, 0 otherwise). Figure 4 shows the posterior predictive survival function forthe different gender and DMF combinations.

4

Multivariate models

When several outcomes are measured on a subject who is examined at regulartime intervals, we obtain multivariate interval-censored observations. With multivariate outcomes, it is natural to ask for the association between the outcomes.
Most of developments have been done for the bivariate case, i.e., when there are
two related survival times T1 and T2 measured in an interval censored manner.
A special case of bivariate interval-censored data are doubly interval-censoredtimes. In that case the T1 measures the onset of the time-at-risk and T2 ≥ T1
measures the time of the event, and again both T1 and T2 are interval-censored.

4.1

Frequentist approaches

Betensky and Finkelstein (1999a) generalized Peto’s and Turnbull’s argument tobivariate interval censored data. That is, information on the bivariate nonparametric survival function is limited to a number of rectangles bearing (possibly)
non-zero mass, again called the regions of possible support. The trigger to develop
the bivariate NPMLE of S for interval-censored observations, was the computation of the association between the two true survival times. However, it turned
out that the bivariate NPMLE is not a good basis for this because too dependent
on the amount right censoring in the data, see Betensky and Finkelstein (1999b).
The absence of statistical software for fitting a rich class of (multi/bi)variate
models (for interval-censored data), restricts the use of parametric modelling for
multivariate responses. Instead one could use copula models, which disentangle the specification of the association structure and the marginal distributions.
The three popular copulas: the Clayton copula, the Gaussian copula and the
Plackett copula have been extended to bivariate interval-censored survival times
and are implemented in the function fit.copula from the R package icensBKL(will accompany the BKL book). Even more flexible are the bivariate smoothing
techniques, such as the bivariate PGM model implemented in the SAS macro%smooth. This macro produced Figure 5 that shows a smooth approximationof the distribution of the true emergence times of the contralateral (left andright) maxillary first premolars (teeth 14 and 24) for boys, collected in the Signal
Tandmobielr study. One can observe that the two emergence times are highly
correlated.

18Interval Censoring

For survival outcomes the association measures Spearman’s rank correlation,
Kendall’s tau and the global and local cross-ratio are in use. These measures
can be estimated by plugging in sample values in the population versions of theassociations. This can be done for parametric models, but when based on the
PGM approach a goodness-of-fit check for the parametric models is obtained.
To graphically explore the association structure of multivariate observations one
can use a biplot. On a biplot the original p-dimensional outcome is projected onto
2 (or 3) dimensions displaying individuals as points and variables as vectors. If the
2-dimensional plot captures most of the original variability, then the projections of
the points on the vectors provide useful visual information on the characteristicsof (groups of) individuals. The biplot has been extended to multivariate intervalcensored observations (Cecere et al., 2013) and implemented in the function ICBiplot of the package icensBKL.
Hierarchical models, called frailty models in the survival context, provide yet
another way to model multivariate interval-censored outcomes. Conditional ona random intercept, the outcomes are then assumed independent. This class ofmodels has been also extended to the interval-censored case. Again various illustrations of methodologies and software can be found in BKL.

4.2

Bayesian approaches

Parametric frailty models can be fit with standard Bayesian software such Win/OpenBUGS
and the SAS procedure MCMC. More challenging is to fit multivariate models
for interval-censored data in a semiparametric manner. A few approaches havebeen suggested to fit the frailty distribution in a flexible manner. The approachof Komárek and Lesaffre (2007) builds on the penalized Gaussian mixture idea.
Let (Ti1 , . . . , Tini )> be independent random vectors representing times-to-eventof the i-th cluster which are observed as intervals blil , uil c and Xil be the covariate vector for the lth observation in the ith cluster (i = 1, . . . , n; , l = 1, . . . , ni ).

f
0.2

0.1

0.1

0.0
14.9
13.5
12.1
10.8
Emergence time of tooth 14 (yrs)
9.4
8.1
6.7 6.7

8.1

9.4

14.9
13.5
12.1
10.8
Emergence time of tooth 24 (yrs)

FIGURE 5. Signal Tandmobielr study. Density of penalized normal mixture
model for emergence of permanent teeth 14 and 24 obtained from SAS using
macro %smooth.

Lesaffre et al.

19

In the random-effects AFT model, the (i, l)-th event time is expressed aslog(Til ) = Xil> β + bi + εil(i = 1, . . . , n; l = 1, . . . , ni ),(6)where εil are (univariately) i.i.d. random errors with a density gε and b1 , . . . , bn
are cluster-specific i.i.d. random-effects with a density gb . The approach thenconsists in expressing either of densities gε and gb as a univariate PGM. The
approach was implemented in the R package bayesSurv and illustrated withdata from the Signal Tandmobielr study. More specifically the software was used
to examine the impact of caries (now or in the past) of deciduous teeth and theirsuccessors. Another option is to use the package DPpackage, which provides
functions that allow for a multivariate semiparametric approach.

5

Discussion

The list of statistical approaches extended to deal with interval-censored data isendless. In fact, each statistical approach developed for fully observed or rightcensored data can be extended to interval-censored data. Additional topics thathave been investigated with interval-censored data: competing risks, multi-statemodels, interval-censored covariates, etc. We also omitted here the discussion of
doubly interval-censored observations, important for HIV/AIDS research.
Finally, the majority of the developments (if not all) have been done under the assumption of non-informative independent censoring. This assumption is violated
when the censoring intervals are associated with the actual and unobserved timeto-event. This may happen more often in practice than assumed, and may affectthe conclusions considerably. Developments that deal with informative censoring
are therefore desirable.
To conclude, there is no reason anymore to bypass interval censoring since there
is ample software available for a great variety of problems.

References
Betensky, R. A. and Finkelstein, D. M. (1999a). A non-parametric maximum likelihood estimator for bivariate interval censored data. Statistics in Medicine,18, 3089 – 3100.
Betensky, R. A. and Finkelstein, D. M. (1999b). An extension of Kendall’s coefficient of concordance to bivariate interval censored data. Statistics in
Medicine, 18, 3101 – 3109.
Bogaerts, K., Komárek, A. Lesaffre, E. (2017). Survival Analysis with IntervalCensored Data: A Practical Approach with examples in R, SAS and BUGS.
Boca Raton: CRC/Chapman and Hall
Calle, M. L. and Gómez, G. (2001). Nonparametric Bayesian estimation frominterval-censored data using Monte Carlo methods. Journal of Statistical
Planning and Inference, 98, 73 – 87.
Cecere, S. and Groenen, P. J. F. and Lesaffre, E. (2013). The interval-censoredbiplot. Journal of Computational and Graphical Statistics, 22, 123 – 134.
Jara, A. (2007). Applied Bayesian non- and semiparametric inference using DPpackage. R News, 7, 17 – 26.
Farrington, C. P. (1996). Interval censored survival data: A generalized linear
modelling approach. Statistics in Medicine, 15, 283 – 292.

20Interval Censoring

Ferguson, T.S. (1973). A Bayesian analysis of some nonparametric problems. The
Annals of Statistics, 1, 209 – 230.
Finkelstein, D. M. (1986). A proportional hazards model for interval-censoredfailure time data. Biometrics, 42, 845 – 854.
Komárek, A. and Lesaffre, E. (2007). Bayesian accelerated failure time model forcorrelated censored data with a normal mixture as an error distribution.
Statistica Sinica, 17, 549 – 569.
Lin, X. and Cai, B. and Wang, L. and Zhang, Z. (2015). A proportional hazards model for interval-censored failure time data. Lifetime Data Analysis,
21, 470 – 490.
Pan, W. (2000). A multiple imputation approach to Cox regression with intervalcensored data. Biometrics, 56, 199 – 203.
Peto, R. (1973). Experimental survival curves for interval-censored data. Applied
Statistics, 22, 86 – 91.
Susarla, V. and Van Ryzin, J. (1976). Nonparametric Bayesian estimation of survival curves from incomplete observations. Journal of the American Statistical Association, 71, 897 – 902.
Turnbull, B.W. (1976). The empirical distribution function with arbitrarily grouped,censored and truncated data. Journal of the Royal Statistical Society, Series B, 38, 290 – 295.
Wang, X. and Chen, M.-H. and Yan, J. (2013). Bayesian dynamic regression models for interval censored survival data with application to children dental
health. Lifetime Data Analysis, 19, 297 – 316.

Language comprehension as a multiple label
classification problem
R. Harald Baayen1 , Tino Sering1 , Cyrus Shaoul2 , Petar Milin3
1

Eberhard Karls University of Tübingen, Germany
Landmark College, USA
3
University of Sheffield, UK
2E-mail for correspondence: harald.baayen@uni-tuebingen.de
Abstract: The initial stage of language comprehension is a multi-label classification problem. Listeners or readers, presented with an utterance, need to discriminate between the intended words and the tens of thousands of other words
they know. We propose to address this problem by pairing a network trained withthe learning rule of Rescorla and Wagner (1972) with a second network trained
independently with the learning rule of Widrow and Hoff (1960). The first network has to recover from sublexical input features the meanings encoded in thelanguage signal, resulting in a vector of activations over all meanings. The second
network takes this vector as input and further reduces uncertainty about the intended meanings. Classification performance for a lexicon with 52,000 entries isgood. The model also correctly predicts several aspects of human language comprehension. By rejecting the traditional linguistic assumption that language is a(de)compositional system, and by instead espousing a discriminative approach(Ramscar, 2013), a more parsimonious yet highly effective functional characterization of the initial stage of language comprehension is obtained.
Keywords: multi-label classification, language comprehension, error-driven learning, Rescorla-Wagner, Widrow-Hoff
Table 1 presents 10 simple sentences. When reading these sentences, the lettersand their combinations succeed in bringing to the fore a small number meanings
while dismissing thousands of others as irrelevant. Sentences present the reader
with a multi-label classification problem.
We address this problem as follows. First, we represent the orthographic input bymeans of letter trigrams. For the first sentence, these are #Ma Mar ary ry# y#p#pa pas ass sse sed ed# d#a #aw awa way ay#(the#symbol represents the space character). Letter trigrams provide a much richer
representation of the visual input than do orthographic words. For the data in
Table 1, there are n = 104 distinct letter trigrams, to which we refer as cues.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

22Language as multi-label classification

The second column lists the lexical meanings (lexomes) that are the targets ofclassification. Lexomes are pointers to locations in a high-dimensional semantic
vector space (defined below). Note that past-tense word forms such as passed(regular) and ate (irregular) are coupled with the lexomes pass and eat as wellas with past tense (past). Likewise, the two word forms apple and pie are coupledwith one lexome applepie, and the three expressions with the word forms kicked
the bucket, passed away, and died, are all linked with the same lexome die.
TABLE 1. Sentences, lexomes in the message, and frequency of occurrence (F ).
The total number of learning events is k = 771.

1
2
3
4
5
6
7
8
9
10

Sentence

Lexomes in the message

Mary passed away
Bill kicked the ballJohn kicked the ball away
Mary died
Mary bought clothes
for the ball
Ann bought a ballJohn filled the bucket
John kicked the bucket
Bill ate the apple pie
Ann tasted an apple

mary die pastbill kick past def balljohn kick past def ball away
mary die pastmary buy past clothes
for danceparty
ann buy past indef balljohn fill past def bucket
john die pastbill eat def applepie
ann taste past indef apple

F
40
100
120
300
20
45
100
10
3
33

Is it possible to discriminate between the targeted lexomes given the letter trigrams in the sentences? We will show that considerable headway can be made by
an error-driven incremental multi-label classifier that comprises two simple networks, each with only an input layer and an output layer. In what follows, we firstprovide a formal definition of the algorithm, and illustrate it for the sentences inTable 1. We then turn to a more realistic example in which lexomes targeted inaround a million of utterances have to be discriminated from some 52,000 otherlexomes.

1

An algorithm for multiple label classification

The problem of incremental learning of multi-label classification is defined by asequence of events at which a set of features (henceforth cues) are present andgenerate predictions about classes (henceforth outcomes), only some of which are
actually present in the learning event. The mismatch between predicted outcomes
and the outcomes actually present in a learning event provides the error driving
learning.
From a total of n distinct cues and m possible outcomes, only small subsets will
be present in a given learning event. Let k denote the number of unique learning
events (learning events may repeat, cf. good morning and tickets please). We index
a specific learning event in the sequence t (of length K ≥ k) of learning events by
t. The classification problem is defined by t, a sparse n × k cue matrix C whichis 1 whenever a given cue is present in a specific event and zero otherwise, and asparse m × k target matrix T that is 1 whenever an outcome is present and zerootherwise.

Baayen et al.

23Classification proceeds in two steps, using two networks. The first network hascues as inputs and outcomes as outputs. It is defined by an m × n matrix W
of connection weights from cues (columns) to outcomes (rows). Given W, thepredicted support (henceforth activation) for a specific outcome given the cuesin the learning event is obtained by summation of the weights on the connections
from these cues to that outcome. The m × k activation matrix A specifies these
activations for all outcomes across all unique learning events:
A = WC.
The classification performance of this first network is assessed by checking whether
the outcomes with the highest activations are those of the targeted lexomes.
As shown by Danks (2003), if over a sequence of learning events no further changes
in the weight matrix take place other than the tiny increments and decrementsthat come with individual updates, i.e., when the weight matrix has entered astate of equilibrium, then, given the incremental learning rule of Rescorla andWagner (1972) (see below), W can be estimated straight from conditional probabilities characterizing the input. Let E specify pairwise conditional probabilitiesof cues given cues,
Pr(c0 |c0 ) Pr(c1 |c0 ) . . . Pr(cn |c0 ) Pr(c0 |c1 ) Pr(c1 |c1 ) . . . Pr(cn |c1 ) 
,
E=


...
...
...
...
Pr(c0 |cn ) Pr(c1 |cn ) . . . Pr(cn |cn )and let F denote a matrix specifying conditional probabilities of outcomes given
cues,


Pr(o0 |c0 ) Pr(o1 |c0 ) . . . Pr(on |c0 ) Pr(o0 |c1 ) Pr(o1 |c1 ) . . . Pr(on |c1 ) 
.
F=


...
...
...
...
Pr(o0 |cn ) Pr(o1 |cn ) . . . Pr(on |cn )Danks’ equilibrium equations state thatF = EWT ,which can be solved using the generalized inverse.
When a weight matrix is calculated in this way, the effect of the exact order oflearning events is lost. Furthermore, a Danks weight matrix dampens the consequences of the frequencies of occurrence of cues and outcomes in the input
space, while highlighting the contrasts that allow cues to discriminate between
outcomes. Thus, the Danks weight matrix is useful when there is no informationon the sequence of learning events (e.g., when only the frequency of learning
events is available but not their order) and when interest is directed specificallyto an idealised endstate of learning.
Preferably, the weight matrix W is estimated by repeated application of thelearning rule of Rescorla & Wagner (1972) to the learning events t. The update
at learning event t,
Wt = Wt−1 + ∆rw
depends on the learning rate η (typically set at 0.001) regulating the magnitudeof the changes to the weight matrix, on the predictions for the outcomes asgauged by the activations of these outcomes given the cues, and on whether the24Language as multi-label classification

outcomes are actually present in the learning event. Specifically, let c denote thetranspose of that column vector of C specifying which cues are present at thecurrent learning event t, and let o denote the transpose of that column vector of
T detailing which outcomes are present at t, and let J denote an m × n all-ones
matrix. Let the (row) vector a1 to specify the activations of those outcomes thatare present in the learning event while setting to zero the activations for all otheroutcomes:
a1 = (((J · o)T · c)T · W)i.
Here, i is a row unit vector of length n. Note that ((J · o)T · c)T is 1 for all cueoutcome combinations that are present in the learning event, and zero elsewhere.
Next, let the (row) vector a0 represent the activations of those outcomes notpresent in the learning event, again given the cues in that learning event, and let
it be zero for all other outcomes:
a0 = (((J · [1 − o])T · c)T · W)i.
((J · [1 − o])T · c)T is 1 for all cue-outcome pairs where the cue is present but theoutcome not, and zero elsewhere. The update to the weight matrix, ∆rw , cannow be defined as follows:∆rw = η{((J · o)T · c)T · (1 − a1 ) − ((J · [1 − o])T · c)T · a0 }.
For cue-outcome pairs that are both in the learning event, the update of theirweight is given by the difference from the maximal activation, 1 by definition.
As the summed activations a1 tend to be less than 1, weights will be strengthened. For cue-outcome pairs where the cue is present but the outcome is not,the corresponding connection weight is decreased by the summed activations a0 .
Estimation of W using incremental updating over the sequence of learning events
is fast, first because only parts of the weight matrix require updating (efferent
weights from cues not present in the learning event are left untouched), and also
because the updates to individual outcomes are independent and hence allow forparallelization.
The activation matrix A = WC specifies, for each unique learning event and foreach outcome, the joint support provided by the cues in that learning event forthat outcome.
Although class predictions based on A can do well for small constructed data sets,
they lack precision for large real data sets. Prediction accuracy can be further
improved by a second network that is given the task to predict the target T fromthe activation matrix A:
T = DA.
The prediction matrix
P = DA
is the resulting approximation of T. Although D can be calculated using thegeneralized inverse of A, computation costs can be prohibitive for large numbersof learning events. It is therefore preferable to estimate D as follows:
T

=

DA

TAT

=

DAAT

Y

=

DX,

Baayen et al.

25

which leads to D = YX−1 . Since X is m × m, and since generally m  k, computational costs are much lower when calculating X+ as compared to calculating
A+ .
The prediction matrix can also be estimated iteratively by means of the update
rule of Widrow and Hoff (1960). This update rule, which specifies the update∆wh to the m × m second weight matrix D, is important, first, as it allows us toassess the consequences of how the order of learning events affects classification,and second, because for large numbers of training events (in the order of hundredsof millions), it is not feasible to actually calculate A (and P).
Let Z denote an m × m matrix initialized with zeroes, let a denote the column
vector of the activation matrix A giving the predicted activations for the current
learning event, and let o denote the transpose of the corresponding column vectorof the target matrix T. The Widrow-Hoff update to Z is:∆wh = η{a(o − aT Z)}.
We take the transpose to obtain D = ZT .
The weights for the two networks (m × n for the Rescorla-Wagner network, andm×m for the Widrow-Hoff network) can be estimated in two ways. One possibility
is to first estimate W and then estimate D. Alternatively, one can update both
networks in tandem for each successive learning event. In this case, it is notnecessary to calculate A. Note that when estimating
P = (WC)+ TWC
we ‘inject’ error twice: once during the estimation of W and again during theestimation of P.
The equilibrium equations are implemented in the ndl package for R on cran.
An efficient Python implementation for incremental learning of W is available atgithub.com/quantling/pyndl. An implementation of incremental learning for R
is available (for linux only) upon request from the authors. Software for efficient
updating of D by Widrow-Hoff is currently under development.
Returning to the example of Table 1, first consider classification performance
when W and D are estimated independently, using incremental updating for theformer, and the generalized inverse for the latter. In this case, for each of the 10
sentences, the lexomes in that sentence have the highest prediction values in P.
When the two networks are updated in tandem, with at each learning event firstan update of W and then an update of D, accuracy varies with the (random)order in which the 771 learning events are made available to the model. For one
such random order, the proper lexomes had the highest ranks in A for 9 out of
10 sentences. The one sentence with an error is John kicked the bucket, wheredef (the lexome for the definite article) intrudes with a higher activation before
die, which is found at the next rank (4).
Figure 1 illustrates this incremental training regime. The left and center panels
show the predictions based on A and P when training proceeds on a random
order of 771 learning events, and the right panel when training proceeds on 7710
learning events. Solid lines represent key lexomes from sentence 8 in Table 1: kickand bucket for the unintended literal reading and die for the intended idiomatic
reading. Dashed lines represent the competitors apple and applepie in sentence
9. The spiky behavior in the left and center panels reflects the learning andunlearning that unfolds as outcomes competing for the same cues are encountered.

Language as multi-label classification

200

400

t

600

800

1.0
0.8
0.2
0.0

0.0

0.0
0

KICK
BUCKETDIE
APPLE
APPLEPIE

0.6

prediction

0.6
0.4

prediction

0.8

KICK
BUCKETDIE
APPLE
APPLEPIE

0.2

0.4

0.6

0.8

KICK
BUCKETDIE
APPLE
APPLEPIE

0.2

prediction

P

1.0

P

1.0

A

0.4

26

0

200

400

t

600

800

0

2000

4000

6000

8000

t

FIGURE 1. Prediction strengths for selected lexomes in the learning events ofsentences 8 and 9 in Table 1, using incremented coupled Rescorla-Wagner andWidrow-Hoff. Left and center panels: frequencies as in the table; right panel:frequencies increased tenfold.

Comparison of the left and center panels shows that the Rescorla-Wagner network
learns much faster than the Widrow-Hoff network. By the end of the learning
sequence, the former, but not the latter network succeeds in giving the intended
lexomes higher prediction scores. The rightmost panel shows that with sufficientexperience, the model learns that kick the bucket means die, and that an apple
pie is not an apple but a particular kind of pie.
An important property of this approach to language comprehension is that thecorrect lexomes are selected without any worries about regular or irregular verbs,
literal versus idiomatic expressions, finding boundaries between words, decomposing words into parts, or disambiguating homographs. Given the assumptionthat understanding drives the recalibration of weights, the rich information available in the combinatorics of sublexical cues and lexomes is sufficient for multiple
label classification to be effective.

2

Multiple label classification with 52,000 classes

To clarify whether this approach scales up, we applied our algorithm to the tasa
corpus (Zeno, 1995), a collection of texts comprising a total of 10,807,146 words
representing 52,401 word types. Lemmatization was carried out with TreeTagger(www.cis.uni-muenchen.de/∼schmid/tools/ TreeTagger/), which distinguished
90,339 lemmata, of which 37,938 occurred once. To keep computations tractable,
the model was trained on all words occurring at least twice and 351 hapax legomena that occurred in a precompiled list of words. Hapax legomena that were notincluded were replaced by the dummy word HAPAX, resulting in a total of 52,401
lexomes. Learning events were sentences in the tasa corpus. Sequences of morethan 8 words were split at the next available occurrence of and or or. This resulted
in a total of 992,752 learning events. The multi-label classification challenge is topredict the appropriate lexomes (out of 52,401) given the letter trigrams of the(possibly inflected) words in the learning events.
Using the ndl2 package for R, W (52,401 lexomes × 11,724 letter trigrams)
was estimated using all learning events. To keep computations tractable for theBaayen et al.

27

second network, two learning events were selected randomly from a precompiled
list of 8866 targeted lexomes, resulting in a total of 17,455 learning events (in 276
cases there was overlap with two or more lexomes in the same event, and for one
word, there was only 1 learning event available). The total number of outcomesin this subset of learning events was 19,020. With these restrictions, the matrices
A (19,020 lexomes × 17,455 learning events), D (19,020 × 19,020 lexomes) andP (19,020 lexomes × 17,455 events) could be estimated straightforwardly.

●

0.6
0.4

cumulative proportion●●●●●● ●● ●● ●● ●
●●
●
●
●● ●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●

●

●

rank Arank P

●

●
●

●

0.2

0.8
0.6
0.4

●

0.2

ratio intruders/targets

0.8 1.0

isolated words

1.0

utterances

●

0.0

●
●

0.0

●

●

0.2

●

●

0.4

●

0.6

0.8

deciles 0 through 0.9

1.0

1

10

100

1000

10000

rank

FIGURE 2. Left: Quantiles of the ratio of intruders (false positives) to targets(correct identifications), full utterances. Right: Rank and corresponding cumulative proportion based on A (red) and P (blue), isolated words.
The left panel of Figure 2 presents the ratio of intruders (lexomes with an activation exceeding that of the least activated target lexome) to the number oftargeted lexomes. The median number of intruders is zero, at the 8th decile theratio is 0.17, and at the 9th, it is 0.33. At the 10th decile, we find cases with vast
numbers of intruders, leading to a maximal ratio of 1208.9. Examples of intruders are down for the sentence The aleuts were housed in abandoned rundown gold
mines or fish canneries, and field and success for the sentence He is an ecologist
who studied succession in abandoned cornfields.
We also tested identification performance when target lexomes were presented inisolation. The right panel of Figure 2 plots in blue cumulative proportion (out of a
total of 7179) against rank based on P: 34% of lexomes had the highest prediction
value, 88% of the targeted lexomes had at most a rank of 16 (indicating 15
intruders with higher activations). As show by the red curve, performance basedon A instead of P is substantially worse. Human lexical decision performance, as
gauged using the British Lexicon Project (blp, Keuleers et al. 2012) was for thepresent data at 90% correct. As the lexical decision task does not require actual
identification, but only sufficient evidence for lexicality, it appears that humansubjects tolerate around 16 intruders.
As shown in Figure 3, the model also predicts power-transformed lexical decision
response times (t0 = −1000t−1 ). For all but the first decile, log activation ai =Wci (with c the vector specifying the present and absent cues in the input, andi indexing a specific lexome) shows a nearly linear effect with negative slope. Log
rank prediction (the log rank of pi = DWci ) has a smaller effect that is again
negative and nearly linear, but now for the first nine deciles. The 90% decile of
the rank is at 18, which is close to the cut-off at rank 17 for lexicality decisions in
the right panel of Figure 2. Apparently, the same range of ranks influences bothdecisions and reaction times.

28

Language as multi-label classification

4
2
0
−4

−2

partial effect (−1000/t)

4
2
0
−2
−4

partial effect (−1000/t)6

edf 4.2, p < 0.0001

6

edf 6.9, p < 0.0001

−12

−10

−8

−6

−4

−2

0

0

2

log activation

4

6

8

log P rank

FIGURE 3. Partial effects in a gam fitted to power-transformed (−1000t−1) reaction times. Left: log activation; Right: log prediction rank. Vertical lines denotedeciles. The 90% decile of log prediction rank is at rank 18 (red lines indicate
deciles). Regression analyses were carried out with gams (Wood, 2006).
PT defines a semantic vector space (cf. Landauer & Dumais, 1997), and lexomes
are indices or pointers for locations in this space. By way of illustration of thesemantic nature of PT , the left panel of Figure 4 presents partial effects forhuman semantic similarity ratings for word pairs (Bruni et al., 2014) as predictedfrom correlations of the corresponding column vectors of PT (left). For 90% of
the data points, a nearly linear relation is observed. Clearly, extreme values areunreliable as predictors. Similarity in PT -space, i.e., similar prediction values
across events and thus greater similarity of experiences communicated, correctlypredicts greater perceived semantic similarity.

20
−10

0

10

partial effect

20
10
−10

0

partial effect30

edf = 5.9, p < 0.0001

30

edf = 8.0, p < 0.0001

−0.1

0.0

0.1

0.2

0.3

cor(t(P))

0.4

0.5

0.6

−0.8

−0.6

−0.4

−0.2

0.0

0.2

0.4

0.6

cor(D)FIGURE 4. Partial effects of the correlations of row vectors of P (left) andcolumn vectors of D as predictors of human similarity ratings for 2,369 wordpairs. Red vertical lines indicate 5% and 95% percentiles. Regression analyses
were carried out with gams (Wood, 2006).
The column vectors of D also define a lexomic space, but similarities in this spaceturn out to be positively correlated with the Levenshtein distance between theorthographic forms of the two words. As shown in the right panel of Figure 4, themore different two word forms are, the lower their perceived semantic similarity.

Baayen et al.

329Concluding remarks

Multi-label classification is a hard problem, not only for statistics, but also forhumans. For instance, in auditory word recognition, isolated words taken fromconversational speech have recognition rates between 20% and 40% (Arnold et al.,2017). In the visual lexical decision task, undergraduate students perform nearchance on the lower-frequency words (Baayen et al., 2017). From this perspective,
the model’s performance, with training on a mere 10 million words, is too good
to be true. This is, of course, due to the model being given perfect feedback,whereas human learning tends to proceed under uncertainty and lack of fullunderstanding.
Given that the model presents a simplified perspective on the first stage of comprehension — understanding the words — several of its features are remarkable.
First, the traditional linguistic assumption that language is a (de)compositional
system is replaced by a perspective in which the language signal is a code that
discriminates between possible messages (Ramscar 2013, Shannon, 1956).
Second, the model is parsimonious with only one free parameter, the learning
rate η. And although W and D can be very large, most of the weights are close
to zero. E.g., for W, only 5,885 weights exceed 0.1 (0.00058% of the total number
of weights), and only 195 weights are greater than 0.5. Arnold et al. (2017) show
for auditory comprehension that W can be pruned down to a fraction of theoriginal weights without noticeable loss of accuracy.
Third, the classifier implements a three-layer network that differs from backpropagation networks in that there is direct error injection twice, once for W
using the Rescorla-Wagner equations, and once for D, using Widrow-Hoff (or thegeneralized inverse). Importantly, the power of the first network should not be
underestimated. Although ever since the criticism of the perceptron by Minsky& Papert (1972), two-layer networks have been regarded as far too restricted
for any classification tasks requiring more than the simplest linear separation,
it turns out that actually, with an appropriate choice of cues, Rescorla-Wagner
networks can solve much more interesting problems. Figure 5 illustrates this fora simple example with two classes (represented by gray and red points) that in
R × R are not linearly separable (left panel). When the data are re-represented
by identifiers for rows and columns (right panel), a Rescorla-Wagner network
correctly predicts the highest activations for around 210 of the 260 elements of
the red class (see Baayen and Hendrix, 2017, for detailed comparison with othermachine learning classifiers, and also Ghirlanda, 2005).
Fourth, more sophisticated features than letter trigrams can be used as cues,
such as the frequency band summary features used by Arnold et al. (2017) for
modeling auditory word recognition, and for reading the histogram of oriented
gradients feature descriptor proposed by Dalal and Triggs (2005).
Finally, the model is transparent to interpretation. W specifies the support provided by sublexical features for lexomes. D transforms activation vectors thatare still strongly influenced by form similarity into vectors closer to the targeted
lexomes, which in turn results in a semantic vector space, PT .

References
Arnold, D., Tomaschek, F., Sering, K., Lopez, F., and Baayen, R.H. (2017). Words
from spontaneous conversational speech can be recognized with human-like

30

Language as multi-label classification

FIGURE 5. A non-linearly separable classification problem with a majority classin gray (2240) and a minority class in red (260). Left: data points in a Cartesian
grid (x = 1, 2, . . . , 50; y = 1, 2, . . . , 50). Right: rerepresentation with row andcolumn identifiers as cues for a Rescorla-Wagner network: hits in blue, misses
and false alarms in red.
accuracy by an error-driven learning algorithm that discriminates between
meanings straight from smart acoustic features, bypassing the phoneme asrecognition unit. PLOS-ONE, 12, e0174623.
Baayen, R. H., and Hendrix, P. (2017). Two-layer networks, non-linear separation, and human learning. In Wieling, M., Kroon, M., van Noord, G., andBouma, G. (Eds.) From Semantics to Dialectometry. Festschrift in honorof John Nerbonne. London, College Publications, 13–22.
Baayen, R. H., Tomaschek, F., Gahl, S., and Ramscar, M. (2017). The Ecclesiastes principle in language change. In Hundt, M., Mollin, S., and Pfenninger, S. (Eds.) The changing English language: Psycholinguistic perspectives. Cambridge, Cambridge University Press.
Bruni, E. and Tran, N.K. and Baroni, M. (2014). Multimodal distributional semantics, Journal of Artificial Intelligence Research, 49, 1–47.
Dalal, N. and Triggs, B. (2005). Histograms of oriented gradients for human detection. In CVPR’05, volume 1, 886–893.
Danks, D. (2003). Equilibria of the RescorlaWagner model. Journal of Mathematical Psychology, 47, 109–121.
Ghirlanda, S. (2005). Retrospective revaluation as simple associative learning.
Journal of Experimental Psychology: Animal Behavior Processes, 31, 107–
111.
Keuleers, E. and Lacey, P. and Rastle, K. and Brysbaert, M. (2012). The British
Lexicon Project: Lexical decision data for 28,730 monosyllabic and disyllabic English words, Behavior Research Methods, 44, 287–304.

Baayen et al.

31

Landauer, T.K. and Dumais, S.T. (1997). A Solution to Plato’s Problem: The
Latent Semantic Analysis Theory of Acquisition, Induction and Representation of Knowledge. Psychological Review, 104, 211–240.
Minsky, M. and Papert, S. (1969). Perceptrons: An introduction to computational
geometry. Cambridge, MA: The MIT Press.
Ramscar, M. (2013). Suffixing, prefixing, and the functional order of regularities
in meaningful strings. Psihologija, 46, 377–396.
Rescorla, R. A. and Wagner, A. R. (1972). A theory of pavlovian conditioning:
Variations in the effectiveness of reinforcement and nonreinforcement. In:
A. H. Black and Prokasy, W. F. (Eds.), Classical conditioning II: Current
research and theory. New York: Appleton Century Crofts.
Shannon, C. E. (1956). The bandwagon, IRE Transactions on Information Theory, 2, 3.
Widrow, B. and Hoff, M. E. (1960). Adaptive switching circuits. 1960 WESCON
Convention Record Part IV, 96–104.
Wood, S. (2016). Generalized additive models. New York: Chapman & Hall.
Zeno, S.M. and Ivens, S.H. and Millard, R.T. and Duvvuri, R. (1995). The educator’s word frequency guide. New York: Touchstone Applied Science.

Part II – Contributed Papers

Predict NBA 2016-2017 Regular Season
MVP WinnerMason Chen1
1

Mason Chen Consulting, San Jose, USA

E-mail for correspondence: mason.chen.training@gmail.com
Abstract: This project is to build a statistical model to predict who will win
the 2017 NBA Most Valuable Player (MVP) Award. The “MVP Index” has been
derived from combining each player’s Z statistics with equal weight as a ”uniform”
model. The team has further derived the ”weighted” model by adding the weightfactor which was calculated based on the dispersion/separation between the top
two MVP winners and the remaining players not in top five. Based on the Power
Model, team can improve the Accuracy Index to 70% at power=3. Our final model
would suggest that Westbrook and Harden shall own the Co-MVP Awards.
Keywords: Descriptive Statistics; Z Transformation; Power Model; Discriminant Cluster; Data Mining.

1

Problem Statement and Project Objective

In major professional sports, the coach and team management are looking forways to win more games. Sports statistical modeling analytics (A. Maymin, 2012;
M. Oh, 2015) is becoming a critical approach to uncover the winning patterns
hidden in sports data collected during each game played. The objective of thispaper is to build a statistical model to predict the NBA 2016-2017 regular season
most valuable player (MVP).

2

Data Collection, MVP Index and Model Accuracy
Index

Section 2 has three subsections: (1) Data Collection, (2) Derive MVP Index tojudge player performance, (3) Derive Model Accuracy Index to assess model
performance.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

36Predict NBA MVP Winner

2.1

Data Collection

In order to predict the 2016-2017 season MVP, team has collected three raw
data of the 2015-2016 season from the public Sports domain: (1) top 50 player
statistics in Figure 1 (NBA Player Statistics, 2015-2016), (2) team win%, and (3)historical MVP winners.

FIGURE 1. Top 50 Player Statistics.

2.2

Derive MVP Index

Before building the model, the player statistics have been standardized to the Z
scale in order to remove any mean and standard deviation effect. This Z transformation can eliminate any statistics bias or domination. The Z scale will also
analyze each player’s performance as compared to the other top 50 NBA players
in the same 2015-2016 season. The “MVP Index” has been derived from summingeach player’s Z statistics as shown in Figure 2.

FIGURE 2. Z Transformation and MVP Index.

2.3

Derive Model Accuracy Index

To evaluate the model accuracy, the author has derived “Accuracy Index” ofpredicting the top five MVP players. MVP index of each top player was sorted
and compared to the actual top 5 MVP winners in that season.

3

Build Statistical Modeling Algorithms

Based on the above analysis, author has derived and compared: (1) Uniform, (2)Weighted, (3) Power, and (4) Discriminant Clustering Model.

Chen

3.1

37

Uniform ModelIn the Uniform model, the “MVP Index” has been derived from combining eachplayer’s Z statistics with equal weight. The “Uniform” model can predict the top
five MVP winners at 47% accuracy.

3.2

Weighted Model

The “Weighted” model added the weight factor which reflects player Z scalestatistics categories are more important. To determine the weight level, the descriptive dispersion statistics between the top two MVP winners and the remaining players not in top 5 was calculated for each Z scale statistics. The “Weighted”
model has further improved the Accuracy Index from 47% to 52% shown in Figure 3. The weighted MVP index formula is shown below:
W eighted M V P Index = 1.03 ∗0 GP − N 0 + 0.73 ∗0 M in − N 0 + 1.71 ∗0 F G% −N 0 + 0.61 ∗0 3pt% − N 0 + 0.67 ∗0 F T % − N 0 + 1.17 ∗0 RB − N 0 + 1.66 ∗0 AST −N 0 + 1.5 ∗0 ST L − N 0 + 1.16 ∗0 BLK − N 0 + 2.7 ∗0 P P G − N 0 + 0.89 ∗0 RB/M IN −N 0 + 1.3 ∗0 AST /M IN − N 0 + 0.96 ∗0 A/T − N 0 + 3.13 ∗0 P P G/M IN − N 0

FIGURE 3. Model Accuracy: Uniform model vs. Weighted model.

FIGURE 4. Model Accuracy: compare all three models.

3.3

Power Model

To further optimize the accuracy, author has added the ”Team Winning” factor.
Most historical MVP winners were from the teams with best or better regular season records. Author has assessed the team winning factor based on the38

Predict NBA MVP Winner“Power” model from power= 0 (equivalent to the Weighted Model), 1 to 6 topower= infinity (pick MVP from the best Team). The weighted MVP Index will
be multiplied by the power of the team winning% in the Power model. Based on
the Power Model, team can improve the Accuracy Index to 70% at Power=3. The
Power model has indicated the importance of team winning % on MVP selection
process. There is little benefit but more over-fit risk to further increase the power
level beyond 3 shown in Figure 4.

3.4

Data Mining Discriminant Model

Data Mining Discriminant analysis (J. Garcia, 2013) has been utilized to identify basketball performance indicators in regular season and playoff games. The
Discriminant model accuracy is at 54.5% not better than Power model shown in
Figure 5.

FIGURE 5. Data Mining Discriminant Model.

4Conclusions

Power model has shown the better prediction capability which has indicated theimportance of team winning performance on the MVP selection process. Power=3
model will be utilized to predict 2017 MVP winner on the first day of each monthstarting from Dec 2016,end in April 2017. The final model prediction will suggest
Co-MVP Awards, and presented in IWSM.

References
Maymin, A., Maymin, P., and Shen, E. (2012). NBA chemistry: Positive and negative synergies in basketball. Proceedings MIT Sloan Sports Analytics Conference.
Oh, M., Keshri, S., and Iyengar, G. (2015). Graphical model for basketball match
simulation. Proceedings MIT Sloan Sports Analytics Conference.
Garcia, J., Ibanez, S., Santos, Leite, N., and Sampaio, J. (2015). Journal of Human Kinetics, 36, 2013, 161 – 168 Section II Sports Training.

Detect Exam Cheating Pattern on
Multiple-Choice Exams
Mason Chen1 , Charles Chen21
2

Mason Chen Consulting, United States
Morrill Learning Center, United States

E-mail for correspondence: mason.chen.training@gmail.com
Abstract: This project will demonstrate on how to apply Data Mining Algorithm on detecting the assessment exam cheating pattern in Schools. In order todetect the cheating pattern, JMP multivariate statistics was used to determine
whether there was any association pattern among the students from the same
exam table. Hierarchical Clustering and Dendrogram Tree were used to identify
the grouping affinity behavior related to exam cheating pattern. This cheatingcase study can also be used to study the students answering patterns on any
particular subject to help instructors on designing their future curriculum basedon the Data Mining Patterns.
Keywords: Data Mining; Heat Map; Clustering Analysis; Dendrogram Tree;
Principle Component Analysis.

1

Problem Statement and Project Objective

For each instructor, designing an effective assessment exam is a critical job (KevinY. etc.). Its much easier to grade any exam with questions of multiple choices
than with comprehensive questions. In this case study, there were 75 students
sat in 25 different small tables (3 students per table) in a very limited space.
Instructor has modified the original exam into three different orders (versions A,B, C). Three students from the same table will take different versions. However,students were still smart enough to synchronize the question sequence quickly.
The objective of this paper is to find a data mining algorithm in order to detect
any cheating pattern from the same table.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

402Detect Cheating Pattern

Data Collection and Multivariate Correlation
Analysis

The raw data includes each students ID, Exam Version, Answers, and Table
Number. In order to reduce the computing time, the bottom 25 scorers havebeen excluded from the analysis in this paper. There is unlikely that we can findany cheating pattern from these bottom scorers.

2.1

Multivariate Correlation Analysis

JMP 12 Multivariate Correlation Analysis was used to study any correlation
between Top 50 Students in Figure 1. From the Multivariate Correlation Analysis,
there are Combination(50, 2) = 1, 225 correlation coefficients between any two
students. This massive correlation table is a good start to visualize any correlation
pattern, but not effectively to draw any systematic pattern conclusion.

FIGURE 1. Multivariate Correlation Analysis.

2.2

Sort Students Score

To further detect any cheating pattern, authors have sorted students score (reference column) from top to bottom and list Table information in Figure 2. Its
clear that, on some tables, some students (No.1, No.15, No.17, and No.4) haveobserved same score or similar score.

3

Build Data Mining Algorithms

Authors are looking for Data Mining Algorithms including JMP 12 Hierarchical
Clustering Dendrogram, Heat Map, and Principle Component Analysis to detect
the cheating pattern with high confidence.

Chen

41

FIGURE 2. Sort Score and Exam Table Information.

3.1

Hierarchical Clustering Dendrogram Analysis

Hierarchical Clustering analysis (Michael A.) is utilized to analyze the cheating
pattern. In Figure 3 Dendrogram Tree, JMP software will calculate the answer
affinity among all 1,225 pairs and group the first pair at the strongest affinity(similar pattern). JMP software then find the next affinity pair until all done asshown in Dendrogram. In Figure 4, clustering history has shown 4 out of 5 pairings sat on the same exam table. Hierarchical clustering analysis can defend thecheating pattern at high confidence based on the low pairing distance separated
from the remaining pairing distance shown in Figure 4. Students from Exam
Tables 1, 4, and 15 have been identified having cheating pattern in the exam.

FIGURE 3. Hierarchical Dendrogram Tree.

FIGURE 4. Clustering History.

3.2

Heat Map Analysis

Instead of visualizing the cheating in Figure 3 Correlation Table, Heat Map in
Figure 5 below was conducted to visualize the cheating pattern among the student
identified in previous Dendrogram. Its very clear from the Heap Map, SID (26,35, 44), SID (36, 43) and SID (49, 50) have similar heap map color pattern.

42

Detect Cheating Pattern

FIGURE 5. Heat Map Analysis.

3.3

Principle Component Analysis

Authors also conducted JMP 12 Principle Component Analysis (Soren H.) shownin Figure 6 based Matrix Eigenvalue and Eigenvector algorithm to derive thetwo strongest principle components in a linear combination of all the answering
variable dimensions. Its very clear that SID (26, 35, and 44), SID (36, 43) andSID (49, 50) are assigned in the same region based on two principle components(in X-Y).

FIGURE 6. Principle Component Analysis.

4Conclusions

Authors have utilized various Data Mining Algorithms to detect the exam cheating patterns. The same concept and algorithm can be applied to any other applications to uncover the hidden patterns such as Sports Analytics, Customer
Relational Management, or Biostatistics.

References
Michael R. Anderberg (1973). Cluster analysis for applications, Academic Press,
New York, ISBN 0120576503.
Soren Hojsgaard (2011). Multivariate analysis Principal component analysis (PCA)
Statistics and Decision Theory Research Unit. 1 – 58.

Simulation-based evaluation of the impact of
singletons on the fixed effects in a linear
mixed model
R. Bruyndonckx12 , N. Hens13 , M. Aerts1
1

Interuniversity Institute for Biostatistics and statistical Bioinformatics (I-Biostat),
Hasselt University, Hasselt, Belgium
2
Laboratory of Medical Microbiology, University of Antwerp, Antwerp, Belgium
3
Centre for Health Economic Research and Modelling of Infectious Diseases(CHERMID), Vaccine & Infectious Disease Institute (VAXINFECTIO), University of Antwerp, Antwerp, Belgium
E-mail for correspondence: robin.bruyndonckx@uhasselt.be
Abstract: Data that are collected in medical sciences often have a hierarchicalstructure. Regardless of sparseness caused by the presence of a large numberof small units or a small number of large units, linear mixed models are used
to account for within-unit correlation. Using a simulation study, we assess theimpact of an increasing proportion of singletons (i.e. one subunit in a unit) at thehighest or lowest level of the hierarchy on the fixed effects in a linear mixed model.
Additionally, we assess whether, when high proportions of singletons are present,
performance improves by removing or grouping singletons, splitting singletonsat the highest level or ignoring the dependency within units at the lowest level.
We show that, in the presence of singletons at the lowest level, the model isquite stable. Ignoring clustering and dropping the singletons come with biased
standard error estimates. Grouping the singletons does not improve the model’s
performance. In the presence of singletons at the highest level, the model isunstable. Grouping, dropping and splitting the singletons either decrease type I
error rate or increase power, while worsening the other. The likelihood ratio test
and Wald test perform poorly. The performance of the permutation test howeveris superior to that of the F test. In conclusion, the linear mixed model is stablein the presence of singletons at the lowest level, but care should be taken in thepresence of singletons at the highest level. In that case, the permutation test
rather than the F test should be used to assess the significance of included fixed
effects.
Keywords: F test, hierarchical data, permutation test, sparsenessThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

441Impact of singletons on the linear mixed model

Introduction

Data that are collected in e.g. medical sciences often have a hierarchical structure,with units at a lower level nested within units at a higher level. Most multi-levelsettings consist of a small number of units that tend to be quite large. An example
is seen in the Ceftriaxone data, which contain information on doses (expressedin mg/kg/day) of ceftriaxone prescribed to hospitalized children. Here, some geographical regions contain one country, while all but one country contain morethan ten children. The opposite, where a large number of units tend to be quite
small, occurs less often but is omnipresent in specific fields such as family research. An example can be seen in the Ceftriaxone data, where 47.6% of included
departments contain only one child. Units containing only one subunit (e.g. regions containing only one country or departments containing only one child) arereferred to as singletons. Regardless of sparseness caused by high proportions of
singletons, hierarchical data are generally analysed with linear mixed models.
In this study, we are interested in the impact of different proportions of singletonson the fixed effects in a linear mixed model. We will use simulation studies toassess the impact of an increasing proportion of singletons at the highest level(i.e. countries within regions) and at the lowest level (i.e. children within departments). We will assess whether, when high proportions of singletons are present,
the model’s performance improves by applying some frequently used techniques
to cope with singletons: ignoring the dependency within units, removing singletons, splitting singletons (at highest level) and grouping singletons in an artificial
unit.

2

Methods

2.1

singletons at the lowest level of the hierarchy

A simulation study, focussing on a two-level setting with 350 children divided
over 50 departments and including variables both at the level of the child (ageand reason to treat) and at the level of the department (size), was set up. We
used a low (0.15), realistic (0.27, from the Ceftriaxone data) and high (0.64) intracluster correlation coefficient (ICC). The proportion of singletons ranged from0 to 95% (in steps of 5%), with the number of singletons rounded upwards (e.g.
5% singletons implies 3 departments with one child) and the remaining children
divided equally amongst the remaining departments. For each of the 60 scenarios,
1000 datasets were simulated.
We fitted a linear mixed model, including fixed effects for department size, reason
for treatment and age, and a random effect for department, to all simulated
datasets. Stability of fixed effects estimates and standard errors was evaluatedusing the relative difference between the average parameter estimate (or standard
error) and the true parameter estimate (or standard error) (RDM (or RDE)).
Additionally, we considered the rejection rate of the F tests for age, reason fortreatment and department size. Because some scenarios contained a very low(or very high) proportion of singletons, we considered ignoring within-cluster
correlation, dropping the singletons and grouping them in an artificial unit.

Bruyndonckx et al.

2.2

45

Singletons at the highest level of the hierarchy

A simulation study, focussing on a three-level setting with 240 children divided
over 5 − 40 countries in five regions, was set up. Data were simulated under thenull hypothesis (H0 ), assuming that region has no effect on dose, as well as undera specific alternative hypothesis (HA ). For each of the 18 scenarios, 1000 datasets
were simulated.
We fitted a linear mixed model, including a fixed effect for region and a random
effect for country, to all simulated datasets. The performance of the F test forregion was assessed using the rejection rate, referred to as type I error rate (underH0 ) or corrected power (under HA ). Because one can think of several ways toeliminate data sparseness, we considered dropping the singletons, grouping themin an artificial region and splitting them into two artificial countries. For one
sparse scenario (Scenario 3), we studied the likelihood ratio test, Wald test andpermutation test as alternatives to the F test.

3

Results

3.1

Singletons at the lowest level of the hierarchy

The RDM fluctuated regardless of the proportion of singletons. The RDE wasstable throughout the simulation study. The rejection rate for effects at the levelof the child increased slightly with an increasing proportion of singletons and ICC,
while the rejection rate for the effect at the level of the department decreasedslightly with an increasing proportion of singletons and ICC.
The RDM for both levels was not affected by ignoring clustering and dropping orgrouping singletons. When ignoring clustering, the RDE was overestimated andthe F test rejection rate underestimated, with the respective over- and underestimation being worse with higher ICC. When ignoring clustering, the RDE at thelevel of the department was underestimated while the F test rejection rate wasoverestimated. When dropping singletons, the RDE increased with an increasing
proportion of singletons and ICC, while the F test rejection rate decreased. Whengrouping the singletons into an artificial department, the RDE decreased slightly
with an increasing proportion of singletons and ICC, while the F test rejection
rate increased slightly

3.2

singletons at the highest level of the hierarchy

In the absence of singletons, the Type I error rate was low, while the corrected
power was high, indicating that the F test performs well. In the presence of singletons, type I error rate was high and power low, indicating that the performanceof the F test is inadequate, and there is a need for an alternative approach.
When dropping or regrouping the singletons, type I error rate and corrected power
decreased. When splitting the singletons, type I error rate and corrected powerincreased. The performance of the Wald test and likelihood ratio test, investigatedfor a scenario with a small number of countries per region (Scenario 3), was worse
than that of the F test. Performance of the permutation test however was superior
to that of the F test.

46

4

Impact of singletons on the linear mixed model

Conclusions

The linear mixed model appears to be stable enough to handle high proportionsof singletons at the lowest level of the hierarchy, even when the intracluster correlation is high. Alternatives which are frequently used, such as ignoring clusteringand removing or grouping the singletons, should be avoided as they provide biased
standard error estimates for the fixed effects.
The linear mixed model appears to be unstable in the presence of singletons at
the highest level of the hierarchy. Frequently used alternatives, such as grouping,
splitting or dropping singletons could solve either the problem of a high type I
error rate, or the problem of low power, while worsening the other. This forced
us to conclude that neither method acts as a solution to the poor performanceof the F test. We showed that the performance of the likelihood ratio test andthe Wald test was comparable to that of the F test, while the permutation test
outperformed the F test, and hence recommend to use the permutation test in
the presence of singletons at the highest level of the hierarchy.

References
Garson, D.G. (2013). Hierarchical linear modeling: guide and applications. London: Sage publications.
Snijders, T.A.B. and Bosker, R.J. (1999). Multilevel analysis: an introduction
to basic and advanced multilevel modeling. London: Sage publications.
Verbeke. G and Molenberghs, G. (2009). Linear mixed models for longitudinal
data. New York: Springer Verlag.

Top-down joint graphical lasso
Eugen Pircalabelu1 , Gerda Claeskens1 , Lourens J. Waldorp2
1
2

ORSTAT and Leuven Statistics Research Center, KU Leuven, Belgium
Dept. of Psychological Methods, University of Amsterdam, The Netherlands

E-mail for correspondence: eugen.pircalabelu@kuleuven.be
Abstract: We develop a new method to estimate simultaneously multiple graphs
and apply it to fMRI data. The method is a ‘top-down’ method which allows
the researcher to zoom repeatedly on edges of the graph obtaining graphs withdifferent levels of detail, in terms of the number of edges and number of nodes. By
repeatedly zooming on edges, we estimate graphs with similar structures acrosscoarseness scales, but with different levels of sparsity.
Keywords: FMRI; Graphical models; Joint graphical lasso; Mixed scale data.

1

Introduction and problem description

For n = 8 participants the cerebral activity has been measured T = 240 times.
The researchers used K = 5 distinct parcelations of the brain containing different
numbers of brain regions of interest (ROIs), obtained in a top-down manner. First
68 ROIs (corresponding to a coarse scale of measurement with anatomically large
brain regions) were defined using an anatomical atlas. Subsequently, the large
volume regions were further split into several smaller regions and the splittingof the larger regions continued several times. In total five coarseness scales wereobtained where the number of ROIs were 68 (for scale 1), 114 (scale 2), 219(scale 3), 446 (scale 4) and 872 (scale 5). The goal is to estimate brain pathways
between the ROIs where each ROI is represented by a node in the graph. We
associate with each scale two graphs: a directed and an undirected one, where
the graphs at the coarsest scale will contain 68 nodes, while the graphs at thefinest scale will contain 872 nodes. Undirected edges indicate a contemporaneous
relation between two nodes and directed edges indicate a lagged effect. The setsof edges are unknown and need to be estimated.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

48

2Top-down joint graphical lasso

Graph estimation

Let n be the number of subjects, K the number of scales, pkn the number of
ROIs at scale k and T the length of the time series (one for each ROI) of brain
measurements. The fMRI measurement at scale k for the ith subject, for the jthk
ROI at time point t is represented by Xi,j,t. The fMRI data at the kth scale forthe ith subject is represented by the matrix Xik which contains as rows the vectors
k
k
T
k
k
k
Xi,·,t= (Xi,1,t
and as columns the vectors (Xi,j,1, . . . , Xi,p, . . . , Xi,j,T)T . Ink ,t )n

its rows Xik contains measurements for all regions at time t and in its columns atimes series for the jth region.
We estimate at each k two graphs: Gk (Euk , V k ) having undirected edges onlyand H k (Edk , V k ) having directed edges only. V k = {1, . . . , pkn } denotes the set of
all nodes for scale k. Euk denotes the set of undirected edges depicted as a − bkbetween a pair of nodes (a, b). Assuming a pkn -dimensional Gaussian vector Xi,·,twith inverse covariance matrix Θ ≡ Σ−1 , the edge set is defined as Euk = {(a, b) ∈V k × V k |a 6= b & Θka,b 6= 0}.
Edk represents the set of directed edges of the form a → b. Following Yin andLi (2011) and Abegaz and Wit (2013), for H k a first order Markov model isused where observations from time t depend only on observations at time t −kk
1. We assume that the vector Xi,·,tconditional on past values, Xi,·,t−1
obeys
k
k
k k
k
Xi,·,t |Xi,·,t−1 ∼ N (Γ Xi,·,t−1 , Σ ). The data generating process depends on scalespecific matrices Γk and Σk . The matrices Γk contain the autoregressive coefficients corresponding to lagged effects between the nodes (the ROIs in theexample) and are used to define the set of directed edges as Edk = {(a, b) ∈V k × V k |Γka,b 6= 0}.
We jointly estimate the matrices A = {Γ1 , . . . , ΓK , Θ1 , . . . ΘK } by minimizing a
top-down penalized negative log-likelihood:
min{−
A

K
X

(log det Θk −trace(SΓk Θk ))+λn1 (k=1X| Θ1a,b | +

a6=b

λn2 (

X
a,b

| Γ1a,b | +KXX| Θka,b |)+k=2 a6=b∈U k
KXX| Γka,b |)},

(1)

k=2 a,b∈D kPnsuch that Θ1 , . . . , ΘK are positive definite and where SΓk = 1/(nT ) i=1
PT
k
k k
k
k k
T
k
k
k
t=2 (Xi,·,t − Γ Xi,·,t−1 )(Xi,·,t − Γ Xi,·,t−1 ) , U = {(a, b) ∈ V × V |a 6=
k−1
k−1
b & Θa,b 6= 0} and Dk = {(a, b) ∈ V k × V k |Γa,b 6= 0}.
Equation (1) uses `1 terms which enforce sparsity (see Friedman et al.,2008) for the directed and undirected graphs. The fine scale edges estimatedat each scale k depend on the coarser edges estimated at scale k − 1 which
creates a zoom effect on edges in a top-down approach where the structureof the coarser graphs influences the structure of the finer graphs making
them similar. Different penalties that enforce sparsity and similarity ofgraphs can be found in Guo et al. (2011) and Danaher et al. (2014).

Pircalabelu et al.

13

2915

13 3
13
29
31010333

28
31
19

1625
23

23 34

1632

30

32

13

3

30
14

30

8

8

12

25

26

26

12

13

13

28

28

16 19
25

31

25

23

23
34

16

23

32

31

23

16

30 7

12

26

30

3030

14

7

12

8

17
14

14

26
26
14 30

26

14

26

14

422

26

26

14
4

7

8

30

8

12 8

30
12 12

26

26

30

22

30

14

22

422

14

4

1212

12

87 8

8

8

30

12
12

12

12

12
1212
12

9
8

29

29 15

16 8

29

29

13

29

16

16

28
28
28

20

3

21

19

3
31
3
1019 19
10
19
16
3

31

31
25

9
1010

25 10252525162531

25 16

25 2523
31
2523
25
10 31
23 23233423 31

25

7

5

17

21

13

13

33
3

29

24
17

28
20 21
28

13

2429 29

29
29
24

28

21

29

15

2929
29
2

25

28

28
13

28

13

2

2

29

29

7

28

28

13

23 9

9

23
25 25
24
25 25 24
31 23
31 34
24
1825 23
25 18
17 17
2323
25
2517 17 25
31
32
34
2423
32
25 24 17
23 7 23
23 23 1823
31 31
17 1830 237237 2318 23 1717
30
11
26
3130
1126
7
26 14
16

9

32

10

7
32

31 16

30

32

32

32

1

307
8

8

8

8

16

9
8

8
88 7

30
730 26

12
8

30

14

12
8

8
12

12

12

12
12

7 30
30
2630 14
26
30 30
26 22

26
14

1426 26 426267

14
26
22
4

30

14

30

4 14
144 1422
22
4
26

22
22
14

12
1212

9
32 8 8

11

26
2626

30 30

7

8
12

32 16

32

30

11
7 30 14 26 26
30 1126
30 1414
30

9

16 9

32

11

1

1

32
9 1

32

9 31

18

16

31

16

31
16

9
3232

7

32

10

9 32

32

8

12
30 12
30

4

30 30 14

1212
12
12

7

31
9

11

9

30

3030

14

16

28
28

15

29
29
15
29 29
27
27
27 27
29 13
292 15
29
2
15
15 29
29

13

10
5
233125252323
10
31 9 25
2325

6

29
29
29

2915

16

31

328 8 32

8

26

4 26

22
22

1623

3232

1

32

7

7
14 30

30

14

26
4
26

1422

30

14

11

7

7

30

26

32

2318

18

11 26 30

11

7

8 8

12 12

23

17

11

26
26

11

9

9
32
32
313116

23

23

3

25
9

16

31
23
23
31

34 23

7

3223

17

24

19 16

2516

25

23

10

2510

18

25

2626

26

30
26
30

88 8 3030
87

25

24
17

17

23
18

7 30

8

8 98

24

18

23

30
7

9

1717
24

2325

23

32

1

1

1725

7 34

16

30

14
22
4

2915

3
13

3 10333
31
19
10
19 10 325 253125 10 25

31

25

23
31 9

23

25

24

25

28 1321
28
3

31

31

2525

13

19

3 19
31
2525
25
16
10
25
16

25
10
25
10
7

25
5

25

9

8
8

12 12

28 29

28
13

19

31

10

9

29

21

21

33

29

29

24

13 13

3

24
1717

18

25 10
25

329 1 3231 32
30

32
32
16 9

2

292924

10

23

8

15
29
29

13

28

21

28

33 1019 19

13

21 20

21

20
28

3

29

15

29

25

16 3123 23 993232 31
32
31

32

30

29
29

28

28

20

28

21

13

2
29 29

5

25

23

16

29

13

29

15

13
28

29

29

15
29
2

29

29

28
28

27

15

7

9 3125322614 30 30 7

12

28
28

28

20

2729

29

29

103 3

31

23

28

28
28

29
29

15

10

163125 25 259 1010 25

23 25

30
26
11

26

22
14

28

28

28

15
27

13

28

28

29

27
29
15
2

29

21

31
19

16

16

1

6

6
29
29
15

13
3
3 13
292
13
3 10333 29

28
28
19

9

7

26

30
12

23

9

4

8

12

13

28 29
13

21

31
19

6

29 15

29

2929
20

30

12

25
16
25

9

12

15

21

12

3

251031

7
10

26 26
264
22
14 14

30

7

8

8

1931

32

26
1426
26

8

3012

22
4

29

28
28

8

16

21
19

10

32

11

30

7

9

8

33

25
23
23 31
25 34
31
24
18
17
17 23
23 31

24

30
14

30

133
3

13
29

29

25
17

7

9

12
12

29

28 28

30

32

1

32

8

12

28

32

31 1
16

8

30

32

20

28
21

17

25

34

9

2828

13

29

17
18
31 23 23
23

32
16

28

28

29
29
24 24 255

25

9

16

9

8
8

14

12

28

16

32

32 8

26

4
22
30 14

8

7

29
25

16

1

14

14264
22

26

2
15

29

5
23
25 7
23
25
31
23 23 1025
25

3232
31
9

307
26 30 7
30 12
30

2

29

3 1033

103

16

9

34

23

30

26

14

12

20

23

10

23
18

26

15
29
29

15

31 10

23 31
23
31

261426

30
7 30

19
31
19

25
25

31

7

24
17 17

3

13

29
29

13

19

31
19

10

16
25
10 9 25

5

25

11 11

7

7

8

23

21

21

28

29
29
15 13

15

27 2729 29

2813

28

28

33
33

29

17
17

18

20

28

24

30

8

9

20

21

13
29

13

28

13

29

24

25

32
1
1
31 9

28

17

25

23

23

2828

29

24

25

9 32
32 31

28

21529

2

5

25
7
10

6

29

28

28
13

29
29

10

25

16

15

29
25

31 9
25

163123

15
2729

29

19

6

28

29

27

28 13

21

23

6
29

29
29

20
28

6

15

29

28

2828

49

30

4

30

30

12

16

7

8

32

16

9
8
8

8

128

8
8

30

12
12

1212

12

12

8

30

26

14
422

8

8

12

12

FIGURE 1. Directed (left panels) and undirected (right panels) graphs for scales
k = 3 (top) and 4 (bottom) estimated using (λn1 , λn2 ) = (.4, .2)

3

fMRI example

Due to space constraints Figure 1 presents only the graphs estimated forscales 3 and 4 from which we conclude that (i) across the scales the graphs
are similar to each other (due to the top-down penalty we use) and (ii)the contemporaneous associations between ROIs are more pronounced forregions from opposite sides of the brain, while the lagged associations are
more pronounced for regions from the same hemisphere.
Figure 2 shows that the estimated directed and undirected graphs havepercentagewise a similar number of edges within the left hemisphere, but
for the right hemisphere the directed graphs estimate percentagewise moreedges than the undirected ones. Moreover, as we move across the coarsenessscales the within hemisphere connections relative to the total connections
become more predominant than the between hemisphere connections. This
points to a functional connectivity shift in the brain pathways, in the sensethat as we increase the splitting of regions this creates partitions that are
more homogenous within the hemisphere than across the hemispheres.

50

Top-down joint graphical lasso

Coarseness Scale, k

5

●
●

●

4

●

3

●

3

●

1

●

●

●

30
60
% Edges within LH

●

●

●

0

●

Directed
Undirected

●

●

0

30
% Edges within RH

●

60

0
30
60
% Edges between LH and RH

FIGURE 2. Percentages of edges connecting ROIs within the left hemisphere(left), right hemisphere (middle) and from different hemispheres (right).

4

Discussion

The proposed method to jointly estimate sparse graphs at different coarseness scales, starts from the coarsest scale and uses a top-down penalty whichconstrains the structure of finer scales graphs to depend on the structureof coarser scale graphs. Using a top-down approach reveals how the graphs
evolve from having a simpler structure at the coarsest scale to having amore complex structure at the finest scale. The resulting graphs offer the
researcher a more complete image about the phenomenon under study.
References
Abegaz, F. and Wit, E. (2013). Sparse time series chain graphical models
for reconstructing genetic networks. Biostatistics, 14(3), 586 – 599.
Danaher, P., Wang, P. and Witten, D. (2014). The joint graphical lasso forinverse covariance estimation across multiple classes. Journal of theRoyal Statistical Society, Series B, 76(2), 373 – 397.
Friedman, J., Hastie, T. and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3), 432 –441.
Guo, J., Levina, E., Michailidis, G. and Zhu, J. (2011). Joint estimation ofmultiple graphical models. Biometrika, 98(1), 1 – 15.
Pircalabelu, E., Claeskens, G., and Waldorp, L.J. (2016). Mixed scale joint
graphical lasso. Biostatistics, 17(4), 793 – 806.
Yin, J. and Li, H. (2011). A sparse conditional Gaussian graphical model
for analysis of genetical genomics data. The Annals of Applied Statistics, 5(4), 2630 – 2650.

Modelling time series of counts with deflation
or inflation of zeros
Marcelo Bourguignon1
1

Universidade Federal do Rio Grande do Norte, Brasil

E-mail for correspondence: m.p.bourguignon@gmail.com
Abstract: In this paper, we introduce a first order non-negative integer valued
autoregressive process with zero-modified geometric innovations based on the binomial operator. This new model will enable one to tackle the problem of deflation
or inflation of zeros inherent in the analysis of integer-valued time series data.
The main properties of the model are derived, such as transition probabilities andzero probability. The methods of conditional maximum likelihood, Yule-Walker
and conditional least squares are used for estimating the model parameters. A
Monte Carlo experiment is conducted to evaluate the performances of these estimators in finite samples. The proposed model is fitted to time series of number
of weekly sales and weekly number of syphilis cases illustrating its capabilities inchallenging cases of deflated and inflated count data.
Keywords: INAR(1) process; Zero-modified geometric distribution.

1

Introduction

McKenzie (1985) proposed the INAR(1) process based on the binomial thinning
operator, i.e., a sequence {Xt }t∈Z is said to be an INAR(1) process if it admits
the representation
Xt = α ◦ Xt−1 + t ,(1)
where 0 ≤ α < 1, {t }t∈Z is a sequence of independent and identically distributed
integer-valued random variables, called innovations, with t independent of Xt−k
for all k ≥ 1, E(t ) = µ and Var(t ) = σ2 , and “◦” is binomial thinning operator,
defined by
 PXt−1
Xt−1 > 0;
j=1 Yj ,α ◦ Xt−1 =0,
Xt−1 = 0,
where the so-called counting series {Yj }j≥1 is a sequence of independent andidentically distributed Bernoulli random variables with Pr(Yj = 1) = 1 − Pr(Yj =0) = α.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

52Modelling time series of counts with deflation or inflation of zeros

While processes for integer-valued time series are now abundant, there is a shortage of similar processes when the time series refer to data with deflation or inflation of zeros, i.e., processes for modeling count time series with excess (or deficit)
of zeros based on thinning operators were discussed by few authors. Jazi et al.
(2012b) introduce a new stationary INAR(1) process with zero inflated Poisson
innovations [ZINAR(1)]. Recently, Barreto-Souza (2015) proposed a first-orderinteger-valued autoregressive process for dealing with count time series with deflation or inflation of zeros [ZMGINAR(1)]. The proposed process has zero-modified
geometric marginals. However, the conditional probabilities of this model don’t
have a simple form and the parameter restrictions aren’t liberal.
This paper aims to give a contribution in this direction. The objective of thispaper is to propose a new INAR(1) process (1) with zero-modified geometric(ZMG) innovations, denoted by INARZMG(1), based on binomial thinning formodeling nonnegative integer-valued time series with deflation or inflation ofzeros.
Let {t }t∈Z be a sequence of discrete i.i.d. random variables following a zeromodified geometric (ZMG) distribution with parameters µ > 0 and π ∈ (−1/µ, 1).
More specifically, we here assume that {t }t∈Z has a probability mass function
given by( 1+πµ,if y = 0,
1+µPr(t = y) =(2)µy
(1 − π) (1+µ),
if y = 1, 2, . . .
y+1
The process {Xt }t∈Z satisfying Equation (1), with {t }t∈Z ∼ ZMG(π, µ), is
Markovian, stationary, and ergodic. Then, the Markov process admits a uniquestationary distribution.
The dispersion index, which is the variance-to-mean ratio, will be given by
IX :=2µ(1 + π)
σX
=1+
,
µX
1+α

it follows that this model presents equidispersion when π = −1; underdispersion
when µ ∈ (0, 1) and π ∈ [−1/µ, −1); and overdispersion when π ∈ (−1, 1).

2

Real data examples

2.1

Modelling deflation of zeros

In the first application, we consider the series of weekly sales of a particular soap
product in a supermarket. The proportion of zeros in the series considered is
3.7%. Then, we have evidence that there is deflation of zeros in the number ofweekly sales. The time series data and their sample autocorrelation and partial
autocorrelation functions are displayed in the Figure 1.
Table 1 provides the estimates of the model parameters and two goodness-of-fit
statistics: Akaike information criterion (AIC) and Bayesian information criterion(BIC). From this table, we observe that the proposed model being better.

53

10 15 20

●
●
●
●
●
●

5

●●

●

●

●

●

●

● ●
●

●

●

●

●●

●
●

●

●

●
●●
●

●●
●
●

●

●

●

●

●
●
●●
●
●
●
● ●
●
● ●
●
● ●
●
●●●
●● ●
●
●
● ●●●●●
●●
●
●
●
● ●
● ● ● ●●●
● ● ● ●●●●
●●
●
● ● ●
●●
●
●
●
●
●

● ●
● ●

●

●

● ●
●
●● ●

●
●

0

Number of weekly sales

Marcelo Bourguignon

●●
●
●

●
●

0

50

●
●

●

●

●
●● ●
●
●●
●●
●
● ●
●
●
●
●
●
●
●●
● ●
●● ●● ●
●
●
●● ●
●
●
● ●
●
●
● ●●
●
●
● ● ● ●●●
● ● ●
●● ●
●●
●
●●
●
● ● ●
●
● ●●
●
● ●●●
●
●●
●● ●
●
●
●
●
●●
●

● ●
● ● ● ●●
● ● ● ●●●
● ● ● ● ●● ●
●● ●
●
● ● ●
●●

●●

●

100

150

200

250

0.4
0.2
0.0
−0.2

−0.2

0.0

ACF

PACF

0.2

0.4

Time

5

10

15

20

5

10

Lag

15

20

Lag

FIGURE 1. Plots of the time series, autocorrelation and partial autocorrelation
functions for the number of weekly sales.
TABLE 1. Estimates of the parameters and goodness-of-fit statistics for the firstdata set.
Model
α
b
µ
b
π
b

CML Estimates
0.3056
3.2094−0.1826

AIC
1252.88

BIC
1263.4INARZMG(1)

α
b
b
λ
ρ
b

0.36234.59720.24151310.911321.4

ZINAR(1)αbpb0.37120.77461260.34

1267.32

INARG(1)2.2

Modelling inflation of zeros

5

x

2

3

4

●

50

●●

100

200

0.1
PACF

0.1

−0.2

0.0
−0.2

ACF

150

0.2

0

●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● ●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●

0.0

0

1

●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

0.2

Weekly number of syphilis cases

For the second application, we consider the weekly number of syphilis cases in
the United States from 2007 to 2010 in Vermont state given in ZIM package. The
proportion of zeros in the series considered is 96%. Then, we have evidence that
there is inflation of zeros in the number of syphilis cases. The series, autocorrelation and partial autocorrelation functions are displayed in Figure 2.

5

10
Lag

15

20

5

10

15

20

Lag

FIGURE 2. Plots of the time series, autocorrelation and partial autocorrelation
functions for the number of syphilis cases.

54

Modelling time series of counts with deflation or inflation of zeros

Table 2 gives the CML estimates, AIC and BIC for the fitted models. Since thevalues of the AIC and BIC are smaller for the INARZMG(1) process comparedto those values of the ZINAR(1) and INARG(1) models, the new model seems acompetitive model for these data.
TABLE 2. Estimates of the parameters and goodness-of-fit statistics for the number of syphilis cases.
Model
α
b
µ
b
π
b

CML Estimates
0.08010.72390.9078

AIC
96.46

BIC
106.49INARZMG(1)

α
b
b
λ
ρ
b

0.0894
1.18930.944497.80

107.83

ZINAR(1)αbpb0.11380.0606110.32117.00INARG(1)References
Barreto-Souza(2015) barreto2015 Barreto-Souza, W. (2015). Zero-Modified Geometric INAR(1) Process for Modelling Count Time Series with Deflation
or Inflation of Zeros. Journal of Time Series Analysis, 36, 839–852.
Jazi et al.(2012a) jazia Jazi, M. A., Jones, G. and Lai, C. D. (2012a). Integer
valued AR(1) with geometric innovations. Journal of the Iranian StatisticalSociety, 11, 173–190.
Jazi et al.(2012b) jazib Jazi, M. A., Jones, G. and Lai, C. D. (2012b). First-orderinteger valued AR processes with zero inflated Poisson innovations. Journalof Time Series Analysis, 33, 954–963.
McKenzie(1985) Mc85 McKenzie, E. (1985). Some simple models for discretevariate time series. Water Resour. Bull., 21, 645–650.

An Estimated Parameter of Local Polynomial
Regression on Time Series Data
Autcha Araveeporn
1

Department of Statistics, Faculty of Science, King Mongkut’s Institute of Technology Ladkrabang, Bangkok 10520, Thailand

E-mail for correspondence: kaautcha@hotmail.com
Abstract: The local linear estimation, Nadaraya-Watson estimation, and locallyweighted scatter plot smoothing (LOWESS) estimation are the way for estimating unknown parameter of local polynomial regression based on nonparametric
regression model, fitting function of independent variable by the weighted leastsquares is computed for time series data. With the local polynomial regression
fitting we can estimate the local linear estimator based on kernel function as a
Gausian density function, and the Nadaraya-Watson estimation which is a partof local linear estimation or called a local constant estimation. Furthermore the
LOWESS is used a weighted least squares to estimate parameter that modified
the kernel function as a tricube kernel. The goal of this article is to show, throughapplication, which three estimators can be used the best for fitting time series
data. The Average Mean Square Errors (AMSE) is considered as a criterion tocheck a bias of fitted estimator. For simulation data, the data is generated byautoregressive process by several coefficients. The Nadaraya-Watson estimation
outperforms other methods by showing the minimum of AMSE values.
Keywords: local linear estimation; local polynomial regression; locally weighted
scatter plot smoothing; Nadaraya-Watson estimation

1

Introduction

Regression analysis is a statistical tool for the investigation of relationship between independent and dependent variables in term of regression model. An estimating parameter of regression model requires an assumption such as the linearity, statistical independence, homoscedasticity, and normality that the form of
the underlying regression analysis. If an inappropriate regression model is used, it
is possible to produce misleading conclusions. To overcome the difficulty causedby the restrictive assumption of the regression function, this approach leads toso-called nonparametric regression.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

56

An Estimated Local Polynomial Regression

Typically, the nonparametric regression methods are overcome this problem, because the fitted model interpolates on the curve of data base on bandwidth values.
The local polynomial regression is a class of nonparametric regression method
which has a long history in the smoothing of time series.

2

Methods of Parameter Estimation

The simple of nonparametric regression model can be written as
yt = f (xt ) + εt ,t = 1, 2, . . . , n,(1)where xt are the independent variable as the time points of time series data, yt
are the dependent variable, and εt denote the measurement errors. The Taylor
expansion can be approximated by a local polynomial regression of degree p andassumed that f (xt ), asf (xt ) = f (x0 ) + (xt − x0 )f (1) (x0 ) +1(xt − x0 )2 f (2) (x0 ) + . . . , t = 1, 2, . . . , n.
2

The local polynomial regression estimator is used the weight least-squares tominimize of(β̂0 , β̂1 , . . . , β̂p ) = arg minnX(yt − f (xt ))2 Kh (xt − x0 ),

(2)

t=1

where K(·) is a kernel function, h > 0 is called the bandwidth.
The estimators of local polynomial regression are depended on the process ofweight least squares and the bandwidth selection. The following methods areshown the parameter estimation on each method.

2.1

The Local Linear Estimation

For the weighted least squares, the local linear estimator can be approximatedby setting p = 1, denote for j = 0, 1 and 2,
Sj (x; h) =nX

Kh (xt − x0 )(xt − x0 )j ,t=1

Tj (x; h) =nX

Kh (xt − x0 )(xt − x0 )j yt .

t=1

The local linear estimator isβ̂0 (x; h) =T0 (x; h) S2 (x; h) − T1 (x; h) S1 (x; h)S0 (x; h) S2 (x; h) − S12 (x; h)

(3)

,and
T1 (x; h) S0 (x; h) − T0 (x; h) S1 (x; h).
(4)S0 (x; h) S2 (x; h) − S12 (x; h)
The quality of the local polynomial regression depends on the bandwidth selection method. Ruppert, Sheather, and Wand (1995) studied the ideas of plug-in
bandwidth selection of kernel estimators.
β̂1 (x; h) =

Araveeporn.

2.2

57

Nadaraya-Watson Estimation

The trend estimator is evaluated by using the Nadaraya-Watson kernel estimator(Nadaraya (1964) and Watson (1964)) and written asPn
Kh (xt − x0 )yt
Pt=1fˆ(xt ) =,(5)nt=1 Kh (xt − x0 )where h is known as the bandwidth parameter. The bandwidth can be chosenusing a cross-validation criteria. The kernel functions K can be chosen as theGaussian density function.

2.3

LOWESS Estimation

Cleveland(1979) introduced an alternative form of the local polynomial regression
which is called LOWESS , locally weighted scatter plot smoothing. The basic
idea was to start the weighted least square as the kernel function in term of thetricube kernel function, and proposed a nearest neighbour bandwidth by settingr = nf + 0.5. For each predictor xt , let
hk =| xk − x |(r) ,where hk is the rth order statistic of the sample | xk − x1 |, | xk − x2 |, . . . , |xk − xn |.

3

Application of Simulation Data

The data is generated in term of autoregressive model in order 1 (AR1) following
yt = ρyt−1 + εt , t = 1, 2, ..., n,where εt ∼ N (0, 1), the coefficient (ρ) of AR(1) is defined by 0.1, 0.5, 0.7, and0.99, and the sample sizes n = 100, 200, 300, and 400. The accuracy of fitting
function is consider by the Mean Square Errors (MSE) as follows:n

M SE =

1X(yt − ŷt )2 ,
n t=1

(6)

where yt denoted the simulated data and ŷt denoted the fitting data of 3 estimators. From Table 1, the results show that the average MSE of NW is the minimum
values in all cases.

4

Conclusion

We have been discussed the method to estimate parameter of local polynomial
regression function. It is concluded that the estimation of NW has indicated agood performance more than LL and LOWESS method. Hence it can be saythat NW leads to a nearly enough class of local polynomial regression to estimate adequately fitted time series data. The cross-validation criteria is a good
performance for fitting model.

58

An Estimated Local Polynomial Regression

TABLE 1. The average MSE on simulated data of local linear (LL),
Nadaraya-Watson (NW), and locally weighted scatter plot smoothing (LOWESS)
estimation.

Sample
sizes
n=100

n=200

n=300

n=400

Methodρ =0.1

ρ =0.5

ρ = 0.7

ρ = 0.99

LL
NW
LOWESSLL
NW
LOWESSLL
NW
LOWESSLL
NW
LOWESS

1.05130.54381.21671.04380.51371.1113
1.03370.50111.07091.03770.5042
1.06720.75930.36041.48100.92990.34941.4033
1.06710.35011.3906
1.13170.34681.37360.60990.2966
1.86350.84890.29181.95881.0527
0.29101.95611.23280.29081.98850.48220.21973.3187
0.63270.22286.50610.78930.21909.93200.96510.222712.3403

References
Cleveland, W.S. (1979). Robust Locally Weight Regression and Smoothing Scatter Plots. Journal of American Statistics Association, 74, 829 – 836.
Nadaraya, E.A. (1964). On Estimating Regression. Theory of Probability and Its
Application, 10, 186 – 196.
Ruppert, D, Sheather, S. J.,Wand M. P. (1995). An Effective Bandwidth Selector for Local Least Squares Regression. Journal of the American Statistical
Association, 90, 1257 – 1270.
Watson, G.S. (1964). Smooth regression analysis. Sankhyā, Series A, 26, 359 –372.

New links for binary regressions: An
application to the coca cultivation in Peru
Artur J. Lemonte1 , Jorge L. Bazán2
12

Universidade Federal do Rio Grande no Norte, Brazil
Universidade de São Paulo, Brazil

E-mail for correspondence: arturlemonte@gmail.com
Abstract: Binary response data arise naturally in applications. In general, thewell-known logistic and probit regression models form the basis for analysing
binary data in practice. These regression models make use of symmetric linkfunctions (logit and probit links). However, many authors have emphasized theneed of asymmetric links in modeling binary response data. In this paper, weconsider a broad class of parametric link functions that contains as special cases
both symmetric as well as asymmetric links. Furthermore, this class of links isquite flexible and simple, and may be an interesting alternative to the usual
regression models for binary data. We consider a frequentist approach to perform inferences, and the maximum likelihood method is employed to estimate
the model parameters. We also propose residuals for the link models to assess
departures from model assumptions as well as to detect outlying observations.
Additionally, the local influence method is discussed, and the normal curvatures
for studying local influence are derived under two specific perturbation schemes.
Finally, an application to the coca leaf cultivation in Peru is considered to show
the usefulness of the proposed link models in practice.
Keywords: Binary response model; Maximum likelihood estimation; Parametriclink function; Symmetric distributions.

1

New link functions

In the generalized linear model setup, we have that
µi = F (ηi ),i = 1, . . . , n,(1)where F (·) is a CDF and its inverse F −1 (·) is typically called link function. The
link function is symmetric when F (·) is a CDF of a symmetric distribution.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

60

New links for binary regressions

1.1

Power symmetric and reciprocal power symmetric
distributions

We have the following definition.
Definition 1 A univariate random variable T is said to have a power distribution with location, scale and shape parameters given by ξ ∈ IR, φ > 0 and λ > 0,
respectively, if its CDF has the form
λt−ξFp (t) = G, t ∈ IR,φ
where G(·) is any absolute continuous CDF. The standard power distribution
arises when ξ = 0 and φ = 1, and its CDF is Fp (z) = G(z)λ , for z ∈ IR.
In the above construction, the function G(·) is refereed to as the baseline CDF.
We can also define the (standard) reciprocal power distribution by considering
Frp (z) = 1 − G(−z)λ , for z ∈ IR. From the above definition, it is evident that
we can introduce several new class of distributions. In this paper, we will assume
that the baseline CDF G(·) belongs to the symmetric family of distributions in
its standardized form (Fang et al., 1990). In this case we can express Frp (z) = 1−
[1 − G(z)]λ . Therefore, we have the standard power symmetric (PS) distribution
as well as the standard reciprocal power symmetric (RPS) distribution.
The PS distributions are skewed to the right if λ > 1 and to the left if 0 < λ < 1,
and the RPS distributions are skewed to the left if λ > 1 and to the right if 0 <λ < 1. The additional shape parameter λ introduces a great degree of skewness
to the distribution. In fact, let us consider the definition of skewness measure interms of the mode for the PS and RPS distributions; that is, if Z ∼ PS(λ), thenγp = 1 − 2Fp (M ) = 1 − 2G(M )λ , where M is the mode of Z; and if Z ∼ RPS(λ),
it follows that γrp = 1 − 2Frp (N ) = 2G(−N )λ − 1 = −γp , where N = −M is themode of Z. Clearly, −1 ≤ γp , γrp ≤ 1. Additionally, note that small values of λyield values of γp and γrp near −1 and 1, respectively. On the other hand, γp andγrp are close to 1 and −1, respectively, for high values of λ. In short, γp → −1(1)as λ → 0+ (∞), whereas γrp → −1(1) as λ → ∞(0+ ). Values of this skewness
measure near 1(−1) will indicate extreme right (left) skewness. Therefore, we canhave a high degree of skewness (positive as well as negative) for the PS and RPS
distributions depending on the values of the additional shape parameter λ. Thus,
asymmetric link functions constructed from the PS (RPS) distributions can bevery skewed. Finally, we have that γp = γrp = 0 when λ = 1, as expected.

1.2

The PS and RPS link models

The first class of asymmetric links uses Fp (ηi ) (i.e. the PS distributions) for F (·)
in (1), that is,µi = Fp (ηi ) = G(ηi )λ , i = 1, . . . , n.
(2)When F (·) in (1) involves Frp (ηi ) (i.e. the RPS distributions) we have the secondclass of asymmetric links given byµi = Frp (ηi ) = 1 − G(−ηi )λ ,i = 1, . . . , n.

(3)

The additional parameter λ > 0 characterizes the skewness of the link functions
associated with models (2) and (3). For λ = 1 the models (2) and (3) are the same

Lemonte and Bazán61and, in addition, the link function is symmetric. These models include commonlyused symmetric links such as the probit and logit links. They also include thesymmetric robit (Student-t) link as a special case. Notice we are considering
wide classes of link functions for binary response data, which contains several
symmetric (and asymmetric) links as special cases.
As expected, the probability µi approaches zero at the same rate as it approaches
one when λ = 1, since in this case the link function is symmetric. When 0 < λ <1 (λ > 1), the probability µi approaches one (zero) at a faster rate than it
approaches zero (one) for the PS models. On the other hand, the probabilityµi approaches one (zero) at a faster rate than it approaches zero (one) whenλ > 1 (0 < λ < 1) for the RPS models. In short, the additional parameter λ can
produce a considerable degree of skewness to the links, since it controls the “tail”
behaviour of the links.
Finally, the parameter λ may be named as “acceleration” parameter for the PS
link models, in the sense that it will accelerate the point (value) of the linear
predictor at which the slope power of the curve becomes greatest. In contrast,
this parameter may be named as “deceleration” parameter for the RPS linkmodels, once now we have an opposing behavior.

2

Application to real data

We have a study on eradication of the coca cultivation in Peru, and we consider
the following regression models for modeling these data:µi = Fp (ηi ) = G(ηi )λ ,µi = Frp (ηi ) = 1 − G(−ηi )λ ,where µi = Pr(Erad = 1), andηi = β1 + β2 Pculti + β3 Cpari + β4 Pccoi + β5 Polevi ,for i = 1, . . . , 1947. We shall consider several link functions to model these binary
data, which are based on the Normal, Logistic, Laplace, Cauchy, Student-t andPE distributions. The degrees of freedom ν > 0 in the Student-t and PS(RPS)Student-t link models as well as the value of k ∈ (−1, 1] in the PE and PS(RPS)PE link models were obtained by using the profile log-likelihood procedure We
also include the complementary log-log (cLogLog) and log-log (LogLog) link functions given by log[− log(1 − µi )] = ηi and − log[− log(µi )] = ηi , respectively, forthe sake of comparison.
To compare the different regression models, we consider selection criteria on thecandidate models; that is, the value of the log-likelihood function evaluated at theb and the AIC (Akaike Information Criterion), BIC (Bayesian
ML estimates (`),
Information Criterion) and HQIC (Hannan-Quinn Information Criterion) criteria (see Table 1). According to the AIC, BIC and HQIC criteria, the PS- and
RPS-Student-t regression models outperform all regression models and therefore
should be preferred, indicating that these regression models may be suitable todescribe these binary data.

References
Fang, K.T., Kotz, S., Ng, K.W. (1990). Symmetric Multivariate and Related Distributions. Chapman and Hall, London.

62

New links for binary regressions

TABLE 1. AIC, BIC, HQIC and `b for the fitted models.

Link model
NormalLogistic
LaplaceCauchyStudent-t (ν = 2.36)
PE (k = 0.42)
PS-Normal
PS-Logistic
PS-Laplace
PS-Cauchy
PS-Student-t (ν = 0.25)
PS-PE (k = −0.2)
RPS-Normal
RPS-Logistic
RPS-Laplace
RPS-Cauchy
RPS-Student-t (ν = 0.19)
RPS-PE (k = −0.34)
cLogLog
LogLogb̀

−1223.03−1222.85−1223.56−1223.30−1222.73−1223.28−1222.95−1222.44−1220.75−1220.66−1218.22−1219.81−1222.85−1222.65−1221.09−1221.30−1218.20−1219.85−1223.10−1224.50

AIC

BIC

HQIC

2456.052455.712457.122456.612455.472456.55
2457.902456.872453.502453.33
2448.45
2451.622457.702457.312454.182454.60
2448.40
2451.692456.202459.00

2483.92
2483.582484.992484.482483.34
2484.422491.35
2490.32
2486.94
2486.772481.892485.06
2491.152490.75
2487.622488.04
2481.842485.142484.07
2486.87

2466.30
2465.962467.37
2466.852465.712466.80
2470.20
2469.172465.792465.622460.74
2463.92
2470.00
2469.60
2466.472466.892460.702463.99
2466.452469.25

Wavelet regression and generalized linear
models with continuous distributions
Aluı́sio Pinheiro1 , Eufrásio Lima Neto2
1
2

Universidade Estadual de Campinas, Campinas, Brazil
Universidade Federal da Paraı́ba, João Pessoa, Brazil

E-mail for correspondence: apinheiro.unicamp@gmail.com
Abstract: We discuss in this paper the use of wavelet regression methods ingeneralized linear models setups. The use of such a non-parametric link function would be interesting in practice for several reasons. The first one is that
researchers in different areas are used to the employment of the paradigm of generalized linear models. The second one is that researchers are more interested in
the set of explanatory variables which adequately predicts the responses than on
the specific form on which this relationship happens. The correct specification of
the link function is paramount to a good fit. The wavelets can be employed inthree ways: (i) by helping choosing the link function from a pre-specified set offundtions; (ii) by providing a fully non-parametric estimation of the mean; and(iii) by providing a non-parametric link function.
Keywords: Non-parametric link function; non-linear regression;semi-parametric
regression

1

Introduction

The generalized linear models (GLMs) represent one of the most important developments in statistical theory over the past several decades[6]. A GLM is characterized by three terms. The first is the random component with the response
variable belonging to the exponential family of distributions. The second termis the systematic component represented by a linear predictor that includes the
explanatory variables. Finally, the third term is the link function which connects
the linear predictor to the response variable mean.
An important step in the GLM model employment is the choice of the link function. This function represents the mathematical structure or the regression equation that relates the random component to the linear predictor in this class ofmodels. Link misspecification can lead to several problems on a GLM application, such as bias in the regression parameters and in the mean response estimates
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

64

Wavelet regression and generalized linear models[2, 3]. A methodology which finds an appropriate link function for a GLM is still
an open problem. Techniques have been proposed to evaluate if a predeterminedlink function is adequate for a fitted GLM model [1, 4, 7]. Most of the current
techniques are straightforward adaptations from linear models’ techniques.
Wavelets have been developed in functional analysis as bases for L2 (R), as well assome of its subspaces. These classes of functions contain a large number of diverse
elements, which makes them suitable for broad theoretical and numerical applications. For instance, they form unconditional bases for some large functionalclasses, which leads to optimal estimators and tests [8].

2The problem

A Multi-Resolution Analysis (MRA) in L2 (R) is a nested sequence of closed
subspaces, {Vj }j∈Z with four basic properties: (i) HierarchyS- Vj ⊂ Vj+1 ⊂L2 (R) ∀j ∈ Z; (ii) Dense Union and Trivial Intersection - j∈Z Vj = L2 (R)T
and j∈Z Vj = {0}; (iii) Self-Similarity - m(2j t) ∈ Vj ⇔ m(t) ∈ V0 ∀j ∈ Z;kand (iv) Natural
Basis - ∃φ ∈
PV0 so that T φ(t) = φ(t − k)∀k ∈ Z spans V0 ,i.e., V0 = m ∈ L2 (R) | f (t) = k∈Z ck φ(t − k) for some appropriate sequence{ck }k∈Z , and {φ(· − k), k ∈ Z} is called an orthonormal basis of V0 . By mirror
filtering another function, ψ, can be built, so that any function m ∈ L2 (R) canbe written in L2 -sense as:XXX
X X
m(t) =
βj,k ψj,k (t) =αj0 ,k φj0 ,k (t) +βj,k ψj,k (t),j∈Z k∈Zk∈Z

j≥j0 ∈Z k∈Z

for an arbitrary j0 , where φj (t) = 2j/2 φ(2j t − k) j ∈ Z k ∈ Z and {ψj,k (t) =j
2 2 ψ(2j t − k), j ∈ Z, k ∈ Z}[5].
Let Y = {y1 , . . . , yn } be a set of observations that represents a random sample of
the response variable Y . Suppose a set of explanatory variables X1 , . . . , Xp andη = g(µ) = Xβ,(1)

where X is the design matrix formed by the observed values of the explanatory
variables X1 , X2 , . . . , Xp , β is vector of parameters, η is the vector of linear predictors, µ is the vector of means of Y , i.e., with η=(η1 , . . . , ηn )T , µ = (µ1 , . . . , µn )Tand β=(β0 , . . . , βp )T . g(µ) is the link function, which connects the response variable mean with the explanatory variables. If Y is continuous, a few functions
available for a GLM are: the identity, logarithmic, inverse, power, among others.
We propose wavelets in three ways. The first procedure identifies the most appropriate link function, g, for a GLM from a set of pre-specified link functions.
The second is a purely non-parametric wavelet regression without any paradigm
related to GLM for the estimation of the mean function m of Y . Finally the third
setup employs a wavelet estimation of the link function g within the GLM model.
The proposals are illustrated by simulation studies and by their application on
three data sets. The data sets present problems on insurance, measurements onbabies and SONAR signal analysis. The Monte Carlo simulation studies withdifferent sample sizes, random components, true link functions and criteria selection are considered and the overall and relative performances of each proposal
are recorded.

3

Simulation Studies

We perform an experimental study to evaluate the performance of the wavelet
proposals. The artificial data sets consider a predefined relationship between the
response variable Y and the linear predictor η = Xβ, where X represents a setof explanatory variables. The synthetic data sets are generated according to 30
different configurations, taking into account 3 sample sizes (128, 256, 512), 3
probability distributions for the response variable Y (Gaussian, gamma, inverse
Gaussian) and 4 link functions (identity, logarithm, inverse, 1/µ2 ). Note that the
1/µ2 link function was considered only in the inverse Gaussian model. We considered one explanatory variable X, uniformly distributed in (0.5, 1.5)]. A Monte
Carlo simulation was considered for each configuration with 1000 replications.
At each time, we generate an artificial data set according to a predefined GLM.
The WM and other eligible GLMs are fitted to this data set. We then compare theperformances of the three procedures. On the first wavelet proposal, we calculate
the frequency of correct classification (choice) of link functions. This was neversmaller than 85% in our simulation studies. The link functions log and inverse
exhibited a better true classification rate when compared with identity link function. The WM demonstrated a better performance in the random components
gamma and inverse Gaussian, when compared with the Gaussian distribution.
These results highlight that the WM can be used for choosing an appropriate
link function when the response variable presents an asymmetric distribution
and a nonlinear relationship between the variables.
For the non-parametric and semi-parametric proposals, performance of the parametric correct link function is superior, specially for smaller samples. This, however, depends on the initial correct link function specification, which is not expected in many situations. Moreover, non-linear and segmented linkages are truein many real-life situations. In these latter cases the non-parametric and thesemi-parametric are superior to the parametric, with a clear advantage for thesemi-parametric in estimation and prediction precision.

4

Concluding remarks

We propose the use of wavelets in generalized linear model setups. The percentage of true link function classification is higher than 85% but for one configuration. Moreover, this degree of accuracy is based on very parsimonious wavelet
representation. If the true link function is compared to the non-parametric andsemi-parametric approaches, it will be superior to them, as expected. However,
the advantage the fully parametric method has depends on the right choice of thelink function which is not necessarily true. Moreover, for many interesting applications the true nature of the link function is not parsimoniously represented byany set of link functions. For these more realistic situations wavelets may be theonly efficient choice.
We believe that wavelets may be useful on GLM setups for the purposes here
specified but further research will yield other ways in which the combined use of
wavelet representation and generalized linear models will provide the practitioner

66

Wavelet regression and generalized linear models

with a very reliable and efficient tool.

Bibliography[1] Cole, M. J. and McDonald, J. W. (1989). Bootstrap goodness-of-link testingin generalized linear models. Statistical Modelling, 57, 84–94.
[2] Czado, C. and Santner, T. J. (1992). The effect of link misspecification on
binary regression inference. Journal of Statistical Planning and Inference, 33,
213–231.
[3] Czado, C. and Raftery, A. E. (2006). Choosing the link function and accounting for link uncertainty in generalized linear models using Bayes factors.
Statistical Papers, 47 (3), 419–442.
[4] Hinkley, D.V. (1985). Transformations diagnostic for linear models.
Biometrika, 72, 487–496.
[5] Morettin, P.A., Pinheiro, A. and Vidakovic, B. (2016). Wavelets in Functional
Data Analysis. Springer, New York, in press.
[6] Nelder, J. and Wedderburn, W. M. (1972). Generalized linear models. Journalof Royal Statistical Society A, 135, 370–384.
[7] Pregibon D. A. (1980). Goodness of link tests for generalized linear models.
Journal of the Royal Statistical Society C, 29 (1), 15–23.
[8] Vidakovic, B. (1999). Statistical Modeling by Wavelets. John Wiley & Sons.

Bivariate residual plots with simulation
polygons
Rafael A. Moral1 , John Hinde2 , Clarice G. B. Demétrio1
1
2

ESALQ/USP, Brazil
NUI Galway, IrelandE-mail for correspondence: rafael.moral@usp.br
Abstract: When using univariate models, goodness-of-fit can be assessed throughmany different methods, including graphical tools such as half-normal plots witha simulation envelope. This is straightforward due to the notion of ordering ofa univariate sample, which can readily reveal possible outliers. In the bivariatecase, however, it is often difficult to detect extreme points and verify whether asample of residuals is a reasonable realisation from a fitted model. We propose
a new framework, implemented as the bivrp R package, available on the Comprehensive R Archive Network. Our framework uses the same principles of thesimulation envelope in a half-normal plot, but as a simulation polygon for eachpoint in a bivariate sample.
Keywords: bivrp package; Bivariate models; Goodness-of-fit; Graphical methods.

1

Introduction

For both univariate and multivariate models, it is possible to obtain summary
goodness-of-fit statistics to compare model fits, such as information criteria (e.g.
Akaike and Bayesian) and adjusted R-squared values. However, while these statistics are useful for comparing models fitted to the same data, they cannot be used
to inform on the suitability of a particular model to the data. In this case, diagnostic analyses play a significant role. For univariate models, a possible alternativeis the use of half-normal plots with simulation envelopes (Moral et al., in press).
Here, we extend this approach to bivariate models. Our proposal makes it possible for one to graphically examine whether the observed data is a plausible
realisation of the fitted model.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

68Bivariate residual plots with simulation polygons

2

Methodology

Let yi = (y1i , y2i )T be a vector of bivariate responses to which a model is fitted.
Also, let ri = (r1i , r2i )T be the vector of bivariate model diagnostics (e.g. residuals, Cook’s distances, leverages). Then, a very basic and straightforward way ofordering the bivariate diagnostics is by calculating the angle
 r2i
αi = tan−1r1ithey form with the origin (see Figure 1(a)), to obtain the ordered diagnostics
r(i) = (r1(i) , r2(i) )T . Here, the ordering is done not to define extremes, but to aid
in the simulation process.

6
4

4

4

1

2

●

−2

−1

0

●

●●

2

●
●

●

3

−3

−2

−1

0

1

2

3

−3

−2

−1

0

1

Residuals 1

Residuals 1

Residuals 1

(d)

(e)

(f)

2

3

4

4

4

6

−3

●

0

Residuals 2

●

●●

●
●

−4

6

−4

−2

●

4 ●5
3

●

●

−4

●●

●
●

12

●

−2

0

8

●
●

●

−2

7
●

0

●

●

9

2

●

10

Residuals 2

2●

Residuals 2

(c)

6

(b)

6

(a)

●
●

● ●

●

● ●

●

●

●

●

−4

−4

●

−3

−2

−1

0

Residuals 1

1

2

3

●
●

0

Residuals 2

●
●

●

● ●

●

●
●

●

−2

●

●

●

●

● ●

●

●

●

●

●

−4

●●

●
●

●

0

Residuals 2

0

●

●
●

●
●

−2

2

●

●

●

−2

Residuals 2

●

2

2

●

●

−4

−2

0
Residuals 1

2

4

−4

−2

0

2

4

Residuals 1

FIGURE 1. The process of constructing the bivariate residual plot with simulation polygons .
ss T
We then simulate 99 bivariate response variables yis = (y1i
, y2i
) , s = 1, . . . , 99from the fitted model, using the same model matrices, error distribution andfitted parameters, and refit the same model to each simulated sample, obtainingss
the same type of model diagnostics rs(i) = (r1(i), r2(i))T , ordered by the angles
they form with the origin. We have, for each bivariate diagnostic r(i) , 99 simulated
bivariate diagnostics rs(i) (see Figures 1(b) and (c)), forming the whole cloud of
simulated diagnostics (see Figure 1(d)). We then obtain the convex hulls of each
set of the s sets of points and obtain a reduced polygon whose area is 95% of theoriginal convex hull’s area, forming the simulated polygon (see Figure 1(e)). The
points are then connected to the centroids of their respective simulated polygons
and, if they lie outside the polygons, they are drawn in red (see Figure 1(e)). For
the final display, the polygons are erased so as to ease visualization (see Figure1(f)).

Moral et al.

3

69

Example and discussion

We illustrate our approach with a simple simulated example with a moderatesample size n = 80.. Let Y1i and Y2i , i = 1, . . . , 80, be two normally distributed
correlated random variables. We may write



  2

Y1i
µ1i
σ1 σ12
Yi =
∼ N2
,
,
(1)
2
Y2i
µ2i
σ12 σ2
which yields, marginally,Yji∼

N (µji , σj2 ),j = 1, 2,(2)

with Cov(Y1i , Y2i ) = σ12 . Hence, the marginal distributions remain unalteredregardless of σ12 , which when equal to zero implies independence between the
random variables.
We supposed a simple linear regression of the form µji = βj0 + βj1 xi between
both variables and a covariate xi , which ranges from 1 to 10 in equal steps of
approximately 0.114, giving 80 distinct xi values. We fixed the true parametervalues as β10 = 2, β11 = 0.4, β20 = β21 = 0.2, σ12 = 2, σ22 = 3, and a negative
covariance σ12 = −1.7.
We simulated the data in R (R Core Team, 2016) using the function rmvnorm
from the package mvtnorm (Genz and Bretz, 2009) with the seed fixed as 2016
to allow for reproducibility. We then fitted the bivariate model using the BFGS
algorithm implemented in the function optim twice, first estimating the covariance σ12 and then the second time fixing σ12 = 0, which is the same as fitting
two separate linear regressions. The parameter estimates were virtually the samefor both approaches, with the obvious exception of the covariance estimate. The
contribution to the likelihood when estimating the covariance was 43.33 on 1 degree of freedom, which is highly significant, giving evidence that σ12 is different
from zero, as expected.
We then proceeded to produce the bivariate residual plot with simulation polygons for both model fits. The raw residuals may be defined asorji= yji − µ̂ji ,µ̂ji = β̂j0 + β̂j1 xi .

(3)

We observe that fitting the models independently for each variable yields a poor
model fit, with many points lying outside of the simulation polygons when compared with the joint model (see Figure 2). We observe that because of the negative correlation between the variables the expected two-dimensional pattern of
the scatterplot of residuals is actually an ellipse (Figure 2(b)) instead of a circle(Figure 2(a)). Hence, even though when inspecting each variable marginally it
might seem that the behaviour of the residuals is as expected by the 99 simulations, this is not true when looking at the bivariate nature of these residuals.
Here, our proposed approach worked well in identifying these bivariate patterns
and was appropriate to aid in model selection and in identifying possible outliers.
We do not rule out other forms of diagnostic checking, either analytical orgraphical. Our approach represents a potentially helpful framework for assessing goodness-of-fit for bivariate models.

Acknowledgments: Special Thanks to FAPESP (proc no. 2014/12903-8 and2014/03310-3).

70Bivariate residual plots with simulation polygons

FIGURE 2. Diagnostic plots for the bivariate normal simulated data using raw
residuals, while assuming there was no covariance between the two random variables (a) or estimating this covariance with a joint model (b). Filled red points
are outside of their respective simulated polygon; there are (a) 45 and (b) 6 points
outside of the simulated polygons out of 80 total points.

References
Genz, A. and Bretz, F. (2009). Computation of multivariate normal and t probabilities Berlin: Springer.
Moral, R.A., Hinde, J. and Demétrio, C.G.B. (in press). Half-normal plots andoverdispersed models in R: the hnp package Journal of Statistical Software.
R Core Team (2015). R: A language and environment for statistical computing.
R Foundation for Statistical Computing, Vienna, Austria. URL: https://www.Rproject.org/.

Gradient test for variance component models
Antonio Hermes Marques da Silva Junior1212Durham University, Durham, UK
Universidade Federal do Rio Grande do Norte, Natal, Brazil

E-mail for correspondence: a.hermes.marques-da-silva-jun@durham.ac.uk
Abstract: Variance component models are an extension for the generalised linear
models for group-wise data. The gradient test is an asymptotic likelihood-based
test with chi-square as reference distribution and its statistic does not makeuse of any matrix operation. We motivate this work by a real data application.
We propose a bootstrap simulation for checking the accuracy of the chi-squareapproximation for the gradient test in comparison to the likelihood test. We also
show numerical confidence regions based on the inversion of the test.
Keywords: asymptotic test; random effects; random intercepts

1

Motivation data and the model

We take the data from an experiment given by Markussen (2017). It is of interest
to investigate how the continuous measurement of redness of pork meat afterslaughter is affected by the storage (in light or darkness), by the time (1, 4 or
6 days) and by the breed (old and new, 10 pigs each). Six chops were takenfrom each pig and allocated according to the scheme shown on Table 1. ThisTABLE 1. Factor allocation [source: Markussen (2017)].

Storage

1 days4 days6 daysDark
Lightchop 1chop 4chop 2chop 5chop 3chop 6gives 2 × 10 × 6 = 120 samples of pork chops in total. Given that the response
variable is strictly positive, we consider that the redness measurements yijk of agiven replicate corresponding to the ith breed, the jth storage and the kth time
are independently distributed as IG(µijk |Z, φ) with means µijk |Z and a fixed
dispersion φ. We also assume the linear predictor is linked to µijk |Z asµijk |Z = Z + αi + τj + βk i = 1, 2, j = 1, 2, k = 1, 2, 3

(1)This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

72

Gradient test for variance component models

where Z is a random intercept representing the base level for each pig, α1 =
τ1 = β1 = 0 and (1) is one of the configurations of the variance component model
defined in Aitkin et al (2009). Because Z is an unknown random variable, the EM
approach in conjunction with the maximum likelihood method can be applied forparameter estimation. We assume that the distribution of Z is unspecified for all
the model adjustments and for estimation purposes we used the Nonparametricmaximum likelihood (NPML) implementation of Einbeck and Hinde (2006).

2

The gradient test

Consider including in (1) the interaction between storage and time, i.e. testing the null hypothesis H0 : ((τ β)22 , (τ β)23 )> = 0. Let ` be the total loglikelihood and θ = (θ1> , θ2> )> the vector of fixed effects parameters where θ1 =((τ β)22 , (τ β)23 )> is our vector of parameters of interest and θ2 is a vector ofnuissance parameters. The unrestricted MLE for θ is θ̂ = (θ̂1> , θ̂2> )> and therestricted to the null hypothesis is θ̃ = (θ10> , θ̃2> )> , where θ10> is an arbitrary
vector (which, in our application is equal to 0). From now on the top accents∧ and ∼ represent the MLE unrestricted and restricted to the null hypothesis, respectively. Let U = ∂`/∂θ = (U1> , U2> )> the respective partitioned score
vector. Terrell (2002) proposed the gradient statistic for testing H0 denoted asξT = Ũ1> (θ̂1 − θ10 ). Note that ξT does not have any matrix computation in its
formula which turns to be its main advantage. In theory, the reference distribution for ξT is χ2q where q denotes the dimension of θ1 . Because of that, ξT iscomparable to the ξLR , the likelihood ratio statistic. The gradient test statistic
formula for generalised linear models has been defined by Marques da Silva Junior
et al (2016) and can be extended for variance component models. Table 2 showsthe estimates for the test statistics, the chi-squared p values and the equivalentbootstrap version.
TABLE 2. Likelihood ratio and gradient statistics for the null hypothesis. The
p values were computed using the chi-square distribution with two degrees of
freedom and p values∗ uses empirical bootstrap to reproduce the reference distributions.

Statisticp valuep value*3

likelihood ratio

gradient

8.794883
0.01230879
0.01880188

10.25232
0.005939328
0.00730073

Bootstrap and confidence intervals

The main purpose of the bootstrap experiment here is to verify how accurateis the chi-square approximation for the test statistic. We propose therefore abootstrap in two levels taking the model under null hypothesis as true. In thefirst level we resample the estimated random intercepts obeying the respective
estimated probabilities and in the second level we generate responses given theMarques da Silva Junior

73

samplenew intercepts. Then the model in (1) is fitted and both likelihood ratio andgradient statistics are computed. We replicate the procedure 9999 times and theresults can be seen in Figure 1. We can produce confidence regions for θ1 inverting
20

20

15

15

10

10

5

5

0

0
0

5

10

15

20

0

5

theoretical

10

15

20

theoretical

FIGURE 1. Bootstrap samples of the likelihood ratio statistic (left) and gradient statistic (right) compared to the theoretical χ22 for the test with hypothesis
H0 : (τ β)22 = (τ β)23 = 0.

(τ β)23

the gradient test however there is no analytic procedure so far. Numerically,
we took a grid of two sequences of 51 values for each (τ β)jk in the interval(τcβ)jk ± 3se((b τcβ)jk ). Then, we fit the model in (1) with (τ β)jk as offset for eachposition of the grid and compute ξT for H0 . Therefore, the region consists on thevalues of θ1 that satisfy ξT < χ22 (1 − α), where 1 − α is the confidence level. The
same procedure has been done for ξLR . The Figure 2 shows the contour maps forthe 90% confidence regions.

-1

-1

-2

-2

-3

-3

-4

-3

(τ β)22

-2

-1

-4

-3

-2

-1

(τ β)22

FIGURE 2. 90% confidence regions in black for (τ β)22 and (τ β)23 based on the
numerical inversion of the likelihood ratio test (left) and the gradient test (right).

74

4

Gradient test for variance component models

Discussion

The gradient test is an alternative to the classic likelihood ratio test for variableselection on variance component models. Also, the gradient statistic is of simple
computation as it does not require matrix operations in its formula. According toour bootstrap simulations, the empirical distribution of the gradient statistic has
better approximation to the chi-square than the likelihood ratio statistic. The
confidence region using the inverse of the gradient test has smaller area than the
one which uses the likelihood ratio test for our real data example. In summary, ourresults show that the gradient test is preferable for variance component models.

Acknowledgments: We wish to thank J. Einbeck for the discussions andhelpful comments. We gratefully acknowledge grant no 9622/13-6 from CAPES
foundation through Brazil’s Science Without Borders Program.

References
Aitkin, M.A., Francis, B., Hinde, J. and Darnell, R. (2009). Statistical modellingin R. Oxford University Press Oxford.
Einbeck, J. and Hinde, J. (2006). A note on NPML estimation for exponential
family regression models with unspecified dispersion parameter. Austrian
Journal of Statistics 35 233 – 243.
Markussen, B. (2017). Lecture notes for Statistical methods for the Biosciences.
Copenhagen: Department of Mathematical Sciences, University of Copenhagen.
Marques da Silva Junior, A.H., Einbeck, J. and Craig, P.S. (2016). Gradient testfor generalised linear models with random effects. In J.-F. Dupuy and J.
Josse, eds., Proceedings of the 31st International Workshop on Statistical
Modelling, vol. 1. 213 – 218.
Terrell, G.R. (2002). The gradient statistic. Computing Science and Statistics,
34 206 – 215.

Constrained Mortality Forecast
Carlo G. Camarda1
1Institut National d’Études Démographiques, Paris, France

E-mail for correspondence: carlo-giovanni.camarda@ined.fr
Abstract: Forecasting mortality can be achieved by parametric models, LeeCarter variants and smoothing approaches. All of these methods either impose
rigid modeling structures or produce implausible outcomes. In this paper, we
propose a novel approach that combines well established smoothing model as
P -splines and demographic prior information. Specifically, we constrain futuresmooth mortality patterns to lay within a range of reasonable age-profiles andtime-trends, both computed from observed patterns. We enforce these shape constraints by an asymmetric penalty approach on forecast mortality. We illustrate
the proposed approach on England & Wales males.
Keywords: Mortality forecast; Smoothing; Demographic constraints; Age-Timepatterns; Asymmetric penalty.

1

Introduction

Modelling and forecasting mortality is crucial in epidemiology and population
studies as well as in the insurance and pensions industry. In the last few decades,
several methodologies have been suggested.
Currie et al. (2004) proposed a model based on two-dimensional P -splines tosmooth mortality over age and time. Despite P -splines outperforming all competitors in modeling mortality, this approach suffers from all the issues thata purely data-driven approach reveals when employed for forecasting purposes.
Forecast mortality simply follows estimated trends in a slavish manner and mortality structure over age and time is not fully considered in the forecast values.
Unreasonable trends from a demographic perspective could consequently emerge.
This paper aims to enhance two-dimensional P -splines, incorporating demographic knowledge to allow for abetter performance in forecasting mortality
trends.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

762Constrained mortality forecast

Data and model

Suppose that we have mortality data, deaths, and exposures to the risk of death,arranged in two matrices, D = (dij ) and E = (eij ), each m × n1 , whose rows andcolumns are classified by age at death, a, m × 1, and year of death, y1 , n1 × 1,
respectively.
We assume that the number of deaths dij at age i in year j is Poisson distributedwith mean µij eij . The value of µij is commonly named force of mortality andits estimation is the object of all mortality model. Forecasting mortality aims toreconstruct trends in µij for n2 future years, y2 , n2 × 1.
In the following we will illustrate the method for England & Wales, males, ages
0-100, years 1960-2013, aiming to forecast up to 2050 (HMD, 2017).
By arranging data as a column vector, that is, d = vec(D) and e = vec(E), we
model our Poisson death counts as follows: η = ln(E(d)) = ln(e)+B α , where B
is the regression matrix computed as a Kronecker product of B-spline bases over
the two dimensions: B = By1 ⊗ Ba . The coefficients vector α are estimated by apenalised version of the iteratively reweighted least squares (IRWLS) algorithm.
In the original paper by Currie et al. (2004), forecasting is treated as a missing
value problem, and data and B-spline bases are augmented as follows:Ĕ = [E : E2 ] ,D̆ = [D : D2 ] ,B̆ = [By1 : By2 ] ⊗ Ba ,where D2 and E2 are filled with arbitrary future values. If we define a weightmatrix V :V = diag(1(m×n1 ) : 0m×n2 ) ,we can adapted the penalised IRWLS algorithm as follows(B̆ T V W̃ B̆ + P )α̃ = B̆ T V W̃ z̃ ,(1)where a difference penalty P enforces smoothness behaviour of mortality overboth age and years.
Outcomes from this approach are portrayed as dashed lines in Figure 2. It isclear that, given the optimal amount of smoothness selected by BIC, a plain
extrapolation of past trend via the penalised coefficients is not satisfactory.
The aim of this paper is to incorporate prior demographic knowledge by enforcing
future mortality to follow data-driven age-profiles and reasonable changes overtime. Operating at the shape level, we first compute the derivatives of the fitted
linear predictor over the observed years. This can be done by a linear combinationof a modified version of the B-splines over the two domains and the estimatedcoefficients. In formulas:∂η = (By1 ⊗ Ca ) α̂ = Day1 α̂,∂a∂η = (Cy1 ⊗ Ba ) α̂ = Dya1 α̂ ,∂y1where, for instance, the matrix Ca is given byCa =
1  q−1
q−1
Bk (a) − Bk−1(a)hwith h, q and k being the knot-distance, degree and positions of Ba .
Figure 1 presents the boxplots of the derivatives of the mortality patterns foreach age over both ages and years. The idea is to incorporate this information

Camarda

●
●

●
●
● ●
●
●
●

●
●
●

●

●
● ● ● ●

● ●
●

● ● ●
● ● ● ●

● ●

0.02

●
● ●

●

●
●
●

●

●

●

● ●

●

●

●
●

●

● ●

●

●

●

●

●

●

●

●
●

●

−0.5

● ● ●
●

●

●
● ●
●
●

●
● ●

●

●

●

●
●
● ●

●

●

●

●
●

●
●

●

●

●
●

●

●

● ● ● ●

● ● ●
●

0.0

● ●

●
●
●

0.00

●

77

●

●
● ●
●
●

∂
η
∂y1

−1.0

∂
η
∂a

●
● ●
●
● ● ●
●
● ●
●
●
●
●
●
●

−0.02

●
●
●

●

−1.5

●
●

●

●
●
● ●

●
●

−0.04

●
● ● ●

●
●

● ●
● ● ●
● ● ● ●

−2.0
●
● ● ●
● ●

●
●

●
●

●

−0.06

−2.5

●
●

●

● ●

● ●
●
● ●

●

● ●
●

●
● ●

● ●
● ●

●
●

●
●

●
●

● ●

● ● ●
● ●

0

10

20

30

40

50

60

70

80

90

100

0

10

20

30

a

40

50

60

70

80

90

100

a

FIGURE 1. Boxplots for each age of the derivatives with respect to ages (left)
and years (right). England & Wales, males, ages 0-100 years 1960-2013.

in the model for constraining future mortality to lay within a range plausible
age-patterns and time-trends.
Due to space restriction, we describe the model for the age-pattern. Let us denoteaa
the 99% confidence intervals of the derivatives with respect to ages,and δUby δLaa
the repetition of these values over both dimensions.
= 1n1 +n2 ⊗ δL
and by gL
We can enforce our shape constraints by adding two asymmetric penalties within
the system described in (1) (Bollaerts et al., 2006):(B̆ T V W̃ B̆ + P + PLa + PUa )α̃ = B̆ T V W̃ z̃ + paL + paU .

(2)

As example, the penalty terms for the lower bounds are given by
PLa

=

a
κ Day1 +y2 T diag(s · vL
) Day1 +y2

paL

=

a
a
κ Day1 +y2 T diag(s · vL
) gL

where
a
vL
=

(
0
1

if
if

a
Day1 +y2 α̃ > gL,a
Day1 +y2 α̃ < gL

and s is a 0/1 vector equal to 1 when the constraint is to be applied (futureyears). The size of κ regulates how strictly the constraints are enforced. In thispaper, we chose κ = 104 . Similar reasoning is applied for the rate-of-change overyears.

3

Results

Figure 2 presents the outcomes of the proposed model. For comparison we add
the outcomes from a plain P -spline approach and a smooth Lee-Carter variant(Delwarde et al., 2007). Our proposed approach outperforms Lee-Carter modelin fitting observed data and it provides reasonable outcomes in future years.

78

Constrained mortality forecast0●●●●●● ●●●●●●●● ●● ●●●
●●

●

−2

●

year 1985
ActualConstrained P−splines
P−splines
Smooth Lee−Carter

−4
●

●
●
●
●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●●● ●

●

−4

●

●

●

●
●

●

●

●

●●

●

●
●

●●●●

●

●●

●●●

●

●

●
●

●

●

●●

●

● ●●

●● ● ●●●●●●●● ●●●●●

age= 0
●

●
●●●●

●●●

●●● ●

●●●● ●

● ●
● ●

●

−6

●● ●●
●
●●●

●
●

●●
●

●●● ● ●
● ●● ● ●● ●●age=20●● ●●●● ●●● ● ●●●●●●● ●

●●●

●
●

●●

●

●

−8

Constrained P−splines
P−splines
Smooth Lee−Carter

●●
●

●●

● ●●●

●

year 2035●●●

age= 70
●● ●

●●●

●

●

−8

●

● ●●
●● ●●●
●

●

●

●●

●

●

●●

●

●●●

●

●

●

●●●●

●

●

●

●

●

●

●●●●●

●

●

●

●●

●

●

●●

●

●
●
●

●

−6

● ●●●●●●●

η, log−mortality●

η, log−mortality

●
●

●

●

●

●

ActualConstrained P−splines
P−splines
Smooth Lee−Carter

−10
−10
0

20

40

60

ages

80

100

1960

1980

2000

2020

2040

years

FIGURE 2. Actual, modelled and forecast death rates (log-scale) over ages forselected years (left) and over years for selected ages (right) by the proposed model
as well as by plain P -spline approach and a smooth Lee-Carter variant. England& Wales, males, ages 0-100 years 1960-2013, forecast up to 2050.

References
Bollaerts, K., Eilers, P. H. C., and van Mechelen, I. (2006). Simple and multiple
P -splines regression with shape constraints. British Journal of Mathematical and Statistical Psychology, 59, 451-469.
Currie, I. D., Durbán, M. and Eilers, P. H. C. (2004). Smoothing and Forecasting Mortality Rates. Statistical Modelling, 4, 279-298.
Delwarde, A., Denuit, M., and and Eilers, P. H. C. (2007). Smoothing the LeeCarter and Poisson log-bilinear models for mortality forecasting: A penalized log-likelihood approach. Statistical Modelling, 7, 29-48.
Human Mortality Database (2016). University of California, Berkeley (USA),and Max Planck Institute for Demographic Research (Germany). Available at www.mortality.org. Data downloaded on January 2017.

Prior Specifications to Handle Monotone
Likelihood in the Cox Regression Model
Enrico A. Colosimo1 , Frederico M. Almeida1 , Vincius D.
Mayrink1
1

Universidade Federal de Minas Gerais, Belo Horizonte, Brazil

E-mail for correspondence: enricoc@est.ufmg.br
Abstract: The phenomenon of monotone likelihood is observed in the fitting
process of a Cox model when the likelihood converges to a finite value while atleast one parameter estimate diverges to infinity. A penalty solution suggested byHeinze and Schemper (2001) leads to finite parameter estimates by means of penalized maximum likelihood estimation. In this paper, we explore other penalties
for the partial likelihood function in the flavor of Bayesian prior distributions.
This work was motivated by a real situation involving a melanoma skin data set.
Keywords: Firth correction, MCMC, Partial likelihood, Survival analysis.

1

Introduction

The proportional hazards model (PHM) is probably one of the most importantstatistical methods for the analysis of censored data. When fitting the PHM tosome data sets, one may observe a phenomenon known as monotone likelihood orseparation (Bryson and Johnson, 1981). The monotone likelihood tends to occur
associated to one category of a categorical covariate. In this sense, the larger thenumber of dichotomous regressors included in the model, the higher is the chanceof monotone likelihood.
This paper was motivated by the analysis of a melanoma data set (Cherobin, etal., 2017). Melanoma is a neoplasm that shows high mortality when diagnosedin advanced stages. The PHM analysis was used to assess factors associatedwith time until metastasis occurrence. However, presence of mitosis, one of mostimportant covariate, had lack of metastasis in those tumors without mitosis in
the histologic exam; this means the occurrence of monotone likelihood.
A solution suggested by Heinze and Schemper (2001) is based on the procedureof Firth (1993). This method produces finite parameter estimates by means ofpenalized maximum likelihood estimation. Penalization is a very general method
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

80Prior Specifications to Monotone Likelihoodof stabilizing estimates, which has both frequentist and Bayesian rationales. Firthmethod is a well known example of penalty which can be derived from a Jeffreys
type of prior in Bayesian inference. However, this approach has some drawbacks,
especially biased estimators and high standard errors (Greenland and Mansournia, 2015).
The goal of this paper is to propose and compare others penalties for the partial
likelihood function in the flavor of prior distributions in the Bayesian context.

2

Notation and Cox Regression Model

The Cox regression model uses the exponential formulation for the hazard function:
λ(t) = λ0 (t) exp(β T x),(1)
in which λ0 (t) is a baseline hazard function (an unknown non-negative functionof time), β is a p × 1 vector of unknown parameters (to be estimated), andx = (x1 , x2 , . . . , xp )T is a covariate vector.
The estimation of coefficients β in Eq. (1) is based on the partial log-likelihood
function:



n
X
X
δi β T xi − log exp(β T xj ) ,(2)l(β) = logL(β) =i=1

j∈R(t )
iin which R(ti ) = {k : tk ≥ ti } is the risk set at time ti , δi is the failure indicator(δi = 1 means failure and δi = 0 means censoring), and xi = (xi1 , xi2 , . . . , xip )T
corresponds to the covariate row vector for the i-th individual. The Maximumb of β is obtained by maximizing Eq. (2).
Partial Likelihood Estimator (MPLE; β)The Firth method for bias reduction estimates β as the maximum of the l(β) =logL(β) penalized by r(β) = log |I(β)|−1 . That is,l∗ (β) = l(β) + (1/2) log |I(β)|.
The augmenting term 1/2 log |I(β)| is the log of a Jeffreys prior, apart from a
constant, and thus the maximizer of l∗ (β) is the posterior mode given this prior.
Other penalties structures have been proposed recently by Greenland and Mansournia (2015). In this paper, the main interest is to evaluate the properties of
the Jeffreys prior and propose other priors in the monotone partial likelihood
situation.

3

Prior Distributions and Bayesian Approach

Partial likelihood modeling version based on Eq. (2) can be used for a Bayesiananalysis. The Bayesian inference using the Cox’s partial likelihood is the topic ofSinha et al. (2003). According to the authors the regression coefficients can bewell estimated in this situation.
The main focus of our study is to investigate the impact of different prior specifications for the coefficients associated with the covariates included in the Cox
regression model used to fit a data set configured with the monotone likelihood
scenario; the following cases are investigated:Colosimo et al.

81• βi ∼ N(m, v), with mean m variance v, for i = 1, . . . , p. Typically, one may
choose m = 0 and v > 0 small/large to explore informative/vague initial
uncertainty.
• βi ∼ Log-F(l1 /2, l2 /2) with l1 and l2 being the degrees of freedom of theoriginal F distribution. Higher variability is associated with small values ofl1 and l2 .
The joint posterior distribution of β, given the observed time points and covariates, does not have a closed form, i.e., a proper probability density cannot be
identified via Bayes’ rule in this case. Markov Chain Monte Carlo (MCMC) algorithm called Gibbs Sampler methods are required to sample from this unknown
target distribution.

4

Melanoma Data Revisited

The prognostics factors under consideration are: (i) Gender β2 (Female); (ii) Histological Type (HT ) with the following levels β31 (Nodular ), β32 (Acral lentiginous) and the extensive malign+superficial spreading level assumed as reference;(iii) Breslow index (CB ) with levels β41 (1 − 4mm), β42 (> 4mm) and (< 1mm)
assumed as the reference level; (iv) Ulceration β5 (Yes). The fifth factor, β1 (mitotic rate or mitosis), is the one associated with the monotone likelihood issue.

β̂1
β̂2
β̂31
β̂32
β̂41
β̂42
β̂5

Standard approach
Cox
Firth
18.52
2.2214-0.651-0.64140.984
0.95780.236
0.29780.110
1.03560.184
1.75980.699
0.6848N (0, 1)1.4033-0.5891.03760.17210.7312
1.42290.7925N (0, 5)2.7724-0.6327
1.07700.2332
1.0612
1.78230 .7594

Bayesian approach
log-F(1, 1) log-F(2, 2)
3.69092.4310-0.6298-0.6203
1.04871.04650.21520.21141.06020.92211.7983
1.64860.73320.7656TABLE 1. Parameter estimates for the Melanoma data set.
Tables 1 and 2 present the main results. As expected, the estimates for the parameters not related to the monotone likelihood issue are very similar for allcases. The most reliable prior distributions, N (0, 5) and log-F(1, 1) (based on
simulation studies, not presented), provide estimates larger than those of Firthcorrection, and standard errors slightly smaller than the one for the standard approach. Indeed, Mitosis is not an important factor according to the Firth method(z = 2.2214/1.5003) and it is closely significant when using the Bayesian approachwith prior N (0, 5).

References
Bryson, M. C., Johnson, M. E. (1991). The Incidence of Monotone Likelihood in
the Cox Model. Technometrics, 23, 381–383.

log-F(9, 9)
1.0801-0.531
0.99400.14880.5263
1.20670.7844

82

Prior Specifications to Monotone Likelihood

σ̂1
σ̂2
σ̂31
σ̂32
σ̂41
σ̂42
σ̂5

Standard approach
Cox
Firth
5131.01.50030.356
0.35540.449
0.45110.634
0.61830.692
0.66570.734
0.71470.385
0.3872N (0, 1)0.6956
0.33420.3872
0.53340.48750.53010.3574

N (0, 5)1.42720.35370.45350.6292
0.66250.69530.3910

Bayesian approach
log-F(1, 1) log-F(2, 2)
2.32151.42290.34840.34790.44360.44050.60710.58710.61350.58940.65050.63180.38590.3756log-F(9, 9)0.55990.31490.36670.45150.40210.43290.3390TABLE 2. Standard error estimates for the Melanoma data set.
Cherobin, A. C. F., Wainstein, A. J., Colosimo, E. A., Goulart, E. A. (2017). Prognostic factors for metastatic melanoma An. Bras. Derm. (in press).
Firth, D. (1993). Bias Reduction of Maximum Likelihood Estimates. Biometrika,
80, 27–38.
Greenland, S., Mansournia, M. A. (2015). Penalization, Bias Reduction and Default Priors in Logistic and Related Categorical and Survival Regressions.
Statistics in Medicine, 34, 23, 3133–3143.
Heinze, G., Schemper, M. (2001). A Solution to the Problem of Monotone Likelihood. Biometrics, 57, 1, 114–119.
Sinha, D., Ibrahim, J. G., Chen, M. H. (2003). A Bayesian Justification of Cox’s
Partial Likelihood. Biometrika, 90, 3, 629–641.

Spatially Balanced Sampling
Jennifer Brown1 , Blair Robertson1 , Trent McDonald2
12

University of Canterbury, New ZealandWestern Ecosystems Technology Inc, USA

E-mail for correspondence: Jennifer.Brown@canterbury.ac.nz
Abstract: Spatially balanced sampling is an emerging area in statistical sampling. These designs are popular because they are one way to ensure the selected
sample has spatial coverage over the entire survey area. This feature of spatial
coverage aids in the resultant sample being representative of the population ofinterest.
One of the first and the most commonly used spatially balanced design is called
GRTS (Generalized Random Tessellation Stratified sampling) where sample effort
is spread evenly over the target region. The term spread evenly in this context
means having coverage of survey effort over the region. The coverage from GRTS
has a stochastic component rather than a fixed interval, regularly spaced coverageas in a systematic sampling design.
We have extended the idea of GRTS to a new design called Balances AcceptanceSampling (BAS). The BAS design allows surveys to be balanced in dimensions
higher then two (n - dimensional space). Until now, most designs have consideredbalance in 2-D geographic space. With BAS we can achieve balance in 3-D space,or in higher dimensions. In some applications these dimensions can be featuresother than the spatial measures of geographic location, and the design allows
aspects such as time for repeat surveys to be incorporated into sample balance.
Keywords: Sample; Environmental Sampling; Spatial Sampling.

1

Introduction

Spatial sampling is a broad category referring to designs that incorporate spatial reference in site selection. The purpose of sampling is usually to estimate
a population parameter, such as the mean or total of some characteristic of interest. For example, in environmental applications surveys may be to estimate
the density or abundance of a plant species of interest. In social science applications the survey may be to estimate household income within a city. Interest in
spatial sampling has increased because of the availability of georeferenced data,
computation capacity, and the overarching need for surveys to be cost-efficient.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

84

Spatial sampling

The gold standard of survey designs is simple random sampling (Cochran 1977),
where selection of individual units in the sample is at random. In the above
examples, individual units could be a quadrat or plot within which plants arecounted, or, individual households from which household income is recorded. A
spatial sample would appear very similar to a simple random sample except thatthe selection process would use information on the spatial location of the sample
units (Wang et al. 2012).
A feature of many natural and social systems is that characteristics observed in
one area are likely to be similar to the ones observed in an adjacent area. For
example, the counts of plants in one plot will be similar to the counts in an adjacent plot (Grafström et al. 2012, Stevens and Olsen 2004). This is because of theunderlying biotic and abiotic processes that drive the species distribution. In human populations individual neighbourhoods tend to have more similar householdincomes than are observed among neighbourhoods, because of socio-economic
factors. One way to view this is that there is limited new information provided
from a sample unit that is spatially adjacent to another unit that has alreadybeen measured. This motivated development of survey designs where the sample
is evenly, or near to evenly, spread over the study area. A design that generates
samples that are well-spread over the population is called a spatially balanced
survey design.
There are many different methods of spatially balanced survey designs (Wanget al. 2012). The design that stimulated the most interest in spatially balanced
sampling, and began the use of this phrase, is Generalized Random Tessellation
Stratified sampling (GRTS), developed by Stevens and Olsen (2003). In this design an invertible mapping technique is used to transform two-dimensional space
into one-dimensional space. Then, a systematic sample is selected along the linear representation. Sampling location geo-references are generated from selecting
points at regular intervals in this one-dimensional space (Brewer and Hanif 1983).
The one-dimensional space is then mapped back to the two-dimensional original
space. By maintaining the spatial properties of the original units, the resultant
sample is spatially balanced, with neither no one area being over-represented withhigh sample intensity nor under-represented with low sample intensity.

2

Other spatially balanced designs

Local Pivotal Method (LPM) is a design for spatially balanced sampling (Grafström et al. 2012). The method is based on a method introduced by Deville andTill (1998). The algorithm involves sequentially updating each point’s inclusion
probability. Starting with N points, neighbouring points compete to be includedin the sample. The winning point of each competition has its inclusion probability increased and the losing point has its decreased. Eventually n points haveinclusion probabilities of one and N-n points have inclusion probabilities of zero.
The resultant sample will have points separated spatially because it is unlikely
two adjacent points will be included in the final sample of size n. The design canbe computationally heavy with large N, and an alternative method, suboptimal
LPM, was suggested for these situations (Grafström et al. 2014), where only asubset of neighbours are used for the comparison, rather than all possible points.
Balanced acceptance sampling, BAS, (Robertson et al. 2013) is a more recent
design for spatially balanced sampling. The method uses the Halton sequence,Brown et al.

85

a quasi-random number sequence (Halton 1960). In two-dimensional geographic
space, Halton points are used to generate the geo-referenced locations of thesample units (starting from a randomly chosen position in the sequence). The
design uses acceptance/rejection sampling to select sample units. If a generated
sample point is beyond the edge of the sample space the sample unit is rejected,
otherwise it is accepted. The design is straightforward computationally, and has
better spatial balance than the comparable GRTS design (Robertson et al. 2013).
The algorithm can be extended into more than two dimensions (e.g., up to five),and this is an appealing feature for some survey situations. One application isfor surveys that involve repeat samples of the same population. Environmental
monitoring is an example, where interest is in how population parameters changeover time. This involves repeat visits to the survey area, often on an annual basis.
Other examples can be in social economic surveys where there is interest in how
indicators are changing over time. In these situations, the design can be viewed
as having three dimension, the two dimensional georeferenced space and the third
dimension that is time. Three dimensional spatial balance can be thought of as away of ensuring that for any one survey there is spatial balance, and no one area
is excessively over- or under-sampled. In addition, over the course of the repeat
surveys (e.g., annual surveys for 10 years), there is no one area that is excessively
repeatedly over - or under-sampled. Until now many survey designs rely on fixedintervals between site revisits, and here BAS offers a method for randomising therepeat interval.

Acknowledgments: Special Thanks to Peter Jaksons and Naeimeh Abi for
their involvemnt in ongoing research, and helpful suggestions.
References
Brewer, K.R.W. and Hanif, M. (1983). Sampling with Unequal Probabilities. Lecture Notes in Statistics. New York: Springer-Verlag.
Cochran, W.G. (1977). Sampling techniques. 3rd edition. John Wiley & Sons.
Deville, J.C. and Tille, Y. (1998). Unequal probability sampling without replacement through a splitting method. Biometrika, 85, 89 – 101.
Grafström , A, Saarela, S. and Ene, L. T. (2014).Efficient strategies for forest inventories by spreading the sample in auxiliary space. Canadian Journal ofForest Research, 44, 1156 – 11164.
Grafström , A, Lundstom, N.L.P. and Schelin, L. (2012). Spatially balanced sampling through the pivotal method. Biometrics, 68, 514 – 520.
Halton, J.H. (1960). On the efficiency of certain quasi-random sequences ofpoints in evaluating multi-dimensional integrals. Numerische Mathematic,
2, 84 – 90.
Robertson, B.L., Brown, J.A., McDonald, T. and Jaksons, P. (2013). BAS: Balanced acceptance sampling of natural resources. Biometrics, 69, 776 – 784.
Stevens, D.L. and Olsen, A.R. (2003). Variance estimation for spatially balanced
samples of environmental resources. Environmetrics, 14, 593 – 610.
Stevens, D.L. and Olsen, A.R. (2004). Spatially balanced sampling of natural resources. Journal of the American Statistical Association, 99, 262 – 278.

86

Spatial sampling

Wang, J.F., Stein, A., Gao, B.B. and Ge Y. (2012). A review of spatial sampling.
Spatial Statistics, 2, 1 – 14.

A comparison of wind power forecast models
Fraser Tough1
1

Renewable Energy Systems

E-mail for correspondence: fraser.tough@res-group.com
Abstract: The forecast of wind speed and subsequent generated power is required to efficiently balance the electricity grid. Within this paper, a number offorecast models are applied to generate wind power predictions at a location in
Scotland, UK. Conventional statistical models are compared with more complex
machine learning techniques using all historic data as well as subsets of historic
data. Local on-site variables collected from the turbines as well as Numerical
Weather Predictions (NWP) are utilised within the framework.
Keywords: wind power forecasting, auto-regressive integrated moving average;
generalised additive model; linear model; regularisation; neural network; autoregressive neural network

1

Introduction

Once energy is generated it must be used immediately as it cannot be stored
in large quantities. Supply must constantly meet demand to prevent outages.
Therefore, grid operators must plan and balance the network accordingly. Conventional energy sources such as coal and gas can be controlled to a large degree
and grid operators can request that operators of these power plants ramp-up or
ramp-down output in real time. However, for renewable energy sources, the energy produced cannot be controlled to the same degree, although turbines canbe shut down via curtailment to reduce power output. To help manage the grid,
wind farm operators must supply energy production forecasts so that the gridcan be efficiently balanced.
This paper compares a number of forecasting techniques applied to a 30 megawatt(MW) wind farm in Scotland, UK. The models are assessed using Mean Absolute
Error (MAE) over a forecast horizon of 24 half hour periods.

2

Data and methodology

The data available for forecasts can be grouped into two categories: on-site historical data via Supervisory Control And Data Acquisition (SCADA), and Numerical
Weather Prediction (NWP) via an external provider.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

88

A comparison of wind power forecast models

SCADA data is collected via an automated system at the wind farm location.
Wind speeds, power output and turbine availability are recorded on a 10 minuteresolution then aggregated to half hours. From SCADA, lagged variables are ofuse for predictions as are time based variables such as month of the year andhour of the day which can be used to represent seasonal variation via a Fourier
series.
NWP utilises mathematical models to make atmospheric forecasts via computer
simulation. Forecasts are generally made at nodes which may be close but not at
the wind farm location, at a certain height above the ground. Therefore, NWP
wind speed forecasts may not reflect the actual weather conditions at the siteand so are corrected utilising historical SCADA data via the statistical/machinelearning models outlined in Section 3. Once relationships between the outcome
variable and the NWP explanatory variables are established, the NWP forecasts
and lagged SCADA data can be fed into the model to make predictions in realtime.

NWP forecasts
Wind speed
Wind direction
Temperature
Pressure

Historical SCADA data
Wind speed
Turbine availability
PowerTime based
Time of day
Month

TABLE 1. Variables
Note that as future turbine availability is required, wind farm operators must
submit this in advance of any forecasts.

3

Models

A number of methods were selected for model comparisons. Models were trainedon data from the start of 2013 to the end of 2015 using k-fold cross validationfor variable selection. Data from 2016 was used to compute MAEs, as shown in
Figure 2.
– Persistence– Linear models (LM)– Linear models with elastic net regularisation (RLM)– Auto-Regressive Integrated Moving Average (ARIMA)– Generalised Additive Models (GAM)– Neural Networks (NN)– Auto-Regressive Neural Networks (ARNN)
The persistence/naive model is a commonly used benchmark, replicating the last
known power such that P̂t+k|t = Pt . It is generally hard to improve upon thepersistence model when forecast horizons are less than a few steps ahead. Powersof variables as well as interactions were used within the both the LMs and RLMs
to quantify non-linear relationships. ARIMA models with exogenous variables(ARIMAX) were also applied but showed no value over linear models including
AR terms and NWP inputs. GAM smooth terms were represented with thin plate
regression splines. Finally, there was one hidden layer within the NN models andthe number of nodes were chosen to minimise the MAE.

Tough489

Analysis

Power can be predicted in two ways, a one stage process of forecasting power
directly, or a two stage process, forecasting wind speed then applying a transformation to convert to power. Converting wind speed to power can be achieved via
the theoretical power curve, or by estimating the on-site power curve while taking into account local conditions (Stathpoulos et al., 2013). The approach within
this paper utilises only the latter, modelling the relationship between on-site windspeed, power and turbine availability via a Gaussian Process (GP) (see Figure 1)
as it was found that forecasting power directly resulted in higher MAEs. The approach leaves open the opportunity to include other variables. This may be usefulfor forecasts where the transformation from wind speed to power is dependent
on wind direction, due to the surrounding terrain, for example.

FIGURE 1. On-site power curve by availability with superimposed GP modelAs wind speeds are Weibull distributed, the response variable was transformed
to approximate normality by firstly fitting a Weibull distribution to the historical wind speed then dividing 3.6 by the shape parameter, α, rendering the dataapproximately normal (Dubey, 1967). Forecasts were back transformed as necessary.
Figure 2 shows MAEs by forecast horizon. It is clear that models based only on
auto regressive terms (ARIMA and ARNN) do not perform as well as those which
include NWP forecasts, out performing the naive approach only towards latter
forecast horizons. The GAM does not perform as well as the others which may
infer that random variation in the training dataset may have been modelled. RLM
models outperform standard LMs and using more recent data (RLM1 and LM1)
resulted in higher MAEs over all horizons. Note that the effect of regularisation
is more prominent while computing parameters with the smaller dataset. Finally,
the NNET model performed the best, outperforming all others over all forecasthorizons.
5 Summary
The results show that all models tested performed well, with MAEs withinroughly 100kW of each other by horizon 24. While the NNET model did outperform the others, it was only marginally better, is much more complex to interpret
and takes much longer to train. This shows that the relationships between windspeed/power and the explanatory variables can be modelled adequately without

90

A comparison of wind power forecast models

FIGURE 2. MAE by forecast horizon

the need for complex black box models such as neural networks. This can be
carried out using linear models with or without regularisation, utilising powersof the input variables and interactions between them.

References
Stathopoulos, C., Kaperoni, A., Galanis, G., & Kallos, G. (2013). Wind power prediction based on numerical and statistical models. Journal of Wind Engineering and Industrial Aerodynamics, 112, 2538.
Dubey, S. Y. D. (1967). Normal and Weibull distributions. Naval Research Logistics (NRL), 14(1), 69-79.

Beyond regression: what does really affect afootball team on championships
Luiz R. Nakamura1 , Pedro H.R. Cerqueira2 , Thiago G.
Ramires23 , Rodrigo R. Pescim4 , Robert A. Rigby5 , Dimitrios
M. Stasinopoulos5
1

Departamento de Informática e Estatı́stica, Universidade Federal de Santa
Catarina, Brazil
2
Departamento de Ciências Exatas, Escola Superior de Agricultura “Luiz deQueiroz”, Universidade de São Paulo, Brazil
3
Institute for Biostatistics and Statistical Bioinformatics, University of Hasselt,
Belgium
4
Departamento de Estatı́stica, Universidade Estadual de Londrina, Brazil
5
Centre of Communications Technology and Mathematics, London Metropolitan
University, United Kingdom
E-mail for correspondence: luiz.rn@gmail.com
Abstract: In this paper we introduce a new regression model, based on the generalized additive models for location, scale and shape, in order to explain the
points rate of football teams in the end of a championship from the four mostimportant leagues in the world: Barclays Premier League (England), Bundesliga(Germany), Serie A (Italy) and BBVA league (Spain) during three different seasons (2011-2012, 2012-2013 and 2013-2014).
Keywords: Collective sports; GAMLSS; Match analysis; Soccer.

1

Introduction

Football can be considered the most popular and important collective sport in
the world. This importance reflects on several aspects, among them the economic,
since a lot of money is invested on football in different areas, such as betting,sponsorship, television rights and in particular the organization of major events.
In this manner an issue that becomes paramount for managers is the assessmentof which factors affect positive outcomes. To evaluate which variables might havesuch relevance, we model the final points rate results (i.e. the proportion of points
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

92

Beyond regression: what does really affect a football team on championships

scored to the maximum points possible) of each team in different national championships using the measured explanatory features. In order to do this task, we
present a new distribution on support (0, 1), called the logitSHASHo distribution
and develop its regression model based on the generalized additive models forlocation, scale and shape (GAMLSS; Rigby and Stasinopoulos, 2005). We foundthat the new model outperformed the beta regression model and some othermodels on support (0,1).

2

GAMLSS framework

GAMLSS is a very flexible class of semi-parametric regression models that involves a distribution for the response variable and may involve parametric and/or
non-parametric smoothing terms when modelling any or all of the parameters ofthe distribution as functions of a set of explanatory variables. This methodology
is already implemented in the gamlss package (Stasinopoulos and Rigby, 2007) in
R, which includes several distributions with up to four parameters (conveniently
denoted by µ, σ, ν and τ ).
In this work we will use only the parametric version of GAMLSS. Generically,
let Y ∼ D(θ), where D represents the response variable distribution and θ isits vector of parameters of length 4. The parametric GAMLSS model is given
by gk (θk ) = ηk = X k β k , for k = 1, 2, 3, 4, where gk (·) is a known monotonic
link function relating the distribution parameter θk to the predictor ηk , X k is aknown design matrix and β k is a parameter vector of length Jk0 .

2.1

The logitSHASHo distribution

As we can see in Hossain et al. (2016), any distribution on the range −∞ < Z <∞ can be transformed to a restrictive range 0 < Y < 1 using an inverse logit
transformation, i.e. Y = (1 + exp {−Z})−1 . In this paper we propose a new very
flexible distribution to model response variables on the interval from zero to one,with four parameters using the logit transformation.
If −∞ < Z < ∞ follows a sinh-arcsinh distribution (Jones and Pewsey, 2009),
then Y = (1 + exp {−Z})−1 will follow a logit sinh-arcsinh distribution, denotedby Y ∼ logitSHASHo(µ, σ, ν, τ ) for 0 < Y < 1, −∞ < µ < ∞ is the location
parameter, σ > 0 is the scale parameter, −∞ < ν < ∞ determinates the skewnessof the distribution (positive skewness corresponds to ν > 0) and τ < 1 and τ > 1
yield heavier and lighter tails than the normal distribution, respectively.

3

Application

3.1

Data set

The following information about four European leagues (Barclays Premier League,
from England, Bundesliga, from Germany, Serie A, from Italy and BBVA league,
from Spain) were considered: points rate (response variable) and the covariates
season, league, shots (Sh.Pg), shots on goal (ShG.Pg), clean sheet (ClSh), offsides (Off.Pg), dribbles (Drbl.pg), fouled, shots conceded (Sh.con.Pg), tackles(Tack.Pg), interceptions (Int.Pg), fouls (Fou.pg), yellow cards (YC), red cards(RC), possession (Poss) and passes (Pass).

Nakamura et al.

3.2

93

Results and discussion

A stepwise selection of explanatory terms was performed for all parameters andvalues of global deviance (GD), Akaike information criterion (AIC) and Bayesian
information criterion (BIC) were computed in order to compare all fitted models using the following distributions: logitSHASHo, logitSST, logitNO, beta andgeneralised beta. The results are displayed in Table 1 where we can see that thelogitSHASHo model is the best fitted model when we use the GD and AIC values(-656.28 and -636.28, respectively), while the beta model is prefered when we use
the BIC criterion.
TABLE 1. Statistics from the fitted models.

Model

Parameters

df

GD

AIC

BIC

4
2
4
2
4

10
7
8
9
9

-656.28-645.19-646.19-644.72-642.48-636.28-631.19-628.19-626.72-624.48-601.72-607.00-597.09-595.62-593.38

logitSHASHo
Beta
logitSST
logitNO
Generalised beta

As we will see in the residual plots, the logitSHASHo model is prefered. Hence, thefinal model from the logitSHASHo distribution under the GAMLSS parametricframework is given byµ̂=−0.19 + 0.49 ShG.Pg + 0.29 ClSh − 0.13 Sh.Pg − 0.05 Sh.con.Pg

σ̂=

exp [−1.96 + 0.13 ShG.Pg]

ν̂=

0.07 − 0.12 YC

τ̂

=

0.68.

ShG.Pg and ClSh are positively associated to µ. This relationship can be explained using simple game facts: a team that avoids being scored (ClSh) andcreates real chances of scoring (ShG.Pg) has more chances to be successfully.
Sh.Pg, surprisingly, has a negative effect on the location parameter µ. Statistically, this can be explained by using the value of the partial correlation between
Sh.Pg and points rate given ShG.Pg, which is negative (-0.11). In practice, we
may say that just arbitrarily shooting has a negative effect, since in many cases a
football team trying to score from anywhere in the pitch, without presenting any
real risk for the opposite team is less likely to score. The last covariate used tomodel µ is Sh.con.Pg which has a negative effect, since it increases the chances
to concede goals and consequently to lose the match.
The value of the scale parameter σ increases according to the number of ShG.Pg
by the football teams. Further, YC has a negative linear effect on the skewness
parameter ν. Finally, the kurtosis parameter τ is a constant smaller than one,i.e. the final model from the logitSHASHo distribution presents heavy tails.
The worm plot for the residual analysis (Figure 1 (a)), for the model based on
the logitSHASHo distribution does not present any trend (vertical shift, slope,
quadratic or cubic shape), thus it fitted really well the skewness and kurtosis
present in the response variable. As for the beta regression model, we can see
from Figure 1(b) that the residuals present a cubic shape, indicating possible
problems in the kurtosis, and three of the dots are not between the upper dotted

94

Beyond regression: what does really affect a football team on championships

curve, which act as 95% pointwise confidence interval. Hence, we conclude thatthe logitSHASHo model is the best model between the five used in this paper toexplain the current data.

(a)

(b)

FIGURE 1. Worm plots of the (a) logitSHASHo and (b) beta GAMLSS models

References
Hossain, A., Rigby, R., Stasinopoulos, M. and Enea, M. (2016). Centile estimation for a proportion response variable. Statistics in Medicine, 35, 895 – 904.
Jones, M.C. Pewsey, A. (2009). Sinh-arcsinh distributions. Biometrika, 96, 761 –780.
Rigby, R.A. and Stasinopoulos, D.M. (2005). Generalized additive models for location, scale and shape. Applied Statistics, 54, 507 – 554.
Stasinopoulos, D.M. and Rigby, R.A. (2007). Generalized additive models for location scale and shape (GAMLSS) in R. Journal of Statistical Software, 23,
1 – 10.

A Continuous Time Multi-State model for
Prostate Cancer Screening
Tiago M. de Carvalho 1 , Ardo van den Hout 2 , Jonathan Miles
3, Stephen Duffy 3 , Nora Pashayan 1
1

Department of Applied Health Research, University College London, UK,
Department of Statistical Science, University College London, UK
3
Wolfson Institute of Preventive Medicine, Queen Mary’s University London,UK
2E-mail for correspondence: t.marques@ucl.ac.uk
Abstract: We use a continuous-time multi-state model to describe the natural
history of prostate cancer. We apply the model to a two-arm randomized control
trial of prostate cancer screening. Our main interest is to estimate the sojourntime (time between pre-clinical and clinical state). The challenge is that ourobservation scheme is unusual: for each individual we observe either the preclinical state, if the person is screen-detected or only the clinical state if thedisease becomes symptomatic. We show that this can be done using standard
software, by using R package msm.
Keywords: Continuous Time Multi-State Models; Prostate Cancer Screening;
Sojourn Time; Survival

1

Introduction

We present a multi-state model, consisting of four states (healthy, pre-clinical,
clinical and death) applied to interval-censored panel data, in a two-arm randomized controlled trial. We apply this model to prostate cancer screening, using
data from the PLCO (Prostate, Lung, Colorectal and Ovarian) cancer screening
trial (Andriole et al, 2012). We are interested in estimating the sojourn time,
the transition time between pre-clinical and clinical states. This is the temporalwindow of opportunity for early detection of cancer. Sojourn time is useful todetermine the frequency of screening and to estimate the potential benefits andharms of screening. However we only observe for each individual either the preclinical state, for men who are screen-detected (Screen Arm) or the clinical state
for men who are clinically diagnosed (either cancer detected between screens orThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

96Multi-State Model

after the screening program ended or in the control arm). There is also censoring
due to the limited time of follow-up of the trial. We show that this can be doneusing standard software, R package msm.

2

Continuous Time Multi-State Model
123

Healthy

Detectable

Symptomatic

4
Dead

FIGURE 1. The 4-state model for the natural history of prostate cancer
A detailed description of this model can be found in, Jackson (2011) and vanden Hout (2016). We describe here the model briefly. An individual is observed
at times, t1 , ..., tJ , with corresponding observed states, y1 , ..., yJ , and a vector ofcovariates, x1 , ..., xJ . The set of health states is S = {1, 2, 3, 4}, where 1 denotes
healthy, 2 pre-clinical disease, 3 clinical disease and 4 death. Additionally, D ={3, 4} is a set of absorbing states. We define prs (tj , tj+1 ) = P (Yj+1 = s|Yj =r), where r, s ∈ SQand j = 1, ..., J. Then the likelihood contribution for eachindividual is L = Jj=2 Lj , where Lj is given by,
Lj = pyj−1 ,yj (tj−1 , tj ),for yj ∈ {1, 2}. When yJ = 4 this becomes,X
LJ =pyJ−1 ,m (tJ−1 , tJ ) qm,yJ ,

(1)

(2)

m=1,2

since we observe the death time exactly, but we do not know the state immediately
before death. When yJ = 3,
LJ = pyJ−1 ,2 (tJ−1 , tJ ) q2,yJ ,(3)since by assumption a person must be in the pre-clinical state 2, before the clinicalstate 3 . Finally, at the end of follow-up we do not observe whether an individual
is in state 1 or 2. The likelihood contribution then becomes,
LJ = p1,1 (tJ−1 , tJ ) + p1,2 (tJ−1 , tJ ).

(4)

Each Lj is composed of an entry of the transition probability matrix P (t),t = tj+1 − tj . For continuous time multi-state models this matrix equals,
P P (t) =exp(Qt), where Q is a matrix with non-diagonal entries, qrs = exp( βrs xrs )(notice we could have aPdifferent set of covariates for each transition), and diagonal elements qss = − k∈S qsk . If the transition is not possible, then we set a
priori, qrs = 0.
For this analysis, we consider two models. Model I denotes a model without
covariates in the transition intensities. In Model II, we add age as a covariate, forde Carvalho 1 et al.

97

the transition between healthy and preclinical. This imposes a time-dependenttransition intensity from healthy to preclinical.
Estimation is possible using R package msm (Jackson, 2011). The general idea
is to maximize L with respect to the β parameters. For users of other statisticalsoftware, any general optimization algorithm could be used for instance, NelderMead or Broyden-Fletcher-Goldfarb-Shanno.

3

Data

PLCO (Prostate, Lung, Colorectal and Ovarian) cancer screening trial is a multicenter randomized control trial. A total of 76, 685 men were randomized aged
between 55 and 74. Randomization occured between 1993 and 2001. The screening protocol consisted of six annual screens, with a PSA test. The latest publishedreport included results at 13 years of follow-up (Andriole et al, 2012). For the current analysis we use a random sample of 4000 men from the 13 years of follow-up
data.

4

Simulation Experiment

The goal of this simulation experiment is to show that it is possible to obtain
unbiased estimates of the qrs ’s and consequently, of the sojourn time, in thecontext of a two-arm trial. The basic idea of the simulation is for each individual,
to simulate time spent in the k-th state, k ∈ S. If there are two states r and s,
where one could move to, from state k, then the individual moves to the next
state according to the rule, min(Tr , Ts ), where Tk is time until event k. The
Tk ’s can be either Exponential or Gompertz distributed, with parameters givenby βrs = (βrs,0 , βrs,1 ), where βrs,1 is included if age is used as a covariate. We
simulate the screen arm, by imposing a grid of observation times, with yearlytime between observations (as in the PLCO data). We also add censoring, byending follow-up at 13 years. We simulate two situations: Without Mixed Design
denotes the natural history during follow-up, and 2-arm Mixed Design denotes
what we would observe in a screening trial. We do 100 simulations, with 1000
individuals each. In Table 1 we verify that we can obtain an unbiased estimateof the β parameter. The main difference lies in the standard errors, which, as
expected, are larger for the mixed design. We see a similar story for the Model II(results not shown).

5Preliminary Results

We used the PLCO data and the multi-state model to estimate prostate cancer
sojourn time. We estimated a sojourn time of about 1.7 years for both models,with AIC value equal to 12669 (Model I ) and 12653 (Model II ). Model II seems
to be better than Model I, i.e., the addition of the age covariate, did improve themodel. The estimated sojourn time is substantially lower than what we would
expect, given previous literature. This is the case since there is a substantial
amount of missclassification in the control arm, namely, many men in state 3,
are actually in the state 2, as they received the PSA screening test (Pinsky et98Multi-State Model

TABLE 1. Simulation Results for Model I

Parameter

Value

β12,0
β14,0
β23,0
β24,0

-2
-4
-2
-3

β12,0
β14,0
β23,0
β24,0

-2
-4
-2
-3

Mean
Bias
Relat. Bias
Without Mixed Design-2.004 -0.004
0.002-3.998 0.002
0.000-1.993 0.007
0.004-3.010 -0.010
0.003
With 2-Arm Mixed Design-2.003 -0.003
0.002-4.005 -0.005
0.001-1.993 0.007
0.004-3.028 -0.028
0.009

St. Dev.

St. Error

0.034
0.112
0.049
0.081

0.037
0.109
0.051
0.090

0.047
0.159
0.099
0.189

0.051
0.154
0.098
0.201

al, 2005). We aim to extend the model, to obtain an unbiased estimate of thesojourn time in the presence of missclassification.
Acknowledgments The research was funded by the National Institute for HealthResearch (NIHR) Collaboration for Leadership in Applied Health Research andCare North Thames at Barts Health NHS Trust. The views expressed are those of
the author(s) and not necessarily those of the NHS, the NIHR or the Department
of Health.

References
Andriole GL, Crawford ED, Grubb RL et al (2012). Prostate cancer screening
in the randomized PLCO Cancer Screening Trial: mortality results after
13 years of follow-up. J Natl Cancer Inst. 104(2):125-32.
Jackson CH (2011). Multi-State Models for Panel Data: The msm package for
R. J Stat Soft. 38(8).
Pinsky PF, Blacka A, Kramer BS, et al (2010). Assessing contamination and compliance in the prostate component of the PLCO Cancer Screening Trial.
Clin Trials. 7(4):303-11.
van den Hout A (2016). Multi-State Survival Models for Interval-Censored Data.
Chapman & Hall-CRC Monographs on Statistics & Applied Probability.

Multinomial and Discrete survival models toevaluate the performance of students
Hildete Pinheiro1 , Rafael Maia1 , Daniel Barboza1
1

Department of Statistics, State University of Campinas, Brazil

E-mail for correspondence: hildete@g.unicamp.br
Abstract: A great interest in Public Policies in Brazil is the comparison of undergraduate students who come from Public High Schools (Pu HS) with those
who come from Private High Schools (Pr HS). In this paper we analyze the performance of undergraduate students in Calculus I and Physics I. The databaseis from the University of Campinas (Unicamp), one of the top Public Research
Universities in Brazil. First, we used a multinomial model to evaluate the proportion of students approved (in none, Calculus I only, Physics I only and inboth subjects) according to socioeconomic status, demographic characteristics
and pre-enrollment tests (e.g. SAT scores). Then, we studied the number of times
the students took these courses until they pass. We used discrete survival models
with the censored observations being those who did not pass the course until theend of the study. For the survival models, the analysis can be done separately
with two univariate models, one for each course (Calculus I and Physics I), or as
bivariate survival models.
Keywords: Multinomial model; Categorical data; Censored data; Discrete survival model; Survival analysis.

1

Introduction

A topic of great interest to educators and administrators of Universities is theperformance of students in subjects like Calculus I and Physics I. They are usuallyrequired courses for all students in Engineering and Exact Sciences majors in thefirst year of enrollment in the university, but there are many questions regarding
the factors which may contribute more to bad/good performance of the students.
The database of this study is from the State University of Campinas (Unicamp),
located in the state of São Paulo, Brazil, which is one of the top research universities in South America with a highly selective entrance exam. Unicamp has an
entrance exam which consists of two phases. The first phase consists on a multiple
choice test of general knowledge. The second phase consists on open questions ofThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

100Models to evaluate performance of students

Mathematics, Portuguese, Geography, History, Biology, Chemistry, Physics andan Essay Writing. We will consider here only the scores on the second phase.
A great interest in Public Policies is the academic performance of those from
Public High Schools (Pu HS) and those from Private High Schools (Pr HS). In
Brazil, most of middle class students go to Pr HS, because the Public High School
system is not very good. In view of this, Unicamp implemented an affirmative
action program which gives a bonus in the second phase of the Entrance Exam
Score (EES), when the candidate studied all High School years in Public Schools.
Therefore, for the models considered here, the type of High School will be a very
important covariate.
The primary goal of this study is to evaluate the performance of students in
Calculus I and Physics I, especially when they are taking theses courses for the
first time. Therefore, we would like to evaluate, when the students are taking
these courses for the first time, the proportion who failed both, approved only in
Calculus I or only in Physics and approved in both, according to a multinomial
model with nominal response. We will be able to see what are the factors which
can better explain the variation of performance of these students.
Our secondary goal is to model the average number of times a student needto take Calculus I/ Physics I until he/she passes. In this case, we can think ofthis as discrete survival model (Collet, 2003) and when the student did not pass
the course, it is a censored observation. Note that there are university policies
to determine when a student should be dropped out from the university, since
public universities in Brazil are completely free for all enrolled students.

2

Data set

The original data set consists of 16,503 records of students who enrolled at Unicamp from 2009 to 2013. It was selected from the database only students with
Calculus I and/or Physics I as required courses, having about 28% from Pu HS
students and 72% from Pr HS. So, there are records for all the times the student
took the subject with their respective grades. We also have entrance exam scores- EES (e.g., SAT scores) in each subject (Mathematics, Portuguese, Geography,
History, Biology, Chemistry and Physics), some academic variables as well associoeconomic status, which are considered as covariates in the models. Amongthe students who have Calculus/Physics as required courses, there are 1259 students who took only Calculus I or Physics I and 5384 who took both Calculus I
and Physics I. Table 1 shows the number of times the students took the subjects
according to type of High School.
Figure 1 shows the proportion of students approved (in none, only Calculus I,
only Physics I and in both) according to type of High School when they aretaking the courses for the first time. One can see that there are more students
from Pr HS who were approved in both courses and more students from Pu HS
who failed both courses. When comparing those who passed only in one of thecourses, the difference is quite small.

3

Statistical models

The first model considered here is a generalized logit model for multinomial response (Agresti, 2013). Let Yi be a r.v. indicating the number of subjects student

Pinheiro et al.

101

TABLE 1. Number of students taking Physics I and Calculus I by type of HighSchool and number of times taking them.

Subject
Physics I

80

Calculus I

High School
PublicPrivatePublicPrivateNumber of times taking the course
1
2
3
4
5
6 ≥7
1126 269 91 49 17 803275 519 143 60 25 501273 328 113 48 23 853704 602 190 96 31 12
7

Approved in
69.3%

None
Calculus I Only
● Physics I Only
● Both●55.13%

40
20

30

Percentage

50

60

70

●

26.78%

16.32%
12.44%
8.1%

6.28%

0

10

5.64%

Pu HS

Pr HS
Type of High School

FIGURE 1. Bar plots of the proportion of students approved (in none, only
Calculus I, only Physics I and in both) according to type of High School when
they are taking the courses for the first time.

i approved, whereYi ∈ {0, 1, 2, 3}, i.e., 0 if student i failed in Calculus I andPhysics I; 1 if student i was approved in Calculus I only; 2 if student i was approved in Physics I only; and 3 if student i was approved in both subjects. Now,
let πij denote the probability of response j for student i, j = 1, 2, . . . 4. When the
final category is the baseline, the j − th logit is


πij
log
= xi β j , for j = 1, 2, 3(1)
πi4
with xi being the ith row of the design matrix and β j the respective vector of
fixed effect for logic j.
For the second analysis, we can use univariate discrete survival models (Collet,
2003; Hosmer and Lemeshow, 1989) The likelihood function here is written interms of the probability of passing the course (Calculus/Physics) in the j − thtime, j = 1, 2, . . . , k. Let Ti be the number of times student i took the course
until being approved. Here Ti ∈ {1, 2, 3, . . .} is a discrete r.v. Let pj (xi ) = P (Ti =102Models to evaluate performance of students

j | Ti ≥ j − 1, xi ) be the probability of individual i pass the course in the j − thtime given that he did not pass the (j − 1) − th time. The likelihood is given byL(β, γ) =





k X 
X
exp(x0i β )exp(x0i β )δij log 1 − γj
+ (1 − δij ) log γj,(2)

j=1 i∈Rj

where δij = 0, if student i did not pass the course in the j − th time and δij = 1,
exp(xi β )otherwise. pj (xi ) = 1 − γj
, with γj = S0 (j)/S0 (j − 1) and S0 (.) is thebaseline survival.

Acknowledgments: We would like to thank Renato Hirata and Flavio Ferraresi, database specialists at the Admissions Committee at Unicamp, and Silvio
de Souza, database specialist at the Academic Records Division at Unicamp, fortheir support regarding the database.
References
Agresti, A. (2013). Categorical Data Analysis. New York: John Wiley & Sons.
Collet, D. (2003). Modelling Survival Data in Medical Research. London: Chapman & Hall.
Hosmer, P.J. and Lemeshow, B.W. (1989). Applied Logistic Regression. New York:John Wiley & Sons.

A Random Effect Mixture Model for
Repeated Measurements of ordinal variables
Rosaria Simone1 , Gerhard Tutz2 , Maria Iannario1
1
2

Department of Political Sciences, University of Naples Federico II, Italy
Ludwig–Maximilians - Universität München, Germany

E-mail for correspondence: maria.iannario@unina.it
Abstract: The focus of the present contribution is on the modelling of subjective heterogeneity for repeated measurements of ordinal variables, which occur,for example, in questionnaires. The proposal is a multivariate random effects
model based on CUB models, a class of mixture distributions for rating datathat accounts for uncertainty of choices. Model performances are assessed on thebasis of a survey on relational goods.
Keywords: Repeated Measurements; Subjective Heterogeneity; Mixture models

1

Introduction and Preliminaries

Within the framework of ordinal data analysis, a general mixture model with anuncertainty component as considered in Iannario and Piccolo (2016) and Tutz etal. (2016) is specified by:
P (Ri = r|xi ) = πi PM (Yi = r|xi ) + (1 − πi )PU (Ui = r), r = 1, . . . , m(1)where Ri , i = 1, . . . , n, is the observed ordinal response and xi is a vectorof subjects’ characteristics. The unobserved variables Yi and Ui stand for thepure-preference choice and the uncertainty, respectively, both taking values in{1, . . . , m}. The distribution PM (Yi = r|xi ) of Yi can be any ordinal model M ,1is assumed to follow a discrete Uniform distribution.
whereas PU (Ui = r) =m
The general model (1) is a CUP model: C ombination of U niform and P reference
structures. In this contribution, we consider the traditional CUB model of Piccolo (2003), in which the distribution of Yi is the (shifted) Binomial distribution
gr (ξi ) with feeling parameter ξi :!
1
m − 1 m−r
P r(Ri = r|xi ) = πi gr (ξi ) + (1 − πi ) , gr (ξi ) =ξ(1 − ξi )r−1 . (2)mr−1 iThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

104Subjective Heterogeneity and Repeated Measurements

In (2), parameters ξi and πi are linked to the row vector of subjects’ covariates
xi by:
logit(ξi ) = γ0 + xi γ,
logit(πi ) = β0 + xi β.
The mixing proportion πi is referred to as the uncertainty parameter, since 1−πi isthe weight for the Uniform component. Thus, πi measures the individual propensity to adhere to a meditated choice for the response, so that each choice takes
place within the individual. As for any mixture model, πi determines the distinction between the two separate sub-groups in the population: those responding
driven by their feeling/preferences, and the others adhering to a random process
for their choices. In the following a multivariate model is proposed that uses CUB
models for the marginal responses and accounts for individual propensities acrossthe repeated measurements. The model contains a subject-specific random effectin the mixing proportion πi , which is considered fixed across the items. In thisvein, CUB models gain a multidimensional perspective avoiding the shortcomings of an item-by-item analysis. Hierarchical CUB models with random effects
for the feeling component have been introduced in Iannario (2012).

2

The Random CUB Models

We consider several measurements on the same subject on a given latent trait, asin questionnaire analysis. In order to simplify the presentation, covariates are notincluded. Let Ri1 , . . . , RiK denote the responses of the person i to a questionnairewith K items, for i = 1, . . . , n, all measured on the same ordinal scale with m
categories. The RCUB model (Random CUB Model) is defined as the following
mixed model for Ri1 , . . . , RiK :0

P r(Rij = r|η , bi ) = πi gr (ξj ) + (1 − πi )
0

1
,
m

r = 1, . . . m, j = 1, . . . , K,

(3)0

with η = (β0 , ξ1 , . . . , ξK ) and ξj being the CUB feeling parameter for item j. A
subject-specific parameter bi is included in the mixing proportion by:πi =exp(β0 + bi ),1 + exp(β0 + bi )bi ∼ N (0, σ 2 ).

(4)

As usual in mixed models, the bi ’s are assumed to be i.i.d. random variables andobservations are conditionally independent given the random effect:0

P r(Ri1 = ri1 , . . . , RiK = riK |η , bi ) =K
Y0P r(Rij = rij |ηj , bi ),i = 1, . . . , n,j=10

where we set ηj = (β0 , ξj ). Note that σ 2 = 0 yields CUB (CUP) models withuncertainty parameter equal for all items to 1/(1 + exp(−β0 )). The rationale behind RCUB models is that the subject-specific tendency to responses determinedby uncertainty is driven by a subject-specific parameter bi , which is constantacross items. Moreover, the parameter β0 quantifies the common uncertainty
among items and respondents, while σ accounts for the unspecified effects. Thisapproach allows for a parsimonious parameterization of repeated ordinal measurements, since the covariance structure of the variables is explained by thevariance component σ 2 .

Simone et al.

105

As customary for mixed-effect models, estimation of the fixed-effect parameters
0
0
0
0
0
0
0
θ = (η , σ 2 ) , with η = (β0 , ξ ), ξ = (ξ1 , . . . , ξK ) is pursued by integratingout the random parameters bi of the full log-likelihood (Pawitan, 2001), thus
obtaining the (marginal) log-likelihood to be optimized:
L(θ) =nX

log Ii ,

Ii =

Z ∞ Y
K

P r(Rij = rij |ξj , β0 , bi )ϕ(bi ; σ)d bi ,(5)−∞ j=1i=1where ϕ(·, σ) denotes the density function of the normal distribution N (0, σ 2 ).
Then, after simple algebraic manipulations and the application of the GaussHermite quadrature formula, the marginal log-likelihood (5) is maximized bymeans of the quasi-Newton method resulting from the Broyend-Fletcher-GoldfarbShanno (BFGS) algorithm.

3

The case study

In 2014, an observational study was carried out by the Department of Political
Sciences, University of Naples Federico II to examine relational goods. Ratings
were collected on a 10-point Likert scale from 1 = “Never, Not at all” to 10 =“Always, Totally, Absolutely Yes”). We fit an RCUB model to selected variables
and report the results of an item-by item analysis with CUB models. It is seenthat there is a significant variation of the tendency to the uncertainty (σ̂ = 1.93).
TABLE 1. Selected Questionnaire Items.
CUB

Parents
Friends
Neighbours
SafetyFamilycond

4

How often do you speak with at least one of your parents?
How good are your relationships with friends?
How good are your relationships with neighbours?
Do you feel safe in the place where you live?
Does your family easily make ends meet?

RCUB

π̂k

ξˆk

0.216
0.742
0.313
0.227
0.427

0.060
0.169
0.353
0.458
0.341

β̂0
-0.648

ξˆk

σ̂
1.93

0.138
0.140
0.328
0.433
0.324

Final remarks

The RCUB proposal stems from the need of running multi-item analysis by focusing on subjective heterogeneity while maintaining the rationale implied by
CUB models. For future work, this approach will be developed to specify thefeeling component on item-basis to include possible overdispersion, shelter effector, more generally, response-styles.

References
Piccolo, D. (2003). On the moments of a mixture of uniform and shifted binomial
random variables. Quaderni di Statistica, 5, 85 – 104.
Iannario, M. and Piccolo, D. (2016). A generalized framework for modelling ordinal data. Statistical Methods and Applications, 25, 163 – 189.

106

Subjective Heterogeneity and Repeated Measurements

FIGURE 1. Selected Questionnaire Items
Parents

RelFriends

RelNeighbours

0.25

0.25

0.25

0.20

0.20

0.20

0.15

0.15

0.15

0.10

0.10

0.10

0.05

0.05

0.05

0.00

0.00
1

2

3

4

5

6

7

8

9

10

0.00
1

2

3

4

5

6

7

8

9

Safety

10

1

2

3

4

8

9

10

5

6

7

8

9

10

EndofMonth

0.25

0.25

0.20

0.20

0.15

0.15

0.10

0.10

0.05

0.05

0.00

0.00
1

2

3

4

5

6

7

8

9

10

1

2

3

4

5

6

7

Iannario, M. (2012). Hierarchical CUB models for ordinal variables. Communications in Statistics. Theory and Methods, 41(16-17), 3110 – 3125.
Pawitan, J. (2001). In All Likelihood: Statistical Modelling and Inference Using
Likelihood. Oxford: Oxford Science Publications, Clarendon Press.
Tutz, G., Schneider, M., Iannario, M. and Piccolo, D. (2016). Mixture models forordinal responses to account for uncertainty of choice. Advances in Data
Analysis and Classification, DOI:10.1007/s11634-016-0247-9.

A Generic Method for Density Forecast
Recalibration
Michael Richard12 , Christophe Hurlin2 , Jérôme Collet1
1
2

EDF R&D division, OSIRIS department, France
Laboratoire d’économie d’Orléans, France

E-mail for correspondence: michael-m.richard@edf.fr
Abstract: We address the calibration constraint of probability forecasting. We
propose a generic method for recalibration, which allows to enforce this constraint. It remains to know the impact on forecast quality, measured by predictive distributions sharpness, or specific scores. We show that the impact on
Pinball-Loss score expectation is weak under some hypotheses and that it is always positive under more restrictive conditions.
Keywords: Density forecasting; Rosenblatt transform; bias correction

1

Introduction

Due to increasing need for risk management, forecasting is shifting from point
forecasts to probabilistic forecasts. According to T. Gneiting, probabilistic forecasting aims to maximize the sharpness of the predictive distributions, subject
to calibration, on the basis of the available information set. It means we havea multi-objective problem, which is necessarily more difficult. That is why an
important step in building a probabilistic forecast is calibration, either in a specific way, either in a more generic manner. We propose here a generic method,using Probability Integral Transform (PIT). PIT is usually a measure of forecast
miscalibration, we show it can be used to recalibrate it.

2

Principle of the recalibration

One stands in the following case: let E be the set of all possible states of theworld; for each forecasting time j the forecaster knows the current state of theworld e(j), and uses it to forecast. An estimated distribution Ge is associated toeach state of the world.
Then, one can calculate the estimated PIT series, it is the series of the values Ge(j) (yj ) j , where yj is the realization for time j.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

108Recalibration

FIGURE 1. Empirical c.d.f of the estimated PIT series and comparison with thec.d.f of the uniform distribution on [0,1] respectively.

The theoretical c.d.f of the estimated PIT is the following:C(y) : = Pr( G(Y) ≤ y)
= E [ E [ 1{G(Y)6y} /E ] ]=−1e pe Fe ◦ Ge (y)



 R

e∈R

P

if E is a discrete R.V.

p(e) Fe ◦ G−1
e (y) deif E is a continuous R.V.

where pe and p(e) are respectively the frequency of appearance or the p.d.f ofstate e, and Fe and Ge the true and the estimated c.d.f of Y conditionally tothe state e. Obviously, if Ge = Fe for each e, then this c.d.f. is the c.d.f. of theuniform distribution on [0,1].
If there is a bias, as shown in figure 1, we propose to use the biased PIT series
to correct the future forecasts.
That makes sense since :Pr( C ◦ G(Y) 6 y) = Pr( G(Y) 6 C −1 (y))= C ◦ C −1 (y)=yHence for instance, if we are interested in forecasting the τ quantile, we will use
the model estimated for the C −1 (τ ) quantile
It remains to study the impact of such a correction in finite sample, as Siegert
does.

3

Impact on sharpness

We first present some theoretical results and then we show a case of positive bias
correction.

3.1

Theoretical results

Under the two following hypotheses :Richard et al.

109• Fe and Ge are close, i.e ∀x, ∀e, | Fe (x) − Ge (x) | 6 ε with F and G thetrue and estimated distributions respectively• the derivative of Ge (x) are lower bounded on the following interval Ig :=
−1
1
[G−1
e (τ − ε) , Ge (τ + ε)], i.e ∀x ∈ Ig , ∀e, ge (x) 6 ξ, with ge the derivativeof Ge
We prove that the impact of our correction on the absolute value of the PinballLoss score expectation is less than 2 ε2 ξ
Under more restrictive hypotheses :• ∀e, e := (Ge ◦ Fe−1 (τ ) − τ ) ∈ [m ± √12 m ] with m the mean of the e
• ∀e,

1
1
∈ [mf −1 ± √12 mf −1 ] with mf −1 the mean of thefe ◦Fe−1 (τ )fe ◦Fe−1 (τ )We prove that the impact of our correction on the Pinball-Loss score expectationis always positive.

3.2

Illustration• we simulate 10 000 independent realizations of a random variable X, withstandard normal distribution,• For each realization x, one generates a realization of a random variable
Y with Student distribution, with degree of freedom equal to 3, and lag
parameter 2x• we model Y with a simple Gaußian linear model Y = 2X + ε, with ε ∼N(0, 1).

When we study the empirical c.d.f of the estimated PIT series of our simple
Gaußian linear model, it fails to pass the validity test. Nevertheless, the correction
allows to pass the validity test.
Note that it would be possible to pass the validity test with a ”climatological”
forecast, which means to forecast, independently of x, the marginal distributionof Y . This forecast is reliable, but very inaccurate.
TABLE 1. Pinball-Loss score expectation for τ =0.5 and median quantile issued
from different models.

climatological

Gaußian

Gaußian, correctedtrue

E[P L0.5 ]

1.24199

0.74155

0.741520.73720

ratio withtrue model

1.68474

1.00590

1.00586

1

As one can see in table 1, the climatological estimation has a greater Pinball-Lossscore expectation than the other even if it passes the validity test, due to loss of
information. Moreover, we remark that the score obtained with our correction islittle better than the score obtained with the simple linear Gaußian model.

110

Recalibration

References
Bentzien, S., and Friederichs, P. (2014). Decomposition and graphical portrayalof the quantile score. Quarterly Journal of the Royal Meteorological Society,140(683), 1924 – 1934.
Bröcker, J. (2009). Reliability, sufficiency, and the decomposition of proper scores.
Quarterly Journal of the Royal Meteorological Society, 135(643), 1512 –1519.
Gneiting, T., and Katzfuss, M. (2014). Probabilistic forecasting. Annual Reviewof Statistics and Its Application, 1, 125 – 151.
Gogonel, A., Collet, J., and Bar-Hen, A. (2013). Improving the calibration of
the best member method using quantile regression to forecast extreme temperatures. Natural Hazards and Earth System Sciences, 13(5), 1161 – 1168.
Michelangeli, P-A., Vrac, M., and Loukos, H. (2009). Probabilistic downscaling
approaches : Application to wind cumulative distribution functions. Geophysical Research Letters, 36, L11708.
Siegert, S. (2016). Simplifying and generalising Murphy’s Brier score decomposition. Quarterly Journal of the Royal Meteorological Society
Siegert, S., Sansom, P. G., and Williams, R. (2015). Parameter uncertainty inforecast recalibration. Quarterly Journal of the Royal Meteorological SocietyAnalysis of social interaction data
Susan Fennell1 , Michael Quayle2 , James P. Gleeson1,3 , Kevin
Burke1
1

Department of Mathematics and Statistics, University of Limerick, Ireland
Department of Psychology, University of Limerick, Ireland
3
MACSI, University of Limerick, Ireland2E-mail for correspondence: Susan.Fennell@ul.ie
Abstract: VIAPPL is a software platform used to conduct experiments in social
interaction. Participants interact by exchanging tokens. We would like to understand why participants choose certain players to interact with and how thesereasons may change over the course of the experiment. We describe two preliminary models developed for the experimental data that indicate reciprocity,
in-group favouritism and charity may occur in the experiments. We discuss why
these models need to be improved upon.
Keywords: Linear Regression; Logistic Regression; Network Science; Social Psychology

1Introduction

Recent developments in social psychology suggest that social identities and norms
develop and change through social interaction. A person does not only develop
their identity through, and adopt the social norms of, the society that they arein, they can affect change in society and the social norms present. Durrheim etal. (2016a, 2016b) believed there was a need for a new type of experiment thatincorporated this strategic nature of social interaction and so developed VIAPPL(Virtual Interaction APPLication), a software platform for carrying out theseexperiments.
Participants of the experiments are avatars in a game-like environment, and arereferred to as players. They observe other players as nodes in a network, as shownin Figure 1, and they interact by exchanging tokens. At the start of a game players
are randomly allocated to one of two groups, of which members are recognisedby the colour of their node.
Each player begins with a number of tokens, which may depend on the group towhich they have been assigned, and must choose a player to give a token to ateach round of the game (there are 40 rounds in total). At the end of each roundthey observe the network shown in Figure 1. The number beside each node refers
to the number of tokens that player has. Arrows between nodes indicate to whom
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

112

Analysis of social interaction data

FIGURE 1. VIAPPL environment.
each player has given their token in that round. The number of tokens belonging
to each group is shown to the left of the screen. The VIAPPL technology allows
for multiple trials per game but the games considered here have only one trial.
What we would like to discover from these games is whether relationships develop(i.e., do players reciprocate token giving), whether charity occurs (do wealthier
players, in terms of token count, give more to poorer players than other wealthy
players) and whether in-group favouritism is present in the games (do players
give more often to players in their own group than the other group). We havedata from 4 VIAPPL games, each of which has two groups of 7 players who startwith 20 tokens.

2

Methods

In order to investigate whether there is reciprocity, etc. in the games, we consider
what influences the number of tokens given from one player to another overthe course of the game, over a number of rounds or even over one round (i.e.,
whether a player gives a token to another player in a given round). The variableof interest, Yi,j,t , is the number of tokens player i receives from player j at, or up
to, round t. The vector of covariates Xt−1 used in the models for Yt can containany information from rounds 1 to t − 1 of the game.
We consider first Yi,j,t = Yi,j,40 to be the total number of tokens player i receives
from player j over the 40 rounds of the game. In this case the vector of covariates isT
Xt−1 = X39 . A normal linear model for Y is assumed, Y40 ∼ N (X39β, σ 2 ), where2
β is the vector of unknown regression coefficients and σ is the error variance.
The covariates are the number of tokens player i had at the end of round 39, thenumber of tokens player j had at the end of round 39 and the number of tokens
given from player i to player j in the first 39 rounds, as well as a categoricalvariable of which three were considered; the first describes whether players i andj are in the same group; the second whether j and i are the same person, in thesame group but not the same person or in different groups; the third indicatesplayer j’s position in the network relative to player i.

Fennell et al.
Tokens Given

Group

10

15

20

25

30

Round

35

40

−0.5
−1.0

Coefficient

−2.0

−1.2
5

−1.5

1.0
0.0

0.5

Coefficient

−0.4
−0.8

Coefficient

0.0

1.5

0.0

Tokens I Have

113

5

10

15

20

25

Round

30

35

40

5

10

15

20

25

30

35

40

Round

FIGURE 2. Coefficients for each variable in the logistic regression models for the
first game.
The final model included the variable describing the number of tokens player igave to player j and the categorical variable describing whether players i and jwere in the same group. The model for the first game, which has R2 = 0.43, isTokensReceived = 1.82 + 0.59 × TokensGiven − 1.21 × Group.
The model predicts that the more tokens you give to another player, the more
tokens you will receive in return and that on average you will receive at least 1
token more over the course of the game from a player in your group than from aplayer in the other group.
We also considered the case where Yi,j,t is the binary variable describing whether
player i received a token from player j in round t. We assume Yt ∼ Bernoulli(pt ),where the probability of receiving a token, pt , is related to the covariates via
Tlogit(pt ) = Xt−1
βt .
A model was developed at each round t of the game, 6 ≤ t ≤ 40, and thecovariates used were similar to those in the linear model. An exception is thevariable for the number of tokens given from player i to player j; in the modelat round t this variable is the number of tokens given from player i to player j
between rounds t − 5 and t − 1 (instead of between rounds 1 and t − 1). This
restriction to the previous 5 rounds assumes players have limited memory andwill base their decisions on the most recent information.
The variables contained in the final logistic regression models are the number oftokens player i has, the number of tokens given from player i to player j and thecategorical variable that tells us whether players i and j are in the same group.
The coefficients of the variables in the final models for the first game are shownin Figure 2. In the majority of rounds the fewer tokens a player has, the more
likely they are to receive a token, indicating that players are charitable. The more
tokens player i has given to player j in the previous 5 rounds, the more likely
player j is to reciprocate (again in the majority of rounds). The coefficients ofthe group variable are all negative (players are less likely to give to players in theother group) suggesting in-group favouritism is present in the game.

3

Discussion

The linear and logistic regression models indicate reciprocation and in-groupfavouritism are present in the games. The logistic models also show charity occurs.

114

Analysis of social interaction data

However these simple models were used only as a first approach to modelling thedata and can be improved upon.
Players only give one token per round which places a sum constraint on the response variable. This means
P observations are not independent. For example thelinear model should have 14
j=1 Yi,j = 40. We are therefore developing a model
which respects the compositional constraint on the vector Yj,t = (y(1,j,t) , . . . , y(14,j,t) )whose elements describe the number of tokens player j has given to each otherplayer up to, or at, round t.
The linear regression model ignored the temporal aspect of the game. Whilelogistic regression models were constructed at each round and showed how theimportance of each variable changed over time, having many models makes it
difficult to interpret what is happening in the game as a whole. We are now
starting to develop temporal network models to enable more detailed analysis of
the dynamic social interaction structures in the data.

Acknowledgments: The first author is funded by the Irish Research Council.
References
Durrheim, K., Quayle, M., Titlestad, K. & Tooke, L. (2016a). The evolution of
ingroup bias in a dynamic experimental environment: toward a social psychology of movement. viappl.org/wp/pubs.
Durrheim, K., Quayle, M., Tredoux, C. G., Titlestad, K. & Tooke, L. (2016b). Investigating the Evolution of Ingroup Favoritism Using a Minimal Group Interaction Paradigm: The Effects of Inter-and Intragroup Interdependence.
PLoS ONE, 11, 1 – 26. e0165974.

Nonlinear quantile regression with clustereddata
Marco Geraci1 , Bo Cai1
1

University of South Carolina, Columbia SC, USA

E-mail for correspondence: geraci@mailbox.sc.edu
Abstract: In this paper, we develop a novel approach to the estimation of nonlinear quantile functions when the data are clustered. The proposed methods areillustrated using a toy example taken from the nonlinear mixed-effects literature.
Keywords: Growth curves; Pharmacokinetics; Random effects.

1

Introduction

Quantile regression analysis of clustered data is a very active area of research.
Since the seminal work of Koenker and Bassett (1978) on methods for crosssectional observations, there have been a number of proposals on how to accommodate for the dependency induced by clustered (e.g., longitudinal) designs. As
briefly outlined by Geraci and Bottai (2014) and then extensively reviewed by
Marino and Farcomeni (2015), approaches to linear quantile regression with clustered data include distribution-free approaches (e.g., Koenker, 2004) and (pseudo)
likelihood-based approaches. The latter mainly adopt the asymmetric Laplace(AL) density (Geraci and Bottai, 2007, 2014; Farcomeni, 2012).
We examined the statistical literature on nonlinear quantile regression with clustered data (in our review, we did not consider nonparametric smoothing). To
the best of our knowledge, there seem to be only a handful of published articles.
Karlsson (2008) considered nonlinear longitudinal data and proposed weighting the standard quantile regression estimator with pre-specified weights. Wang(2012), taking their cue from Geraci and Bottai (2007), used the AL distribution to define the likelihood of a Bayesian nonlinear quantile regression model.
Finally, Oberhofer and Haupt (2016) established the consistency of the L1 -norm
nonlinear quantile estimator under weak dependency.
In this paper, we propose an extension of Geraci and Bottai’s (2014) linear quantile mixed models (LQMMs) to the nonlinear case.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

116Nonlinear quantile regression with clustered data

TABLE 1. Estimated fixed parameters of logistic curves at quantile levels{0.05, 0.1, 0.5, 0.9, 0.95} by genotype.

2

τ

0.05

β1
β2
β3

16.24
55.18
7.96

β1
β2
β3

8.97
53.31
7.38

0.1

0.5

0.9

Genotype P
16.97 19.51 24.02
55.35 52.98 52.97
8.40
8.08
8.76
Genotype F
9.73 17.77 20.72
51.85 55.06 54.14
6.93
8.10
8.34

0.95
30.66
56.37
9.73
21.48
53.56
8.31

Methods

Consider data from a two-level nested design in the form (xTij , yij ), for j =1, . . . , ni and i = 1, . . . , M , where xij is a given vector of covariates and yij
is the jth observation of the response vector yi = (y11 , . . . , y1ni )T for the ithcluster. We define the nonlinear conditional quantile regression functionQτ (yij |ui ) = fij (βτ , ui , xij ) ,(1)

where f is a nonlinear, smooth function of the p × 1 fixed parameter βτ , theq × 1 random parameter ui , and the covariates xij . For given ui , model (1) canbe equivalently written as yij = fij (βτ , ui , xij ) + τ,ij , where τ,ij ∼ AL (0, στ ),which denotes the AL distribution with location 0 and scale στ . This convenient assumption leads to the quantile restriction Qτ (τ,ij ) = 0. We assume
ui ∼ N (0, Στ ), independently from ij , though, in principle, one can consider
different distributions for the random effects. Our goal is to maximise the (pseudo)
marginal log-likelihoodZX` (βτ , Στ , στ ; y) =log
p (yi |ui , βτ , στ ) p (ui |Στ ) dui
(2)
i

Rq

We applied the Laplacian and importance sampling (IS) approximations to compute the integral in (2). Since these approximations make use of second-order
Taylor expansions, we first smoothed the absolute residuals in the AL’s kernel
exponent using the approximation of Chen (2007). As a result, we obtained approximated log-likelihoods akin to those used in nonlinear mixed models (Pinheiro
and Bates, 1995). The R language (R Core Team, 2016) was used to implement
the proposed methods and to carry out the data analysis in the next section.

3

Growth of Soybean Plants

We presents some results using the Soybean dataset, available in the R package
nlme, which consists of measurements from an experiment to compare growth10 15 20 25 30
0

14

23

34

42

55

69

Time since planting (days)

84

117

●
●
●
●

5

Leaf weight/plant (g)10 15 20 25 30
50Leaf weight/plant (g)Geraci and Cai

●●

●
●
●● ●

20

●
●
●●
●●
●
● ●

30

●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●

●
●
●
●
●●
●●
●●●
●
●
●
●
●●
●●
●
●
●

40

●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●

50

●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●

●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●

●●
●●
●
●
●
●
●●
●●
●
●●

●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●

●
●
●
●
●
●
●
●
●
●
●
●
●

●
●
●
●
●
●

60

70

80

Time since planting (days)

FIGURE 1.
Left plot: Boxplots of average leaf weights per plant by
day since planting. Right plot: Fitted logistic quantile curves at levelτ ∈ {0.05, 0.1, 0.5, 0.9, 0.95} for genotypes P (dashed red line) and F (solid blackline) superimposed to observed measurements (grey dots).

patterns of two genotypes of soybeans: an experimental strain (P) and a commercial variety (F). We considered the logistic function applied by Pinheiro andBates (2000, p.289) to these data and specified the following quantile model:yij =βτ,1 + u1,i+ τ,ij ,1 + exp {(βτ,2 + u2,i − xij )/(βτ,3 + u3,i )}separately for the two strains, where yij is the average leaf weight (g) per plant inplot i = 1, . . . , 48 at xij days after planting, measured on occasion j = 1, . . . , ni .
We assumed a diagonal matrix Στ and then estimated five quantiles at levels{0.05, 0.1, 0.5, 0.9, 0.95} using the Laplacian approximation (similar results wereobtained with the IS approximation).
The estimates of the fixed effects are given in Table 1, while a graphical presentation of the results is shown in Figure 1. Genotype P showed larger asymptotes(β1 ) than genotype F at all quantiles. However, asymptotes were more similar
between genotypes at τ = 0.5. Curves were steeper (as suggested by the reciprocal of β3 ) at lower quantiles in both genotypes, and genotype F had faster
growth than genotype P at all quantiles, except for the median. There was considerable variation in the growth curves among plots and heterogeneity differedby quantiles in both strains.

Acknowledgments: This research has been supported by the National Institutes of Health – National Institute of Child Health and Human Development(Grant Number: 1 R03 HD084807-01A1).
References
Chen, C. (2007). A finite smoothing algorithm for quantile regression. Journalof Computational and Graphical Statistics, 16, 136–164.

118

Nonlinear quantile regression with clustered data

Farcomeni, A. (2012). Quantile regression for longitudinal data based on latent
Markov subject-specific parameters. Statistics and Computing, 22, 141–152.
Geraci, M. and Bottai, M. (2007). Quantile regression for longitudinal data using the asymmetric Laplace distribution. Biostatistics, 8, 140–154.
Geraci, M. and Bottai, M. (2014). Linear quantile mixed models. Statistics andComputing, 24, 461–479.
Karlsson, A. (2008). Nonlinear quantile regression estimation of longitudinal data.
Communications in Statistics-Simulation and Computation, 37, 114–131.
Koenker, R. (2004). Quantile regression for longitudinal data. Journal of Multivariate Analysis, 91, 74–89.
Koenker, R. and Bassett, G. (1978). Regression quantiles. Econometrica, 46, 33–
50.
Marino, M. F. and Farcomeni, A. (2015). Linear quantile regression models forlongitudinal experiments: an overview. METRON, 73, 229–247.
Oberhofer, W. and Haupt, H. (2016). Asymptotic theory for nonlinear quantile
regression under weak dependence. Econometric Theory, 32, 686–713.
Pinheiro, J. and Bates, D. (1995). Approximations to the log-likelihood functionin the nonlinear mixed-effects model. Journal of Computational and Graphical Statistics, 4, 12–35.
Pinheiro, J. C. and Bates, D. M. (2000), Mixed-effects models in S and S-PLUS.
New York: Springer Verlag.
R Core Team (2016). R: A language and environment for statistical computing.
Vienna, Austria: R Foundation for Statistical Computing.
Wang, J. (2012). Bayesian quantile regression for parametric nonlinear mixed
effects models. Statistical Methods & Applications, 21, 279–295.

Statistical Inference in the Duffing Systemwith the Unscented Kalman Filter
Michela Eugenia Pasetto1 , Dirk Husmeier2 , Umberto Noè2 ,Alessandra Luati1
1
2

Department of Statistics, University of Bologna, Italy
School of Mathematics and Statistics, University of Glasgow, Scotland

E-mail for correspondence: michela.pasetto2@unibo.it
Abstract: We investigate the accuracy of inference in a chaotic dynamical system (Duffing oscillator) with the Unscented Kalman Filter, and quantify thedependence on the sample size, the signal to noise ratio and the initialization.
Keywords: Bayesian filtering, Unscented Kalman Filter, Chaotic dynamical system, Parameter estimation

1

Introduction

We focus on the analysis of the deterministic Duffing process, defined as
dx1t /dt = x2t ,dx2t /dt = −(cx2t + αx1t + βx31t ),(1)where x1t and x2t are the position and the velocity, respectively, of the oscillation
at time t, g(x) = αx1t + βx31t is a restoring force, α is the natural frequency ofthe vibration, β the mode of the restoring force (hard or soft spring), and c is thedamping term. The Duffing system (1) describes a periodically forced oscillator
with a nonlinear elasticity, and has been widely used in physics, economics andengineering (Kovacic and Brennan, 2011). A characteristic feature is its chaotic
behaviour, which makes statistical inference challenging. In the present paper we
present an approach based on the Unscented Kalman Filter (UKF).

2

Methodology

The UKF algorithm is a non-linear generalization of Kalman filter which relieson the unscented transform (Julier and Uhlmann (2004)) in order to construct
a Gaussian approximation to the filtering distribution. The UKF performs a
Bayesian estimation of a state-space model:
xt = f (xt−1 ) + ε,yt = h(xt ) + η(2)This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

120

Statistical Inference in the Duffing System with the Unscented Kalman Filter
(a)

(b)

(c)

(d)

FIGURE 1. UKF estimates for the deterministic Duffing system with SNR=31
and n = 1000. (a) Signal estimate. (b) Estimate of parameter α. (c) Estimate ofparameter β. (d) Estimate of parameter c.
where xt ∈ RM is the (hidden) state at time t, yt ∈ RD is the measurement,ε ∼ N (0, Σε ) is the Gaussian system noise and η ∼ N (0, Ση ) is the Gaussian
observation noise. The non-linear differentiable functions f and h are, respectively, the transition and observation models. UKF passes a deterministicallychosen set of points (sigma points) through f to obtain the predictive distribution p(xt |y1:t−1 ). Then, the sigma points are transformed using model h tocompute the filtering distribution p(xt |y1:t ). As suggested in Sitz et al. (2002),
we merge the signal with the parameter vector λ = [α β c]T in a joint state vector
jt = [xt , λt ]T = [(f (xt−1 , λt−1 ) + ε), λt−1 ]T , and yt = h(jt ) + η. In our case,the function f of model (2) is given by the numerical solution of system (1), h is
the identity function, and ε = 0.
(a)

(b)

(c)

(d)

FIGURE 2. UKF estimates for the deterministic Duffing system with SNR=10
and n = 1000. (a) Signal estimate. (b) Estimate of parameter α. (c) Estimate ofparameter β. (d) Estimate of parameter c.

3

Simulations

We simulate system (1) through the ode23 MATLAB function with a stepsize of
integration δt = 0.01 and starting values for the numerical integration [1, 0]. Measurements are obtained from the first component, x1t , by adding observational
noise ηt ∼ N (0, ση2 ) with known variance. The time interval is t = 1, . . . , 20, andthe presented results are averaged over 10 simulations. The UKF algorithm isperformed with the EKF/UKF toolbox of Hartikainen et al. (2011). To investigate
the behaviour of the Duffing process and the UKF performance, we have simulated several scenarios, varying the Signal to Noise Ratio, SNR ∈ {30, 10, 1},

Pasetto et al.
(a)

(b)

(c)

121

(d)

FIGURE 3. UKF estimates for the deterministic Duffing system with SNR=1
and n = 1000. (a) Signal estimate. (b) Estimate of parameter α. (c) Estimate ofparameter β. (d) Estimate of parameter c.
(a)

(b)

(c)

(d)

FIGURE 4. UKF estimates for the deterministic Duffing system with SNR=10
and n = 100. (a) Signal estimate. (b) Estimate of parameter α. (c) Estimate ofparameter β. (d) Estimate of parameter c.
(a)

(b)

(c)

(d)

FIGURE 5. UKF estimates for the deterministic Duffing system with SNR=10
and n = 50. (a) Signal estimate. (b) Estimate of parameter α. (c) Estimate ofparameter β. (d) Estimate of parameter c.

and the sample size, n ∈ {1000, 100, 50} (Figures 1–5). To evaluate the impact
of initialization, we considered different offsets as starting values for the parameters. The offsets are sampled randomly from a Gaussian distribution in which
the mean is defined by a percentage deviation from the true parameter values
and the variance is 10% of the mean (Table 1).

4

Results and Discussion

Figures 1–5 show that the UKF successfully learns the parameters from thenoisy data, and that at the end of the filtering phase the true parameters always

122

Statistical Inference in the Duffing System with the Unscented Kalman Filter

TABLE 1. Impact of the initialization for the deterministic Duffing system fordifferent offsets (as percentage of the true parameter values) in term of Euclidean
norm prior inference and post inference.

α
100%
150%
200%
250%

β

c

Prior

Post

Prior

Post

Prior

Post

1.00
1.52
2.03
2.48

0.05
0.12
0.23
0.65

2.04
3.02
3.90
4.61

0.24
0.50
0.94
2.12

0.10
0.15
0.21
0.25

0.01
0.01
0.02
0.04

lie within the predicted standard error around the estimate. This suggests that
Bayesian filtering offers a successful paradigm for inference in chaotic dynamicalsystems. The prediction uncertainty depends on the sample size n, and the levelof noise, quantified by the SNR. As one would expect, the uncertainty increases
with decreasing n and decreasing SNR, i.e. as information in the data is lost, andour study allows a quantification of this trend. The increase in uncertainty particularly affects the parameter β, which is associated with the nonlinear term andthe source of the chaotic behaviour. Table 1 shows the effect of the initialization,measured in terms of the Eucliedean distance in parameter space. This distanceis consistently reduced in the filtering process, and the posterior distance (after
filtering) is always smaller than the prior distance (before filtering). However,the posterior distance increases with the prior distance, suggesting that a goodinitialisation will improve the inference results.

References
Hartikainen, J., Solin, A., Särkkä, S. (2011). EKF/UKF Toolbox for MATLAB.
http://becs.aalto.fi/en/research/bayes/ekfukf/.
Julier, S. J., and Uhlmann, J, K. (2004). Unscented Filtering and Nonlinear Estimation. Proceedings of the IEEE, 92(3), 401-422.
Kovacic I., Brennan, M. J., eds. (2011). The Duffing Equation: Nonlinear Oscillators and their Behaviour. New York: John Wiley & Sons.
Sitz, A., Schwarz, U., Kurths, J. and Voss, H. U. (2002). Estimation of parameters and unobserved components for nonlinear systems from noisy timeseries. Phys. Rev., E 66(1), 016-210.

Effect of Health Risk Factors on All-Causes
Mortality in Elderly People: A Joined
Survival Analysis using Flexible Parametric
Models of Cohorts coming from Brazil,
Argentina and Italy.
Liciana Vaz de Arruda Silveira1 , Alberto Osella2 , Maria delPilar Diaz3
1

Biostatistics Department. Institute of Biosciences. Sao Paulo State University,
Botucatu, Brazil.
2
Laboratory of Epidemiology and Biostatistics, Istituto di Ricovero e Cura a
Carattere Scientifico (IRCCS) Saverio de Bellis, Castellana Grotte, Italy.
3
Biostatistics Unit, School of Nutrution and INICSA-CONICET, University ofCórdoba, Córdoba, Argentina.
E-mail for correspondence: pdiaz@fcm.unc.edu.ar
Abstract: Aim: to estimate the effect of High Blood Pressure on mortality amongpeople aged 65 years old in three scenarios with different socio-economic background and urbanization process by using flexible parametric survival models.
Methods: Three cohorts coming from Brazil (n=365), Argentina (n=1800) and
Italy (n=2472) were considered and only people with 65 years old included. Timeto death (months) from enrolment and all-causes mortality were considered. Statistical analysis included Frailty Coxs Model and Flexible Parametric Survival
Analysis. Due to the two-level structure of variability multilevel mixed-effect survival regression models were fitted. Main Results: Frailty Cox model showed significant positive effects of age and an effect modification of High Blood Pressure
and Non-Communicable Diseases (NCD) but not effects proportionality. Multilevel modelling evidenced a positive statistically significant effect of Age, NCD
and Smoking. There was also an effect modification of NCD on High Blood Pressure. Overall Multilevel Flexible Models estimates were more conservative withan increase in their precision. Conclusions: High Blood Pressure and NCD areimportant causal components and strong risk factors of cardiovascular mortality
in these countries. The model obtained indicates a common causal model in threecohorts coming from very different environments.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

124Multilevel Survival Models in Elderly Mortality

Keywords: Flexible survival model; Mortality; Elderly; Hypertension; Mixed
model.

1

Introduction

The present work considers three different scenarios and proposes a joined survival analysis of elderly populations coming from different cohort studies conducted in Brazil, Argentina and Italy. The increase in the proportion of elderly
population worldwide is one of the most important demographic facts both indeveloped countries and developing ones. Health problems affecting the elderly
population are generally chronic, then factors associated with lower survival havebeen described and related to the risk of death. The aim of this work was to estimate the effect of High Blood Pressure (HBP) on all-causes mortality amongpeople 65 years old in three scenarios with different socio-economic backgroundand urbanization process by using flexible parametric survival models while considering countries variability.

1.1

Cohorts

Three cohorts coming from Brazil, Argentina and Italy were included. Brazil:
individuals aged 60 years old were enrolled in 2003 and followed-up until 2010 at
Botucatu (São Paulo State). A two-stage random sample (365 subjects) was used
to collect information about socio-demographic characteristics, health status andpersonal history.
Argentina: Patients from a medical care center were enrolled in two cities ofCórdoba province (January 2004 and still ongoing). Complete history aboutchronic diseases was recorded for 1,142 subjects.
Italy: The study was conducted at Castellana Grotte, (Bari province) enrolling
2472 subjects. A structured standard interview was performed to collect information about socio-demographic data, health status and personal history.
Time from date of enrolment to death, migration or end of study (in months),
whichever occurred first. Follow-up ended at December 31st, 2010 in Brazil, June
30th, 2014 in Argentina and June 30th, 2015 in Italy. In this work only individualsof 65 years old were considered.

1.2

Modelling

To explore survival probabilities Kaplan-Meier method was applied. A Frailty Cox
model assuming gamma-distributed latent random effects was fitted to handle
unobserved heterogeneity due to pooling the studies. As proportional hazard
assumption of Cox model was not satisfied, flexible parametric Royston-Parmar
survival models (RPM, Royston & Parmar 2002) were chosen. This models use
natural cubic splines to model baseline g[S0 (t)] within the Aranda-Ordaz familyof link function. Due to the two-level structure of variability (subjects nested
in countries) of the overall dataset, multilevel mixed-effect survival regression

Vaz de Arruda Silveira et al.

125

models were fitted (Crowther et al. 2014), as it follows:log[Hij (t)]

=

0

0

s(log(t)|γ, k0 ) + Xij β + Zi bi +PXs(log(t)|δp , kp )Xijp ,

p=1

=

0

s(log(t)|γ, k0 ) + Xij β + bi ,with Xijp covariate, s(log(t)|δp , kp ) spline and coefficients vector denoted as δp ,random bi intercept (bi ∼ N (0, V )). Gender, age, HBP, Diabetes Mellitus (DM),smoking habit and other NCDs were selected as covariates.
Post-estimation predicted hazards by categories of HBP at 65 and 80 years old
were obtained. All statistical analyses were performed using Stata statistical software, version 14.1 (StataCorp LP, College Station, Texas).

2

Results and Conclusions

We refer to Table 1 for a summary of our main results of modelling. Overall
Mixed-RPM estimates were more conservative with a gain in precision. Figure 1
shows predicted hazard for HBP for five years survival at 65 an 80 years old. There
were increased hazards at 80 years old as evidenced on the vertical axis. Whereas
there was an initial decreased hazard (which spans for different time periods) formen in Brazil and Italy, hazard for Argentina behaved in an irregular way.
The major findings of this study were the positive statistically significant main
effect of Age, Smoking and NCDs on all-causes. There was also a strong effectmodification of NCDs on HBP. These results were more plausible and attenuatedestimates while gain in precision. The modelling process uncover a common causal
model among different environments whereas maintain countries peculiarities,
such as survival probabilities which remain substantially different.
TABLE 1. Frailty Cox Model and Two-level Mixed Survival Parametric Model.
Brazil, Argentina and Italy.

Frailty Cox Model

Multilevel Flexible Model

VariableHR

CI0.95

HR

CI0.95Age
Sex(Female)
HBP(Yes)DM(Yes)NCD(Yes)
Smoking(Yes)HBP*DM(Yes/Yes)HBP*NCDs(Yes/Yes)HBP*Smoking(Yes/Yes)

1.04**
0.68
0.99
0.51
2.15**
1.77*
1.95
4.58**
1.05

1.01-1.08
0.42-1.11
0.59-1.91
0.27-2.13
1.31-3.56
1.05-2.98
0.42-2.12
1.14-18.37
0.51-1.81

1.06*
0.66
1.07
0.95
2.12**
1.61*
1.31
2.30**
1.76

1.03-1.08
0.43-1.13
0.61-1.87
0.54-1.66
1.28-3.45
1.03-2.62
0.74-2.29
1.28-4.11
0.89-2.46

Multilevel Survival Models in Elderly Mortality
Italy

0

10 20 30 40
Survival (months)

50

Hazard Rate
.005
.01
0

0

Hazard Rate
.005
.01

.015

Argentina

0aBrazilHazard Rate
.005
.01

.015

65 years

.015

126

0

20
40
Survival (months)

60

0

20
40
Survival (months)

60

0

20
40
Survival (months)

60

0

20
40
Survival (months)

60

0

10 20 30 40
Survival (months)

50

.015
0

Hazard Rate
.005
.01

.015
Hazard Rate
.005
.01
0

0

Hazard Rate
.005
.01

.015

80 years

Normal Blood Pressure

High Blood Pressure

FIGURE 1. Hazard Rate by Country and Ages 65 and 80 years old for High Blood
Pressure Status estimated using Multilevel Mixed Survival Parametric Model.

Acknowledgments: Special Thanks to all participants in the studies for theirvaluable time and information. This work was supported by São Paulo Research
Foundation (FAPESP, Brazil) - Proc. No.2014/22856-7.

References
Rabe-Hesketh, S. and Skrondal, A. (2008). Multilevel and Longitudinal Modeling
Using Stata. Texas: StataCorp.
Royston P., Parmar M.K.B. (2002). Flexible parametric proportional hazards andproportional odds models for censored survival data, with application toprognostic modelling and estimation of treatment effects. Statistics in Medicine,21, 2175 – 2197.
Crowther M.J., Look M.P., Riley R.D. (2014). Multilevel mixed effects parametric survival models using adaptive Gauss-Hermite quadrature with application to recurrent events and individual participant data meta-analysis.
Statistics in Medicine, 33, 3844 – 3858.

A new approach for clustering of effects inquantile regression
Gialuca Sottile1 and Giada Adelfio1
1

University of Palermo, Dip. di Scienze Economiche Aziendali e Statistiche, Italy

E-mail for correspondence: giada.adelfio@unipa.it
Abstract: In this paper we aim at finding similarities among the coefficientsfrom a multivariate regression. Using a quantile regression coefficients modeling,
the effect of each covariate, given a response (also multivariate) is a curve in themultidimensional space of the percentiles. Collecting all the curves, describing
the effects of each covariate on each response variable, we could be able to assess
if only one or more covariates have same effects on different responses.
Keywords: curves clustering; quantile regression coefficients modeling; multivariate analysis; functional data.

1

Introduction

Looking for curve similarity could be a complex issue characterized by subjective
choices related to the continuous transformation of observed discrete data. Here,the alignment problem is handled introducing a new, simple and efficient procedure, based on a similarity measure between curves. A more general approachis based on the alignment of curves using a target function (Silverman, 1995).
Adelfio et al. (2012) introduced a simple procedure to identify clusters of multivariate waveforms based on a simultaneous assignation and alignment procedure.
This approach has been extended in Adelfio et al. (2016), where the authors focussed on finding clusters of multidimensional curves with spatio-temporal structure, applying a variant of a k-means algorithm based on the principal component
rotation of data. Quantile regression can be used to fully describe the conditional
distribution of an outcome and the effect of the covariates on it (Koenker andBassett, 1978). In Frumento and Bottai (2015), the authors suggest adopting
a parametric model for the coefficient function. They refer to this estimation
approach as quantile regression coefficients modeling (QRCM).
In this paper we provide a new perspective of the curve similarity approachconsidering as curves the coefficient functions after applying a QRCM. A new
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

128

A new approach for clustering of effects in quantile regression

algorithm is proposed to assess if two coefficients functions have the same behaviour. The rest of the article is organized as follows. Section 2 briefly presents
the algorithm, Section 3 shows a simulation study and Section 4 provides conclusions.

2

Methods

In Frumento and Bottai (2015), the authors suggest to adopt a parametric modelfor the coefficient function of a quantile regression. Conversely to standard quantile regression which works in a quantile-by-quantile fashion, in the QRCM framework different quantiles are estimated one at the time. This modelling approachfacilitates estimation, inference, and interpretation of the results, and generally
yields a gain in terms of efficiency. Let us consider a response variable y and aset of q-covariates x, the coefficients β(p) are defined as functions of p ∈ (0, 1)that depend on a finite-dimensional parameter θ,β(p | θ) = θb(p),where b(p) = [b1 (p), . . . , bk (p)]T is a set of k known functions of p. In a multivariate framework, let y = [y1 , . . . , yj , . . . , ym ] be a set of m response variables,
correlated or not, and x be a set of q covariates. Applying the QRCM on each response variable, we estimate the coefficient functions β1j (p, θ), . . . , βqj (p, θ) overthe percentiles. In this paper, starting from the QRCM estimation of curve effects, we propose a new algorithm to identify those covariates with the same
effect on a single response, or, similarly, to identify the responses that are related
by similar effect of a given covariate. In a generic framework, we investigate thesimilarities among n general curves, parametrized by βi (p), i = 1, ..., n. The clustering approach proposed in this paper is based on a new dissimilarity measures
based both on shape and distance. More in the detail, we define a new dissimilarity measure, based on two measures accounting both for the shape and for the
distance. Let βi (p) be the coefficient function approximated by a spline functionsi (p), for p = 1, ..., Np , i = 1, ..., n. Considering two different curves βi (p) and
βi0 (p) with i 6= i0 , we define
0

00
00
dii
shape (p) = I(sign(si (p)) × sign(si0 (p)) = 1)0

diidistance (p) = I(|βi (p) − βi0 (p)| ≤ f (α, dist(p))),where s00i (·) is the second derivative of βi (·) and f (·, ·) is a cut-off function, that
depends on α, a probability value, and dist(p), that is the vector of the distances
between all the pairs of curves for each value of p. Computed the distribution ofdist(p) for each value of p, the cut-off function selects the corresponding α−th percentile vector. Therefore, the proposed dissimilarity measure between two curves
is defined as:
Npi00
1 X h ii0
dii = 1 −dshape (p) · dii(1)distance (p)Np 1=1In the proposed approach, the new dissimilarity measure is used to define adissimilarity matrix, useful for the application of a hierarchical clustering method.

Sottile and Adelfio

3

129

Simulation Study

Let us consider a multivariate scenario in which the quantile function is simulatedasQ(p | x, θ) = β0 (p | θ) + β1 (p | θ)x1 + · · · + βq (p | θ)xq ,where x1 , x2 , . . . , xq are independent U(0, 5) variables and p ∈ U(0, 1). In the firstsimulation scenario, the intercept is modelled as a quantile normal distributionfunction (φ) for its flexibility. Other choices, as suggested in the original paper of
Frumento and Bottai (2015), could be also considered. We use q = 2 covariates
and define three groups of quantile functions
Q1 (p | x, θ) = (1 + φ(p)) + (.5 + .5p + p2 + 2p3 )x1 + (.5 + 2p + p2 + .5p3 )x2 ,Q2 (p | x, θ) = (1 + φ(p)) + (−3 + .5p + p2 + .5p3 )x1 + (−1.5 − p − .5p2 + p3 )x2 ,Q3 (p | x, θ) = (1 + φ(p)) + (.3 − .5p − p2 + 2p3 )x1 + (−.5 + p − .5p2 − p3 )x2 ,

−2

0

−1

1

0

1

2

2

3

3

4

4

Ten responses are generated for each quantile function (Q1 , Q2 , Q3 ). Applying the
QRCM method to these responses we obtained the 30 coefficients curves, namely
curves effect, and their lower and upper bounds, useful to select the optimal
number of clusters, for both covariates. The proposed algorithm is able to selectthe correct number of clusters and to discriminate all the curves effect. In Fig. 1,
the curves for both covariates are represented in the three clusters. Results are
summarized in terms of average cluster distances within clusters, which highlightcloseness of curves, and silhouette widths to assess the cohesion of each curve
compared to the other clusters. In particular, results show a valid clustering ofcurves, since silhouette widths are all greater than 0, and for clusters 2 and 3
these values are greater than 0.5. Moreover, all the average cluster distances arelower than or equal to 0.5.

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

FIGURE 1. The simulated 30 curves (solid black lines) clustered in 3 clusters,
conditioning to the first (left) and the second covariate (right), respectively. Redlines are the mean curves; the shaded areas are identified by the mean lower andupper bands within each cluster. The dotted line corresponds to zero.

4

Conclusions

The proposed method for curve-clustering provides a new perspective for the
study of similarity, since it is used in a context of the coefficient functions of a

130

A new approach for clustering of effects in quantile regression

quantile regression model. The proposed algorithm, can be actually used in any
general context, since the main purpose is finding both close and similar shape
curves. Although in this paper we briefly summarize some results, showing a good
performance of the method, the curve similarity should be performed just on thesubsets of quantiles where the effects are significant. This allows to account fora more comprehensive analysis of the relationship between variables along the
distribution of the outcome variables.

Acknowledgments: This paper has been supported by the national grant of the MIUR
for the PRIN-2015 program, “Prot. 20157PRZC4 - Research Project Title Complex space-timemodeling and functional analysis for probabilistic forecast of seismic events. PI: Giada Adelfio”

References
Adelfio, G., Chiodi, M., DAlessandro, A., Luzio, D., DAnna, G. and Mangano, G.
(2012). Simultaneous seismic wave clustering and registration. Comput
Geosci, 44, 60 – 69.
Adelfio, G., Di Salvo, F. Chiodi, M. (2016). Space-time FPCA clustering of multidimensional curves. In: Proc. of the 48th Scientific Meeting of Ital. Stat.
Soc., 1 – 12.
Frumento, P. and Bottai, M. (2015). Parametric modeling of quantile regression
coefficient functions. Biometrics, 72 74 – 84.
Koenker, R. and Bassett, G., Jr. (1978). Regression quantiles. Econometrica, 46,
33 – 50.
Silverman, B.W. (1995). Incorporating parametric effects into functional principal components analysis. J R Stat Soc Series B, 57, 673 – 689.

Social and material deprivation in the
population aged over 50 in the Czech
Republic in 2013
Ivana Malá
1

University of Economics in Prague, Czech Republic

E-mail for correspondence: malai@vse.cz
Abstract: Finite normal mixture model is used to model the distribution ofindices of deprivation in the Czech Republic. Two composite indicators are analysed (for material and social deprivation) based on the Survey of Health, Ageing
and Retirement in Europe (survey SHARE) for the population aged more than 50
in the Czech Republic in 2013. The maximum likelihood estimates are comparedto the moment method based on L-moments with the use of AIC information
criterion. The numeric approach is used for the evaluation of quantiles of a mixture.
Keywords: L-moments; finite mixture of distributions; SHARE; deprivation

1

Methods

The finite normal mixture density is defined as (for K components)
f (x; ψ) =KX

πj f (x; µj , σj ),j=1where f (x; µj , σj ), j = 1, ..., K are component normal densities and π is a vectorof weights of the components in the mixture. The vector of unknown parameters in the model ψ consists of the parameters (2K parameters) of component
distributions µj , σj , j = P
1, ..., K and K − 1 free parameters πj fulfilling obvious
constraints 0 ≤ πj ≤ 1 ,πj ≤ 1. Usually the maximum likelihood estimates ofparameters are used (in this text the package mixtools, Benaglia et all. (2009) wasapplied). In addition to this process, the moment method of estimation based on
L-moments is proposed. For 3K − 1 parameters in the model the same numberof ”theoretical” L-moments are evaluated by the formula (r = 1, ..., 3K − 1)
Z 1
∗
Lr =
Q(P )Pr−1(P )dP,
0This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

132Social and material deprivation in the Czech Republic

where QP is a quantile function (of the mixture) and Pr∗ is the rth shifted Legendre polynomial. Unfortunately, there is not a closed formula for the quantile
function of the mixture, however the weighted average of component quantiles is a
useful approximation and initial value for a numeric method. Quantiles QP were
evaluated numerically on the grid (as a root of an equation F (QP ) = P , where F isa cumulative distribution function of the analysed mixture) and substituted into
the formula for L-moments. Sample L-moments l1 to l3K−1 were evaluated using
Lmoments package (Karvanen, 2006) and equations Lj = lj , j = 1, ..., 3K − 1were solved with respect to unknown parameters in the model. Standard errorsof estimates were evaluated using bootstrap. Both fits are compared with AIC
criterion.

2

Data and Results

6
5
4
0

1

2

3

density30

1

2

density4

5

6

The Survey of Health, Ageing and Retirement in Europe (SHARE, Börsch-Supan(2016)) is a multidisciplinary and cross-national panel database of micro data ofEuropean population aged over 50. Data from 6 waves (from 2004 to 2015) areavailable at present. The module of deprivation is based on the work of (Adena
at all., 2015). Values of two composite indicators, depsoc for social deprivationand depmat for material deprivation from this module are used to model the distribution of these indices in the Czech Republic in 2013. Depmat is an aggregate
measure of material conditions of individuals aged over 50 in Europe using a
set of 11 items that refer to two broad domains: the failure in the affordabilityof basic needs and financial difficulties. Depsoc is an index for measuring social
deprivation, for this purpose 15 items from the survey were used. Answers withalternative values yes (in case of problems)/no are weighted into composite indices. Both indicators are transformed into 0 − 1 scale, 0 means no deprivationand 1 is the highest deprivation.

0.0

0.4

0.8

material deprivation

0.0

0.4

0.8

material deprivation

FIGURE 1. Histogram and kernel density estimate for the index ofmaterial deprivation; the whole sample (left), positive values (right);
quartiles are given with dashed lines.
In the sample, n =4 091 respondents from the Czech republic aged above 50
are included (2 366 female and 1 725 male) with observed values of indices of

133

2
0

1

density

3

4

Malá

0.0

0.2

0.4

0.6

0.8

1.0

social deprivation

FIGURE 2. Histogram and kernel density estimate for the index ofsocial deprivation; quartiles are given with dashed lines.

interest. A normal mixture with two components was selected for the index ofsocial deprivation depsoc (see Figure 1). For the index depmat there is 41% ofzeroes (no deprivation) in the data (Figure 2). For this reason, the model witha discrete part (value 0 with probability π3 ) and a continuous part of K = 2
normal distributions for positive values. The value of π3 was estimated a relative
frequency of respondents with depmat = 0. Then for both models there are 5
parameters in the model and first five theoretical and sample L-moments mustbe evaluated to obtain five equations Lj = lj , j = 1, ..., 5 to be solved with respect
to unknown parameters. Both (MLE and moment method) approaches can becompared using AIC criterion. Fitted distributions are shown in the Figure 3.
According to the AIC criterion both methods were comparable, however the chisquare test rejects the hypotheses of proper models (P < 0.0001). We consider
both models to be acceptable for a description of probability distribution ofanalysed indices. The models enable identification of subgroups of individuals(Figure 3); two components with estimated expected values 0.113 (L-moments
0.106) and 0.367 (L-moments 0.356) for social deprivation and three with 0 (nomaterial deprivation with estimated probability 0.406) and two components with
0.150 (L-moments 0.169) and 0.381 (L-moments 0.411).
The analysed indices, based on the SHARE data, correspond (at least for the
Czech Republic) to the data regularly published by the Czech Statistical Office.
Percentages of deprived (in all questions included in both surveys) are similar forage groups 50-65, 65+, that are used in official statistics. The mixture model iswell applicable for the modelling of both distributions.

Acknowledgments: The institutional support of a long-term conceptual advancement of science and research number IP 400 040 at the Faculty of Informatics and Statistics, University of Economics is gladly acknowledged.

Social and material deprivation in the Czech Republic

201

density

3

4

134

0.0

0.2

0.4

0.6

0.8

1.0

deprivation

FIGURE 3. Fitted probability distributions of the indices of deprivation; material deprivation solid lines, social deprivation dashed lines.
Blue lines MLE, black lines moment method estimator.

References
Adena, M., Myck, M., and Oczkowska, M. (2015). Innovation for better understanding deprivation index. In: Ageing in Europe - Suporting Policies foran Inclusive Society, De Gruyter.
Benaglia, T., Chauveau, D. and Hunter, D. R. (2009). mixtools: An R Package
for Analyzing Finite Mixture Models Journal of Statistical Software, 32,
1 – 29.
Börsch-Supan, A. (2016). Survey of Health, Ageing and Retirement in Europe(SHARE) Wave 5. Release version: 5.0.0. SHARE-ERIC. Data set.
Karvanen, J. (2006). Estimation of quantile mixtures via L-moments and trimmed
L-moments. Computational Statistics & Data Analysis, 51, 947 – 959.

On generalized additive partial linear models
for correlated data
Roberto F. Manghi1 , Francisco J. A. Cysneiros1 , Gilberto A.
Paula2
1
2

Departamento de Estatı́stica, Universidade Federal de Pernambuco, Brazil
Instituto de Matemática e Estatı́stica, Universidade de São Paulo, Brazil

E-mail for correspondence: giaapaula@ime.usp.br
Abstract: In this paper we discuss estimation and diagnostic in generalized additive partial linear models (GAPLM) for analyzing correlated data. A reweighed
iterative process based on the back-fitting algorithm is derived for the parameter estimation and discussions on the extension of some diagnostic proceduresfor GAPLM are given. A motivating data set is reanalyzed by the methodology
developed in the paper.
Keywords: Diagnostic methods; B-splines; Generalized estimating equations;Semiparametric models.

1

Introduction

Generalized additive partial linear models (GAPLM) (see, for instance, Wang etal.,2014) comprise an important approach for modelling correlated data. Suchmodels have the feature of jointly modelling the mean structure by parametricand nonparametric components, with the information only on the marginal distributions as well as on the within-subject correlation structure. The GAPLM classcombines two well known approaches, generalized estimating equations (GEE)(Liang and Zeger, 1986) and generalized additive models (Hastie and Tibshirani,1990).
The aim of this paper is to discuss estimation and diagnostic in GAPLM. The
model is presented in section 2, whereas an iterative process for the joint estimation of the parametric and nonparametric parameters is derived in section
3. Discussions on the extension of some diagnostic procedures are also given. A
motivating data set is reanalyzed in the last section.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

136

2

On generalized additive partial linear models for correlated data

The model

Let yi = (yi1 , . . . , yimi )T be the mi × 1 response vector for the i-th subject, fori = 1, . . . , n. We will assume that the marginal distribution of yij belongs tothe one-parametric exponential family of distributions, namely yij ∈ EF(µij , φ),where E(yij ) = µij , Var(yij ) = φ−1 Vij , φ−1 > 0 denotes the dispersion parameter
and Vij is the variance function. In addition, one has a link function g(·) suchthatqXg(µij ) = ηij = xT
fk (tijk ),ij β +
k=1

where xij = (xij1 , . . . , xijp )T contains values of explanatory variables, β is a pdimensional parameter vector and fk (·) is an unknown smooth function of somecontinuous
explanatory variable tk . In matrix notation one has ηi = Xi β +PqTTT
k=1 Nik fk , where Xi = (xi1 , . . . , ximi ) , Nik is the k-th incidence matrix forthe i-th subject and fk = (fk (t01k ), . . . , fk (t0rk k ))T with (t01k , . . . , t0rk k ) denoting
the distinct and ordered values of tk . The working correlation matrix (withinsubject) will be denoted by Ri (ρ), where ρ = (ρ1 , . . . , ρs )T , for i = 1, . . . , n,
j = 1, . . . , mi and k = 1, . . . , q.

3

Parameter estimation and diagnostics

Applying the Gauss-Seidel method to solve the penalized GEE the (b + 1)-th stepof the iterative process for obtaining the penalized estimates of β and f1 , . . . , fq ,by fixing ρ and the smoothing parameters α1 , . . . , αq , may be expressed as
β (b+1)

=

n
n
X
X
X
(b+1)
(b)
(b)
(b)
−1
Ni` f`}][XT
XT{
i Wi {zi −i Wi Xi }

=

n
n
X
X
X
(b)
(b)
(b)
(b+1)
−1
[NT
NT
W
N
+
α
M
}
Ni` f`}],{
ikkk
ik Wi {zi −
ik
i

i=1
(b+1)

fk

i=1

i=1

`6=0

i=1

`6=k
1

1

for b = 0, 1, . . . and ` = 0, 1, . . . , q, where Wi = Di {Vi2 Ri (ρ)Vi2 }−1 Di denotes
the weight matrix, zi = ηi + D−1
i (yi − µi ) is the modified dependent responsewith Di denoting a diagonal matrix of derivatives dµij /dηij , Mk are relatedwith the spline basis functions, Ni0 = Xi and f0 = β, for i = 1, . . . , n, j =1, . . . , mi and k = 1, . . . , q. The iterative process above should be alternated withconsistent estimates for ρ and the solution for f = (f1 , . . . , fq )T leads to natural
cubic splines. The smoothing parameters are estimated by the AIC criterion.
Approximate standard errors are derived from the penalized estimated robustTasymptotic variance-covariance matrix for (β̂ T , f̂ )T .
Approximate leverage measures, normal conformal curvatures under some usual
perturbation schemes and approximate standardized Pearson residuals are derived. Some simulation studies, under marginal distributions in the EF as well as
within-subject correlation structures, indicate for a good agreement between theempirical distribution of the residuals and N(0,1).

Manghi et al.

4137

Application

3.5

3.5

For illustration, we will consider a balanced study described in Myers et al. (2002,section 6.5) in which n = 30 rats had a leukemic condition induced. Threechemotherapy treatments were applied in groups of 10 rats, and each rat wasobserved in four time periods. The number of cancer cell colonies, the number ofwhite blood cells (wbc) and the number of read blood cells (rbc) were observed ateach period. Fig. 1(left) describes the dispersion graph between the log(number ofcancer cells) and the number of wbc, and a nonlinear tendency may be observed.
The dispersion graph between the log(number of cancer cells) and the number ofrbc (omitted here) presents approximately a linear positive tendency.

Treatment 1

2.5

1.5

1.0

2.0

1.5

2.0

Nonparametric function

3.0

Treatment 3

2.5

Log(cancer cells)

3.0

Treatment 2

5

10

15

20
Wbc

25

30

35

5

10

15

20

25

30

35

Wbc

FIGURE 1. Dispersion graph between the log(number of cancer cells) and wbc(left) and the simultaneous 95% confidence intervals for f (wbc) (right).
Denoting by yij the number of cancer cells for the i-th rat at the j-th instant,
we propose to explain µij = E(yij ) the following GAPLM: (i) yij ∼ P(µij ), (ii)log(µij ) = β1 Treat1i + β2 Treat2i + γrbcij + f (wbcij ) and (iii) corr(yij , yij 0 ) =0ρ|j−j | , for i = 1, . . . , 30 and j, j 0 = 1, 2, 3, 4, where Treat1 and Treat2 denotes
dummy variables.
The model was fitted in R (codes developed by the authors) and we found theparameter estimates (approximate standard errors) β̂1 = −0.249(0.067), β̂2 =−0.031(0.065), γ̂ = 0.015(0.007) and ρ̂ = 0.508, with f (wbc) being significant.
The smoothing parameter and the effective degrees of freedom were estimated as
α = 636.84 and df(α) = 6.80(3 + 3.80), respectively. Fig. 1(right) presents thesimultaneous 95% confidence intervals for f (wbc). The normal probability plot
with the standardized Pearson residual (Fig. 2(left)) does not present unusual
features and a measure related with rat #28 is pointed out in Fig. 2(right). Thisrat received treatment 3, and has the lowest number of cancer cells and the largest
number of wbc counts.
Therefore, based on the above results, one may conclude that treatment 1 seems
to reduce the average count of cancer cell colonies, whereas the other treatments
do not seem to have significant effects. In addition, the average count of cancer
cells colonies seems to decrease as the number of wbc increases and seems toincrease as the number of rbc increases.

On generalized additive partial linear models for correlated data

0.12

138

0.08
0.06
0.02

0.04

0
−2

0.00

−4

Robust residual2

Conformal curvature

0.10

4

4 : 28

−2

−1

0

1

Quantile of N(0,1)

2

0

5

10

15

20

25

30

Rat

FIGURE 2. Normal probability plot for the standardized Pearson residual (left)
and the index plot of the normal conformal curvature under the case-weight
perturbation scheme (right).

Acknowledgments: Special Thanks to CNPq and FAPESP, Brazil.
References
Hastie T.J. and Tibshirani R.J. (1990). Generalized Additive Models. London: Chapman & Hall.
Liang, K-Y. and Zeger, S.L. (1986). Longitudinal data analysis using generalized
linear models. Biometrika, 73, 13 – 22.
Myers, R.H., Montgomery, D.C. and Vining, G.G. (2002). Generalized Linear Models: with Applications in Engineering and the Sciences. New York: Wiley.
Wang, L., Xue, L., Qu, A. and Liang, H. (2014). Estimation and model selection
in generalized additive partial linear models for correlated data with diverging number of covariates. The Annals of Statistics, 42, 592 – 624.

What is the best approach to analyze
longitudinal bounded scores? Application to
Quality of Life data
Anne-Françoise Donneau1 , Cibele Russo2 , Nadia Dardenne1 ,Murielle Mauer3 , Corneel Coens3 , Andrew Bottomley3 ,Emmanuel Lesaffre4
1

Biostatistics, University of Liège, CHU Sart Tilman, Liège, Belgium
Department of Applied Mathematics and Statistics, University of Sao Paulo,
Brazil3
EORTC Headquarters, Departments of statistics and quality of life, Brussels,
Belgium
4
L-Biostat, KU Leuven, Leuven, Belgium
2E-mail for correspondence: afdonneau@ulg.ac.be
Abstract: The present study compares the linear mixed-effects model and thebeta regression model to analyze longitudinal health-related quality of life bounded
outcome scores.
Keywords: Bounded outcome scores; Longitudinal; Health-related quality of
life.

1

Aims

In medical studies, it is a common practice to assess at prespecified time intervals
the patient’s quality of life (QoL) by means of Likert-type items questionnairethat covers different domains of the QoL (e.g., physical, emotional, social healthissues). Usually, the items are summated and linearly transformed to construct
a bounded score ranging from 0 to 100. The aim of the present study is tocontrast different approaches, namely the linear mixed-effects model and the beta
regression model, to analyze longitudinal bounded outcome scores.

2

Methods

The EORTC QLQ-C30 Fatigue scale and the Social Functioning scale from the537 patients in the EORTC 26981 trial, a randomized multicenter phase III in
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

140

Analysis of QoL longitudinal bounded scores

glioblastoma evaluating the addition of temozolomide (TMZ) to radiotherapy(RT), were compared between the two arms using different statistical approaches
for bounded outcome scores (see, for instance, Aaronson et al., 1993 and Taphoorn
et al., 2005). Quality of life was assessed using the EORTC QLQC30 version 2questionnaire: at baseline; during radiotherapy at week 4; 4 weeks after completion of radiotherapy; at the end of the third and sixth cycle of adjuvant temozolomide; and every 3 months thereafter until disease progression for patients
allocated RT+TMZ, and at equivalent time points for those allocated radiotherapy alone. The statistical approaches selected in the present study included thebeta regression model (for bounded values) from Ferrari and Cribari-Neto (2004)
and the commonly used linear mixed-effects model (for continuous values) from
Verbeke and Molenberghs (2000). Scores were divided by 100 to fit to the condition of application of the beta distribution.

3

Results

The magnitude of the P-values for the treatment difference derived under eachstatistical approach slightly varied at some assessment time points. However, theestimations of the treatment mean scores at each time point were comparable(See Table 1).

TABLE 1. Estimated mean scores in both arms and P-value related to treatment
effect at each assessment time for the Fatigue scale.

Baseline
During RT
After RT
1st follow-up
2nd follow-up
3rd follow-up
4th follow-up-2 Log Lik
AIC
BIC

Linear mixed model
RT RT+TMZ P-value

Beta regression model
RT RT+TMZ P-value

0.36
0.36
0.37
0.33
0.31
0.29
0.30

0.34
0.35
0.36
0.31
0.28
0.30
0.31

0.35
0.40
0.41
0.40
0.32
0.31
0.27

0.76
0.07
0.08
0.05
0.90
0.74
0.72
-305.9-273.9-205.4

0.34
0.39
0.40
0.40
0.30
0.34
0.30

0.99
0.11
0.12
0.04
0.63
0.54
0.90
-785.6-753.6-685.0RT = Radiotherapy / RT+TMZ = Radiotherapy and temozolomide

Donneau et al.

4141Conclusion

The preliminary analysis of these two QoL scales showed that both statistical
approaches led to the same conclusion when considering the treatment effect,
P-values and the mean scores. However, the beta regression model presented abetter model fit for the QoL scales. This indicates that incorporating the bounded
outcome assumption into the analysis methods can improve QoL hypothesis testing. Other models for longitudinal bounded outcome scores, such as truncatedregression model and coarsening approach, will be investigated and compared tothese preliminary results.

Acknowledgments: Second author thanks to Fundação de Amparo à Pesquisa
do Estado de São Paulo - FAPESP, Brazil.
References
Aaronson, N. et al. (1993). The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute,85, 365 – 376.
Ferrari, S. and Cribari-Neto, F. (2004). Beta Regression or Modeling Rates andProportions. Journal of Applied Statistics, 31(7), 799 – 815.
Taphoorn, M. et al. (2005). Health-related quality of life in patients with glioblastoma: A randomized controlled trial. Lancet Oncology, 6(12), 937 – 944.
Verbeke, G., and Molenberghs, G. (2000). Linear mixed models for longitudinal
data. New York: Springer.

Nonlinear mixed-effects models with scale
mixtures of skew-normal distributions
Marcos Antonio Alves Pereira1 , Cibele Maria Russo2
1
2

Federal University of Piauı́, Floriano-Brazil
University of São Paulo, São Carlos-Brazil

E-mail for correspondence: cibele@icmc.usp.br
Abstract: This work presents a mixed-effects nonlinear regression model with aclass of skewed distributions, which provide alternatives to the normal model andother symmetric distributions and may allow better fit to correlated nonlinear
data. Many studies have been proposing models with scale mixtures of skewnormal (SMSN) family of distributions, including skew distributions with heavytailed. A real data analysis with this new proposal is performed.
Keywords: nonlinear model, scale mixtures of skew-normal distributions, skewness, mixed-effects.

1

Introduction

Models with mixed effects can be applied in several areas of knowledge, in particular when data are correlated. Nonlinear models with mixed effects have been
explored recently because of their flexibility to deal with measures in areas suchas economics and pharmacokinetics. Russo et al. (2009) developed a nonlinear
model with random effects assuming elliptic distributions for the random components of the model. In this work we present an asymmetric nonlinear model with
mixed-effects in which the associated error follows a distribution belonging to thescale mixtures of skew-normal (SMSN) family of distributions, which comprises
distributions such as skew-normal, skew-Student-t, skew-contaminated normal,
skew-exponential power and skew-slash and the symmetric versions of these distributions.
A random vector y ∈ Rp follows a SMSN distribution, as discussed in Zeller
et al. (2011), Y ∼ SM SNp (µ, Σ, λ, H), with location vector µ ∈ Rp , matrix
scale Σ positive definite with dimension p × p and skewness vector λ ∈ Rp if its
probability density function (pdf) is given byZ ∞φp (y|µ, κ(u)Σ)Φ1 (κ−1/2 (u)λ> Σ−1/2 (y − µ))dH(u; ν),f (y) = 20This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Pereira and Russo

143

where κ(.) is a strictly positive function, U is a positive random variable withcumulative distribution function (cdf) H(u; ν), being ν supposedly known, d =(y − µ)> Σ−1 (y − µ) is the Mahalanobis distance, φp (.|µ, Σ) is the pdf of the pvariate normal distribution and Φ1 (.) is the cdf of the univariate standard normal
distribution. In the SMSN family, if the vector of parameters λ is null, we will
have a SMN family of distributions.

2

The model

A nonlinear regression model with mixed effects and under the SMSN family of
distributions, where the observed responses vector yi , i = 1, . . . , n, of dimension
mi × 1, according to Vonesh and Carter (1992), can be written as
yi = f (α, xi ) + Zi bi + i ,(1)>

where α = (α1 , . . . , αp ) is the vector of unknown parameters, Zi is a mi × r
matrix of known constants. The random effects bi = (bi1 , . . . , bir )> and the errors
i = (i1 , . . . , imi )> are uncorrelated and follow multivariate SMSN, with null
location vector and covariances matrices D and σ 2 Imi , respectively, and λi = 0.
So we have the model


yi
bi




∼ SM SNqf (α, xi )
0

 
,

Σi
DZ>
i

Zi D
D

 


λyi
,
, H . (2)λbi

2where q = mi + r and Σi = Zi DZ>
i + σ Imi and yi has marginal distribution
SM SNmi (f (α, xi ), Σi , λyi ; H).

3

Maximum likelihood estimation

The estimation of the parameters θ = (α> , σ 2 )> is obtained numerically via the
EM-type algorithm and Fisher scoring algorithm. The model (2), with κ(ui ) =1/ui , can be represented hierarchically as
yi |Ti = ti , Ui = ui

ind.

∼

2
Nmi (f (α, xi ) + ∆i ti , u−1
i σ Imi ),(3)

Ti |Ui = uiind.

HN1 (0, u−1i ) and(4)

h(ui ; ν),

(5)

Ui

∼

ind.

∼

1/2
where HN1 denotes the univariate half normal distribution, ∆i = Σi δ i , δ i =
>
1/2
2
>
>
>
λi /(1 + λi λi )and σ Imi = Σi − ∆i ∆i , thus Zi DZi = ∆i ∆i .
> >> >Let y = (y1> , . . . , yn> )> , u = (u>and t = (t>1 , . . . , un )1 , . . . , tn ) , under the hi-

erarchical representation (3)-(5), the complete log-likelihood function associatedwith yc = (y> , u> , t> )> is given byn`(θ|yc )

∝

1 Xn
ui
− log |σ 2 Imi | − 2 (yi − f (α, xi ))> (yi − f (α, xi ))
2 i=1
σ

ut2i >
2uti+ 2 ∆>∆∆ii (yi − f (α, xi )) −
i
σ
σ2

(6)

144

Nonlinear mixed-effect models with SMSN

c2 = E(Ui T 2 |θ =b yi ), ut
b yi ) and ut
ci = E(Ui Ti |θ = θ,where ubi = E(Ui |θ = θ,
i
i
(k)
(k)
b
b
b
θ, yi ) (see Lachos et al. 2010). Maximizing Q(θ|θ ) = E[`c (θ|yc )|y, θ ] in the(k)k-th iteration of the EM-type algorithm, about θb , we obtainc2 (k+1)σ=b (k+1)
∆
i

=


n 
(k)
(k)> (k)
(k)>
(k)
1X
(k)
(k) (k)> (k)
2
c
b
b
b
b
u
bi b
ri b
ri − 2uti ∆i b
ri + uti ∆i ∆i
and
n i=1
(k)
b i(k)but
ri

c2 (k)uti(k), where b
rib (k) , xi )).
= (yi − f (αAt each iteration of the EM-type algorithm, the estimates of α(k) are obtained
by Fisher scoring algorithm
#
" n
o −1
X n (k) (k)> (k)(k)(k) (k)>
(k+1)
(k)
bi
b + ut
b
b
bi L
b
b
∆
J
α
=α
+
u
b J
i

i

i

i

i=1

#
" n
o
X n (k)>  (k) (k)
(k)
(k)
b
b
b i ∆iri − ut
,
u
bi b
Ji
i=1

b (k) = ∂ 2 f (αb(k) = ∂f (αb (k) , xi )/∂ αb (k) ∂ αb (k)> .
b (k) , xi )/∂ αb (k)> and L
where Jii

The prediction of the random effects bi was obtained by empirical Bayes method.

4

Application

The data from a study of the kinetics of the theophylline anti-asthmatic agent
presented in Pinheiro and Bates (2000) were analyzed with this new proposal andcompared with symmetric distributions, thus, nonlinear models were fitted underskew-normal distribution and skew-Student-t distribution (skew-t) and compared
to normal distribution and Student-t distribution (t), whose fits are presented in
Russo (2010). We consider the models (1) and (2) where α = (lke , lka , lcl ), withyi = (yi1 , . . . , yimi )> the concentration measurement vector Ci (t) and xi = ti =(ti1 , . . . , timi )> the vector representing the time measurements. The nonlinear
model for theophylline concentration using time as a covariate can be represented
by

yij =

lka
Di (elke +lka +lcl ) −tij elke(e−e−tij e )+ij , i = 1, . . . , n and j = 1, . . . , mi .
elka − elke

where Di is the dose aplicated of the drug (in mg/kg) to the ith individual.
We observe that the model under the skew-Student-t distribution presented thelowest standard errors for the parameters lke , lka and lcl , but for the parameterσ 2 the smallest standard error occurred with the skew-normal distribution andaccording to the AIC and BIC information criteria obtained, the more appropriate distribution for this application is the skew-normal distribution because it
presents lower values for these statistics.

Pereira and Russo

5

145

Discussion

The mixed nonlinear model presented is a general case of nonlinear mixed models
that include symmetric and asymmetric distributions. The proposed model can beused when atypical observations or asymmetry are present, due to the flexibilityof the SMSN family of distributions.

Acknowledgments: Special thanks to FAPESP - Fundação de Amparo àPesquisa do Estado de São Paulo, Brazil and to UFPI - Federal University ofPiauı́, Brazil.

References
Lachos, V. H., Ghosh, P. and Arellano-Valle, R. B. (2010). Likelihood based inference for skew-normal independent linear mixed models. Statistica Sinica.
20(1), 303 – 322.
Pinheiro, J. C. and Bates, D. M. (2000). Mixed-Effect models in S and S-PLUS.
Springer.
Russo, C. M., Paula, G. A. and Aoki, R. (2009). Influence diagnostics in nonlinear mixed-effects elliptical models. Comput. Statistics & Data Analysis,
53(12), 4143 – 4156.
Russo, C. M. (2010). Modelos não lineares elı́pticos para dados correlacionados.
PhD thesis (in Portuguese), Universidade de São Paulo, Brasil.
Vonesh, E. F. and Carter, R. L. (1992). Mixed-effects nonlinear regression for unbalanced repeated measures. Biometrics, 48(1), 1 – 17.
Zeller, C. B., Lachos, V. H. and Vilca-Labra, F. E. (2011). Local influence analysis for regression models with scale mixtures of skew-normal distributions.
Journal of Applied Statistics, 38(2), 343 – 368

Modeling spatio-temporal earthquake
dynamics using generalized functional
additive regression
Alexander Bauer1 , Fabian Scheipl1 , Helmut Küchenhoff1 ,Alice-Agnes Gabriel2
1
2

Department of Statistics, Ludwig-Maximilians-Universität, Munich, Germany
Department of Geophysics, Ludwig-Maximilians-Universität, Munich, Germany

E-mail for correspondence: Alexander.Bauer@stat.uni-muenchen.de
Abstract: We present a generalized functional additive regression model for
spatio-temporal functional data representing surficial ground velocities of simulated earthquakes. The data come from a large-scale in silico experiment toinvestigate the effects of physical parameters describing the conditions at thetriggering fault on the large-scale dynamics of earthquakes. We describe the performance of a recently developed efficient inference algorithm on this huge data
set with a complex effect structure. The estimated effects are geophysically plausible and the methodological approach seems promising.
Keywords: functional regression; generalized additive model; penalized splines;
geophysics; big data

1

Introduction and data

The physics of seismic waves emanated by earthquakes is well-understood, butchallenging with respect to the natural complexity of topography, subsurface velocity structure and earthquake source effects on large scales relevant for earthquake hazard assessment. Furthermore, current earthquake models rarely include
the effects of earthquake faulting dynamics. To tackle both of these challenges,
we use a generalized functional additive regression to quantify how physical conditions at an earthquake fault affect the surficial ground velocity measured overtime.
The simulated earthquake data we analyse are derived from a large scale computer
experiment with the open source software SeisSol (Breuer, 2014) and are based ona real earthquake that took place in Northridge (USA) in 1994. Absolute ground
velocities were simulated solving elastic wave equations and are defined as theThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Bauer et al.

147

absolute ground velocity [m/s](isotropic) L2 norm of the ground velocities in east-west, north-south and verticaldirection. Each of the simulations used a different set of physical conditions at thefault. Surficial ground velocities were then recorded at 6146 virtual seismometers.
For the analysis, the first 15s of the absolute ground velocity measurements from135 simulations were used in a resolution of 2Hz. Leading zeros were discarded
until the first relevant observation (≥ 0.01).
Typical observations
by hypocentral distance4

hypocentral distancesmall
medium
large

3
2
1
0
0

5

10

15

time [s]FIGURE 1. Left: Categorized mean absolute ground velocity in one simulation
over the area under study, darker colours correspond to increased velocity. Right:
Typical observations of absolute ground velocity over time. The initial peak of
the ground velocity is delayed and smaller as hypocentral distance increases.
The evaluated predictors were all constant over time. Our main interest was in fivephysical parameters, which were pre-set in each simulation and which consistedof three frictional resistance variables, the direction of the regional tectonic background stress and the binary soil material of the whole area ({rock, sediment}).
Additionally included were the moment magnitude as the classical measure ofearthquake size, the local topography as well as the height and the hypocentral
distance of each seismometer.

2

Methodology

Following Scheipl, Gertheiss and Greven (2016), the generalized functional additive model for function-on-scalar
regression is written in the form yi (tl ) ∼P
F (µil , ν), where g(µil ) = R
r=1 fr (X ri , tl ), with seismometers i = 1, . . . , n, absolute seismic ground velocity yi (tl ) recorded at time point tl and the numberof additive effects R with associated covariates X ri . Conditional on the additive
predictor and known response function g(·), yi (tl ) is assumed to come from somegiven distribution F with conditional expectation µil and dispersion and shapeparameters ν. Each functional additive effect fr (X ri , t) is represented in termsof a (tensor product) P-spline basis. A prediction error based approach was used
for tuning basis sizes as the high-dimensional data dominate the penalization
prior in the estimation. The model is fitted on 468,503 data points and comprises 18 smooth effects with a total of 633 spline coefficients and 25 smoothing
parameters. We used a recently developed highly performant algorithm for penalized likelihood-based inference from Wood et al. (2016) that makes estimationof this complex model on such large data feasible. Its major advances are the use

148

Modeling earthquake dynamics

of a highly efficient and parallelizable block-wise Cholesky decomposition and acompressed representation of the (marginal) spline basis.
Local small-scale (radius 300m) and large-scale (2000m) topography at each measuring station was included via the Topographic Position Index (TPI) introducedby Weiss (2001), defined as the height difference between the seismometer andthe mean height in a circular neighborhood.

3

Results

Absolute ground velocity
by hypocentral distance [m](95% confidence intervals)
0.7

20000
40000
60000
80000
100000

0.6
0.5
0.4
0.3
0.2
0.1
0.0
0

5

10
time [s]15absolute ground velocity [m/s]absolute ground velocity [m/s]

A Gamma model with exponential response function g(·) explained 70.7% of thenull deviance. Excluding moment magnitude from the model improves model performance measured by test set MSE. This is in line with a secondary finding thatmoment magnitude can be predicted quite precisely (98.2% explained deviancein a linear model with log-transformed response) based on the five parametersdescribing the simulation setup.
Absolute ground velocity
by dynamic coefficient of friction(95% confidence intervals)
0.1
0.3
0.5

4
3
2
1
0
0

5

10

15

time [s]FIGURE 2. Pointwise 95% confidence intervals for the predicted mean of absolute
ground velocity for various values of the two most important predictors, withremaining covariates being held constant at realistic values. Left: hypocentral
distance. Right: dynamic coefficient of friction.
Among the remaining predictors the hypocentral distance and the dynamic coefficient of friction show by far the strongest effects, a visualization is given in
Figure 2. Additional results can be found in Bauer (2016). All the estimatedeffects seem geophysically plausible.
In total the model has an acceptable fit, although absolute peak ground velocities
are often underestimated. Remaining spatial structure of the mean residuals isshown in Figure 3.

4

Conclusion

Functional additive regression models are a promising approach in modeling surficial ground velocity. Major effects are the hypocentral distance and the dynamic
coefficient of friction, with higher values leading to decreased ground velocities forboth. The estimated effects all seem geophysically plausible. In future research,

Bauer et al.

149

Residual mean●(−1.00,−0,25](−0.25,−0.05](−0.05, 0.05]( 0.05, 0.25]( 0.25, 1.00]

FIGURE 3. Mean residuals (averaged over time and over simulations) over thearea under study. The black dot marks the epicenter.

the model will be refined further, e.g. by explicitly modeling spatial correlation
and by relaxing the strict assumption of the hypocenter as fixed point source forall simulated earthquakes.

References
Bauer, A. (2016). Auswirkungen der Erdbebenquelldynamik auf den zeitlichen
Verlauf der Bodenbewegung. MA thesis. Ludwig-Maximilians-Universität,
Munich, Germany. Available: https://epub.ub.uni-muenchen .de/31976/
Breuer, A. et al. (2014). Sustained petascale performance of seismic simulations
with seissol on supermuc. Supercomputing - 29th International Conference,
ISC 2014, Leipzig, Germany, 1 – 18. Springer International Publishing.
Scheipl, F., Gertheiss, J., Greven, S. (2016). Generalized functional additive mixed
models. Electronic Journal of Statistics, 10.1, 1455 – 1492.
Weiss, A. (2001). Topographic position and landforms analysis. Poster presentation, ESRI user conference, San Diego, CA, 200.
Wood, S.N. et al. (2016) Generalized additive models for gigadata: modelling theUK black smoke network daily data. Journal of the American Statistical
Association. DOI: 10.1080/01621459.2016.1195744.

A Destructive Series Power Cure Rate Model
Gladys D.C. Barriga1 , Vicente G. Cancho2 , Gauss M.
Cordeiro3 , Edwin M. M. Ortega1 , Michael W. Kattan4
1

Universidade de São Paulo, So Carlos, SP, Brazil.
Universidade Estadual Paulista, Bauru, SP, Brazil.
3
Universidade Federal de Pernambuco, Recife, PE, Brazil.
4
Department of Quantitative Health Sciences-Cleveland Clinic, USA.
2

E-mail for correspondence: glad@feb.unesp.br
Abstract: A new flexible cure rate survival model is developed where the initial
number of competing causes of the event of interest (say lesions or altered cells)follows a power series distribution. This model provides a realistic interpretationof the biological mechanism of the event of interest as it models a destructive
process of the initial competing risk factors and records only the damaged portionof the original number of risk factors. Our proposed survival models are used forpredicting breast carcinoma survival in women who underwent mastectomy. The
postmastectomy survival rates are often based on previous outcomes of large
numbers of women who had a disease, but they cannot predict what will happen
in any particular patient’s case. Pathologic explanatory variables such as diseasemultifocality, tumor size, tumor grade, lymphovascular invasion and enhanced
lymph node staining are prognostically significant to predict these survival rates.
Keywords: Cure rate models; power series distribution; proportional hazard
models; regression model; survival models

1

Introduction

The event of interest in many survival studies or cancer-relapse trials can be
the death of a patient (due to various competing causes) or a tumor recurrence(attributed to metastasis-component tumor cells left active after an initial treatment). However with the recent advances in (cancer) treatment therapies, a highportion of the subjects are expected to be cured, i.e. remaining disease free after
prolonged follow-ups. In this vein, there is now a vast literature on ‘cure rate
models’ for survival data, though majority of these stems from either one of
the 2-component Mixture Cure model of Berkson & Gage(1952), or the Bounded
Cumulative Hazard (or, BCH model) of Yakovlev & Tsodikov(1996).
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Barriga et al.

151

Recently, Rodrigues et al.(2011) have extended the BCH model of Yakovlev &Tsodikov(1996) through a special case of the destructive (compound) weighted
Poisson distribution. The main purpose of this study is to propose a general classof destructive survival cure rate model.

2

Model

For an individual in the population, let M denote the unobservable number ofcauses of the event of interest for this individual, for example, M can represent
the number of altered cells before the treatment initial. Consider that M follows
a power series (PS) distribution with probability mass function (pmf),
P (M = m; θ) =am θm
, m = 1, 2, . . . ,
A(θ)

(1)

P∞
m
where am ≥ 0 and A(θ) =and θ ∈ (0, s) (s can be ∞) is such
m=1 am θthat A(θ) is finite. The immediate consequence of a prolonged treatment is theformation (or not) of precancerous lesions. Given M = m, let Wj , j = 1, . . . , m,
be independent random variables (independent of M ) following a Bernoulli distribution with success probability φ indicating presence of the j-th lesion (orcompeting cause), or the probability of an undestroyed clonogenic cell. The variable N , denoting the total number of altered cells among the M initial cells(competing causes), which are not destroyed or eliminated by the treatment, isdefined by
N = W1 + W2 + · · · + WM .
(2)
By damaged or destruction, we mean N ≤ M . The initial M competing causes is
primary initiated malignant cells, where Wj in ( 2) denotes the number of living
malignant cells that are descendants of the initiated malignant cell j duringsome time interval. In this case, N denotes the total number of living malignant
cells at some specific time. The conditional distribution of N (given M = m) is
referred to as the damaged distribution. The time to event for the jth competing
cause is represented by Zj , j = 1, . . . , N . We assume that conditional on N ,the Zj are iid random variables with cumulative distribution function (cdf) F (t)and survival function S(t) = 1 − F (t) that does not depend on N. The total
number of competing causes N and the time Zj are not observable (latent).
The observable time to the event of interest is defined by the random variable
T = min(Z1 , . . . , ZN ), and T = ∞ if N = 0 with P (T = ∞|N = 0) = 1. Under
this setup, the survival function (Rodrigues et al., 2009) for the entire population
is given by
Spop (t) =A(θ[1 − φF (t)])
.
A(θ)

(3)

The cured fraction is given p0 = limt→∞ Spop (y). From (2), we obtain p0 =limt→∞ Spop (t) = A(θ[1−φ])
> 0,
A(θ)
this indicating that (2) is not a proper survival function. The density and hazard
0
(t)])
0
function associated to (2) are given by fpop (t) = −Spop(t) = A (θ[1−φFφ θ f (t),A(θ)
0(θ[1−φF (t)])
and hpop (t) = AA(θ[1−φFφ θ f (t) where A0 (θ[1 − φF (t)]) = A0 (θ) |θ(1−φF (t))(t)])
and f (t) = −dS(t)/dt.

152

3

A A Destructive SP Cure Rate Model

Application

To illustrate our proposed modeling discussed so far, we consider the data set
collected by Kattan et al.(2004), where a total of n = 284 women who had
been treated with mastectomy and axillary lymph node dissection at Memorial
Sloan-Kettering Cancer Center (New York, NY) between 1976 and 1979 and met
the following requirements for study inclusion: confirmation of the presence ofinvasive mammary carcinoma, no receipt of neoadjuvant or adjuvant systemic
therapy, no previous history of malignancy, and negative lymph node status asassessed on routine histopathologic examination. We report the survival times(T ) until the patient’s death or the censoring times at the end of the study.
Some explanatory variables are associated with pathologic characteristics of thetumor. The tumor grading was performed using the standard modified BloomRichardson system. The lymphovascular invasion was obtained using morphologic
criteria. The lymph node status was measured according to immunohistochemistry (IHC) and hematoxylin and eosin (H&E) stains. After deleting subjects withincomplete data and missing observation times, we have a subset of n = 365 patients with approximately 78% of censoring. We consider survival times until thepatient’s death (in years) as the response variable. The following variables werecollected from each patient: ti : observed time ti (in years); x1i : age (in years);
xi2 : multifocality (0: no, 1:yes); xi3 : tumor size (in cm); xi4 : tumor grading (0:I, 1: II, II and lobular); xi5 : lymphovascular invasion (0: no, 1: yes) and xi6 :lymph node status (0: IHC+ IHC- and H&E-, 1: IHC+ and H&E+).
To these data, we fit some members of the model described in Section 2 such as thedestructive Poisson, geometric and logarithmic models with all covariates on the
proportions of undestroyed clonogenic cell (φ) and on the short-term survivors.
exp(β0 +β1 x1i +···+β6 xi6 )
i.e, φi = 1+exp(β. Also, based on the proportional hazard func0 +β1 x1i +···+β6 xi6 )tion, we incorporate the covariates through h(y|λ) = h0 (y|α)exp{γ0 + γ1 x1i + · · · + γ6 xis },bwith h0 (y|α) = αtα−1 . We apply the selection criteria AIC = −2`(ϑ)+2#(ϑ)
forthe candidate models, according to the AIC, we select the destructive geometriccure rate (DGCR) model as our working model. Considering the likelihood ratio
statistic, we test the effect of these covariates in the probability of undestroyedcauses and short-term survivors, i.e., H0 : γ1 = γ2 = γ3 = γ4 = γ5 = γ6 = β1 =β2 = 0 yielding wn = 0.179 (p-value ≈ 1), indicating that the covariates age,
multifocality and lymphovascular invasion are not significant for the probabilityof undestroyed causes and any of the covariates have a significant effect on shortterm survivors. Hence, the MLEs and their standard errors for the parametersof the DGCR model with significant covariates are listed in Table 1. We conclude that the tumor size, tumor grading, lymphovascular invasion and lymph
node status are significant prognostic variables for determining survival time andmortality risk in women with breast carcinoma.

References
Berkson, J. & Gage, R. P. (1952). Survival curve for cancer patients following
treatment. Journal of the American Statistical Association, 47(259), 501–
515.
Johnson, L. N., Kemp, A. W. & Kotz, S. (2005). Univariate Discrete Distribution. Wiley, New York, NY, third edition.

Barriga et al.

153

TABLE 1. MLEs of the parameters for the reduced DGCR model.

Parameter
θ
α
γ0
β0
β3
β4
β5
β6

Estimate (est)
0.005
1.933
-4.075-5.216
0.391
3.165
0.719
1.900

Standard error (se)
0.076
0.205
0.463
1.163
0.168
1.102
0.394
0.518

|est| / se
—
—
8.805
4.483
2.331
2.871
1.824
3.666

Rodrigues, J., Cancho, V. G., de Castro, M. & Louzada-Neto, F. (2009). On theunification of the long-term survival models. Statistics & Probability Letters, 79, 753–759.
Rodrigues, J., de Castro, M., Balakrishnan, N. & Cancho, V. G. (2011). Destructive weighted poisson cure rate models. Lifetime data analysis, 17(3), 333–
346.
Yakovlev, A. Y. & Tsodikov, A. D. (1996). Stochastic Models of Tumor Latency
and Their Biostatistical Applications. World Scientific, Singapore.

Latent vector autoregressive models:Application to evaluate evolution of oral
health and general health
Trung Dung Tran1 , Joke Duyck2 , Geert Verbeke1 , EmmanuelLesaffre1
1
2

KU Leuven, I-BioStat, Leuven, Belgium
KU Leuven, Department of Oral Health Sciences, Belgium

E-mail for correspondence: trungdung.tran@kuleuven.be
Abstract: Vector autoregressive models (VAR) and random effects models areused to flexibly describe the evolution of a multivariate response. However, with alarge number of responses at each occasion and many evaluations, a lot of parameters need to be estimated. We propose a latent vector autoregressive model that
describes the evolution of the common factors used to summarize the responses
via a combination of factor analysis (FA) models and random effects models.
The latent VAR model makes use of q common factors for p (p  q) observed
variables. The proposed model is applied to the BelRAI database to identify theeffect of oral health (OH) and general health (GH) on the development of theother. Parameter estimation is done via Stan package.
Keywords: CFA; Longitudinal data; Oral and general health; VAR models.

1

Introduction

In most longitudinal studies, multiple outcomes are repeatedly measured overtime. In the BelRAI database (De Almeida Mello et al. 2012) three OH and four
GH items are recorded longitudinally on elderly people at baseline and regularly
at every six months. It was of clinical interest to explore the joint evolution oforal and general health, and to know whether the information from OH provides
additional information on GH and vice versa. To evaluate the evolution of all theitems simultaneously, one could make use of the VAR models or random effects
models. However, joint modeling of all responses leads to over-parameterization
with VAR models (Koop and Korobilis, 2010) and computational problems withrandom effects models (Verbeke et al. 2014). To avoid this problem, we propose alatent VAR model which is a VAR model applied to the latent structure obtainedThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Tran et al.

155

from a model that combines confirmatory factor analysis and random effects
models.

2

Motivating data set: BelRAI database

To obtain a picture of the joint dependence of OH and GH, we extracted 3450
individuals from the BelRAI database. The elderly subjects were examined on 2
to 6 occasions providing in total 8320 observations. Three binary OH indicators:
non-intact teeth, chewing difficulty, and dry mouth were selected to represent
OH and four ordinal GH measures: activity of Daily Living (ADL), cognitive
performance scale (CPS), depression rating scale (DRS), and changes in health,end-stage disease, signs, and symptoms scale (CHESS) were recorded. In each of
the responses there are missing values ranging from 5% to 20%, assumed to be
at random.

3

VAR model for the latent structure

3.1

Basic VAR model

Let Yt be a continuous (p×1) multivariate response recorded at time t. The basic
k-lag vector autoregressive (VAR(k)) model has the form:
Yt = c + Γ1 Yt−1 + ... + Γk Yt−k + δt ,t = 1, ..., T

where c is an unknown vector of intercepts, Γi (i = 1, . . . , k) are coefficient matrices, and δt is a zero mean, serially uncorrelated vector of errors with covariancematrix ∆. Thus a VAR model is a generalization of an autoregressive model bymodeling the evolution of each variable based on its own lags and the lags of theothers.

3.2

Latent VAR model with random effects for ordinaloutcomes

For ordinal outcomes, we first assume that each of them is a manifestation of alatent standard normal distributed variable, Z, discretized by a set of cut-points.
These latent variables are modeled using a FA model with q (q  p) common
factors combined with a random effects model:
Zt = τt + Λt ξt + b + εt .

(1)

We then apply the VAR model to the latent structure of the common factors,i.e., at time point t:
ξt = Γ1 ξt−1 + ... + Γk ξt−k + δt ,(2)

where ξt are the common factors, Λt is the factor loading matrix, b, a vector of p
random intercepts, b ∼ N (0, D) with a diagonal matrix D, and εt ∼ N (0, Ψ) withdiagonal matrix Ψ. The restrictions τ1 = ... = τT and Λ1 = ... = ΛT are applied
to preserve the interpretation of the common factors over time. For identification,
variances of the common factors at the first time point are kept at one.

156

4

VAR models for OH and GH

Application of the latent VAR model to oral andgeneral healthFirst, model (1) is used to summarize the OH and GH items in the BelRAI
database using two common factors. VAR model (2) is then fitted to the common
factor (Figure 1):
  

  
ξt
γ11 γ12
ξt−1
δ
=
+ ξ ,
ηt
γ21 γ22ηt−1δη   

ξ1
0
1 ρ
where
∼ N
,
, δξ ∼ N (0, σξ2 ), δη ∼ N (0, ση2 ). With this
η10
ρ 1
model, the cross-lagged parameter γ12 (resp. γ21 ) indicates the amount of extra
information that the current OH (resp. GH) indicator provides the future GH(resp. OH) indicator.
The model is implemented in R using the rstan package. Four chains are runand convergence is claimed when the estimated potential scale reduction factor
is less than 1.10 and Monte Carlo standard error is less than 5% of the posterior
standard deviation.
Estimates, provided in Table 1, indicate that given current GH, OH is still predictive of future GH and vice versa.

FIGURE 1. VAR(1) model for the latent factors derived from oral health (OH)and general health (GH) indicators.

5

Discussion

A similar idea in psychology was proposed by Oort (2001) for continuous variables
with a balanced dataset. Our procedure allows for unbalanced data and works

Tran et al.

157

TABLE 1. Posterior mean, SE and equal-tail 95% CI for the γ’s.

Parameter

Est.

SE

γ11
γ12
γ21
γ22

0.958
0.032
0.039
0.994

0.004
0.009
0.012
0.001

95% CI
0.951
0.015
0.016
0.991

0.967
0.048
0.062
0.996

also for ordinal data, making use of a latent structure as proposed by Liu and
Hedeker (2006), Cagnone et al. (2009) but their approaches were restricted tojust one latent factor.
References
Cagnone, S., Moustaki, I., Vasdekis, V. (2009). Latent variable models for multivariate longitudinal ordinal responses. British Journal of Mathematicaland Statistical Psychology, 62, 401 – 415.
De Almeida Mello, J., Van Durme, T., Macq, J., Declercq, A. (2012). Interventions to delay institutionalization of frail older persons: design of a longitudinal study in the home care setting. BMC Public Health, 12:615.
Koop, G., Korobilis, D. (2010). Bayesian multivariate time series methods for
empirical macroeconomics. Foundations and Trends in Econometrics, 3,267 – 358.
Liu, L., Hedeker, D. (2006). A mixed-Effects regression model for longitudinal
multivariate ordinal data. Biometrics, 62, 261 – 268.
Introduction to Multiple Time Series Analysis. Berlin: Springer-Verlag.
Oort, F. J. (2001). Three-mode models for multivariate longitudinal data. British
Journal of Mathematical and Statistical Psychology, 54, 49 – 78.
Verbeke, G., Fieuws, S., Molenberghs, G., Davidian, M. (2014). The analysis ofmultivariate longitudinal data: A review. Statistical Methods in Medical
Research, 23, 42 – 59.

Multivariable fractional polynomials forcorrelated dependent variables using
generalized estimating equations
Maren Vens12 , Jördis Stolpmann2
1

Institut für Medizinische Biometrie und Epidemiologie, Universitätsklinikum
Hamburg-Eppendorf, Hamburg, Germany
2
Institut für Medizinische Biometrie und Statistik, Universität zu Lübeck, Universitätsklinikum Schleswig-Holstein, Campus Lübeck, Lübeck, Germany
E-mail for correspondence: m.vens@uke.de
Abstract: Generalized estimating equations (GEE) are used for estimating therelationship between correlated outcome variables and several risk factors. For
independent observations, non-linearity in continuous independent variables canbe taken into account by multivariable fractional polynomials (MFP). We propose an MFP procedure for GEE models (GEE-MFP) which combines a function
selection procedure with a stepwise elimination procedure. Quasi-likelihood information criteria are compared to a new Bayesian quasi-likelihood information
criterion (BQIC) for model selection. GEE-MFP is illustrated by re-analyzing
data on hypertension from the Framingham Heart Study. It is shown that the
GEE-MFP algorithm using the BQIC provides the best fit to the real data, while
the use of the established criteria overfit the data. The GEE-MFP algorithm
provides a flexible approach for modeling the functional relationship between
correlated dependent variables and several continuous independent variables.
Keywords: Akaike information criterion; Bayesian information criterion; Nonlinear relationship; Quasi likelihood information criterion; Stepwise selection.

1

Introduction

Generalized estimating equations (GEE) can adequately take into account thecorrelation between the dependent variables, e.g., repeated measurements or family members (Zeger and Liang, 1986). The main advantage is that only themean structure needs to be correctly specified but the correlation between thedependent variables may be misspecified.Many extensions to GEE have been
proposed (Ziegler, 2011). GEE is, however, still limited in its capability to modelThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Vens et al.

159

continuous independent variables. In all software packages, the functional formof the covariates needs to be specified in advance.
One approach to allow for nonlinearities in the independent variables are fractional polynomial transformations (FPs; Royston and Sauerbrei, 2008).
The advantages of GEE and FP can be combined to a GEE-MFP method. A
first simple algorithm for two independent variables has been proposed by Cui
et al. (2009), which we generalize to handle an arbitrary number of independent
variables. Due to the large amount of possible different models, it is impossible
to search the entire model space and there is a clear need for a simple searchstrategy to find the best-fitting model. To this end, we propose a simple feasible
algorithm.

2

The GEE-MFP algorithm and the Bayesian QIC

The function selection procedure (FSP, Fig. 1, left) is combined with a P valuebased stepwise procedure (SWP; Fig. 1, right).

(a)

(b)

FIGURE 1. Left: Function selection procedure of the proposed GEE-MFP approach. Right: Stepwise procedure of the proposed GEE-MFP approach.
To measure the goodness-of-fit (GOF) of a GEE model, Pan (2001) proposed the
QIC, an extension of the Akaike information criterion. GOF is investigated bymeasuring the difference between the estimated model-based covariance matrixÂ and the estimated robust covarianceĈ. Therefore, the QIC is estimated −1 matrix
dby QIC(R)= −2ϕ̂Q(µ̂) + 2tr Â Ĉ where ϕ̂Q(µ̂) is the quasi likelihood
function, ϕ the dispersion parameter, µ̂1and Q(µ̂) are estimated using the modelwith working correlation matrix R (Pan, 2001). For model selection of the mean
structure, the QIC is simplified by assuming that Q equals the independency
matrix (QICu ).

160

GEE meet MFP

Cui et al. (2009) used the QICu for selecting the best FP transformations. Disadvantages of the AIC are that models have a large number of independent
variables, and tend to overfit the data. We propose to use the Bayesian quasi\u (R) = −2ϕ̂Q(µ̂) + pln(N ) , where N is
likelihood information criterion: BQIC
the total number of observations in the study. BQICu penalizes additional parameters more strictly if the sample size is larger than 8. As a result, fewer FP2
transformations will be observed in the model building process and overfitting isprevented.

3

Application to the Framingham Heart Study

To illustrate the GEE-MFP approach, data from the Framingham Heart Study,
cohort 2 as provided for the Genetic Analysis Workshop 13, was re-analyzed(Cupples et al., 2003). The dichotomous outcome high blood pressure at any
investigation (HBP) was modeled using explanatory binary, categorical, and continuous variables observed at baseline.
Analysis showed that models derived using BQICu included fewer variable variable transformations compared with QIC and QICu models. FP2 transformation
were never for one of the covariates using BQICu , and transformations were morestable in the sense that less different FP transformations were chosen for eachcovariate. The SWP using BQICu yielded in lower FP transformations for eachcovariate and less covariates were chosen.
The effect of different transformations on the model fit is illustrated for thevariable age (Fig. 2). The age distribution of study participants was divided into
deciles, and the proportion of subjects with HBP within each decile was plottedagainst the median age per age decile group. A good fit would be close to thesepoints.
1.00

High blood Pressure

0.75

0.50

0.25

0.00

1

2

3

4

5

6

Age [years/10]
GEE with linear influence

QIC and QICu before SP

QIC after SP

QICu after SP

BQICu before SP

BQICu after SP

FIGURE 2. Comparison of fits using the different selection and goodness of fit
criteria before and after stepwise procedure.
QIC and QICu models (FP2 for age) were not close to the estimated proportions.
Models using BQICu (linear in age) provided a good fit to the data. Results weresimilar for other variables kept in the models after SWP.

Vens et al.

4161

Conclusion

We propose to use fractional polynomial transformations together with GEE
for the joint analysis of an arbitrary number of independent binary, categorical,
and continuous variables together on correlated possibly non-normal dependent
outcome. The approach may be combined with a SWP using a quasi-likelihood
criterion based on the BIC.

Acknowledgments: Authors acknowledge the Framingham Heart Study of
the National Heart, Lung and Blood Institute of the National Institutes of Healthand Boston University School of Medicine, supported by the National Heart, Lung
and Blood Institutes Framingham Heart Study (Contract No. N01 HC 25195)
for providing data for independent analysis by our group. The Genetic Analysis Workshops are supported by NIH grant R01 GM031575 from the National
Institute of General Medical Sciences.

References
Cui, J. et al. (2009). Fractional polynomials and model selection in generalized
estimating equations analysis, with an application to a longitudinal studyin Australia. American Journal of Epidemiology, 169, 113 – 121.
Cupples, L.A. et al. (2003). Description of the Framingham Heart Study data
for Genetic Analysis Workshop 13. BMC Genetics, 4 Suppl 1, S2.
Pan, W. (2001). Model selection in estimating equations. Biometrics, 57, 529 –534.
Royston, P. and Sauerbrei W. (2008). Multivariable model-building. New York:John Wiley & Sons.
Zeger, S.L. and Liang, K.Y. (1986). Longitudinal data analysis for discrete andcontinuous outcomes. Biometrics, 42, 121 – 130.
Ziegler, A. (2011). Generalized estimating equations. New York: Springer.

An application of mixture models to estimate
the size of Salmonella infected flock data
through the EM algorithm using validation
information
Carla Azevedo1 , Dankmar Böhning1 , Mark Arnold2 , Antonello
Maruotti1
1
2

University of Southampton, United Kingdom
Animal and Plant Health Agency, United Kingdom

E-mail for correspondence: cfa1e14@soton.ac.uk
Abstract: Capture-recapture methods are used to estimate the total size Nof a population when it is partially observed. Due to an incomplete identification/registration mechanism in real life applications, we observe only the positive counts representing the number of repeated identifications, and in orderto estimate N , we wish to predict the number of units of the unobserved part.
Sometimes, a validation sample is available in the study, providing complete information on the unobserved units. The estimate of the total population size canbe obtained by fitting jointly a zero-truncated distribution to the truncated data
and an untruncated distribution of the same class to the untruncated data bymeans of the EM algorithm. In this paper, we consider a flexible non-parametricmixture model approach allowing the heterogeneity of the data by means of anested EM algorithm using validation information. A simulation study illustrates
the major ideas of this application.
Keywords: capture-recapture; validation information; mixture models.

1

Introduction and background

The aim of this work is to determine the size N of an elusive target population.
Let us assume that the members of the population are identified at m observational occasions where m is considered fixed. For each member i the count ofidentifications Xi returns a count in 0, 1, ..., m and i takes values from 1 to N . It
is assumed that Xi is observed if unit i has been identified for at least one occasion. We have then that Xi is observed and let X1 , ..., Xn denote the observed
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Azevedo et al.

163

counts with n representing the total number of recorded individuals. We assume
w.l.o.g. that Xn+1 = ... = XN = 0. Let fx be the frequency of units with countX = x. The population can be described by a probability density function px (θ)
which denotes
Pthe probability of exactly x identifications for a generic unit wherepx ≥ 0 and ∞x=0 px = 1. It is possible to incorporate a validation sample into
the modelling and decrease the bias in the estimation process. In this sample all
the counts are observed. We will denote by g0 , g1 , ..., gm the frequency distribution associated with this sample. Situations of heterogeneity in the population
can be modelled using non-parametric mixture models. We start with the simple
homogeneous case of one component using the binomial distribution through the
EM algorithm. Mixture models were utilized to allow for more components andinclude validation information. Simulation studies allowed to enhance the valueof including validation information into the modelling to estimate the total sizeof the population.

2

Case study - Salmonella data

The following data was provided by the Animal and Plant Health Agency, UK.
A European Union wide baseline survey of Salmonella infection was carried out
between October 2004 and September 2005. The results of that survey were used
as a basis for setting flock prevalence reduction targets for Salmonella national
control programmes in each member state of the EU. In the UK, a randomized
sample of 454 commercial layer flock holdings was tested for Salmonella. In order
to be able to monitor the progress of control measures for Salmonella, it is important to be able to obtain an accurate estimate of the initial prevalence at thetime of the EU baseline survey. The goal of this study is to determine the numberof farms which had Salmonella infected poultry but for which result in the survey
was negative. 53 holdings tested positive for Salmonella in one or more samplesof the survey using a EU survey method which consists of a total of 7 tests, soeach farm could have 0,1,...,7 positives as Table 1 (second row) shows.
TABLE 1. Positive and validation samples for salmonella data.

x

0

1

2

3

4

5

6

7

fx

?

17

9

5

6

5

5

6

gx

3

1

3

2

3

3

4

2

The same method was conducted in 21 of the infected farms which establishedthe validation sample as shown in Table 1 - third row. This sample also allows
to observe infected holdings with all repeated tests negative.

3

Non-parametric mixture models

Our interest lies in estimating the size N of a target population knowing thatno zero counts have been captured. To model px = px (θ) we need to find an
estimate θ̂ for θ so that p̂x = px (θ̂). Since we are dealing with a fixed number

164

A capture-recapture approach using validation informationof sampling occasions m = 7, a binomial distribution to model the data seems
appropriate and we can estimate θ using the EM algorithm. However, specificassumptions such as independence of observations and homogeneity are requiredby the simple binomial model. Also, the benefits of having a validation sample
have not yet been considered. Mixture models allow relaxing these assumptions
offering a flexible approach in modelling heterogeneity. Let f (x, θ) denote thesimple, parametric binomial
P density function. The finite mixture distribution is
given by px = p(x, Q) = kj=1 f (x|θj )qj and appears as the marginal distribution with respect to some variable Z having distribution Q, where Q
P is the mixing distribution giving non-negative weights qj to θj . Notice that kj=1 qj = 1.
The mixing distribution can be seen as the heterogeneity distribution of thelisting parameter of the population. If Q is available, we can estimate N , fornexample, by means of the Horvitz-Thompson estimator as (1−p(0,Q)). Therefore, we need to estimate Q and this will be done by maximum likelihood. We
have two likelihoods for capture-recapture modelling: the unconditional likelifj +gj
 
Q
Pk
m l
hood LU (θ, N ) = m
θj (1 − θj )m−l
and the conditional,
j=0
l=1
l
 
fj

m
 θ l (1−θj )m−l ql
j

Pk
Q
l
 ×

P
observed joint likelihood LC (θ) = m
m 
j=1 
l=1 1− kj=1 qj (1−θj )
gj
 
m l
m−l
. The estimation of the unconditional like(1
−
θ
)
θ
j
j
j=0
l=1
l
U
lihood would imply the maximization
 of the unknown f0 . However, L (Q) =N
LC (Q) × B(N, Q) where B(N, Q) =pf00 (1 − p0 )n . Hence, the unconditionalnlikelihood depends to a large extent from the conditional likelihood which the EM
algorithm for mixtures maximizes very easily. In the following we will consider
only the conditional likelihood.
Qm

4



Pk

Application of the EM algorithm with mixtures tothe case study

This theory was applied to the Salmonella data using a mixture model with 2
and 3 components using just the positive sample (Pos) or both samples (Pos-Val).
Note that only the positive models and the positive with validation information,
respectively, are comparable. Simulation studies allow to conclude that we getan estimate for the population size more accurate and with less bias using avalidation sample (details not reported here).

5

Conclusion

The mixture model approach using only the positive sample can be extended toinclude information from the validation sample, the untruncated sample including also zero counts which are not observed in conventional capture-recapture
settings. It was done here using the binomial mixture model with 2 and 3 components. We have focused on the model selection criteria AIC and BIC to chooseAzevedo et al.

165

TABLE 2. Details of the obtained mixture models. The first column indicatesthe samples used in the model, fˆ0 is the number of unreported cases, N̂ is total
size of the population, k is the number of components of the mixture model, θ̂jand q̂j are the parameters of the mixture model.

Model

fˆ0

N̂

k

θ̂j

q̂j

log-likelihood

AIC

BIC

Pos

1

54

1

0.45

-

-146.05

294.1

296.40

Pos-Val

1

54

1

0.48

-

-187.79

377.58379.88

Pos

9

62

2

5.47
1.25

0.34
0.66

-98.75203.49209.40

Pos-Val

7

60

2

5.47
1.46

0.41
0.59

-144.64295.28301.19

Pos

20

73

3

6.49
3.99
0.78

0.13
0.25
0.62

-96.91203.82213.67

Pos-Val

9

62

3

4.24
6.38
1.16

0.31
0.17
0.52

-143.13296.26306.11the more appropriate model. According to that, the mixture model with 2 components might be used for both samples. More trust can be developed in themodel for the unobserved part using validation information.

References
Arnold, M.E., Martelli, F., McLaren, I., Davies, R.H. (2014). Estimation of therate of eggs contamination from salmonella infected chickens. Zoonoses andPublic Health 61: 18-27.
Böhning, D., Dietz, E., Kuhnert, R., Schön, D. (2005). Mixture models for capturerecapture count data. Statistical Methods and Applications DOI: 10.1007/BF02511573,
14: 29–43.

Retrospective estimation of growth ofChilean common hake based on the otolithmeasurements
Cristian Villegas1 , Emmanuel Lesaffre2 , Vı́ctor Espejo3
1

Escola Superior de Agricultura Luiz de Queiroz (ESALQ), Piracicaba, Brazil
L-Biostat, KU Leuven, Leuven, Belgium
3
Subsecretarı́a de Pesca y Acuicultura, Valparaı́so, Chile
2

E-mail for correspondence: clobos@usp.br
Abstract: The age-at-capture and length-at-capture of common hake (Merluccius gayi gayi) was collected in San Antonio and Talcahuano areas in Chile
between 1967 and 2010. In addition, the number of annual rings in sagittal otolith,
a common estimate of the age of the fish, is determined. This estimate is, however, interval-censored. Research questions such as the comparison of the age-atcapture over the years and the relationship of age- with length-at-capture requireappropriate modeling techniques. It was also of interest to know whether there aredifferent growth patterns inferred from the otolith radii in the hake and whether
there is a trend in growth over the many years data were collected. This involves
mixture modeling of longitudinal data. Because of the complexity of the data, a
Bayesian approach was used to address the research questions.
Keywords: common hake; radius-at-capture; length-at-capture; Bayesian inference; repeated measurements

1

Introduction

Measurements were taken on the common hake (Merluccius gayi gayi) in mainly
two maritime zones in Chile, i.e. between the IV (central zone) and the X Region(south zone) in Chile. It is of scientific but also commercial interest to determine
growth parameters of the hake and their evolution over time. At capture, thelength of the fish is determined as well as the otolith is extracted (a destructive
measurement) to determine the age of the fish. Based on the number of rings,
but also the lengths of the different otolith radii, it is of interest to estimate
the length of the hake at different ages. It is also of interest to examine therelationship of the otolith radii and the length of the fish, and to explore trends
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Villegas et al.

167

length−at−capture2

length−at−captureof this relationship over time. Knowledge of fish age characteristics is necessary
for stock assessments, and to develop management or conservation plans. The
latter allows for prediction of the volume of the future captures. Some of theresearch questions involve techniques that allow for interval-censored responses(a)(b)
and/or interval-censoredcovariates. But also mixture modelingis involved, andhierarchicalmodelling of longitudinal measurements.
Because of the complexity
80
80
of the data, Bayesian modeling is envisaged.
60

60

40
40
Backgroundand descriptionof the hake data set
20

20

The unpublished data set consists of measurements from 8347 common hakeotoliths
areas in Chile from Fisheries
0 from the San Antonio and Talcahuano
0Development(IFOP).
The14data set is0 composed
of
hake
0
2 Institute
4
6
8
10
12
2
4
6
8
10lengths,
12
14 andmeasurements taken
on otoliths collected during the years
1967-2010. Ages-atradius−at−captureradius−at−capture
capture fish range from 1 to 20 years for females and from 1 to 15 years formales. An historical analysis of otolith growth could help to describe the ways
4.3 Fitting non-linear model considering length vs age by Sex
in which a given population grows and shrinks over time, as controlled by birth,
The vonand
Bertalanffy
growth model
given
bydeath,
migration.
It is isthebasis
for understanding changing fishery patterns
and issues such as habitat destruction, predation and optimal harvesting rates.
Li = L∞ (1 − exp{−K (ti − t0 )}) + εiwith εi ∼ N (0, σ 2 ),(1)
The population dynamics
of fisheries is used byfisheries scientists to determine
sustainable
where Li is theyields.
length-at-capture and ti is the age-at-capture for the ith fish, respectively. The parameterinterpretationof thepurposes,
von Bertalanffy
modelinis givenby 1 the length-at-capture, radius-atFor
illustrative
wegrowth
present
Table• L∞andis theradii
asymptotic(average)lengthcaptureof theotolith
for each−1annulus for a female fish caught in 2002.
• K is the growth rate coefficient (units are years ), andIn Figure
1 wetheshow
the(average)
length-at-capture• t0 represent
age whenlength was zero. as a function of age-at-capture forfemaleand2 show
maleus hake
the fitted
growthcurve
The Figure
the dataand
and fitted
von Bertalanffy
growthmodelmodel.
considering length-at-capture versus
age-at-capture for each sex. In addition, we could see from Figure 2 the age-at-capture is between 1 and 20(years) for females and between 1 and 15 (years) for males.

(a)

(b)
80

length−at−captrure

length−at−captrure

80

60

40

20

0

60

40

20

0
0

5

10

15

age−at−capture

20

0

5

10

15

20

age−at−capture

FIGURE 1. Hake study: Length-at-capture of fish as a function of age-at-capture((a) = females, (b) = males) obtained 3from number of rings in otolith, together
with estimated length obtained from von Bertalanffy growth model ignoring interval-censored character of age measurement.

168

Retrospective estimation of growth of Chilean common hake

TABLE 1. Hake study: Measurements taken at capture of a four years old female
fish caught in 2002.

Length-at-capture (cm)44

3Radius-at-capture (µm)
6.9Radius at year (µm)
1 2 3
4
3 5 6
6.7

Statistical modelling approach

A popular growth curve model in biology is the von Bertalanffy growth model
given by
Li = L∞ (1 − exp{−K (ti − t0 )}) + εiwithεi ∼ N (0, σ 2 ),where Li is the length-at-capture and ti is the age-at-capture for the ith fish, respectively. The parameters of the von Bertalanffy growth model are interpreted
as follows: L∞ is the asymptotic (average) length, K is the growth rate coefficient (units are years−1 ), and t0 is the age when (average) length was zero. To
estimate the length of the hake as a function of sex, year of capture and age determined by the number of rings in the otolith necessitates a regression model withan interval-censored covariate, see e.g. Bogaerts, Komarek and Lesaffre (2017).
Model estimation, selection and checking will be done using a Bayesian approach.
Another possibility is to regress on the otolith maximal radius-at-capture. This
regression model will be compared to the previous model with respect to fit to
the fish lengths.
There is also interest in exploring the annual radii and to see how that has evolved
over time during the study period. This analysis involves a time series model on
longitudinal profiles where year is taken as time measurement and the annual
radii in the hake as longitudinal measurements. Notice that these longitudinalmeasurements are unbalanced across the fish.
All calculations were and will be done using R in combination with Bayesian
software such as JAGS and Stan.

4

Conclusion

This rich data set allows for many interesting explorations and complex modellingusing a Bayesian approach, which will be illustrated at the meeting.

Acknowledgments: FAPESP from Brazil and IFOP from Chile.
References
Bogaerts, K.; Komarek, A. and Lesaffre, E. (2017). Survival Analysis with IntervalCensored Data: A Practical Approach with examples in R, SAS and BUGS.
CRC/Chapman and Hall, Boca Raton.

Forecasting Quality of Hard Disk Drive with
Logistic Regression and Neural Networks
Somsri Banditvilai1 , Chammaree Chubuathong11King Mongkut’s Institute of Technology Ladkrabang, Thailand

E-mail for correspondence: somsri b2000@yahoo.com
Abstract: The purpose of this research is to develop a model to predict thequality of hard disk drives in an outgoing reliability test process. This would
reduce the loss of resources, such as cost reduction and testing time. The amountof data in this research is huge, therefore 2 data mining techniques which arelogistic regression analysis and neural networks are employed in this study. SASEnterprise Miner Workstation 13.2 was used to analyze the data. The data isseparated into 3 sets which are training set 60%, validation set 20% and test
set 20%. There are 197 quantitative independent variables. In order to reducethe independent variables, the relationship between each independent variable
and the quality of the hard disk drive was tested and it was found that only
25 variables were related to the quality of the hard disk drive. Therefore, 25
independent variables were used to build the neural network model. Since logistic
regression needs to eliminate independent variables that creates multicollinearity
problem. Following this 17 variables were left to build a logistic regression model.
The results show that the neural network model has 95.98% accuracy while thelogistic regression model has 72.37% accuracy. Therefore, it is seen that the neuralnetwork model performs better predictions for hard disk drive quality than thelogistic regression model.
Keywords: Hard Disk Drive; Logistic Regression; Neural Networks.

1

Introduction

Currently, the hard disk drive which is an electronic storage device, is very important. It is used for storing information and its size is growing exponentially.
The hard disk industry has expanded from the computers market to other markets, such as mobile phone, DVD player, TV and CCTV. In this research, theresearchers focus on the prevention of waste in the process of outgoing reliability test (ORT) before deliver them to the customers. It was found that there
are many types of abnormalities in read/write heads. Therefore, the researchers
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

170Forecasting Quality of Hard Disk Drive

would like to study and create a model to predict the hard disk drive whether theread/write heads were fault. Since there are 197 independent variables and data
is huge. Therefore, 2 data mining techniques which are logistic regression andneural networks are employed. Logistic regression model is a regression model
where the dependent variable is categorical. It used to predict whether an event
will occur. By configuring one or more variables that are expected to affect theoccurrence of the event. (Freedman. D. A. 2009) In the case of binary logistic regression, two values are defined as 0 and 1. The error term e is a random variable.
It is assumed that E(e) = 0 , the error are uncorrelated, and no multicollinearity among independent variables. Logistic regression requires sample size n larger
than regular regression analysis. The sample size is n>30p, where p is the numberof independent variables. Artificial neural networks has been inspired by attempt
to simulate human biological neural systems. The human brain consists mainlyof neurons, The linking of neurons through fibers called axons is used to deliver
signal from one neuron to another. Whenever nerve cells are stimulated, nerve
cells are connected to other axon via dendrites. The connectors between dendrites and axons are called synapses. Artificial neural network is widely used inclassification, modeling, forecasting, controlling, and clustering. (Tan, P.N.2005)

2

Methodology

2.1

The study of outgoing reliability test.

Examine hard disk abnormalities occurring in the pre-delivery test process andfound that the error most occur in the read/write heads set. Therefore, thisresearch focuses on modeling prediction of hard disk test results of malfunction
at read/write heads set. The data is divided into 60 % learning set, the 20 %verification set and the 20% testing set. Each set has the same ratio of normal
hard disk and hard disk failure. The data in the learning set is used in modeling.
The data in the validation set is used to evaluate the accuracy and the test set
is used to measure the accuracy of the final model.

2.2

Data preparation.

Select relevant information, verification of data integrity, consolidate data frommultiple sources and transform the data into the format that is appropriate forthe modeling techniques.

2.3

Build the model.

The model is build with SAS Enterprise Miner workstation 13.2, and using logistics regression analysis and artificial neural networks. Using data from April to
November 2015 (8 months). Study the factors affecting the hard disk drive quality.
There are 197 independent variables that affect or correlate with malfunction of
the read/write head set in post-assembly hard disk test and using the chi-square
test of association to eliminate the independent variables. In this research, if thePearson correlation value is greater than 0.8, then the independent variables arestrongly correlated.(Campbell, I. 2007) The dependent variable refers to the hardSomsri Banditvilai, Chammaree Chubuathong171

disk drive quality which can be defined to be 1 as ”PASS” and 0 as ”FAIL”. After testing the relationship between independent variables and hard disk quality,
it was found that there were only 25 independent variables were used to build
the model. For logistic regression analysis, in order to save all 25 independent
variables and eliminate multicollinearity, this research used principal component
analysis to regroup independent variables, and left 17 independent variables toconstruct the logistic regression model. Based on the stepwise method, only 7
independent variables had a p-value < 0.05, indicating that these variables wereeffectively or significantly correlated with hard disk quality. For artificial neuralnetworks, we employed multilayer neural networks and backpropagation training.
There is no specific theory for the number of nodes in hidden layer. However,(Satish K, 2012) proved that it should be between 1 and n-1 where n is the number of independent variables. Therefore, this research set the nodes in hidden
layer from 1-24. The final model had 1 input layer with 25 input nodes which
are equal to the number of independent variables, and one hidden layer with 5
hidden nodes, and 1 output layer with 1 node and defined the learning rate of
0.1.

2.4

Select of the models

The results of learning set, validation set, and test set were compared. A confusion
matrix is a summary of prediction results on a classification problem. The numberof correct and incorrect predictions are summarized in the table. This researchuses accurary which is the ratio of correct predictions to total predictions made.
The correct predictions are composed of True Positive (TP) which is hit and True
Negative(TN) which is correct rejection. Incorrect predictions are composed ofFalse Positive (FP)which is Type I error, and False Negative(FN) which is TypeII error.

3

Result and conclusion

The results from artificial neural networks had 95.98% accuracy as shown in
Table 2 and the logistic regression analysis had 72.37% accuracy as shown in
Table 1. From Table 1, 2, the accuracy of training set, validation set and testing
set are similar for both logistic regression and neural networks. There is no overfitting problem occurred (Chen, et. al., 2004). Therefore, the neural networks
analysis was used to test the hard disk drive before the outgoing reliability test.
It was found that the model was able to prevent the hard disk with malfunction
read/write heads. The results showed that 90 % of the predicted hard disk failure
are correct and it could prevent scraps from hard disk drive test process by 90%.
Thus, it could be concluded that the model derived from neural network analysis
had the ability to predict the quality of the hard disk effectively.

References
Campbell, I. (2007). Chi-squared and Fisher-Irsin tests of two-by-two tables withsmall sample recommendation: Statistics in Medicine. 3661 – 3675.
Chen, C. Liaw, A. and Breiman, L. (2009). Using Random Forest to Learn Imbalanced Data. California: University of California. Report ID:666

172

Forecasting Quality of Hard Disk Drive

TABLE 1. Logistic regression results

Data set

FN

TN

FP

TP

Accuracy

Training set
Validation set
Testing set
Total

110
40
41
191

72337
24152
24089
120578

27977
9286
9351
46614

335
108
108
551

72.37 %
73.34%
73.30%
72.37%TABLE 2. Neural networks results

Data set

FN

TN

FP

TP

Accuracy

Training set
Validation set
Testing set
Total

13
12
14
39

95823
32010
31999
159812

4511
1428
1441
7380

432
136
135
703

95.92 %
96.10%
96.05%
95.98 %

Freedman, D. A. (2009). Statistical Models:Theory and Practice. Cambridge: Cambridge University Press. p.128.
Satish, K. (2012). Neural Networks: A Classroom Approach (Second Edition)
Columbus: McGraw Hill Education.
Tan, P.N., Steinbac, M. and Kumar, V. (2005). Introduction to Data Mining. Boston:Pearson International, Inc.

Growing Bigger and More Accurate with
GSBPM (part one)Arbi Setiawan1 , Ni Lessari2
12

National Statistical Training Center, Indonesia
STIS National Statistical College, Indonesia

E-mail for correspondence: nilessari25@gmail.com, hanik.devia@gmail.com
Abstract: Indonesia has single government agency disseminating official data sothat national statistics accuracy can be as simple as mostly a matter of single
agency. Higher national statistics accuracy is viable through elements of genericstatistical business process model [gsbpm]. GSBPM provides pathways to be moreaccurate with bigger data. Big data in layman’s sense could be a trade-off between quantity and quality (including accuracy). With a wider availability ofsmart phones, guide book and manual soft copies can easily be loaded and retrieved. Question libraries can be made more handy. There are more than threehundred local languages in our country. Currently all variable descriptions available only in single national language. Interviewer may not be able to accuratelytranslate variable descriptions from single national language to local language.
This incapability can lead to different interpretation by respondent and in turndifferent raw data obtained. Accuracy can be improved with bilingual variable descriptions. Paper questionnaire space is limited for identified potential problems,
errors, discrepancies to be hand written. A personal computer on every desk offers
far more ease and flexibility for review, validate, edit sub-process. Distributive
computing power will make possible a repository of data and relevant metadata
in almost all subject matter specialist office. Currently archive is meaningful only
for small number of permanent staff members. To our thought with minimal costmore staff members with limited skill can be exposed to data and relevant metadata. Every year there are around half a thousand new statistics under-graduatelevel employees having personally-owned notebook computer at home and available for after office hours limited back-up. There are staff members having morethan twenty years tenure and ready for assessing how well the data reflect theirinitial expectations. It can be done by viewing the data from different perspectives using different tools and media. Our thought is that we can introduce aformal written documentation in addition to informal oral assessment.
Keywords: Big data; Smart phones; Distributive Computing.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

174

Accurate with GSBPM (part one)Advertised landed house price data among other things can be used to evaluate
interest rate.

FIGURE 1. sub process 1·3 2·3 3·3 4·3 5·3 6·3 7·3 8·3 9·3 gsbpm version four.
1.3 Any organization which is serving central bank may establish output objectives such how many regions will be covered. Will it be all five hundred Indonesian
regions ? To be accurate will it be of equal importance for all regions ? Otherwise
do some regions having central bank office are more important ? To be accurate
will it be of equal dead line for all regions ? Otherwise do some regions having
central bank office are less tolerant to delay ? That is some regions expectation
is well ahead of deadline.

FIGURE 2. Sub process 1·3 only gsbpm version four.
2.3 Will same primary raw data collection methodology be applied to all regions? If same primary raw data collection methodology is applied will it be person toperson interview ? Will it be paper-based interview ? Otherwise do some regions
having central bank office may use cellular-phone-based personal interview ? For
further accuracy do some regions having central bank office may use advertised
house price eliminating direct personal contact with primary raw data provider?

FIGURE 3. Sub process 2·3 only gsbpm version four.
3.3 Configure workflows for more accurate primary raw data collection. In general
the smaller a region the less accurate primary raw data in any given collection
time. In general the more remote a region the less accurate primary raw data in
any given collection time. Remote small region understandably has least accurate
primary raw data in any given collection time. Therefore accuracy improvement
is different for workflow in accessible regions with good transport. Workflow inlimited-communication regions will be different.
4.3 Accuracy improvement during run collection of cellular-phone-based personal
interview can be achieved in several subtle steps such as adherence to standard operating procedure improvement. Always recharge before power supply depletion.
Prepare fully recharged portable power bank. Refresh cellular-phone memory toarbi·setiawan

175

FIGURE 4. Sub process 3·3 only gsbpm version four.

accommodate raw data. Prepare secondary portable magnetic storage. Remotesmall region needs back up of second generation (2G or 2·5G) cellular-phone.

FIGURE 5. Sub process 4·3 only gsbpm version four.
5.3 One acceptable fact on big data is that any particular raw data may haveaccumulated in some sort of series. Depending on intelligence of raw data analyst
this sort of series can be used to review for accuracy improvement.

FIGURE 6. Sub process 5·3 only gsbpm version four.
House price data in any region may be associated to opening of newly-built nonintersection road. House price data in other region may be associated to additionof toll-road access. House price may be related more to development of commuter
railway and less to development of high speed train railway.
6.3 First sub process establish output objectives expect a responsibility to scrutinize raw data. Contrasting objectives against preliminary product is carried out.
Raw data originating from unusual circumstances deserves different treatment
to improve accuracy. Unusual circumstances applies both to more favorable condition as well as less favorable condition. Preliminary product scrutiny is donewith ease in case of way ahead of schedule activities or any expense far less thanactivities budget.

FIGURE 7. Sub process 6·3 only gsbpm version four.
7.3 Central bank publishes result of residential property survey for primary house.
First page of report has shown methodology of quarterly survey covering sixteen

176

Accurate with GSBPM (part one)cities spread over 15 regional offices. The report does not mention stratification explicitly. Reader of report may to certain extent interpret that Balikpapan
represents a stratum of Eastern Kalimantan and and newly renamed Northern
Kalimantan. Interpret that Manado represents a stratum of Northern Sulawesi
and Gorontalo and Western Sulawesi. To be more accurate central bank may
conduct a press conference.

FIGURE 8. Sub process 7·3 only gsbpm version four.

Acknowledgments: The author and co-authors declared there is not any potential conflict of interest with respect to the research, authorship, and/or publication of this article. The views expressed here are those of the individual author
and co-authors and not necessarily those of STIS or its board, or officers, or staff.

References
McCullagh, Peter. (2002). What Is A Statistical Model?. In: Proceedings of The
Annals of Statistics, Volume 30, University of Chicago, 5, 1225 – 1310 see
page 1247.
UNECE Secretariat. (2009). Generic Statistical Business Process Model Version
4.0. UNECE Secretariat Steven Vale (steven.vale@unece.org), based on
previous work by Statistics New Zealand (for the first seven phases) andStatistics Canada (for the Archive phase), with considerable input andfeedback from the members of the METIS group.

Imputation in High-dimensional Mixed-Typedata by Nearest Neighbors
Shahla Faisal1 , Gerhard Tutz1
1

Ludwig-Maximilians-Universität München, Germany

E-mail for correspondence: shahla.ramzan@stat.uni-muenchen.de
Abstract: In modern industrial and biomedical research, the data often contains
a large number of variables measured at mixed data types (continuous, nominalor binary) but the information on some variables is missing. Imputation is a common solution where the downstream analyses require a complete data matrix. A
number of imputation methods are available that work under some distributional
assumptions. We propose an improvement over the popular nonparametric nearest neighbor imputation method that requires no particular assumptions. The
proposed method makes practical and effective use of the information on association among the variables. In particular we propose a weighted version of Lq
distance for mixed-type data that uses the information from a subset of importantvariables only. The performance of the proposed method is investigated under avariety of data settings. The results show a smaller imputation error and better
performance when compared to other approaches. It is shown that the proposed
imputation method works efficiently even when the number of samples is lessthan the number of variables.
Keywords: Weighted Nearest neighbors; Missing values; Mixed-type data; Kernel function; High-dimensional data.

1

Introduction

Missing values are a common phenomenon in practical research. A number ofdifferent methods are available that can be used to fill the missing values inmetrically scaled data only (e.g. Troyanskaya, et al, 2001) or handle nominal data
only (e.g. Schwender, 2012). Since one has to deal with combination of continuousand nominal variables in many real world applications, the methods to impute
mixed data become more crucial. Since the multiple imputation techniques failto impute high dimensional missing data, the nonparametric simgle imoutation
methods are gaining more popularity. We propose an improved version of the
popular nonparametric nearaest neighbors method which uses information onlyThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

178

Imputation in High-dimensional Mixed-Type data by Nearest Neighborson potentially relevant neighbors to impute missinng values. More specifically,
We introduce a distance function that is more appropriate for mixed data. It isan extension of Tutz and Ramzan (2015) and uses information on association
among variables.

2

Weighted Distance for Mixed-type Data

Let R = (Ris ) be the n×(p+m) data matrix with p continuous and m categoricalcovariates defined by R = (X, Z), where X = (xig ), g = 1, · · · , p, with xig
denoting ith observation of the g th continuous covariate, and Z = (zil ), l =1, · · · , m, with zil denoting ith observation of the lth categorical covariate. Let
O = (ois ) is the corresponding n × (p + m) matrix with ois = 1 if Ris is observed,
otherwise ois = 0.
The categorical observations zil in the data matrix, can assume values cl ∈{1, . . . , kl }, l = 1, . . . , m, where kl is the number of categories that the lth attribute can take. Then the ith row of the data matrix R can be written as
RiT = (xTi , ziT )with xTi = (xi1 , · · · , xip ) and ziT = (zi1 , · · · , zim )For the computation of distances, the categorical variables are transformed intoTbinary variables. Thus the observation zil becomes a vector, zil= (zil1 , . . . , zilks )with zilr = 1 if Zil = r. Thus, the ith row of the transformed matrix RD has theform
TT[(xi1 , . . . , xip ), (zi1
, · · · , zim)]Twith dummy vectors zil , l = 1, · · · , m.
Let Ris be a missing entry in the data matrix R, that is Ois = 0. Then the
distance between the i-th and the j-th rows specific for a missing value in thesth covariate is defined byd(Ri , Rj )

=

γ1

p
X

|xig − xjg |q I(oig =1) I(ojg =1) .C(δsg ) +

g=1

γ2

kl
m X
X

!1/q
q

|zilc − zjlc | I(oil =1) I(ojl =1) .C(δsl ),(1)

l=1 c=1

where I(.) .I(.) is the number of valid components. C(.) is a convex function definedon the interval [−1, 1]. We use C(δsl ) = |ρsl |c , where c is a tuning parameter. γ1and γ2 are tuning parameters with γ1 = 1 − γ2 and δsl is a measure of association
between covariates s and l.

3

Weighted Imputation by Nearest Neighbors

Let a value is missing in the ith row of the data matrix R. One has two possible
options: (i) metric covariate has the missing value (ii) a categorical covariate has
the missing value. One finds k nearest neighbor observation vectors Rk based on
the distancesDD
R(1), . . . , R(k)withd(Ri , R(1) ) ≤ · · · ≤ d(Ri , R(k) )Faisal and Tutz

179

Imputing Categorical Missing Value For the imputation of the values
zis , we use the weighted estimatorπ̂isc =kX

w(Ri , R(j) )z(j)sc ,

(2)

j=1

where c = 1, . . . , ks . The weighted imputation estimate of a categorical missing
value zis issẑis = arg maxkc=1π̂isc ,

(3)i.e. the value of c ∈ {1, . . . , ks } with highest value of π̂ is .

Imputing Continuous Missing Value The weighted imputation estimateof continuous missing value, xis , is defined byx̂is =kX

w(Ri , R(k) )x(j)s

(4)

j=1

The weights in equation (2) and (4) are defined byk(d(Ri , Rj )/λ)w(Ri , Rj ) = Pk,
l=1 k(d(Ri , Rj )/λ)(5)where K(.) is a kernel function and λ is a tuning parameter. We use Gaussian
kernel as in the initial simulations it yielded smaller imputation errors. The tuning
parameter λ is chosen by cross validation.

4

Simulation Study

We generated S = 200 samples of size n = 100 for p = 30 predictors drawn froma multivariate normal distribution with N (0, Σ). The correlation matrix Σ has
an autoregressive type of order 1 with ρ = 0.9. Some randomly selected variables
are converted to nominal scale. We construct categories from the continuous databy setting cut points. In each sample, miss = 10%, 20%, 30% of the total values
were replaced by missing values completely at random (MCAR).
As initial step we investigate the performance of different kernel functions (Gaussian and Triangular), value of q (q =1,2) and the convex functions (linear andpower). The results showed that the Gaussian kernel for q = 2 using power function provides smallest imputation error. We use these findings to compare theperformance of wNNSelmix with existing methods. For comparison purpose, we
compute imputation errors for continuous and categorical variables separately
and add them to get final imputation error.
The weighted nearest neighbors approach (wNNSelmix ) is compared with k-nearest
nrighbors (kNN), multiple imputation by chained equations (MICE) and random
forest (RF) methods. Figure 1 shows the results for 10% missing data. Clearly,
the proposed weghted nearest neighbors method provides smallest imputation
errors.

Imputation in High-dimensional Mixed-Type data by Nearest Neighbors
miss = 10%

●
●

●
●

0.6

Imputation Error
0.8
1.0
1.2

1.4

180

N
kN

CE

MI

RF

ix

N
wN

lm
Se

FIGURE 1. Boxplots of average imputation error obtained by different imputation methods. Solid circles within boxes show mean values.

References
Tutz, G. and Ramzan, S. (2015). Improved methods for the imputation of missing data by nearest neighbor methods. Computational Statistics and Data
Analysis, 90, 84 – 99.
Schwender, H. (2012). Imputing missing genotypes with weighted k nearest
neighbors. Journal of Toxicology and Environmental Health, Part A, 75(810), 438 – 446.
Troyanskaya, O., Cantor, M., Sherlock, G., Brown, P., Hastie, T., Tibshirani, R.,
Botstein, D., and Altman, R. B. (2001). Missing value estimation methods
for DNA microarrays. Bioinformatics, 17 (6), 520 – 525.

Using hierarchical hidden Markov models forjoint inference at multiple temporal scales
Timo Adam1 , Vianey Leos-Barajas2 , Roland Langrock1 , Floris
M. van Beest3
1

Bielefeld University, Germany
Iowa State University, USA
3
Aarhus University, Denmark2E-mail for correspondence: timo.adam@uni-bielefeld.de
Abstract: Hidden Markov models are prevalent in the field of animal movement
modeling, where they can be used to infer behavioral modes from various typesof movement data. Due to substantial improvements in tagging technology, suchdata can nowadays be collected at much finer time scales than only a few years
ago. Behavioral modes, however, do not necessarily manifest themselves at thefine scales at which the data are collected, but may effectively operate on much
cruder time scales. To address the mismatch between data resolution on the
one hand and biologically meaningful resolution on the other hand, we discussa modeling framework that allows to jointly infer behavioral modes at multiple
temporal scales. The approach is illustrated by modeling vertical movements ofa harbor porpoise throughout the northeastern part of the North Sea.
Keywords: animal movement; statistical ecology; time series.

1

Introduction

In recent years, hidden Markov models (HMMs) and related state-switching models have emerged as increasingly popular tools for the analysis of animal behavior
data (see, e.g., Leos-Barajas et al., 2017, DeRuiter et al., 2017), where they provide a versatile framework to infer behavioral modes (e.g. foraging or traveling)from various types of movement or general behavior data. Some types of animal
behavior of potential interest, e.g. migratory behavior, may however not directly
manifest themselves at the fine temporal grids at which data nowadays tend to
be collected, but may rather be operating on much cruder scales.
As an example, the vertical movements of a harbor porpoise can be observed at
a dive-by-dive resolution. However, behavioral modes such as foraging or traveling cannot easily be inferred directly from single dives (which may for example
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

182

Hierarchical hidden Markov models

be shallow or deep), but rather become apparent from collections of dives (e.g.
a sequence of many shallow dives, occasionally interspersed with deeper dives,
typically indicates traveling behavior).
Here we discuss an extension to the basic HMM framework that regards thetime series observations as stemming from two different underlying behavioral
processes that operate on different time scales. The first process, which we callthe internal state process, determines the behavioral mode at the crude scale,
while the second process, which we call the production state process, determines
the fine-scale mode within the crude-scale mode.

2

Methodology

A basic HMM comprises an observable state-dependent process {Yt }Tt=1 which isdriven by an unobservable state process {St }Tt=1 . The state process is determinedby an N -state Markov chain with transition probability matrix (t.p.m.) Γ =(γij ), where γij = Pr(St = j | St−1 = i), i, j = 1, . . . , N . The initial distributionδ = (δi ), where δi = Pr(S1 = i), gives the probability of the first observation
y1 belonging to state i. Given St = i, the state-dependent process is assumed
to generate an observation from a state-dependent distribution with density (orprobability) function fi (yt ).
To extend the basic framework in a way that allows for joint inference at multiple
temporal scales, we initially segment the fine scale (e.g. dive-by-dive) observations
into M distinct (e.g. hourly) chunks, denoted by ym , m = 1, . . . , M . Each chunkis connected to one of K production HMMs, which generate the observations at(P )the fine scale. The likelihood Lk,m of the mth chunk being generated by the kth
production HMM is given by
(P )

(P )

Lk,m = δk P(P ) (y1 )

T
Y

(P )

Γk P(P ) (yt )1,

(1)

t=2
(P )

where P(P ) (yt ) = diag(f1 (yt ), . . . , fN (yt )). Considering the likelihoods Lk,m , anadditional, internal K-state Markov chain {Hm }M
m=1 is considered to model theswitches among the K production HMMs, so that the likelihood L(I) of thehierarchical HMM is given by
L(I) = δ (I) P(I) (y1 )MY

Γ(I) P(I) (ym )1,

(2)

m=2
(P )

(P )

where P(I) (ym ) = diag(L1,m , . . . , LK,m ). The model structure is illustrated in
Figure 1. Notably, due to the simplifying conditional independence assumptions,the likelihood can efficiently be evaluated and hence directly (numerically) maximized (Zucchini et al., 2016).

3

Illustrating example

To illustrate the application of the suggested methodology, we model verticalmovements of a harbor porpoise in the northeastern part of the North Sea. Raw

Adam et al.

···

···

···

Yt 1,mYt,m

Yt+1,m

St 1,mSt,mSt+1,m

···

···

183

observable···

hidden
···

Hm 1

Hm

Hm+1

···

Figure 1: Dependence structure in hierarchically structured HMMs.

FIGURE 1. Dependence structure in hierarchical HMMs.
the K ⇥ K t.p.m. for the internal state process, and define

data on dive depth were processed into measures of the dive duration, the maximum depth and thedive wiggliness (as represented by the absolute vertical disP(I) (ym ) = diag Lp (ym |Hm = 1), . . . , Lp (ym |Hm = K) .
tance covered at the bottom of each dive) to characterize the porpoise’s verticalmovements at a dive-by-dive resolution.
The Thefittedstate-dependent gamma distributions (extended by a zero point mass
likelihoods Lp (ym |Hm = k), k = 1, . . . , K, have the form as given in (1), andfor the dive wiggliness) displayed in Figure 2 suggest three different dive types.
only 1vary
across kthein shortest,
terms of theimplied production-levelt.p.m.
(andpotentially
also
State
capturesshallowest
and smoothest
dives
with
small variance.
State 3 exhibits opposite features, namely very long, deep and wiggly dives, withthe production state-dependent distributions). Then the likelihood for the hierarchicallyhighvariation. State 2 lies in-between these two extremes, with all three variables
takingon moderately
large values.
structured
HMM is,Mfor the internal state process were estiThe off-diagonal transition probabilitiesY(I) (I)(I) (I)(y1 )The corresponding
P (ym )1
mated as γ̂12 = 0.211 andLhγ̂=0.219.
stationary distribution
21 = P
m=2
is (0.509, 0.491), indicating that about half of the observations are generated byeachof estimatedthe two production
production
HMMs.
The t.p.m.sforbehaviors,
the twoallowing
production
As thestatesare generally
proxies forfor onlyHMMs,
thedetermining the switching behavior among the three production states, were est.p.m. toas
vary across the K HMMs leads to an interpretation of the K internal states (loosely
timated
connected to K0.406
behavioral
processes)
as distinct manners in which
an animal
will persist
0.277
0.153
0.570 and
0.443
0.150
(P
)
(P )
= 0.124 0.248 0.628 ,= 0.240 0.600 0.159 and Γ̂Γ̂
switch1 among the production states. As long as the2individual time series’ likelihoods, Lp ,0.057 0.087 0.856
0.196 0.366 0.437
can be evaluated in an efficient manner, we can evaluate the likelihood of the hierarchicallywith corresponding stationary distributions (0.277, 0.506, 0.217) and (0.083, 0.110, 0.807),
structured HMM
via theforwardalgorithm,
generalnot di↵er
respectively.
These
figures
implythat since
whenthethe
first structure
HMM isdoes
active,thenfromin thelong run about 28%, 51% and 22% of the observations are generated in producthat of the basic HMM, and thus maximize it directly (Zucchini et al., 2016). The Viterbition states 1, 2 and 3, respectively, whereas when the second HMM is active, thenabout
8%,can11%
81%
of theare generated
in sequence
the respective
production
algorithm
be andused forglobalstatedivesdecoding,
i.e. finding theof the most
likelystates. Biologically, the second production HMM can be seen as a proxy of foraging behavior, which particularly manifests
itself in the extensive dive wiggliness9
suggesting prey-chasing. The first production HMM, which is characterized bya large proportion of short, shallow and smooth dives, could be associated withresting or traveling behavior. Further differentiation would require the inclusionof additional variables, e.g. the horizontal step length performed during dive.

4

Conclusion

The proposed hierarchically structured HMM constitutes a novel framework allowing for joint inference at multiple temporal scales. Environmental covariates

184

Hierarchical hidden Markov models
0.3

0.5

Pr(X=0|S=1)=0.309
Pr(X=0|S=2)=0.008
Pr(X=0|S=3)=0.000

0.4

0.10

0.05

production state 1

production state 3
0.00
50

100

150

dive duration (seconds)

production state 1
0.2

production state 2
production state 3

0.1

production state 3
0.0

0

0.3

production state 2

0.1

production state 2

densitydensitydensity

0.2

production state 1

0.0
0

20

40

60

maximum depth (meters)

80

0

20

40

60

80

dive wiggliness (meters)

FIGURE 2. Fitted state-dependent distributions.

may be included in the model to investigate their effects on state occupancy
and the dynamics of variation in behavioral modes at cruder scales than thoseat which the data are collected. The suggested methodology can be extended
to allow for simultaneous modeling of multiple data streams collected at different scales (e.g. dive depths, which are observed at a dive-by-dive resolution, andhourly step lengths).

Acknowledgments: The harbor porpoise movement data were collected aspart of the DEPONS project (www.depons.au.dk) funded by the offshore winddevelopers Vattenfall, Forewind, SMart Wind, ENECO Luchterduinen, East Anglia Offshore Wind and DONG Energy.
References
DeRuiter, S.L., Langrock, R., Skirbutas, T., Goldbogen, J.A., Calambokidis, J.,Friedlaender, A.S. and Southall, B.L. (2017). A multivariate mixed hidden
Markov model for blue whale behaviour and responses to sound exposure.
Annals of Applied Statistics, in press.
Leos-Barajas, V., Photopoulou, T., Langrock, R., Patterson, T.A. Murgatroyd,M., Watanabe, Y.Y. and Papastamatiou, Y.P. (2017). Analysis of accelerometer data using hidden Markov models. Methods in Ecology and Evolution,in press.
Zucchini, W., MacDonald, I.L. and Langrock, R. (2016). Hidden Markov Models
for Time Series. An Introduction Using R. Boca Raton: CRC.

Bayesian cure rate models induced by frailty
in survival analysis
Vicente Garibay Cancho1 , Daiane de Souza1 , Josemar
Rodrigues1 , N. Balakrishnan 2
1

Department of Applied Mathematics and Statistics, University of São Paulo,São Carlos, Brazil
2
Department of Mathematics and Statistics, McMaster University, Hamilton,
Ontario, Canada
E-mail for correspondence: garibay@icmc.usp.br
Abstract: Frailty models provide a convenient way of modeling unobserved
dependence and heterogeneity in survival data which, if not accounted for duly,
would result incorrect inference. Gamma frailty models are commonly used forthis purpose, but alternative continuous distributions are possible as well. However, with cure rate being present in survival data, these continuous distributions
may not be appropriate since individuals with long-term survival times encompass zero frailty. So, we propose here a flexible probability distribution induced
by a discrete frailty, and then present some special discrete probability distributions. We specifically focus on a special hyper-Poisson (hP) distribution andthen develop the corresponding Bayesian simulation, influence diagnostics andan application to real dataset by means of intensive Markov chain Monte Carlo(MCMC) algorithm. These illustrate the usefulness of the proposed model as wellas the inferential results developed here.
Keywords: Bayesian inference, frailty models, proportional hazard models, longterm survivors, hyper-Poison distribution.

1

Introduction

Studies on the frailty models generally assume a non-negative and continuous
frailty random variable (Hougaard ,1984). Usually, the gamma distribution, inverse Gaussian and positive stable are used to model the frailty random variable.
From the computational point of view, these distributions are convenient since
it is easy to derive closed-form expressions for the survival, density and hazard
functions using the Laplace transform. However, continuous frailty distribution
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

186Bayesian cure rate models

do not allow zero risk. Zero frailty indicates that there is a subgroup of nonsusceptible individuals among whom the event of interest has not happened evenafter a long period of observation. The event of interest in many survival studies or cancer-relapse trials can be the death of a patient or a tumor recurrence.
However, due to recent advances in (cancer) treatment therapies, a high portionof the subjects are expected to be cured, i.e., remaining disease-free after prolonged follow-ups. For this reason, there is a vast literature on cure rate models
for survival data, also called survival models with a surviving fraction or longterm survival models. Most of these models were obtained in a competing risks
scenario (Tsodikov et al.,2003), but they can also be obtained from the proportional hazard models with discrete frailty distributions (Caroni et al.,2010). In
this paper, we focus on this direction from the Bayesian point of view.

2

The model

The frailty model assumes a proportional hazard structure conditional on the
random effect Z. This random effect is a non-negative frailty variable whichindicates the individual level of risk. So, the frailty model is basically specified
by the following hazard rate function (hrf) h(t|Z) = Z hB (t), where hB (t) isthe baseline hrf that can be equal to hB (t) exp(x> β) in the proportional hazard
scenary (Cox & Oakes, 1984). The corresponding survival function, conditionalon Z, is given by
S(t|Z) = P (T > t|Z) = exp {−ZHB (t)} = SB (t)Z ,(1)
Rt
where HB (t) = 0 hB (u)du is the baseline cumulative hazard function and SB (t)is the corresponding survival function. A new survival model is obtained when
we assume that the frailty variable Z in (1) follows a hyper-Poisson distribution
with probability mass function
P (Z = z; η, θ) = p(z; η, θ) =1
θz
; z = 0, 1, . . . ,1 F1 (1; η; θ) (η)z

for a > 0 , r is a positive
where η, θ > 0, (a)r = a(a + 1) . . . (a + r − 1) = Γ(a+r)Γ(a)P(a)r wr
integer, and 1 F1 (a; b; w) = ∞istheconfluenthypergeometric series.
r=0 (b)r r!
Thus, the unconditional survival function can be expressed as
S(t) =

1 F1 (1; η; θSB (t))
1 F1 (1; η; θ)(2)and the fraction of ”cured” individuals is given by p0 = limt→∞ S(t). From1(2), p0 = limt→∞ S(t) = 1 F1 (1;η;θ)> 0 implying that (2) is not a proper survival function. The density function associated with (2) is given by f (t) =1 F1 (2;η+1;θSB (t)) θf (t), where fB (t) = −dSB (t)/dt. The corresponding hazard
η B
1 F1 (1;η;θ)1 (2;η+1;θSB (t)) θfunction is given by h(t) = 1 F1 F
fB (t). Note that when η = 1,
1 (1;η;θSB (t)) η
the model reduces to the cure rate survival model investigated by Yakovlev &Tsodikov(1996).

Cancho et al.

3

187

Application

We illustrate the application of the proposed model to a data set from a Phase
III cutaneous melanoma clinical trial conducted by the Eastern Cooperative Oncology Group (Kirkwood et al.,2000). These data are part of an assay for theevaluation of postoperative treatment performance with a high dose of a certain
drug (interferon alpha - 2b) in order prevent recurrence. Patients were includedin the study from 1991 to 1995, and follow-up was conducted until 1998. After
deleting subjects with incomplete data and missing observation times, we have asubset of n = 417 patients with approximately 56% of censoring. The observed
time has mean equal to 3.18 and standard deviation equal to 1.69. The following variables were associated with each participant, i = 1, . . . , 417: ti : observed
time (in years); xi1 : treatment (0: observation, n = 204; 1: interferon, n = 213);
xi2 : age (0: ≥ 48 years, n = 197; 1: < 48 years, n = 220); xi3 : nodule (nodule
category is coded from the number of lymph nodes involved in the disease: 1:
n = 111; 2: n = 137 ; 3: n = 87; 4: n = 82); xi4 : sex (0: male, n = 263 ; 1:
female, n = 154); xi5 : p.s (performance status-patient’s functional capacity scale
as regards the daily activities: 0: fully active, n = 363; 1: other, n = 54); xi6 :tumor (tumor thickness in tenth of a millimeter).
We initially consider the hP cure rate model with with all regression variables,θi = exp{β0 + β1 xi1 + · · · + β6 xi6 }, i = 1, . . . , 417.
For the Bayesian analysis, we assume a Weibull distribution for the baseline
distribution function in (1), with SB (t; γ) = exp{(−t/γ2 )γ1 }. From posterior
summaries for the parameters of the hP model with all covariates, we observethrough the percentiles that the variable x3 (nodule category) has a significant
effect in the model since the interval does not contain zero.
TABLE 1. Posterior summaries for the hP model with only nodule category as
covariate fitted to the melanoma dataset.

Parameter

Mean

η
γ1
γ2
β0
β3

6.014
1.783
2.867
0.485
0.293

Standard
deviation
0.5414
0.121
0.274
0.169
0.053

Percentile
2.5% 97.5%
5.000 7.000
1.549 2.022
2.477 3.549
0.130 0.808
0.189 0.400

Table 1 shows posterior summaries of the parameters of the simplified hP model
retaining only the nodule category (x3 ) as the covariate.
We finally conclude our melanoma analysis by considering the estimation of thecure rate (p0 ) by employing the hP cure rate model. We stratified the patients
1
according to the nodule category. From Section 2, we have p0j = 1 F1 (1,η,θ
, withj)θj = exp(β0 + jβ3 ) for j = 1, . . . , 4. In Table 2, the posterior summaries revealthat cure rate (p0j ) is decreasing with respect to the j-th nodule category, forj = 1, 2, 3, 4.

188

Bayesian cure rate models

TABLE 2. Posterior summaries of the cure rate p0 for the hP model according
to nodule category (1-4)

Cure rate

Mean

p01
p02
p03
p04

0.654
0.552
0.428
0.289

Standard
deviation
0.040
0.037
0.039
0.051

Percentile
2.5% 97.5%
0.594 0.748
0.497 0.633
0.349 0.491
0.183 0.393

References
Caroni, C., Crowder, M. & Kimber, A. (2010). Proportional hazards models withdiscrete frailty. Lifetime Data Analysis, 16, 374–384.
Cox, D. & Oakes, D. (1984). Analysis of Survival Data. Chapman & Hall, London.
Hougaard, P. (1984) (1984). Life table methods for heterogeneous populations:distributions describing the heterogeneity. Biometrika, 71, 75–83.
Kirkwood et al. (2000).. High- and low-dose interferon alfa-2b in high-risk melanoma:
First analysis of Intergroup Trial E1690/S9111/C9190. Journal of ClinicalOncology, 18, 2444–2458.
Tsodikov, A. D., Ibrahim, J. G. & Yakovlev, A. Y. (2003). Estimating cure rates
from survival data: An alternative to two-component mixture models. Journal of the American Statistical Association, 98, 1063–1078.
Yakovlev, A. Y. & Tsodikov, A. D. (1996). Stochastic Models of Tumor Latency
and Their Biostatistical Applications

Robust Generalised Autoregressive Score
ModelsBernardi Mauro1 , Ruli Erlis1
1

Department of Statistical Sciences, University of Padova, Italy

E-mail for correspondence: mauro.bernardi@unipd.it
Abstract: Dynamic conditional score models are observation driven alternatives to classical state space models which are optimal in the sense of reducing
the local Kullback–Liebler divergence between the true and the model implied
conditional density. In this paper we apply the density power divergence approachof Basu et al. (1998) to robustify the dynamic conditional score equation withrespect to outliers. The density power divergence approach minimises an appropriately modified divergence governed by an additional parameter that controls
for the efficiency–robustness trade–off. Our approach combines the optimality ofthe score dynamics with the robustness properties of the class of M–estimatorsand it is a viable alternative to the Student–t approach of Harvey and Luati(2014). The proposed approach is illustrated through two examples of time series
being strongly affected by outlying observations.
Keywords: Generalised autoregressive score models, robustness, minimum distance estimation, estimating equations.

1

Introduction

The class of Generalised Autoregressive Score (GAS) processes, recently introduced by Creal et al. (2013) and Harvey (2013), is becoming increasingly popular tool for modeling the time varying behaviour of unobservable parametersin an observation driven environment. The key feature of GAS models is thatthe predictive score of the conditional density is used as forcing variable into the
updating equation of a time–varying parameter. Two main reasons for adopting
the GAS updating mechanism has been advocated by the literature. Firstly, theGAS dynamics can be seen as an approximation to a filter for a model driven by astochastic latent parameter that is by definition unobservable. Secondly, the conditional score can be considered as a steepest ascent direction for improving themodel’s local fit given the current parameter, position as usually happens into a
Newton–Raphson type algorithm. Moreover, the class of GAS models nests a large
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

190Robust GAS

number of observation driven models, such as the ARCH–type models of Bollerslev (1986). As a practical justification the GAS updating mechanism circumvents
the problem of choosing a non–adequate forcing variable. Score driven processes
have been proved to be effectively used in many applications. Harvey and Luati(2014) and Caivano and Harvey (2014) motivate the use of score dynamics by
the need to reduce the potentially relevant effect of outliers on the extracted signal by assuming a fat–tailed distribution for the conditional innovation. In thispaper, we adopt a different perspective and we robustify the dynamic conditionalscore equation to incidental influential observations and outliers by means of theDensity Power Divergence (DPD) approach of Basu et al. (1998). Our approachcombines the optimality of the score dynamics with robustness properties of theclass of M–estimators and it is a viable alternative to the Student–t approach ofHarvey and Luati (2014).

2

GAS models

Formally, assume that the observed random variable yt ∼ p yt | ϑt|t−1 , η , wherep (·) is a probability density and ϑt|t−1 is a set of time–varying parameters andη is a vector of time–independent parameters. The GAS(1,1) updating equation
for the time–varying parameters ϑt|t−1 isϑt+1|t = ω + ϕϑt|t−1 + κst(1)st = St ϑt|t−1 , η ∇ yt , ϑt|t−1 , η ,(2)where ∇ yt , ϑt|t−1 , η is the score of the conditional density function of the observed random variable evaluated at ϑt|t−1 , i.e.
∂ ln p yt , ϑt|t−1 , η∇ yt , ϑt|t−1 , η =,∂ϑt|t−1and St ϑt|t−1 , η is a positive definite, possible
parameter–dependent scalingmatrix. A convenient choice for St ϑt|t−1 , η is usually given by St ϑt|t−1 , η =
h
i
−α


I ϑt|t−1 , η, where I ϑt|t−1 is the Fisher Information matrix and α =
0, 12 , 1 .

3Robust extension

The density power procedure has been introduced by Basu et al. (1998) as alternatives to the classical maximum likelihood approach for robust parameter
estimation. Specifically, let pϑ (Y ) be a parametric family of distributions indexed by the parameter θ and let qϑ0 (Y ) be the true unknown density of therandom variable Y . To estimate the unknown parameter ϑ0 , Basu et al. (1998)
proposed the family of density power divergences which depends on the tuning
parameter β > 0, defined as



1+β
q
(Y )
R β+1pϑ (Y ) − 1+βqϑ0 (Y ) pβϑ (Y ) + ϑ0 β
dY, β > 0βDβ (pϑ , qϑ0 ) = R(3) q (Y ) (log q (Y ) − log p (Y )) dY,
β = 0,
ϑ0ϑ0ϑBernardi and Ruli

191

and discussed the corresponding estimator obtained by minimising the empiricalcounterpart of equation (3). The tuning parameter β controls for the trade–off
between robustness and efficiency. Here, we apply the methodology of Basu etal. (1998) to robustify dynamic conditional score models. The advantages of robustifying the DCS models using the approach of Basu et al. (1998) are twofold.
First, it provides a general estimation methodology that bridges the gap between
the maximum likelihood and the robust L2 estimation where deviations fromefficiency are controlled for by the tuning parameter β. Second, it allows forthe definition of a robustified score entering the transition equation as forcing
variable. Furthermore, coherently with the theory of the DCS models, the robustified version of the score is defined as the first derivative of the probability
density function of the observed variable and the robustified score is redescending. The robustified version of the GAS updating equation for the time–varying
parameters defined in equations (1)–(2) becomes
ϑt+1|t = ω + ϕϑt|t−1 + κs∗ts∗t = St∗ ϑt|t−1 , η ∇∗ ϑt|t−1 , η − ξ ,

(4)
(5)



β
roubstified score
where ∇∗ yt , ϑt|t−1 , η = ∇ yt , ϑt|t−1 , η p yt , ϑt|t−1 , η is


R the
∗
yt ,ϑt|t−1 , η p yt , ϑt|t−1 , η dytof the conditional density function, evaluated at ϑt|t−1
,
ξ
=
∇

is the mean of the robustified score and St∗ ϑt|t−1 , η = Jβ ϑt|t−1 , η is a positive
defined, possible parameter–dependent scaling matrix with
Z


0

Jβ ϑt|t−1 , η = ∇∗ z, ϑt|t−1 , ϑ ∇∗ z, ϑt|t−1 , η p z, ϑt|t−1 , η dz − ξξ 0 .

4Application

We apply the robust GAS to the problem of signal extraction from two series
which are characterised by the presence of extreme outliers: the daily series ofelectricity prices from the Nord Pool Energy Market from January 1, 2013 toNovember 30, 2016 and the series of US average weekly hours in manufacturingfrom 1940 to 2016. The two series are plotted in the top panels of Figure 1. The superimposed blue and red lines denote the signal extracted using a GAS(1,1) modelwith Gaussian innovations and its roubstified version with β = 0.05, respectively.
The optimal tuning parameter has been selected as suggested by Ghosh and Basu
(2013). Bottom panels plot the dynamic evolution of the scaled score that enters the GAS updating equation for the two models. It is evident that the signal
extracted by the robust GAS filter is less influenced by the presence of outliers
than the filter based on the Gaussian GAS model.

References
Basu, A., Harris, I.R., Hjort, N.L., and Jones, M.C. (1998). Robust and efficientestimation by minimising a density power divergence. Biometrika, 85, 549 –559.
Bollerslev, T. (1986). Generalized autoregressive conditional heteroskedasticity.
Journal of Econometrics, 31, 307 – 327.

192

Robust GAS
150

44
42

100
40

50
38

0
01/13

09/13

05/14

01/15

09/15

06/16

06/49

02/58

10/66

06/75

02/84

10/92

06/01

06/49

02/58

10/66

06/75

02/84

10/92

06/01

0

0
−2
01/13

36
10/40
50

2

09/13

05/14

01/15

09/15

06/16

−50
10/40

FIGURE 1. Signal extraction form the series of electricity price for the Nord Poolmarket and the series of US average weekly hours in manufacturing.

Caivano, M. and Harvey, A. (2014). Time series models with an EGB2 conditional distribution. Banca d’Italia.
Creal, D., Koopman, S.J., and Lucas, A. (2013). Generalized autoregressivescore models with applications. Journal of Applied Econometrics, 28, 777 –795.
Ghosh A. and Basu, A. (2013) Robust estimation for independent non-homogeneous
observations using density power divergence with applications to linear regression. Electronic Journal of Statistics, 7, 2420 – 2456.
Harvey, A. and Luati, A. (2014). Filtering with heavy tails. Journal of theAmerican Statistical Association, 109, 1112 – 1122.
Harvey, A.C. (2013). Dynamic Models for Volatility and Heavy Tails: With Applications to Financial and Economic Time Series, volume 52. Cambridge
University Press.

Monte Carlo modified profile likelihood insurvival models for clustered censored data
Claudia Di Caterina1 , Giuliana Cortese1 , Nicola Sartori1
1

Department of Statistical Sciences, University of Padova, Italy

E-mail for correspondence: dicaterina@stat.unipd.it
Abstract: When analyzing failure time datasets within stratified contexts, themain focus is usually not on the clustering variables and hence the group-specificparameters are treated as nuisance. If a fixed effects formulation is preferred andthe total number of clusters is large relative to the single-stratum sizes, standard
frequentist techniques are often misleading and the use of adjustments to makereliable inference on the parameter of interest is complicated by the presence of
censored data. Here we show how Monte Carlo simulation may be exploited tocompute a modification of the profile likelihood in general regression settings forsurvival models with unspecified censoring mechanism.
Keywords: Fixed effects; Incidental parameters; Monte Carlo simulation.

1

Introduction

Stratification of time-to-event data subject to censoring is frequent in many applied areas, although the primary concern of the study is typically not the intercluster variability. To avoid the assumptions imposed on the survival analysis byfrailty models, a fixed effects approach considering the group-related traits as
unknown nuisance parameters can be adopted.
Under such a formulation, if the amount of groups is much larger than the withincluster sizes, usual asymptotic results are no longer valid and maximum likelihood
inference on the component of interest may be inaccurate (Sartori, 2003). One
solution to tackle this issue is offered by the modified profile likelihood (Severini,1998). Its good performance has been proved in the literature for various models
with incidental parameters, but in regression contexts with failure data it isunclear how to properly deal with random censoring schemes (Pierce and Bellio,2006). We present here a convenient strategy to calculate the modified profile
likelihood in stratified regressions for survival times with unspecified censoringmechanism.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

194Monte Carlo modified profile likelihood for clustered survival data

Cortese and Sartori (2016) opted for eliminating the group-specific parametersfrom the likelihood of Weibull-distributed survival data by integration. The samemodel, within an extended regression framework, is studied below to illustrate ourless computationally intensive procedure, whose basic principle is yet immediately
applicable to different supposed distributions.

2

Modified profile likelihood in survival models under
unknown random censoring scheme

2.1

Weibull model

Let independent grouped failure times ỹit ≥ 0, i = 1, . . . , N, t = 1, . . . , Ti , berealizations of a Weibull random variable with probability density function
ξ−1

ξ
pit (ỹit ; ψ; λi ) = ηit ξ ηit ỹitexp − ηit ỹit,(1)Twhere ηit = e−(λi +xit β) . The parameter of interest is ψ = (ξ, β), while the nuisancecomponent consists of the single intercepts λ = (λ1 , . . . , λN ). One can see thatξ > 0 is the common shape parameter and β = (β1 , . . . , βk ) contains the regression
coefficients associated with the k-dimensional vector of fixed covariates xit .
Failures are right-censored, so data are observations from the random pair Yit , ∆it ,
where Yit = min Yeit , Cit with Cit censoring time and ∆it is the censoring indicator such that ∆it = 1(0) if Yeit ≤(>)Cit . The censoring mechanism is only hypothesized to be independent and non-informative, thus each Cit is unrelated to theother survival or censoring times and its continuous distribution does not dependon θ = (ψ, λ). In particular, we avoid to parametrically specify the distributionof Cit .
PT
For ease of exposition, suppose T1 = . . . = TN = T . Writing δi· =t=1 δit andPN
δ·· = i=1 δi· allows to express the profile log-likelihood for ψ as
)
(
T
N X
T
N
X
X
X
ξ −ξ(xT
it β)
−ξ
yit
e
δit (xT
δi· log δi· − loglP (ψ) =
it β)
i=1

t=1

+ δ·· (log ξ − 1) + (ξ − 1)i=1 t=1
N X
TX

δit log yit ,i=1 t=1ˆ β̂). Note that the ith element of the constrained estimate
with maximizer ψ̂ = (ξ,λ̂ψ = (λ̂1ψ , . . . , λ̂N ψ ) is obtained as a function of ψ by equating to 0 the scalarpartial score
lλi (θ) = −ξδi· + ξTX(ηit yit )ξ ,i = 1, . . . , N,

(2)

t=1

and solving the equation for λi . We define then θ̂ψ = (ψ, λ̂ψ ) and θ̂ = (ψ̂, λ̂ψ̂ ).

2.2

Monte Carlo modified profile likelihood

The modified profile log-likelihood (MPL) of Severini (1998) takes the formlM (ψ) = lP (ψ)+M (ψ), where the modification term is expressed by

Di Caterina et al.

M (ψ) =

N 
X
1
i=1

2


log jλi λi (θ̂ψ ) − log Iλi λi (θ̂ψ ; θ̂) .

195

(3)

While jλi ,λi is readily available from the double differentiation of the log-likelihoodwith respect to λi , explicitly calculating the expected value Iλi λi (θ̂ψ ; θ̂)=Eθ̂ lλi (θ̂ψ )lλi (θ̂)
requires a parametric assumption on the distribution of Cit . Such restriction is
not needed to compute the MPL via Monte Carlo simulation instead, adapting
the general idea already used by Bartolucci et al. (2016) in econometric fixed
effects models for panel data.
The expectation in (3) can be approximated through the empirical meanR

Iλ∗i λi (θ̂ψ ; θ̂) =1 X rlλ (θ̂ψ )lλr i (θ̂),
R r=1 ii = 1, . . . , N,(4)where, in our case, lλr i is the partial score (2) for the rth Monte Carlo sample
r
r
(yit
, δit
), derived nonparametrically via the Kaplan-Meier estimator. Eventually,rrr
we have yit= min(ỹit, crit ) with corresponding indicators δit.

3

Simulation studies

Two experiments with 2000 iterations are run to discuss inference on ψ under
diverse overall proportion of censored data (Pc = 0.2, 0.4) in the survival model
described above, with k = 2. Subjects of comparison are lP (ψ) and its Monte Carloadjustment lM ∗ (ψ) = lP (ψ)+M ∗ (ψ), where M ∗ (ψ) equals (3) with Iλi λi (θ̂ψ ; θ̂)replaced by (4). Dimensions of the artificial datasets are T = 4, 6, 10 and N =50, 100, 250. In every group, the first covariate is set to 0 if t = 1, . . . , T /2 and to 1otherwise, the second is sampled independently from a N (0, 1) distribution. We
suppose ξ = 1.5 and β = (−1, 1), while the independent incidental parameters are
drawn as λi ∼ N (0.5, 0.52 ). Censoring times are realizations of an Exp(ς) randomvariable, with ς chosen according to the selected Pc like in Cortese and Sartori(2016, Sec. 5).
Table 1 shows some measures of inferential accuracy for ξ from the study referred
to Pc = 0.2. Empirical bias (B), root mean squared error (RMSE) and ratio
SE/SD, where SE is the average over simulations of likelihood-based estimatedstandard errors and SD is the standard deviation of the estimates, are reported
along with actual coverages of 0.95 Wald confidence intervals (CI). These results,
similar to those recorded when Pc = 0.4 and for β, attest how the Monte Carlomodification remarkably improves point and interval estimations supplied by the
profile likelihood.

References
Bartolucci, F., Bellio, R., Salvan, A. and Sartori, N. (2016). Modified profile likelihood for fixed-effects panel data models. Econometric Reviews, 35, 1271 –1289.
Cortese, G. and Sartori, N. (2016). Integrated likelihoods in parametric survival
models for highly clustered censored data. Lifetime Data Analysis, 22,382 – 404.
Davison, A.C. and Hinkley, D.V. (1987). Bootstrap Methods and their Application. Cambridge University Press.

196

Monte Carlo modified profile likelihood for clustered survival data

TABLE 1. Inference on ξ in the stratified Weibull regression model with unknown
censoring distribution and Pc = 0.2. Figures based on 2000 trials and R = 500.
N

T

Method

B

RMSE

SE/SD

0.95 CI50

4lP (ψ)
lM ∗ (ψ)

0.392
0.010

0.418
0.112

0.858
0.979

0.111
0.956

6

lP (ψ)
lM ∗ (ψ)

0.231
0.008

0.252
0.088

0.884
0.964

0.291
0.943

10

lP (ψ)
lM ∗ (ψ)

0.124
0.005

0.141
0.061

0.976
1.029

0.517
0.961

4

lP (ψ)
lM ∗ (ψ)

0.371-0.006

0.385
0.079

0.840
0.966

0.015
0.936

6

lP (ψ)
lM ∗ (ψ)

0.219-0.000

0.230
0.060

0.903
0.987

0.063
0.947

10

lP (ψ)
lM ∗ (ψ)

0.119
0.001

0.128
0.044

0.938
0.989

0.259
0.943

4

lP (ψ)
lM ∗ (ψ)

0.366-0.009

0.372
0.050

0.847
0.972

0.000
0.939

6

lP (ψ)
lM ∗ (ψ)

0.214-0.005

0.218
0.039

0.890
0.972

0.000
0.934

10

lP (ψ)
lM ∗ (ψ)

0.116-0.002

0.120
0.028

0.943
0.993

0.018
0.949

100

250

Pierce, D.A. and Bellio, R. (2006). Effects of the reference set on frequentist inferences. Biometrika, 93, 425 – 438.
Sartori, N. (2003). Modified profile likelihoods in models with stratum nuisanceparameters. Biometrika, 90, 533 – 549.
Severini, T.A. (1998). An approximation to the modified profile likelihood function. Biometrika, 85, 403 – 411.

Using State Space models to track Measles
Infections
Kirsten Eilertson1 , Matthew Ferrari2 , John Fricks3
1

Penn State Statistics Department, USA
Penn State Center for Infectious Disease Dynamics, USA
3
Arizona State Statistics Department, USA
2E-mail for correspondence: eilertson@psu.edu
Abstract: We present a state space model for estimating the measles disease
burden at the country level. Our approach estimates model parameters of a dynamic epidemic model using surveillance data from each country, and builds offof previous work by incorporating age distribution information and explicitly
estimating the uncertainty in the estimates for use in forward projections. The
primary aim of this work is to improve our understanding of the impact of publichealth measures.
Keywords: state space model; disease burden; particle filters.

1

Introduction

We present a state space or hidden markov model of measles disease burden,
and estimate parameters of the model from surveillance data at the country level
collected by the World Health Organization (WHO). The aim is to predict the unobserved values of the number of individuals in a population who are infected andsusceptible each year using the observed information of number of cases reported.
From this we hope to provide valueable information on the impact of regular andsupplemental vaccine campaigns both historically and in the near future. Fitting
the model to each of 193 countries will enable the setting of goals for vaccination
programs, allocation of resources, and evaluation of program success on a country
by country basis.
Our method builds off of earlier work by Chen, Fricks and Ferrari (2012). Like thisprevious work it combines expert knowledge in the dynamics of measles disease
burden with surveillance data, but additionally it incorporates age distributionof the popluation and modifies the model so that parameter estimation and forward projections are tenable. First we present the basic model for progression ofThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

198

Measles Burden Estimation

measles through time and then discuss our approach to fitting the model using aparticle filter.

2

Model

The model form is a dynamic non-linear epidemic model known as susceptibleinfected-recovered or an SIR model. In this framework individuals are assumed tobelong to one of three classes: susceptible (S, after birth), infected (I), and recovered (R, who are immune to subsequent infection) (Anderson and May (1991),
Bjornstad et al. (2002)). The number of susceptibles in year t can be modeled asa function of the number of susceptibles from year t − 1, the number of births (B)that year, the impact of vaccination campaigns, and infections from the previous
year.
St = St−1 + Bt − Vt (Bt , St−1 ) − It−1
Our model specifies that the impact of vaccines on the number of susceptibles
Vt (Bt , St−1 ) incorporates the age distribution of the population, the efficacy of
vaccinations based on age of administration in that country, and “pulsed” orsuplemental vaccine campaigns which target specific age groups.
The resulting number of infections in the year’s susceptible population is a function of the susceptible population size relative to the total population St /Nt andother factors captured with the term et where et ∼ N (0, σe ).
It ∼ Bin(St , πt (St , Nt , et ))St
πt = logit−1 (β0 + β1
+ et )
Nt
Using this logistic form to model the probability of infection as a function of
the proportion of susceptibles in the population is phenomonologically consistent
with herd immunity.
The surveillance data reflects the number of reported cases, Ct , a subset of thenumber of actual cases.
Ct ∼ Bin(It , pt )Our model states that cases are reported with probability p, where p is specificto that country and genearlly assumed to be constant. For some countries, someyears have been identified (based on outside factors) as “high reporting years”.
These are years where due to an epidemic or other reasons a country is thought tohave a higher than usual probability of reporting cases. Thus, the probability ofreporting may vary across these “high reporting” years. These years are usuallyless than 10% of the years observed.

3

Fitting

We are primarily interested in the number of individuals infected each year. We
would like to “predict” these values both for each year in the past It (where t is ayear within the span of surveillance data) conditioning on all years of observed ofdata. Additionally, if our parameters (β0 , β1 , σe ) are well-estimated than we canEilertson et al.

199

use the model to run forward projections under different scenarios of vaccination
campaigns.
To this end we will use a particle filter. A filter in this context allows us to use
the conditional density of the state of a system (the pair [S, I]t in our case) attime t given the observed part of the system up to time t (C1 , ..., Ct ]). Briefly,
the filtering step is as follows:
1. draw proposals [S, I]1t , ..., [S, I]Rt from f ([S, I]) for j = 1, ..., R
2. evaluate conditional likelihood f (Ct |[S, I]jt ) to calulate weight wj
3. resample M observations from [S, I]1t , ..., [S, I]Rt according to weights wj
These steps are then repeated to progress through the data. The evaluation of
the filter provides the value of the likelihood function of the system for a given
set parameter values. This will allow us to use the particle filter to numerically
optimize this likelihood function to give maximum likelihood estimators for theparameters. In addition to the filter, a smooth may also be calculated where allof the observed values of C both past and future states are used to predict It .
In this way, the particle filter approach allows us a “best guess” for the state attime t given the observations through the conditional expectation; as well as ameans to obtain forward projections under various vaccine campaign scenarios.

4

Simulation Example

Using simulated data we examine the reliability and sensitivity of our estimation
procedure. Simulated data is generated by sampling values for β0 , β1 , and p,
as well as sampling a vaccine campaign history. Using preliminary fits to realcountry data from 193 countries, we obtained a range of parameter values to
explore. Refer to Figure 1 for an example of a simulated country history.
From simulations such as these we will establish expected coverage of particle
clouds for past years as well as future projections.

References
Anderson, R. M. and R. M. May (1991). Infectious diseases of humans: dynamics and control. Oxford: Oxford University Press.
Bjornstad, O. N., B. F. Finkenstadt, and B. T. Grenfell (2002). Dynamics of measles
epidemics: Estimating scaling of transmission rates using a time series sir
model. Ecological Monographs 72(2), 169 – 184.
Chen, S., Fricks, J., Ferrari, M (2012). Tracking measles infection through nonlinear state space models. Journal of the Royal Statistical Society: Series C(Applied Statistics) 61.1 117-134.

200

Measles Burden Estimation

FIGURE 1. Particle cloud coverage of number of infections. On the x-axis isyear, and on the y-axis is number of infections. The red dots indicate the “true”
simulated number of infections. The dramatic reduction in measles cases in year
27 is a result of a supplemental vaccine campaign included in the simulation.

DiNAR: web application for Differential
Network Analysis and visualisation in R
M. ZAGORŠČAK1 , A. BLEJEC1 , Ž. RAMŠAK1 , M.
PETEK1 , K. GRUDEN1
1

National Institute of Biology, Slovenia

E-mail for correspondence: maja.zagorscak@nib.si
Abstract: To these days most widely used approach for comparing biologicalsamples under multiple condition is differential expression (DE) analysis. As thegenerated expression data sets are becoming larger and more complex, rather
than to investigate change in the gene behaviour as an isolated occurrence, more
popular question to answer is which parts of system are perturbed and rewired
during an altered response. To access and understand difference in signalling
relationships at the system level differential network analysis (DiNA) is used.
DiNA requires as an input reference and condition specific networks. Accurate
reconstruction of true biological network topology due to a smaller number ofsamples than the number of genes is quite a complicated problem. Majority ofexisting methods are relying on correlation based analysis for construction of gene
regulatory networks which entails noise like spurious inference, reaction oversimplification and missed latent and mediated effects. Other more accurate methods
are computationally exhaustive even when focused only on two conditions and in
the case of long time series experiments completely impractical. As a simple andstraightforward approach we propose to integrate abundant biological knowledge
and condition specific experimental data by mapping obtained DE onto verified
signalling pathways.
Here we present DiNAR, a Shiny web application that allows researcher to analysemultiple condition data sets in the biological network context using an interactive
graphical interface with user defined thresholds and colour palettes. As a reference
network the large knowledge network of Arabidopsis thaliana, created by merging
expert knowledge based interactions with large scale experimentally validated
interactions was introduced. Translation tables based on PLAZA orthologues
information allow the automated transfer of knowledge from model organism
to some non-model plant species. Condition specific networks are automaticallygenerated when experimental data files are uploaded. Changes in processes ofinterest can be monitored through a simple animation.
DiNAR is written in R and customised using JavaScript, HTML and CSS. App isThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

202

DiNAR

accessible over the web at https://nib-si.shinyapps.io/DiNAR or it can be run directly from the GitHub repository using shiny:::runGitHub(”NIB-SI/DiNARscripts”,”NIB-SI”) command.
Although DiNAR is demonstrated on high-throughput biological data it can handle any properly formatted custom network and associated quantitative data set.
Keywords: Condition specific network; Differential network analysis; Shiny

References
Ideker, T. and Krogan, N. J. (2012). Differential network biology. Molecular Systems Biology, 8, 565
Henry,V.J., Bandrowski, A.E., Pepin, A.-S. et al. (2014). OMICtools: an informative directory for multi-omic data analysis. Database (Oxford), 2014,bau069Banf M. and Rhee S.Y. (2017). Computational inference of gene regulatory networks: Approaches, limitations and opportunities. Biochimica et Biophysica Acta (BBA) - Gene Regulatory Mechanisms., 1860, 41 – 52.

Median unbiased estimator for thetwo-parameter logistic model
Euloge Clovis Kenne Pagui1 , Edoardo Michielon2 , Alessandra
Salvan1 , Nicola Sartori1
1
2

Department of Statistical Sciences, University of Padua, Italy
SDG Consulting, Milan, Italy

E-mail for correspondence: kenne@stat.unipd.it
Abstract: We propose a median modification of the score equation for largedimensional parameter estimation in fixed effects one-parameter and two-parameter
logistic models, well known as Rasch models. These models are of great importance in item response theory for the analysis of data generated by aptitude tests
or questionnaires. The number of parameters grows together with the number ofsubjects or items and complicates the estimation procedure. Similarly to logistic
regression, the model often presents cases of complete data separation. As a consequence, all or some components of the maximum likelihood estimate may be
infinite. Moreover, the maximum likelihood estimates are often not available forthe two-parameter logistic model due to numerical irregularities of the likelihood.
The proposed method estimates the whole vector of parameters and also solves
the infinite estimate problem. The performance of the approach is evaluated bysimulation studies and compared with mean bias reduction.
Keywords: Infinite estimates; Likelihood; Logistic model; Median unbiased;
Modified score

1

Introduction

The two-parameter logistic (2PL) model is a generalization of Rasch (1960) oneparameter logistic (1PL) model, originally proposed in the context of reading
ability tests. These models are of great importance in item response theory forthe analysis of data generated by aptitude tests or questionnaires. Without latentvariables assumptions, the number of parameters grows together with the number of subjects or items and complicates the estimation procedure. Additionally,
the data could present cases of complete or quasi-complete separation implying
nonexistence of maximum likelihood estimates. Many authors have studied theThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

204Median bias reduction of mle

properties of the standard estimators of these models, such as marginal, conditional and joint maximum likelihood estimators (see for instance Andersen, 1980,
Ch. 6). Although joint maximum likelihood was the preferred method of estimation due to its intrinsic robustness, it soon became clear that numerical issues
were involved in the estimation of the 2PL model (Baker, 1987, 1988). For thisreason, marginal maximum likelihood with a random effects assumption on thesubjects abilities is nowadays the standard choice.
Firth (1993) proposed an implicit method for bias reduction of the maximum
likelihood estimator by adding a modification term to the score function. The
corresponding modified estimating equation does not depend explicitly on the
maximum likelihood estimate and has been found to overcome infinite estimate
problems. An alternative modification of the score equation is proposed in Kenne
Pagui et al. (2016), aiming at median centering of the estimator. This method
respects equivariance under componentwise reparameterizations and is also effective in preventing infinite estimates. The modification is obtained by equatingeach score component to the approximate median of its profiled version. Bothbias and median bias reduction allow fixed effects estimation of the 2PL model,thus avoiding the latent variable assumption.

2

The 1PL and 2PL models

Let ysi be the answer for subject s to item i, s = 1, . . . , S, i = 1, . . . , I. Assume
that the observations ysi are realisations of independent random variables Ysi ∼Bin(1, πsi ). Here, πsi is viewed as the probability that subject s succeeds in item iof an achievement test. Under the 1PL model, log{πsi /(1−πsi )} = γs +αi , whereγs is interpreted as a measure of the ability of subject s, while αi is a measureof easiness of item i. Thus, a large value of αi corresponds to high probability ofa correct response. When πsi is viewed as a function of γs , it is called the item
characteristic curve (ICC). The constraint α1 = 0 allows identifiability of themodel so that there are S + I − 1 unknown parameters.
The 2PL model generalises the 1PL model by assuming log{πsi /(1 − πsi )} =βi γs + αi , where βi is interpreted as the discrimination parameter of item i. The
larger the discrimination parameter, the steeper the ICC curve is. Here, with theconstraints α1 = 0, β1 = 1, the model is identifiable and has S +2(I −1) unknown
parameters (San Martin et al., 2015).
The number of parameters drastically increases with the number of subjects andhence a suitable estimation method is needed. For the 1PL model, the conditional
likelihood works well in estimating the item parameters since the maximum likelihood estimator is asymptotically biased. However, in the 2PL model, maximum
likelihood estimates are often affected by numerical instabilities and no conditional likelihood is available. Therefore, the standard solution assumes γs as arandom effect, usually N (0, 1). In Section 3, we propose a method that makes noassumption on γs .

3Median bias reduction method

Consider a regular statistical model with parameter θ = (θ1 , . . . , θp ). We denoteby `(θ) the log likelihood, i(θ) Fisher information, and θ̂ the maximum likelihood

Kenne Pagui E.C. et al.

205

estimator. Let irs be a generic entry of i(θ) and irs an entry of its inverse,
r, s, . . . = 1, . . . , p. Let Ur and Urs be partial derivatives of `(θ) with respect toelements of θ with indices r and s. In particular, Ur is the rth component of
the score function. Moreover, expected values of log likelihood derivatives aredenoted as νr,st = Eθ (Ur Ust ) and νr,s,t = Eθ (Ur Us Ut ).
In the following, summation is understood over repeated indices. The medianbias reduced estimator, θ̃, is obtained as a simultaneous solution of the modified
score equationsŨr = Ars Us − κ1r + 16 κκ3r
, r = 1, . . . , p, with Ars = irs /irr ,2rwhere κjr , j = 1, 2, 3, are approximate cumulants of the profiled version of Ur ,with the following expressions κ1r = − 21 irs ν tu (νs,tu + νs,t,u )/irr , κ2r = 1/irr
and κ3r = irs irt iru νs,t,u /(irr )3 , where ν tu = itu − itr iur /irr . See Kenne Pagui
et al. (2016) for details and properties. In particular, the asymptotic distribution of θ̃ is the same as that of θ̂. For implementation in 1PL and 2PL models, we vectorise the S × I data matrix obtaining a n × 1 vector of the formy = (y11 , . . . , yS1 , y12 , . . . , yS2 , . . . , y1I , . . . , ySI )T , with a n × p design matrix X
and where n = S × I. Due to the special structure of Rasch models, summation
over three indices in Ũr can be avoided by exploiting sparsity of arrays involvedin the modification. A further significant computational gain is achieved by implementing the modification term using Rcpp rather than plain R.

4

Numerical studies

We conducted a simulation study to assess the performance of the median bias
reduced (MBR) estimator in 2PL model. We compare the new estimator withthe mean bias reduced (BR) estimator of Firth (1993) in terms of error distribution. The BR estimates are obtained through the R package brRasch available on
GitHub. Here, the simulation is under the usual assumption of a random effects
model. In particular, the results are obtained with 1,000 replications, I = 10,
S = 100 and assuming γs ∼ N (0, 1). From Figure 1, it appears that the BR and
MBR estimators of α are almost comparable in terms of estimation error, while
the MBR estimator of β is preferable. Unreported simulation results in the more
extreme settings show that the method still gives reasonable results.

References
Andersen, E. B. (1980). Discrete Statistical Models with Social Science Applications. North-Holland, Amsterdam.
Baker, F. B. (1987). Methodology review: Item parameter estimation under theone-, two-, and three-parameter logistic models. Applied Psychological Measurement, 11, 111 – 141.
Baker, F. B. (1988). The item log-likelihood surface for two-and three-parameter
item characteristic curve models. Applied Psychological Measurement, 12,387 – 395.
Firth, D. (1993). Bias reduction of maximum likelihood estimates. Biometrika,
80, 27 – 38.
Kenne Pagui, E. C., Salvan, A. and Sartori, N. (2016). Median bias reduction of
maximum likelihood estimates. http://arxiv.org/abs/1604.04768.

Median bias reduction of mle
BR alpha
●
●
●
●
●

●
●
●
●
●
●
●

●
●
●
●
●
●
●

●
●
●
●
●
●
●

●
●
●
●
●
●

●
●
●
●

●
●
●

●
●

●
●
●
●
●
●
●

●
●
●
●
●
●
●
●
●

●

●
●
●

●

−1

−0.7 −0.5

0

0.5

0.7

1●

●
1.64

2
1

●
●
●

●
●
●
●
●

●
●
●
●
●
●
●
●
●

●
●
●
●
●
●
●
●

−1.5

●
●
●
●
●
●
●
●
●
●
●
●
●
●

−1

●
●
●

●
●
●
●
●
●
●
●
●

●

●
●
●

−0.7 −0.5

●

●
●
●
●
●
●
●
●
●
●
●

0

0.5

0.7

1

●
●
●
●
●
●

●
●
●
●
●
●
●
●
●
●
●
●
●

●
●
●
●
●
●
●
●
●
●
●
●
●

●

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●
●
●
●
●
●
●
●
●
●
●
●
●

●

●

●

●
●

●

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

1.31
●
●
●
●

0.82
●
●
●

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●
●
●

●
1.48
●

1.15
●

●

0.66

●

●

●

●
●
1.8
●

0.5

●

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●

MBR● beta●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●
●

●
●
●
●
0.99

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●
●

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

0

●
●
●
●
●

●

●
●

1.5

1.64

1.31

0.82

1.48

1.15

0.66

0.99

1.8

0.5

●

●
●

●
●
●
●
●
●

●

−1

−1

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●

●
●

●
●

●

1

●
●
●
●
●
●
●
●
●

●

0

●
●

●
●
●
●

●

3

●
●
●
●

●
●
●
●
●
●
●
●
●
●

●
●

●
●

2

●
●
●
●

MBR alpha

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●
●
●

●

1.5

●

●

●
●
●

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

●

0

●
●
●
●
●
●
●

−1.5

−2

●
●
●

−1

−1
−2

●

1

●
●
●
●
●
●

●
●
●
●
●
●
●
●
●
●
●

0

●
●

●
●
●

●
●
●
●
●
●
●
●
●

3

2

●
●

●

1

●
●

BR beta

●
●
●

2

206

●

FIGURE 1. Estimation error simulated distribution for the BR and MBR estimators of (α, β) with I = 10, S = 100 and γs ∼ N (0, 1).

Rasch, G. (1960). Probabilistic Models for some Intelligence and Attainment Tests.
Studies in Mathematical Psychology I. Danish Inst. Educational Research,
Copenhagen.
San Martı́n, E., González, J., and Tuerlinckx, F. (2015). On the unidentifiabilityof the fixed-effects 3PL model. Psychometrika, 80, 450 – 467.

Modelling and categorisation of seismic waves
Duncan McGregor1 , Andrew Bell2 , Bruce J. Worton1
1

School of Mathematics and Maxwell Institute for Mathematical Sciences, The
University of Edinburgh, Edinburgh, UK
2
School of GeoSciences, The University of Edinburgh, Edinburgh, UK
E-mail for correspondence: Bruce.Worton@ed.ac.uk
Abstract: A key problem in seismology is assessing the similarity or difference
between events, and constructing categorisations based on these measures. Clustering algorithms remain an active area of research, but many approaches are well
documented. This paper assesses the suitability of a selection of these approaches
to the problem of seismic waves, with reference to a data set taken from Tunguruhua, Ecuador between 6–13 April 2015. In addition, approaches to modellingthe waves, both parametric and nonparametric are fitted and assessed, and thesuitability of certain data transformations are considered.
Keywords: Seismic waves, Clustering, Nonparametric.

1

Introduction

To better understand the processes occurring within volcanoes, seismologists
study the seismic waves generated within — these are vibrations that propagate
through the earth, and are measured by seismometers located near the volcano.
The raw seismic data is processed to identify distinct ‘seismic events’ where theseismic activity rises above some level. There are many aspects to the study ofthese events, however one key problem concerns their categorisation with theintention of identifying events that are similar and likely to have arisen froma common source. Seismologists can then use this categorisation to trace theevolution of the number of events occurring before, during, and after periods ofactivity, and make inferences about the nature of the events giving rise to activity.
This process of categorisation is currently not well-defined, with a variety of similarity measures and ad-hoc approaches to clustering in use. Plausible approaches
exist and are in wide use, but these are computationally expensive and havepractical shortcomings we will discuss. The analysis is further complicated by
the presence of ambient vibrations arising from natural and man-made sources,
and any categorisation must allow for the presence of significant noise in theevent signals.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

208

2

Modelling seismic waves

Data set and modelling

We study modelling seismic data for 4805 events between 06/04/2015 and 13/04/2015
recorded at the station next to the Tungurahua volcano. Each event data item
records the velocity of the vibration at 3001 distinct equally spaced points intime over a period of 30 seconds.
Our attempts to fit flexible parametric models to the data proved very challenging
due to computational issues. An exhaustive search of suitable models would be
impossible, but whilst the failure of our efforts cannot confirm the task is impossible. However, we have considered various possible nonparametric models (Hastie
et al, 2009). All appear to give reasonable fits to the data, but wavelets appear
to best capture the behaviour of the model (Donoho and Johnstone, 1994), andlend themselves readily to the smoothing of the wave (to remove ambient noise)and downsampling to reduce the dimension of the problem.

3

Results

Figure 1 gives an example of an event. Note the noisy nature of the data whichleads to problems with parametric fits.
The gap statistic analysis provides useful summary information concerning theclusters. For example, on Day One there is no strong evidence for more than 6groups. A typical example of a dendrogram (see Figure 2) for a day of events israther crowded (making it difficult to identify individual events). However thehigh level structure is clearly visible and it is instructive to compare the distanceof the different numbers of clusters with the Gap Statistic.

FIGURE 1. An event: example of the type of data being modelled.

McGregor et al.

209

FIGURE 2. Spectral data, Euclidean distance, Ward’s method on Day 1. Thissuggests 2 to 6 groups.

4

Conclusions

We started this work with the intention of investigating methods of categorising
seismic waves. At the end, we have arrived at the following conclusions.
1. Seismic events are complex, and not readily modelled using a simple parametric approach.
2. Smoothing of events can be performed through a variety of non-parametrictechniques, or by low-band pass.
3. Seismic events data require transformation before they can be compareddirectly. Transformations include: (a) Scaling to match amplitude of events;(b) Translation to align events; (c) Windowing to eliminate regions where
the signal to noise ratio is too low to be useful; (d) Smoothing to eliminate
high frequency noise; (e) Fourier Transform to shift the event from thetime domain to the frequency domain; (f) Principal Component Analysis
to reduce the dimension of the problem.
4. Scaling and translation are essential for clustering techniques using conventional distance measures.
5. Windowing appear to stabilise the clustering under different clustering
methods and distance measures.

210

Modelling seismic waves

6. Clustering methods give quite different groupings for smoothed signals.
Relatedly, groupings are not invariant under decimation of the waveletsmoothed signal (Donoho and Johnstone, 1994).
7. Principal Component Analysis can reduce the dimension of our data from3001 dimensions to 200 and retain over 80% of the observed variance.
8. The existing technique of cross-correlation looks sensible and fit for purpose, however we propose an alternative technique of carrying our Gap
Analysis on the Spectral Intensity Data (Tibshirani et al, 2001). The brief
simulation study we carried out suggests the technique requires refinement,
and that spectral intensity data may not be the optimal choice, and applying Gap analysis on the raw data using cross correlation as a similarity
measure may be a superior technique.

Acknowledgments: We are particularly grateful to The Instituto Geofisicoof the Escuela Politecnica Nacional (IGEPN) of Ecuador for all their hard monitoring work and providing the data.

References
Donoho, D.L. and Johnstone, I.M. (1994). Ideal spatial adaptation by wavelet
shrinkage. Biometrika, 81, 425 – 455.
Hastie, T., Tibshirani, R. and Friedman, J. (2009). The Elements of Statistical
Learning, Second Edition. New York: Springer.
Tibshirani, R., Walther, G. and Hastie, T. (2001). Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal StatisticalSociety, Series B, 63, 411 – 423.

Bayesian Modelling of the Exponential
Random Graph Models with Non-observed
Networks
Abdolreza Mohammadi1 , Ernst C. Wit2
12

Tilburg University, Netherlands
University of Groningen, Netherlands

E-mail for correspondence: a.mohammadi@uvt.nl
Abstract: Across the sciences, one of the main objectives is network modellingto discover complex relationships among many variables. The most promising statistical models that can be used for network modelling is the Exponential Random
Graph Models (ERGMs). These models provide an insightful probabilistic model
to represent a variety of structural tendencies that define complicated dependence patterns hardly modelled by other probabilistic models. However, they are
restricted to the models that regarded the network as given, observed networks
data. In the present paper, we develop a novel Bayesian statistical framework
which combines the class of ERGMs with graphical models capable of modelling
non-observed networks. Our proposed method greatly extends the scope of the
ERGMs to more applied research areas. We discuss possible extensions of themethod.
Keywords: Bayesian inference; Exponential random graph models; Graphicalmodels; Birth-death process; Markov chain Monte Carlo.

1

Introduction

Network modeling pervades all of science since one of the main objectives ofscience is to discover complex relationships among large numbers of variables.
For the prevention of epidemics, social science relies on a keen understandingof interactions among individuals. One way to describe these kinds of complex
relationships is by means of an abstract network.
Exponential random graph models are promising and flexible family of statistical models for modelling network topology. These models have been used mainlyin the social science literature since they allow to statistically account for theThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

212Bayesian Modelling of the Exponential Random Graph Models

complexity inherent in many network data (Snijders 2002). In ERGMs, the basic assumption is that the topological structure of an observed network can be
explained by the a vector of network statistics that capture the generative structures in the network (Snijders et al. 2006). However, up till now, these models
are restricted to the observed network data. In most applications, we have multivariate data which are not the observed networks, like in biology in neuroscience.
The main question is that is it possible to extend the ERGMs to the multivariate
data that are not observed network? A possible solution is combining the classof ERGMs with graphical models.
Graphical models Lauritzen(1996) provide an appealing and insightful way to obtain uncertainty estimates for inferring network structure. The close relationshipbetween the topology of the underlying graphs and their probabilistic properties is a main aspect in graphical models, and it provides the potential tools tointerpret the underling graph structure. In this regard, Bayesian approaches provide a mainly straightforward tools, and much recent progress has been made ingraphical models (Mohammadi and Wit 2015). However, graphical models are
powerful approaches only for estimating the underlying graph structure, they are
not designed for network modeling.
In this paper, we develop a new Bayesian statistical framework for ERGMs, whichis capable not only network modeling but also estimating underlining graph formultivariate data which are not observed networks. The proposed method greatly
extends the scope of the ERGMs to more applied research areas, which not limited
only in social science. In our method, to apply the ERGMs to non-observed
networks data, we combine the class of ERGMs with graphical models capableof modelling non-observed networks. In particular, in our Bayesian framework,
we design a computationally efficient search algorithm to explore all the graphspace to distinguish not only important edges but also key features and detect
the underlying graph structure. This search algorithm is based on birth-death
Markov chain Monte Carlo algorithm proposed by Mohammadi and Wit (2015)
for Gaussian graphical models.

2

Exponential families and graphical models

Exponential random graph models models are the families of statisticalmodels that provide a flexible way to model the complex dependence structureof networks. The aim to model data as observed networks consisting of nodes
and edges, which in the social network context represent actors and relationships
between these actors, respectively.
In an ERGMs, the random matrix G = {gij } is defined over the graph space on
a set of p nodes, with each variable in G representing the presence or absence ofa particular edge ( gij = 1 if there is a link from i to j, and gij = 0 otherwise).
Edges connecting a node to itself are not allowed so gii = 0. For a graph, the
ERGM is then given by
P (G|θ) =1
exp{θt S(G)},
Z(θ)

(1)

where θ ∈ Θ represents a vector of unknown parameters, and Z(θ) is a normalizing constant and S(G) term is a network statistic of interest that gives theMohammadi and Wit

213

ERGMs much of its explanatory power. The vector S(G) can contain statistics
to capture the generative structures of connectivity in the network.
Note, in ERGMs data are observed networks, which is a strong limitation. In
most of the applications, data are measured on variables, such as gene expression
data and cell signalling data. The question we intend to answer is whether it ispossible to extend the idea of ERGMs to those types of data? We extend the
ERGMs to multivariate data (that are not observed networks) by combining itwith graphical models.
Graphical models Lauritzen(1996) use a graph concept to represent conditional dependence relationships among random variables, as non-observed networks. When observed data come from noisy measurements of the variables, thengraphical models present an appealing and insightful way to describe graph-based
dependencies between the random variables. A graph G = (V, E) denotes a set ofvertices V = {1, 2, ..., p} – where each node corresponds with a random variable– and a set of existing edges E. In this class of models, nodes in the graph G
correspond to the random variables. The absence of an edge between two nodes
determines the two corresponding variables are conditionally independent given
the remaining variables. Graphical models that follow the multivariate Gaussian distribution are called Gaussian graphical models (GGMs), also known ascovariance selection models.

3

Bayesian hierarchical model for ERGMs with
non-observed networks

We can display the hierarchical model schematically as below
θ −→ G −→ K −→ X = (X1 , ..., Xn ).
Thus, we consider the joint posterior distribution of the parameters as bellow
P (θ, G, K | X) ∝ P (X | θ, G, K) P (K | G) P (G | θ) P (θ).

(2)

In our methodology, we assume the observed data follows a multivariate Gaussian
distribution.
For prior specification on graph, by consider the idea of exponential randomgraph, we use a prior on the graph as follows
P (G | θ) =1
exp{θt S(G)}
Z(θ)

(3)

where S(G) is a vector of statistics of the graph (e.g., the number of edges,
triangles, etc.) and θ ∈ Θ denotes the parameter vector of the model. For theprior distribution of the precision matrix, we use the G-Wishart distribution.

3.1

MCMC sampling scheme

The MCMC algorithm is in three steps as follows
Step 1: Sample from θ, based on exchange algorithm (Murray et al. 2012).
Step 2: Sample from graph space, based on birth-death MCMC sampling algorithm proposed by Mohammadi and Wit (2015).

214

Bayesian Modelling of the Exponential Random Graph Models

Step 3: Sample from precision matrix, based on exact sampling algorithm form
G-Wishart distribution.
For step 1, We sample from conditional distribution of θ based on exchange
algorithm (Murray et al. 2012). For step 2, we using computationally efficientbirth-death MCMC sampler proposed by Mohammadi and Wit (2015) for Gaussian graphical models. Their algorithm explores the graph space by adding ordeleting an edge in a birth or death event, in which the events are based on acontinuous time birth-death Markov process.

References
Lauritzen, S.L. (1996). Graphical models. Oxford University Press.
Mohammadi, A. and Wit, E. (2015). Bayesian Structure Learning in Sparse Gaussian Graphical Models. Bayesian Analysis, 10(1), 109 – 138.
Mohammadi, A. and Wit, E. (2016). BDgraph: An R Package for Bayesian Structure Learning in Graphical Models. Journal of Statistical Software.
Murray, I. and Ghahramani, Z., David (2012). MCMC for doubly-intractable distributions. arXiv preprint arXiv:1206.6848.
Snijders, T. (2002). Markov chain Monte Carlo estimation of exponential randomgraph models. Journal of Social Structure, 3(2), 1 – 40.
Snijders, T. and Pattison, P. and Robins, G. and Handcock, M. (2006). New specifications for exponential random graph models. Sociological methodology,
36(1), 99 – 153.

Analysis of differential effects in multi-group
regression methods
Adrian Quintero1 , Emmanuel Lesaffre1 , Geert Verbeke1 ,Benedict Orindi1 , Luk Bruyneel1
1

KU Leuven, I-BioStat, Leuven, Belgium

E-mail for correspondence: luisadrian.quinterosarmiento@kuleuven.be
Abstract: In regression analysis, the data sample is often composed by subpopulations and the impact of some of the covariates may differ across groups.
A model with group interactions can be fit to avoid biased estimates, but thepower to test for significant effects is importantly reduced, especially when thenumber of sub-populations is large. We propose a prior distribution which combines the information of the groups with a similar covariate effect. This increases
importantly the power whilst allowing to study differential effects across subpopulations. The method is applied to analyse patients’ satisfaction in seven
European countries.
Keywords: Differential effects; Multiple groups; MCMC methods.

1

Introduction

In regression analysis, the response is usually modelled as a function of the explanatory variables assuming that the effect of each covariate is the same for allobservations. Such a model can be expressed asyi = β0 x0,i + β1 x1,i + · · · + βp xp,i + εi ,(1)where the covariate x0 corresponds to the intercept and the error is assumed tofollow εi ∼ N (0, σ 2 ) for i = 1, . . . , n. However, data often come from populations
with well-defined groups such as geographical region, level of education, ethnicity
or gender. Ignoring group effects may have serious consequences on the estimation of regression coefficients. As a simple example, let us consider the case inwhich a specific covariate has a strong negative effect on the response for womenand a strong positive effect for men. Fitting a model with a common regression
coefficient for both groups would lead to a biased estimate and perhaps inferring non-significance of the covariate effect. Therefore, the following model withThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

216

Differential effects in multi-group regression analysis

interaction terms may be more appropriate in multi-group studies
yi(g) = β0(g) x0,i(g) + β1(g) x1,i(g) + · · · + βp(g) xp,i(g) + εi(g) ,(2)where yi(g) is the response for individual i = 1, . . . , ng in group g = 1, . . . , G. The
regressor xj,i(g) corresponds to the interaction between covariate xj,i and groupmembership and εi(g) ∼ N (0, σ 2 ). Hence, the effect of covariate xj in group g is
βj(g) for j = 0, . . . , p.
However, model (2) embraces low power to detect significant effects when thenumber of groups is large. To illustrate this, let us assume that the varianceσ 2 is known and that all covariates in model (1) are mutually independent,i.e., Cov(xj , xj 0 ) = 0 for j 6= j 0 . If the covariates are mean centered and standardized, the variance of the regression coefficient estimators under model (1) is
Var(β̂j ) = σ 2 for j = 1, . . . , p. On the other hand, with the interaction model (2)2the variance of the estimators is substantially larger, namelyP) =2 GσP 2 Var(β̂j(g)for j = 1, . . . , p. This equality holds assuming thati xj,i(g 0 ) ,i xj,i(g) =Pj,i(g) = 0 for all groups and that the interaction cross-products are null,i xPi.e., i xj,i(g) xj 0 ,i(g) = 0 for j 6= j 0 . Therefore, including the interaction terms
is unfavourable when a covariate has the same effect across groups, because themodel without interactions describes adequately the data and presents higher
power for significance testing.
We propose an approach that is a compromise between the two alternatives, allowing for a differential effect across groups but taking into account that thecovariate effect may be similar for some or all of the sub-populations. Using a
Bayesian hierarchical model, we assume that for a given covariate xj , the interaction coefficients across groups βj(1) , . . . , βj(g) can be classified into 3 components:
sub-populations where the effect of the covariate is negative, groups where there
is no effect on the response and sub-populations where the impact is positive.
If no differential effects are present for the covariate, all the interaction terms
can be classified in the same component. The proposed method combines theinformation of the groups with a similar covariate effect, which improves thepower in significance testing, whilst allowing to study differential effects across
sub-populations.

2

Proposed Bayesian hierarchical model

A latent variable γj(g) = {−1, 0, +1} is introduced to classify the interaction
coefficients of the model in the three defined components: the covariate xj has
no impact in group g when γj(g) = 0, the effect is negative if γj(g) = −1 and theeffect is positive when γj(g) = +1. The distribution of the interaction coefficients
conditionally on the latent variable is defined as2βj(g) |γj(g) = −1 ∼ N µj(−) , σj(−)
,
(3)



2
βj(g) |γj(g) = +1 ∼ N µj(+) , σj(+) ,where µj(−) < 0 and µj(+) > 0 for j = 0, . . . , p. If γj(g) = 0 we have a degenerate
distribution and P r(βj(g) = 0) = 1. For each covariate, the proportion of subgroups where xj has no impact is πj(0) = P r(γj(g) = 0), whereas the overall proportions with a negative and positive effect are respectively πj(−) = P r(γj(g) =Quintero et al.

217−1) and πj(+) = P r(γj(g) = +1). The model specification is completed by assuming independence for γj(g) ’s across covariates and sub-populations conditionallyon πj = (πj(−) , πj(0) , πj(+) ).
This model for the analysis of differential effects can be estimated via MCMC
methods. We select as prior distribution for the component probabilities πj ∼Dir(0.25, 0.5, 0.25) to have P r(βj(g) = 0) = P r(βj(g) 6= 0). A vague inverse
gamma density IG(0.001, 0.001) is assigned to the variance of the error. In themethod, the response variable y is mean centered and standardized to facilitate
the choice of priors for the parameters in the mixture model (3). We select a prioriµj(−) ∼ U (−2, 0) and µj(+) ∼ U (0, 2) for j = 0, . . . , p, since the case βj = ±1
would imply a perfect correlation between the covariate xj and the response.
Finally, for the variances of the mixture components we specify σj(−) , σj(+) ∼U (0, 1).
After running the MCMC algorithm, a sample of the indicator component γj(g) is
obtained for each interaction in the model. If the proportion of MCMC iterations
in which γj(g) = −1 is larger than 0.5, this indicates that the effect of xj in groupg is negative. Similarly, if the proportion in which γj(g) = +1 is larger than 0.5,
the effect is positive. In any other case, we conclude that there is no impact of
the covariate in that group.

3

Simulation study

To assess the proposed method, a simulation study was carried out based on themodel yi(g) = β0(g) x0,i(g) + β1(g) x1,i(g) + β2(g) x2,i(g) + β3(g) x3,i(g) + εi(g) withng = 20 observations for each sub-population g = 1, . . . , 30. The intercept takes
the values β0(g) = {−3, −2.8, −2.6, . . . , 2.8} across groups, whereas β1(g) = 0
and β3(g) = 3 for all sub-populations. The covariate effect of x2 is β2(g) = 0 for
g = 1, . . . , 15 and β2(g) = 3 for g = 16, . . . , 30.
The proposed model (3) is compared with the method advocated by Benjaminiand Hochberg (1995) for multiple testing. The so called BH approach controls
the false discovery rate (FDR) which is defined as the proportion of hypothesis
H0 : βj(g) = 0 mistakenly rejected among all (mistakenly and accurately) rejected
hypothesis. This allows to control the familywise error rate in a weak sense andadmits more powerful procedures compared to Bonferroni-type methods, which
are well-known to be conservative.
In the simulation study, 100 replicated data sets were generated from the model.
Besides the FDR, we compared the methods based on: false non-discovery rate
(FNDR), type I error rate, the power and total number of interaction effects
that are misclassified (TotalMiss). The results in Table 1 show that BH presents
a lower average FDR but the power is 0.44, very low compared to 0.92 in the
proposed model (3). When looking at TotalMiss, we see that around 35% of the
120 interactions are being misclassified by BH whereas in our proposed method
it is only 9%.

4Analysis of patient overall satisfaction

In the registered nurse forecasting (RN4CAST) study, patients were asked torate their hospitals on a scale from 0 (worst) to 10 (best). Additionally, the subjects responded 16 items related to doctor communication, nurse communication,

218

Differential effects in multi-group regression analysis

TABLE 1. Results from the simulation study.

Method
BH
Prop.

FDR
0.017
0.055

FNDR
0.476
0.120

Type I
0.013
0.093

Power
0.446
0.917

TotalMiss
42.37010.440physical environment, pain control and discharge information. Hence, it is of interest to understand how the patient experiences (reflected in the 16 items) relate
to the overall rating given by patients to the hospitals.
Furthermore, the RN4CAST is a multi-country study, so the 2942 patients inthe sample (after discarding observations with missing information) belong toseven different countries: Belgium, Switzerland, Spain, Finland, Greece, Irelandand Poland. Therefore, it is important to determine if the impact of the 16 items
on overall satisfaction is the same across countries or, on the contrary, whether
any patient experiences are more relevant in some specific nations.
Fitting model (1) indicates that 13 of the items are related to overall patientsatisfaction. To analyse differential effects, the BH method is applied to model(2) finding that in 5 of those items, none of the interactions is significant. On theother hand, the proposed method (3) indicates that all of the 13 items revealed
by model (1) have an important effect in at least four of the seven countries.

References
Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: apractical and powerful approach to multiple testing. Journal of the Royal
Statistical Society, Ser. B, 57, 289 – 300.

Gene regulatory network reconstruction withprior knowledge over mRNA data for COPD
patients and controls
Victor Bernal 12 , Corry A. Brandsma3 , Alen Faiz5 , Victor
Guryev4 , Wim Timens3 , Maarten van den Berge5 , Rainer
Bischoff2 , Marco Grzegorczyk 1 , Peter Horvatovich2
1

Johann Bernoulli Institute (JBI), Groningen Rijksuniversiteit, Groningen, NL.
Department of Pharmacy, Analytical Biochemistry, Rijksuniversiteit Groningen, NL.
3
Universitair Medisch Centrum Groningen (UMCG), Department of Pathology
and Medical Biology, Rijksuniversiteit Groningen, NL.
4
Universitair Medisch Centrum Groningen (UMCG), ERIBA, Rijksuniversiteit
Groningen, NL.
5
Universitair Medisch Centrum Groningen (UMCG), Department of Pulmonary
Diseases, Rijksuniversiteit Groningen, NL.
2E-mail for correspondence: v.a.bernal.arzola@rug.nl
Abstract: Chronic obstructive pulmonary disease (COPD) is a type of lungdisease characterized by persistent bronchitis and emphysema. Current therapy
is restricted to alleviate lung tissue inflammation, but is not able to stabilize or
improve lung function of patients making necessary to understand the underlying
molecular mechanisms of COPD. Genome-wide gene expression of lung tissue
provides a powerful tool to elucidate molecular mechanism of COPD patients. In
particular, Bayesian Networks (BNs) have been applied to infer genetic regulatoryinteractions from microarray gene expression data. In this study we aim obtaina clearer understanding of the genes interaction in COPD patients by learning a
BN over microarray expression data. A subset of genes was selected for the study
fulfilling that i) the genes were significantly expressed in COPD stage 4 and ii)
there is reported gene-gene experimental association. The reported associations
are introduced as prior biological knowledge in the reconstruction.
Keywords: COPD; Bayesian Networks; Microarray Gene Expression; Introduction of prior knowledge.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

220

1

Gene regulatory network over COPD mRNA

Introduction

Chronic obstructive pulmonary disease (COPD) is characterized by distinct phenotypes as, emphysema, chronic bronchitis and fibrosis. The exact nature of theabnormal lung tissue repair processes in COPD lungs is unknown, hence a better understanding the underlying molecular mechanisms is thus crucial. Bayesian
Networks (BNs) have been applied to infer genetic regulatory interactions frommicroarray gene expression data. This inference problem is particularly hard because of the relatively small size of the data sets. Moreover the asessment of
the inference for COPD results is unfeasible as there is a lack of known gold
standards from the biological literature. The aim of the present study is the reconstruction of a regulatory network from a microarray gene expression data set
for COPD (stage 4) patients using BNs with prior biological knowledge comingfrom STRING (database of known and predicted protein-protein interactions).
Inference is done using with Markov Chain Monte Carlo (MCMC) sampling following the approach of Werhli et al. (2007). Two networks are obtained andcompared, one for the controls and one for the COPD data.

2

Bayesian Networks with prior knowledge

Bayesian Networks (BNs) represent probabilistic relationships between interacting agents (e.g. genes) in the form of conditional independence. For BNs themodel is a Directed Acyclic Graph (DAG), a set of nodes that are connected by(directed) edges without cycles. We denote by G the network structure, which isthe set of nodes and edges, and by q the interaction strength. Our objective isto learn the network structure G directly from the scratch microarray data D.
Nodes are associated with genes and the edges indicate interactions between thegenes. A Bayesian learning approach uses the posterior P (G|D)
P (G|D) = P (G|D)P (G)/P (D)where P (G) is called the prior over graphs, P (D|G) the marginal likelihood,
P (D) is a scaling constant and P (G|D) is called the posterior probability. The
term P (D|G) is the result of integrating the likelihood P (D, q|G) over all thepossible values of the parameters q. Calculating P (G|D) for all possible network
structure G is computationally impossible due to P (D) is intractable. Gene expression data implies a large number of genes with only a relative small numberof measurements. As a consequence the posterior P (G|D) will not have a clear
maximum G∗ . A solution is to use a MCMC sampling based on Metropolis Hasting algorithm (MH). This produces a Markov Chain that reaches P (G|D) as its
stationary distribution (under some conditions). Generally different DAGs canrepresent the same probabilistic relationships (i.e they can be equivalent). The
set of equivalent DAGs is called the equivalence class and is represented by a
CPDAG.
Introducing prior knowledge for learning BNs is done following Werhli et al.
(2007). The prior knowledge is encoded in a matrix B whose entries Bij ∈ [0, 1]
represent the belief about the presence/absence of an edge. The elements Bij
greater than/less than 0.5 reflects evidence in favor/against the presence of theedge Gij , and Bij = 0.5 as uninformative. The prior over graphs takes the form

Bernal et al.

221

of a Gibbs distribution,
P (G|β) = e−βE(G) /Z(β)
The function E(G) measures the agreement between a sampled network G and B(e.g. using the Hamming distance). The parameter β indicates the weight of therespective source of knowledge to the data, and Z(β) is a normalization constant.

3

Evaluation of the COPD dataset and
Implementation

The data consist of genome-wide gene expression profiling in lung tissue samplesof 72 subjects, where 48 exhibited COPD (stage 4). The others 24 are nonCOPD tissues (Controls). The 60 most significant genes based on p-value andlog2 (F C) > 2 were checked in STRING data base, obtaining that 7 genes (FGG,
RORC, FGA, OSMR, NR1D1, CSF3, THBS1) had reported experimental and/or
data base associations. This set of genes is augmented as follows: i) the Pearson’s
correlation matrix ρ is built for Controls and COPD, and ii) a pair of genes isselected if |ρcontrols | < 0.25 and |ρCOP D | > 0.75 . In this way 8 additional genes
are obtained, namely MT1M, IL1RL1, MT1P3, MT1A, SLCO4A, GPA33, TTN,ZBED2. In total 2 networks were reconstructed for the forementioned genes i)
Controls with prior knowledge, and ii) COPD with prior knowledge. The coupling
parameter β is defined in [0, 30]. The MCMC was implemented with the REV
move from Grzegorczyk (2008) for number of 9x105 iterations, with thining every
1000 DAGs. The first half of the sample was discarded as part of the burn in phaseand Model Averaging was performed over the final set of 501 CPDAGs.

4

Results and Conclusions

After the burn in phase the 75th percentile of β (in Controls and COPD cases)
is less than 1. This suggest that COPD’s gene regulation has no apparent commonalities with previously reported interaction. Table 1 shows a summary of theinteractions that considerably changed between Controls and COPD. In particular interactions like FGG-FGA, and MT1P3-MT1A are reasonable as they belong
to the same family. On the other hand NR1D1 has no a well known association
with fibrogenes, this interaction (FGG-NR1D1) is present in Controls (mediatedby FGA) and COPD (directly). Our next research plan is to extend the gene set.
We will provide more precise biological interpretations in the presentation.

References
Grzegorczyk, M., Husmeier, D. (2008). Improving the structure MCMC sampler
for Bayesian networks by introducing a new edge reversal move. Machine
Learning, 71, 265 – 305.
Werhli A., Husmeier, D. (2007). Reconstructing Gene Regulatory Networks with
Bayesian Networks by Combining Expression Data with Multiple Sourcesof Prior Knowledge.Stat Appl Genet Mol Biol, 6(1), 1 – 45.

222

Gene regulatory network over COPD mRNA

FIGURE 1. Reconstructed regulatory network for Control and COPD.
On the right: Network from STRING with experimental and database associations. Doble arrows stand for reported associations. On the left: Reconstructed
network. The blue edges stand for gene interactions in the Controls. The segmented red edges stand for gene interactions in the COPD. There are no commonedges. Only edges with a frequency above 0.75 are shown.

TABLE 1. Marginal edge frequency changes This table presents a set of relevant gene-gene interaction whose edge frequency had a notable change between
the network for the Controls and the one for COPD.

FGG-FGA
FGA-NR1D1
MT1P3-MT1A
THBS1-MT1A
RORC-NR1D1

Controls

COPD

0.770
0.906
0.890
0.932
0.760

0.314
0.078
0.614
0.502
0.114

Resolving the Lord’s Paradox
Priyantha Wijayatunga1
1

Department of Statistics, Umeå University, Umeå SE-90187, Sweden

E-mail for correspondence: priyantha.wijayatunga@umu.se
Abstract: An explanation to Lord’s paradox using ordinary least square regression models is given. It is not a paradox at all, if the regression parameters areinterpreted as predictive or as causal with stricter conditions and be aware oflaws of averages. We use derivation of a super-model from a given sub-model,
when its residuals can be modelled with other potential predictors as a solution.
Keywords: Effect; Predictive; Causal; Confounding.

1

Introduction

In 1967 Frederic Lord posed following question (see Lord 1967 and Pearl 2016)that became a paradox among applied statistical community. To see effects andif there is any sex difference of diet provided in a university weights of students
at time of their arrival and those a year later are recorded. The data are independently examined by two statisticians. The first examines the mean weight of
the girls at the beginning and at the end of the year, and finds that they are tobe identical, i.e., frequency distribution of the weight for the girls is not changed,
so is for the boys. The second statistician finds that the slope of the regression
line of the final weight on the initial weight is essentially the same for both sexes
but the regression coefficient of the variable sex to be statistically significant andconcludes that the boys showed significantly more gain in weight than the girls
when proper allowance is made for differences for initial weight.
Conclusions of the two statisticians seem to contradict with each other; the firstis predictive and the second is both predictive, and causal if the initial weightis the only confounder of causal relation between the sex and the final weight.
The second has given causal effect of the sex on the final weight (weight gain)
by a regression coefficient. In fact, to give it by comparing, two supports of theconfounder of both sexes should coincide. But one can assume that the population
initial weight ranges of boys and girls coincide even though sample counterparts
differ (so, extrapolation is meaningful).
Let the initial weight, final weight and sex are denoted by WI , WF , S respectively(S = 0, a girl and S = 1, a boy) and weight gain be D = WF − WI . If the effect ofThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

224

Lord’s Paradox

S on D is found by difference of conditional means, E{D|S = 1} − E{D|S = 0}then it is no effect. This can be found by running regression of D on S. Note that
E{WF |S = 1} = E{WI |S = 1}, (say, µB ) and E{WF |S = 0} = E{WI |S = 0},(say, µG ). If E{D|WI = i, S = 1} and E{D|WI = i, S = 0} are calculated simply
by partitioning the data by taking WI to be discrete or as a functions of i, thenthe difference E{D|WI = i, S = 1} − E{D|WI = i, S =P0} may not be zerofor each i, so may
P be difference of their weighted means, i E{D|WI = i, S =1}p(WI = i) − i E{D|WI = i, S = 0}p(WI = i). If the effect of S on WF iscalculated by it then it is different from former value (paradoxical!).
Now let us see why two types of differences of averages differ by simple algebra,
that will say that they should have two different interpretations. First assume that
we have a number of subgroups of boys and, for simplicity, the same is true for1girls. Let Dij
be the weight gain of the j-th boy in the i-th subgroup of boys where0
sub-group size is ni and DP
girls where sub-group size isPmi and
ij be that of the P
furthermore, let fi1 = ni / k nk , fi0 = mi / k mk and fi = (ni + mi )/ k (nk +mk ) for j = 1, ..., ni and i = 1, ..., a. And P
let A1 be difference Pof the average
0
1/mi for/ni and D̄i0 = j Dij
weight gain of the boys and the girls, D̄i1 = j Dij
i = 1, ..., a. So,
Pa PniPa Pmi 0
1
a
a
X
X
Dij
i=1
j=1 Dij
i=1
Pa
Pa j=1A1 =−=D̄i1 fi1 −D̄i0 fi0
i=1 ni
i=1 mi
i=1
i=1
6=

a
a
a
a
1 n X 1 1 X 1 0 X 0 0 X 0 1oD̄i fi −D̄i fi −D̄i fi ; generallyD̄i fi +
2 i=1
i=1
i=1
i=1

=

a
a
o X
1nX 1
(D̄i1 − D̄i0 )fi = A2
(D̄i − D̄i0 )(fi1 + fi0 ) 6=
2 i=1
i=1

P
P
where fi = αfi1 + (1 − α)fi0 for i = 1, ..., a such that α = ai=1 ni / ai=1 (ni +mi ) and A2 is the difference of weighted averages of the sub-group weight gain
averages. So, the difference of group averages A1 (which is zero in our case) isdifferent from the difference of pooled-weighted average of the sub-group averages
A2 . The second statistician compares the boys and the girls subgroup-wise andfinds that it is a constant gain for the boys over the girls across the subgroups, i.e.,D̄i1 − D̄i0 is constant for all initial weight i. Therefore he finds that the boys gain
more weight than the girls in corresponding sub-groups. Note that P
for simplicity
we have taken initial P
weights as discrete values. In fact, A2 =i E{D|S =1, WI = i}p(WI = i) − i E{D|S = 0, WI = i}p(WI = i) is the causal effect of S
on D if WI is the only confounder, under the linear assumption. It is different from
A1 unless E{D|S, WI } = E{D|S}. The confounding effect (A2 − A1 ) depends on
how different f 1 and f 0 are (can have a measure from them).

2

Regression Solution

Now we define interpretation of ordinary least square (OLS) estimates of the regression coefficients (parameters). The OLS estimation is based on the variationof the response variable Y for a given functional form of the values of explanatory
factors. Regression coefficients are estimated so that sum of squared prediction
errors for the data in the sample is the minimum. So, reverse regression is not
generally obtainable from forward regression and may not be consistent with theWijayatunga et al.

225

latter. For simple linear regression one can easily establish that the reverse regression and the forward regression are consistent with each other if and only if oneof the regressions have symmetric residuals about and uni-modal at conditional
expectation of response, that implies other regression too.
Now consider the OLS linear regression model Y = β0 + β1 X1 + β2 X2 + , thenlinear effect of X1 on Y when X2 is held unchanged is given by β1 if Y values
are symmetric about and uni-modal at β0 + β1 X1 + β2 X2 . It is clear that the
supports of X2 for each value of X1 are the same (or extrapolation is meaningful
if empirical supports differ). Symmetry and uni-modality of Y values for given
values of X1 and X2 are observed if all other factors that affect or are associated,but are not taken into consideration are allowed to vary pure randomly. This isa fundamental assumption used in statistical modelling often implicitly.
Let us do a regression of WF on the binary variable S. Then we get the model
WF = µG +(µB −µG )S +1 . where the regression co-efficient of S is the predictive
effect of S on WF provided that above requirement is fulfilled. The residuals of
the model are just individual values of D, i.e., 1 = D for each subject and it iseasy to see in Fig. 1 of Lord 1967, that the residuals are predictive by WI for eachsex category separately, 1 6⊥ WI |S. However, it may be that 1 ⊥ S. So, if thetwo clusters of values of WF for two sexes are symmetric about and uni-modal
at the respective means then the effect of S on WF is the regression coefficient of
S in the model. But it is uncontrolled confounders that are associated with WF ,then it should be interpreted accordingly. That is, it is the predictive effect of sex
differences and causal if there are no confounders such as WI . And we see that
we get zero predictive effect from the meal change since the regression coefficientis the same as that when the girls and boys had previous meal type.
Let we can write theR distribution of residuals for each value s of S, say, f (1 |s) as a
mixture, f (1 |s) = g(1 |x, s)π(x, s)dx for some random variable X, and for eachvalue x of X theRcomponent distribution g(1 |x, s) may have non-zero mean suchthat E{1 |s} = E{1 |x, s}π(x, s)dx = 0 and then we have that V ar{1 |x, s} ≤V ar{1 |s} where π(x, s) = h(x|s)p(s); here h(x|s) is the conditional probability
density of X given S = s and p(s) is the marginal probability distribution of
S. If X could be identified meaningfully, then model should include such feature
variables too. In this case, X could be identified as the initial weight WI . Then
one should accept the upgraded model that includes WI too. It has residuals thathave a smaller conditional standard deviation given WI and S. Furthermore, if
WI is the only confounding factor and when it is also included in the model thethe coefficient of S is the causal effect of S on WF .
Let the residual 01 corresponds to the context that WI = wI and S = s and
I ,s
I ,s
is the expectation of it. So,
then it can be written as 01 = µw
+ 2 where µw
1
1
we have E{2 |WI = wI , S = s} = 0 and also that V ar{2 |WI = wI , S = s} ≤I ,0V ar{1 |S = s}. And furthermore, we can have that µw= a0 + b0 wI for s = 0
1
wI ,1and µ1 = a1 + b0 wI for s = 1 where a0 , b0 and a1 are constants. Now, giventhat WI = wI and S = s, for s = 0, 1, and I(A) = 1 when A is a true statement

226

Lord’s Paradox

and I(A) = 0 otherwise, we have
WF

=

I ,s
+ 2
WF = µG + (µB − µG )s + µw
1=µG + (µB − µG )s + (a0 + b0 wI )I(S = 0) + (a1 + b0 wI )I(S = 1) + 2=

µG + (µB − µG )s + a0 I(S = 0) + a1 I(S = 1) + b0 wI + 2=

µG + (µB − µG )s + a0 (1 − s) + a1 s + b0 wI + 2=

µG + a0 + (µB − µG − a0 + a1 )s + b0 wI + 2

So we can obtain a super-model (regression) from a given regression model (it isa sub-model of the former) as long as its residuals are predictive (linearly in thiscase) with another explanatory variable. The predictive effect of S on WF whencontrolled for WI is µB − µG − a0 + a1 that is generally different from earlier
value of µB − µG and for each individual model prediction is more accurate thanthat of the previous model, therefore new model is preferred to the previous
one. If WI is only a confounder but not an intermediate variable between thecausal pathway between S and WF , and has a common support for all valuesof S, then β1 is the average causal effect of S on WF in the linear case. In our
example, sample supports of WI for S = 1 and S = 0 differ but we can assume
that they are the same in the population (so, extrapolation is meaningful). Notethat the above arguments can be generalised. For restrictions of space, we avoid
presenting solution to the paradox, that is based on causal diagrams. We object
recent solution by Pearl. Our explanations comply with Lord’s initial comments.

References
Lord, F. M. (1967). A Paradox in the Interpretation of Group Comparisons. Psychological Bulletin, 68(5), 304 – 305.
Pearl, J. (2016). Lord’s Paradox Revisted - (Oh Lord Kumbaya!). Journal of
Causal Inference, 4(2). DOI: 10.1515/jci-2016-0021.

On a mixture regression model for
left-censored dataMário F. Desousa1 , Helton Saulo2 , Manoel Santos-Neto3 ,Vı́ctor Leiva4
1

Department of Statistics, Universidade Estadual de Campinas, Brazil
Department of Statistics, Universidade de Brası́lia, Brazil
3
Department of Statistics, Universidade Federal de Campina Grande, Brazil
4
School of Industrial Engineering, Pontificia Universidad Católica de Valparaı́so,Chile
2

E-mail for correspondence: heltonsaulo@gmail.com
Abstract: Most of the models designed for modeling censored data rely on theassumption of normality for the error distribution. It is well known that not all
applications are well modeled by this distribution. Some efforts have relaxed thenormality assumption by considering more flexible distributions such as t and logalpha-power. Nevertheless, these models do not consider partial observations fromthe assumed distribution which potentially leads to biased inference. We have explored a real data example of measles vaccine in Haiti and confirmed both thepossibility of partial observation and asymmetry problems. Then, to solve suchproblems, we propose a mixture model consisting of the Birnbaum-Saunders andBernoulli distributions. We discuss estimation of the model parameters basedon the maximum likelihood method. We then carry out a Monte Carlo simulation study to evaluate the performance of the maximum likelihood estimators.
We use the R software in all computations and the results favor the proposed
methodology.
Keywords: Mixture model; Censoring; Birnbaum-Saunders Distribution.

1

Introduction

The determination of antibody concentration by quantitative assays is a very
important topic of research, because there is always a concentration value (T )below which an exact measurement cannot be obtained regardless of the employedtechnique. Nevertheless, this antibody concentration value (T ) is a function of
the associated assay. When left-censoring is present in data from an assay, theThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

228

Mixture regression model

lower detection limit (LDL) can be used to substitute a value for the censoredobservation, namely, the value T . In special, this substitution is applied in a safety
and immunogenecity study of measles vaccine in Haiti presented by Moulton andHalsey (1995), an example explored in this paper.
A two-part model for the situation of zero excess was considered by Cragg (1971).
The Cragg model considers the possibility of having observations from the assumed distribution f and from the point mass distribution. In this model, thelog-normal distribution was considered for the independent variable. The Cragg
model, however, does not consider the existence of both a lower limit and someobservations below this limit. Moulton and Halsey (1995) proposed a straightforward generalization of the two-part model, called Bernoulli/lognormal model,
by considering the possibility of some limiting responses resulting from interval
censoring associated with f . The generalized two-part model allows the possibility of a observation i, if located below T to be either a partial observation fromf or a realization of the point mass distribution.
The main objective of this paper is to propose a regression model for censoreddata based on the mixture between the Birnbaum-Saunders (BS) and Bernoulli
distributions, that is, a censored continuous distribution and a point mass distribution located below the detection limit. The Birnbaum-Saunders (BS) distribution is positively skewed and has a failure rate with upside-down bathtub shapeand a close relation with the normal distribution; see Birnbaum and Saunders(1969) and Johnson et al. (1995). The proposed model extends to the BS case
the Moulton and Halsey (1995)’s Bernoulli/lognormal model.

2

The Bernoulli/BS mixture model

2.1

Formulation

We propose a mixture model between the Bernoulli and BS distributions (Bernoulli/BS),that is,







c1
y − µ12y − µ1c 
2g(yi ) = p + (1 − p) Φ ζi2 Ii +(1 − p)coshexp − 2 sinh(1−Ii ),
α
2
α
2
(1)

√

>
c
where c1 = 1/ 2π, µ1 = x>(1) β(1) , ζi2 = (2/α) sinh (y0 − x(1) β(1) )/2 ,(1, if y ≤ y0 ,Ii =0, if y > y0 ,(2)and Φ(·) is the CDF of the standard normal distribution, with x(1) being thecovariates associated with β(1) . We assume a logit link for the random variable
D, thus it is possible to include covariates as follows
logit [P (D = 1|x(2) )] = x(2) > β(2) ,(3)where x(2) are the covariates associated with β(2) . The formulation of the logit
link defined in Equation (3) becomes,τ i = 1 − pi =exp (x(2) > β(2) ),1 + exp (x(2) > β(2) )(4)

Desousa et al.

229

where the vector x(2) has a dimension q, which can be, and usually is, different
from the dimension of the vector x(1) .
The log-likelihood function for the mixture Bernoulli/BS is given by`(θ) = −(n − m) log(2) − (n − m) log(2π)
+
2


>
c
I
log
1
+
exp(x
β
)
[Φ
(ζ
)
−
1]
−
log
1
+
exp
x>
(2) (2)(2) β(2)
i2
i=1 i
 >

Pn
2
+ m+1 (1 − Ii ) x(2) β(2) + log (ζi1 ) − 12 ζi2− log 1 + exp(x>,(2) β(2) )

Pm

(5)



c
where ζi2
is defined in Equation (1) and ζi1 and ζi2 are defined in Equation (6),!
!
yi − x>y i − x>22(1)i β(1)(1)i β(1)ζi1 = cosh
, ζi2 = sinh
.
(6)
α
2
α
2

3

Application

We analyse a data set provided by Moulton and Halsey (1995) from a study ofmeasles vaccines. Neutralization antibody levels were collected from 330 children
at 12 months of age. The LDL was 0.1 international units (IU) or −2.306 inlogarithm scale. Around 86 (26.1%) of the observations fell below the LDL andthen recorded as 0.1. The following covariates were considered: X1 indicates thetype of vaccine used (0 if Schwartz and 1 if Edmonston-Zagreb); X2 is the levelof the dosage (0 if medium and 1 if high); and X3 is the gender where 0 is maleand 1 is female.
We here present the estimation results for the Bernoulli/BS model along withthose of the standard tobit, tobit-BS (Chapter 2) and Bernoulli/LPN Martı́nezFlórez et al. (2013) models. The Bernoulli/BS and Bernoulli/LPN have botha logit link. The covariates EZ and HI were used only in the logit component,
and covariate FEM entered only in the continuous component of the models.
Table 1 shows the ML estimates, Akaike information criterion (AIC) values andstandard errors for the considered models. A glance at the results indicates that,in the Bernoulli/BS model, the receiver of Edmonston-Zagreb strain does notcontribute to the odds ratio of being above the detection limit, however, thereceiver of a high dose impacts exp(1.499) = 4.472 in the odds of being above
the detection limit. Moreover, the Bernoulli/BS model suggests that girls haveexp(−0.078) − 1 = −0.075 less concentration of measles antibody concentration
than boys. We observe that the Bernoulli/LPN and Bernoulli/BS models do notagree on the sign of the coefficient corresponding to the FEM variable, while
the first model indicates that girls have a higher measles antibody concentration
than boys, the other indicates the opposite. Also from Table 1, we note that the
Bernoulli/BS model provides a better fit compared to the other models based on
the AIC values.

Acknowledgments: This study was partially supported by CAPES from theBrazilian government.

References
Birnbaum, Z.W., and Saunders, S.C. (1969). A new family of life distributions.
Journal of Applied Probability, 6, 319 – 327.

230

Mixture regression model

TABLE 1. ML estimates (with SE in parentheses) and AIC values for the indicated models with the measles vaccine data
Continuous component
Model
tobit

AIC
1299.27α0.945***(0.047)INT
0.597**(0.288)EZ
0.225(0.297)HI−0.228(0.295)FEM
0.271(0.296)

tobit-BS1168.60

1.545***(0.048)−0.910***(0.105)0.188*(0.111)0.074(0.109)0.121(0.110)

Bernoulli/LPN

976.488.918**(3.922)−2.869***(0.582)0.222*(0.134)

Bernoulli/BS

760.641.560***(0.108)0.123***(<0.001)−0.078***(<0.001)

Obs: Rejects H0 at *10% of significance,** 5% of significance and ***1% of significance.

Cragg, J.G. (1971). Some statistical models for limited dependent variables withapplication to the demand for durable goods. Econometrica, 39, 829 – 844.
Johnson, N., Kotz, S., and Balakrishnan, N. (1995). Continuous Univariate Distributions. Volume 2. New York: Wiley.
Martı́nez-Flórez, G., Bolfarine, H., and Gómez, H.W. (2013). Asymmetric regression models with limited responses with an application to antibody response to vaccine. Biometrical Journal, 55, 156 – 172.
Moulton, L.H., and Halsey, N.A. (1995). A mixture model with detection limits
for regression analyses of antibody response to vaccine. Biometrics, 51,
1570 – 1578.

On asymmetry models and decompositions ofsymmetry for square contingency tables withordered categories
Kouji Tahata1
1

Tokyo University of Science, Japan

E-mail for correspondence: kouji tahata@is.noda.tus.ac.jp
Abstract: To analyze square contingency tables with ordered categories, herein amodel that indicates the structure of asymmetry for cell probabilities is proposed.
This model is the closest to the symmetry model in terms of the f -divergence
under certain conditions and incorporates various types of asymmetry. It is shownthat the symmetry model can be separated into the proposed model and othermodels. This may reveal the origin for the poor fit of the symmetry model when
it occurs for a real dataset.
Keywords: f -divergence, moment equality, quasi-symmetry.

1

Introduction

To analyze square contingency tables with ordered categories, an issue of symmetry (rather than independence) arises naturally. So, we are interested in considering a structure of symmetry. Many other statisticians, including Kateri and Papaioannou (1997), Kateri and Agresti (2007), and Tahata and Tomizawa (2011),
have proposed various models of symmetry and asymmetry. In the present paper,
we propose a model that is the closest to the symmetry model in terms of the
f -divergence (Csiszár and Shields, 2004) under certain conditions. Our model isa generalization of the symmetry model in the sense that it includes the various
types of asymmetry models.
Caussinus (1965) proved the theorem that the symmetry model holds if andonly if both the quasi symmetry model and the marginal homogeneity modelhold. When the symmetry model does not fit for a real dataset, the separationof symmetry may be useful to identify the origin for the poor fit of symmetry.
For example, Kateri and Papaioannou (1997), Tahata and Tomizawa (2011), andSaigusa, Tahata and Tomizawa (2015) have given the separation of the symmetry
model. In the present paper, we show that the symmetry model can be separated
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

232

On asymmetry models and decompositions of symmetry

into some models. One is the proposed model and others are concerned with theequality of marginal moments. The results include the separation of the symmetry
model given by Caussinus (1965) as a special case.
The present paper is organized as follows. Section 2 proposes the new modeland gives the separation of the symmetry model. Section 3 provides a numericalexample.

2

Asymmetry models

Consider an r×r square contingency table with ordered categories. Let πij denotethe probability that an observation will fall in the (i, j)th cell of the contingency
table (i = 1, . . . , r; j = 1, . . . , r). Let π = (πij ) and q = (qij ) be two bivariate
probability distributions. The f -divergence between π and q is defined as


XX
πij
I C (π : q) =
qij f
,
qij
i
j
where f is a convex function on (0, +∞) with f (1) = 0. Also, we take f (0) =limt→0 f (t), 0·f (0/0) = 0, and 0·f (a/0) = a limt→∞ [f (t)/t] (Csiszár and Shields,2004).
Let {ui } be a set of known scores u1 ≤ u2 ≤ · · · ≤ ur (with u1 < ur ). In
an analogous manner to Kateri and Papaioannou (1997), for a given k (k =1, . . . , r − 1) we propose that the asymmetry based on the f -divergence (ASk [f ])model is defined as
!
k
X
S −1
h
πij = πij F(i = 1, . . . , r; j = 1, . . . , r),ui αh + γij
h=1S
where γij = γji , πij= (πij + πji )/2, f is a twice-differential and strictly convex
S
function, and F (t) = f 0 (t). From the relation πij= (πij + πji )/2, the parametersof the ASk [f ] model must satisfy
!
!
k
k
X
X
uhj αh + γji = 2.
uhi αh + γij + F −1
F −1
h=1

h=1

Note that when α1 = · · · = αk = 0, (i) the ASk [f ] model is reduced to thesymmetry (S) model (Bowker, 1948), (ii) the AS1 [f ] model is reduced to the
OQS[f ] model (Kateri and Agresti, 2007) and (iii) the ASr−1 [f ] model with u1 <u2 < · · · < ur is reduced to the QS[f ] model (Kateri and Papaioannou, 1997).
The ASk [f ] model is the closest
in terms of the f -divergence
P Pmodel to thePS modelP
under the condition that i j uhi πij (or i j uhj πij ) for h = 1, . . . , k as wellas the sums πij + πji for i = 1, . . . , r; j = 1, . . . , r are given.
When we set f (t) = t log(t), t > 0, the ASk [f ] model can be expressed ask

Y uhi −uhj
πij
=
βh
,
πji
h=1

where βh = exp[αh ]. Above equation with {ui = i} is the k-th linear asymmetry
model proposed by Tahata and Tomizawa (2011).

Tahata233

Let X and Y denote the row and column variables, respectively. Assume that a
set of known scores {us } can be assigned to both the rows and the columns. Let
X1 = ui when X = i (i = 1, . . . , r), and X2 = uj when Y = j (j = 1, . . . , r). For
a given positive integer k (k = 1, . . . , r − 1), consider a model defined byE(X1h ) = E(X2h ) (h = 1, . . . , k),PPP
where E(X1h ) = s=1 rt=1 uhs πst , E(X2h ) = rs=1 rt=1 uht πst . For a given k,
we shall refer to this model as the marginal kth moment equality model for thescores {us } (denoted by MEk ).
This leads to the following theorem.
Pr

Theorem 1. For a fixed k (k = 1, . . . , r − 1), the S model holds if and only if
both the ASk [f ] and MEk models hold.
Assume that a set of known scores {ui = i} is assigned. We now obtain that thetest statistic for the S model is asymptotically equivalent to the sum of those for
the ASk [f ] model and the MEk model under the S model.

3

ExampleTable 1, which is taken directly from Stuart (1955), is the cross-classificationof the unaided distance vision of 7477 women aged 30–39 employed in Royal
Ordnance factories in Britain from 1943 to 1946. The S model fits the data poorly,
yielding the likelihood ratio statistic G2 = 19.249 with 6 degrees of freedom (df).
TABLE 1. Unaided distance vision of 7477 women aged 30–39 employed in Royal
Ordnance factories in Britain from 1943 to 1946; from Stuart (1955).

Right eye
grade

Highest (1)Highest (1)Second (2)Third (3)Lowest (4)
Total

1520
234
117
36
1907

Left eye grade
Second (2) Third (3)
266
1512
362
82
2222

124
432
1772
179
2507

Lowest (4)

Total

66
78
205
492
841

1976
2256
2456
789
7477

We set f (t) = (1 − t)2 and used the integer scores {ui = i}. Consider the hypothesis that the AS2 [f ] model holds under the assumption that the AS3 [f ]model holds; namely, the hypothesis that α3 = 0. According to the test basedon the difference between the G2 values for the AS2 [f ] and AS3 [f ] models, thehypothesis is accepted at the 0.05 level because 7.267 − 7.262 = 0.005 with 1
df. In a similar manner, the hypothesis that the AS1 [f ] model holds under theassumption that the AS2 [f ] model holds is accepted at the 0.05 level because
7.271 − 7.267 = 0.004 with 1 df. Therefore, the AS1 [f ] model may be preferable
to the AS2 [f ] and AS3 [f ] models. The AS1 [f ] model fits the data well. On theother hand, the ME1 model fits the data poorly, yielding G2 = 11.978 with 1 df.
From Theorem 1 with k = 1, the poor fit of the S model is due to the lack ofstructure of the ME1 model (rather than the AS1 [f ] model).

234

On asymmetry models and decompositions of symmetry

References
Bowker, A. H. (1948). A test for symmetry in contingency tables. Journal of theAmerican Statistical Association, 43, 572 – 574.
Caussinus, H. (1965). Contribution à l’analyse statistique des tableaux de corrélation.
Annales de la Faculté des Sciences de l’Université de Toulouse, Série 4, 29,
77 – 182.
Csiszár, I. and Shields, P. C. (2004). Information Theory and Statistics: A Tutorial. Now Publishers, Hanover, Massachusetts.
Kateri, M. and Agresti, A. (2007). A class of ordinal quasi-symmetry models forsquare contingency tables. Statistics and Probability Letters, 77, 598 – 603.
Kateri, M. and Papaioannou, T. (1997). Asymmetry models for contingency tables. Journal of the American Statistical Association, 92, 1124 – 1131.
Saigusa, Y., Tahata, K. and Tomizawa, S. (2015). Orthogonal decomposition ofsymmetry model using the ordinal quasi-symmetry model based on f divergence for square contingency tables. Statistics and Probability Letters,
101, 33 – 37.
Stuart, A. (1955). A test for homogeneity of the marginal distributions in a twoway classification. Biometrika, 42, 412 – 416.
Tahata, K. and Tomizawa, S. (2011). Generalized linear asymmetry model anddecomposition of symmetry for multiway contingency tables. Biometrics
and Biostatistics, 2, 1 – 6.

Analysis of the t linear mixed model andevaluation of a direct estimation method
M. Regis1 , N. Nooraee1 , A. Brini1 , E. R. van den Heuvel1
1

Eindhoven University of Technology, Department of Mathematics and Computer Science, P.O. Box 513, 5600 MB Eindhoven, The Netherlands

E-mail for correspondence: m.regis@tue.nl
Abstract: The t linear mixed model naturally occurs from the linear mixed
model with inverse gamma distributed heterogeneous variances. Various publications emerged with the aim of proving superiority with respect to traditional linear mixed models, extending to more general settings and proposing more efficientestimation methods. However, little attention has been paid to the mathematicalproperties of the model itself and to the evaluation of the proposed estimation
methods in finite samples with many repeated outcomes. In this paper we propose an in depth analysis of the t linear mixed model, with particular focus on its
identifiability and on the evaluation of a maximum likelihood estimation method
via an intensive simulation study.
Keywords: Heterogeneous variances; outliers; latent variable; model identifiability; variance components.

1

Introduction

The linear mixed model (LMM) introduced by Laird and Ware (1982) has been
largely investigated and applied to diverse longitudinal studies for their appealing
feature of capturing both the intra- and inter-subject variability, together with abroad set of possible correlation structures. However, they rely on strong normality assumptions for both the random effects and the residuals, properties often
not satisfied in real datasets which can be heavy-tailed, with outlying observations and heteroscedastic residual variances. The t linear mixed model (tLMM)
is a particular extension of the classical LMM in which the response variable isjointly t distributed, which makes it robust to outliers. The literature on tLMM
is vast, but little has been explored on the properties of the model itself andthe accuracy of the associated estimation methods in finite samples with many
repeated outcomes. In this paper we thoroughly investigate tLMMs, study its
properties and derive identifiability conditions. According to these restrictions,This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

236

Rigorous analysis of the t linear mixed model

we conduct an intensive simulation study to assess the accuracy of the estimation
method under different scenarios. We implement the maximum likelihood (ML)estimation method introduced by Lin and Lee (2006), extending it by allowing
the inclusion of both random intercept, slope and AR(1) correlation structure at
the same time.

2

The t linear mixed model

We consider the t linear mixed model defined asyi = Xi ζ + Zi ui + ei ,where yi is the vector of ni observations for subject i = 1, . . . , m, Xi and Zi are
known ni × p and ni × q matrices corresponding to the vectors ζ and ui of the
fixed and random effects respectively, and ei the vector of residuals. The model
extends the traditional LMM by introducing a latent variable γi for the varianceof the residuals and of the random effects,
γi = 1/vi ∼ Gamma(α, β).
Given γi , the random effects are assumed to be independent of the residuals withui |γi ∼ N (0, vi G) and ei |γi ∼ N (0, vi R). Here G is the covariance matrix for the
random effects which may take a general unstructured formτ12
η1,2 τ1 τ2 . . . η1,q τ1 τq2
η1,2 τ1 τ2
τ2. . . η2,q τ2 τq 

G=
 ...
...
...
... 
η1,q τ1 τq η2,q τ2 τq . . .
τq2
and R is an AR(1) correlation matrix. Under the stated distributional assumptions, yi is t distributed with 2α degrees of freedom and variance-covariancematrix Var(yi ) = vi Ωi = vi (Zi GZiT + R).

3

Identifiability

We assume here the definition of identifiability conditions as restrictions on theparameter space Θ = {θ = (ζ1 , . . . , ζp , τ1 , . . . , τq , η, σ 2 , ρ1 , . . . , ρni , α, β)} suchthat the mapping θ −→ Li (θ) is one to one, namelylog(Li (θ1 )) = log(Li (θ2 )) ⇐⇒ θ1 = θ2 ,where Li is the contribution to the total likelihood from subject i = 1, . . . , m. In
general, in case of n observations, identifiability conditions require the number Nof parameters of the variance-covariance structure to be less or equal to n(n+1)/2
since this is the maximum number of distinct elements in Ωi . Assuming this condition to be satisfied, Xi and Zi to be full rank, and the variance-covariancematrices G and R themselves identifiable with respect to the corresponding parameters, the problem reduces to the identifiability of−(α+ni /2)Γ(α + ni
)
1
1 2
2
fyi =
.
1
+
∆
Γ(α) (2πβ)ni /2 |Ωi |1/2
2βRegis et al.

237

Here ∆2 = (yi − Xi ζ)T Ω−1i (yi − Xi ζ), and so formulated the problem is not
identifiable. In the literature it is commonly assumed α = β to overcome thisissue and reduce identifiability conditions to those for classical LMM. Althougha requirement on the ratio between α and β is necessary, that can be somewhatlooser, as the parameter β can be easily be compensated by Ωi . For example byreparametrizing Ω̃i = ω Ωi , and imposing a constraint on the ratio ω = β/α,
it is possible to show that for specific combinations of G and R, the model is
identifiable.

4

Simulation study

We introduce a parametrization in compliance with identifiability conditions andprevious literature, with the latent variable γi gamma distributed with bothparameters equal to α and an extra positive parameter σ 2 to rescale the varianceof the residuals ei |γi ∼ N (0, vi σ 2 R). We consider a tLMM including three fixed
effects (e.g. gender, age and time) and two random effects (intercept and slope),yij = ζ0 + ζg xig + ζa xia + ζt tj + δi + ωi tj + eij .
The coefficients for the fixed effects are set to determined values, while we choose
a set of possible values for each of the variance-covariance parameters, the combination of which results in 64 simulation settings. We initially investigate themethod accuracy with 50 subjects and then increase it to 100 for the criticalcases. Each setting is simulated 1000 times to ensure stability. In Figure 1 weshow the relative bias in estimating the variance-covariance parameters, as the
fixed effects can be estimated with good accuracy. The results from the simulation study show that overall the method estimates the parameters with highaccuracy, and the estimation in the critical cases can be consistently improvedby increasing the number of subjects involved in the study.

5Conclusion

In this work we derive identifiability conditions for the tLMM and test the accuracy of a ML-based estimation method for tLMM via an intensive simulation
study. Our results show that the model should be applied with caution, as one
may occur in the wrong estimates by simply using routines implemented in theavailable softwares. When identifiability conditions are accounted for, the proposed estimation method is shown to be accurate.

Acknowledgments: Research performed within the framework of the Data
Science Flagship.

References
Laird, N.M., and Ware, J.H. (1982) Random-effects models for longitudinal data.
Biometrics, 963 – 974.
Lin, T.I. and Lee, J.C. (2006) A robust approach to t linear mixed models applied to multiple sclerosis data. Statistics in medicine, 25(8), 1397 – 1412.

238

Rigorous analysis of the t linear mixed model

tau1

10

Density

5

4

0

0

2

Density

6

8

15

alpha

0.00

0.05

0.10

0.15

0.20

0.25

−0.15

−0.10

−0.05

0.00

N = 64 Bandwidth = 0.01925

N = 64 Bandwidth = 0.01053

tau2

eta

3

Density100−0.025−0.015−0.005

0.005

−0.1

0.0

0.1

0.2

0.3

0.4

N = 64 Bandwidth = 0.03143

sigma2

rho

0.5

60
40
20

Density

10 15 20 25

80

N = 64 Bandwidth = 0.002206

0

0

5

Density

2

60
40
20

Density

4

80

5

−0.05

−0.02

0.00

0.02

0.04

0.06

N = 64 Bandwidth = 0.00654

0.08

−0.02

−0.01

0.00

0.01

N = 64 Bandwidth = 0.002207

FIGURE 1. Relative bias in estimating the variance-covariance parameters inall the settings.

Combining information at different scales inspatial epidemiology: a Composite Link
Generalized Additive Model approach
Diego Ayma1 , Dae-Jin Lee2 , Marı́a Durbán1
12Department of Statistics, Universidad Carlos III de Madrid, Madrid, Spain
Basque Center for Applied Mathematics, Bilbao, Spain

E-mail for correspondence: dlee@bcamath.org
Abstract: Mortality data provide valuable information for the study of the spatial distribution of mortality risk, in disciplines such as spatial epidemiology,
medical demography, and public health. However, they are often available in an
aggregated form over irregular geographical units, hindering the visualization of
the underlying mortality risk and the detection of meaningful patterns. It is also
common that factors that may affect mortality are usually measured in a different
spatial resolution than mortality data. In this paper, we propose the use of thecomposite link models to combine information such as covariates measured at
different spatial scales (i.e. municipalities, districts or census tracts levels) withina generalized additive framework. We called this model CL-GAM (“Composite
Link - Generalized Additive Model”). We illustrate our proposal with the analysis
of deaths by cardiovascular diseases in Madrid, Spain (period 1996–2003).
Keywords: Disease mapping; penalized Composite Link Models; Mixed models.

1

Introduction

Disease maps deal with public health data that are usually available in an aggregated form over geographical units, like counties, districts, and municipalities.
Epidemiologists, health care practitioners, and other related researchers use thesedata to study the spatial distribution of mortality caused by an specific disease,and thus identify areas of excess and their potential risk factors. In this paper,
we propose the extension of the spatial penalized composite link model (P-CLM)
in Ayma et al. (2016) for the case of spatial area-to-area (ATA) disaggregation(i.e. from coarse geographical units to smaller units), where mortality maps fromaggregated counts are combined with information at different scales.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

240

CL-GAM for combining spatial information at different scales
Raw log(SMR)

ATA smoothed log(SMR)

TetuánChamartínChamberíSalamanca
CentroRetiro[−0.108,0.041][−0.155,−0.108á
[−0.196,−0.155á
[−0.229,−0.196á
[−0.255,−0.229á
[−0.284,−0.255á
[−0.31,−0.284á
[−0.337,−0.31á
[−0.379,−0.337á
[−1.234,−0.379á

[−0.108,0.041][−0.155,−0.108á
[−0.196,−0.155á
[−0.229,−0.196á
[−0.255,−0.229á
[−0.284,−0.255á
[−0.31,−0.284á
[−0.337,−0.31á
[−0.379,−0.337á
[−1.234,−0.379á

Arganzuela
Census tract boundaries
District boundaries
Selected district boundaries

FIGURE 1. Left: raw log(SMR) in Madrid. Right: ATA (from municipalities tocensus tracts disaggregation) smooth log(SMR).

2

Deaths by cardiovascular disease in the Communityof Madrid (1996–2003)

The data correspond to the number of observed and expected female deaths bycardiovascular diseases in the community of Madrid, Spain, over the period 1996–
2003, which are available at different (aggregated) spatial levels. We consider
the area-to-area case (i.e. from municipalities to census tracts). Let y be thevector of observed deaths and assume that y is Poisson distributed with meanµ. Also, consider the longitude and latitude centroid coordinates of units vj∗ ,(x1j , x2j ), given by the longitude and latitude vectors x1 = (x11 , ..., x1m )T andx2 = (x21 , ..., x2m )T , respectively. Then, the P-CLM for the area-to-area case isgiven by:
µ = Cγ = C (ef ∗ exp(f (x1 , x2 ))) ,(1)
where γ denotes the vector of latent expectations at the fine unit level and C is
an area-to-area composition matrix, whose entries are obtained as cij = 1 if vj∗
is contained in unit vi and 0 otherwise. The vector γ in model (1) is expressedin terms of the vector of exposures at the fine unit level (ef ). Hence, we can use
the standardised mortality ratio (SMR), i.e. SM Ri = yi /ei , instead of counts.
The function f (x1 , x2 ), represents the latent spatial trend that is modelled via a
Tensor product bivariate P-spline.
Ayma et al. (2016), derived the mixed model reparameterization of the P-CLM,
obtaining the mixed model equations with the so-called working matrices: X̆ =W−1 CΓX and Z̆ = W−1 CΓZ, with W = diag(µ) and Γ = diag(γ). The covariance matrix of the random effects G depends on two variance components τ1
and τ2 that controls the spatial smoothness.
Finally, defining the working vector: z = X̆β + Z̆α + W−1 (y − µ). Hence, given
b and α,b we use penalized quasi-likehood (PQL), the solution is achieved by iterβb αb and the restricted or residual maximum likelihood (REML)
ation between β,
criteria until convergence. Figure 1, illustrates the ATA P-CLM allowing for abetter visualisation of the central districts of Madrid.

241

1.0
0.5

smooth % unemployment effect

0.0

0.0
−0.5
−1.5

−0.5

−1.0

smooth % ageing effect

0.5

1.5

1.0

2.0

Ayma et al.

0

10

20

30
% ageing

40

50

60

70

0

5

10

15

20

25

30

% unemployment

FIGURE 2. Smooth effects for covariates measured at municipality level for ATA
disaggregation of mortality rates by cardiovascular disease. Left: % of population> 65. Right: % of unemployment.

3

Extending the CLMM to combine information at
different spatial scales

In many occasions it might be of interest to include covariates in the model which
are collected at a different scales of the response variable, or covariates measured
at different scales. The CLM provides a framework in which both situations
can be easily resolved, yielding what we call composite link generalized additive
mixed model (CL-GAMM). Given that in all cases we are interested in spatial
disaggregation of counts, we can consider different situations: (a) the covariates
are measured at the fine level; (b) the covariates are measured at the aggregatedlevel; and (c) when we have covariates measured both at aggregated and finescale levels, which generalizes the previous cases a) and b).
Including covariates measured at census tract level. In order to study
relationships between different socio-economic indicators and mortality rates, weconsidered two covariates: i) an indicator of ageing (people whose ages are greateror equal than 65 years old); ii) an indicator of unemployment for people whoseages are greater or equal than 16 years old. Figure 2, shows the smoothed additiveeffects.
Comparing mortality of two populations. Let y = (yfemale , ymale )T be thevector of observed number of counts for females and males, and ef = (ef-female , ef-male )T
be the vector of expected number of counts for females an males at the fine unit
level. The model is extended to include a factor variable for sex and hence able
to compare both populations. Figure 3, shows the resulting ‘contrast map’ where
the difference of the mortality surfaces at census tract levels between females andmales are classified into three categories based on the smoothed spatial trends
and standard errors.

Acknowledgments: This research was supported by the Basque Government
through the BERC 360 2014-2017 and by the Spanish Ministry of Economy andCompetitiveness MINECO and FEDER: BCAM Severo Ochoa excellence accred-

242

CL-GAM for combining spatial information at different scales

FIGURE 3. Classification of mortality differences between females and males
into three categories.

itation SEV-2013-0323 and MTM2016-74931-P.

References
Ayma, D., Durbán, M., Lee, D.-J., and Eilers, P.H.C. (2016). Penalized composite link models for aggregated spatial count data: a mixed model approach.
Spatial Statistics, 17, 179 – 198.
Eilers, P.H.C. (2007). Ill–posed problems with counts, the composite link modeland penalized likelihood. Statistical Modelling, 7(3): 239 – 254.

Detecting Match-Fixing in the Italian Serie B
Using Flexible Regression
Marius Ötting1 , Christian Deutscher1 , Roland Langrock1
1

Bielefeld University, Germany

E-mail for correspondence: marius.oetting@uni-bielefeld.de
Abstract: In recent years, several match-fixing scandals in soccer took placein Europe and all over the world. In order to avoid match-fixing, the literatureand fraud detection systems analyse odds movements from bookmakers. In ourwork, we analyse not only odds movements but also total volume placed on bets.
For the volume of money placed on bets, we use data from the betting exchange
platform Betfair. We focus on the second division in Italian Soccer (Serie B)
from season 2009/10 until 2015/16, since for this league it has effectively been
proven that some matches were fixed. To model both the mean and the varianceof the betting volume, a Generalized Additive Model for Location, Shape andScale (GAMLSS) is employed. Both the mean and the variance of the bettingvolume are related to several explanatory variables such as the type of bet, thematchday or the popularity of the teams. Possible smooth functions are used toestimate effects of the non-categorical variables. Suspicious matches are detected
via normalised quantile residuals. Using this approach, our model obtains a true
positive rate of 21%, i.e. 21% of the fixed matches are detected.
Keywords: detecting corruption; soccer; flexible regression

1

Introduction

Between 2009 and 2015, several match-fixing occurrences were detected in Italiansoccer, especially in the second division (Serie B). This resulted in forced relegation and point deduction for various teams, potentially endangering the integrityof this league. Detecting match-fixing before the start of the game is challenging.
Fraud detection systems typically analyse odds movements from bookmakers.
Reade and Akie (2013) and Feustel and Rodenberg (2015) argue that in theabsence of any attempt to fix a match, the probability of a particular outcome
stated by bookmakers should be very close to the probability stated by a statistical model, because most of the relevant information for estimating the probability is available, e.g. position in the league table or home field advantage. Feustel
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

244Match-Fixing in the Serie B

and Rodenberg (2015) estimate odds for certain outcomes in football matches
in England, USA, France and Italy and compare these odds with the odds frombookmakers.
In this work, we propose to model both betting odds and the total volume of bets
placed, instead of considering only the former. We demonstrate our approach using betting volume data on the Serie B obtained from the betting exchange platform Betfair. The aim of the empirical analysis is to demonstrate that modelling
betting volumes, in addition to the usual consideration of odds movements, canhelp to more accurately identify suspicious matches. In this regard, the Serie B isa useful case study, since here for several matches it has effectively been proventhat they were fixed. Furthermore, we try to minimise false positives, i.e. flagging
of matches which were not proven to be fixed.

2

Data

In order to find matches with a strong indication of match-fixing activities, pregame data from the betting exchange platform Betfair are analysed. The data
include the seasons 2009/10 to 2015/16. In this period, 3234 matches were played,from which 3224 matches offer full information and are included in the data set.
In addition to betting volumes, several explanatory variables are included in
the data, such as home and away team, the date of the match, the type of bet
and the matchday. For the type of the bet, we focus on bets on the winningteam and on the number of goals scored per match. These type of bets are very
popular compared to others, e.g. bets on a particular player scoring a goal, andtherefore the total volume of these bets is typically higher on average. Due tothe high volumes, placing heavy bets on these markets by match fixers is not
as conspicuous as placing high volume bets on less popular markets. For thenumber of goals scored in a match, the betting types over/under 1.5/2.5/3.5
goals are considered.
To account for popularity effects and the importance of the match, additionalvariables are collected, e.g. Facebook likes of the teams and a dummy variableindicating whether the match is important for the team in terms of the outcomeof the season. There is a large variation between the betting volumes placed on
individual games and types of bets, ranging from a few Euro to ∼ 3 million Euro.

3

Methodology

From visual inspection of the data, we found that the distribution of the bettingvolumes varies substantially across several non-categorical covariates, and that it
would be difficult to accommodate these effects within a linear model. Thus, we
use the very flexible class of Generalized Additive Models for Location, Scale andShape (GAMLSS), which allows 1) to simultaneously model several parametersof the distribution of the response variable (e.g. mean and variance) and 2) toestimate smooth functional effects of non-categorical covariates (Stasinopoulos
and Rigby, 2007). We apply the semi-parametric additive formulation of GAMLSS
formulated by Stasinopoulos and Rigby (2007), given by
gk (θk ) = ηk = Xk βk +

Jk
X
j=1

hjk (xjk ),(1)

Ötting et al.

245

where θk is a parameter of the distribution assumed for the response variable Y ,0
gk (·) is a known link function, Xk is an n × Jk design matrix, βk is a vector of0
regression coefficients of length Jk , and the hjk are unknown smooth functions.
In this work, we assume a normal distribution for the response variable Y , here
the logarithm of the betting volumes. Thus, the explanatory variables are linked
to two parameters of the distribution of Y , namely the mean µ (= θ1 ) and thestandard deviation σ (= θ2 ), leading to the following special case of (1):g1 (µ) = η1 = X1 β1 +J1X

hj1 (xj1 )(2)

hj2 (xj2 )

(3)

j=1

g2 (σ) = η2 = X2 β2 +

J2
X
j=1

The explanatory variables described in Section 2 enter the model in (2) and (3) indifferent parts. Dummy variables for each season, for the type of bet, for the dayof the week and for indicating whether the match is important are included in thelinear parts both in (2) and (3). The effects of the non-categorical variables, i.e.
the matchday and the Facebook likes of the teams, are estimated using P-splinesmoothers. A major advantage of the GAMLSS framework here is the option tomodel both the mean and the variance. The betting volume data is heteroscedastic, as can be seen for example by comparing the betting volumes across the 42
matchdays. The variance in the betting volume increases for the last matchdays,
which can be accommodated within the GAMLSS framework by modelling thevariance via the covariate matchday. By being able to use smooth functional
effects for the non-categorical covariates — the second major advantage of the
GAMLSS framework — the model fit is substantially improved.
We apply this model is to detect outliers, i.e. suspicious matches. Outlier detection
is done by using the normalised quantile residuals, which are standard normally
distributed provided the model is correct (Dunn and Smyth, 1996). Therefore,
observations with normalised quantile residuals larger than three are flagged as
outliers and hence as suspicious matches.

4

Preliminary Results

Out of 19 matches that have effectively been proven to have been fixed, ourmodel flags four, equalling a true positive rate of 21%. There is little information
available on how many matches exactly were fixed in the Serie B. Thus, there
may very well have been more than 19 fixed matches from 2009 to 2015. In total,
59 matches out of 3224 matches are flagged by the GAMLSS applied. From these,
55 are not currently known to have been fixed. Assuming that these matches wereindeed free from irregularities, the GAMLSS model hence returns 55/3224 ≈ 1.7%
false positives, i.e. matches to be flagged as suspicious but unconfirmed to be
fixed.

5

Discussion

Future research should attempt to further increase the number of true positives,
and to reduce the number of false positives. For example, it may be beneficial to246

Match-Fixing in the Serie B

additionally model betting odds using game and team characteristics, in order
to identify deviation between actual and fair betting odds. Betting odds weremodelled for example in Reade and Akie (2013) and in Feustel and Rodenberg(2015), and are also considered within the procedures commonly applied in fraud
detection systems. Odds can be modelled for example by fitting Poisson regression
models to the number of scored goals by a team in a match, then calculating theodds of winning by simulation. Extensions of the Poisson distribution, such as
the zero-inflated Poisson or the bivariate Poisson, may be useful here. These
distributions involve several parameters, such that the GAMLSS framework isagain a good candidate.

References
Dunn, P.K. and Smyth, G.K. (1996). Randomized quantile residuals. Journalof Computational and Graphical Statistics.
Feustel, E.D. and Rodenberg, R.M. (2015). Sports (betting) integrity: detecting
match-fixing in soccer. Gaming Law Review and Economics.
Reade, J.J. and Akie, S. (2013). Using forecasting to detect corruption in international football. Proceedings of the 4th International Conference onMathematics in Sport.
Stasinopoulos, D.M. and Rigby, R.A. (2007). Generalized additive models forlocation scale and shape (GAMLSS) in R. Journal of Statistical Software.

Copula VAR models
Lucia Benetazzo1 , Ernst Wit2
1
2

Università degli studi di Padova, Italy
JBI, University of Groningen, Netherlands

E-mail for correspondence: lucia.benetazzo@gmail.com
Abstract: We propose a VAR graphical model for the analysis of time series
genomic data; our goal is to relax the normality assumption of Gaussian graphical models employing a semiparametric Gaussian copula. We adopt the lasso
penalized likelihood inference approach in order to obtain sparse estimates.
Keywords: Chain graphical model; Time series; Gaussian copula; Lasso penalty;
Gene expression.

1

Introduction

In the last decades, significant developments in DNA microarray technology haveallowed us to collect observations of gene expression levels over time. Time-series
data provide a deeper insight on the biological process under study. This work
can be considered as an extension of the Time Series Chain Graphical Model(TSCGM) by Abegaz and Wit (2013), since we aim to tackle the issue of nonnormality with the use of the Gaussian copula.

2

Methodology

2.1

Time series chain graphical models

TSCGMs assume longitudinal microarray data in which n replications indexed by
i = 1, . . . , n of continuous measurements across p genes indexed by j = 1, . . . , p
are repeated T times. Our aim is to model both contemporaneous and dynamic(delayed) relationships among the genes. This will be accomplished with a timeseries chain graph G = (V, E), where V is a finite set of vertices and the set ofedges E is a subset of the set V × V of ordered pairs of distinct vertices. The
time
ST series chain graph is based on the partitioning of V into a number of blocks
t=1 Vt . Links within a time step are undirected and represent contemporaneous
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

248

Copula VAR models

interactions; links between time steps are directed and they explain dynamic ordelayed interactions between genes in time.
Let Xt = (X1t , . . . Xpt ) be a p-variate random variable associated with the nodes
in Vt . Xt represents the gene expression levels at time t. We assume that Xt follows a Markovian dynamics, which can be translated into a vector autoregression
process of order 1 (VAR(1)):
Xt = ∆Xt−1 + t .

(1)

In the TSCGM setting, the following normality assumption is made: t ∼ Np (0, Ω−1 ).
In this project, our aim is to make no assumption about the distribution of gene
expressions. Particularly, in order to weaken the gaussian assumption on t , wewill exploit the Gaussian copula.

2.2

Copula VAR models

Let Zt = (Z1t , . . . Zpt ) be a multivariate latent variable. The Gaussian copulatransformation is defined as:
Zjt = Φ−1 (Fjt (Xjt )),for j = 1, . . . , p,t = 1, . . . , T,(2)where Xjt is the gene expression level for gene j at time t, Fjt is the marginal
CDF of gene j at time t and Φ−1 is the quantile function of the univariate normal
distribution N (0, 1). The latent variable Zt follows a dynamics equivalent to thatin (1):
Zt = ΓZt−1 + ηt ,where Zt |Zt−1 ∼ Np (ΓZt−1 , Θ−1 ). The matrices Γ and Θ are in general different
from ∆ and Ω respectively, but they have the same nonzero entries, resulting inthe same graphs. Therefore, Γ and Θ contain all the information about the dependencies among the variables. In particular, nonzero entries in Γ are equivalent
to directed edges; likewise, contemporaneous interactions (undirected edges) are
related to nonzero entries of the concentration matrix, also known as precision
matrix, Θ:(α, β) ∈ Vt−1 × Vt ⇔ Γαβ 6= 0,(α, β) ∈ Vt × Vt ⇔ Θαβ 6= 0.

We now define the log-likelihood for n replicates each at T time steps as`(Γ, Θ) = −

npT
nT
nT
log(2π) +log Θ −
T r(SΓ Θ) + D
2
2
2

(3)

where D P
is the P
sum of the logs of the Jacobians of the transformation in (2) andn
T>
1SΓ = nTi=1
t=1 (Zit −ΓZi,t−1 )(Zit −ΓZi,t−1 ) . We exploit the marginal rank
likelihood by Hoff (2007), which can be seen as a type of marginal likelihood
function for estimation in the presence of nuisance parameters (the marginal
distributions). This allows us to neglect the term D in (3).

2.3

Kernel density estimate

The marginal CDFs involved in the transformation given in (2) are unknown: weestimate them in a nonparametric fashion via weighted kernel CDF estimation.
Weights exponentially decay to zero as the time interval becomes larger: i.e., inBenetazzo and Wit

X1t−2

X1t−1

X1t

X2t−2

X2t−1

X2t

X3t−2

X3t−1

X3t

249

FIGURE 1. Example of graphical representation of a Copula VAR model oforder 1 for p = 3 time series variables. Directed edges represent nonzero entriesof the transition matrix Γ and undirected edges (dashed links) characterize thenonzero entries of Θ.
order to estimate the CDF in a time point, we use observations from the futureand from the past giving more importance to those observations closer in time.
Hence, the CDF estimate for the j-th gene at time t isF̂jt (xijt ) =



T X
n
X
xijt − xkjl
Φ
wtl ,
hj
l=1 k=1|t−l|where wtl = PTω ω|t−j| , Φ is the CDF of the standard normal (which is thej=1chosen kernel), hj is the bandwidth parameter (different for each gene) and ω isa constant which determines the decay velocity. We choose ω = 0.925.

2.4

Penalized log-likelihood

Gene networks are known to be sparse, i.e. a gene will not interact with all thegenes in the network but only with a relatively small subset of them. This resultsin the matrices Γ and Θ to have many zero entries. For the purpose of obtaining
sparse estimates, we add two lasso penalties to the likelihood, for both Γ and Θ.
Therefore, the objective function for optimization is defined as`pen (Γ, Θ) = log Θ − T r(SΓ Θ) − λ1

p
X
i6=j

|θi,j | − λ2pX|γi,j |

i6=jwhere θi,j and γi,j are the entries of Θ and Γ. The estimation is accomplished
via a two-stage iterated procedure as in Rothman (2010). The first stage consistsof the following optimization problempXΘ̂ = argmax log Θ − T r(SΓ Θ) − λ1
|θi,j |
with Γ fixed.
Θ

i6=j

250

Copula VAR models

We use the graphical lasso algorithm to solve it. The second stage is given bypX|γi,j |
with Θ fixed.
Γ̂ = argmax − T r(SΓ Θ) − λ2Γi=jThe solution is computed using a cyclical-coordinate descent algorithm.

2.5

Model selection

The regularization parameters of the lasso penalty must be carefully tuned. One
can adopt the Bayesian information criterion (BIC) to select the parameters,
which is defined as
BIC(λ1 ) = −nT {log Θ̂λ1 − T r(SΓ̂λ Θ̂λ1 )} + k log(nT ),2

where k is the number of nonzero estimated parameters. The values of λ1 and λ2that jointly minimize the BIC criterion have to be chosen.

References
Abegaz, F. and Wit, E. (2013). Sparse time series chain graphical models forreconstructing genetic networks. Biostatistics, 14, 3, 586 – 599.
Hoff, P.D. (2007). Extending the rank likelihood for semiparametric copula estimation. The Annals of Applied Statistics, 265 – 283.
Rothman, A. J., Levina, E., and Zhu, J. (2010). Sparse multivariate regression
with covariance estimation. Journal of Computational and Graphical Statistics, 19.4, 947 – 962.

An exact test for comparing two predictive
values in small-size clinical trials
Kanae Takahashi1 , Kouji Yamamoto1
1

Department of Medical Statistics, Osaka City University Graduate School of
Medicine, Japan

E-mail for correspondence: takahashi.kanae@med.osaka-cu.ac.jp
Abstract: The positive predictive value and the negative predictive value describe the performance of a diagnostic test. There are several methods to test
the equality of predictive values of two binary diagnostic tests. However, thesemethods were premised on large sample size theory and they may not be suitable
for small-size clinical trials.
In this study, we propose an exact test for conducting a small-size clinical trialthat investigates the equality of predictive values of two binary diagnostic tests.
In addition, we execute simulation studies to evaluate the performance of the
proposed exact test and existing methods in small-size clinical trials.
The proposed test can calculate exact p-value and, as a result of simulations,the performance of it is not much different from the other methods. Therefore,
it is considered that the proposed exact test may be useful for small-size clinical
trials.
Keywords: exact method; negative predictive value; positive predictive value;small-size clinical trials.

1

Introduction

In medicine, diagnostic tests are important for early detection and treatment ofdisease. The positive predictive value (PPV) and the negative predictive value(NPV) describe the performance of a diagnostic test. The PPV is the probabilityof having the disease when the diagnostic test result is positive, and the NPV is
the probability of not having the disease when the diagnostic test result is negative. The PPV and NPV are useful clinically, and may influence the treatment
decision.
There are several methods to test the equality of predictive values of two binary
diagnostic tests (Leisenring et al, 2000; Moskowitz and Pepe, 2006; Wang et.al,
2006; Kosinski, 2013). Some researchers use regression frameworks, and others
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

252

An exact test for comparison of predictive values in small-size clinical trials

utilize the multivariate central limit theorem and the delta-method. These methods were premised on large sample size theory. For this reason, these methods
may not be suitable for small-size clinical trials.
In this study, we propose an exact test for conducting a small-size clinical trialthat investigates the equality of predictive values of two binary diagnostic tests.
Furthermore, we execute simulation studies to evaluate the performance of the
proposed exact test and existing methods in small-size clinical trials.

2

Methods

We performed a simulation study to assess the performance of two existing methods; one method is based on the delta-method with identical link (Tm) and theother is weighted generalized score statistic (Twgs) in a regression framework.
The performance measure were actual type 1 error rate and empirical power. We
executed 1,000,000 repeated simulations for each method. The simulation was
conducted in the same way as Kosinskis article. The odds ratio of diseased groupwas set to ORD+ =5, and ORD− =2 for the odds ratio of non-diseased group. The
total sample size N was assumed to be 30, 40 and50 because this study assumesa small-size clinical trial. PPV for new diagnostic test (PPV1) was set to 0.75 or
0.85 and PPV for the existing diagnostic test (PPV2) was 0.75. NPV for new diagnostic test (NPV1) was set to 0.85 or 0.95, NPV for the existing diagnostic test
(NPV2) was 0.95. The disease prevalence was assumed to be 0.4. The nominal
type 1 error rate was set to 0.05 (two-sided test).
In addition, we performed a simulation study to assess the performance of the
proposed exact test. The exact test is based on the permutation test. The simulation was conducted in the same way as above.

TABLE 1. Actual type 1 error rate (PPV1=PPV2=0.75)

N

Tm

Tgs

Exact

30
40
50

0.078
0.068
0.064

0.043
0.048
0.049

0.025
0.034
0.037

TABLE 2. Actual type 1 error rate (NPV1=NPV2=0.80)

N

Tm

Tgs

Exact

30
40
50

0.044
0.051
0.053

0.035
0.042
0.045

0.024
0.032
0.037

Takahashi and Yamamoto

253

TABLE 3. Empirical power (PPV1=0.85 PPV2=0.75)

N

Tm

Tgs

Exact

30
40
50

0.133
0.143
0.162

0.101
0.130
0.154

0.089
0.124
0.148

TABLE 4. Empirical power (NPV1=0.90 NPV2=0.80)

3

N

Tm

Tgs

Exact

30
40
50

0.190
0.274
0.346

0.162
0.248
0.321

0.130
0.217
0.294

Results

Table 1 shows the actual Type 1 error rate for comparing two PPVs. It showsthat the actual type 1 error rate of Twgs and Exact test does not exceed thenominal type 1 error rate 0.05. Table 2 shows the actual Type 1 error rate forcomparison of NPVs. It shows almost same findings as the case of PPV.
Table 3 shows the empirical power of PPVs. It shows that the empirical powerof Tm is the highest among three methods, and the empirical power of proposed
exact test is not much different from that of Twgs. Table 4 shows the empirical
power of NPVs. It also shows similar findings to the case of PPVs.

4

Discussion

By the result of the simulation studies, we consider that the proposed exact test
may be useful for small-size clinical trials because it can calculate exact p-value,
and the exact test does not have so less power than other methods.

References
Kosinski, A.S. (2013). A weighted generalized score statistic for comparison ofpredictive values of diagnostic tests. Statistics in medicine, 32, 964 – 977.
Leisenring, W. et al. (2000). Comparisons of predictive values of binary medical
diagnostic tests for paired designs. Biometrics, 25, 2215 – 2229.
Moskowitz, C.S. and Pepe, M.S. (2006). Comparing the predictive values of diagnostic tests: sample size and analysis for paired study designs. Clinical
Trials, 3, 272 – 279.
Wang, W. et al. (2006). Comparison of predictive values of two diagnostic tests
from the same sample of subjects using weighted least squares. Statistics
in Medicine, 25, 2215 – 2229.

Comparison of Two Testing Procedures inClinical Trials with Multiple Binary
Endpoints
Takuma Ishihara1 , Kouji Yamamoto1
1

Department of Medical Statistics Osaka City University Graduate School of
Medicine, Japan

E-mail for correspondence: ishihara.takuma@med.osaka-cu.ac.jp
Abstract: The test treatment in confirmatory clinical trials is often assessed byusing multiple primary endpoints. We considered the efficacy of the trials that isconfirmed only when non-inferiority for all endpoints and superiority for at least
one endpoint. Perlman and Wui2004jand Nakazuru et al.i2014jproposed testingprocedures when the multiple primary endpoints are continuous variables. However, it is not yet discussed the case that a trial has multiple binary endpoints.
In this presentation, we consider a testing procedure when the multiple primary
endpoints are binary, and show the performance through Monte Carlo simulations.
Keywords: Clinical trial; Multivariate Bernoulli distribution; Non-inferiority;
Superiority.

1

Introduction

In confirmatory clinical trials, the efficacy of a test treatment are sometimes assessed by using multiple primary endpoints. However, it is difficult to demonstratethat each endpoint is significantly when the number of endpoints is not small.
For the reason, this manuscript deals with a case where it is superior for at least
one of the endpoint and not clinically inferior for the remaining endpoints. Perlman and Wu (2004) proposed a testing procedure that is applicable to the abovecase. This procedure has a disadvantage that cause the inflation of type I error
when correlation coefficients among the endpoints are highly positive. Moreover,
Nakazuru et al. (2014) proposed a modified testing procedure for Perlman andWu method. These procedures are able to be applied only when endpoints arecontinuous variables. Therefore, we propose a new testing procedure when thecase that multiple primary endpoints are binary.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Ishihara and Yamamoto2

255

Notations and Hypotheses

To simplify this case, we consider comparing p binary endpoints with 2 treatmentgroups comprising n1 and n2 subjects. Let Yijk (i = 1, 2; j = 1, ..., p; k = 1, ..., ni )denote the response variable of the kth subject to the ith treatment at the jth
endpoint, Ȳij (i = 1, 2; j = 1, ..., p) is the sample mean for the jth endpoint to
the ith treatment, and set X = (X1 , ..., Xp )t with Xj = (Ȳ1j − Ȳ2j )(j = 1, ..., p),
where the superscript t denotes transpose.
A null hypothesis H0 is expressed by 
H0 : max µj ≤ 0 ∪ min (µj + j ) ≤ 0 ,
1≤j≤p

1≤j≤p

where µj = µ1j − µ2j is mean of the treatment effect, and j (j = 1, ..., p) is noninferiority margin of jth endpoint that denotes prespecified positive constants.

3

Proposal of a New Procedure

3.1

Perlman and Wu Procedure

Perlman and Wui2004jproposed a testing procedure that rejects above null hypothesis H0 if and only if
kX − πs (X; N p )k2s > c∗αpcn1 ,n2 (Xj + j )/ σ̂jj > tα

and
for

j = 1, ..., p,

where c2n1 ,n2 = n1 n2 /(n1 + n2 ), and let σ̂jj denote jth diagonal element of thepooled sample covariance matrix Σ̂, then S ≡ (n1 + n2 − 2)Σ̂ is distributed as a
Wishart distribution Wp (Σ, n1 + n2 − 2) when n1 + n2 > p + 2. Here N p is thenon-positive orthant in Rp (p-dimensional Euclidean space), kxk2s ≡ xt S −1 x is
the Euclidean norm determined by S, πs (X; N p ) is the projection of X onto N p
with respect to this norm. Then c∗α is the critical value of size-α one-sided LRT(likelihood ratio test) for superiority part of H0 determined by 1




χ2p−1
χ2p
1
1
α = Pr 2> c∗α + Pr 2
> c∗α ,
2
χn1 +n2 −p2
χn1 +n2 −p−1
where χ2n is a chi-square variable with n degrees of freedom.

3.2

Nakazuru et al.’s Procedure

Nakazuru et al.i2014jproposed a more powerful procedure rather than Perlman
and Wu procedure by modifying the Glimm et al. (2002)’s method, and it rejects
the null hypothesis if and only ifmin(ū2A , ū2B ) > cpcn1 ,n2 (Xj + j )/ σ̂jj > tα

and
for

j = 1, ..., p.

Here uA ≡ (uA1 , ..., uAp )t = cn1 ,n2 AX and uB ≡ (uB1 , ..., uBp )t= ( detA)2/p cn1 ,n2 BX are the test statistics that distributed as a p variable standetB
dard normally distribution. Then ū2A and ū2B are as follows:256

Comparison of Two Testing Procedures in Clinical Trials with Multiple Binary Endpointsū2A =pX

max(uAj , 0)2

j=1

ū2B =pX

max(uBj , 0)2 .

j=1

The matrix A and B are omitted here. For further details, see Nakazuru et al.
(2014). Then c is determined by
p−1

1 X
p!
1
·
n1 +n2 −1−p+j
2p j=0 j!(p − j)! B( p−j,)22Z c/(n1 +n2 −1)n1 +n2 −1−p+j
p−j
1
−1
2
r 2 −1 (1 − r)
dr + p = 1 − α20where B(a, b) is beta function defined with gamma function.

3.3

A new testing procedure

However, these methods are discussed when the efficacy of a test treatment arecontinuous variables. Suppose that the vector of responses Yik = (Yi1k , ..., Yipk )are distributed as a P -variate Bernoulli distribution with E(Yijk ) = πij , V (Yijk ) =0
πij (1 − πij ), and corr(Yijk , Yij 0 k ) = ρjj
for all j 6= j 0 (1 ≤ j < j 0 ≤ p). Then, weican apply the same way as Nakazuru et al.’s procedure to the case that multiple
primary endpoints are binary by using an approximation based on a multivariate
central limit theorem. Let π̂ij (i = 1, 2; j = 1, ..., p) be a sample proportion of thejth endpoint to the ith treatment, and ∆j = π̂1j − π̂2j denote the difference ofjth endpoint. It rejects the null hypothesis if and only ifmin(ū2A , ū2B ) > c∆j + jq> Zαπ̂2j (1−π̂2j )π̂1j (1−π̂1j )
+
n1
n2

and
for

j = 1, ..., p,

where Zα is the upper α-percentile of standard normally distribution. On the
day of the presentation, we introduce the other procedure that is applied Westfall
PH and Troendle JF (2008) method to the part of superiority in test statistics,
and compare the performance of two procedures.

3.4

Numerical comparison

The performance of the proposed procedure was examined through Monte Carlosimulations. In this simulation, we confirmed difference of 2 proportions in thelevel of significance was set at 0.05. For evaluating powers, we assumed that
(π11 , π12 ) = (0.6, 0.4), (π21 , π22 ) = (0.4, 0.4), the simulated data was repeated
10,000 times. In addition, we assumed that (π11 , π12 ) = (0.5, 0.5), (π21 , π22 ) =(0.5, 0.5), the simulated data was repeated 100,000 times for evaluating type I
error rates. The sample size is set as n = 30 and n = 100. The correlation
coefficient is set as ρ = 0, ρ = 0.4 and ρ = 0.8 in each case.

Ishihara and Yamamoto

257

TABLE 1. Estimated powers and type I error rates.

PowersType I error rates

ρn = 30n = 100n = 30n = 100

0.8
0.4
00.57170.47750.47280.91320.89860.90090.045400.036960.030360.000250.000260.00026

References
Nakazuru, Y., Sozu, T., Hamada, C. and Yoshimura, I. (2014). A new procedureof one-sided test in clinical trials with multiple endpoints. Japanese Journalof Biometrics, 35, 17-35.
Glimm, E., Srivastava, M. and Lauter, J. (2002). Multivariate tests of normalmean vectors with restricted alternatives. Communications in Statistics, B
31, 589-604.
Perlman, M.D. (1969). One-sided testing problems in multivariate analysis. The
Annals of Mathematical Statistics, 40, 549-567.
Perlman, M.D. and Wu, L. (2004). A note on one-sided tests with multiple
endpoints. Biometrics, 60, 276-280. 26, 1193-1207.
Westfall PH and Troendle JF. (2008). Multiple testing with minimal assumptions. Biometrical Journal, 50(5), 745-755.

Parameter Inference in the Pulmonary Blood
Circulation of Mice
L. Mihaela Paun1 , M. Umar Qureshi2 , Mitchel Colebank2 ,Mansoor A. Haider2 , Mette S. Olufsen2 , Nicholas A. Hill1 , Dirk
Husmeier1
1School of Mathematics and Statistics, University of Glasgow, Glasgow G12
8SQ, UK
2
Department of Mathematics, NC State University, Raleigh, NC 27695, USA
E-mail for correspondence: l.paun.1@research.gla.ac.uk
Abstract: This study focus on parameter inference in a pulmonary blood circulation model for mice. It utilizes a fluid dynamics network model that takes someparameter values and aims to mimic features of the pulmonary haemodynamics under normal physiological and pathological conditions. This is of medical
relevance as it allows monitoring the progression of pulmonary hypertension.
Constraint nonlinear optimization is successfully used to learn the parameters.
Keywords: Pulmonary hypertension; Parameter Inference; Constraint Nonlinear Optimization; Partial Differential Equations; Windkessel model.

1

Introduction

Pulmonary hypertension (PH) is a leading cause of right heart failure. It involves
vascular remodelling including stiffening of the large and small arteries. Clinically,
PH is diagnosed by analysing blood pressure (BP) measured invasively in thelarge pulmonary arteries. However, key parameters, including arterial stiffness,cannot be measured in-vivo. This creates the need for methods to indirectly estimate parameters from the measured hemodynamic blood flow and pressure data.
This study uses a 1D fluid dynamical network model that predicts blood flow
and pressure in the large pulmonary arteries (for details see Qureshi et al., 2017).
The model is used to predict blood flow and pressure in healthy and hypoxic
mice, for which data were acquired invasively (Tabima et al., 2012). The method
discussed here is not specific to mice, but can easily be extended to analysis ofsimilar data from humans for whom repeated invasive procedures are requiredfor diagnostic and treatment purposes. The ultimate goal behind this model, andThis paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Paun et al.

259

hence the motivation behind inferring the parameters, is to minimize the number of invasive procedures for PH patients, as well as to assist the clinicians indevising better treatment strategies. Thus, this study focuses on inference of key
parameters pertinent to disease detection and treatment. We have shown thatby using our statistical method we can improve BP prediction in both healthy
and hypoxic mice. This leads to enhanced reliability of key parameter estimatesobtained using the model.

2Mathematical Model

The 1D fluids model studied here is derived from the incompressible axisymmetric
Navier-Stokes equations for a Newtonian fluid, and coupled with a constitutive
wall model predicting stiffness of the blood vessels. The arterial network geometry,
including length and radii for the 13 largest vessels in the pulmonary vasculature
is obtained from a micro CT image of a healthy mouse lung (see figure 1(a)). In
order to solve the equations, boundary conditions are specified at the inlet andoutlet vessels in the network. The system is driven by imposing an invasively
measured flow profile at the inlet of the MPA whereas conservation of blood flow
and continuity of pressure are ensured across the bifurcations. At the 7 terminal
outlet vessels, 3-element Windkessel models (represented by two resistors R1 , R2
and a capacitor C) are attached. The outflow boundary conditions account for
the lumped effects of pulmonary hemodynamics beyond the truncated networkof large arteries. The model takes several parameters as input and predicts theflow and pressure at different locations along the large pulmonary arteries. Oneof these parameters is the arterial stiffness, k, which significantly changes its
behaviour during initial stages of PH.

3

Methodology

Let the statistical model be defined by: yi = f (xi ; θ) + i , where yi ∈ y are thenoisy measured flow and pressure, f (.) describes the system behaviour that comesfrom numerically solving the fluids model, θ are the parameters that we wish toinfer from the observed flow and pressure, xi ∈ x denote other input variablesand  are the errors, which we assume are i.i.d following a Gaussian distribution.
The objective function to be minimised using Constraint Nonlinear Optimizationis the Residual Sum of Squares:
X
RSS = (y − f (x; θ))2 =(yi − f (xi ; θ))2
(1)
i

A Sequential Quadratic Programming (SQP) gradient-based method is used tominimise the RSS (Wilson, 1967).

4

Simulations

Simulations are set up to mimic experimental waveforms, which are recorded in
the main pulmonary artery in healthy and hypoxic mice (Tabima et al., 2012).
The parameter set to be inferred includes: θ = (k, r1 , r2 , c), where r1 , r2 , c are

260

Parameter Inference in the Pulmonary Blood Circulation

resistances and capacitor factors used to predict parameters assigned at the outlet
and k is the elastance factor used to predict stiffness in all vessels. Since theparameters are on different scales, to avoid having an ill-conditioned problem, we
rescale the parameters to have the same magnitude. There are certain parameter
configurations that violate the model assumptions, these are marked by setting
RSS to a high value (1010 ). In this study, the RSS is calculated for pressure andwe aim to find parameters that minimise the RSS. The initial parameter values
examined by the SQP algorithm are uniformly drawn from a Sobol sequence toensure a good coverage of the multidimensional parameter space (Bratley et al.,1988). The algorithm is iterated until it satisfies the convergence criterion, i.e.
|θi − θi+1 | < 1e − 11.

5

Results and Discussion

Regardless of the initial value, the algorithm converges towards the same parameter values for both the healthy and hypoxic mouse. Figure 1 shows our optimisedpressure waveform, plotted along the measured and the reference pressure. Panel(d) shows the pressure fit for the hypoxic mouse. The optimized fit predicts data
better than nominal parameter values, supported by a significantly smaller RSS
than the one between the reference and the measured pressure (panel (b)). As
for the healthy mouse (panel (c)), the simulated pressure closely follows the measured pressure except near the peak, where an offset is registered. Nevertheless,in this case too, a clear improvement is achieved over the reference pressure. We
hypothetise that this peak shift is a consequence of: (i) the model specifying theelastic behaviour of the blood vessels and/or the boundary conditions, (ii) uncertainty of the geometry measurements which are not specific to a given mouse,(iii) a combination of (i) and (ii). The overall model prediction appears betterfor the hypoxic than the healthy mouse. Therefore, future work could include
improvements in the fluids model and inferring geometry measurements. Finally,
we also aim to apply our statistical methods presented here to data from humanpatients.

Acknowledgments: This work is part of the research programme of the Centre for multiscale soft tissue mechanics with application to heart & cancer (SoftMech), funded by the Engineering and Physical Sciences Research Council (EPSRC) of the UK, grant reference number EP/N014642/1. Olufsen, Haider andQureshi were supported by National Science Foundation (NSF-DMS # 1615820).
Data were made available by N. Chesler, Department of Biomedical Engineering,
University of Wisconsin, Madison.

References
Bratley, Bennett (1988), Algorithm 659: Implementing Sobol’s Quasirandom Sequence Generator. ACM Trans Math Softw, 14(1), 88 – 100.
Qureshi, Haider, Chesler, Olufsen (2017), Simulating effects of hypoxia on pulmonary haemodynamics in mice. In: Proc CMBE. 271 – 274.
Tabima, D.M. et al. (2012), Persistent vascular collagen accumulation alters hemodynamic recovery from chronic hypoxia. J. Biomech, 45(5), 799 – 804.

Paun et al.

(a)

300

1

3

4
7

5

328

Optimised
Reference

200

RSS

2

(b)

6

13

135

100

8
9

10

11

47

12

23

35

(c)

Measured
Reference
Optimised

15

Healthy1000.05Time (s)0.1

Pressure (mmHg)

Pressure (mmHg)

0

20

261

Healthy

Hypoxic(d)Measured
Reference
Optimised

30
25
20

Hypoxic

15
10
0

0.05

0.1

Time (s)FIGURE 1. (a): The arterial network for the fluid dynamical model, (b): comparison of RSS between reference and optimised pressure simulations, (c) & (d):comparison of simulated pressure using reference and optimised parameters values for the healthy and hypoxic mice.

Wilson, R.B. (1967). Simplicial Method for Convex Programming. PhD thesis,
Harvard University.

Exploring brain-behaviour interactions
during auditory oddball detection withgeneralized additive modeling
Toivo Glatz12 , Wim Tops12 , Natasha Maurits23 , Ben
Maassen12
1

Center for Language and Cognition (CLCG), University of Groningen, The
Netherlands
2
Research School of Behavioural and Cognitive Neurosciences (BCN), Universityof Groningen, The Netherlands
3
Department of Neurology, University Medical Center Groningen (UMCG), The
Netherlands
E-mail for correspondence: t.k.glatz@rug.nl
Abstract: We conducted an auditory oddball experiment with Dutch speaking
first grade children using electroencephalography. While conventional analyses
usually involve averaging over items, subjects, and time windows, we used generalized additive modeling with single trial data and explored possible interactions
with continuous behavioral predictors.
Keywords: ERP; MMN; LDN; GAM.

1

Background and Objective

The mismatch negativity (MMN) is an event related potential (ERP) component, which is widely considered to be an indicator of auditory discriminatory
capabilities (Näätänen et al., 2007). The MMN has been shown to correlate withbehavioral judgements, as well as to reflect behavioural training effects over time
(Lovio et al., 2012). In addition, the MMN acquired from newborns was predictive for their reading fluency many years later (Leppänen et al., 2010). The
MMN usually peaks around 100-230ms after stimulus deviancy in an oddballparadigm, and it is sometimes followed by a long-lasting component named thelate discriminative negativity (LDN) from 250ms onwards (see Figure 1). As thenature of these two components is not fully understood, the latter is also sometimes called MMN; alternatively, both together may be referred to as mismatch
response (MMR).
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Glatz et al.

263

In this work, which is part of a bigger study, we are investigating possible interactions of the MMR components with other behavioral (e.g. IQ, phonologicalawareness (PA)) and biographical measures (e.g. age, gender, handedness, familial risk for reading difficulties) of Dutch speaking first grade children using anovel approach with generalized additive modeling.

2

Design and Methods
Fig. 2 illustrates the improvement of bothgroups with a Mann-Whitney U
We groups
recruitedon
40 thefirstCELF
grade children
playedcomputer game,
which
eitherand thewho
PFB.
Weadid120.5,p=.126) and are illustrat
trained reading related skills such as PA (N=20), or simple arithmetic (N=20), on
not find a statistical difference between the3.
a daily basis for six weeks. Behavioral tests measured reading-related abilities beimprovementoftheintervention
group
and
fore, during, and after the gaming. Electroencephalography (EEG) was recorded
4.2 Mismatch
controlgroupbothtothe
CELF
and PFB
beforeand after
the fortraining
study
the MMRto syllabic speechsounds as a Negativity
potential
correlateofgame-based
phonemediscrimination
improvement.
with respectively an independent sample t
The EEG data were preprocessed using the EEGLAB toolbox forThe
Matlab,applygrand-mean
waves for th
test and a Mann-Whitney U test (t(35) =ing bandpass filtering (0.3 - 30Hz), re-referencing to the average of the mastoids,
standard
anddeviant
for each of th.174, pocular= .863)(Ucorrection,= 177, pas=well.918).
The
ICA-based
artifact
as pre-stimulus
baseline correction.
types:
vowel,
scoresthe post
test also
not generalized
For reading
the analysisof theatsingle
trial ERP
data, did
we usedadditive
mod- consonant and dur
separately
shown in Fig. 4, 5 andelingshow(GAM;
mgcv package
in R; Wood,2006). Subsequently,
wetested which
a significant
difference
betweenthebehavioural measures interact with the MMR to consonant, vowel and duration
deviants of speech sounds in a stepwise procedure. Starting with a basic model
including smooths over time per condition, we added a random effects structure
for participants, and subsequently tested all behavioural measures as main effects
and interactions.

1
0
−1
−2

LDN

T1CF_Total

MMN

Phonological Awareness (z-scores)

Difference
Consonant
Di

0

FIGURE 1. Conventional grand-average ERP analysis averaged over items andparticipants. Solid line: standard stimulus; dashed line: deviant stimulus; red line:
mismatch response; shaded area: confidence interval for mismatch response.

100

MM

918). The
so did not
tween thestandard and deviant for each of the deviant
types: vowel, consonant and duration, areseparately shown in Fig. 4, 5 and 6.
264

Brain-behaviour interactions with GAM

7

1

0

difference, excl. random

−1

0

−7

−2

T1CF_Total

LDN

Phonological Awareness (z-scores)

Difference
between CD and CS
ConsonantDiscrimination
MMR

0

100

200

MMN

Time

300

time

LDN

FIGURE 2. GAM predicted tensor smooth for the nonlinear interaction of theconsonant mismatch response (deviant subtracted from standard, color-coded)
and phonological awareness.

3

Results and Discussion

We found a significant nonlinear 3-way interaction of PA skills and condition(standard vs. deviant syllable) over time, when predicting single trial ERP amplitudes (see Figure 2). Interestingly, this interaction was only found for the
consonant deviancy condition (i.e. /ti/ and /pe/ changing to /pi/ and /te/, respectively), but not for the vowel or duration change. While we expected to findan interaction of MMN with PA skills, we saw that only the LDN is modulated by
PA skills. More specifically, only the LDN to consonant change was modulated
by PA skills, which is interestingly also the condition with the smallest MMN
response.
It is not yet fully understood what cognitive processes the early and late components of the MMR reflect. To some extent, this may be due to the fact thatthe vast majority of ERP studies rely on signal averaging over items, subjects, aswell as time points (i.e., when averaging over time windows instead of considerfor vowel at Fz scalping
location
before and after training period in the
single sampling points) when analyzing the MMR together with behaviouralstandard. ------ deviant, data.
gray lineis
MMN (the difference wave).
Generalized additive modeling has recently started to be used with EEG
data (e.g. Meulman et al., 2015) as it allows to consider single trial data, and tointegrate continuous behavioural predictors into the analysis in a way that had
not been possible before. This should allow researchers to further narrow down
the functions which certain ERP components 13
reflect.

Glatz et al.

265

Acknowledgments: We are very thankful to Martijn Wieling, Iris te Stroete,
Maaike Rietdijk, Jenny Aussum, and all the children who participated in ourstudy!
References
Leppänen, P. H., Hämälainen, J. A., Salminen, H. K., Eklund, K. M., Guttorm,
T. K., Lohvansuu, K., ... & Lyytinen, H. (2010). Newborn brain eventrelated potentials revealing atypical processing of sound frequency and thesubsequent association with later literacy skills in children with familial
dyslexia. Cortex, 46(10), 1362-1376.
Lovio, R., Halttunen, A., Lyytinen, H., Näätänen, R., & Kujala, T. (2012). Reading skill and neural processing accuracy improvement after a 3-hour intervention in preschoolers with difficulties in reading-related skills. Brain
research, 1448, 42-55.
Meulman, N., Wieling, M., Sprenger, S. A., Stowe, L. A., & Schmid, M. S. (2015).
Age effects in L2 grammar processing as revealed by ERPs and how (not)to study them. PloS one, 10(12), e0143328.
Näätänen, R., Paavilainen, P., Rinne, T., & Alho, K. (2007). The mismatch negativity (MMN) in basic research of central auditory processing: a review.
Clinical Neurophysiology, 118(12), 2544-2590.
Wood, S.N. (2006). Generalized Additive Models: An Introduction with R. Chapman and Hall/CRC.

Interval Shrinkage Estimation Parameter ofBurr type III Distribution
Parviz Nasiri
1

Department of Statistics, University of Payam Noor, 19395-4697 Tehran, I.R.
Iran

Abstract: In this article, the Burr type III distribution is defined and for unknown parameter, different point estimates are derived. In shrinkage estimation,
we believe that the parameter belong to a finite interval. So we propose for suchsituations an interval shrinkage approach which combines in a coherent way anunbiased conventional estimator and non-sample information about the rangeof plausible parameter values. At the end, we conclude those interval shrinkage
estimators are better than the maximum likelihood estimation.
Keywords: Maximum likelihood estimation, Interval information, Shrinkage estimation, Means square error

1Introduction

Burr in 1942 introduced a family of the twelve cumulative distribution functions
for modeling lifetime data. In 2011, Asgharzadeh and Fallah done estimationand prediction for exponential family of distributions based on records. The two
important members of the family are Burr type III and XII. Burr type III distribution allows for a wider region for skewness and kurtosis plane, which covers
several distributions including the log-logistics, and the Weibull and Burr type
XII distributions. Let us consider the cumulative distribution function, probability density function of the Burr type III distribution are given respectivelyby,F (x; θ, c) = (1 − x−c )−θ
−(c+1)

f (x; θ, c) = θcx,x>0−c −(θ+1)(1 − x

)

,

θ>0
,

,

x>0

c>0

(1)
(2)

where the parameters c > 0 and θ > 0 are the shape parameters of the distribution. Our approach is closely related to the interval shrinkage estimators. The
idea of shrinkage is providing a balanced trade-off between a conventional estimator and a shrinkage target (see Pandey, B. N. (1983)). Recently Nasiri (2016)
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Parviz267

introduces Interval shrinkage estimators for location parameter of the exponential
distribution.
The rest of the paper is organized as follows. In section 2 and 3, we deal with the
maximum likelihood and feasible interval shrinkage estimation of parameter. In
section 4, we compare the Bias and MSE of the estimates empirically.

2

Maximum Likelihood Estimation

Let X1 , X2 , . . . , Xn be a random sample of size n of the Burr type III distribution.
The log-likelihood function for based on a random sample isl(θ) = n log(θ) − (θ + 1)nXlog(1 + x−ci ) + n log(c) + (c + 1)

i=1

n
X

log(xi )(3)

i=1

The maximum likelihood estimate of the unknown parameter is obtained by maximizing the log-likelihood function l(θ) with respect to θ. The likelihood equation
which is obtained from the derivatives of l(θ) with respect to the parameter,
become
n
∂l(θ)
n X
= −
log(1 + x−c
i )
∂θ
θ
i=1

(4)

The maximum likelihood estimator θ̂ of θ can be obtained by solving the likelihood equation ∂l(θ)= 0, so∂θθ̂ = Pnn(5)log(1 + x−c
i )

i=1

3

Feasible interval shrinkage estimator

Let (X1 , X2 , . . . , Xn ) be the random sample of size n taken form Burr type III
distribution. Proposed shrinkage estimator and its properties following Thompson(1968), a shrinkage estimator for the parameter θ and θg , a guess values of θ isavailable, is defined asθ̂sh = θg + ω(θ̂ − θg )(6)

A shrinkage factor is defined based on guessed value. Let θ̂sh be the shrinkage
estimation of scale parameter. Thenθ̂sh = θg + ω(θ̂ − θg )To find we have to consider MSE of estimator as:
M SE(θ̂sh ) = {(θ̂sh − θ)2 }= ω 2 M SE(θ̂, θ) + (ω − 1)2 (θ − θg )2 + 2ω(ω − 1)(θ − θg )E(θ̂ − θ)268Interval Shrinkage Estimation Parameter

Now, we have to minimize the (θ̂sh ),dM SE(θ̂sh )=0dω(θg − θ)2 + (θg − θ)E(θ̇ − θ)ω=
M SE(θ̂ − θ) + (θg − θ)2 + 2(θg − θ)E(θ̂ − θ)(7)

In 2011, Golosony and Liesenfeld show the shrinkage estimator towards the interval θ ∈ [θ0 , θ1 ] ⊂ Θ for unbiased conventional sample estimator of θ̂ withE(θ̂) = θ is given by





q
h
i
V (θ̂)
V (θ̂)+(θ1 −θ)2
θ1 −θ
θ0 −θ
θ̂
√
√
θ̃sh =θ̂+ V (θ̂) · θ1θ−
arctan−arctan+ln−θ02(θ1 −θ0 )V (θ̂)+(θ −θ)2
V (θ̂)

V (θ̂)0andE(θ̃sh ) = θ̂ +"#V (θ̂) + (θ1 − θ̂)2
V (θ̂)ln2(θ1 − θ0 )V (θ̂) + (θ0 − θ̂)2

(8)

For E(θ̂) = θ, we have"#
V (θ̂) + (θ1 − θ̂)2
V (θ̂)lnθ̃sh = θ̂ +2(θ1 − θ0 )V (θ̂) + (θ0 − θ̂)2

(9)

The estimator obtained in (9) is not linear in θ̂ and becomes identical to theconventional estimator θ̂ as the V (θ̂) tends to zero or as (θ1 − θ0 ) increases. It
also tends to θ̂ as θ̂ moves away from the interval [θ0 , θ1 ]. Furthermore if θ̂ isequal to the midpoint of the interval θm = (θ0 − θ1 )/2, the estimator θ̃sh isequal to θ̂. The exact stochastic properties of the feasible estimator θ̂, including
its means and variance, are hard to derive analytically since it is a complex
nonlinear function in θ̂. Golosony and Liesenfeld (2011) suggest to approximateθ̃sh by a first-order Taylor expansion around the center of the shrinkage interval
θm . The second derivative of θ̃sh equals zero at the point θm , so the first-order
Taylor expansion in this case is equivalent to second-order one. Since θ̃sh has an
inflection point at the midpoint of the interval θ̂ = θm , this expansion provides
a reasonable approximation to θ̃sh within the shrinkage interval. So it defined analternative useful feasible interval shrinkage estimator with mean and variancethat can be derived in a closed form. Let the half-length of the shrinkage interval
be denoted by θd = (θ1 − θ0 )/2. The second-order Taylor approximation of theestimator at the point θm is linear in θ̂ and is given by"˜θ̃sh = θ̂ 1 −#V (θ̂)v(θ̂) + θd2+ θm

V (θ̂)
V (θ̂) + θd2

θ

(10)

With mean and variance as˜E(θ̃sh ) = θ − (θ − θm )V (θ̂)
V (θ̂) + θd2

˜θ , V (θ̃sh ) = V (θ̂) 1 −˜
It can be shown that M SE(θ̃sh ) ≤ V (θ̂).

V (θ̂)
V (θ̂) + θd2!2

Parviz

4269

Numerical Study

To have some idea about mean square error (MSE) of maximum likelihood (θ̂mle )˘and shrinkage interval (θ̆) estimation, we perform sampling experiments using R
software and the results are shown in Tables 1 to Table 3. The MSE of both
estimators decrease when sample size increase. Comparing the MSE of two estimators, states that the interval shrinkage estimator works better than maximum
likelihood estimator.

References
Asgharzadeh, A. and Fallah, A. (2011). Estimation and prediction for exponentiated family of distributions based on records. Communication in statistics, theory and methods, 40: 68-83.
Burr, I. W. (1942). Cumulative frequency distribution, Annals of Mathematical
Statistics, 13, pp. 214-232.
Nasiri, P. (2016). Interval shrinkage estimators for location parameter of the exponential distribution. Research Journal of Applied Sciences, 5(11), 229231.
Pandey, B. N. (1983). Shrinkage estimation of the exponential scale parameter.
IEEE Trans. Reliability 32.
Thompson, J. R. (1968). Some shrinkage techniques for estimating the mean. J.
Amer. States. Assoc. 63, 113-123.

Fitting binomial mixtures to RNA-seq data
to find discrete gene regulation mechanisms
Adriaan van der Graaf1
1

Universitair Medisch Centrum Groningen, Netherlands

E-mail for correspondence: adriaan.vd.graaf@gmail.com
Abstract: Gene expression measurements are becoming more and more relevant
to detect how cellular processes operate. In this paper binomial mixtures are fit
to RNA-seq data, searching for certain discrete gene regulation mechanisms. My
results suggest that gene expression regulation is performed through a continuous
process and not by a small number of discrete ones. One other option could bethat the data used was of insufficient quality to provide accurate results.
Keywords: RNA-seq; binomial mixtures

1

Introduction

Next generation RNA sequencing (RNA-seq) is the current gold standard ofgene expression measurements. Unlike microarray gene expression measurements,
RNA-seq is able to detect novel transcripts. Counts of detected transcripts in
RNA-seq are a proxy of cellular gene expression, but because of the count natureof the data, statistical techniques that deal with microarray expression are notalways suitable because of their assumption of Gaussian distributed data.
Variance of gene expression can only be partially explained by an organisms genetics, Lloyd-Jones et al. (2017), implying that there are multiple factors driving
gene expression. Modelling possibly unknown factors that influence gene expression can be attempted through a mixture of expression distributions, each ofwhich could represent some biological process.
This short paper describes the steps taken to pre-process Geuvadis RNA-seq
data from Lappalainen et al. (2013), fitting a binomial mixture distribution using a custom EM algorithm and finally doing model selection to find the best
performing model and discussing biological implications.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

van der Graaf

2

Methods

2.1

Data

271

The Geuvadis mRNA-seq dataset from Lappalainen et al. (2014) was used. Thisconsisted of 660 human individuals of which RNA-seq data was available fromlymphoblastoid cell lines, the total sequencing depth had a median of 4.97 million
transcripts per individual. For every gene on the first chromosome, transcripts
were counted and used as input data for the EM algorithm.

2.2

EM algorithm

The rationale for modeling RNA-seq using the binomial distribution is the following: If there are ni RNA-seq transcripts detected in individual i, then, finding
a number ki RNA-seq transcripts from some gene, can be modeled through a binomial process, with some probability π. Where a success is considered as findinga transcript on the gene and a failure if the transcript is not detected on the gene.
The biological expectation in this study is that total expression can be considereda mixture of binomial distributions, each of which may represent some biologicalprocess.
The EM algorithm that is often used to estimate Gaussian mixtures can beadapted to fit other probability functions, in our case the binomial. I consider
some vector of i.i.d. transcripts k = (k1 , ..., kN ) for some gene, and a vector of
total transcripts for the individual n = (n1 , ..., nN ) where N is the number ofindividuals of which RNA-seq data is available. Analyzing each gene separately,
the log likelihood is maximized for a mixture of M ∈ {1, 2, ..., 50} binomial distributions.
!
M
N
X
X
ni
ki
l(π|p, k, n, ∆) =∆i,m ·· πmpm· (1 − πm )ni −ki
(1)
ki
m=1
i=1
Where k and n are N dimensional vectors with entries ki and ni respectively. π
and p areP an M dimensional vector with entries πm ∈ [0, 1] and pm respectively,
where M
m=1 pm = 1, represents the relative proportion of a distribution in the
mixture. ∆ is an N -by-M matrix with each element ∆i,m ∈ {0, 1}, indicating if
individual i is part of distribution m.
An EM algorithm for binomial mixtures was used to estimate the distribution parameters, the EM algorithm is initialized four times, and after convergence, the model with the maximum log likelihood is retained. Initialization
parameters were: πm ∼ U (ARGM INi (ki /ni ), ARGM AXi ( ki /ni )) and pm =1/M . In the E step, membership probability of an individual is determined m:
i ,ki ,π̂m )
γi,m = PMBin(n. In the M step, the maximum likelihood estimators
Bin(n ,k ,π̂ )i
m=1PN
i=1 γi,m

i

m

PN

γ

k

i=1 i,m i
are p̂m =for probability of membership and π̂m = PN
for theN
i=1 γi,m nibinomial proportion. The log likelihood and BIC is determined, by taking thehighest γi,m for each individual, and setting it’s respective ∆i,m = 1. The algorithm is repeated until there is less than a 10−10 difference between new and oldπ̂m parameters, or if the maximum number of iterations (1000) has been reached.

272

3

Binomial mixtures to fit RNA-seq

Results

Some genes showed non-convergence, which was mostly attributable to numericalprecision problems, when there were a low number of transcripts. If only genes
were considered median(k) > 10, 89.2 % of initializations reached convergence.
The BIC of four representative genes are shown in figure 1.
BIC criteria of four representative genes
35000

ENSG00000000457.8

G

30000
G

20000
40000
35000
30000
25000
20000
5e+05
4e+05
3e+05
2e+05
1e+05
0e+00

G

G G G G G G G G G G G G G G G G G G
G G G G G G G G G G G G G G G G G G G G G G G G G G G G G

G

G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G

GENSG00000000460.11

G

G

ENSG00000000938.7

BIC statistic

25000

G

G
G

G

G

G

G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G

ENSG00000001460.12

G

26000
24000

G

22000

G

G G
G G G G G G G G G G G G
G G G G G
G G G G G G
G G G G G G G G
G G G G G
G G G GG G G G G

0

10

20

30

40

50

Number of distributions fitted (m)geneIDG

ENSG00000000457.8

G

ENSG00000000460.11

G

ENSG00000000938.7

G

ENSG00000001460.12

FIGURE 1. The BIC statistics of a number of genes, over a different number of
mixture distributions. The Genes are identified by their ENSEMBL ID

Best fit distributions over median read counts

5
4
3
2

100
0

50

Frequency

150

log10(Median read count per gene)

200

Histogram of best fitting number of distributions

10

20

30

40

Best fitting number of distributions, per gene

50

10

20

30

40

50

Best fitting number of distributions, per gene

FIGURE 2. Left: A histogram of the optimal number of mixtures, based onminimized BIC. Right: A scatter plot of optimal number of mixtures compared
to median transcript count of the gene.
A histogram showing the best fitting number of mixtures is shown in figure 2

van der Graaf

273(left). This would indicate that most genes are under control of a small numberof regulatory processes.
However, when plotted against the median transript count of a gene, there is aclear correlation between the number of best fitting mixtures and the mediantranscript count of a gene (figure 2, right). This indicates that the accurate detection of binomial mixtures is dependent on the number of reads kj from agene.
Simulations were also done to validate the EM algorithm. Reads were simulated
according to convergence parameters from the algorithm, based on parameters
found in converging EM algorithms. In all cases, the algorithm converged to thesame solutions

4

Conclusion and discussion

This short paper details the steps taken to fit up to 50 different binomial mixtures
to RNA seq data. Fitted using an EM algorithm. Evidence for a small and discrete
number of distributions which regulate gene expression was not found. Perhaps
because of sample size issues, but it could also indicate that gene expression isregulated by more continuous mechanisms.
One thing to note is that RNA-seq data can be overdispersed, so using a binomial
distribution for modelling is not always correct, Sun (2011) therefore used thenegative binomial distribution for regression of expression, using an organisms
genotype as a dependent variable. Adapting this to the negative binomial could
provide more accurate results.

Acknowledgments: Special Thanks to Marco Grzegorczyk, for much neededguidance, in this unfamiliar field.
References
Lloyd-Jones, L. R. et al. (2017). The genetic architecture of gene expression in
peripheral blood American Journal of Human Genetics 100 228 – 237.
Lappalainen, T. et al. (2014). Transcriptome and genome sequencing uncovers
functional variation in humans Nature 501 506 – 511.
Sun, W (2011). A Statistical Framework for eQTL Mapping Using RNA-seq
Data Biometrics 68 1 – 11.

Modelling weekly temperatures in Eelde
using Bayesian Structural Time Series
Hung Chu 1
1

Johann Bernoulli Institute for Mathematics and Computer Science, the Netherlands

E-mail for correspondence: hung c@hotmail.com
Abstract: In this paper we study the weather temperature estimation performance of a Bayesian approach based on a structural time series model. The
approach makes use of Monte Carlo Markov Chain (MCMC) sampling to approximate the joint distribution of the model parameters. For our empirical analysis
we consider data with a strong seasonal pattern, namely weekly weather data.
Keywords: Bayesian Structural Time Series Model; MCMC; Estimation.

1

Introduction

Traditional methods to estimate time series data are the Multiple Linear Regression (MLR) and the autoregressive integrated moving-average (ARIMA) model.
We apply the Bayesian Structural Time Series (BSTS) model, which was introduced by Scott et al. (2013), for the estimation of weather temperature using
contemporaneous predictors. The BSTS allows for 1. decomposing the time series
data into several latent components that can describe the underlying dynamics
of the data, such as trend, seasonality and regression, 2. variable selection and 3.
Bayesian model averaging. The structural time series model of an observation yt
at time t is
yt = µt + γt + β T xt + εt ,µt = µt−1 + δt−1 + η1t
δt = δt−1 + η2tγt = −S−1
X(1)

γt−13s + η3t ,s=1

where εt ∼ N (0, σε2 ) for some positive constant σε2 , ηt ≡ (η1t , η2t , η3t )T ∼ N (0, Q)
for some diagonal variance matrix Q, xt a vector of contemporaneous or lagged
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Chu

275

predictors and S the number of seasons. Furthermore, each season is assumed
to have a duration of exactly 13 weeks. The model is decomposed into four
components: µt , γt , β T xt and εt respectively describe the general trend, seasonalpattern, regression effect and error. The model parameters are σε2 , β and Q.

1.1

Spike-and-slab regression

The BSTS model incorporates the spike-and-slab regression for variable selection.
Let K denote the number of regressors. For k = 1, . . . , K, let(
1 if βk 6= 0γk =,0 if βk = 0
and assume that the prior p(γ) is an independent Bernoulli:
γ∼

K
Y

γ

πkk (1 − πk )1−γk .

k=1Intuitively, πi is the probability of inclusion of βi .
Let βγ denote the subset of β for which βk 6= 0. The spike-and-slab prior on the
regression coefficients is then given byp(β, σε2 , γ) = p(βγ |γ, σε2 )p(σε2 |γ)p(γ).
See Scott et al. (2013) for the conditional priors p(σε2 |γ) and p(βγ |γ, σε2 ).

1.2

Conditional posterior distributions

Let y = (y1 , . . . , yT )T and α = (α1 , . . . , αT )T , where T is the size of the data.
Observe that the model in (1) can be represented as a state space time seriesmodel:
yt = Z T αt + εtαt = T αt−1 + Rηt−1 .
Therefore we can use the Kalman filter and smoother to obtain the posterior
distribution of the states given the data, i.e. p(α|y, θ), where θ is the set of model
parameters. However, we cannot just generate each αt from p(αt |y, θ), because
we have to take into account the autocorrelation between αt and αt+1 . For thiswe use the simulation smoother of Durbin et al. (2002) to generate samples fromp(α|y, θ).
The conditional posterior p(γ|y, α) is obtained by a Gibbs sampling algorithm,
where each γi is drawn conditional on every other element of γ. The conditional
posteriors p(Q|α), p(σε−2 |y, α, γ) and p(β|y, α, σε−2 , γ) are closed-form, from which
we can generate samples directly. See Scott et al. (2013) for the details.

276

2

Bayesian Structural Time Series

Model estimation

Let φ ≡ (θ, α)T and let ψ denote the set of parameters other than β and σε2 .
Considering an initial θ = θ(0) generated from the prior distributions, we obtaina stationary distribution p(φ|y) using MCMC as follows.
1. Simulate α from p(α|y, θ) using the simulation smoother.
2. Simulate ψ from p(ψ|y, α, β, σε2 ).
3. Simulate β and σε2 from p(β, σε2 |y, α, ψ).
By cycling through Steps 1 − 3 for M times, we obtain a sequence of MCMC
draws φ(1) , . . . , φ(M ) . The first m samples (also known as burn-in samples) may
not be representative for the target posterior distribution and hence will be discarded. The remaining sequence of MCMC draws is used to estimate the posterior
distribution of yt by means of (1) and the fit by:
ŷt =

3

M
X
1
(i)
ŷ .
M − m i=m+1 t

Empirical analysis

For the empirical analysis we use weekly weather data from Eelde (the Netherlands) from 20–12–2011 until 20–12–2016, provided by Koninklijk Nederlands
Meteorologisch Instituut (KNMI). We have yt denoting the average weekly temperature and xt containing nineteen candidate predictors, such as average windspeed, relative humidity, global radiation, but also last week’s temperature. Furthermore, we perform M = 5, 000 MCMC iterations, where the first m = 1, 000
samples are considered for burn-in.
2
TABLE 1. R2 and Radjof the fitted models.

Approach

BSTS

BSTSsub

MLR

MLRsub

ARIMA(1,1,1)

R2
2
Radj

0.93750.93230.93380.93190.9191
0.91270.91100.90880.76780.7669The set of predictors with inclusion probability greater than 0.500 are reference evapotranspiration (1.000), global radiation (0.999), last week’s temperature (0.995), minimum relative humidity (0.655) and average relative humidity(0.531). We compare the fit of the BSTS model with all nineteen predictors to
the BSTS model with the set of five predictors above (BSTSsub ), MLR with all
nineteen predictors, MLR with the set of five predictors above (MLRsub ) and the
ARIMA(1,1,1) model. The estimation performance of the models in terms of the2(adjusted) coefficient of determination (R2 and Radj
) can be found in Table 1.
The result in the table indicates that BSTS and even BSTSsub provide a better fit
than the traditional methods. The fitted temperature and individual components
by BSTS (with all predictors) are depicted in Figure 1. The true temperature isincluded in the first subplot (red dashed line).

Chu

277

FIGURE 1. Plot of the true (red dashed line) and fitted temperature by the
BSTS model as well as the individual fitted components (black solid lines).

4

Conclusion

Considering a dataset with a strong seasonal pattern, we empirically showed thatthe BSTS model (even with the selected subset of predictors) provides a better2
fit than the MLR and ARIMA(1,1,1) model, based on R2 and Radj
. The BSTS
model has the advantage that it allows to visualize and study the underlying
components of the time series data as well as variable selection. Moreover, thisstudy demonstrates that the BSTS model performs the variable selection and thefitting of weather temperature very well.

References
Durbin, J. and Koopman, S.J. (2002). A Simple and Efficient Simulation Smootherfor State Space Time Series Analysis). Biometrika, 89, 603 – 613.
Scott, S.L. and Varian, H.R. (2013). Bayesian Variable Selection for Nowcasting
Economic Time Series. NBER Working Paper 19567.

A comparison between GMLVQ and Random
Forests
Thomas Nijman1
1

Rijksuniversiteit Groningen, Netherlands

E-mail for correspondence: thomasrutger@gmail.com
Abstract: This paper presents a comparison of two classification algorithms,
the first one is an advanced prototype based classifier known as General Matrix
Learning Vector Quantization. Which is an extension of Learning vector Quantization and is a relatively new algorithm. The second algorithm is the Random
Forest algorithm which is a very popular classification method that has provento be a very reliable classifier. It was found that GMLVQ gave slightly better results than the Random Forest algorithm for the first two datasets. On the third
dataset however, the Random Forest algorithm performed much better (13 %of test examples were misclassified by GMLVQ as opposed to only 7 % by the
Random Forest algorithm).
Keywords: GMLVQ; Random Forest; Comparison.

1

Introduction

This section gives some theoretical background on GMLVQ and Random Forests.
As the Random Forest classifier is already well known, only a brief description is
given.

1.1

GMLVQ

GMLVQ (General Matrix LVQ) [Schneider et. al., 2009] is an extension of the
prototype based classification method Learning Vector Quantization, in which
a set of labeled prototypes are fitted to the data. A new data point with an
unknown class label is then assigned to the class of the closest prototype. Any
distance measure can be used, but usually Euclidean distance is used. GMLVQ
extends LVQ by using an advanced distance measure (eq. 1), using a matrix Λ ofadaptive relevances, depicting the relevance for each feature as well as correlations
between features.
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

NijmandΛ (ξ, w) = (ξ − w)T Λ(ξ − w)

279

(1)

The matrix Λ transforms each feature vector to an alternative feature space
which provides more discriminative power. Consider the closest 2 prototypes toa training sample (ξ, y) with c(wJ ) = y and c(wJ ) 6= y. The updating of the
prototypes is done by minimising a cost function:
EGM LV Q =

P
X
i=1

Λ
Φ(µΛ
i ) with µi =ΛdΛJ (ξi ) − dK (ξi )ΛdΛJ (ξi ) + dK (ξi )(2)

The model is trained by minimising EGLV Q with respect to the model parameters[Schneider et. al., 2009].
The diagonal entries of Λ reflect the importance of each feature, while off-diagonal
elements account for correlations between features. Feature vectors can be projected onto the 2 eigenvectors of Λ corresponding to the 2 largest eigenvectors of
Λ, Giving a 2D visualisation of the dataset. This makes a GMLVQ classifier easy
to interpret.

1.2

Random Forest

Random Forest [Breiman, 2001] is an ensemble classification method that builds
many decision trees and combines them to form a new classifier. Each decision
tree is trained on a bootstrap sample of the training data. It eliminates some of
the key issues of decision trees. In particular, decision trees tend to be non-robust
and may lack accuracy when compared to other approaches [Gareth et. al., 2015].
Classification can be done by outputting the class that is most frequently given
as output by the individual trees.

2

MethodComparing two classification methods is not easy as classification methods canhave many different parameters. Some parameters such as the learning rates for
GMLVQ have been heuristically chosen. The number of prototypes per class was
set to 1 for all the experiments.
For GMLVQ the intitial prototype learning rate used was 1 and the initial relevance Matrix learning rate used was 2. Furthermore, the learning rates are reduced by a factor of 1.5 after every training epoch (1 ”sweep” through the training
set). 100 training epochs were used, this seemed to be a sufficient amount of time
for the prototypes to converge in all the experiments without showing overfitting
effects.
For Random Forests, the number of trees used was 128. 128 was selected heuristically as it was found that using more trees did not do much for the classification
performance in the experiments. It has also been shown that the error does notdecrease linearly as the number of trees increases [Oshiro et. al., 2012].
10-fold cross validation was used for the experiments. Perhaps not a necessity,
but it could help in avoiding random splits that are beneficial for one classifier.

280

3

A comparison between GMLVQ and Random Forests

Results

The experiments were done using three different datasets. They were all obtained from the ”UCI Machine Learning Repository” [M. Lichman, 2013]. The firstdataset is the ”Wine” dataset. Which is a 2 class problem, containing 178 12
dimensional feature vectors. The second dataset is the ”Wisconsin Diagnostic
Breast Cancer” dataset. A 2 class problem with 569 30 dimensional feature vectors. The third dataset is the ”Image Segmentation” dataset. This is a 7 classproblem with 2310 19 dimensional feature vectors. The third feature has been
removed as it is constant over all the feature vectors to obtain 18 dimensionalfeature vectors.
The experiment described in ”Method” was repeated 10 times for all the datasetsfor both GMLVQ and Random Forests. The test errors over all the validation
runs were averaged and can be seen in Table 1. These numbers were obtainedby dividing the number of incorrect classifications by the total number of test
examples.

4

Discussion

When looking at Table 1 it can be seen that GMLVQ shows robust performance
when compared to Random Forests. The error for the first 2 datasets is relatively
low. It is even a little bit lower than the error for Random forests. When looking
at the Image Segmentation dataset, Random Forest does perform significantly
better, however. This problem is also somewhat harder due to the fact that ithas 7 classes. Performance here for GMLVQ might still be optimized by tweaking
the learning parameters. Especially increasing the number of prototypes might
increase classification performance as some classes may be spread out over a
subspace that is too large for 1 prototype.

5

Conclusion

In this short study it can be seen that GMLVQ is a suitable classifier for certain
problems. Rivaling Random Forests in the Wine and Breast Cancer datasets.
An advantage of GMLVQ is that it is sophisticated, yet easy to interpret. One
obvious downside is that there are plenty of parameters to tweak that may affectclassification performance. Random Forest clearly wins when it comes to the
Image Segmentation dataset, but little improvements could perhaps still be madeby tweaking the parameters of GMLVQ.
TABLE 1. Test Errors

GMLVQ
Random Forest

Wine

Breast Cancer

Image Segmentation

0.01070.01810.03090.04140.13050.0724

Nijman

References
Breiman, L. (2001).
Random Forests.
Public Library of Science
In: Machine Learning, 45(1), 5 – 32.
Gareth, J., Witten, D., Hastie, T. and Tibshirani, R. (2015).
An Introduction to Statistical Learning.
New York: Springer.
Lichman, M. (2013).
UCI Machine Learning Repository.
url: http://archive.ics.uci.edu/ml
University of California, Irvine, School of Information
and Computer Sciences
Oshiro T.M., Perez P.S., Baranauskas J.A. (2012).
How Many Trees in a Random Forest?
Machine Learning and Data Mining in Pattern Recognition:
8th International Conference, MLDM 2012, Berlin, Germany,
July 13-20, 2012. Proceedings.
Springer Berlin Heidelberg Berlin, Heidelberg, 154 – 168.
Schneider, P., Biehl, M. and Hammer, B. (December 2009).
Adaptive Relevance Matrices in Learning Vector Quantization.
MIT Press.
In: Neural Comput., 21(12), Cambridge, MA, USA, 3532 – 3561.

281

Polynomial and spline regression
D.A. Langbroek1
1

RUG, Groningen

E-mail for correspondence: davidlangbroek@hotmail.com
Abstract: In this article I compare polynomial regression and cubic spline regression methods on some functions. I summarise some advantages and disadvantagesof both regression methods, and give insight when to apply either method.
Keywords: Polynomial; Spline; Regression

1

Introduction

In this paper I take a look at polynomial regression and cubic spline regression.
Polynomial regression fits a function of the form ck+1 xk + ck xk−1 + · · · + c2 x + c1
to the data, where k is the degree and each ci to be determined constant. The
flexibility in the model comes from the degree. Spline regression fits a polynomial
as well but with the addition of dividing the range of X into regions, on which eachseparately polynomial regression takes place. Cubic spline regression specifies
that the to be fit spline polynomials have degree 3 and connect on the section
knots in a continuous and second order smooth manner. The flexibility of splines
is mostly generated by the number of regions. In this paper I first look at theRunge function, to illustrate some weaknesses of polynomial regression and how
splines can help. Then I will show in two cases that splines can be applied to morefunctions than polynomial regression. Finally I show some downsides of splines.

2

Statistical methods

All the figures, plots, table and simulation have been made in R with build inregression functions. The simulations have been made by taking 100 random
points uniform distribution (1000 for the cosine) over the domain of the underlying function as data points to fit the regression methods with. These points
are the same for all figures. The error in Figure 4 is created by adding a random
value between [−0.2, 0.2] taken form uniform distribution. The adjusted R2 values are from the corresponding built in function aswell. Table 1 has an overview
This paper was published as a part of the proceedings of the 32nd International Workshop on Statistical Modelling (IWSM), Johann Bernoulli Institute,
Rijksuniversiteit Groningen, Netherlands, 3–7 July 2017. The copyright remains
with the author(s). Permission to reproduce or extract any parts of this abstract
should be requested from the author(s).

Langbroek.

283

of indication for all figures how well each model fits. A sample R code can befound in the appendix.

3

Runge function

In this section I will use both polynomial and cubic spline regression methods on
1
the Runge function. The Runge function is defined as: y = 1+25x
2 . In Figure 1 I
have plotted the regression polynomials of degree 4,9,16 over the original Runge
function and cubic spline with 3 and 7 knots at respective data quantiles. The

FIGURE 1. Runge function polynomial/spline regression
Figure and Table 1 show that a more flexible polynomial leads to a better fit.
However, at the boundaries, the polynomials are extremely poor. The cubic splinewith 7 knots is more flexible then the 3 knots spline, and fits the function better.
Note that the left boundary is approximated well but that the right boundary
still has a bit of fluctuation. Indicating that splines can alleviate the boundary
problems, but not necessarily outright solve them.

4

Spline and polynomials fit

In this section I will show that a spline can approximate some functions thatpolynomial regression can not while not performing worse on polynomials. FirstI will approximate the polynomial: y = x6 + x5 − 3x4 − 3x3 − 2x2 + x with splineand polynomial regression. The original function is a sixth degree polynomial,
and as a result polynomial regression of degree equal or more then 6 give an (almost) perfect fit. In Figure 2 I have plotted the original function, and polynomial
and spline approximations. The cubic splines are capable of approximating thispolynomial as well and are better fits as indicated in Table 1. The other functionthat I approximate is the periodic cosine function on the domain x ∈ (−4π, π). In
this domain the cosine function achieves 9 extreme values. In Figure 3 the polynomials of degree 9 and 16 have been plotted over the cosine. I have not included
a figure where a cubic spline is fit, as the cubic spline with appropriately chosenknots (knots at multiples of π) fits the underlying function near perfect.

5

Weakness of splines

So far splines outperform polynomials. However splines are not necessarily better,and in the section I will show two explicit downsides of splines: knot placements

284

Regression methods

FIGURE 2. Polynomial degree 6 polynomial/spline regression

FIGURE 3. Cosine polynomial regression

and variance. Previously I have claimed that a cubic spline with appropriately
chosen knots can perfectly approximate a cosine function. However if the knots
are chosen at a different location or another amount of knots is chosen, then the
result is not necessarily as good a fit. In the following Figure 4 I have plotted a
cubic spline with 9 knots, positioned at multiples of 10% of the regression data
points. As well as a cubic spline with 7 knots, at the respective data quantiles.
Table1 indicates that both are worse fits than the polynomial of degree 16. To
indicate the variance when using spline I will try to approximate the Runge
function again, but this time the data points have an random error of up to±0.2. The results are shown in Figure 4, including the data points used for theapproximation. The spline with 20 knots is very flexible and wobbly, typical
overfit.

FIGURE 4. Cosine Spline regression/ Runge function with error regression

Langbroek.

285

TABLE 1. Overvieuw adjusted R2 error of fits

6

Fits of first sub-Figure

Fits of second sub-Figure

Plots

Fit 1

Fit 2

Fit 3

Fit 1

Fit 2

Figure 1

0.76550.95230.99850.93990.9963Figure 20.90810.92630.99130.9993Figure 30.25790.9991Figure 40.8841

0.99600.81560.8339Fit 3

0.8418Conclusion

Polynomial regression is a simple method to approximate functions. However atboundary values it may become poor, and increasing the degree does not always
improve the results. Moreover there are functions that can not be well approximated using polynomials. In all these cases, spline regression can yield better
results. All while not performing worse in situations that polynomial regression
suffices. It seems that in general spline performs better. However it also comeswith its own downsides. It is overly complex and has the tendency to overfit thedata. Also the position and number of knots matters for the results, but finding the proper values can be difficult. Lastly, the boundary problems can persist
when using splines.

Acknowledgments: Special Thanks to dr. M.A. Grzegorczyk for his feedbackon the writing of this paper

References
Hastie, T., James, G., Tibshirani, R., and Witten, D. (1994). An Introduction to
Statistical Learning, with apllication in R. New York: Springer.

IndexÖtting, Marius, 243

Coens, Corneel, 139
Colebank, Mitchel, 258
Collet, Jérôme, 107
Colosimo, Enrico A., 79
Cordeiro, Gauss M., 150
Cortese, Giuliana, 193
Cysneiros, Francisco J. A., 135

Adam, Timo, 181
Adelfio, Giada, 127
Aerts, M., 43
Almeida, Frederico M., 79
Araveeporn, Autcha, 55
Arnold, Mark, 162
Ayma, Diego, 239
Azevedo, Carla, 162

de Carvalho, Tiago M., 95
de Souza, Daiane, 185
del Pilar Diaz, Maria, 123
Demétrio, Clarice G. B., 67
Desousa, Mário F., 227
Deutscher, Christian, 243
Di Caterina, Claudia, 193
Donneau, Anne-Françoise, 139
Duffy, Stephen, 95
Durbán, Marı́a, 239
Duyck, Joke, 154Böhning, Dankmar, 162
Baayen, R. Harald, 21Banditvilai, Somsri, 169
Barboza, Daniel, 99
Barriga, Gladys D.C., 150
Bauer, Alexander, 146Baz, Jorge L., 59
Bell, Andrew, 207
Benetazzo, Lucia, 247
Bernardi, Mauro, 189
Bernel, Victor, 219
Bischoff, Rainer, 219
Blejec, A., 201
Bogaerts, Kris, 10
Bottomley, Andrew, 139
Bourguignon, Marcelo, 51
Brandsma, Corry A., 219
Brini, A., 235
Brown, Jennifer, 83
Bruyndonckx, R., 43
Bruyneel, Luk, 215
Burke, Kevin, 111

Eilertson, Kirsten, 197
Erlis, Ruli, 189
Espejo, Vı́ctor, 166
Faisal, Shahla, 177
Faiz, Alen, 219
Fennell, Susan, 111
Ferrari, Matthew, 197
Fricks, John, 197
Gabriel, Alice-Agnes, 146
Geraci, Marco, 115
Glatz, Toivo, 262
Gleeson, James P., 111
Goeman, Jelle, 7
Gruden, K., 201
Grzegorczyk, Marco, 219
Guryev, Victor, 219

Cai, Bo, 115
Camarda, Carlo G., 75
Cancho, Vincente G., 150
Cancho, Vincente Garibay, 185
Cerqueira, Pedro H.R., 91
Chen, Charles, 39
Chen, Mason, 35, 39
Chu, Hung, 274
Chubuathong, Chammaree, 169
Claeskens, Gerda, 47

Haider, Mansoor A., 258
Hens, N., 43
Hill, Nicholas A., 258
Hinde, John, 67
Horvatovich, Peter, 219

287

288

INDEX

Hurlin, Christophe, 107
Husmeier, Dirk, 119, 258
Iannario, Maria, 103
Ishihara, Takuma, 254
Küchenhoff, Helmut, 146
Kenne Pagui, Euloge Clovis, 203
Klattan, Michael W., 150
Komárek, Arnošt, 10
Langbroek, D.A., 282
Langrock, Roland, 3, 181, 243
Lee, Dae-Jin, 239
Leiva, Vı́ctor, 227
Lemonte, Arthur J., 59
Leos-Barajas, Vianey, 181
Lesaffre, Emmanuel, 10, 139, 154,
166, 215
Lima Neto, Eufrásio, 63
Luati, Alessandra, 119
Maassen, Ben, 262
Maia, Rafael, 99
Malá, Ivana, 131
Manghi, Roberto F., 135
Marques da Silva Junior, Antonio
Hermes, 71
Maruotti, Antonello, 162
Mauer, Murielle, 139
Maurits, Natasha, 262
Mayrink, Vincius D., 79
McDonald, Trent, 83
McGregor, Duncan, 207
Michielon, Edoardo, 203
Miles, Jonathan, 95
Mohammadi, Abdolreza, 211
Moral, Rafael A., 67

Pasetto, Michela Eugenia, 119
Pashayan, Nora, 95
Paula, Gilberto A., 135
Paun, L. Mihaela, 258
Pereira, Marcos Antonio Alves, 142
Pescim, Rodrigo R., 91
Petek, M., 201
Pinheiro, Aluı́sio, 63
Pinheiro, Hildete, 99
Pircalabelu, Eugen, 47
Quayle, Michael, 111
Quintero, Adrian, 215
Qureshi, M. Umar, 258
Ramšak, Ž., 201
Ramires, Thiago G., 91
Regis, M., 235
Richard, Michael, 107
Rigby, Robert A., 91
Robertson, Blair, 83
Rodrigues, Josemar, 185
Russo, Cibele Maria, 139, 142
Salvan, Alessandra, 203
Sangalli, Laura M., 4
Santos-Neto, Manoel, 227
Sartori, Nicola, 193, 203
Saulo, Helton, 227
Scheipl, Fabian, 146
Sering, Tino, 21
Shaoul, Cyrus, 21
Simone, Rosaria, 103
Solari, Aldo, 7Sottile, Gianluca, 127
Stasinopoulos, Dimitrios M., 91
Stolpmann, Jördis, 158

Nadia Dardenne, 139
Nakamura, Luiz R., 91
Nasiri, Parviz, 266
Nijman, Thomas, 278
Noè, Umberto, 119
Nooraee, N., 235

Tahata, Kouji, 231
Takahashi, Kanae, 251
Timens, Wim, 219
Tops, Wim, 262
Tough, Fraser, 87
Tran, Trung Dung, 154Tutz, Gerhard, 103, 177

Olufsen, Mette S., 258
Orindi, Benedict, 215
Ortega, Edwin M. M., 150
Osella, Alberto, 123

van den Berge, Maarten, 219
van den Heuvel, E. R., 235
van den Hout, Ardo, 95
van der Graaf, Adriaan, 270

INDEX
Vaz de Arruda Silveira, Liciana,
123
Vens, Maren, 158
Verbeke, Geert, 154, 215
Villegas, Cristian, 166

Wijayatunga, Priyantha, 223
Wit, Ernst C., 211, 247
Worton, Bruce J., 207

Waldorp, Lourens J., 47

Zagorščak, M., 201

Yamamoto, Kouji, 251, 254

289

International Workshop on Statistical Mo
32rd edition3 – 7 July 2017
Groningen, The Netherlands
is sponsored by

We are very grateful to the following organisations for sponsoring the32nd IWSM 2017:

International
Workshop• The Statistical
Modelling
Society on Statistical Modelling

32rd edition
International Workshop on Statistical M
3 – 7 July 2017
32rd edition
Groningen, The Netherlands
NWO Star Cluster

3 – 7 July 2017
• Rijksuniversiteit Groningen
Groningen, Theon
Netherlands
International
Workshop
Statistiis(RUG)
sponsored by
NWO Star Cluster

32rd
editionis sponsored by
3 – 7 July 2017
• Johann Bernoulli Institute, RUG International Workshop on Statistical Mod
32rd The
editionNetherl
Groningen,
3 – 7 July 2017
Groningen, The Netherlands

is sponsored by• Netherlands Organisation for Scientific Research (NWO), (NWO sponsors our Open Access Session)NWO Star Cluster

is sponsored by• STAR clusterronde, NWO
NWO Star Cluster• Koninklijke Nederlandse Akademie van Wetenschappen
NWO Star Cluster• Toyota Motor CorporationNWO Star Cluster

to be continued

NWO Star Cluster

32rd edition3 – 7 July 2017
Groningen, The Netherlands
292

INDEX

is sponsored by• C.R. Rao Stichting, Groningen
NWO Star Cluster

ational Workshop on Statistical Modelling• Center for Language and Cognition, RUG
32rd edition3 – 7 July 2017
Groningen, The Netherlands
is sponsoredby and Cognitive Neurosciences, RUG• Behavioural

NWO Star Clu

NWO
Star Cluster• R StudioInternational Workshop on Statistical Modelling
32rd edition3 – 7 July 2017
Groningen, The Netherlands
is sponsored by• Statistical Analysis Software (SAS)• The welcome reception
is sponsored by:
NWO Star Cluster– The Province of Groningen– The Muncipality of Groningen

NWO Star Cluster

NWO Star Cluster– The Rijksuniversiteit Groningen (RUG)