# Tutorial: Modern Regression Techniques for HCI Researchers

Martin Schmettow

► To cite this version:

Martin Schmettow. Tutorial: Modern Regression Techniques for HCI Researchers. 15th Human-Computer Interaction (INTERACT), Sep 2015, Bamberg, Germany. pp.651-654, 10.1007/978-3-319-22723-8_86. hal-01610866

HAL Id: hal-01610866

https://inria.hal.science/hal-01610866

Submitted on 5 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Tutorial: Modern regression techniques for HCI researchers

Martin Schmettow

m.schmettow@utwente.nl

University of Twente

## 1 Objectives

Despite a century of progress in statistics since the introduction of ANOVA and Pearson correlation, many researchers are still squeezing their precious data into the tight corset of those dated statistical models. This is particularly limiting in applied disciplines such as HCI, where impact factors can be numerous, heterogeneous and difficult to control experimentally. The proposed tutorial aims at liberating applied researchers from constraints and concerns associated with legacy statistics. After the tutorial, attendees will be able to:

1. build *general linear models*, with multiple categorical and metric predictors

2. model and interpret *interaction effects*

3. identify the optimal set of predictors through *model selection*

4. analyze data from complex research designs using *mixed-effects models*

5. use all above-mentioned techniques to conceive *advanced research models*

6. use all above-mentioned techniques to create more *efficient research designs*

## 2 Content

The tutorial spans one day, divided into four sessions of 90 minutes. All start with a motivating example, then explain the technique and demonstrate its use in R (or SPSS, if desired). In all sessions, attendees have opportunity to work on exercises.

### 2.1 Session 1: From classic to general linear models (GLM)

Attendees are picked up where they most likely stand: outcome variables are continuous and predictors are either metric or categorical. After briefly rehearsing ANOVA and regression, the *general linear models (GLM)* framework is introduced, which allows to free combinations of any number of categorical and metric predictors.

### 2.2 Session 2: Interaction and selection

With multiple predictors, the outcome variable sometimes shows non-linear trends, which often can conveniently be modeled as *interaction effects*. First, interactions between factors and metric predictors are introduced. Then, common interaction pat-
terns, such as ceiling effects and amplifiers are explained by examples. Finally, we advance to hypothesis forming with interaction effects. In view of the gain in flexibility, the Akaike Information Criterion (AIC) is introduced as alternative to F-tests.

## 2.3 Session 3: Basic mixed-effects models

In experimental or observational research, repeated measures often are preferable with regard to efficiency and data richness. But, neither classic, nor general linear models can deal with repeated measures. (Tweaks exist, but these are limited in statistical power and flexibility.) *Linear mixed-effects models* as the modern way to deal with repeated measures, efficiently and gracefully (Gueorguieva & Krystal, 2004).

## 2.4 Session 4: Advanced mixed-effects models

The final session unleashes the power of mixed-effects models by introducing hierarchical and cross-classified random effects. By example of Molich et al. (2010), it is shown how to deal with complex sampling schemes. Finally, a paradigm for quantitative design research is introduced, that uses samples of designs (Monk, 2004).

# 3 Intended audience

The tutorial is primarily targeted at HCI researchers who are conducting or planning observational and experimental studies. Knowledge of legacy parametric statistical methods is required, such as linear regression and ANOVA. The first part of the tutorial is a rehearsal, bringing all attendees on the same level before proceeding to more advanced techniques. Familiarity with SPSS and/or R is assumed.

4 Suggested readings

For preparation, attendees may best pick their favorite textbook on statistics and review the respective chapters on ANOVA and linear regression. An introduction to everything covered in the tutorial (and beyond) is Gelman & Hill (2007). It is recommended to also take a few hours to get acquainted with the basics of R and Rstudio (www.rstudio.com).

5 Background of the tutor

The instructor, Dr. Martin Schmettow, is assistant professor in psychology at the University of Twente (The Netherlands). He has more than a ten year experience in HCI research and education. At his home university, he gives a number of well received courses in Psychology bachelor and master education. Besides his background in Psychology and HCI, he is an expert in modern statistics, including modern regression techniques, exploratory techniques, maximum likelihood estimation, psychomet-
ric methods and Bayesian analysis. He has used advanced regression techniques in
several publications (Hund, Schmettow, & Noordzij, 2012; Schmettow, Bach, &
Scapin, 2014; Schmettow, Noordzij, & Mundt, 2013; Schmettow & Havinga, 2013),
but is also known for his mathematical contributions to the problem of sample size
estimation in usability testing studies (Schmettow, 2012).

For further background information, visit the instructors page on ResearchGate
(https://www.researchgate.net/profile/Martin_Schmettow), as well as the tutorial
handout (http://rpubs.com/schmettow/regression_in_r).

6 Pedagogical concept

Generally, the tutorial presents modern statistical techniques in a non-intimidating way. Focus is on understanding practical implications of the techniques and train applications in R (or SPSS). Mathematical formalism is avoided as much as possible. The tutorial consists of four sessions that are structured as follows:

1. Every session starts with a mini lecture that first describes a research problem where a particular limitation of classic statistical techniques exists, for example: *How can one simultaneously compare the impact of age and level of education (factor) on web browsing performance?*

2. Then, the problem is analyzed, and a new technique is introduced, for example: *Factors and metric predictors can be mixed, by capitalizing on sum of squares and by introducing dummy variables.*

3. Third, it is demonstrated by an example, how the technique is used in R, for example: *General linear models can be estimated using lm command with a formula expressions. User don't have to deal with dummy variables themselves.*

4. Fourth, attendees will work in pairs, applying the newly learned technique to a given research problem, for example: *Examine the combined effects of sleep deprivation and noise on performance (data set: Sleep).*

The proposed tutorial is based on a lecture unit of the Psychology master course *Research methods in Human Factors and Engineering Psychology*. In January 2013, a compact version has been given to PhD students at the University of Twente. In December 2013, a one-day workshop has been given to PhD students and post docs at the University of Würzburg, Germany (host: Prof. Dr. Jörn Hurtienne).

7 Material and required resources

Attendees receive the following material:

1. Handout containing more than 100 crafted presentation slides

2. Online tutorial for regression analysis in R

3. Written tutorial, explaining linear mixed-effects models by the example of four case studies (23 pages), SPSS instructions included.

4. Collection of ten real and synthetic data sets
5. Annotated list of recommended books

6. List of over 50 references for use in publications

It is assumed that attendees bring a notebook with R and Rstudio installed. Those, who want a SPSS demonstration alongside, need SPSS installed (version 19 upwards). The tutorial uses a very small subset of the R programming language. Basic programming skills are beneficial.

## 8 References

Gelman, A., & Hill, J. (2007). *Data analysis using regression and multilevel/hierarchical models*. New York: Cambridge University Press.

Retrieved from

http://www.imamu.edu.sa/Scientific_selections/abstracts/Math/Data Analysis

Using Regression and Multilevel Hierarchical Models.pdf

Gueorguieva, R., & Krystal, J. H. (2004). *Move Over ANOVA. Archives of General Psychiatry*, *61*, 310–317.

Hund, A. M., Schmettow, M., & Noordzij, M. L. (2012). The impact of culture and recipient perspective on direction giving in the service of wayfinding. *Journal of Environmental Psychology*, *32*(4), 327–336. doi:10.1016/j.jenvp.2012.05.007

Molich, R., Chattratichart, J., Hinkle, V., Jensen, J. J., Kirakowski, J., Sauro, J., ... Traynor, B. (2010). Rent a Car in Just 0, 60, 240 or 1,217 Seconds? –

Comparative Usability Measurement, CUE-8. *Journal of Usability Studies*,

6(1), 8–24. Retrieved from http://eprints.kingston.ac.uk/16473/

Monk, A. (2004). The Product as a Fixed-Effect Fallacy. *Human-Computer Interaction*, *19*(4), 371–375. doi:10.1207/s15327051hci1904_6

Schmettow, M. (2012). Sample size in usability studies. *Communications of the ACM*, *55*(4), 64. doi:10.1145/2133806.2133824

Schmettow, M., Bach, C., & Scapin, D. (2014). Optimizing Usability Studies by Complementary Evaluation Methods. In *Proceedings of the 28th British HCI conference*. Southport, UK.: BCS Learning and Development Ltd.

Schmettow, M., & Havinga, J. (2013). Are users more diverse than designs? Testing and extending a 25 years old claim. In S. Love, K. Hone, & Tom McEwan (Eds.), *Proceedings of BCS HCI 2013- The Internet of Things XXVII*. Uxbridge, UK: BCS Learning and Development Ltd.

Schmettow, M., Noordzij, M. L., & Mundt, M. (2013). An implicit test of UX: Individuals Differ in What They Associate with Computers. In *CHI '13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA '13*

(pp. 2039–2048). New York, New York, USA: ACM Press.

doi:10.1145/2468356.2468722