# ManyBabies 5: A large-scale investigation of the proposed shift from familiarity preference to novelty preference in infant looking time Pre-data collection manuscript for peer-review The ManyBabies 5 Team

Jessica Elizabeth Kosie, Martin Zettersten, Rana Abu-Zhaya, Dima Amso, Mireille Babineau, Heidi Baumgartner, Marina Bazhydai, Margherita Belia, Silvia Benavides-Varela, Christina Bergmann, et al.

► To cite this version:

Jessica Elizabeth Kosie, Martin Zettersten, Rana Abu-Zhaya, Dima Amso, Mireille Babineau, et al.  
ManyBabies 5: A large-scale investigation of the proposed shift from familiarity preference to novelty  
preference in infant looking time Pre-data collection manuscript for peer-review The ManyBabies 5  
Team. 2024. [hal-04383754](https://hal.science/hal-04383754v1)

HAL Id: hal-04383754

https://hal.science/hal-04383754v1

Preprint submitted on 9 Jan 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# ManyBabies 5: A large-scale investigation of the proposed shift from familiarity preference to novelty preference in infant looking time

Pre-data collection manuscript for peer-review

The ManyBabies 5 Team
**Abstract**

Much of our basic understanding of cognitive and social processes in infancy relies on measures of looking time, and specifically on infants’ visual preference for a novel or familiar stimulus. However, despite being the foundation of many behavioral tasks in infant research, the determinants of infants’ visual preferences are poorly understood, and differences in the expression of preferences can be difficult to interpret. In this large-scale study, we test predictions from the Hunter and Ames model of infants’ visual preferences.¹ We investigate the effects of three factors predicted by this model to determine infants’ preference for novel versus familiar stimuli: age, stimulus familiarity, and stimulus complexity. Drawing from a large and diverse sample of infant participants (N = XX), this study will provide crucial empirical evidence for a robust and generalizable model of infant visual preferences, leading to a more solid theoretical foundation for understanding the mechanisms that underlie infants’ responses in common behavioral paradigms. Moreover, our findings will guide future studies that rely on infants’ visual preferences to measure cognitive and social processes.

**Keywords:** looking time methods; familiarity preference; novelty preference
**Introduction**

Scholars and parents alike have long been fascinated by questions concerning the infant mind. What are infants thinking, seeing, hearing, and feeling, and from where and when do these abilities arise? Because it is not possible to query infants directly, scientists must rely on indirect techniques to study early development. However, many of the measures and procedures used with older children and adults, such as button presses or responding to questions, are difficult or impossible to use with infants. This has led researchers to adopt an indirect behavioral measure – infants’ looking toward visual stimuli – as one of the primary measures for uncovering the development of psychological processes in infancy.²

Much of the prior research using infants’ looking time is based on the seminal finding that infants, following repeated exposure to a particular stimulus, typically demonstrate a preference to look at a relatively new stimulus.³ This novelty preference has been well-documented for both visual and auditory stimuli.⁴⁻⁷ The approach in these looking time studies is often to present one or more stimuli until infants’ interest in the familiar stimuli is reduced, thereby inducing preferences for novelty.⁸ However, infants’ responses, as well as scientists’ interpretations of these responses, vary substantially.² Although many studies have found that infants show a robust novelty preference⁹⁻¹¹, there are also conditions under which infants prefer familiar stimuli over novel stimuli⁴,¹²⁻¹⁵. Even within specific research areas, there are usually examples of both directions of preference, and there are no clear reasons as to why ⁶,¹⁶.

Furthermore, in some cases, the same type of stimulus that draws infants’ attention in one paradigm can fail to draw attention in another.¹⁷,¹⁸

Despite this variability, scientists have generally focused on novelty preferences as a basis for making inferences about infants’ cognitive processing. This idea is consistent with
arguments from multiple disciplines suggesting that stimulus novelty may serve as a key signal for exploration and learning.¹⁹ That is, the novelty of a stimulus may engage intrinsic motivation mechanisms that, in turn, drive organisms to invest cognitive resources in learning under conditions of uncertainty.²⁰ Preferences for novelty have been identified in numerous diverse species²¹ and have been linked with both basic survival processes like locomotor adaptation²² and high-level constructs like curiosity²³⁻²⁶. Understanding the determinants of infants’ familiarity and novelty preferences, therefore, is important not only for methodological reasons but also for clarifying mechanisms of cognitive development.

The most influential account of looking preferences in the human infant literature is the conceptual model put forth by Hunter and Ames (henceforth, the Hunter and Ames model)¹, which was based on several studies.⁴,²⁷,²⁸ Hunter and Ames aimed to account for the circumstances under which infants show familiarity vs. novelty preferences while processing information presented in stimuli. The model assumes that when one stimulus is repeatedly presented, infants gradually develop a preference for this familiar stimulus compared to a second, novel stimulus. After continued exposure to the familiar stimulus, infants’ preferences eventually switch, and they begin to show a preference for the novel stimulus (see Figure 1). The key premise of the model is that the degree to which infants prefer familiar vs. novel stimuli depends on three main factors and their interactions (see Figure 1):

(1) **Familiarization time.** With increasing familiarization to a stimulus, infants transition from preferring a familiar stimulus to preferring a novel stimulus.

(2) **Infant age.** The transition from preferring a familiar stimulus to preferring a novel stimulus is faster for older infants than younger infants.
**(3) Task difficulty.** As task difficulty increases (operationalized in the current study as **stimulus complexity**), infants make a slower transition from preferring a familiar stimulus to preferring a novel stimulus.

At the time of writing, this influential model has been used to explain findings within a broad range of papers examining visual attention, speech segmentation, object categorization, face recognition, quantification, memory, knowledge of the physical world, and social cognition, among others.<sup>5,12,29–31</sup>

<figure><img src="image_2.png" /><figcaption>*Figure 1.* This figure (adapted from Bergmann and Cristia<sup>6</sup>) depicts Hunter & Ames' model<sup>1</sup> of infant looking to familiar (plotted up) and novel (plotted down) stimuli at different levels of pre-test familiarization time (shown on the x-axis). The dashed line indicates equal preference for familiar and novel stimuli. The grey line represents the model's prediction for older infants, and the black line for younger infants. The “lower complexity” and “higher complexity” arrows indicate how the proposed relationship between age and familiarization time might shift with variation in stimulus complexity (i.e., the current operationalization of task difficulty).</figcaption></figure>
However, this model has not undergone a rigorous and comprehensive test, and the predictions from the model have not been supported unequivocally by the existing literature. Whereas several studies have demonstrated the proposed shift from familiarity to novelty preferences³²⁻³⁵, other studies have found limited evidence for this shift.⁶,³⁶⁻³⁸ More generally, there is minimal consensus on the factors that drive a familiarity preference in some conditions and a novelty preference in others. This gap in understanding about a core component of visual preferences in infancy has raised numerous interpretive challenges over the decades, including when to predict a preference for novelty versus familiarity and how to interpret unexpected, unstable, or null preferences across studies.³³,³⁴,³⁹

A large-scale investigation into the fundamental tenets of the Hunter and Ames model is an important step toward quantifying relationships between infant age, familiarization time, task difficulty, and preference for familiar vs. novel stimuli. In the present study, we bring together researchers from around the world to create and implement a best-test of key predictions of the Hunter and Ames model. This investigation will enable a broad community of researchers to improve their interpretations of infants’ visual preferences and thereby enable new and more robust insights into the infant mind.

There are two main challenges in evaluating the Hunter and Ames model. First, most behavioral studies with infants have lacked sufficient statistical power due to the inherent limitations within any given lab in testing large numbers of participants.⁴⁰,⁴¹ Recruiting participants and obtaining a high data ‘yield’ are notoriously challenging in infant research. Moving forward, infant research is in need of solutions to achieve higher statistical power in order to improve its reliability and replicability.⁴² A second, related challenge is that most behavioral studies with infants have reported findings based on a geographically and culturally
restricted sample, causing limited generalizability across populations. This is in part a
consequence of the over-representation of researchers in North America and Western Europe,
who tend to recruit primarily local families who have the means and the time for a lab visit. The
result of this tradition of convenience has been an overreliance on samples of White infants from
middle-class families – a biased sample of the world’s population when making conclusions
about both human development and human nature.⁴³,⁴⁴ Overcoming these challenges is essential
to the development of a robust and generalizable model of infants’ visual preferences.

We address these challenges by providing a large-scale, high-powered test of the Hunter
and Ames model through an international collaboration, harnessing the infrastructure of
ManyBabies.⁴⁵ ManyBabies is a six-continent network of scientists who are interested in
understanding key theoretical questions about early child development, promoting best practices
in developmental research, and broadening participation to include scientists and families from a
broad range of communities and cultures. Drawing on this network, we designed an experiment
testing visual preferences for familiar and novel stimuli that will include an infant sample of
unprecedented size and diversity (N = XX, from XX labs in XX countries), enabling an
evaluation of the extent to which the Hunter and Ames model generalizes across cultures and
communities. We test three specific hypotheses (see Table 3): Infants’ preferences for novel
stimuli will be stronger (1) following longer familiarization time, (2) as infants grow older, and
(3) for simpler vs. more complex stimuli.

Method

Ethics

Prior to beginning data collection, all labs contributing data will be asked to complete a
survey that asks for information about, among other things, their lab's ethics approval
procedures. Labs will indicate whether they have new or existing approval from their local ethics
committee or institutional review board to conduct this study or whether the current study does
not require ethics approval at their institution.

For all labs, we will require the following consent procedure: A parent or legal guardian
will provide informed consent for each participating infant, though the specifics of consenting
procedures will vary across labs to comply with local legal requirements and cultural norms.
Families will be compensated according to each lab's standard practices as approved by their
local ethics board. All de-identified data will be stored on the ManyBabies5 OSF repository
([https://osf.io/g3udp/?view_only=0f4f6e3b48d8456999d6459c0bfe5510](https://osf.io/g3udp/?view_only=0f4f6e3b48d8456999d6459c0bfe5510)).

**Participation Details**

An initial open call to participate in “ManyBabies 5: Hunter and Ames” was issued on
July 23, 2020 via social media and developmental science listservs. Over the next two years,
scientists from around the world (see Figure 2) contributed in various ways to study design. The
initial Stage 1 manuscript was submitted for review on January 10, 2023. Between February 1,
2023 and June 30, 2023 we will address reviewer comments, finalize study design and
procedural documents, and work with individual teams to get IRB approval. Data collection will
begin on July 1, 2023 with a goal of finishing one year later on June 30, 2024. Data analysis and
Stage 2 manuscript writing will begin on July 1, 2024 with a target submission date of December
31, 2024.
<figure><img src="image_3.png" /><figcaption>*Figure 2. Map depicting the geographic locations of the ManyBabies 5 contributors.*</figcaption></figure>

Each participating lab will be asked to contribute a sample of at least 32 infants between the ages of 3 and 15 months; effort will be made across labs to ensure that age is distributed evenly across this age span, to the extent possible. Because many of our analyses will examine effects across labs rather than within a single lab, we will also allow labs to contribute a “half sample” of 16 infants. As in the ManyBabies1 project⁴⁶, allowing “half samples” will increase the number of labs capable of contributing to data collection and will enable participation from more labs, especially those from under-resourced and/or underrepresented communities. We will ask that the sample size per lab (a full sample of 32 or half-sample of 16) includes any infant that
enters the lab and not the number of infants retained after exclusion criteria are implemented
because final decisions about participant exclusion will take place centrally.

## Participants

The final sample will include XX infants between the ages of 3 and 15 months, recruited and tested by XX labs across XX countries and XX continents (see Table 2 for by-lab details).

Based on initial surveys of potential contributors, we expect to be able to recruit a sample size of at least 1,280 infants (at least 40 labs plan to contribute data from approximately 32 infants each). The 3- to 15-month age span was chosen because it covers the majority of ages for which the Hunter and Ames model has been used to describe infants' responses to stimuli in previous research. Because the ManyBabies model makes it possible to collect a sizable sample of infants with a variety of developmental histories, we will adopt an inclusive recruitment strategy. Labs will be encouraged to recruit (using lab-standard practices) any infant who meets just two criteria: (1) the infant's age falls within the 3- to 15-month age range, and (2) the infant has no known issues that would directly impede their ability to process visual stimuli (i.e., visual impairments). However, we will additionally collect data about the criteria on which infants are typically excluded in developmental studies, such as prematurity, developmental delays or disorders, family history of colorblindness, and hearing status. For the analyses of interest in the current paper, we will exclude infants when there is a reason to expect that the predictions of our model would differ based on their developmental history (such as prematurity; see inclusion criteria below). However, all data will be retained for potential future analyses beyond the scope of the current paper (e.g., investigating the effects of prematurity on infants' preferences for novel versus familiar stimuli⁴⁷).
In some labs, we expect that infants will also be tested in other experiments during their visit. In these cases, we will strongly encourage labs to administer the current study first because it is commonly observed in infant studies that “second session” experiments result in greater dropout rates. If labs absolutely must administer this study second, we will ask them to report whether infants participated in the current study first or second and, when the infant participated in the current study second, we will ask for information about the first study. All infants will be included in the primary analyses (following previous ManyBabies studies⁴⁶), but information about first versus second session will be retained for exploratory analyses examining the effect of first versus second session testing.

<caption>**Table 2**</caption>

*Overview of participating labs and contributed participants (see below for additional details about method and procedure).*

<table><thead><tr><th>Lab Name</th><th>University</th><th>Method</th><th>Procedure</th><th>Mean age (days)</th><th>N</th><th>Country</th></tr></thead><tbody><tr><td>XX</td><td>XX</td><td>central fixation / eye-tracking</td><td>fixed-length / infant-controlled</td><td>XX</td><td>XX</td><td>XX</td></tr><tr><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td></tr></tbody></table>

**Participant Demographics**

Basic demographic data will be collected for all participants using the ManyBabies Demographics questionnaire (Singh, Barokova, Baumgartner, Lopera, Okyere, et al., unpublished manuscript). This questionnaire asks about participants’ biographical information, race and ethnicity, gestational age, caregiver information, family socio-economic status, language exposure, and relevant developmental concerns. Researchers from different communities, nations, and cultures will be able to modify the content of the questionnaire in
structured ways to ensure appropriateness of the questions for local contexts while preserving, to
the extent possible, comparability across test sites.

Materials

*Visual Stimuli.* In the current experiment, we chose to operationalize *task difficulty* in terms of stimulus complexity: we assume that task difficulty increases as stimuli become more complex. Because the sample size possible in ManyBabies projects provides a unique opportunity to test generalizability across multiple stimulus types, we chose two different categories of stimuli: *fribbles*48–52 and *fractals*53,54. Fribbles are colorful, computer-generated models of 3D figures. All fribbles are made up of one of three possible body shapes and up to four appendages that can be chosen from a set of possible options for heads, legs, and two parts of a tail. Fractals are geometric shapes in which the same structure is repeated iteratively at different levels of scale. These two classes of stimuli were chosen because: (1) they allow for variation in stimulus complexity within the same stimulus category (i.e., by manipulating the number of features); (2) infants are unlikely to have previous experience with either set of stimuli; and (3) the inclusion of two stimulus sets allow us to begin examining the generalizability of any observed effect.

We created 12 unique fribbles and 12 unique fractals, operationalizing complexity as the number of features present (these features are appendages in the case of fribbles and iterations of the same structure for fractals; see Figure 3). Within each stimulus type, 6 items were designed to be low-complexity and 6 were designed to be high-complexity. Specifically, low-complexity fribbles are constructed of one body shape and one appendage while high-complexity fribbles have one body shape and four appendages. In the case of fractals, low-complexity items consist
of 3 iterations of the same fractal pattern whereas high-complexity items are made of 9 iterations of the same pattern.

<table><thead><tr><th></th><th>Fribbles</th><th>Fractals</th></tr></thead><tbody><tr><th>Low Complexity</th><td></td><td></td></tr><tr><th>High Complexity</th><td></td><td></td></tr></tbody></table>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /><figcaption>Figure 3. Examples of low- and high-complexity Fribble and Fractal stimuli.</figcaption></figure>

## Procedure

Full procedural instructions provided to each lab will be available in Supplementary Materials on the OSF. Before data collection begins, all labs will be asked to complete a pre-data-collection survey (which will be viewable on the OSF) and submit a walkthrough video depicting a standard testing session (described in further detail below). During the experiment, infants will be seated on their caregiver's lap or in a high-chair or car seat (whichever option corresponds to labs' standard procedures for testing infants, as reported in the pre-data-collection survey). Each trial will be made up of a familiarization phase, in which infants are first exposed to a stimulus, and two test phases where infants are presented with both the item they viewed during the familiarization phase and a novel test item. These phases will be repeated on each of 12 total trials and are described in further detail below.

**Infant-controlled versus fixed-length methods.** Familiarization can be achieved either by presenting a fixed amount of exposure, i.e. a *fixed-length design*, or by ending the
familiarization phase of an experiment after infants have acquired a certain amount of looking at the target, i.e. an infant-controlled design. Both approaches have been used in the literature. In the fixed-length condition the familiarization stimulus is presented for a fixed duration regardless of how much the infant looks.⁵⁵ This procedure allows for variation in how much time infants look at the stimulus before their visual preference is assessed while controlling for the amount of time from the start of the familiarization phase to the test phase. Thus, in the fixed-length design, the length of the familiarization phase is fixed while the duration of visual attention directed towards the stimulus varies.

In the infant-controlled design, in contrast, the familiarization stimulus is presented until infants accumulate the same amount of looking.⁴,⁵⁶ This procedure ensures that all infants receive the same amount of familiarization before their preferences for novel stimuli are tested. It also means that the length of the familiarization phase is determined by how long it takes infants to accumulate the required amount of looking (e.g., if they have one or two long looks, with little looking away time, or multiple short looks interspersed with variable length looks away). Thus, in the infant-controlled design, the length of the familiarization phase varies while the duration of infants’ visual attention directed towards the stimulus remains consistent.

The consensus across the ManyBabies 5 community was that an infant-controlled design, which facilitates equating infants’ familiarization time, represented the best test of our main research questions. Participating labs will be encouraged to use and supported in their use of an infant-controlled procedure. However, because infant-controlled procedures require either well-trained testers who can reliably code infant looking time in the moment or eye-tracking systems that can support gaze-contingent protocols, it may not be possible for some labs to use this procedure. In line with ManyBabies’ philosophy of inclusive participation, we will attempt to
support all labs in implementing infant-controlled familiarization, but allow for use of a fixed-length procedure in cases in which infant-controlled is not possible (i.e., labs lacking the required personnel or equipment to implement an infant-controlled procedure). This will allow labs to flexibly choose the option that works best for their resources and maximizes overall data collection, while also working towards a high proportion of labs implementing the desired procedure. In total, XX labs will use an infant-controlled procedure and XX labs will use fixed-length familiarization (as reported in the pre-data-collection survey). To ensure that any observed differences across labs using infant-controlled and fixed-length procedures will not be due to selection effects (as labs self-selected into one paradigm or the other), a subset of XX labs who use the infant-controlled design will be asked to collect a second sample of infants using a fixed-length procedure.

***Trial structure.*** The trial sequence is patterned after classic work by Rose and colleagues.⁴ On each trial, a single stimulus is presented centrally on the screen for a familiarization phase, followed by two test phases in which the now-familiar stimulus is paired with a novel stimulus (the novel stimulus will be the same across the two test phases). Following Rose and colleagues⁴, each test phase will be 5 s in duration, and they will differ only in the left-right placement of the novel and familiar stimulus, i.e. the location of the paired test item will be switched between the two test phases.

In order to test the effect of *familiarization time*, we will vary the duration of the familiarization phase, yielding three types of trials. In the infant-controlled design, following Rose and colleagues⁴, the familiarized stimulus will be presented until infants accumulate 5, 10, or 15 s of looking to the familiarization stimulus (thus, the actual trial length will vary across infants depending on how long it takes each infant to accumulate the required looking time). We
will additionally implement a maximum trial length criterion; if infants do not accumulate the required looking time within twice the target familiarization time (i.e., 10, 20, or 30 s), the familiarization phase will end. In the fixed-length design, the familiarized stimulus will be presented for 5, 10, or 15 s total, regardless of the duration of infants' looking.

Immediately following familiarization (in both designs), a central-fixation stimulus – looming circles accompanied by chimes (as used in ManyBabies 1<sup>46</sup>) – will be presented in the center of the screen for a fixed duration of 500 ms to reorient infants' gaze in both the infant-controlled and the fixed-length design. The central-fixation stimulus will be immediately followed by the two test phases, which each will be a fixed length of 5 s, for a total of 10 s of test.⁴ The test phases will be a fixed duration, even if familiarization was infant-controlled. In each test phase, infants will now see a familiar and novel item side by side, left-right position of the two stimuli reversed on the second phase. The novel item for each test will be from a different family within the same stimulus set (i.e., fribbles or fractals) that is the same level of complexity as the one they were familiarized with. The central-fixation stimulus will additionally be presented between the two test phases. Figure 4 depicts the structure of an example trial.

To retain infants' attention between trials, an attention-getter will be played before the first trial and between every trial. Labs will have the option of using a default attention-getter video of a laughing baby, which is effective in drawing infants' attention to the screen, but will be encouraged to use any attention-getter that is a better match for their lab or community. For example, a different attention-getter may be needed for researchers working with deaf or hearing-impaired infants.
<figure><img src="image_12.png" /></figure>

*Figure 4.* This figure depicts the design of an example trial. At familiarization, a single image is shown for 5, 10, or 15 s. After the desired familiarization time is reached (based on the infant-controlled or fixed-length design criteria), infants are presented with a central fixation stimulus for 500 ms. During the two test phases (5 s each), infants view the same image to which they were familiarized paired with a new stimulus from the same set (e.g., fribbles or fractals) and the same level of complexity. The side on which the familiar and novel images are presented is counterbalanced across test phases, but the images remain the same.

**Counterbalancing.** Each infant will view both stimulus sets (fribbles and fractals), with presentation blocked by stimulus set and presentation order (fribbles first or fractals first) and counterbalanced across infants. Additionally, within each stimulus set, infants will experience all combinations of complexity level and familiarization time. As a result, infants will view a total of 6 trials per stimulus set, or 12 trials total. If infants successfully view all 12 trials, this represents approximately 4.5 minutes of looking to the screen (note that the total study time will
increase when the amount of time it takes for infants to accumulate each familiarization time is taken into account in the infant-controlled version).

We will additionally impose a number of constraints during counterbalancing. First, we will ensure that: (1) the familiar item does not occur on the same side in Test Phase 1 on more than two trials in a row, (2) no more than two trials in a row feature stimuli from the same level of complexity, (3) the same familiarization time is not used for more than two trials in a row. Additionally, each familiar and novel stimulus will only be used in one trial, as repeating stimuli will impact how familiar or novel a given stimulus is for the infant. Across infants, we will counterbalance which stimulus is used as the familiar or novel stimulus.

**Apparatus.** To facilitate standardization across labs, we will create infant-controlled and fixed-length versions of the experiment on a variety of software platforms (e.g., Habit, PyHab, EyeLink Experiment Builder, Tobii Studio; or, in the case of the fixed-length design, experimenters will be able to use video-presentation software<sup>57,58</sup>). Individual labs will be instructed to present the program using the setup with which they are most familiar (e.g., TV screen, projection screen, or computer monitor). Labs will additionally choose to collect data using central fixation methods or an eye tracker (see Table 2).

**Coding.** In labs that assess looking time by coding all or part of their videos offline, coding of infants' looking to the familiar versus novel stimulus at test (for both the infant-controlled and fixed-length procedures) and during the familiarization phase (during the fixed-length procedure only) will be conducted via standard procedures in each lab. We will additionally provide a "best practices" manual that includes information about training and reliability standards that can be used by labs without existing standard procedures or those interested in revising their current procedures.
While labs will be allowed to code videos according to their existing lab practices, labs will be asked to report inter-coder reliability by independently re-coding a minimum of 20% of videos and reporting the average frame agreement, i.e., the proportion of frames on which both coders agree on the gaze location. The target reliability for this coding is 90% frame agreement.

**Minimizing bias.** To minimize caregiver bias, each lab will be asked to ensure (using lab-standard methods) that caregivers remain unaware of the visual stimuli being presented (e.g., by asking them to wear darkened glasses, close their eyes, or keep their gaze directed toward the infant's head). To minimize experimenter bias, experimenters making online decisions about infants' looking to or away from the screen will not be aware of which visual stimulus is being presented nor the side of the familiar stimulus during the test phase. Off-line coding, including reliability coding, will be conducted with these same efforts to minimize bias. Reliability coders will also not have access to the original coder's decisions.

## General Lab Practices

Each lab will be responsible for maintaining good practices, conducting this study using the same rigorous standards applied to all other studies in their lab. Labs will be asked to provide information about their standard protocols for testing, such as researcher training practices and basic practices for greeting families, obtaining consent, and debriefing. Labs that are new to infant testing will be paired with more established laboratories for support and guidance throughout the data collection process.

Prior to data collection, labs will be asked to submit a walkthrough video depicting a standard testing session. This video can be of an infant participant or of a placeholder toy (e.g., a teddy bear). Before labs are invited to move on to data collection, this video will be reviewed by the ManyBabies 5 study implementation team to address any questions or inconsistencies that
arise in study implementation. Labs will also be asked to upload a practice data file to ensure that each lab's data format is in compliance with the ManyBabies 5 data structure. After data collection is complete, participating investigators will submit their data to the analysis team. We will ask labs to refrain from analyzing the data independently prior to submission and analysis by the ManyBabies 5 team (with exceptions for trainees' educational timelines). Labs will be asked to provide experimental and participant data for any participant who enters the lab to participate regardless of whether the participant began or completed the study. The purpose of this request is to ensure that researchers do not selectively submit data based on their own impressions of data quality. For each participant, labs will be asked to report on any unusual events, such as equipment failure, parental interference, infant fussiness, or infants taking a break during the session. Whenever possible (i.e., the lab has appropriate permissions from an ethics board), sessions will be video recorded, and video recordings will be submitted together with coded data or uploaded to repositories such as Databrary.

**Inclusion Criteria**

All data collected for the study will be handed to the analysis team for confirmatory analyses (i.e., demographic and experimental data for every infant who entered a participating lab, regardless of how many trials they complete). As mentioned previously, infants will be included in the primary analyses if they meet the following criteria: (1) their age falls within the 3- to 15-month age range, (2) they have no known vision or eye movement problems (e.g., no nystagmus), (3) they were born at 37 or more weeks gestation (if known) and/or do not meet local definitions of preterm birth, and (4) they do not have known developmental delays. In some cases, infants who do not meet these criteria will have been recruited in error (e.g., infants recruited outside of the target age range). Thus, XX infants (XX% of the total sample) will be
excluded for failing to meet the recruitment criteria. Additionally, infants will be excluded for
session-level errors, such as experimenter error, equipment failure, or other forms of interference
that impede infants' ability to attend to at least one trial in the experiment or that affect all trials
(N = XX, XX% of infants remaining after recruitment-based exclusions have been implemented).
All infants will otherwise be eligible for inclusion in analyses.

Finally, we will exclude individual trials for which issues are reported (e.g., infant
fussiness, incorrect stimulus, single instance of parental interference, failure to accumulate the
required familiarization time). A total of XX (XX%) trials will be excluded due to trial-level
errors. To be included in the final analyses, eligible infants (as defined above) will be required to
contribute nonzero looking times for at least 1 trial after trial-level exclusions are applied. XX
infants (XX%) will be excluded for failing to meet this criterion.
<caption>Table 3.</caption>

Design Table.

<table><thead><tr><td>Question</td><td>Hypothesis</td><td>Sampling Plan</td><td>Analysis Plan</td><td>Interpretation given to different outcomes</td></tr></thead><tbody><tr><td>How does familiarization time influence infants' novelty preference?</td><td>(H1A) As familiarization time increases, infants' novelty preference increases.</td><td>The design (n=1,280) provides 96.8% power to detect an effect of d=0.5 or larger (see Table 5).</td><td>We will use a linear mixed-effects model to predict infants' novelty preference (proportion looking to the novel stimulus) from familiarization time (H1A), age (H1B) and stimulus complexity (H1C). The model will include all two- and three-way interactions between the three predictors (these correspond to exploratory tests for interactions between the main predictors; Table 4). The model will include random effects for participant, item, and lab (see main text for full model specification). The three simple effects of familiarization time, age, and stimulus complexity correspond to the three main hypotheses of interest.</td><td>A significant coefficient in the predicted direction (positive: familiarization time, age; negative: stimulus complexity) will be interpreted as evidence consistent with the corresponding hypothesis. A significant coefficient in the opposite direction will be interpreted as providing evidence in the opposing direction from that predicted by the Hunter & Ames model. A non-significant coefficient will be interpreted as not providing evidence for or against the corresponding hypothesis (i.e., we cannot reject the null hypothesis that the corresponding predictor has no effect on infants' novelty preference).</td></tr><tr><td>How does novelty preference change as infants age?</td><td>(H1B) As infant age increases, infants exhibit a stronger novelty preference.</td><td>The design (n=1,280) provides >99% power to detect an effect of d=0.5 or larger (see Table 5).</td><td></td><td></td></tr><tr><td>How is novelty preference affected by the complexity of stimuli?</td><td>(H1C) Novelty preference is larger when the familiarized stimulus is simpler than when it is more complex.</td><td>The design (n=1,280) provides >99% power to detect an effect of d=0.5 or larger (see Table 5).</td><td></td><td></td></tr></tbody></table>
# Data Analysis Plan

## Dependent and Independent Variables

The analyses will include the following variables:

* **Novelty preference (model term: novelty preference).** Infants' preference for the novel stimulus over the familiar stimulus during the entire test phase for each trial (summed across the two test phases) will be measured as the proportion looking to the novel stimulus.⁴

$$ \text{novelty preference} = \frac{\text{novel looking time}}{\text{novel looking time} + \text{familiar looking time}} $$

* **Infant age (age).** Infant age (mean-centered and scaled), will be a continuous predictor.

* **Familiarization time (familiarization_time).** Familiarization time will be treated as a continuous predictor and centered. Specifically, the three familiarization times 5, 10, and 15 s will be coded as -0.5, 0, 0.5.

* **Stimulus complexity (stimulus complexity).** The two levels of stimulus complexity will be centered and coded as -0.5 (low) and 0.5 (high).

* **Participant.** The unique identifier for each individual infant participating in the study.

* **Item.** The unique familiar/novel stimulus pair in a given trial.

* **Lab.** The unique identifier for each individual lab that collected data for the study.

## Statistical Modeling Approach

**Modeling Approach.** In the main model, we will use a linear mixed-effects model to predict infants' novelty preference from age, familiarization time, and stimulus complexity, as well as all two- and three-way interactions between the three predictors. We will include random effects for participant, item, and lab. The planned model will include the maximal random effects structure ⁵⁹, which we will prune as necessary to allow the model to converge (see
**Supplementary Materials** S1 for details on our planned approach for handling model non-convergence). Models will be fit using the lme4 package in R⁶⁰,⁶¹, and *p*-values will be estimated using the Satterthwaite approximation implemented in the R package lmerTest.⁶²

**Model.** The main model will be specified as:

novelty preference ~ 1 + age + familiarization_time + stimulus_complexity +

age : familiarization_time +

age : stimulus_complexity +

familiarization_time : stimulus_complexity +

age : familiarization_time : stimulus_complexity +

(1 + familiarization_time * stimulus_complexity | participant) +

(1 + age * familiarization_time * stimulus_complexity | lab) +

(1 + age * familiarization_time | item)

**Hypothesis Tests**

Our main hypotheses are that familiarization time, age, and stimulus complexity will each systematically predict infants’ novelty preference (see Table 3). In addition to the main effects of interest, the model will also allow us to test for two- and three-way interactions between the main effects of interest. These possible interaction effects are not the main focus of our confirmatory analyses, and thus we will interpret these tests as exploratory (rather than confirmatory). Table 4 lists the question tested by the two- and three-way interactions in the model and how significant coefficients for each of these terms will be interpreted.
<caption>**Table 4.**</caption>

*Interpretation of the interactions in the linear mixed-effects model*

<table><thead><tr><td>Question</td><td>Model term</td></tr></thead><tbody><tr><td>To what extent does the effect of familiarization time on novelty preference change with infant age?</td><td>age * familiarization_time</td></tr><tr><td>To what extent does the effect of stimulus complexity on novelty preference change with infant age?</td><td>age * stimulus_complexity</td></tr><tr><td>To what extent does the effect of stimulus complexity on novelty preference change as familiarization time increases?</td><td>familiarization_time * stimulus_complexity</td></tr><tr><td>To what extent does the interaction between familiarization time and complexity of the stimulus depend on infant age?</td><td>age * familiarization_time * stimulus_complexity</td></tr></tbody></table>

**Power Analysis**

We used a simulation-based approach to evaluate whether the expected sample size (i.e., 1,280 infants) can provide sufficient power to detect effect sizes of interest. All code associated with these simulations is publicly available on the project's OSF page (https://osf.io/g3udp/?view_only=0f4f6e3b48d8456999d6459c0bfe5510), and the main simulation-based results are summarized in our Data Simulation and Power Analysis Supplement. In order to calculate the statistical power from this expected sample size, we simulated 500 datasets with a Cohen's *d* effect size of 0.2, 0.3, 0.4, and 0.5 for the three main effects of interest (i.e., *stimulus type*, *age*, and *familiarization time*) as well as their two- and three-way interactions. We chose these effect size magnitudes based on estimates from previous large-scale preferential-looking studies (e.g., in ManyBabies 1, the main meta-analytic effect size was *d* = 0.35, with effect sizes ranging from *d* = 0.24 to *d* = 0.51 depending on the experimental method) as well as median effect size estimates in meta-analyses within infant populations.40,46
For each of these simulated datasets, we ran a linear mixed-effects regression model with the following structure:

$$ 
\begin{aligned}
\text{novelty\_preference} &\sim \text{1} + \text{stimulus\_complexity} * \text{age} * \text{familiarization\_time} + \\
&\quad (1 + \text{stimulus\_complexity} * \text{familiarization\_time} | \text{participant}) + \\
&\quad (1 | \text{lab}) + \\
&\quad (1 | \text{item})
\end{aligned}
$$

Based on preliminary simulations, we chose to focus on this model because it was the maximal random effects structure that reliably converged on the vast majority of simulation runs (i.e., >90%). Given past evidence that models including random slopes may not converge ⁴⁶, we also simulated power for a model with a simpler random effects structure that included only random intercepts for participants, labs, and items. We then evaluated the statistical power by counting the number of times the effect displayed a significantly non-zero (i.e., *p* < .05) coefficient. The main goal was to establish that we had sufficient power to detect each of the three main effects of interest (age, familiarization time, and stimulus type), i.e., to test the three main predictions of the Hunter and Ames model. As shown in Table 5, the analysis indicates ample power to detect each of these three main effects for medium effect sizes (>80% power for all main effects and *d* = 0.4 to 0.5), with somewhat weaker power to detect a main effect of familiarization time at the lower end of the effect size range (*d* = 0.3: power = 63.4%; *d* = 0.4: power = 86.4%). The power analysis also demonstrates reasonable power to detect age and any interactions, including age, even if the effect size is small (i.e., *d* = 0.2). Notably, the current design has reasonable power to detect a three-way interaction between the main effects (>80% power for *d* = 0.2 or larger). Overall, the power analysis indicates that the current design has sufficient power to detect each of the three main predictions of the Hunter and Ames model.
<caption>**Table 5.**</caption>

Overview of results in power simulation for the model with the maximal converging random effects structure. The numbers in each effect size column indicate the percentage of significant results in the 500 simulated datasets. *stim_type* refers to stimulus type, *fam_time* refers to familiarization time, and *age* refers to infant age in months. Sample size is 1,280 included participants, with 24 trials. See section 7 in the Data Simulation and Power Analysis Supplement for details.

<table><thead><tr><th>Outcome Types</th><th>Model Terms</th><th>d = 0.2</th><th>d = 0.3</th><th>d = 0.4</th><th>d = 0.5</th></tr></thead><tbody><tr><td rowspan="3">main effects</td><td>stim_type</td><td>47.4%</td><td>83.8%</td><td>96.6%</td><td>99.2%</td></tr><tr><td>age</td><td>99.0%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>fam_time</td><td>36.0%</td><td>63.4%</td><td>86.4%</td><td>96.8%</td></tr><tr><td rowspan="3">two-way interactions</td><td>stim_type*fam_time</td><td>12.4%</td><td>24.4%</td><td>35.0%</td><td>48.6%</td></tr><tr><td>stim_type*age</td><td>89.4%</td><td>98.6%</td><td>100%</td><td>100%</td></tr><tr><td>fam_time*age</td><td>86.4%</td><td>96.6%</td><td>99.4%</td><td>100%</td></tr><tr><td rowspan="2">three-way interaction</td><td>age*stim_type*</td><td>80.8%</td><td>85.6%</td><td>93.8%</td><td>95.6%</td></tr><tr><td>fam_time</td><td></td><td></td><td></td><td></td></tr></tbody></table>

**Robustness and Exploratory Analyses**

The analytic strategy described above represents our main confirmatory approach, consistent with contemporary analytic approaches used in the infant literature, and our main conclusions will be based on the results of these confirmatory models. However, the large dataset
collected in the current study presents a unique opportunity to explore the relationship between
the underlying assumptions of these models and the distributional properties of infant-looking
time data. We will therefore examine the robustness of the effects of interest across various
analytic decisions and inclusion criteria. The main results from the confirmatory models will be
interpreted in light of the consistency or variability of the main findings across these exploratory
analyses (see S.2 and S.3 in the *Supplementary Materials* for details on specific robustness and
exploratory analyses).
## Data Availability

Full data will be publicly available at

https://osf.io/g3udp/?view_only=0f4f6e3b48d8456999d6459c0bfe5510.

## Code Availability

All analysis code is publicly available at

https://osf.io/g3udp/?view_only=0f4f6e3b48d8456999d6459c0bfe5510.

## Acknowledgements

This work was financially supported by the Early Career Award from the Einstein Foundation (awarded to Jessica E. Kosie and Martin Zettersten), a grant from the Princeton Data-Driven Social Science Initiative (awarded to Martin Zettersten, Jessica E. Kosie, and Casey Lew-Williams), a Global Collaborative Network Grant from Princeton University (awarded to Casey Lew-Williams), an SSHRC Partnership Development Grant (awarded to J. Kiley Hamlin and members of the ManyBabies Governing Board), National Science Foundation SBE 2004983 to Jessica E. Kosie, and by grants from the Eunice Kennedy Shriver National Institute Of Child Health & Human Development of the National Institutes of Health under Award Numbers F32HD110174 (awarded to Martin Zettersten) and F32HD103439 (awarded to Jessica E. Kosie).

The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

## Author Contributions

Authorship order was determined as follows: The first two authors are the primary co-leads of the project and share first authorship. The last author was the primary senior researcher supervising the project. All other authors are listed in alphabetical order. An overview of
authorship contributions following the CRediT taxonomy can be viewed here:

https://docs.google.com/spreadsheets/d/e/2PACX-

1vRbYFrQYIUeGdXLEOw7miYjzfNSBLR3QKKAQqU52F-

J6nGCupIzgQZywez55NzaN9Vbm3cMh-XK-Fr/_pubhtml
References

1. Hunter, M. A. & Ames, E. W. A multifactor model of infant preferences for novel and familiar stimuli. in *Advances in infancy research* vol. 5 69–95 (Ablex Publishing, 1988).

2. Aslin, R. N. What's in a look? *Dev. Sci.* **10**, 48–53 (2007).

3. Fantz, R. L. Visual Experience in Infants: Decreased Attention to Familiar Patterns Relative to Novel Ones. *Science* **146**, 668–670 (1964).

4. Rose, S. A., Melloy-Carminar, P. & Bridget, W. H. Familiarity and Novelty Preferences in Infant Recognition Memory: Implications for Information Processing. *Dev. Psychol.* **18**, 704–713 (1982).

5. Rose, S. A., Feldman, J. F. & Jankowski, J. J. Infant visual recognition memory. *Dev. Rev.* **24**, 74–100 (2004).

6. Bergmann, C. & Cristia, A. Development of infants’ segmentation of words from native speech: a meta-analytic approach. *Dev. Sci.* **19**, 901–917 (2016).

7. Tsuji, S. & Cristia, A. Perceptual attunement in vowels: A meta-analysis: Infant Vowel Attunement: A Meta-Analysis. *Dev. Psychobiol.* **56**, 179–191 (2014).

8. Oakes, L. M. Using Habituation of Looking Time to Assess Mental Processes in Infancy. *J. Cogn. Dev.* **11**, 255–268 (2010).

9. Colombo, J. & Bundy, R. S. Infant response to auditory familiarity and novelty. *Infant Behav. Dev.* **6**, 305–311 (1983).

10. Kirkham, N. Z., Slemmer, J. A., Richardson, D. C. & Johnson, S. P. Location, Location, Location: Development of Spatiotemporal Sequence Learning in Infancy. *Child Dev.* **78**, 1559–1571 (2007).

11. Rose, S. A. & Feldman, J. F. Infant visual attention: Stability of individual differences from
6 to 8 months. *Dev. Psychol.* **23**, 490–498 (1987).

12. Damon, F., Quinn, P. C. & Pascalis, O. When novelty prevails on familiarity: Visual biases for child versus infant faces in 3.5- to 12-month-olds. *J. Exp. Child Psychol.* **210**, 105174 (2021).

13. Johnson, S. P. et al. Abstract Rule Learning for Visual Sequences in 8- and 11-Month-Olds. *Infancy* **14**, 2–18 (2009).

14. Lew-Williams, C. & Saffran, J. R. All words are not created equal: Expectations about word length guide infant statistical learning. *Cognition* **122**, 241–6 (2012).

15. Marquis, A. R. & Sugden, N. A. Meta-analytic review of infants’ preferential attention to familiar and unfamiliar face types based on gender and race. *Dev. Rev.* **53**, 100868 (2019).

16. Bergmann, C., Rabagliati, H. & Tsuji, S. What's in a looking time preference? Preprint at https://doi.org/10.31234/osf.io/6u453 (2019).

17. Hamlin, J. K., Wynn, K. & Bloom, P. Social evaluation by preverbal infants. *Nature* **450**, 557–559 (2007).

18. Sommerville, J. A., Schmidt, M. F. H., Yun, J. & Burns, M. The Development of Fairness Expectations and Prosocial Behavior in the Second Year of Life. *Infancy* **18**, 40–66 (2013).

19. Ahmadlou, M. et al. A cell type–specific cortico-subcortical brain circuit for investigatory and novelty-seeking behavior. *Science* **372**, eabe9681 (2021).

20. Kakade, S. & Dayan, P. Dopamine: generalization and bonuses. *Neural Netw.* **15**, 549–559 (2002).

21. Hall, B. A., Melfi, V., Burns, A., McGill, D. M. & Doyle, R. E. Curious creatures: a multi-taxa investigation of responses to novelty in a zoo environment. *PeerJ* **6**, e4454 (2018).

22. Ruitenberg, M. F. L., Koppelmans, V., Seidler, R. D. & Schomaker, J. Novelty exposure
induces stronger sensorimotor representations during a manual adaptation task. *Ann. N. Y. Acad. Sci.* **1510**, 68–78 (2022).

23. Berlyne, D. E. Curiosity and exploration. *Science* **153**, 25–33 (1966).

24. Gottlieb, J. & Oudeyer, P. Y. Towards a neuroscience of active sampling and curiosity. *Nat. Rev. Neurosci.* **19**, 758–770 (2018).

25. Kidd, C. & Hayden, B. Y. The psychology and neuroscience of curiosity. *Neuron* **88**, 449–460 (2015).

26. Twomey, K. E. & Westermann, G. Curiosity-based learning in infants: A neurocomputational approach. *Dev. Sci.* 1–13 (2017) doi:10.1111/desc.12629.

27. Hunter, M. A., Ames, E. W. & Koopman, R. Effects of Stimulus Complexity and Familiarization Time on Infant Preferences for Novel and Familiar Stimuli. *Dev. Psychol.* **19**, 338–352 (1983).

28. Wetherford, M. Developmental Changes in Infant Visual Preferences for Novelty and Familiarity. *Child Dev.* **44**, 416–424 (1973).

29. Balaban, M. T. & Waxman, S. R. Do words facilitate object categorization in 9-month-old infants? *J. Exp. Child Psychol.* **64**, 3–26 (1997).

30. Kidd, C., Piantadosi, S. T. & Aslin, R. N. The Goldilocks effect: Human infants allocate attention to visual sequences that are neither too simple nor too complex. *PLoS One* **7**, e36399 (2012).

31. Thiessen, E. D., Hill, E. A. & Saffran, J. R. Infant-directed speech facilitates word segmentation. *Infancy* **7**, 53–71 (2005).

32. Bahrick, L. E. & Pickens, J. N. Infant Memory for Object Motion across a Period of Three Months: Implications for a Four-Phase Attention Function. *J. Exp. Child Psychol.* **59**, 343–
371 (1995).

33. Courage, M. L. & Howe, M. L. The Ebb and Flow of Infant Attentional Preferences: Evidence for Long-term Recognition Memory in 3-Month-Olds. *J. Exp. Child Psychol.* **70**, 26–53 (1998).

34. Roder, B. J., Bushnell, E. W. & Sasseville, A. M. Infants' Preferences for Familiarity and Novelty during the Course of Visual Processing. *Infancy* **1**, 491–507 (2000).

35. Thiessen, E. D. Effects of Inter- and Intra-modal Redundancy on Infants’ Rule Learning. *Lang. Learn. Dev.* **8**, 197–214 (2012).

36. Fisher-Thompson, D. Exploring the emergence of side biases and familiarity-novelty preferences from the real-time dynamics of infant looking. *Infancy* **19**, 227–261 (2014).

37. Fisher-Thompson, D. & Peterson, J. A. Infant side biases and familiarity - Novelty preferences during a serial paired-comparison task. *Infancy* **5**, 309–340 (2004).

38. Rabagliati, H., Ferguson, B. & Lew-Williams, C. The profile of abstract rule learning in infancy: Meta-analytic and experimental evidence. *Dev. Sci.* **22**, e12704 (2019).

39. Houston-Price, C. & Nakai, S. Distinguishing novelty and familiarity effects in infant preference procedures. *Infant Child Dev.* **13**, 341–348 (2004).

40. Bergmann, C. et al. Promoting replicability in developmental research through meta-analyses: Insights from language acquisition research. *Child Dev.* **89**, 1996–2009 (2018).

41. Byers-Heinlein, K. et al. Building a collaborative psychological science: Lessons learned from ManyBabies 1. *Can. Psychol. Psychol. Can.* **61**, 349–363 (2020).

42. Oakes, L. M. Sample Size, Statistical Power, and False Conclusions in Infant Looking-Time Research. *Infancy* **22**, 436–469 (2017).

43. Henrich, J., Heine, S. J. & Norenzayan, A. The weirdest people in the world? *Behav. Brain
Sci. 33, 61–83; discussion 83-135 (2010).

44. Singh, L., Cristia, A., Karasik, L. B. & Oakes, L. *Diversity and Representation in Infant Research: Barriers and bridges towards a globalized science*. https://osf.io/hgukc (2021)
doi:10.31234/osf.io/hgukc.

45. Frank, M. C. et al. *A Collaborative Approach to Infant Research: Promoting Reproducibility, Best Practices, and Theory-Building*. *Infancy* **22**, 421–435 (2017).

46. The ManyBabies Consortium. Quantifying sources of variability in infancy research using the infant-directed speech preference. *Adv. Methods Pract. Psychol. Sci.* **3**, 24–52 (2020).

47. Rose, S. A., Feldman, J. F., Jankowski, J. J. & Rossem, R. Pathways From Prematurity and Infant Abilities to Later Cognition: Pathways to Later Cognition. *Child Dev.* **76**, 1172–1184 (2005).

48. Baumgartner, H. A. Understanding the Role of Non-Contrastive Variability in Word Learning and Visual Attention in Infancy. (University of California, Davis, 2014).

49. Deligianni, F., Senju, A., Gergely, G. & Csibra, G. Automated gaze-contingent objects elicit orientation following in 8-month-old infants. *Dev. Psychol.* **47**, 1499–503 (2011).

50. Ellis, E. M., Borovsky, A., Elman, J. L. & Evans, J. L. Novel word learning: An eye-tracking study. Are 18-month-old late talkers really different from their typical peers? *J. Commun. Disord.* **58**, 143–157 (2015).

51. Ellis, E. M., Borovsky, A., Elman, J. L. & Evans, J. L. Toddlers’ Ability to Leverage Statistical Information to Support Word Learning. *Front. Psychol.* **12**, 600694 (2021).

52. Williams, P. & Simons, D. J. Detecting Changes in Novel, Complex Three-dimensional Objects. *Vis. Cogn.* **7**, 297–322 (2000).

53. Ellis, C. T. et al. Evidence of hippocampal learning in human infants. *Curr. Biol.* **31**, 3358-
3364.e4 (2021).

54. Gustafsson, E., Francoeur, C., Blanchette, I. & Sirois, S. Visual exploration in adults: Habituation, mere exposure, or optimal level of arousal? *Learn. Behav.* **50**, 233–241 (2022).

55. Fagan, J. F. Infant Recognition Memory: The Effects of Length of Familiarization and Type of Discrimination Task. *Child Dev.* **45**, 351–356 (1974).

56. Rose, S. A. Differential Rates of Visual Information Processing in Full-Term and Preterm Infants. *Child Dev.* **54**, 1189–1198 (1983).

57. Kominsky, J. F. PyHab: Open-source real time infant gaze coding and stimulus presentation software. *Infant Behav. Dev.* **54**, 114–119 (2019).

58. Oakes, L. M., Sperka, D., DeBolt, M. C. & Cantrell, L. M. Habit2: A stand-alone software solution for presenting stimuli and recording infant looking times in order to study infant development. *Behav. Res. Methods* **51**, 1943–1952 (2019).

59. Barr, D. J., Levy, R., Scheepers, C. & Tily, H. J. Random effects structure for confirmatory hypothesis testing: Keep it maximal. *J. Mem. Lang.* **68**, 255–278 (2013).

60. Bates, D., Maechler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. *J. Stat. Softw.* 1–48 (2015).

61. R Development Core Team. R: A language and environment for statistical computing. (2022).

62. Kuznetsova, A., Brockhoff, P. B. & Christensen, R. H. B. lmerTest Package: Tests in linear mixed effects models. *J. Stat. Softw.* **82**, (2017).
# Supplementary Materials

## S1. Model Non-Convergence and Pruning Random Effects

If the maximal model does not converge using default settings in the lme4 package, our first step will be to attempt a series of remedies to help the model including the full random effects structure achieve convergence (Brauer & Curtin, 2018). Specifically, we will (a) increase the number of iterations in the estimation procedure, (b) provide the model with improved starting values, and (c) check whether convergence is achieved using any of the other optimization functions available with the lme4 package.

If the model does not converge after attempting these remedies, we will next prune random effects until convergence is achieved. We will first remove random effects that are of lesser theoretical interest before removing random effects that are more crucial to the main hypotheses of the study (i.e., random slopes of critical main effects of interest and random intercepts). In practice, we expect that models including random slopes (although consistent with the experiment design) will be difficult to fit and will require further pruning of the random effects structure. For example, in ManyBabies 1 (ManyBabies Consortium, 2020), the maximal random effects structure that allowed the main model to converge included random intercepts only.

We will proceed as follows in pruning random effects step-by-step. In each instance, we would only proceed to the next pruning step if the model still fails to converge. Each of steps 1-3 will be repeated within each of the three types of random effects (item, lab, participant) separately until the model with the overall maximal random effects structure across the three random effect types is achieved. We will first prune random slopes (following steps 1-3) for
item-based random effects, followed by random slopes for lab-based random effects. Random slopes and covariances for participant-based random effects will be pruned last.

1. We will first sequentially remove **random slopes for interaction terms** since the interactions are of lesser theoretical interest (compared to the main effects), checking for non-convergence after each interaction term is removed. Random slopes for the three-way interaction will be removed first, followed by the interaction for stimulus complexity and age, stimulus complexity and familiarization time, and finally, familiarization time and age.

2. We will next remove **any covariances between random effects that approach 0 or 1** (likely leading to a singular fit in the main linear mixed-effects model).

3. If convergence is still not achieved, we will successively remove **random slopes for each of the main effects** of interest. We will remove random slopes in the following order, checking for convergence after each random slope is removed: (a) stimulus complexity, (b) infant age, and (c) familiarization time.

Finally, if the model still fails to converge after removing all random slopes for item, lab, and participant, we will successively remove **random intercepts**, first removing random intercepts for item, followed by random intercepts for lab and (if necessary) random intercepts for participant last.

## S2. Analysis of Model Assumptions and Robustness

In this section, we outline a series of analyses to test for violations of model assumptions and to investigate the robustness of the results from the main model across alternative model specifications.

### S2.1. Model Diagnostics
First, we will conduct diagnostic tests to inspect two key properties of the data and model: missingness and heteroskedasticity.

### S2.1.1. Missingness

We expect there to be a significant amount of missing data due to a variety of infant-related (e.g., “fussiness”) and experimenter-related (e.g., technical errors) factors. A relatively large proportion of missing data is common in infant research (e.g., around 20% of the data in ManyBabies1; The ManyBabies Consortium, 2020). Taking missingness into account is particularly important in the context of the current study because the main phenomenon of interest — infants’ visual attention — is systematically connected with a possible major source of missingness, namely infant inattention or lack of engagement leading to trial exclusion or failure to measure infant looking behavior. A possible risk is that our estimates of infants’ patterns of novelty preference may be biased. Therefore, we will investigate whether and how different factors influence the presence of missingness in the data. Specifically, we will fit a logistic mixed-effects model predicting the trial-by-trial presence or absence of the data (i.e., coded as 0 = data is present; 1 = data is missing) from several key predictors: the main predictors of interest (age, complexity, familiarization time) and trial number (the strongest predictor of missingness in ManyBabies1). We will include random effects for participant, lab, and item, following the same approach for specifying and pruning random effects as in the main model.

The model will be specified as:

$$ NAs \sim \text{familiarization\_time} + \text{age} + \text{stimulus\_complexity} + \text{trial\_number} + (\dots | \text{participant}) + (\dots | \text{lab}) + (\dots | \text{item}) $$

In exploratory analyses, we will also test a more complex model including a series of additional predictors of interest (e.g., time of testing, time of last nap). If we find evidence of systematicity
in missingness, we will explore routes for handling this systematicity, in particular conducting
multiple imputation, and discuss limitations in the interpretation of the results from the main
model in light of missingness issues.

**S2.1.2. Heteroskedasticity**

A particular concern in evaluating our main model is the possibility of heteroskedasticity.
Specifically, it is likely that variance in looking times will increase across trials, which may in
turn lead to higher variance in our measures of preferential looking at later trials in the
experiment. To check for issues related to heteroskedasticity, we will use the performance
package in R (Lüdecke et al., 2021) to conduct a Breusch-Pagan test (Breusch & Pagan, 1979) of
non-constant error. If we find evidence of heteroskedasticity, we will explore the use of location-
scale modeling and general additive modeling approaches to model and account for sources of
variability (e.g., trial number).

**S2.2. Alternative Link Functions**

We chose the primary dependent variable, infants’ novelty preference score, based on its
prevalence in the literature on infant preferential looking (e.g., Rose et al., 1982). However,
because this dependent measure is a proportion variable, it raises several potential analytical
issues. For example, one possible concern is that the residuals (i.e., the error variance) from the
model may not be normally distributed, violating an assumption of linear mixed-effects models.
Therefore, in the event that the observed data do violate this assumption, we will explore
alternative models that incorporate specialized link functions (i.e., beta regression) that may be
better suited for the properties of proportion score data.

**S3. Exploratory Analyses**
Below, we outline a series of analyses we plan to conduct to explore other possible sources of variation in infant-looking behavior and the impact of specific methodological choices (e.g., inclusion criteria, stimulus choices).

## S3.1. Infant-Controlled vs. Fixed-Length Familiarization

We will test for a moderating effect of familiarization method (infant-controlled vs. fixed-length familiarization) (a) in the overall dataset and (b) in the subsample of the data contributed by labs randomly assigned to collect both infant-controlled and fixed-length samples. Specifically, we will fit the 4-way interaction model predicting novelty preference from stimulus type, age, familiarization time, and familiarization method.

novelty preference ~

1 + age * familiarization_time * stimulus_complexity * familiarization_method +

(1 + familiarization_time * stimulus_complexity | participant) +

(1 + age * familiarization_time * stimulus_complexity * familiarization_method | lab) +

(1 + age * familiarization_time * familiarization_method | item)

If familiarization method exerts a moderating influence on any of the main predictors of interest, we will interpret its implications in the General Discussion.

## S3.2. Stimulus Fixation Time vs. Familiarization Manipulation

In the main confirmatory model, we do not distinguish between infants’ accumulated looking to the familiarized stimulus and the duration of the familiarization phase. Since labs can choose to either use an infant-controlled or a fixed familiarization design, this will allow us to assess each of these predictors independently. We will therefore fit a linear mixed-effects model in which we predict novelty preference from the duration of the familiarization phase
(familiarization_duration) and infants' time fixating the stimulus during the familiarization phase
(stimulusoprovision) while controlling for age and stimulus complexity.

novelty preference ~ 1 +

age + familiarization\_duration + stimulus\_fixation + stimulus\_complexity +

(1 + familiarization\_duration + stimulus\_fixation + stimulus\_complexity | participant) +

(1 + age + familiarization\_duration + stimulus\_fixation + stimulus\_complexity | lab) +

(1 + age + familiarization\_duration + stimulus\_fixation | item)

We expect that familiarization duration and stimulus fixation will be correlated. We will
therefore run diagnostic tests to check for issues related to multicollinearity (specifically, we will
test for variance inflation). If the variance inflation factor indicates a concerning degree of
multicollinearity (i.e., exceeds a value of at least 5), we will treat any results from the model with
caution. If the model including both factors is high, we will instead consider a model
comparisons approach in which we fit two models, one including only familiarization duration
(in addition to the other main effects of age and stimulus complexity) and one with only stimulus
fixation. This will allow us to investigate how well each of the two possible operationalizations
of familiarization time predict preferential looking.

S3.3. Investigating Stimulus Types Separately

One possible concern with the analysis outlined above is that there may be fundamental
differences in how infants process the two types of stimuli, fribbles and fractals (and, e.g., their
associated levels of relative complexity). In particular, fitting a model including both item kinds
may lead to poor model fit if the estimates for the two item types systematically differ. To check
for inconsistencies in the model predictions across the two stimulus types, we will therefore fit a
linear mixed-effects model in which we predict novelty preference from the 4-way interaction
between the main effects of interest (age, familiarization time, and stimulus complexity) and
stimulus type (centered; coded as fribbles = 0.5 vs. fractals = -0.5):

novelty preference ~ 1 +

age * familiarization_time * stimulus_complexity * stimulus_type +

(1 + familiarization_time * stimulus_complexity * stimulus_type | participant) +

(1 + age * familiarization_time * stimulus_complexity * stimulus_type | lab) +

(1 + age * familiarization_time | item)

We will follow the same random effects pruning approach as with the main model. Effects of the
model – especially interactions between stimulus type and the effects of age, familiarization
time, and stimulus complexity – will be qualified in light of any interactions observed with
stimulus type.

**S3.4. Inclusion Criteria**

We will also explore the consequences of varying the inclusion criteria on the main
effects of interest, in particular:

*   **Minimum looking times:** We will also fit the main model at various cutoffs for minimum looking times. Specifically, we will fit the main model after excluding any trials that do not meet successively more restrictive inclusion criteria for a given trial:

o Trials must include 1 second of looking in each of the two 5s test periods.

o Trials must include 2 seconds of looking in each of the two 5s test periods.

*   **Number of trials completed:** We will fit the main model with successively more strict inclusion criteria based on trial number. We will fit the model when only including infants who complete at least 4, 8, 12, 16, 20, and 24 trials and discuss any observed systematic changes in effect magnitude.
### S3.5. Variation across Demographic Predictors

We will explore the degree to which preferential-looking behavior varies across a range of demographic factors and participant characteristics that will be collected along with infants' looking behavior, including variation across testing location, cultural and linguistic experiences, and the backgrounds of infants and their families.

#### References

Brauer, M., & Curtin, J. J. (2018). Linear mixed-effects models and the analysis of nonindependent data: A unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. *Psychological Methods*, 23(3), 389–411. https://doi.org/10.1037/met0000159

Breusch, T. S., & Pagan, A. R. (1979). A Simple Test for Heteroscedasticity and Random Coefficient Variation. *Econometrica*, 47(5), 1287. https://doi.org/10.2307/1911963

Lüdecke, D., Ben-Shachar, M., Patil, I., Waggoner, P., & Makowski, D. (2021). performance: An R Package for Assessment, Comparison and Testing of Statistical Models. *Journal of Open Source Software*, 6(60), 3139. https://doi.org/10.21105/joss.03139

Rose, S. A., Melloy-Carminar, P., & Bridget, W. H. (1982). Familiarity and Novelty Preferences in Infant Recognition Memory: Implications for Information Processing. *Developmental Psychology*, 18(5), 704–713.

The ManyBabies Consortium. (2020). Quantifying sources of variability in infancy research using the infant-directed speech preference. *Advances in Methods and Practices in Psychological Science*. https://doi.org/10.1177/2515245919900809
# ManyBabies 5: The Hunter and Ames Model of Infant Looking Preference

## Supplementary Materials: Data Simulation and Power Analysis

ManyBabies Analysis Team

## Contents

<table><tr><td>1</td><td>Data Simulation</td><td>2</td></tr><tr><td>2</td><td>Visualisation of Simulated Data</td><td>4</td></tr><tr><td>2.1</td><td>Familiarization:</td><td>4</td></tr><tr><td>2.2</td><td>Complexity:</td><td>5</td></tr><tr><td>2.3</td><td>Age:</td><td>6</td></tr><tr><td>2.4</td><td>Age*Familiarization:</td><td>7</td></tr><tr><td>2.5</td><td>Age*Complexity:</td><td>8</td></tr><tr><td>2.6</td><td>Familiarization*Complexity:</td><td>9</td></tr><tr><td>2.7</td><td>Variability by Lab</td><td>11</td></tr><tr><td>2.8</td><td>Variability by Item</td><td>12</td></tr><tr><td>3</td><td>Power Calculation with Full Data and Varying Intercepts and Varying Slopes</td><td>13</td></tr><tr><td>3.1</td><td>Effect Size = 0.5</td><td>13</td></tr><tr><td>3.2</td><td>Effect Size = 0.4</td><td>13</td></tr><tr><td>3.3</td><td>Effect Size = 0.3</td><td>13</td></tr><tr><td>3.3.1</td><td>Visualise Estimates for Fixed Effects:</td><td>14</td></tr><tr><td>3.3.2</td><td>Visualise Estimates for Random Effects:</td><td>15</td></tr><tr><td>3.4</td><td>Effect Size = 0.2</td><td>16</td></tr><tr><td>3.5</td><td>Effect Size = 0.1</td><td>16</td></tr><tr><td>4</td><td>Power Calculation with Full Data and Varying Intercepts</td><td>17</td></tr><tr><td>4.1</td><td>Effect Size = 0.5</td><td>17</td></tr><tr><td>4.2</td><td>Effect Size = 0.4</td><td>17</td></tr><tr><td>4.3</td><td>Effect Size = 0.3</td><td>17</td></tr><tr><td>4.3.1</td><td>Visualise Estimates for Fixed Effects:</td><td>18</td></tr><tr><td>4.3.2</td><td>Visualise Estimates for Random Effects:</td><td>19</td></tr><tr><td>4.4</td><td>Effect Size = 0.2</td><td>20</td></tr><tr><td>4.5</td><td>Effect Size = 0.1</td><td>20</td></tr></table>
<table>
  <tbody>
    <tr>
      <td>5</td>
      <td>Power Calculation with 20 pct. Missing Data and Varying Intercepts and Varying Slopes</td>
      <td>21</td>
    </tr>
    <tr>
      <td>5.1</td>
      <td>Effect Size = 0.5</td>
      <td>21</td>
    </tr>
    <tr>
      <td>5.2</td>
      <td>Effect Size = 0.4</td>
      <td>21</td>
    </tr>
    <tr>
      <td>5.3</td>
      <td>Effect Size = 0.3</td>
      <td>22</td>
    </tr>
    <tr>
      <td>5.3.1</td>
      <td>Visualise Estimates for Fixed Effects:</td>
      <td>22</td>
    </tr>
    <tr>
      <td>5.3.2</td>
      <td>Visualise Estimates for Random Effects:</td>
      <td>23</td>
    </tr>
    <tr>
      <td>5.4</td>
      <td>Effect Size = 0.2</td>
      <td>24</td>
    </tr>
    <tr>
      <td>5.5</td>
      <td>Effect Size = 0.1</td>
      <td>24</td>
    </tr>
    <tr>
      <td>6</td>
      <td>Power Calculation with 50 pct. Missing Data and Varying Intercepts and Varying Slopes</td>
      <td>25</td>
    </tr>
    <tr>
      <td>6.1</td>
      <td>Effect Size = 0.5</td>
      <td>25</td>
    </tr>
    <tr>
      <td>6.2</td>
      <td>Effect Size = 0.4</td>
      <td>25</td>
    </tr>
    <tr>
      <td>6.3</td>
      <td>Effect Size = 0.3</td>
      <td>26</td>
    </tr>
    <tr>
      <td>6.3.1</td>
      <td>Visualise Estimates for Fixed Effects:</td>
      <td>26</td>
    </tr>
    <tr>
      <td>6.3.2</td>
      <td>Visualise Estimates for Random Effects:</td>
      <td>27</td>
    </tr>
    <tr>
      <td>6.4</td>
      <td>Effect Size = 0.2</td>
      <td>28</td>
    </tr>
    <tr>
      <td>6.5</td>
      <td>Effect Size = 0.1</td>
      <td>28</td>
    </tr>
    <tr>
      <td>7</td>
      <td>Overview of Power Simulation Results</td>
      <td>29</td>
    </tr>
    <tr>
      <td>7.1</td>
      <td>Summary Statistics for Power Calculation with Full Data and Varying Intercepts and Varying Slopes:</td>
      <td>29</td>
    </tr>
    <tr>
      <td>7.2</td>
      <td>Summary Statistics for Power Calculation with Full Data and Varying Intercepts:</td>
      <td>29</td>
    </tr>
    <tr>
      <td>7.3</td>
      <td>Summary Statistics for Power Calculation with 20 pct. Missing Data and Varying Intercepts and Varying Slopes:</td>
      <td>29</td>
    </tr>
    <tr>
      <td>7.4</td>
      <td>Summary Statistics for Power Calculation with 50 pct. Missing Data and Varying Intercepts and Varying Slopes:</td>
      <td>30</td>
    </tr>
    <tr>
      <td>8</td>
      <td>Overview of Bias Results</td>
      <td>30</td>
    </tr>
    <tr>
      <td>9</td>
      <td>Increasing Variance with Trial Number</td>
      <td>31</td>
    </tr>
    <tr>
      <td>9.1</td>
      <td>Extracting estimates of increasing variance from the adult pilot data</td>
      <td>31</td>
    </tr>
    <tr>
      <td>9.2</td>
      <td>Visualisation of adult data in proportions</td>
      <td>32</td>
    </tr>
    <tr>
      <td>9.3</td>
      <td>Visualisation of adult data in z-score</td>
      <td>33</td>
    </tr>
    <tr>
      <td>9.4</td>
      <td>Modification of simulation function to include variance increase</td>
      <td>34</td>
    </tr>
    <tr>
      <td>9.5</td>
      <td>Visualise the variance increase</td>
      <td>37</td>
    </tr>
    <tr>
      <td>9.6</td>
      <td>Model building and bias assessment</td>
      <td>38</td>
    </tr>
    <tr>
      <td>9.7</td>
      <td>Test how many of the models violate homoskedasticity</td>
      <td>42</td>
    </tr>
    <tr>
      <td>9.8</td>
      <td>Overview of how many of the models exhibit homoskedasticity</td>
      <td>42</td>
    </tr>
    <tr>
      <td>9.9</td>
      <td>Bayesian robust location-scale regression model</td>
      <td>42</td>
    </tr>
    <tr>
      <td>9.9.1</td>
      <td>Prior-Posterior Update Checks</td>
      <td>45</td>
    </tr>
    <tr>
      <td>10</td>
      <td>Data missing not completely at random</td>
      <td>51</td>
    </tr>
    <tr>
      <td>10.1</td>
      <td>Simulation of missing data according to increasing infant age</td>
      <td>51</td>
    </tr>
    <tr>
      <td>10.1.1</td>
      <td>Simulation of models</td>
      <td>52</td>
    </tr>
    <tr>
      <td>10.1.2</td>
      <td>Visualise Estimates for Fixed Effects:</td>
      <td>53</td>
    </tr>
    <tr>
      <td>10.1.3</td>
      <td>Visualise Estimates for Random Effects:</td>
      <td>54</td>
    </tr>
    <tr>
      <td>10.2</td>
      <td>Simulation of missing data according to increasing trial number</td>
      <td>55</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>10.2.1 Simulation of models</td>
      <td>56</td>
    </tr>
    <tr>
      <td>10.2.2 Visualise Estimates for Fixed Effects:</td>
      <td>57</td>
    </tr>
    <tr>
      <td>10.2.3 Visualise Estimates for Random Effects:</td>
      <td>58</td>
    </tr>
    <tr>
      <td>10.3 Simulation of missing data according to increasing familiarisation</td>
      <td>59</td>
    </tr>
    <tr>
      <td>10.3.1 Simulation of models</td>
      <td>60</td>
    </tr>
    <tr>
      <td>10.3.2 Visualise Estimates for Fixed Effects:</td>
      <td>61</td>
    </tr>
    <tr>
      <td>11 Summary Statistics for Power Calculation with Data missing not completely at random</td>
      <td>63</td>
    </tr>
    <tr>
      <td>12 Logistic Regression Models</td>
      <td>64</td>
    </tr>
  </tbody>
</table>
# 1 Data Simulation

```r
my_sim_data <- function(
  n_subj = 1280, # number of subjects
  n_simple = 12,  # number of complex stimuli
  n_complex = 12,  # number of complex stimuli
  n_small_fam = 8, # small familiarization time
  n_medium_fam = 8, # medium familiarization time
  n_high_fam = 8,  # high familiarization time
  n_lab = 40,
  beta_0 = 0, # intercept; i.e., the grand mean
  beta_c = 0.3, # main effect for complexity
  beta_f = 0.3, # main effect for familiarization time
  beta_a = 0.3, # main effect for age
  beta_ca = 0.3,
  beta_af = 0.3,
  betaCF = 0.3,
  beta_cfa = 0.3, # main effect for interaction between complexity and familiarization.
  subject_0 = 0.2, # by-subject random intercept sd
  subject_c = 0.2, # by-subject slope complexity sd
  subject_f = 0.2, # by-subject slope familiarization sd
  subject_a = 0.2, # by-subject slope age sd
  subject_ca = 0.2, # by-subject slope for interaction between age and complexity sd
  subject_af = 0.2, # by-subject slope for interaction between age and familiarization sd
  subjectCF = 0.2, # by-subject slope complexity*familiarization sd
  subject_cfa = 0.2, # by-subject slope for interaction between age, complexity and familiarization sd
  subj_rho = 0.2, # correlations between by-subject random effects
  lab_0 = 0.2, # by-lab random intercept sd
  lab_c = 0.2, # by-lab slope complexity sd
  lab_f = 0.2, # by-lab slope familiarization sd
  lab_a = 0.2, # by-lab slope age sd
  lab_ca = 0.2, # by-lab slope for interaction between age and complexity sd
  lab_af = 0.2, # by-lab slope for interaction between age and familiarization sd
  labCF = 0.2, # by-lab random slope complexity*familiarization sd
  lab_cfa = 0.2, # by-lab slope for interaction between age, complexity and familiarization sd
  lab_rho = 0.2, # correlations between by-lab random effects
  item_0 = 0.2, # by-item random intercept sd
  item_c = 0.2, # by-item slope complexity sd
  item_f = 0.2, # by-item slope familiarization sd
  item_a = 0.2, # by-item slope age sd
  item_ca = 0.2, # by-item slope for interaction between age and complexity sd
  item_af = 0.2, # by-item slope for interaction between age and familiarization sd
  itemCF = 0.2, # by-item random slope complexity*familiarization sd
  item_cfa = 0.2, # by-item slope for interaction between age, complexity and familiarization sd
```
item_rho = 0.2, # correlations between by-item random effects

sigma = 0.3 # residual (error) sd
) { # residual (standard deviation)

# simulate a sample of items

items <- data.frame(
  category = rep(c("simple", "complex"), c(n_simple, n_complex)),
  X_c = rep(c(-0.5, 0.5), c(n_simple, n_complex)),
  familiarization = rep(c("short", "medium", "long"), (n_simple + n_complex)/3),
  X_f = rep(c(-0.5, 0, 0.5), (n_simple + n_complex)/3),
  faux::rnorm_multi(
    n = n_simple + n_complex, mu = 0, sd = c(item_0,
      item_c,
      item_f,
      item_a,
      item_ca,
      item_af,
      item_cf,
      item_cfa), r = item_rho,
    varnames = c("I_0", "I_c", "I_f", "I_a",
      "I_ca", "I_af", "I_cf",
      "I_cfa"))
  ) %>
  mutate(item_id = faux::make_id(nrow(.), "I"))

# simulate a sample of subjects

subjects <-
  faux::rnorm_multi(
    n = n_subj, mu = 0, sd = c(subject_0,
      subject_c,
      subject_f,
      subject_a,
      subject_ca,
      subject_af,
      subject_cf,
      subject_cfa), r = subj_rho,
    varnames = c("S_0", "S_c", "S_f", "S_a",
      "S_ca", "S_af", "S_cf",
      "S_cfa")
  ) %>
  mutate(subj_id = faux::make_id(nrow(.), "S")) %>
  mutate(X_a = runif(n_subj, min = -0.5, max = 0.5))

#add subject age measure, sample from distribution from -0.5 to 0.5. #subjects$subj_id <- 1:n_subj

labs <- faux::rnorm_multi(
  n = n_lab, mu = 0, sd = c(lab_0, lab_c, lab_f, lab_a,
    lab_ca, lab_af, lab_cf,
    lab_cfa), r = lab_rho,
  varnames = c("L_0", "L_c", "L_f", "L_a",
    "L_ca", "L_af", "L_cf",
    "L_cfa")
) %>
  mutate(lab_id = faux::make_id(nrow(.), "L"))

#create lab and subj nesting structure

#Number of subjects must be a multiple of number of labs

lab_multiplier = n_subj/n_lab

lab_subj_dict <- data.frame(
  subj_id = subjects$subj_id,
```r
dat_sim <- my_sim_data()
```

# 2 Visualisation of Simulated Data

## 2.1 Familiarization:

```r
dat_sim_plot_familiarization <- dat_sim %>%
  group_by(X_f) %>%
  dplyr::summarise(med_DV = median(DV))

plot_familiarization <- dat_sim %>%
  mutate(X_f = as.factor(X_f)) %>%
  ggplot() + geom_point(aes(y = DV, x = X_f), position = "jitter",
    alpha = 0.2, size = 0.2) + geom_violin(aes(y = DV, x = X_f,
    fill = familiarization), alpha = 0.2) + geom_line(aes(y = med_DV,
    x = as.factor(X_f), group = 1), data = dat_sim_plot_familiarization) +
  geom_point(aes(y = med_DV, x = as.factor(X_f)), alpha = 0.8,
    size = 2, data = dat_sim_plot_familiarization) + scale_fill_manual(values = viridis(n = 3)) +
  ggtitle("Familiarization") + xlab("Familiarization") + themeBW()

plot_familiarization <- plot_familiarization + theme plot.title = element_text(hjust = 0.5,
  size = 20))

plot_familiarization
```
Familiarization

<figure><img src="image_13.png" /></figure>

2.2 Complexity:

```
dat_sim_plot_complexity <- dat_sim %>%
group_by(X_c) %>%
dplyr::summarise(med_DV = median(DV))
```

```
plot_complexity <- dat_sim %>%
  mutate(X_c = as.factor(X_c)) %>%
  ggplot() + geom_point(aes(y = DV, x = X_c), position = "jitter",
    alpha = 0.2, size = 0.2) + geom_violin(aes(y = DV, x = X_c,
    fill = category), alpha = 0.2) + geom_line(aes(y = med_DV,
    x = as.factor(X_c), group = 1), data = dat_sim_plot_complexity) +
  geom_point(aes(y = med_DV, x = as.factor(X_c)), alpha = 0.8,
    size = 2, data = dat_sim_plot_complexity) + scale_fill_manual(values = viridis(n = 2)) +
  ggtitle("Stimulus Complexity") + xlab("Stimulus Complexity") +
  theme_bw()
```
```r
plot_complexity <- plot_complexity + theme plot.title = element_text(hjust = 0.5, size = 20))
plot_complexity
```

## Stimulus Complexity

<figure><img src="image_14.png" /></figure>

## 2.3 Age:

```r
plot_age <- dat_sim %>
  ggplot() + geom_point(aes(y = DV, x = X_a), position = "jitter",
    alpha = 0.2, size = 0.2) + geom.smooth(method = "lm", se = TRUE,
    formula = y ~ x, aes(y = DV, x = X_a)) + ggtitle("Age") +
    xlab("Age") + themeBW()
```

```r
plot_age <- plot_age + theme( plot.title = element_text(hjust = 0.5,
    size = 20))
plot_age
```
<figure><img src="image_15.png" /></figure>

2.4 Age\*Familiarization:

```r
plot_age_familiarization <- dat_sim %>%
  ggplot() + geom_point(aes(y = DV, x = X_a), position = "jitter",
    alpha = 0.2, size = 0.2) + geom.smooth(method = "lm", formula = y ~
    x, se = TRUE, aes(y = DV, x = X_a)) + facetWrap(~X_f) +
  ggsTitle("Age x Familiarization Interaction") + xlab("Age") +
  themeBW()

plot_age_familiarization <- plot_age_familiarization + theme plot.title = element_text(hjust = 0.5,
  size = 20))

plot_age_familiarization
```
Age x Familiarization Interaction

<figure><img src="image_16.png" /></figure>

2.5 Age*Complexity:

```r
plot_age_complexity <- dat_sim %>%
  ggplot() + geom_point(aes(y = DV, x = X_a), position = "jitter",
    alpha = 0.2, size = 0.2) + geom.smooth(method = "lm", formula = y ~
    x, se = TRUE, aes(y = DV, x = X_a)) + facetWrap(~X_c) +
  ggtitle("Age x Complexity Interaction") + xlab("Age") + themeBW()

plot_age_complexity <- plot_age_complexity + theme plot古代 = element_text(hjust = 0.5,
  size = 20))

plot_age_complexity
```
Age x Complexity Interaction

<figure><img src="image_17.png" /></figure>

2.6 Familiarization*Complexity:

```
dat_f_c_intersection <- dat_sim %>%
  mutate(X_c = as.factor(X_c)) %>%
  mutate(X_f = as.factor(X_f)) %>%
  group_by(X_f, X_c) %>%
  dplyr::summarise(med_DV = median(DV))
```

## `summarise()` has grouped output by 'X_f'. You can override using the `.groups`
## argument.

```
plot_familiarization_complexity <- dat_sim %>%
  mutate(X_c = as.factor(X_c)) %>%
  mutate(X_f = as.factor(X_f)) %>%
  ggplot() + geom_point(aes(y = DV, x = X_f), position = "jitter",
```
alpha = 0.2, size = 0.2) + geom_point(aes(y = med_DV, x = as.factor(X_f)),
alpha = 0.8, size = 2, data = dat_f_c_intersection) + geom_line(aes(y = med_DV,
x = as.factor(X_f), group = 1), data = dat_f_c_intersection) +
geom_violin(aes(y = DV, x = X_f, fill = familiarization),
alpha = 0.2) + scale_fill_manual(values = viridis(n = 3)) +
facet-wrap(~X_c) + ggtitle("Familiarization x Complexity Interaction") +
xlab("Familiarization") + themeBW()
plot_familiarization_complexity <- plot_familiarization_complexity +
theme plot.title = element_text(hjust = 0.5, size = 20))
plot_familiarization_complexity

<figure><img src="image_18.png" /><figcaption>Familiarization x Complexity Interaction</figcaption></figure>
## 2.7 Variability by Lab

Variability by Lab

<figure><img src="image_19.png" /></figure>
## 2.8 Variability by Item

### Variability by Item

<figure><img src="image_20.png" /></figure>
# 3 Power Calculation with Full Data and Varying Intercepts and Varying Slopes

## 3.1 Effect Size = 0.5

```r
# Number of simulations:
reps <- 350

# Simulation function:
run_sims <- function(filename_full, ef) {
  dat_sim <- my_sim_data(
    beta_c = ef,
    beta_f = ef,
    beta_a = ef,

    beta_ca = ef,
    beta_af = ef,
    beta Cf = ef,

    beta_cfa = ef
  )

  mod_sim <- lmer(DV ~ 1 + X_a * X_c * X_f +
    (1 + X_c * X_f | subj_id) +
    (1 | lab_id) +
    (1 | item_id),
  data = dat_sim)

  sim_results <- broom.mixed::tidy(mod_sim)

  # append the results to a file
  append <- file.exists(filename_full)
  write_csv(sim_results, filename_full, append = append)

  # return the tidy table
  sim_results
}

filename_full_0.5 = 'run_sims_full_0.5.csv'
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_0.5, ef = 0.5))
end_time <- Sys.time()
end_time - start_time
```

## 3.2 Effect Size = 0.4

```r
filename_full_0.4 = "run_sims_full_0.4.csv"
start_time <- Sys.time()
sims <- purrr::map�(df(1:reps, ~run_sims(filename_full = filename_full_0.4,
end_time <- Sys.time()
end_time - start_time
```

## 3.3 Effect Size = 0.3

```r
filename_full_0.3 = "run_sims_full_0.3.csv"
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_0.3,
```
ef = 0.3))
end_time <- Sys.time()
end_time - start_time

### 3.3.1 Visualise Estimates for Fixed Effects:

sims_full_0.3 <- read_csv(filename_full_0.3, col_types = cols(group = col_factor(ordered = TRUE), term = col_factor(ordered = TRUE)))

fixed_full Plot <- sims_full_0.3 %>% filter(effect == "fixed") %>% ungroup() %>% arrange(term, estimate) %>% mutate(row = rep(seq(1:reps), 8)) %>% ggplot(aes(x = row, y = estimate, ymin = estimate - std.error, ymax = estimate + std.error)) + facetWrap(~term, scales = "free") + geom_pointrange(fatten = 1/2) + ylab("Estimates") + xlab("Simulations") + ggtitle("Estimates of Fixed Effects for Full Data and Varying Intercepts and Varying Slopes, ef = 0.3") + themeBW()

fixed_full Plot <- fixed_full Plot + theme(Plot.title = element_text(hjust = 0.5, size = 20))

fixed_full Plot
Estimates of Fixed Effects for Full Data and Varying Intercepts and Varying Slopes, ef = 0.3

<figure><img src="image_21.png" /></figure>

3.3.2 Visualise Estimates for Random Effects:

```
ran_full Plot <- sims_full_0.3 %>%
  filter(effect == "ran_pars") %>%
  ungroup() %>%
  arrange(group, term, estimate) %>%
  mutate(row = rep(seq(1:reps), 13)) %>%
  ggplot(aes(x = row, y = estimate)) + geom_point(alpha = 0.7) +
  facetWrap(~group + term, scales = "free_y") + themeBW() +
  ylab("Estimates") + xlab("Simulations") + ggtitle("Estimates of Random Effects Full Random-Effects Structure")

ran_full Plot <- ran_full Plot + theme plot.title = element_text(hjust = 0.5, size = 20))
ran_full Plot
```
Estimates of Random Effects Full Random-Effects Structure, ef = 0.3

<figure><img src="image_22.png" /><figcaption>Simulations</figcaption></figure>

3.4 Effect Size = 0.2

filename_full_0.2 = "run_sims_full_0.2.csv"
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_0.2,
  ef = 0.2))
end_time <- Sys.time()
end_time - start_time

3.5 Effect Size = 0.1

filename_full_0.1 = "run_sims_full_0.1.csv"
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_0.1,
```
ef = 0.1))
end_time <- Sys.time()
end_time - start_time
```

# 4 Power Calculation with Full Data and Varying Intercepts

## 4.1 Effect Size = 0.5

```r
# Simulation function:
run_sims <- function(filename_full, ef) {
```

```r
dat_sim <- my_sim_data beta_c = ef,
                      beta_f = ef,
                      beta_a = ef,
```

```r
                      beta_ca = ef,
                      beta_af = ef,
                      betaCF = ef,
```

```r
                      beta_cfa = ef)
```

```r
mod_sim <- lmer(DV ~ 1 + X_a * X_c * X_f +
                 (1 | subj_id) +
                 (1 | lab_id) +
                 (1 | item_id),
                 data = dat_sim)
```

```r
sim_results <- broom.mixed::tidy(mod_sim)
```

```r
# append the results to a file
append <- file.exists(filename_full)
write_csv(sim_results, filename_full, append = append)
```

```r
# return the tidy table
sim_results
```

```r
filename_full_int_0.5 = 'run_sims_full_int_0.5.csv'
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_int_0.5, ef = 0.5))
end_time <- Sys.time()
end_time - start_time
```

## 4.2 Effect Size = 0.4

```r
filename_full_int_0.4 = "run_sims_full_int_0.4.csv"
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_int_0.4,
                                     ef = 0.4))
end_time <- Sys.time()
end_time - start_time
```

## 4.3 Effect Size = 0.3
filename_full_int_0.3 = "run_sims_full_int_0.3.csv"
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_int_0.3,
ef = 0.3))
end_time <- Sys.time()
end_time - start_time

4.3.1 Visualise Estimates for Fixed Effects:

sims_full_int_0.3 <- read_csv(filename_full_int_0.3, col_types = cols(group = col_factor(ordered = TRUE), term = col_factor(ordered = TRUE)))
fixed_full_int plot <- sims_full_int_0.3 %>%
  filter(effect == "fixed") %>%
  ungroup() %>%
  arrange(term, estimate) %>%
  mutate(row = rep(seq(1:reps), 8)) %>%
  ggplot(aes(x = row, y = estimate, ymin = estimate - std.error,
             ymax = estimate + std.error)) + facetWrap(~term, scales = "free") +
  geom_pointrange(fatten = 1/2) + ylab("Estimates") + xlab("Simulations") +
  ggtitle("Estimates of Fixed Effects for Full Data and Random Intercepts, ef = 0.3") +
  themeBW()
fixed_full_int plot <- fixed_full_int plot + theme plot.title = element_text(hjust = 0.5,
size = 20))
fixed_full_int plot
Estimates of Fixed Effects for Full Data and Random Intercepts, ef = 0.3

<figure><img src="image_23.png" /></figure>

4.3.2 Visualise Estimates for Random Effects:

```
ran_full_int plot <- sims_full_int_0.3 %>
filter(effect == "ran_pars") %>
ungroup() %>
arrange(group, estimate) %>
mutate(row = rep(seq(1:reps), 4)) %>
ggplot(aes(x = row, y = estimate)) + geom_point(alpha = 0.7) +
facetWrap(~group, scales = "free_y") + themeBW() + ylab("Estimates") +
xlab("Simulations") + ggtitle("Estimates of Random Effects for Full Data, ef = 0.3") +
themeBW()

ran_full_int plot <- ran_full_int plot + theme plot.title = element_text(hjust = 0.5,
size = 20))

ran_full_int plot
```
Estimates of Random Effects for Full Data, ef = 0.3

<figure><img src="image_24.png" /></figure>

4.4 Effect Size = 0.2

filename_full_int_0.2 = "run_sims_full_int_0.2.csv"
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_int_0.2,
  ef = 0.2))
end_time <- Sys.time()
end_time - start_time

4.5 Effect Size = 0.1

filename_full_int_0.1 = "run_sims_full_int_0.1.csv"
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_int_0.1,
```
ef = 0.1))
end_time <- Sys.time()
end_time - start_time
```

# 5 Power Calculation with 20 pct. Missing Data and Varying Intercepts and Varying Slopes

## 5.1 Effect Size = 0.5

```r
run_sims_missing <- function(filename_missing, ef) {
  dat_sim <- my_sim_data(
    beta_c = ef,
    beta_f = ef,
    beta_a = ef,

    beta_ca = ef,
    beta_af = ef,
    beta Cf = ef,

    beta_cfa = ef
  )

  missing Samples <- dat_sim %>%
    mutate(nas = rbinom(nrow.dat_sim), 1, 1 - .20)) %>%
    mutate(DV = ifelse(nas == 1, DV, NA)) %>%
    drop_na()

  mod_sim <- lmer(DV ~ 1 + X_a * X_c * X_f +
    (1 + X_c * X_f | subj_id) +
    (1 | lab_id) +
    (1 | item_id),
  data=missingSamples)

  sim_results <- broom.mixed::tidy(mod_sim)

  # append the results to a file
  append <- file.exists(filename_missing)
  write_csv(sim_results, filename_missing, append = append)

  # return the tidy table
  sim_results
}

filename_20_missing_0.5 = 'run_sims_20_missing_0.5.csv'
start_time <- Sys.time()
sims_missing <- purrr::map DF(1:reps, ~run_sims_missing(filename_missing = filename_20_missing_0.5, ef = 0.5))
end_time <- Sys.time()
end_time - start_time
```

## 5.2 Effect Size = 0.4

```r
filename_20_missing_0.4 = "run_sims_20_missing_0.4.csv"
start_time <- Sys.time()
sims_missing <- purrr::map DF(1:reps, ~run_sims_missing(filename_missing = filename_20_missing_0.4,
  ef = 0.4))
end_time <- Sys.time()
end_time - start_time
```
## 5.3 Effect Size = 0.3

```r
filename_20_missing_0.3 = "run_sims_20_missing_0.3.csv"
start_time <- Sys.time()
sims_missing <- purrr::map DF(1:reps, ~run_sims_missing(filename_missing = filename_20_missing_0.3,
  ef = 0.3))
end_time <- Sys.time()
end_time - start_time
```

### 5.3.1 Visualise Estimates for Fixed Effects:

```r
# read saved simulation data
sims_20_missing_0.3 <- read_csv(filename_20_missing_0.3, col_types = cols(
  # makes sure plots display in this order
group = col_factor(ordered = TRUE),
term = col_factor(ordered = TRUE)
))

fixed_missing_plot <- sims_20_missing_0.3 %>%
  filter(effect == "fixed") %>%
  ungroup() %>%
  arrange(term, estimate) %>%
  mutate(row = rep(seq(1:reps), 8)) %>%
  ggplot(aes(x = row, y = estimate, ymin = estimate-std.error, ymax = estimate+std.error)) +
  facetWrap(~term, scales = "free") +
  geom_pointrange(fatten = 1/2) +
  ylab("Estimates") +
  xlab("Simulations") +
  ggtitle('Estimates of Fixed Effects for 20 pct. Missing Data, ef = 0.3') +
  themeBW()

fixed_missing_plot <- fixed_missing plot + theme plot.title = element_text(hjust = 0.5, size=20))
fixed_missing plot
```
Estimates of Fixed Effects for 20 pct. Missing Data, ef = 0.3

<figure><img src="image_25.png" /></figure>

5.3.2 Visualise Estimates for Random Effects:

```r
ran_missing_plot <- sims_20_missing_0.3 %>%
  filter(effect == "ran_pars") %>%
  ungroup() %>%
  arrange(group, term, estimate) %>%
  mutate(row = rep(seq(1:reps), 13)) %>%
  ggplot(aes(x = row, y = estimate)) + geom_point(alpha = 0.7) +
  facetWrap(~group + term, scales = "free_y") + themeBW() +
  ylab("Estimates") + xlab("Simulations") + ggtitle("Estimates of Random Effects for 20 pct. Missing Data, ef = 0.3")

ran_missing_plot <- ran_missing_plot + theme(plot.title = element_text(hjust = 0.5, size = 20))

ran_missing_plot
```
Estimates of Random Effects for 20 pct. Missing Data, ef = 0.3

<figure><img src="image_26.png" /><figcaption>Simulations</figcaption></figure>

5.4 Effect Size = 0.2

filename_20_missing_0.2 = "run_sims_20_missing_0.2.csv"
start_time <- Sys.time()
sims_missing <- purrr::map DF(1:reps, ~run_sims_missing(filename_missing = filename_20_missing_0.2,
  ef = 0.2))
end_time <- Sys.time()
end_time - start_time

5.5 Effect Size = 0.1

filename_20_missing_0.1 = "run_sims_20_missing_0.1.csv"
start_time <- Sys.time()
sims_missing <- purrr::map�(df(1:reps, ~run_sims_missing(filename_missing = filename_20_missing_0.1,
```
ef = 0.1))
end_time <- Sys.time()
end_time - start_time
```

# 6 Power Calculation with 50 pct. Missing Data and Varying Intercepts and Varying Slopes

## 6.1 Effect Size = 0.5

```r
run_sims_missing <- function(filename_missing, ef) {
  dat_sim <- my_sim_data(
    beta_c = ef,
    beta_f = ef,
    beta_a = ef,

    beta_ca = ef,
    beta_af = ef,
    beta Cf = ef,

    beta_cfa = ef
  )

  missing Samples <- dat_sim %>%
    mutate(nas = rbinom(nrow.dat_sim), 1, 1 - .50)) %>%
    mutate(DV = ifelse(nas == 1, DV, NA)) %>%
    drop_na()

  mod_sim <- lmer(DV ~ 1 + X_a * X_c * X_f +
    (1 + X_c * X_f | subj_id) +
    (1 | lab_id) +
    (1 | item_id),
  data=missingSamples)

  sim_results <- broom.mixed::tidy(mod_sim)

  # append the results to a file
  append <- file.exists(filename_missing)
  write_csv(sim_results, filename_missing, append = append)

  # return the tidy table
  sim_results
}

filename_50_missing_0.5 = 'run_sims_50_missing_0.5.csv'
start_time <- Sys.time()
sims_missing <- purrr::map DF(1:reps, ~run_sims_missing(filename_missing = filename_50_missing_0.5, ef = 0.5))
end_time <- Sys.time()
end_time - start_time
```

## 6.2 Effect Size = 0.4

```r
filename_50_missing_0.4 = "run_sims_50_missing_0.4.csv"
start_time <- Sys.time()
sims_missing <- purrr::map DF(1:reps, ~run_sims_missing(filename_missing = filename_50_missing_0.4,
  ef = 0.4))
end_time <- Sys.time()
end_time - start_time
```
## 6.3 Effect Size = 0.3

```r
filename_50_missing_0.3 = "run_sims_50_missing_0.3.csv"
start_time <- Sys.time()
sims_missing <- purrr::map DF(1:reps, ~run_sims_missing(filename_missing = filename_50_missing_0.3,
  ef = 0.3))
end_time <- Sys.time()
end_time - start_time
```

### 6.3.1 Visualise Estimates for Fixed Effects:

```r
# read saved simulation data
sims_50_missing_0.3 <- read_csv(filename_50_missing_0.3, col_types = cols(
  # makes sure plots display in this order
group = col_factor(ordered = TRUE),
term = col_factor(ordered = TRUE)
))

fixed_missing_plot <- sims_50_missing_0.3 %>%
  filter(effect == "fixed") %>%
  ungroup() %>%
  arrange(term, estimate) %>%
  mutate(row = rep(seq(1:reps), 8)) %>%
  ggplot(aes(x = row, y = estimate, ymin = estimate-std.error, ymax = estimate+std.error)) +
  facetWrap(~term, scales = "free") +
  geom_pointrange(fatten = 1/2) +
  ylab("Estimates") +
  xlab("Simulations") +
  ggtitle('Estimates of Fixed Effects for 50 pct. Missing Data, ef = 0.3') +
  themeBW()

fixed_missing_plot <- fixed_missing_plot + theme plot.title = element_text(hjust = 0.5, size=20))
fixed_missing_plot
```
Estimates of Fixed Effects for 50 pct. Missing Data, ef = 0.3

<figure><img src="image_27.png" /></figure>

6.3.2 Visualise Estimates for Random Effects:

```
ran_missing_plot <- sims_20_missing_0.3 %>
filter(effect == "ran_pars") %>
ungroup() %>
arrange(group, term, estimate) %>
mutate(row = rep(seq(1:reps), 13)) %>
ggplot(aes(x = row, y = estimate)) + geom_point(alpha = 0.7) +
facetWrap(~group + term, scales = "free_y") + themeBW() +
ylab("Estimates") + xlab("Simulations") + ggtitle("Estimates of Random Effects for 50 pct. Missing Data, ef = 0.3")
themeBW()

ran_missing_plot <- ran_missing_plot + theme plot.title = element_text(hjust = 0.5,
size = 20))

ran_missing_plot
```
Estimates of Random Effects for 50 pct. Missing Data, ef = 0.3

<figure><img src="image_28.png" /><figcaption>Simulations</figcaption></figure>

6.4 Effect Size = 0.2

filename_50_missing_0.2 = "run_sims_50_missing_0.2.csv"
start_time <- Sys.time()
sims_missing <- purrr::map DF(1:reps, ~run_sims_missing(filename_missing = filename_50_missing_0.2,
  ef = 0.2))
end_time <- Sys.time()
end_time - start_time

6.5 Effect Size = 0.1

filename_50_missing_0.1 = "run_sims_50_missing_0.1.csv"
start_time <- Sys.time()
sims_missing <- purrr::map�(df(1:reps, ~run_sims_missing(filename_missing = filename_50_missing_0.1,
```
ef = 0.1))
end_time <- Sys.time()
end_time - start_time
```

# 7 Overview of Power Simulation Results

## 7.1 Summary Statistics for Power Calculation with Full Data and Varying Intercepts and Varying Slopes:

<table><caption>Table 1: Power for Simulations with Full Data and Varying Intercepts and Varying Slopes</caption><thead><tr><th>term</th><th>power, ef = 0.1</th><th>power, ef = 0.2</th><th>power, ef = 0.3</th><th>power, ef = 0.4</th><th>power, ef = 0.5</th></tr></thead><tbody><tr><td>(Intercept)</td><td>0.050</td><td>0.056</td><td>0.070</td><td>0.056</td><td>0.026</td></tr><tr><td>X_a</td><td>0.844</td><td>0.990</td><td>1.000</td><td>1.000</td><td>1.000</td></tr><tr><td>X_c</td><td>0.170</td><td>0.474</td><td>0.838</td><td>0.966</td><td>0.992</td></tr><tr><td>X_f</td><td>0.118</td><td>0.360</td><td>0.634</td><td>0.864</td><td>0.968</td></tr><tr><td>X_a:X_c</td><td>0.690</td><td>0.894</td><td>0.986</td><td>1.000</td><td>1.000</td></tr><tr><td>X_a:X_f</td><td>0.748</td><td>0.864</td><td>0.966</td><td>0.994</td><td>1.000</td></tr><tr><td>X_c:X_f</td><td>0.072</td><td>0.124</td><td>0.244</td><td>0.350</td><td>0.486</td></tr><tr><td>X_a:X_c:X_f</td><td>0.770</td><td>0.808</td><td>0.856</td><td>0.938</td><td>0.956</td></tr></tbody></table>

## 7.2 Summary Statistics for Power Calculation with Full Data and Varying Intercepts:

<table><caption>Table 2: Power for Simulations with Full Data and Varying Intercepts</caption><thead><tr><th>term</th><th>power, ef = 0.1</th><th>power, ef = 0.2</th><th>power, ef = 0.3</th><th>power, ef = 0.4</th><th>power, ef = 0.5</th></tr></thead><tbody><tr><td>(Intercept)</td><td>0.050</td><td>0.056</td><td>0.050</td><td>0.080</td><td>0.058</td></tr><tr><td>X_a</td><td>0.822</td><td>0.994</td><td>1.000</td><td>1.000</td><td>1.000</td></tr><tr><td>X_c</td><td>0.162</td><td>0.482</td><td>0.810</td><td>0.998</td><td>0.966</td></tr><tr><td>X_f</td><td>0.136</td><td>0.352</td><td>0.644</td><td>0.970</td><td>0.884</td></tr><tr><td>X_a:X_c</td><td>0.868</td><td>0.958</td><td>0.990</td><td>1.000</td><td>0.998</td></tr><tr><td>X_a:X_f</td><td>0.848</td><td>0.932</td><td>0.990</td><td>1.000</td><td>0.998</td></tr><tr><td>X_c:X_f</td><td>0.092</td><td>0.136</td><td>0.220</td><td>0.564</td><td>0.348</td></tr><tr><td>X_a:X_c:X_f</td><td>0.798</td><td>0.832</td><td>0.882</td><td>0.964</td><td>0.934</td></tr></tbody></table>

## 7.3 Summary Statistics for Power Calculation with 20 pct. Missing Data and Varying Intercepts and Varying Slopes:

<table><caption>Table 3: Power for Simulations with 20 pct. Missing Data and Varying Intercepts and Slopes</caption><thead><tr><th>term</th><th>power, ef = 0.1</th><th>power, ef = 0.2</th><th>power, ef = 0.3</th><th>power, ef = 0.4</th><th>power, ef = 0.5</th></tr></thead><tbody><tr><td>(Intercept)</td><td>0.040</td><td>0.054</td><td>0.042</td><td>0.064</td><td>0.052</td></tr><tr><td>X_a</td><td>0.832</td><td>0.996</td><td>1.000</td><td>1.000</td><td>1.000</td></tr><tr><td>X_c</td><td>0.162</td><td>0.476</td><td>0.806</td><td>0.950</td><td>0.998</td></tr><tr><td>X_f</td><td>0.122</td><td>0.348</td><td>0.658</td><td>0.842</td><td>0.972</td></tr><tr><td>X_a:X_c</td><td>0.686</td><td>0.928</td><td>0.982</td><td>1.000</td><td>1.000</td></tr><tr><td>X_a:X_f</td><td>0.674</td><td>0.872</td><td>0.974</td><td>0.996</td><td>1.000</td></tr><tr><td>X_c:X_f</td><td>0.062</td><td>0.132</td><td>0.266</td><td>0.376</td><td>0.498</td></tr><tr><td>X_a:X_c:X_f</td><td>0.762</td><td>0.800</td><td>0.862</td><td>0.926</td><td>0.950</td></tr></tbody></table>
## 7.4 Summary Statistics for Power Calculation with 50 pct. Missing Data and Varying Intercepts and Varying Slopes:

<table><caption>Table 4: Power for Simulations with 50 pct. Missing Data and Varying Intercepts and Slopes</caption><thead><tr><th>term</th><th>power, ef = 0.1</th><th>power, ef = 0.2</th><th>power, ef = 0.3</th><th>power, ef = 0.4</th><th>power, ef = 0.5</th></tr></thead><tbody><tr><td>(Intercept)</td><td>0.036</td><td>0.058</td><td>0.054</td><td>0.044</td><td>0.046</td></tr><tr><td>X_a</td><td>0.822</td><td>0.986</td><td>1.000</td><td>1.000</td><td>1.000</td></tr><tr><td>X_c</td><td>0.168</td><td>0.540</td><td>0.840</td><td>0.970</td><td>0.996</td></tr><tr><td>X_f</td><td>0.120</td><td>0.366</td><td>0.658</td><td>0.836</td><td>0.964</td></tr><tr><td>X_a:X_c</td><td>0.666</td><td>0.902</td><td>0.992</td><td>0.998</td><td>1.000</td></tr><tr><td>X_a:X_f</td><td>0.680</td><td>0.826</td><td>0.966</td><td>0.992</td><td>1.000</td></tr><tr><td>X_c:X_f</td><td>0.066</td><td>0.174</td><td>0.232</td><td>0.372</td><td>0.460</td></tr><tr><td>X_a:X_c:X_f</td><td>0.690</td><td>0.754</td><td>0.840</td><td>0.896</td><td>0.942</td></tr></tbody></table>

## 8 Overview of Bias Results

sim_stats_full Bias_int slope %>%
kbl(caption = "Bias for Simulations with Full Data and Varying Intercepts and Varying Slopes",
digits = 3, align = "c") %>%
kable_styling(full_width = T, latex_options = c("striped",
"HOLD_position"))

<table><caption>Table 5: Bias for Simulations with Full Data and Varying Intercepts and Varying Slopes</caption><thead><tr><th>term</th><th>bias, ef = 0.1</th><th>bias, ef = 0.2</th><th>bias, ef = 0.3</th><th>bias, ef = 0.4</th><th>bias, ef = 0.5</th></tr></thead><tbody><tr><td>(Intercept)</td><td>0.000</td><td>0.003</td><td>0.003</td><td>-0.003</td><td>0.003</td></tr><tr><td>X_a</td><td>0.000</td><td>-0.001</td><td>-0.003</td><td>0.002</td><td>-0.003</td></tr><tr><td>X_c</td><td>0.001</td><td>-0.003</td><td>-0.007</td><td>0.000</td><td>-0.004</td></tr><tr><td>X_f</td><td>0.012</td><td>0.002</td><td>0.002</td><td>0.014</td><td>-0.021</td></tr><tr><td>X_a:X_c</td><td>-0.001</td><td>0.004</td><td>0.005</td><td>0.006</td><td>0.006</td></tr><tr><td>X_a:X_f</td><td>-0.007</td><td>0.013</td><td>-0.007</td><td>0.000</td><td>-0.002</td></tr><tr><td>X_c:X_f</td><td>0.024</td><td>-0.001</td><td>-0.015</td><td>0.001</td><td>0.011</td></tr><tr><td>X_a:X_c:X_f</td><td>0.009</td><td>0.003</td><td>0.008</td><td>0.018</td><td>-0.018</td></tr></tbody></table>

sim_stats_full Bias_int %>%
kbl(caption = "Bias for Simulations with Full Data and Varying Intercepts",
digits = 3, align = "c") %>%
kable_styling(full_width = T, latex_options = c("striped",
"HOLD_position"))

<table><caption>Table 6: Bias for Simulations with Full Data and Varying Intercepts</caption><thead><tr><th>term</th><th>bias, ef = 0.1</th><th>bias, ef = 0.2</th><th>bias, ef = 0.3</th><th>bias, ef = 0.4</th><th>bias, ef = 0.5</th></tr></thead><tbody><tr><td>(Intercept)</td><td>0.002</td><td>0.006</td><td>0.001</td><td>0.003</td><td>-0.003</td></tr><tr><td>X_a</td><td>0.001</td><td>0.002</td><td>-0.002</td><td>-0.098</td><td>0.094</td></tr><tr><td>X_c</td><td>0.007</td><td>0.000</td><td>-0.003</td><td>-0.103</td><td>0.095</td></tr><tr><td>X_f</td><td>0.004</td><td>0.012</td><td>-0.001</td><td>-0.094</td><td>0.094</td></tr><tr><td>X_a:X_c</td><td>-0.011</td><td>0.008</td><td>-0.013</td><td>-0.105</td><td>0.105</td></tr><tr><td>X_a:X_f</td><td>-0.003</td><td>0.006</td><td>-0.011</td><td>-0.105</td><td>0.093</td></tr><tr><td>X_c:X_f</td><td>-0.005</td><td>0.014</td><td>-0.005</td><td>-0.136</td><td>0.101</td></tr><tr><td>X_a:X_c:X_f</td><td>0.004</td><td>0.027</td><td>-0.003</td><td>-0.075</td><td>0.113</td></tr></tbody></table>

sim_stats_20_missing Bias %>%
kbl(caption = "Bias for Simulations with 20 pct. Missing Data and Varying Intercepts and Slopes",
digits = 3, align = "c") %>%
kable_styling(full_width = T, latex_options = c("striped",
"HOLD_position"))

<table><caption>Table 7: Bias for Simulations with 20 pct. Missing Data and Varying Intercepts and Slopes</caption>
  <thead>
    <tr>
      <th>term</th>
      <th>bias, ef = 0.1</th>
      <th>bias, ef = 0.2</th>
      <th>bias, ef = 0.3</th>
      <th>bias, ef = 0.4</th>
      <th>bias, ef = 0.5</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>(Intercept)</td>
      <td>0.001</td>
      <td>0.002</td>
      <td>0.002</td>
      <td>-0.005</td>
      <td>0.001</td>
    </tr>
    <tr>
      <td>X_a</td>
      <td>0.010</td>
      <td>0.000</td>
      <td>-0.006</td>
      <td>0.006</td>
      <td>0.001</td>
    </tr>
    <tr>
      <td>X_c</td>
      <td>0.002</td>
      <td>-0.003</td>
      <td>-0.011</td>
      <td>0.007</td>
      <td>0.000</td>
    </tr>
    <tr>
      <td>X_f</td>
      <td>0.000</td>
      <td>0.001</td>
      <td>-0.006</td>
      <td>0.005</td>
      <td>-0.002</td>
    </tr>
    <tr>
      <td>X_a:X_c</td>
      <td>-0.001</td>
      <td>-0.009</td>
      <td>-0.005</td>
      <td>0.004</td>
      <td>-0.014</td>
    </tr>
    <tr>
      <td>X_a:X_f</td>
      <td>0.014</td>
      <td>0.005</td>
      <td>-0.007</td>
      <td>0.001</td>
      <td>0.000</td>
    </tr>
    <tr>
      <td>X_c:X_f</td>
      <td>0.008</td>
      <td>0.002</td>
      <td>-0.008</td>
      <td>0.011</td>
      <td>-0.008</td>
    </tr>
    <tr>
      <td>X_a:X_c:X_f</td>
      <td>0.009</td>
      <td>-0.014</td>
      <td>-0.003</td>
      <td>0.003</td>
      <td>0.020</td>
    </tr>
  </tbody>
</table>

sim_stats_50_missing Bias %>

kbl(caption = "Bias for Simulations with 50 pct. Missing Data and Varying Intercepts and Slopes",
     digits = 3, align = "c") %>

kable_styling(full_width = T, latex_options = c("striped",
"HOLD_position"))

<table><caption>Table 8: Bias for Simulations with 50 pct. Missing Data and Varying Intercepts and Slopes</caption>
  <thead>
    <tr>
      <th>term</th>
      <th>bias, ef = 0.1</th>
      <th>bias, ef = 0.2</th>
      <th>bias, ef = 0.3</th>
      <th>bias, ef = 0.4</th>
      <th>bias, ef = 0.5</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>(Intercept)</td>
      <td>-0.006</td>
      <td>-0.003</td>
      <td>0.002</td>
      <td>0.000</td>
      <td>-0.003</td>
    </tr>
    <tr>
      <td>X_a</td>
      <td>-0.002</td>
      <td>0.002</td>
      <td>-0.004</td>
      <td>0.004</td>
      <td>0.000</td>
    </tr>
    <tr>
      <td>X_c</td>
      <td>0.006</td>
      <td>-0.011</td>
      <td>0.000</td>
      <td>-0.005</td>
      <td>0.000</td>
    </tr>
    <tr>
      <td>X_f</td>
      <td>-0.003</td>
      <td>-0.008</td>
      <td>0.007</td>
      <td>0.016</td>
      <td>0.005</td>
    </tr>
    <tr>
      <td>X_a:X_c</td>
      <td>0.000</td>
      <td>-0.002</td>
      <td>0.002</td>
      <td>0.004</td>
      <td>-0.007</td>
    </tr>
    <tr>
      <td>X_a:X_f</td>
      <td>0.004</td>
      <td>0.004</td>
      <td>0.010</td>
      <td>0.012</td>
      <td>0.007</td>
    </tr>
    <tr>
      <td>X_c:X_f</td>
      <td>0.010</td>
      <td>-0.031</td>
      <td>0.029</td>
      <td>-0.012</td>
      <td>0.014</td>
    </tr>
    <tr>
      <td>X_a:X_c:X_f</td>
      <td>0.000</td>
      <td>0.016</td>
      <td>0.000</td>
      <td>0.020</td>
      <td>-0.009</td>
    </tr>
  </tbody>
</table>

9 Increasing Variance with Trial Number

9.1 Extracting estimates of increasing variance from the adult pilot data

# download data from adult pilot study:
adult_data <- read.csv("02_process

stimulus_info df <- adult_data %>
filter(phase == "pref") %>
mutate(familiar_item = case_when(left_stimulus == "familiar" ~
stimulus_process
stimulus_process
"novel" ~ stimulus_process
"novel" ~ stimulus_process
select(subject, block_number, familiar_item, novel_item) %>
distinct(subject, block_number, .keep_all = TRUE)

lt_comparison df <- adult_data %>
filter(phase == "pref") %>
group_by(subject, exposure_time, block_number, complexity,
gaze_location_type) %>
summarise(sum_dwell_time = sum(dwell_time)) %>
pivot_wider(names_from = gaze_location_type, values_from = sum_dwell_time) %>
ungroup() %>
```r
left_join(stimulus_info df, by = c("subject", "block_number")) %>%
  rename(familiarization_time = exposure_time, stimulus_complexity = complexity,
        trial_number = block_number, familiar_lookout_time = familiar,
        novel_lookout_time = novel, participant = subject) %>%
  # currently excluding all the empty trials
filter(!is.na(familiar_lookout_time) & !is.na(novel_lookout_time))
```

```r
## `summarise()` has grouped output by 'subject', 'exposure_time', 'block_number',
## 'complexity'. You can override using the `.groups` argument.
```

```r
lt_comparison df <- lt_comparison df %>%
  mutate(familiarization_time彦 scaled彦 centered = log10(familiarization_time) -
        log10(2000), contrast_coded_complexity = ifelse(stimulus_com complexity ==
        "complex", 0.5, -0.5))
```

```r
proportion df <- adult_data %>%
  filter(phase == "pref") %>%
  group_by(block_number, exposure_time, complexity, gaze彦 location彦 type,
    subject) %>%
  summarise(sum_dwell_time = sum(dwell_time, na.rm = TRUE)) %>%
  pivot彦 wider(names彦 from = gaze彦 location彦 type, values彦 from = sum_dwell_time) %>%
  mutate(familiar = ifelse(is.na(familiar), 0, familiar), novel = ifelse(is.na(novel),
    0, novel)) %>%
  mutate(novelty彦 looking彦 proportion = novel/(familiar + novel))
```

```r
## `summarise()` has grouped output by 'block_number', 'exposure_time',
## 'complexity', 'gaze彦 location彦 type'. You can override using the `.groups`
## argument.
```

```r
dwell彦 proportion彦 mean <- mean(proportion df $novelty彦 looking彦 proportion,
    na.rm = T)
dwell彦 proportion彦 sd <- sd(proportion df $novelty彦 looking彦 proportion,
    na.rm = T)
```

```r/adult彦 data彦 z <- proportion df %>%
  mutate(dwell彦 time彦 z = (novelty彦 looking彦 proportion - dwell彦 proportion彦 mean)/dwell彦 proportion彦 sd)

data彦 on彦 increase彦 in彦 sd <- adult彦 data彦 z %>%
  group彦 by彦 (block彦 number) %>%
  summarise(n = n(), dwell彦 time彦 median = median(dwell彦 time彦 z,
    na.rm = T), sd彦 block彦 = sd(dwell彦 time彦 z, na.rm = T))

increase彦 sd彦 value <- data彦 on彦 increase彦 in彦 sd $sd彦 block彦 [12] - data彦 on彦 increase彦 in彦 sd $sd彦 block彦 [1]
increase彦 sd彦 value
```

## [1] 0.3098641

## 9.2 Visualisation of adult data in proportions

```r
adult proportion plot <- proportion df %>%
  ggplot(aes(x = as.factor(block_number), y = novelty彦 looking彦 proportion,
    color = complexity)) + geom_jitter(alpha = 0.3, width = 0.2) +
  stat_summary(aes(x = as.factor(block_number), y = novelty彦 looking彦 proportion,
    color = complexity), position = position彦 dodge(width = 0.2)) +
  geom_hline(yintercept = 0.5) + scale_color_manual(values = viridis(n = 3)) +
  ggtitle("Adult Pilot Data in Proportions") + themeBW()
adult proportion plot <- adult proportion plot + theme( plot.彦 title = element彦 text(hjust = 0.5,
    size = 20))
adult proportion plot
```
Adult Pilot Data in Proportions

<figure><img src="image_29.png" /></figure>

9.3 Visualisation of adult data in z-score

adult_data_z_plot <- adult_data_z %>%
ggplot(aes(x = as.factor(block_number), y = dwell_time_z,
           color = complexity)) + geom_jitter(alpha = 0.3, width = 0.2) +
stat_summary(aes(x = as.factor(block_number), y = dwell_time_z,
               color = complexity), position = position_dodge(width = 0.2)) +
geom_hline(yintercept = 0) + scale_color_manual(values = viridis(n = 3)) +
ggtitle("Adult Pilot Data in z-scores") + themeBW()

adult_data_z Plot <- adult_data_z Plot + theme(plot.title = element_text(hjust = 0.5,
size = 20))
adult_data_z Plot
Adult Pilot Data in z-scores

<figure><img src="image_30.png" /></figure>

9.4 Modification of simulation function to include variance increase

my_sim_data_Variance <- function(
  increase_sd = increase_sd_value, # Add variance to increase with trial number
  n_subj = 1280, # number of subjects
  n_simple = 12, # number of complex stimuli
  n_complex = 12, # number of complex stimuli
  n_small_fam = 8, # small familiarization time
  n_medium_fam = 8, # medium familiarization time
  n_high_fam = 8, # high familiarization time
  n_lab = 40,
  beta_0 = 0, # intercept; i.e., the grand mean
  beta_c = 0.3, # main effect for complexity
  beta_f = 0.3, # main effect for familiarization time
)
beta_a = 0.3, # main effect for age

beta_ca = 0.3,

beta_af = 0.3,

betaCF = 0.3,

beta_cfa = 0.3, #main effect for interaction between complexity and familiarization.

subject_0 = 0.2, # by-subject random intercept sd

subject_c = 0.2, # by-subject slope complexity sd

subject_f = 0.2, # by-subject slope familiarization sd

subject_a = 0.2, # by-subject slope age sd

subject_ca = 0.2,# by-subject slope for interaction between age and complexity sd

subject_af = 0.2, # by-subject slope for interaction between age and familiarization sd

subjectCF = 0.2, # by-subject slope complexity*familiarization sd

subject_cfa = 0.2, # by-subject slope for interaction between age, complexity and familiarization sd

subj_rho = .2, # correlations between by-subject random effects

lab_0 = 0.2, # by-lab random intercept sd

lab_c = 0.2, # by-lab slope complexity sd

lab_f = 0.2, # by-lab slope familiarization sd

lab_a = 0.2, # by-lab slope age sd

lab_ca = 0.2, # by-lab slope for interaction between age and complexity sd

lab_af = 0.2, # by-lab slope for interaction between age and familiarization sd

labCF = 0.2, # by-lab random slope complexity*familiarization sd

lab_cfa = 0.2, # by-lab slope for interaction between age, complexity and familiarization sd

lab_rho = 0.2, # correlations between by-lab random effects

item_0 = 0.2, # by-item random intercept sd

item_c = 0.2, # by-item slope complexity sd

item_f = 0.2, # by-item slope familiarization sd

item_a = 0.2, # by-item slope age sd

item_ca = 0.2, # by-item slope for interaction between age and complexity sd

item_af = 0.2, # by-item slope for interaction between age and familiarization sd

itemCF = 0.2, # by-item random slope complexity*familiarization sd

item_cfa = 0.2, # by-item slope for interaction between age, complexity and familiarization sd

item_rho = 0.2, # correlations between by-item random effects

sigma = 0.3 # residual (error) sd
) { # residual (standard deviation)

# simulate a sample of items

items <- data.frame(
  item_id = seq_len(n_simple + n_complex),
  category = rep(c("simple", "complex"), c(n_simple, n_complex)),
  X_c = rep(c(-0.5, 0.5), c(n_simple, n_complex)),
  familiarization = rep(c("short", "medium", "long"), (n_simple + n_complex)/3),
  X_f = rep(c(-0.5, 0, 0.5), (n_simple + n_complex)/3),
  faux::rnorm_multi(
```r
n = n_simple + n_complex, mu = 0, sd = c(item_0,
                                           item_c,
                                           item_f,
                                           item_a,
                                           item_ca,
                                           item_af,
                                           item_cf,
                                           item_cfa), r = item_rho,
varnames = c("I_0", "I_c", "I_f", "I_a",
             "I_ca", "I_af", "I_cf",
             "I_cfa")
)
```

```r
) %>
mutate(item_id = faux::make_id(nrow(.), "I"))
```

```r
# simulate a sample of subjects
subjects <-
  faux::rnorm_multi(
    n = n_subj, mu = 0, sd = c(subject_0,
                                 subject_c,
                                 subject_f,
                                 subject_a,
                                 subject_ca,
                                 subject_af,
                                 subject_cf,
                                 subject_cfa), r = subj_rho,
    varnames = c("S_0", "S_c", "S_f", "S_a",
                 "S_ca", "S_af", "S_cf",
                 "S_cfa")
  ) %>
mutate(subj_id = faux::make_id(nrow(.), "S")) %>
mutate(X_a = runif(n_subj, min = -0.5, max = 0.5))
```

```r
#add subject age measure, sample from distribution from -0.5 to 0.5. #subjects$subj_id <- 1:n_subj
labs <- faux::rnorm_multi(
  n = n_lab, mu = 0, sd = c(lab_0, lab_c, lab_f, lab_a,
                           lab_ca, lab_af, lab_cf,
                           lab_cfa), r = lab_rho,
  varnames = c("L_0", "L_c", "L_f", "L_a",
               "L_ca", "L_af", "L_cf",
               "L_cfa")
)
```

```r
) %>
mutate(lab_id = faux::make_id(nrow(.), "L"))
```

```r
# Number of subjects must be a multiple of number of labs
lab_multiplier = n_subj/n_lab
lab_subj_dict <- data.frame(
  subj_id = subjects$subj_id,
  lab_id = rep(labs$lab_id,lab_multiplier)
)
```

```r
) %>
mutate(lab_id = faux::make_id(nrow(.), "L"))
```

```r
# create lab and subj nesting structure
lab_multiplier = n_subj/n_lab
```

```r
# Number of subjects must be a multiple of number of labs
lab_multiplier = n_subj/n_lab
lab_subj_dict <- data.frame(
  subj_id = subjects$subj_id,
  lab_id = rep(labs$lab_id,lab_multiplier)
)
```

```r
) %>
mutate(trial_num = rep(seq(n_simple + n_complex), n_subj))
```

```r
# cross subject and item IDs
temp <- crossing(subjects, items) %>
  left_join(lab_subj_dict, by = "subj_id") %>
  left_join(labs, by = "lab_id") %>
  group_by(subj_id, item_id) %>
  mutate(item_id = sample(item_id)) %>
  ungroup() %>
  mutate(trial_num = rep(seq(n_simple + n_complex), n_subj))
```

```r
temp <- temp %>
  left_join(lab_subj_dict, by = "subj_id") %>
  left_join(labs, by = "lab_id") %>
  group_by(subj_id, item_id) %>
  mutate(item_id = sample(item_id)) %>
  ungroup() %>
  mutate(trial_num = rep(seq(n_simple + n_complex), n_subj))
```

```r
temp <- temp %>
  left_join(lab_subj_dict, by = "subj_id") %>
  left_join(labs, by = "lab_id") %>
  group_by(subj_id, item_id) %>
  mutate(item_id = sample(item_id)) %>
  ungroup() %>
  mutate(trial_num = rep(seq(n_simple + n_complex), n_subj))
```
```r
temp %>%
  mutate(
    B_0 = beta_0 + S_0 + L_0 + I_0,
    B_c = beta_c + S_c + L_c + I_c,
    B_f = beta_f + S_f + L_f + I_f,
    B_a = beta_a + S_a + L_a + I_a,
    B_ca = beta_ca + S_ca + L_ca + I_ca,
    B_af = beta_af + S_af + L_af + I_af,
    BCF = betaCF + S_CF + L_CF + I_CF,
    B_cfa = beta_cfa + S_cfa + L_cfa + I_cfa,
    e si = rnorm(nrow(temp), mean = 0, sd = sigma) + trial_i_var,
    DV = B_0 +
      (B_a * X_a) + (B_c * X_c) + (B_f * X_f) +
      (B_CF * X_c * X_f) + (B_af * X_a * X_f) + (B_ca * X_c * X_a) +
      (B_cfa * X_c * X_f * X_a) + e si
  )
}

dat_sim <- my_sim_data_Variance()
```

## 9.5 Visualise the variance increase

```r
# The increase in variance is over 24 trials in this
# ManyBabies5 case:
dat_sim <- my_sim_data_Variance(increase_sd = increase_sd_value)

plot帅num <- dat_sim %>%
  mutate(trial_num = as.factor(trial_num)) %>%
  ggplot() + geom_point(aes(y = DV, x = trial_num), position = "jitter",
    alpha = 0.2, size = 0.2) + geom_violin(aes(y = DV, x = trial_num,
    fill = trial_num), alpha = 0.2, show.legend = FALSE) + scale_fill_manual(values = viridis(n = 24)) +
  ggtitle("Increase in Variance across Trial Number") + xlab("Trial Number") +
  themeBW()

plot帅num <- plot帅num + theme plot.title = element_text(hjust = 0.5,
  size = 20))

plot帅num
```
Increase in Variance across Trial Number

<figure><img src="image_31.png" /></figure>

9.6 Model building and bias assessment

```
dat_sim <- my_sim_data(increase_sd = increase_sd_value)
```

```
mod_sim <- lmer(DV ~ 1 + X_a * X_c * X_f + (1 | subj_id) + (1 | lab_id) + (1 | item_id), data = dat_sim)
```

```
summary(mod_sim)
```

```
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
```

```
## lmerModlmerTest]
```

```
## Formula: DV ~ 1 + X_a * X_c * X_f + (1 | subj_id) + (1 | lab_id) + (1 |
```

```
## item_id)
```

```
## Data: dat_sim
```

```
##
```
## REML criterion at convergence: 35957.5

##

## Scaled residuals:

## Min 1Q Median 3Q Max

## -4.6460 -0.6486 -0.0005 0.6571 4.5024

##

## Random effects:

## Groups Name Variance Std.Dev.

## subj_id (Intercept) 0.04078 0.2019

## lab_id (Intercept) 0.02656 0.1630

## item_id (Intercept) 0.06894 0.2626

## Residual 0.17278 0.4157

## Number of obs: 30720, groups: subj_id, 1280; lab_id, 40; item_id, 24

##

## Fixed effects:

## Estimate Std. Error df t value Pr(>|t|)

## (Intercept) 2.737e-02 5.978e-02 2.993e+01 0.458 0.65040

## X_a 3.720e-01 2.133e-02 1.243e+03 17.437 < 2e-16 ***

## X_c 5.855e-01 1.073e-01 2.000e+01 5.457 2.43e-05 ***

## X_f 4.687e-01 1.314e-01 2.000e+01 3.567 0.00193 **

## X_a:X_c 1.976e-01 1.627e-02 2.941e+04 12.144 < 2e-16 ***

## X_a:X_f 3.844e-01 1.992e-02 2.941e+04 19.294 < 2e-16 ***

## X_c:X_f 6.772e-01 2.628e-01 2.000e+01 2.577 0.01801 *

## X_a:X_c:X_f 7.447e-01 3.985e-02 2.941e+04 18.690 < 2e-16 ***

## ---

## Signif. codes: 0 '**' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

##

## Correlation of Fixed Effects:

## (Intr) X_a X_c X_f X_a:X_c X_c:X_f X_c:X_f

## X_a -0.002

## X_c 0.000 0.000

## X_f 0.000 0.000 0.000

## X_a:X_c 0.000 0.000 -0.001 0.000

## X_a:X_f 0.000 0.000 0.000 -0.001 0.000

## X_c:X_f 0.000 0.000 0.000 0.000 0.000 0.000 0.000

## X_a:X_c:X_f 0.000 0.000 0.000 0.000 0.000 0.000 0.000 -0.001

plot(mod_sim)
<figure><img src="image_32.png" /></figure>

qqnorm(resid(mod_sim))

qqline(resid(mod_sim))
Normal Q-Q Plot

<figure><img src="image_33.png" /></figure>

check_normality(mod_sim)

## OK: residuals appear as normally distributed (p = 0.250).

check_outliers(mod_sim)

## OK: No outliers detected.

## - Based on the following method and threshold: cook (0.92).

## - For variable: (Whole model)

check_collinearity(mod_sim)

## # Check for Multicollinearity

##

## Low Correlation

##
Term VIF VIF 95% CI Increased SE Tolerance Tolerance 95% CI

| Term | VIF | VIF 95% CI | Increased SE | Tolerance | Tolerance 95% CI |
|---|---|---|---|---|---|
| X_a | 1.00 | [1.00, ] | 1.00 | 1.00 | [ , 1.00] |
| X_c | 1.00 | [1.00, Inf] | 1.00 | 1.00 | [0.00, 1.00] |
| X_f | 1.00 | [1.00, Inf] | 1.00 | 1.00 | [0.00, 1.00] |
| X_a:X_c | 1.00 | [1.00, Inf] | 1.00 | 1.00 | [0.00, 1.00] |
| X_a:X_f | 1.00 | [1.00, Inf] | 1.00 | 1.00 | [0.00, 1.00] |
| X_c:X_f | 1.00 | [1.00, Inf] | 1.00 | 1.00 | [0.00, 1.00] |
| X_a:X_c:X_f | 1.00 | [1.00, Inf] | 1.00 | 1.00 | [0.00, 1.00] |

check_heteroscedasticity(mod_sim)

## Warning: Heteroscedasticity (non-constant error variance) detected (p = 0.014).

## 9.7 Test how many of the models violate homoskedasticity

reps <- 200

run_sims_heteroskedasticity <- function(filename_heteroskedasticity) {

dat_sim <- my_sim_data-variance(increase_sd = increase_sd_value)

mod_sim <- lmer(DV ~ 1 + X_a * X_c * X_f + (1 | subj_id) +
(1 | lab_id) + (1 | item_id), data = dat_sim)

heteroskedasticity_results <- as_tibble(check_heteroscedasticity(mod_sim)[1])

# append the results to a file

append <- file.exists(filename_heteroskedasticity)

write_csv(heteroskedasticity_results, filename_heteroskedasticity,
append = append)

# return the tidy table

heteroskedasticity_results

filename_heteroskedasticity = "sims/filename_heteroskedasticity.csv"

start_time <- Sys.time()

heteroskedasticity_results_data <- purrr::map DF(1:reps, ~run_sims_heteroskedasticity(filename_heteroskedasticity)

end_time <- Sys.time()

end_time - start_time

## 9.8 Overview of how many of the models exhibit homoskedasticity

heteroskedasticity_results <- read.csv(filename_heteroskedasticity)

homoskedastic_models <- heteroskedasticity_results %>%
  filter(value > 0.05)

print(paste0(nrow(homoskedastic_models), "/", nrow(heteroskedasticity_results),
" of the models exhibit homoskedasticity"))

## [1] "156/200 of the models exhibit homoskedasticity"

## 9.9 Bayesian robust location-scale regression model
```
model_formula <- bf(DV ~ 1 + X_a * X_c * X_f +
                   (1 | subj_id) + (1 | lab_id) + (1 | item_id),
                   sigma ~ 1 + trial_num + (1 | subj_id) + (1 | lab_id) + (1 | item_id))
```

```r
#get prior(model_formula, data = dat_sim, family = student)
```

```
priors1 <- c(prior(normal(0, 0.5), class = Intercept),
            prior(normal(0, 0.5), class = b),
            prior(normal(0, 0.5), class = b, dpar = sigma),
            prior(normal(0.25, 0.3), class = sd),
            prior(gamma(2, 0.1), class = nu)
)
```

```
heteroskedasticity_fit <-
  brm(data = dat_sim,
    family = student,
    model_formula,
    prior = priors1,
    sampleaman = "yes",
    iter = 4000,
    warmup = 500,
    #backend = "cmdstanr",
    #threads = threading(2),
    file = "heteroskedasticity_fit",
    cores = 64,
    chains = 2,
    save_pars = save_pars(all = TRUE))
```

```r
pp_check(heteroskedasticity_fit, ndraws = 100)
```
<figure><img src="image_34.png" /></figure>

heteroskedasticity_fit

## Family: student

## Links: mu = identity; sigma = log; nu = identity

## Formula: DV ~ 1 + X_a * X_c * X_f + (1 | subj_id) + (1 | lab_id) + (1 | item_id)

## sigma ~ 1 + trial_num + (1 | subj_id) + (1 | lab_id) + (1 | item_id)

## Data: dat_sim (Number of observations: 30720)

## Draws: 2 chains, each with iter = 4000; warmup = 500; thin = 1;

## total post-warmup draws = 7000

##

## Group-Level Effects:

## ~item_id (Number of levels: 24)

## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS

## sd(Intercept) 0.00 0.00 0.00 0.01 1.00 3967 4291

## sd(sigma_Intercept) 0.01 0.01 0.00 0.02 1.00 3111 3651

##

## ~lab_id (Number of levels: 40)
## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sd(Intercept) 0.17 0.02 0.13 0.22 1.00 1881 3066
## sdsigma_Intercept) 0.04 0.01 0.03 0.06 1.00 3176 5234

##

## ~subj_id (Number of levels: 1280)

## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sd(Intercept) 0.21 0.01 0.20 0.22 1.00 2481 3755
## sd(

##

## Population-Level Effects:

## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## Intercept -0.01 0.03 -0.06 0.05 1.00 1051 1816
## sigma_Intercept -1.07 0.01 -1.09 -1.04 1.00 6208 5953
## X_a 0.26 0.02 0.22 0.31 1.00 2180 3628
## X_c 0.30 0.01 0.29 0.32 1.00 11577 5326
## X_f 0.39 0.01 0.38 0.40 1.00 14033 5762
## X_a:X_c 0.45 0.02 0.42 0.49 1.00 17307 5375
## X_a:X_f 0.67 0.02 0.62 0.71 1.00 14751 5860
## X_c:X_f 0.73 0.01 0.70 0.75 1.00 16307 5246
## X_a:X_c:X_f 0.36 0.04 0.27 0.44 1.00 15476 5133
## sigmaowealnum 0.02 0.00 0.02 0.02 1.00 7882 5137

##

## Family Specific Parameters:

## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## nu 81.74 19.02 52.07 125.80 1.00 11984 5331

##

## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS

## and Tail_ESS are effective sample size measures, and Rhat is the potential

## scale reduction factor on split chains (at convergence, Rhat = 1).

9.9.1 Prior-Posterior Update Checks

# Sample the parameters of interest:

Posterior_m1 <- as

# Plot the prior-posterior update plot for the intercept:

ggplot( posterior_m1 ) + geom_density( aes( prior_Intercept ), fill = "steelblue ", color = "black", alpha = 0.6 ) + geom_density( aes( b_Intercept ), fill = "#FC4E07", color = "black", alpha = 0.6 ) + theme_classic()
<figure><img src="image_35.png" /><figcaption>```r
ggplot(Posterior_m1) + geom_density(aes(prior_bsigma), fill = "steelblue",
                                  color = "black", alpha = 0.6) + geom_density(aes(bsigma_trial_num),
                                  fill = "#FC4E07", color = "black", alpha = 0.6) + theme_classic()
```</figcaption></figure>
<figure><img src="image_36.png" /><figcaption>```r
ggplot(Posterior_m1) + geom_density(aes(prior_sd_item_id), fill = "steelblue",
  color = "black", alpha = 0.6) + geom_density(aes(sd_subj_id__Intercept),
  fill = "#FC4E07", color = "black", alpha = 0.6) + geom_density(aes(sd_lab_id__Intercept),
  fill = "#FC4E07", color = "black", alpha = 0.6) + geom_density(aes(sd_item_id__Intercept),
  fill = "#FC4E07", color = "black", alpha = 0.6) + theme_classic()
```</figcaption></figure>
<figure><img src="image_37.png" /></figure>

```r
ggplot(Posterior_m1) + geom_density(aes(prior_b), fill = "steelblue",
  color = "black", alpha = 0.6) + geom_density(aes(b_X_a),
  fill = "#FC4E07", color = "black", alpha = 0.6) + geom_density(aes(b_X_c),
  fill = "#FC4E07", color = "black", alpha = 0.6) + geom_density(aes(b_X_f),
  fill = "#FC4E07", color = "black", alpha = 0.6) + theme_classic()
```
<figure><img src="image_38.png" /></figure>

```r
ggplot(Posterior_m1) + geom_density(aes(prior_b), fill = "steelblue",
  color = "black", alpha = 0.6) + geom_density(aes(`b_X_a:X_c`),
  fill = "#FC4E07", color = "black", alpha = 0.6) + geom_density(aes(`b_X_a:X_f`),
  fill = "#FC4E07", color = "black", alpha = 0.6) + geom_density(aes(`b_X_c:X_f`),
  fill = "#FC4E07", color = "black", alpha = 0.6) + geom_density(aes(`b_X_a:X_c:X_f`),
  fill = "#FC4E07", color = "black", alpha = 0.6) + theme_classic()
```
<figure><img src="image_39.png" /><figcaption>```r
ggplot(Posterior_m1) + geom_density(aes(prior_nu), fill = "steelblue",
                                  color = "black", alpha = 0.6) + geom_density(aes(nu), fill = "#FC4E07",
                                  color = "black", alpha = 0.6) + theme_classic()
```</figcaption></figure>
<figure><img src="image_40.png" /></figure>

# 10 Data missing not completely at random

## 10.1 Simulation of missing data according to increasing infant age

```r
# back to assumptions of equal variance across trials:
dat_sim <- my_sim_data()
```

```r
# Proportion of missing data increases with age:
missingSamplesAge <- dat_sim %>
mutate(nas = rbinom(n(), 1, 0.95 - ifelse(X_a > -0.5, (X_a + 0.5) * 0.5, 0))) %>
mutate(DV = ifelse(nas == 1, DV, NA))
```

```r
missing_age_plot <- missingSamplesAge %>
drop_na() %>
```
```r
ggplot() + ggtitle("DV as a function of age, missing data") +
  geom_point(aes(x = X_a, y = DV), alpha = 0.3, size = 0.2,
  position = "jitter") + geomsmooth(aes(x = X_a, y = DV),
  method = "lm", se = TRUE, formula = y ~ x) + themeBW()

missing_age_plot <- missing_age_plot + theme plot.title = element_text(hjust = 0.5,
size = 20))
missing_age_plot
```

DV as a function of age, missing data

<figure><img src="image_41.png" /></figure>

10.1.1 Simulation of models

\# Number of simulations:

reps <- 100

\# Simulation function:
```r
run_sims <- function(filename_full, ef) {
  dat_sim <- my_sim_data(
    beta_c = ef,
    beta_f = ef,
    beta_a = ef,

    beta_ca = ef,
    beta_af = ef,
    beta Cf = ef,

    beta_cfa = ef
  )

  missingSamples_age <- dat_sim %>%
    mutate(nas = rbinom(n(), 1, 0.95 - ifelse(X_a > -0.5, (X_a+0.5)*0.50, 0))) %>%
    mutate(DV = ifelse(nas == 1, DV, NA))

  mod_sim <- lmer(DV ~ 1 + X_a * X_c * X_f +
    (1 | subj_id) +
    (1 | lab_id) +
    (1 | item_id),
  data=missingSamples_age)

  sim_results <- broom.mixed::tidy(mod_sim)

  # append the results to a file
  append <- file.exists(filename_full)
  write_csv(sim_results, filename_full, append = append)

  # return the tidy table
  sim_results
}

filename_full_0.3_missing_age = 'sims/run_sims_0.3_age_missing.csv'
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_0.3_missing_age, ef = 0.3))
end_time <- Sys.time()
end_time - start_time
```

### 10.1.2 Visualise Estimates for Fixed Effects:

```r
# read saved simulation data
sims_50_missing_age_0.3 <- read_csv(filename_full_0.3_missing_age, col_types = cols(
  # makes sure plots display in this order
group = col_factor(ordered = TRUE),
term = col_factor(ordered = TRUE)
))

reps <- 100

fixed_missing_age_plot <- sims_50_missing_age_0.3 %>%
  filter(effect == "fixed") %>%
  ungroup() %>%
  arrange(term, estimate) %>%
  mutate(row = rep(seq(1:reps), 8)) %>%
  ggplot(aes(x = row, y = estimate, ymin = estimate-std.error, ymax = estimate+std.error)) +
  facetWrap(~term, scales = "free") +
  geom_pointrange(fatten = 1/2) +
  ylab("Estimates") +
  xlab("Simulations") +
  ggtitle('Estimates of Fixed Effects for Missing Data with Age, ef = 0.3') +
```
themeBW()

fixed_missing_age_plot <- fixed_missing_age_plot + theme( plot.title = element_text(hjust = 0.5, size=20))
fixed_missing_age plot

Estimates of Fixed Effects for Missing Data with Age, ef = 0.3

<figure><img src="image_42.png" /></figure>

10.1.3 Visualise Estimates for Random Effects:

ran_missing_age_plot <- sims_50_missing_age_0.3 %>
filter(effect == "ran_pars") %>
ungroup() %>
arrange(group, term, estimate) %>
mutate(row = rep(seq(1:reps), 4)) %>
ggplot(aes(x = row, y = estimate)) + geom_point(alpha = 0.7) +
facetWrap(~group + term, scales = "free_y") + themeBW() +
ylab("Estimates") + xlab("Simulations") + ggtitle("Estimates of Random Effects for Missing Data with Age, ef = 0.3")
```r
theme bw()

ran_missing_age_plot <- ran_missing_age_plot + theme plot.title = element_text(hjust = 0.5, size = 20))

ran_missing_age plot
```

## Estimates of Random Effects for Missing Data with Age, ef = 0.3

<figure><img src="image_43.png" /></figure>

## 10.2 Simulation of missing data according to increasing trial number

```r
# Proportion of missing data increases with trial number:
missingSamplesTrial <- dat_sim %>
  mutate(nas = rbinom(n(), 1, 0.95 - ifelse(trial_num > 0,
                                         trial_num * 0.025, 0))) %>
  mutate(DV = ifelse(nas == 1, DV, NA))

missingTrialnumPlot <- missingSamplesTrial %>
  drop_na() %>
```
```r
ggplot() + geom_point(aes(x = trial_num, y = DV), alpha = 0.3,
size = 0.2, position = "jitter") + geomsmooth(aes(x = trial_num,
y = DV), method = "lm", se = TRUE, formula = y ~ x) + ggtitle("DV as a function of trial number, missing data")
themeBW()

missing帅num plot <- missing帅num plot + theme plot.title = element_text(hjust = 0.5,
size = 20))
missing帅num plot
```

<figure><img src="image_44.png" /><figcaption>DV as a function of trial number, missing data</figcaption></figure>

10.2.1 Simulation of models

\# Number of simulations:

reps <- 100

\# Simulation function:
```r
run_sims <- function(filename_full, ef) {
  dat_sim <- my_sim_data(
    beta_c = ef,
    beta_f = ef,
    beta_a = ef,
    beta_ca = ef,
    beta_af = ef,
    beta Cf = ef,
    beta_cfa = ef
  )
  missingSamplesTrial <- dat_sim %>%
    mutate(nas = rbinom(n(), 1, 0.95 - ifelse(trial_num > 0, trial_num*0.025, 0))) %>%
    mutate(DV = ifelse(nas == 1, DV, NA))
  mod_sim <- lmer(DV ~ 1 + X_a * X_c * X_f +
    (1 | subj_id) +
    (1 | lab_id) +
    (1 | item_id),
    data=missingSamplesTrial)
  sim_results <- broom.mixed::tidy(mod_sim)
  # append the results to a file
  append <- file.exists(filename_full)
  write_csv(sim_results, filename_full, append = append)
  # return the tidy table
  sim_results
}

filename_full_0.3_missing trial = 'sims/run_sims_0.3 trial_missing.csv'
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_0.3_missing trial, ef = 0.3))
end_time <- Sys.time()
end_time - start_time
```

### 10.2.2 Visualise Estimates for Fixed Effects:

```r
# read saved simulation data
sims_50_missing trial_0.3 <- read_csv(filename_full_0.3_missing trial, col_types = colS(
  # makes sure plots display in this order
group = col_factor(ordered = TRUE),
term = col_factor(ordered = TRUE)
))

fixed_missing trial plot <- sims_50_missing trial_0.3 %>%
  filter(effect == "fixed") %>%
  ungroup() %>%
  arrange(term, estimate) %>%
  mutate(row = rep(seq(1:reps), 8)) %>%
  ggplot(aes(x = row, y = estimate, ymin = estimate-std.error, ymax = estimate+std.error)) +
  facetWrap(~term, scales = "free") +
  geom_pointrange(fatten = 1/2) +
  ylab("Estimates") +
  xlab("Simulations") +
  ggtitle('Estimates of Fixed Effects for Missing Data with Trial Number, ef = 0.3') +
  themeBW()
```
fixed_missing trial plot <- fixed_missing trial plot + theme ( plot.title = element_text ( hjust = 0.5, size = 20))
fixed_missing trial plot

Estimates of Fixed Effects for Missing Data with Trial Number, ef = 0.3

<figure><img src="image_45.png" /></figure>

10.2.3 Visualise Estimates for Random Effects:

ran_missing帅 trial plot <- sims_50_missing帅 trial_0.3 %>
filter(effect == "ran_pars") %>
ungroup() %>
arrange(group, term, estimate) %>
mutate(row = rep(seq(1:reps), 4)) %>
ggplot(aes(x = row, y = estimate)) + geom_point(alpha = 0.7) +
facetWrap(~group + term, scales = "free_y") + themeBW() +
ylab("Estimates") + xlab("Simulations") + ggtitle("Estimates of Random Effects for Missing Data with Trial, e
themeBW()
ran_missing帅 trial plot <- ran_missing帅 trial plot + theme (plot.title = element_text(hjust = 0.5,
```r
size = 20))
ran_missing帅 trial plot
```

<figure><img src="image_46.png" /><figcaption>Estimates of Random Effects for Missing Data with Trial, ef = 0.3</figcaption></figure>

10.3 Simulation of missing data according to increasing familiarisation

# Proportion of missing data increases with

# familiarisation:

```r
missingSamples_fam <- dat_sim %>%
  mutate(nas = rbinom(n(), 1, 0.95 - ifelse(X_f > -0.5, (X_f +
    0.5) * 0.6, 0))) %>%
  mutate(DV = ifelse(nas == 1, DV, NA))
```

```r
missingSamples_fam %>%
  mutate(X_f = as.factor(X_f)) %>%
  drop_na() %>%
```
```r
ggplot() + geom_point(aes(x = X_f, y = DV), alpha = 0.3,
size = 0.2, position = "jitter") + geom_violin(aes(y = DV,
x = X_f, fill = familiarization), alpha = 0.2) + themeBW()
```

<figure><img src="image_47.png" /></figure>

### 10.3.1 Simulation of models

```r
# Number of simulations:
reps <- 100
```

```r
# Simulation function:
run_sims <- function(filename_full, ef) {
```

```r
dat_sim <- my_sim_data beta_c = ef,
beta_f = ef,
beta_a = ef,
```
beta_ca = ef,
beta_af = ef,
beta Cf = ef,
beta_cfa = ef)

missingSamples_fam <- dat_sim %>
mutate(nas = rbinom(n(), 1, 0.95 - ifelse(X_f > -0.5, (X_f+0.5)*0.6, 0))) %>
mutate(DV = ifelse(nas == 1, DV, NA))

mod_sim <- lmer(DV ~ 1 + X_a * X_c * X_f +
  (1 | subj_id) +
  (1 | lab_id) +
  (1 | item_id),
  data=missingSamples_fam)

sim_results <- broom.mixed::tidy(mod_sim)

# append the results to a file
append <- file.exists(filename_full)
write_csv(sim_results, filename_full, append = append)

# return the tidy table
sim_results

}

filename_full_0.3_missing_fam = 'sims/run_sims_0.3_fam_missing.csv'
start_time <- Sys.time()
sims <- purrr::map DF(1:reps, ~run_sims(filename_full = filename_full_0.3_missing_fam, ef = 0.3))
end_time <- Sys.time()
end_time - start_time

10.3.2 Visualise Estimates for Fixed Effects:

# read saved simulation data
sims_50_missing_fam_0.3 <- read_csv(filename_full_0.3_missing_fam, col_types = cols(
  # makes sure plots display in this order
group = col_factor(ordered = TRUE),
term = col_factor(ordered = TRUE)
))

fixed_missing_fam_plot <- sims_50_missing_fam_0.3 %>
filter(effect == "fixed") %>
ungroup() %>
arrange(term, estimate) %>
mutate(row = rep(seq(1:reps), 8)) %>
ggplot(aes(x = row, y = estimate, ymin = estimate-std.error, ymax = estimate+std.error)) +
facetWrap(~term, scales = "free") +
geom_pointrange(fatten = 1/2) +
ylab("Estimates") +
xlab("Simulations") +
ggtitle('Estimates of Fixed Effects for Missing Data with Familiarisation, ef = 0.3') +
themeBW()

fixed_missing_fam_plot <- fixed_missing_fam Plot + theme plot.title = element_text(hjust = 0.5, size = 20))
fixed_missing_fam Plot
Estimates of Fixed Effects for Missing Data with Familiarisation, ef = 0.3

<figure><img src="image_48.png" /></figure>

Visualise Estimates for Random Effects:

```r
ran_missing_fam_plot <- sims_50_missing_fam_0.3 %>
filter(effect == "ran_pars") %>
ungroup() %>
arrange(group, term, estimate) %>
mutate(row = rep(seq(1:reps), 4)) %>
ggplot(aes(x = row, y = estimate)) + geom_point(alpha = 0.7) +
facetWrap(~group + term, scales = "free_y") + themeBW() +
ylab("Estimates") + xlab("Simulations") + ggtitle("Estimates of Random Effects for Missing Data with Familiarisation")
themeBW()

ran_missing_fam_plot <- ran_missing_fam_plot + theme plot.title = element_text(hjust = 0.5,
size = 20))

ran_missing_fam_plot
```
Estimates of Random Effects for Missing Data with Familiarisation, ef = 0.3

<figure><img src="image_49.png" /></figure>

11 Summary Statistics for Power Calculation with Data missing not completely at random

<table><caption>Table 9: Power for Simulations with Non-Random Missing Data and Varying Intercepts</caption><thead><tr><th>term</th><th>missing with age</th><th>missing with trial</th><th>missing with fam</th></tr></thead><tbody><tr><td>(Intercept)</td><td>0.07</td><td>0.01</td><td>0.06</td></tr><tr><td>X_a</td><td>1.00</td><td>1.00</td><td>1.00</td></tr><tr><td>X_c</td><td>0.79</td><td>0.79</td><td>0.78</td></tr><tr><td>X_f</td><td>0.66</td><td>0.62</td><td>0.58</td></tr><tr><td>X_a:X_c</td><td>0.98</td><td>0.99</td><td>0.99</td></tr><tr><td>X_a:X_f</td><td>0.94</td><td>1.00</td><td>0.98</td></tr><tr><td>X_c:X_f</td><td>0.22</td><td>0.24</td><td>0.12</td></tr><tr><td>X_a:X_c:X_f</td><td>0.81</td><td>0.85</td><td>0.85</td></tr></tbody></table>
<table><caption>Table 10: Bias for Simulations with Non-Random Missing Data and Varying Intercepts</caption><thead><tr><th>term</th><th>age bias, ef = 0.3</th><th>trial bias, ef = 0.3</th><th>fam bias, ef = 0.3</th></tr></thead><tbody><tr><td>(Intercept)</td><td>-0.010</td><td>0.020</td><td>-0.011</td></tr><tr><td>X_a</td><td>-0.008</td><td>-0.005</td><td>0.008</td></tr><tr><td>X_c</td><td>-0.003</td><td>-0.001</td><td>0.004</td></tr><tr><td>X_f</td><td>-0.019</td><td>-0.003</td><td>0.029</td></tr><tr><td>X_a:X_c</td><td>-0.001</td><td>-0.009</td><td>-0.007</td></tr><tr><td>X_a:X_f</td><td>0.005</td><td>0.014</td><td>-0.012</td></tr><tr><td>X_c:X_f</td><td>0.019</td><td>-0.031</td><td>0.055</td></tr><tr><td>X_a:X_c:X_f</td><td>0.022</td><td>-0.002</td><td>0.006</td></tr></tbody></table>

## 12 Logistic Regression Models

```
m_missing_age <- glmer(nas ~ X_f + X_a + X_c + trial_num + (1 | subj_id) + (1 | lab_id) + (1 | item_id), data = missingSamples_age, family = binomial)
```

```
summary(m_missing_age)
```

```
m_missing_fam <- glmer(nas ~ X_f + X_a + X_c + trial_num + (1 | subj_id) + (1 | lab_id) + (1 | item_id), data = missingSamples_fam, family = binomial)
```

```
summary(m_missing_fam)
```

```
m_missing帅 <- glmer(nas ~ X_f + X_a + X_c + trial_num + (1 | subj_id) + (1 | lab_id) + (1 | item_id), data = missingSamples帅, family = binomial)
```

```
summary(m_missing帅)
```