Generating novel experimental hypotheses from language models: A case
  study on cross-dative generalization
Authors: name: Kanishka Misra; name: Najoung Kim
Published: 2024-08-09 14:17:36

Abstract: Neural network language models (LMs) have been shown to successfully capture
complex linguistic knowledge. However, their utility for understanding language
acquisition is still debated. We contribute to this debate by presenting a case
study where we use LMs as simulated learners to derive novel experimental
hypotheses to be tested with humans. We apply this paradigm to study
cross-dative generalization (CDG): productive generalization of novel verbs
across dative constructions (she pilked me the ball/she pilked the ball to me)
-- acquisition of which is known to involve a large space of contextual
features -- using LMs trained on child-directed speech. We specifically ask:
"what properties of the training exposure facilitate a novel verb's
generalization to the (unmodeled) alternate construction?" To answer this, we
systematically vary the exposure context in which a novel dative verb occurs in
terms of the properties of the theme and recipient, and then analyze the LMs'
usage of the novel verb in the unmodeled dative construction. We find LMs to
replicate known patterns of children's CDG, as a precondition to exploring
novel hypotheses. Subsequent simulations reveal a nuanced role of the features
of the novel verbs' exposure context on the LMs' CDG. We find CDG to be
facilitated when the first postverbal argument of the exposure context is
pronominal, definite, short, and conforms to the prototypical animacy
expectations of the exposure dative. These patterns are characteristic of
harmonic alignment in datives, where the argument with features ranking higher
on the discourse prominence scale tends to precede the other. This gives rise
to a novel hypothesis that CDG is facilitated insofar as the features of the
exposure context -- in particular, its first postverbal argument -- are
harmonically aligned. We conclude by proposing future experiments that can test
this hypothesis in children.