DOI: 10.5061/dryad.83bk3jb0v
title: Emergent neural dynamics and geometry for generalization in a transitive inference task

description: Relational cognition, the ability to infer relationships that generalize
 to novel combinations of objects, is fundamental to human and animal
 intelligence. Despite this importance, it remains unclear how relational
 cognition is implemented in the brain due in part to a lack of hypotheses
 and predictions at the levels of collective neural activity and behavior.
 Here we discovered, analyzed, and experimentally tested neural networks
 (NNs) that perform transitive inference (TI), a classic relational task
 (if A &gt; B and B &gt; C, then A &gt; C). We found NNs that
 (i) generalized perfectly, despite lacking overt transitive structure
 before training, (ii) generalized when the task required working memory
 (WM), a capacity thought to be essential to inference in the brain, (iii)
 emergently expressed behaviors long observed in living subjects, in
 addition to a novel order-dependent behavior, and (iv) adopted different
 task solutions yielding alternative behavioral and neural predictions.
 Further, in a large-scale experiment, we found that human subjects
 performing WM-based TI showed behavior inconsistent with a class of NNs
 that characteristically expressed an intuitive task solution. These
 findings provide neural insights into a classical relational ability, with
 wider implications for how the brain realizes relational cognition.
description: Neural network (and other) models trained and analyzed on a
 transitive inference task ("delay TI"), as described in the
 study "Emergent neural dynamics and geometry for generalization in a
 transitive inference task". Models were defined
 and trained using PyTorch.
description: # Emergent neural dynamics and geometry for generalization in a transitive
 inference task
 [https://doi.org/10.5061/dryad.83bk3jb0v](https://doi.org/10.5061/dryad.83bk3jb0v) Python (pickle) files containing PyTorch-based models trained on a transitive inference task ("delay TI"). These models are described and analyzed in the study "Emergent neural dynamics and geometry in a transitive inference task" (available as a preprint). These models are readable using Jupyter Notebooks and code that has been deposited on Zenodo and is also available on GitHub (link in the Related Works section). Below is the list of model types and their corresponding directories, each directory (titled "ti#") corresponding to a batch of model instances. Each file in the directory corresponds to a single model instance and contains the model parameters and additional information from training. For any clarifications or requests please contact Kenneth Kay ([kaykenneth@gmail.com)](mailto:kaykenneth@gmail.c). ###### Model files * Feedforward models * ti254 — MLP, 100 seeds (regularization: 0.001) * ti253 — LR, 100 seeds (regularization: 0.1) * Basic-Delay RNN * ti350 — tanh, delay=20, input_train=1, output_train=1, 200 instances "f-RNN" * ti351 — tanh, delay=20, input_train=0, output_train=0, 200 instances "r-RNN" * ti352 — tanh, delay=20, input_train=0, output_train=1, 200 instances "r-RNN" with feedforward outputs trainable * Extended-Delay RNN * ti353 — tanh, delay=60, input_train=1, output_train=1, 200 instances "f-RNN" * ti354 — tanh, delay=60, input_train=0, output_train=0, 200 instances "r-RNN" * Variable-Delay RNN * ti360 — tanh, delay=60, input_train=1, output_train=1, jit=0.67, 200 instances "f-RNN" * ti361 — tanh, delay=60, input_train=0, output_train=0, jit=0.67, 200 instances "r-RNN" * Feedforward-trainable RNN ("ff-RNN") * ti365 — tanh, delay=20, input_train=1, output_train=1, 200 seeds, 500 instances Basic-delay * ti366 — tanh, delay=60, input_train=1, output_train=1, 200 seeds, 500 instances Extended-delay * ti367 — tanh, delay=60, input_train=1, output_train=1, jit=0.67, 200 seeds, 500 simulations Variable-delay