CopTain/neuroGainnet

GainNet — sentence-level neural gain

Code: https://github.com/JumptierFive/neuroGainnet

We trianed our data on Ming Zhou, Zhengxin Gong, Yuxuan Dai, Yushan Wen, Youyi Liu, and Zonglei Zhen (2026). A large-scale fMRI dataset for human action recognition. OpenNeuro. [Dataset] doi: doi:10.18112/openneuro.ds004488.v2.0.1 Thanks for their public data conrtibution

Two sets of weights estimated from fMRI. Both score a single sentence and return one scalar: the weight the brain assigns to that sentence when it describes a brief visual scene.

Files

File	Contents	Shape	Parameters
gainnet_fp16.pt	full ensemble, one network per (participant, region)	2,700 networks	39,521 each, 106.7 M in total
gainnet_shared_1931.pt	shared gain network of the simplified model	one network	1,931
gains_sentence_region.npz	gains of the 66,270 in-sample sentences, averaged over participants	(66270, 90)	—
sentences.csv.gz	the sentences those gains belong to	—	—
selected_regions.json	the 20 AAL regions drawn for each participant in the shared model	—	—

gainnet_fp16.pt carries subject and region index arrays, subject-major (participant 1: regions 1–90, participant 2: regions 1–90, …). region is the AAL index used throughout the paper.

Architectures

Full ensemble (gainnet_fp16.pt)

Linear(384, 100) -> ReLU -> Linear(100, 10) -> ReLU -> Linear(10, 1) -> Sigmoid

Shared network (gainnet_shared_1931.pt)

Linear(384, 5) -> ReLU -> Linear(5, 1) -> Sigmoid

The shared network is the gain branch of the simplified model. It was trained jointly across participants and across 600 region heads (30 participants × 20 randomly drawn AAL regions, listed in selected_regions.json) and contains 1,931 parameters. The region-specific readouts that accompany it in the paper are not part of this file.

Input and output

• Input: the 384-d sentence embedding from sentence-transformers/all-MiniLM-L12-v2, unnormalised, one sentence at a time.
• Output: one scalar in (0, 1) per sentence. Within a description the scalars are normalised to sum to one and used as mixture weights for pooling.

How the weights were estimated

30 participants watched 21,600 two-second video clips during fMRI; each clip was described by a vision–language model, giving 66,270 sentences. For each participant and each of the 90 AAL territories, a network was fitted to predict the regional response from the weighted sum of sentence embeddings, with the weights produced by that network. Sessions 1–10 were used for training and 11–12 for validation, within participant. See DATA.md.

Evaluation

Property	Result
Held-out prediction, estimated weights vs fixed weights	0.0041 vs 0.0007 (t = 18.9, P = 9.5 × 10⁻⁷⁵); positive in 93% of regions and 77% of participants
Permuted pairing between sentences and responses	0 of 19 permutations reached the true value; null maximum 0.0031
Agreement with the image-specificity residual	r = 0.26–0.60 across four corpora
Retrieval, weighting vs uniform pooling	+2.29 pp (19,538 generated descriptions), +1.68 pp (45,619 COCO descriptions), +0.89 pp (19,310 human-written descriptions)
Retrieval where flat pooling is undecided	24.4% vs 4.2% top-1 in the near-miss quartile of the generated corpus

License

CC BY 4.0 for the weights. See DATA.md for the terms attached to the source data.

Citation

See CITATION.cff.