DM-Diaz/VEDB-SimCLR-ResNet18-Baseline

VEDB SimCLR ResNet-18 — Baseline

This repository contains the Baseline SimCLR ResNet-18 checkpoint from:

Diaz, D. M., & Henderson, M. M. (2026). Eccentricity-Constrained CNN Training Reveals Adaptive Information Coding Around the Visual Field. Proceedings of the Conference on Cognitive Computational Neuroscience 2026.

DOI: 10.32470/0416gfsq
arXiv: 2607.19316
Contributed Talk: CCN 2026 presentation on YouTube

This model is part of the Eccentricity-Constrained SimCLR Models (VEDB) collection, containing checkpoints pretrained under four visual-field conditions: Baseline, Fovea-Gaze, Periph, and Periph-NF.

Model Description

This model uses a ResNet-18 visual encoder pretrained with SimCLR self-supervised contrastive learning on naturalistic egocentric imagery with synchronized human gaze data from the Visual Experience Dataset (VEDB).

The associated study investigated whether constraining visual experience to different portions of the visual field produces systematic differences in learned representations, downstream task performance, and alignment with human visual cortex.

Baseline Condition

The Baseline condition serves as the full-field reference model. It was pretrained on the original, unedited 224 × 224 VEDB frames following the common frame preprocessing pipeline and therefore received no additional central or peripheral visual-field restriction.

The associated model variants manipulate the same source frames to isolate different forms of visual-field information:

• Fovea-Gaze: gaze-centered central-only input
• Periph: peripheral-only input produced by masking the gaze-centered central region
• Periph-NF: peripheral-only input with a NeuroFovea transform applied before central masking

Release Status

Component	Status
Pretrained checkpoint	Available
Model card	Available
Training code	Forthcoming
Evaluation code	Forthcoming
VEDB imagery	Not redistributed; available via Databrary

The complete training and evaluation codebase is currently being consolidated and documented and will be linked here upon public release.

The checkpoint is being released in advance of the codebase to provide access to the model artifact used in the published study.

Technical Provenance Note and Discrepancies

Technical provenance note: The checkpoint metadata, architecture, training parameters, and implementation details documented in this model card have been re-verified against the released model checkpoint and, where available, the original training code and launch configuration. If a technical detail concerning the released model artifact differs between the associated paper and this model card, the model card should be treated as the authoritative description of the released checkpoint and its implementation. The associated paper remains the primary source for the study's scientific analyses, results, and interpretation.

Checkpoint

File:

simclr_resnet18_baseline_epoch120.pth.tar

This repository provides the original PyTorch training checkpoint from epoch 120 of Baseline SimCLR pretraining.

The checkpoint is serialized as a dictionary containing:

Key	Contents
epoch	Final training epoch (120)
arch	Backbone architecture (resnet18)
state_dict	Model parameters and registered buffers
optimizer	Adam optimizer state at the time of saving

The state_dict contains 124 entries and includes both the ResNet-18 encoder and SimCLR projection head.

Model parameters use the backbone.* namespace. The projection head is stored as:

• backbone.fc.0: Linear(512, 512)
• backbone.fc.2: Linear(512, 128)

The checkpoint therefore contains the full SimCLR model state rather than encoder weights alone.

Architecture

Overview of the VEDB preprocessing, SimCLR pretraining, downstream linear probes, and voxelwise encoding workflow.

Component	Specification
Backbone	ResNet-18
Framework	PyTorch
Learning paradigm	Self-supervised contrastive learning
Objective	SimCLR / NT-Xent
Input resolution	224 × 224
Backbone representation	512-dimensional
Projection head	Linear(512, 512) → ReLU → Linear(512, 128)
Projection dimension	128

The standard ResNet-18 classification layer was replaced during SimCLR pretraining by a two-layer projection head:

nn.Sequential(
    nn.Linear(512, 512),
    nn.ReLU(),
    nn.Linear(512, 128),
)

Training Data

Visual Experience Dataset (VEDB)

The model was pretrained using imagery from the Visual Experience Dataset (VEDB), a large-scale dataset of naturalistic egocentric experience containing more than 200 hours of integrated egocentric video, eye-movement, and odometry recordings.

VEDB resources:

• Dataset: VEDB on Databrary
• Dataset paper: Greene et al. (2024), The Visual Experience Dataset: Over 200 recorded hours of integrated eye movement, odometry, and egocentric video
• Project resources: VEDB on OSF

The original VEDB imagery is not redistributed through this repository. Researchers wishing to reproduce training should obtain VEDB through the official distribution and comply with its applicable access and usage requirements.

Dataset Construction

A total of 717 VEDB sessions were initially retrieved through Databrary. Sessions without synchronized gaze data were excluded, leaving 514 sessions for processing and analysis.

Within task-relevant portions of each retained session:

• Frames were sampled every 2 seconds.
• Sampling used the native 25 FPS video rate.
• A maximum of 1,000 frames per session was sampled.
• The resulting SimCLR dataset contained 433,564 frames.

All data splits were performed at the video-session level to prevent leakage from temporally adjacent and environmentally correlated frames.

SimCLR Dataset Split

Split	Sessions	Frames	Frame proportion
Train	455	377,462	87.06%
Validation	28	26,026	6.00%
Test	31	30,076	6.94%
Total	514	433,564	100%

The split was constructed as an approximately 80/10/10 session-level split using stratification to balance task-label representation. Because sessions contain different numbers of sampled frames, the resulting frame percentages differ from the session-level proportions.

Validation and test sessions were held out from SimCLR representation learning.

Frame Preprocessing

Sampled VEDB frames were processed using a deterministic common pipeline:

1. Decode the sampled video frame.
2. Convert the image to RGB.
3. Bicubic resize to 256 px.
4. Center crop to 224 × 224.

For the Baseline condition, no eccentricity-specific transformation was applied after this preprocessing.

Condition-specific image construction for Fovea-Gaze, Periph, and Periph-NF occurred before the shared SimCLR augmentation pipeline.

SimCLR Pretraining

The four VEDB conditions used the same SimCLR architecture (PyTorch-SimCLR), optimization procedure, and augmentation pipeline. They differed only in the visual-field manipulation applied to the source imagery before SimCLR augmentation.

Hyperparameter	Value
Backbone	ResNet-18
Input size	224 × 224
Epochs	120
Batch size	512
Optimizer	Adam
Learning rate	6 × 10^-4
Weight decay	1 × 10^-4
Loss	NT-Xent
Temperature (τ)	0.07
Learning-rate schedule	CosineAnnealingLR (scheduler stepping begins after epoch 10)
Projection head	Linear(512,512) → ReLU → Linear(512,128)
Mixed precision	FP16

SimCLR Augmentations

The common SimCLR augmentation pipeline included:

• random resized cropping,
• random horizontal flipping,
• color jitter (p = 0.8),
• grayscale conversion (p = 0.2), and
• Gaussian blur with σ ~ U(0.1, 2.0).

The same augmentation pipeline was used across all four VEDB conditions and was applied after condition-specific frame construction. See PyTorch-SimCLR for further SimCLR implementation details.

Evaluation

Following SimCLR pretraining, the frozen ResNet-18 backbone was evaluated using linear probes for in-domain and out-of-domain classification and voxelwise encoding models for neural prediction.

Comparative Evaluation Results

The table below reproduces the summary metrics reported in the associated paper across all VEDB-trained conditions and reference models. Rows corresponding to this repository's Baseline checkpoint are bolded.

Task	Condition	Val Loss	Top-1 (%)	Top-5 (%)	Best Macro-F1 (%)
SimCLR	Baseline	0.4331	87.60	—	—
SimCLR	Fovea-Gaze	0.3749	90.43	—	—
SimCLR	Periph-NF	0.4548	90.04	—	—
SimCLR	Periph	0.4545	89.26	—	—
In-Domain	Baseline	0.9811	—	—	42.17
In-Domain	Fovea-Gaze	1.2031	—	—	43.64
In-Domain	Periph-NF	1.3090	—	—	30.93
In-Domain	Periph	1.0623	—	—	36.56
In-Domain	STL-10	1.6666	—	—	25.41
In-Domain	ImageNet-100	1.2342	—	—	41.23
In-Domain	ImageNet-1K	0.9713	—	—	43.33
VGGFace2	Baseline	7.8101	5.21	11.73	3.26
VGGFace2	Fovea-Gaze	7.9104	4.58	10.76	2.70
VGGFace2	Periph-NF	8.0232	3.39	8.17	1.90
VGGFace2	Periph	8.1681	2.54	6.39	1.35
VGGFace2	STL-10	6.9973	9.55	18.96	7.43
VGGFace2	ImageNet-100	6.7985	10.77	21.07	8.71
VGGFace2	ImageNet-1K	6.7964	10.74	21.08	8.77
Places365	Baseline	3.9690	25.63	51.90	23.16
Places365	Fovea-Gaze	4.2347	21.86	46.21	19.14
Places365	Periph-NF	4.2621	20.51	44.58	17.86
Places365	Periph	4.2671	20.26	44.10	17.65
Places365	STL-10	3.8281	26.57	53.47	24.82
Places365	ImageNet-100	3.9207	24.99	51.21	23.32
Places365	ImageNet-1K	3.6264	30.17	58.46	28.36

Note: SimCLR Top-1 is computed from the self-supervised contrastive objective and is not directly comparable to downstream supervised classification accuracy. For downstream tasks, the pretrained ResNet-18 backbone was frozen and only a linear classifier was trained; the backbone weights were not fine-tuned. Classifier checkpoints were selected by best validation Macro-F1. In-domain Top-1 accuracy is omitted because label imbalance across frames can make accuracy misleading; Macro-F1 is reported as the primary class-balanced metric. STL-10, ImageNet-100, and ImageNet-1K are treated as out-of-domain baselines because they were not pretrained on VEDB.

For in-domain classification, Macro-F1 was used as the primary class-balanced metric because of label imbalance across VEDB frame categories.

Neural Encoding Evaluation

The pretrained model was additionally evaluated using voxelwise encoding models of human fMRI responses from the Natural Scenes Dataset (NSD).

NSD contains 7T whole-brain fMRI responses to complex natural scenes. The analysis used data from 8 human participants.

For each model:

• features were extracted from Conv1, Layer1.1, Layer2.1, Layer3.1, Layer4.1, and Avgpool,
• convolutional features were spatially downsampled,
• PCA was used to retain the top 200 components per feature set,
• features were concatenated and z-scored across images, and
• regularized L2 linear regression was used to predict individual voxel responses.

For each participant, the 1,000 NSD images shared across all participants served as the held-out test set, while the remaining 9,000 images viewed by that participant were used to fit the encoding models.

Importantly, original intact NSD images were presented to every pretrained model during encoding evaluation. The Baseline, Fovea-Gaze, Periph, and Periph-NF visual-field transformations were applied during SimCLR pretraining and were not reapplied to NSD stimuli at the encoding stage.

Encoding performance was quantified using held-out voxelwise R².

For complete ROI-level prediction accuracy, statistical comparisons, variance-partitioning analyses, and comparisons across eccentricity conditions, see the associated paper.

Intended Use

This checkpoint is provided primarily for research involving:

• self-supervised visual representation learning,
• egocentric visual experience,
• central versus peripheral information processing,
• visual-field eccentricity,
• transfer learning and linear probing,
• computational modeling of visual cortex, and
• model-to-brain comparisons.

The checkpoint may also be used as a pretrained ResNet-18 initialization for methodological extensions or comparisons with alternative visual-field manipulations.

Out-of-Scope Use

This model was developed as a research representation-learning model and was not designed or validated as:

• a production image-classification system,
• a general-purpose computer-vision foundation model,
• a biological simulation of the human visual system, or
• a system for making decisions about individuals.

The Baseline model contains no explicit simulation of retinal or cortical eccentricity.

Limitations

VEDB consists of naturalistic first-person visual experience and is consequently more temporally correlated and semantically constrained than large curated computer-vision datasets.

Only one SimCLR pretraining run per VEDB condition was used in the published study. These checkpoints therefore do not characterize variation across independent pretraining seeds.

The learned representations are specific to the architecture, training objective, augmentations, data-sampling procedure, visual-field manipulation, and preprocessing choices used in the study. Alternative implementations may produce different representations or downstream performance.

These weights should therefore be interpreted as reproducible artifacts of the published experimental conditions. The authors do not make the claim that the particular implementation is the uniquely optimal method for modeling visual-field eccentricity.

Loading the Model

The checkpoint contains the complete SimCLR model state, including the ResNet-18 encoder and projection head.

import torch
import torch.nn as nn
from torchvision.models import resnet18


class SimCLRResNet18(nn.Module):
    def __init__(self):
        super().__init__()

        self.backbone = resnet18(weights=None)

        self.backbone.fc = nn.Sequential(
            nn.Linear(512, 512),
            nn.ReLU(),
            nn.Linear(512, 128),
        )

    def forward(self, x):
        return self.backbone(x)


checkpoint = torch.load(
    "simclr_resnet18_baseline_epoch120.pth.tar",
    map_location="cpu",
    weights_only=True,
)

model = SimCLRResNet18()
model.load_state_dict(checkpoint["state_dict"], strict=True)
model.eval()

Extracting Backbone Features

To use the pretrained ResNet-18 representation without the SimCLR projection head:

# x should be a preprocessed image tensor with shape [B, 3, 224, 224]

encoder = model.backbone
encoder.fc = nn.Identity()

with torch.no_grad():
    features = encoder(x)

print(features.shape)
# torch.Size([1, 512])

Related Models

This checkpoint belongs to the Eccentricity-Constrained SimCLR Models (VEDB) collection.

• VEDB SimCLR ResNet-18 — Baseline
• VEDB SimCLR ResNet-18 — Fovea-Gaze
• VEDB SimCLR ResNet-18 — Periph
• VEDB SimCLR ResNet-18 — Periph-NF

Citation

If you use these model weights in academic work, please cite the associated study:

@inproceedings{diaz2026eccentricity,
  author    = {Diaz, Dylan M. and Henderson, Margaret M.},
  title     = {Eccentricity-Constrained CNN Training Reveals Adaptive Information Coding Around the Visual Field},
  booktitle = {Proceedings of the 9th Conference on Cognitive Computational Neuroscience},
  address   = {New York, NY, USA},
  year      = {2026},
  doi       = {10.32470/0416gfsq}
}

Proceedings: Conference on Cognitive Computational Neuroscience 2026
Preprint: arXiv:2607.19316

VEDB Citation

Researchers using the underlying Visual Experience Dataset should also cite:

Greene, M. R., et al. (2024). The Visual Experience Dataset: Over 200 recorded hours of integrated eye movement, odometry, and egocentric video. Journal of Vision, 24(11), 6.

See the VEDB dataset paper for the complete author list and citation information.

License

The model checkpoint in this repository is released under the Apache License 2.0.

The VEDB dataset and other third-party resources used in the associated study remain subject to their respective licenses, access requirements, and terms of use. This repository does not redistribute the full VEDB dataset; a small number of example frames are included for illustration of the published visual-field manipulations.