Teen-Different/pcn-bridge-lab-connectors

PCN Bridge Lab Connector Weights

Connector-only weights for CLIP ViT-B/32 to Qwen2.5-0.5B-Instruct CLEVR experiments.

Code, training scripts, results, and plots: https://github.com/REDDITARUN/pcn-bridge-lab

This is an AI-assisted research/engineering experiment with a small human-in-the-loop workflow for direction, review, and decisions.

Base models are not included:

• Vision encoder: openai/clip-vit-base-patch32
• Language model: Qwen/Qwen2.5-0.5B-Instruct

Each .pt file contains only:

• connector state dict
• run config
• best validation loss
• candidate-choice test metrics

Optimizer state and full frozen model weights are intentionally omitted.

Results

Rank	Run	Choice Accuracy	File
1	pcn-depth3-s1-eqprop-momentum	47.2%	connectors/pcn-depth3-s1-eqprop-momentum.pt
2	pcn-depth3-s1-adamw	46.3%	connectors/pcn-depth3-s1-adamw.pt
3	pcn-depth3-s6-adamw	45.9%	connectors/pcn-depth3-s6-adamw.pt
4	pcn-depth3-s12-eqprop-momentum	45.5%	connectors/pcn-depth3-s12-eqprop-momentum.pt
5	pcn-depth3-s6-sgd	45.5%	connectors/pcn-depth3-s6-sgd.pt
6	pcn-depth3-s1-sgd	45.2%	connectors/pcn-depth3-s1-sgd.pt
7	linear-adamw	45.0%	connectors/linear-adamw.pt
8	pcn-depth3-s12-sgd	45.0%	connectors/pcn-depth3-s12-sgd.pt
9	mlp-depth3-adamw	43.8%	connectors/mlp-depth3-adamw.pt
10	pcn-depth3-s6-eqprop-momentum	43.2%	connectors/pcn-depth3-s6-eqprop-momentum.pt
11	pcn-depth3-s12-adamw	43.0%	connectors/pcn-depth3-s12-adamw.pt

Loading

Use the code in the GitHub repository to instantiate ConnectorVLM, then load the connector state dict from a file in connectors/.

import torch
from src.model import ConnectorVLM

ckpt = torch.load('connectors/pcn-depth3-s1-eqprop-momentum.pt', map_location='cpu')
model = ConnectorVLM(ckpt['config'])
model.connector.load_state_dict(ckpt['connector'])

Caveat

The primary metric is candidate-choice accuracy over valid CLEVR answers. Free-form generations remain noisy and should be treated as diagnostic only.