yyilmazecrin/class-retrieval-2023

Poolformer experiments: Retrieval

Overview

This is an experimental Poolformer codebase for Retrieval. It keeps the nano setup intentionally manageable so architecture changes can be inspected before a full training run.

Expect rough edges — this is a working sketch, not a polished release.

Limitations

1. weights are random-init, so outputs carry no signal
2. robustness, fairness and domain transfer are untested
3. the API may change without notice while the design settles

Repository Status

1. The Python file contains the model and runnable example or training entry point.
2. config.json records the generated architecture settings.
3. training_args.json records the default experiment recipe.
4. model.safetensors is a valid initialization checkpoint for smoke tests; it is not presented as a trained benchmark checkpoint.
5. No benchmark score is claimed in this repository.

Default Experiment Recipe

The generated recipe uses rmsprop with a exponential schedule.

These values exist so the script runs end-to-end. A meaningful experiment needs its own tuning budget and at least three seeds.

Architecture

1. Architecture: Poolformer
2. Scale: nano
3. Attention: standard
4. Fusion: cross attention
5. Activation: approx gelu
6. Normalization: scalenorm

Training Details

1. Weights initialized with a kaiming scheme.
2. Runs on a single consumer GPU.
3. Full smoke test takes minutes, not hours.
4. No distributed setup required.

Files

File	Purpose
train.py	primary artifact
README.md	this documentation
config.json	architecture configuration
training_args.json	default experiment settings
model.safetensors	initialization checkpoint

License

Released under bsd-2-clause. Review the source-data terms separately when this repository is used with external datasets.