Sschneideremil/poolformer-contrastive-alpha

A compact Poolformer for Contrastive

Overview

A small Poolformer implementation for Contrastive, packaged with an explicit configuration and an initialization checkpoint. The tiny variant is a reproducible starting point, not a trained model release.

Citation

If this repository is useful for your work, a link back is appreciated. No formal citation is required yet.

Default Experiment Recipe

Defaults: rmsprop optimizer, polynomial schedule. Treat these as starting points; a real run should sweep learning rate and batch size before drawing conclusions.

Evaluation Guidance

A first evaluation could use a task-specific held-out set. Report the metric across at least three seeds and include a matched-capacity baseline; keep logs and library versions with any published number.

Repository Status

1. Single-file implementation; the __main__ block runs a smoke test.
2. config.json and training_args.json hold the generated settings.
3. model.safetensors is freshly initialized — plumbing, not results.
4. No benchmark numbers are reported on purpose.

Limitations

1. weights are random-init, so outputs carry no signal
2. robustness, fairness and domain transfer are untested
3. the API may change without notice while the design settles

Training Details

1. Framework: PyTorch
2. Precision: fp32
3. Init scheme: trunc normal
4. Hardware: a single consumer GPU is enough
5. Typical run: minutes, not hours

Files

1. eval.py — primary artifact
2. README.md — this documentation
3. config.json — architecture configuration
4. training_args.json — default experiment settings
5. model.safetensors — initialization checkpoint

License

Distributed under the apache-2.0 license. External datasets keep their own terms — check them separately.