Working Poolformer implementation for Retrieval
Overview
A small Poolformer implementation for Retrieval, packaged with an explicit configuration and an initialization checkpoint. The huge variant is a reproducible starting point, not a trained model release.
The whole setup fits in a single module on purpose.
Limitations
The checkpoint is untrained and unaudited. Treat the repository as an experimental starting point; anything learned from a future trained version must be documented separately.
Default Experiment Recipe
The generated recipe uses rmsprop with a linear warmup schedule.
These values exist so the script runs end-to-end. A meaningful experiment needs its own tuning budget and at least three seeds.
Repository Status
- Single-file implementation; the
__main__block runs a smoke test. config.jsonandtraining_args.jsonhold the generated settings.model.safetensorsis freshly initialized — plumbing, not results.- No benchmark numbers are reported on purpose.
Training Details
- Weights initialized with a xavier uniform scheme.
- Runs on a single consumer GPU.
- Full smoke test takes minutes, not hours.
- No distributed setup required.
Quick Check
Run python model.py --help for the built-in smoke test. Because this is a custom implementation, generic automatic loading APIs require an explicit adapter before use.
Files
| File | Purpose |
|---|---|
model.py |
primary artifact |
README.md |
this documentation |
config.json |
architecture configuration |
training_args.json |
default experiment settings |
model.safetensors |
initialization checkpoint |
examples/sample_usage.py |
usage sketch |
License
Distributed under the cc-by-4.0 license. External datasets keep their own terms — check them separately.
- Downloads last month
- 11