thlegrand/poolformer-multitask-2024

Working Poolformer implementation for Multitask

Overview

This repository is a compact, custom PyTorch implementation of Poolformer for Multitask. The giant configuration is intended for code review, smoke tests, and small controlled experiments rather than as a production-ready pretrained release.

---

Repository status

One Python module contains the model and a runnable example. Generated configuration lives in config.json; the recipe in training_args.json. The checkpoint is an initialization artifact and should not be read as a trained model. Scores are deliberately absent.

---

Default experiment recipe

Defaults: novograd optimizer, step schedule. Treat these as starting points; a real run should sweep learning rate and batch size before drawing conclusions.

---

Limitations

• weights are random-init, so outputs carry no signal
• robustness, fairness and domain transfer are untested
• the API may change without notice while the design settles

---

Evaluation guidance

• pick a task dataset such as a task-specific held-out set
• fix a tuning budget in advance
• compare against a capacity-matched baseline
• publish raw logs next to any claim

---

Citation

@misc{workingpoolformerimp},
  title = {Working Poolformer implementation for Multitask},
  year = {{2026}},
  publisher = {{Hugging Face}},
}

---

Quick check

Run python eval.py --help for the built-in smoke test. Because this is a custom implementation, generic automatic loading APIs require an explicit adapter before use.

---

Files

• eval.py — primary artifact
• README.md — this documentation
• config.json — architecture configuration
• training_args.json — default experiment settings
• model.safetensors — initialization checkpoint
• USE_POLICY.md — use and redistribution notes
• examples/sample_usage.py — usage sketch

---

License

Distributed under the cc-by-4.0 license. External datasets keep their own terms — check them separately.