Petitad07/knowledge-distillation-review50

Knowledge Distillation — open questions

Overview

A structured set of research notes on Knowledge Distillation, with concrete evaluation references and open questions. Plans and hypotheses are kept separate from completed results.

---

Open Questions

1. comparability of baselines under matched capacity
2. minimum data exposure for a usable signal
3. confounders that would invalidate a naive comparison

---

Scope And Limitations

Exploratory by design: no benchmark claims, no completed ablations, no released code, no trained checkpoint. The references are a starting point for verification, not evidence that the study has run.

---

Related Work

References use a numeric-apa citation style and will be consolidated once the reading list settles.

---

How To Read This Repository

The note follows a intro-method-exp-related-conclusion outline and a theoretical-rigorous writing style. Plans and hypotheses are kept apart from results; any future numbers should ship with dataset versions, commands, seeds and raw logs.

---

What Is Covered

1. research question scope and likely confounders
2. one proposed comparison against matched baselines
3. evaluation context such as task-appropriate public benchmarks named in the main note
4. reproducibility checks and open questions

---

Files

1. paper_notes.md — primary artifact
2. README.md — this documentation
3. config.json — architecture configuration
4. training_args.json — default experiment settings
5. model.safetensors — initialization checkpoint
6. NOTES.md — informal to-do notes

---

License

Released under cc-by-4.0. Review the source-data terms separately when this repository is used with external datasets.