A SMART Mnemonic Sounds like "Glue Tonic": Mixing LLMs with Student Feedback to Make Mnemonic Learning Stick
Authors: Nishant Balepur, Matthew Shu, Alexander Hoyle, Alison Robey, Shi Feng, Seraphina Goldfarb-Tarrant, Jordan Lee Boyd-Graber


Reviewer paper summaries: This paper focuses on an intriguing application where LLMs are utilized to enhance students’ vocabulary learning through mnemonics. The system, named SMART, is built using LLaMA fine-tuned on a dataset of user-submitted mnemonics and further refined through alignment with student feedback collected via a flashcard app. The study demonstrates that SMART, while maintaining lower deployment costs, achieves performance on par with GPT-4 in generating effective mnemonics, as evaluated by mnemonic experts.
Summary of strengths: - The task of generating mnemonics is practical and interesting, providing a valuable downstream application for LLMs.
- The design of the experiments is well-considered, and the evaluations with students convincingly validate the effectiveness of the proposed system.
Summary of weaknesses: - The motivation behind the designed reward model in Section 3.1 is unclear. Is there any evidence supporting the use of these metrics?
- The details about data statistics in Section 3.2 are not clear. What is the distribution of mean time? Does it show statistically significant differences?
- The observed preferences in Section 3.2.2 lack some details. For example, how do you determine whether a student truly remembers a term?
Comments, suggestions and typos: Refer to the questions listed in the weaknesses.
Overall assessment scores: 4