A SMART Mnemonic Sounds like “Glue Tonic”: Mixing LLM s with Student Feedback to Make Mnemonic Learning Stick (2024)

Abstract: Keyword mnemonics are memorable explanations that link new terms to simpler keywords.Prior work generates mnemonics for students, but they do not train models using mnemonics students prefer and aid learning.We build SMART, a mnemonic generator trained on feedback from real students learning new terms.To train SMART, we first fine-tune LLaMA-2 on a curated set of user-written mnemonics.We then use LLM alignment to enhance SMART: we deploy mnemonics generated by SMART in a flashcard app to find preferences on mnemonics students favor.We gather 2684 preferences from 45 students across two types: **expressed** (inferred from ratings) and **observed** (inferred from student learning), yielding three key findings.First, expressed and observed preferences disagree; what students *think* is helpful does not always capture what is *truly* helpful.Second, Bayesian models can synthesize complementary data from multiple preference types into a single effectiveness signal.SMART is tuned via Direct Preference Optimization on this signal, which resolves ties and missing labels in the typical method of pairwise comparisons, augmenting data for LLM output quality gains. Third, mnemonic experts assess SMART as matching GPT-4 at much lower deployment costs, showing the utility of capturing diverse student feedback to align LLMs in education.

## an initial smart mnemonic model given a vocabulary (vocab) term v (e.g. benevolent), we desire a keyword mnemonic m students can use to remember the meaning of v. for optimal benefits @xcite , m should link to a similarly sounding and simpler keyword k (e.g. benevolent sounds like benefit), and then explain how k and v are linked (e.g. a boss who gives their employees benefits is kind-or benevolent). we now train an initial smart model to generate keyword mnemonics. we collect a high-quality dataset of user-written mnemonics ( §2.1, §2.2) and fine-tune smart on this mnemonic dataset ( §2.3). ## collecting mnemonic preferences only fine-tuning does not explicitly guide smart toward mnemonics that users prefer and help them learn-our overall goal. thus, we use alignment @xcite : tuning llms to preference labels capturing which outputs users favor. to align smart, we need a preference dataset d pref with entries of a term v, mnemonic pair (m a , m b ) for v created by the initial model p 0 (m | v), and preference label @xmath0, b, tie} noting the mnemonic in the pair users favor. to build d pref , we create mnemonic pairs ( §3.1), define our preference labels ( §3.2), and describe our user study details ( §3.3). ## preference analysis we study the relation of our preference types ( §3.2) and uncover that students cannot fully predict what aids learning ( §4.1, §4.2). thus, we conclude that a mnemonic's overall helpfulness cannot be captured by just one preference type, inspiring the design of our final model that combines all preferences ( §5). ## training a smart-er mnemonic model our goal of collecting student preference labels for mnemonic pairs is to identify more helpful or effective mnemonics, using this signal to guide smart's outputs. but as preference types disagree ( §4), how should we identify more effective mnemonics? pairwise comparisons are typically used for this purpose, but they do not always match our goal of aiding learning ( §4.1). further, when pairwise preferences are missing or have a tie, we could draw from other preferences to break these ties and iden-tify the better mnemonic, instead of discarding the pair for training ( §6.2). conversely, using observed preferences to select more effective mnemonics is promising as it matches our learning goal, but also using expressed preferences could help us avoid bizarre or offensive (kroll and tu, 1988) mnemonics that may still aid learning (see appendix d.1). given the multi-faceted nature of mnemonic effectiveness, we develop a bayesian model to learn effectiveness from all preferences and tune smart via direct preference optimization @xcite on this learned effectiveness signal ( §5.2). ## related work below, we review relevant literature on mnemonic generation ( §7.1) and human preferences ( §7.2). ## conclusion we design smart, the first keyword mnemonic generator guided by student feedback. smart is trained on new fine-tuning and preference datasets, both of which are released. while curating data, we reveal low agreement in expressed preferences and our introduced observed preferences, showing that students cannot predict their learning. combining expressed and observed preferences via dpo and bayesian modeling yields a smaller, more efficient mnemonic model matching gpt-4. however, our human writer surpasses both models, especially in keyword simplicity and explanation quality, motivating mnemonic generation as a challenging task. to further challenge llms, researchers could explore personalizing mnemonics for students, adapting smart to different domains, languages, and modalities, or optimizing on other learning signals (e.g. long-term learning) via student feedback. while expressed preferences are currently used for alignment, they do not measure what truly helps users. thus, there is a need for more work in capturing observed preferences and user goals. ensuring llms pursue such goals safely requires alignment methods to steer llms toward both expressed and observed preferences, and we design a method to combine them; our method also resolves ties and missing labels to augment datasets. we hope our study of the disagreements and benefits of diverse mnemonic preferences will motivate future work in safely aligning llm outputs to true user needs.