Navigating the Landscape of Hint Generation Research: From the Past to the Future (2025)

Abstract: Digital education has gained popularity in the last decade, especially after the COVID-19 pandemic. With the improving capabilities of large language models to reason and communicate with users, envisioning intelligent tutoring systems that can facilitate self-learning is not very far-fetched. One integral component to fulfill this vision is the ability to give accurate and effective feedback via hints to scaffold the learning process. In this survey article, we present a comprehensive review of prior research on hint generation, aiming to bridge the gap between research in education and cognitive science, and research in AI and Natural Language Processing. Informed by our findings, we propose a formal definition of the hint generation task, and discuss the roadmap of building an effective hint generation system aligned with the formal definition, including open challenges, future directions and ethical considerations.

## introduction prior research has established a correlation between the student-teacher ratio and a student's overall performance @xcite . however, private tutoring is not accessible to everyone, and finding expert tutors is often difficult and incurs considerable costs @xcite . intelligent tutoring systems (itss) hold the key to addressing these educational challenges, notably the need for personalized learning in a system often reliant on instructional teaching and standardized testing @xcite . the hallmark of intelligent tutoring systems is their ability to provide step-by-step guidance to students while they work on problems, and hints play a critical role in their ability to provide this * corresponding authoranubhav@cs.columbia.edu. within learning science, hints refer to the clues, prompts, questions, or suggestions provided to learners to aid them in solving problems, answering questions, or completing tasks, thereby encouraging critical thinking, problem-solving skills, and independent learning. in figure 1 , we provide an example of a hint generation system capable of the reasoning required to answer the question, acknowledging the wrong attempt by the learner and providing informative hints linked to the learner's existing knowledge. this framework of scaffolding is well established in education @xcite , and expert tutors are guided to incorporate it in their teaching practices @xcite . with the aim of developing hint generation systems with capabilities such as the ones showcased in figure 1 , we consolidate the dispersed efforts on hint generation, bridging the gap between research in education and cognitive sciences on the one hand, and research in ai and natural language processing (nlp) on the other. grounded in the findings from our literature review, we provide a roadmap for future research on automatic hint generation. we summarize the key characteristics of a successful hint as observed by research with human tutors in section 2 and review the automated hint generation systems in section 3. we identify the gaps and propose a roadmap for future research in hint generation in section 4. we provide a rethinking of the formal definition of the hint generation task (section 4.1), a brief review of research areas that can inform the design of a hint generation system that aligns with the formal definition (section 4.2), open challenges for future directions for effective automatic hint generation systems (section 4.3), and ethical considerations (section 4.4). our major contributions include: ## anatomy of a hint in this section, we draw on research from education and cognitive sciences to describe the key characteristics of an effective hint formulation process. we start by describing the pragmatics of a hint ('context'), covering some prominent traits exhibited by expert tutors and educators while generating hints, and then dive deeper into the anatomy of a hint by discussing the semantic ('what to say') and the stylistic ('how to say it') aspects of a hint. ## survey of computational approaches in this section, we provide a comprehensive overview of the recent advancements of computational approaches for automatic hint generation. we first describe the extensively studied hint generation techniques for computer programming that focus on revealing code snippets to help learn data input expected or generated outputs source itap question-hint generated by rivers and koedinger (2017)-write a program that if a given day is weekend. type: replace learner's response-old expression: ''saturday'' def isweekend(day): new expression: ''saturday'' return bool(@xmath0. response strategyprovide a solution strategy so mike has cookies left? response intention-learner's response -help student understand the lessons topic or solution strategy he has 10 cookies left. response-great try! let's try to draw a picture. let's start with 4 cookies and erase the 3 that mike eats. squad question-who became the most respected entrepreneur hint generated by jatowt et al. (2023)in the world according to financial times in 2003? the searched person held the position of chief executive officer. expected answer-bill gates trivialqa question-hint generated by mozafari et al. (2024b)-in which city are the headquarters of the international monetary fund? the city is known for its neoclassical architecture. ## roadmap for future research in hint generation great progress has been made in automatic hint generation over the last two decades; however, there is still room for improvement. existing hint generation frameworks do not personalize the hints to the learner's prior knowledge (section 2.1), and are only evaluated in short-term studies. we still do not know the effects of long-term exposure to these interventions on the learners. these frameworks are also limited to certain domains and have not been widely explored in other domains, including different branches of science and social science. to improve upon these factors, we discuss a roadmap for future efforts in automatic hint generation. we propose a computational hint generation framework that draws on education and cognitive sciences (figure 3 ).