Proactive Agent: Learning to Proactively Predict Implicit Human Intents
Authors: Yaxi Lu, Shenzhi Yang, Cheng Qian, Guirong Chen, Qinyu Luo, Yesai Wu, Huadong Wang, Xin Cong, Zhong Zhang, Yankai Lin, Zhiyuan Liu, Fangming Liu, Maosong Sun


Reviewer paper summaries: - This paper focuses on improving how agents can predict and respond to human needs by understanding user behavior and intent. 
- The authors introduce a new dataset, ProactiveBench, which includes data from three settings: Coding, Writing, and Smart Home. 
- The goal is to train a proactive agent to predict and assist with user intents based on the sequence of events in these environments.
- The proposed LLaMA-3-Proactive model outperforms existing methods in recognizing and responding to user intents.
- This study emphasizes improving the precision and timeliness of intent predictions for better human-agent interactions.
Summary of strengths: - Introducing a unique dataset with ~12k events from various scenarios (Coding, Writing, Smart Home) - could be valuable for the community to train intent prediction models.
- Proposing an agent capable of predicting user intents by analyzing event sequences and updating its understanding based on user behavior.
- Simulating realistic user responses to predicted intents, ensuring accurate and reliable model performance.
- Good evaluation using key metrics (Coverage Ratio, False-alarm Ratio, Accuracy, Pass Rate) with LLaMA-3-Proactive, showing impressive performance.
- Highlighting common challenges in predicting user intents and providing insights for improving model performance.
Summary of weaknesses: - The dataset relies on simulated data and public blueprints, which might not always represent real-world scenarios.
- While it covers many intents, the prediction accuracy is still low. Plus, the model frequently predicts intents that aren’t there, leading to irrelevant actions.
- The method is complicated and might be hard to apply in different contexts. Plus, some aspects of the proposed approach are not well-explained, making it difficult to understand or replicate. 
- The findings might not apply broadly beyond specific contexts like smart homes and coding.
Comments, suggestions and typos: - The authors can provide more explanation and details of the methods so that readers can easily understand and replicate them.
- More examples of how the proposed model works in different scenarios could help. 
- The authors may try to include or simulate real-world data to show how well the model performs in practical situations (outside of controlled environments).
- A clearer explanation of how you measure the success of this model would help (e.g., why certain metrics were chosen).
- The authors may expand the limitations section and suggest ways to overcome them in future work.
- Minor issues: Please check the paper for spelling or grammatical errors to improve readability and quality.
Overall assessment scores: 2.5