###### Abstract  
The increasing number of daily notifications generated by smartphones and wearable devices increases mental burdens, deteriorates productivity, and results in energy waste. These phenomena are exacerbated by emerging use cases in which users are wearing and using an increasing number of personal mobile devices, such as smartphones, smartwatches, AirPods, or tablets because all the devices can generate redundant notifications simultaneously. Therefore, in addition to distraction, redundant notifications triggered by multiple devices result in energy waste. Prior work proposed a notification management system called _PASS_, which automatically manipulates the occurrence of notifications based on personalized models. However, machine-learning-based models work poorly against new incoming notifications because prior work has not investigated behavior changes over time. To reduce the gap between modeling and real deployment when the model is to be used long-term, we conducted a longitudinal study with data collection over long-term periods. We collected an additional 11,258 notifications and analyzed 18,407 notifications, including the original dataset. The total study spans two years. Through a statistical test, we identified time-invariant features that can be fully used for training. To overcome the accuracy drop caused by newly occurring data, we design windowing time-invariant online learning (WTOL). In the newly collected dataset, WTOL improves the F-score of the original models based on batch learning from 44.3% to 69.0% by combining online learning and windowing features depending on time sensitivity. n pacs: 23.60.-s +
Footnote †: journal: Applied sciences  
automatically defers phone-preferable notification delivery and consecutively forwards them when a watch-preferable notification occurs. However, there is still a lack of longitudinal studies to understand user behavior changes over time and to evaluate the quality of models. Most of the models were built and evaluated using batch learning, which splits data into training and testing sets. batch-learning-based models face a drop in accuracy when deployed _in the wild_ for long-term periods owing to a lack of updates on newly available data. The accuracy drop is mainly caused by the existence of changing data, which is different from the situation when the model was trained; the changes include changes in users' context, personalized content in a notification, social relationships, or breaking news. Over passing periods, such drastic changes may lead to incorrect classification. Therefore, there is a need to retrain these models. However, to the best of our knowledge, this has not been entirely investigated and remains an open issue in notification management in multi-device environments. In this paper, we focus on a longitudinal study to reduce energy consumption in multi-device environments. Our study is the first to investigate the effects of user behavior change over time in intelligent notification systems. From the longitudinal study, we aimed to answer the following research questions: (i) How accurate are batch-learning-based models for long-term periods? (ii) How can invariant features be distinguished over long-term periods? (iii) How can the accuracy drop caused by user behavior change over long-term periods be overcome? To answer these research questions, we reused the existing models trained in _PASS_[17] that reduce redundant notifications in multi-device environments and directly deployed the trained models to multiple users for multiple years. After an average of 593 days from the end of the prior experiment [17], we collected an additional 11,258 notifications over 68 days on average from five users who participated in the original experiment. Data collection spanned approximately two years. With the new dataset, we performed the following experiments and discovered interesting aspects that are related to the aforementioned research questions.