Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work provides a reinforcement learning solution for the insight selection problem and use two experiments to verify the feasibility of the proposed framework. The main claimed contribution is that the framework can provide insights that are both relevant to user preferences and improve users' healthcare. Preliminary experimental result on the American Time Use Survey 2003-2020 shows that the proposed RL solution outperforms insights from multiple pre-defined objectives.
strength_and_weaknesses: Strengths:
1. It is an interesting idea to generate and select insights using the reinforcement learning diagram. 
2. The paper is well organized, with a good hierarchical structure and clear chapter headings.

Weaknesses:
1. This paper concentrates on two kinds of insights, insights that are appreciated by the user and insights that are beneficial to their life quality; what is the relationship between them? The article does not clearly explain.
2. Since the American Time Use Survey (ATUS) 2003-2020 dataset exists, can the supervised learning method and the reinforcement learning framework proposed in this paper be compared with the experimental results?
3. If it is difficult to compare with a supervised learning framework, at least compare with a framework that is also modeled by MDP, such as works in the survey by Afsar et al. (2021).
4. In this paper, real life is modeled in the offline dataset ATUS by state machines. If a state outside of the dataset emerges in a real-world application, how does the insight selection network make decisions?
5. Some writing errors, such as " Those subjects have been selected to tell to the policy network about what measurement is the insight, if it compares it to the benchmark value and to which day of the week it refers to.", and "On figure 6 is presented the behavior of the policy network after 12.000.000 steps of training." and so on.
clarity,_quality,_novelty_and_reproducibility: The paper is well-writhing and motivated. The main contribution is the use of RL algorithm in the insight selection problem.

summary_of_the_review: The article provides a reinforcement learning solution to the insight selection problem, and it would be nice to have a more detailed experimental comparisons, analyses, and discussion to highlight the technical contribution of this work besides only using the RL algorithm in a new task.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: Summary: The growth of sensors and Internet of Things techniques has made it possible to collect an increasing amount of data, which can then be analyzed for its patterns in order to enhance the applications that are linked with those patterns, especially in the field of personal health, where a large amount of data may be applied to the understanding of users' living behaviors and the indirect improvement of their way of life. Systems that are able to recognize these patterns and translate them into a text format that is easily readable are referred to as insight generators. The authors of this study present a unique reinforcement learning (RL) framework for insight selection. This framework has the potential to be utilized in order to both evaluate the authors' lifestyle quality and capture the authors' usage interests. Experiments have shown that RL has the potential to improve the selection of insights toward a number of different pre-defined goals.


strength_and_weaknesses: Strengthes:
1. The research topic of this paper is personal health, which is meaningful and socially impactful.
2. The authors provide a practical RL framework to comprehend user preferences.

Weaknesses:
1. The presentation of this paper is not good. Readers cannot quickly get the main contributions and novelties. 
2. The novelty of this paper is limited. All the key concepts and techniques have existed in a lot of literature. 
3. This paper's technical depth is limited. The writers did not derive an adequate research concept from the issue. Numerous paragraphs are used to explain data extraction and processing procedures. They are, however, too insignificant. This work identifies a suitable application subject, however, the authors should propose their own contributions in addition to applying RL to the problem. 
4. The authors didn't provide code and data links to improve their reproducibility. 
5. Experimental design is limited. The authors should design more case studies and ablation studies to illustrate the effectiveness.
clarity,_quality,_novelty_and_reproducibility: I have provided detailed comments related to clarity, quality, novelty, and reproducibility in the weaknesses section.
summary_of_the_review: This paper studies an interesting research topic and proposes a practical framework. But it has the following limitations: 
1. The presentation of this paper is bad. 
2. The novelty of this paper is limited. 
3. The technical depth of this paper is restricted.
4. The reproducibility of this paper is bad.
5. The experimental design of this paper is incomplete.
Based on above limitations, I prefer to reject this paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose a new framework for insights selection in health data based on reinforcement learning. Insights are actionable interpretations of analysis of data that originates from users' behavior.
The paper proposes to create a large list of candidate insights, which are then scored and filtered based on these scores. The RL problem is then formulated as an insight selection problem, where the RL agent learns to pick which insight from the candidates to present to the user, and the reward depends on the topic of the insight selected.

Through a simulation, the authors show that using an RL framework they are able to influence a user behavior to achieve positive outcomes. 
strength_and_weaknesses: Strengths:
I found the problem and use of RL to be really interesting in this scenario. 
The authors explanation of how they modeled user behavior and the decisions made to create the simulator; while the work itself has the potential to be impactful.


Weaknesses:
There are a few key limitations in the work presented:
1 - The user is assumed to only be in one state at a time, but people do more than one thing at a time regularly. For example, it is not uncommon that people eat while traveling or working.

2 - In section 3.2.1, the authors state that  promotion of time Working is greater than Shopping on weekdays, and the other way around on weekends. This really depends on the job, which leads me to think that there's no notion of personalization in the state transitions.

3 - The paper results rely solely on the model used for the simulation, where the reward function has been design to give a clear learning signal. It is still unknown to what extent the insights picked by RL would influence a user's behavior.
clarity,_quality,_novelty_and_reproducibility: Clarity:
- I was a bit confused about the example in the first paragraph of section 3.1, it's not very clear what "sleep_period:1 measurement measurement_benchmark: 2" means.
- "Insight generation is performed using the insight generator described in ....", the paper referenced is very recent, so it would be helpful for the reader to get a quick overview of the method so the paper is self contained.
Other than that, I found the paper easy to follow.

Quality & Originality:
The paper is well organized, and contains a detailed description of the methods used and experimental setup. 
I also found the work to be original in its use of RL for insight selection.
summary_of_the_review: Overall, this is an interesting paper with promising results. 

However, given that all the evaluation was done in simulation, it is hard to tell what the impact of such a system truly is. 
It is already known that for a well defined MDP such as the one in Figure 2a, RL systems will be able to improve their return, so that in itself is not novel. The real question would be if users respond in such a way at this system.

As it stands, to me this looks like a promising work that's missing that key component in the evaluation.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Insight selection is an uncommon and potentially novel application topic. This paper is not very convincing, however. Most importantly, no real-world data is involved, only a simulation that is rather simplistic and limited in many ways. It is not clear what we can learn from this. Additionally, the actual RL method is not very novel.
justification_for_why_not_higher_score: Lack of novelty, simplistic simulation, no real-world data.
Decision: Reject