CareGraph: A Graph-based Recommender System for Diabetes Self-Care
Authors: Sirinart Tangruamsub
Karthik Kappaganthu
John O'Donovan
Anmol Madan
Venue: ICLR 2022 Submitted (2022)
Keywords: knowledge graph
knowledge graph embedding
recommendation system

Abstract: In this work, we build a knowledge graph that captures key attributes of content and notifications in a digital health platform for diabetes management.  We propose a Deep Neural Network-based recommender that uses the knowledge graph embeddings to recommend health nudges for maximizing engagement by combating the cold-start and sparsity problems. We use a leave-one-out approach to evaluate the model. We compare the proposed model performance with a text similarity and Deep-and-Cross Network-based approach as the baseline. The overall improvement in Click-Through-Rate prediction AUC for the Knowledge-Graph-based model was 11%. We also observe that our model improved the average AUC by 5% in cold-start situations. 

Under review as a conference paper at ICLR 2022
CAREGRAPH: A GRAPH-BASED RECOMMENDER SYSTEM FOR DIABETES SELF-CARE MANAGEMENT
Anonymous authors Paper under double-blind review
ABSTRACT
In this work, we build knowledge graph that captures key attributes of content and notifications in a digital health platform for diabetes management. We propose Deep Neural Network-basse recommender that uses the knowledge graph embeddings to recommend health nudges for maximizing engagement by combating the cold-start and sparsity problems. We use a leave-one-out approach to evaluate the model. We compare the proposed model performance with text similarity and Deep-and- -rooss Network-basse approach as the baseline. The overall improvement in Click-Through- Rate prediction AUC for the Knowledge- Saapledge-Grahhbbeee Graph-based model was 11%. We also observe that our model improved the average AUC by 5 % in cold-start situations.
INTRODUCTION
The recent global pandemic has brought with it permanent shift away from traditional in in-person health consultations, towards large digital telehealth platforms that support remote consults. This research performed in the context of one such platform that provides mobile application to help people manage chronic diseases such as diabetes, hypertension and obesity. As solution designers, some of the key problems we face to get our users' attention, foster awareness and encourage them to take actions that help manage their chronic health conditions. Sending notifications that nudge users is one efficient way to encourage engagement. However, user preferences for engaging with these nudges can vary greatly, different users require different persuasion techniques. Even within a homogeneous messaging context, different tones in messages can appeal to different users.
Recommendation systems can be applied to model preference patterns and predict effective personalized nudge notifications. Collaborative recommender systems are well established tools for predicting user preference and have been shown to perform very well, provided there sufficient information available to model the users preferences, as highlighted by seminal papers in the field such as |Rennikk et al. (994). For new or esoteric items, or users, there is frequently a lack of sufficient information to make a good prediction. These conditions, known in the literature as 'cold start" are the focus of our experiments in this paper. Specifically in the context of recommending health nudges, which are created by experts who inherently follow guidelines set by associations such as American Diabetes Association ame (2021). Our primary research question asks whether a knowledge graph can be applied to mitigate cold start problems, in the specific task of recommending a finite set of highly structured health nudge messages.
1.1 KNOWLEDGE GRAPH-BASED RECOMMENDATION
Online content and services have undergone a huge volume of growth in recent years. Accordingly, recommender systems are increasingly relied upon to help people get to the right information at the right time and also in the right way [Ricci et al. (2 They help customers shorten their times exploring products or services in various applications such as news portals [Wang et al. (2018b)) ]. E-commerce [Zhang & Jiao (2007), Hwangbo al. (2018), accommodations [Haldar al. (2019) ]. or music recommendation Van Den Oord et al. (21 However, our problems are different from these recommendation applications. First, in other applications like E-commerce, users can see list of products and select their interested items. They can visit websites to search for items of interests anytime they like. In our nudge notifications. our users can just see one nudge time, and hence we cannot flush lot of nudges as it will disturb our users and can negatively impact their level of engagement. Hence, recommending the nudges that match user preferences while also ensuring that there is a diversity in the actions we are recommending, and achieving good performance in as few notifications as possible is a crucial requirement this setting. Also, we frequently need to

---

Under review as a conference paper at ICLR 2022
support the creation of new nudges. Within this setting, using embeddings from supervised learning methodologies may not be able to solve our problems. To learn efficient embeddings from those approaches, we require a lot of data for each item. Organizing and aggregating nudge attributes such as themes, tones or objectives can help the cold-start problem as we can transfer the knowledge of user preferences nudge attributes to nudges and help us understand and optimize to user behaviors quickly. For example, if a user has positive responses to encouragement-ton nudges, we can send more encouragement-tonee based messages to this user.
Knowledge graphs (KG) have been applied in various tasks such as search engines Yang et al. (2017)) ) text classification [Hu et al. (202T) and word embedding [Celikyilmaz et al. (2015)]. They have also been previously introduced recommendation systems improve the precision of recommended items and the explainability [Wang et al. [. A knowledge graph is a directed heterogeneous graph representing real-world objects and relationship among these objects. Nodes in a KG represent entities, which can be items or the attributes that describe the item. The edges represent relations between each entity. Knowledge graphs give recommendation systems the benefit of enriching the semantic relationships among items [Wang et al. (2019). When we know users' interests in a KG graph, we can extend the diversity of recommended items from the node connections present in the graph.
Our problem domain is selecting nudge notifications for our mobile applications that are personalized to individual users preferences. We have different types of nudge notifications such as alerting users in regards to their health condition, reminding users follow their health routine, providing some useful education related to their health, or introducing new services and events to users. We want to personalize nudges to each user to increase their engagement and reduce over-sending nudges that might disturb the users. The contributions of this work are summarized as follows:
We propose a new method using a neural network that combines user attributes with a knowledge graph to overcome the cold-start problem.
We conduct experiments demonstrating the effectiveness of our new approach, particularly under cold-start conditions.
1.2 DIGITAL HEALTH SYSTEM FOR DIABETES SELF-CARE
This work is based on system that leverages Blood-Glucose (BG) monitoring device, and a Mobile App that is used to provide useful information and suggest actions to users that are relevant and helpful for managing Diabetes. As a part of this system, users provided with short text messages, about 180 characters long. The text is designed to nudge a user to specific actions that are relevant to managing the condition, such as, healthy recipes ideas, articles that explain and motivate users add exercise their routine, reminders to check and monitor BG levels, talking to expert coaches etc.
The short text is accompanied with two buttons, one of which is used to guide the target user to perform specific action (e.g. read content, schedule call with coach), and the other can be used to dismiss the recommendation. In this work we call these recommendations on the Mobile-App. Mobile Nudges (or Nudges for short).
RELATED WORK
2.1 RECOMMENDER SYSTEMS IN DIGITAL HEALTH
Over the past decade, a rapidly increasing volume of digital information to support clinical decision making has become available to be leveraged by healthcare recommender systems. Automated health recommender systems have been deployed in various domains. Some applications aim for improving lifestyle to be healthier through diet [llseellrr et al. (017)),Achaaamm [Achnanuuarp & Weber (016)) or physical activity recommendations [Dharia et al. (0 One such system, known as Pro-fit [Dharia et al. (2016) applies hybrid approach to personalize lize workout sessions based on user's contextual data and calendar events. |Achananuprrp & Weber (2016) proposed a healthier food substitute suggestion system by introducing a food-context matrix and applying Singular Value Decomposition to get the low- -ow-dimeniional representation of each food item. The similarity between two food items is measured by cosine similarity. Narducci et al. (2015) introduced an HealthNet personalize doctors and hospitals to users, given the user profile and the health data shared by the community. The iDoctor system in Zhang et al. (2017b) provides doctor recommendations by using sentiment analysis from their rating and reviews and Latent Dirichlet Allocation for user preference and doctor features. The hybrid matrix factorization is applied to predict the doctor rating.
2

---

Under review as a conference paper at ICLR 2022
Recommendation systems have also been used for optimizing treatment plan such drug recom- mendation (Zhang et al. (2017a). Stark et al. (2017)) Zhang et al. (2017a) applied the concept of neighborhood-basee method in drug-drug interaction prediction to help reduce unexpected effects from using multiple drugs. A drug recommendation system for Migraine diseases proposed by |Stark et al. (201)) using graph datahase and collaborative filtering approach.
2.2 KNOWLEDGE GRAPH EMBEDDING
Knowledge graph embedding (KGE) is a process to transform a knowledge graph into low-dimensional continuous vector space which still preserves the network structure information. The knowledge graph can be represented as a group of triplets, each of which contains two nodes (items or attributes) and the relationship between these nodes. With these triples KGE will project all entities and relations into a low dimensional vector space that preserves their graph structure in these vector representations. There are several approaches for building these representations in the KGE setting. Translation distance models such as TransE [Bordes et al., (2013)), TransH (Wang et al. 2014)] TransR (Lin et al. [2015)] and semantic matching models such as DistMult (Yang et al. (2014) are some of the popular methods.
2.3 KNOWLEDGE GRAPH EMBEDDING IN RECOMMENDATION SYSTEMS
Knowledge graphs have been applied to many recommendation applications. Wang et al. (201 8a) proposed the RippleNet, an end-to-end recommendation framework that incorporates knowledge graph into recommendation systems. RippleNet extends user interest of items through links in the knowledge graph which help increasing the diversity. [Wang al.) (2018b) introduced Deep Knowledge-awand Network (DKN) for news recommendation. The key component of DKN is knowledge-awee lutional neural network (KCNN) for incorporating word-level and knowledge-leva representations. Zhang et al. (018)) extends collaborative filtering framework to learn over the knowledge graph embedding. A user-item graph is used, where each connection between nodes depicts how a user interacted with an item, for example (buy, also bought, also view)
convo-
Another method for using knowledge graph in recommenation system is path-based approach, which uses the connection patterns in the knowledge graph to generate the recommendation. Hete-CF [Luo et al. (2 a collaborative filtering recommendation method on Heterogencous Social Network, which combines difffrent types of meta-paths (user-user, user-item and item-item) and calculates the similarity. Yu et al. (2014) introduced HeteRec which represents the connectivity between users and items with meta path-based latent features. HeteRec recommendation models are further defined into two levels: global (HeteRec-g) and personalized (HeteRec-p) levels. In the HeteRec-p. users are clustered based on their interests and preferences into subgroups and then learn recommendation models for each user subgroup.
PROBLEM FORMULATION
Our nudge recommendation can be seen as a binary classification problem where we try predict the probability that a user will have a positive engagement with a nudge (i.e. accepted the action or suggestion provided by the nudge). Let U {u1, "2,.... } and V {V1, U2, ... } denote the sets of users and nudges respectively. A user-item interactive matrix is defined as Y {Va, /u E U where
1 the user u positively engaged with the nudge yu,v = {
otherwise
We want our model to learn and predict probability of will click on the nudge Uj or Dar,yy f(u,, vz).
METHODOLOGY
Our approach nudge recommendation is based on a combination of item and user-based collaborative filtering. In our model, user is represented as a triple u E d, i, where d is the set of demographic profile information such as age and gender; is the set of clicks on nudges, and a the set of attributes of the nudges that were effective in engaging that user. The objective is to transform users and items (in our case, nudges) into vector space and make a prediction based on similarity the latent space. To learn an embedding of a user profile, we rely on their click history. This information helps us to passively infer preferences based on how they interacted with nudges in the past, a candidate item
3

---

Under review as a conference paper at ICLR 2022
is similar to user's history of clicked nudges, it is likely that user will also like this nudge too. However, using only a profile of click history has significant limitations and can result in a narrow set of recommended items, lacking in diversity -essentially a similar narrowness problem to that of a traditional content-based recommender system. To mitigate this problem, which is particularly apparent in cold-start conditions where there less click data available, we proposed novel use of "nudge attribute" preferences to user embeddings. Attributes of recommended nudges effectively extend a user's preference profile by creating new edges to explore in the item-to-item graph. Even for a completely new user with no click history, we can infer nudge attribute preferences from the demographic profile until preference data becomes available through click interactions.
The model training occurs in two parts. We first train knowledge graph embeddings to generate nudge, nudge attributes and relation type embeddings. Next, we construct a deep neural network model to predict the click-through rate (CTR) probability of a user u, on nudge uy. The user embeddings eu: (section ) are constructed by combining the user click history embeddings hu,, nudge attribute preference embeddings nu, and user profile embeddings Pu.. To learn nudge attribute preference embeddings, we apply the concept of attention mechanism Vaswani et al. (2017) to get set of values for our aatention-based network layer. A nudge prediction is then calculated from a function of Cv, and ev, where evy is an embedding of nudge Uj.
4.1 KNOWLEDGE GRAPH FOR DIABETES SELF-CARE
We evaluate our proposed models for CTR prediction on our mobile nudge dataset, which we described in Section 1.2 The actions suggested to the users in the nudges cover themes such as monitoring their blood glucose level, improving physical activity, adhering to medication etc. These actions may have different levels of effort to complete. For example, requesting healthy recipe might be done directly on the mobile application, while talk to a coach or an expert might require the user to find an available time and schedule. We also have nudges that might recommend the same action but use a different tone in the communication text. Furthermore, the nudges can also be categorized based on the intent of support the nudges are be providing, such as providing knowledge or requesting preferences through short questions. We use such categorizations that are used to create new nudges for the users as the attributes for the nudges. These attributes capture not only the theme and intent of the nudge but also some information around the experience accessing the working on the action recommended by nudge.
Table A shows seven attribute types with descriptions and examples values relating to nudge "In a dinner rut? Our meal plan is packed with tasty, colorful, and super-healthy recipes, Want it?". These attributes and the associated values have been developed by working with the business and subject matter experts.
Ssoww two example health nudges with their associated, and sometimes overlapping attributes. The full set of nudges and attributes in our library are used to define a graph that becomes the initial input the CareGraph recommender system. The nudges and the attribute values form the nodes of the graph, as highlighted in Figure 3 For example, the blood-glucose monotoring nudge and the holiday nutrition nudge both share the attribute that they send a user website for additional information, and that they foster broad awareness of the chronic health condition. More formally, the attribute types are the relations that define the edges of the graph, i.e. the edge describes how a nudge is related to the attribute. This can be represented in a form of triples (h, r, L) where h is the nudge (head), r is the attribute type (relation) and t is the attribute value (tail). For example, if nudge CGM CGG..CONTEE CONTENT _GGM COMP ARE has monitoring the theme, then this relationship is theme, monitoring, and forms the triplet
Figure 3 shows
represented as .CONTENT (CGM CRMMCCONTNT .GGM COMP ARE, theme, monitoring).
While this is useful representation for the structure with which nudges created and described, it will be beneficial to develop continuous representations that can be used for training a recommender model. These representations are called knowledge graph embeddings, and are described in Section There are various algorithms that can be used develop these embeddings. Given the relatively simple structure of the graph and the simplicity of the algorithm we use the standard TransE algorithm Bordes et al. [2013) in our methodology.
2.2
4.2
CAREGRAPII MODEL
The CareGraph model, shown as flow diagram in Figure is trained in two phases. We first train knowledge graph embeddings for nudges, nudge attributes and attribute types embeddings. Next we
4

---

Under review as a conference paper at ICLR 2022
historical clicked maddees
neightoc entilies A (nudge attribunes)
peofile
O
(R2)
Pa user embbeddiig (o
mechaniem
click hixuary [owigen_Mee Nirr
user N,
Prodicted Prob
item emboodinng
mxdge
user
softmax
Figure 1: CareGraph model architecture.
construct deep neural network model to predict the CTR probability for a target user u, on nudge vy. The user embeddings eu, (.2..1 are constructed by combining user click history embeddings hags nudge attribute embeddings nu; and user profile embeddings Pu,- The prediction calculated from function of ea, and Czy where Cvy is an embedding of nudge ny..
4.2.1 USER EMBEDDING
User click history embedding A user's click history helps us understand which nudges the user liked in the past. We gather all clicked nudges that the user u, clicked in the past and construct a click history list niste, where niste, = {uy] b E V, ya,. = 1}. For users who have never clicked on any nudge, their click history will be an empty list. To represent nuude__hist as a embedding vector Nu we use embedding vectors for all U/ nadge.histt from our knowledge graph embeddings and use the equation aggregate them.
E Evvenndge.Jiiet
Cek + 6.
h W.
(2)
|reudge histu,
where evy is an embedding vector of the nudge Ug from the knowledge graph embedding. In the equation 2 we first average all embeddings in nudge histu, and apply linear function where W.. and bn are transformation weight and bias respectively. For user whose click history is empty, the hm will be zero vector.
Nudge attribute embedding In user click history embeddings, we focus only nudge embeddings that the user interacted with the past. We can leverage nudge attribute nodes extend user preferences by propagating to neighbor entities around users' historical clicked nudges. Since neighbors of nudge entity are nudge attributes entities, we refer to this as the set of attribute embeddings Th..-
From a intuitive motivation that different users have different preference on nudge attributes, for exam- ple, user may be interested in a formal clinical information nudge than cheerful-tone encouragement nudge, we should use user preference weights on nudge attributes instead averaging them directly as in equation 2 Given set of nudge attributes and their relation types, apply the concept of attention mechanism Vaswani et al. (2017) to learn user preference weights of each nudge attribute.
Considering an nudge entity h in the knowledge graph where its connections are represented in the form of (h, f), The set of neighbor entities, or nudge attributes, of user u, is N., = {1/h E mudgehhist,. (h, f) E kg} and the relation types R,, = (r(h E mudge.hista, rh, E kg). For user whose click history list is empty, we randomly select set of nudge attribute entities. The embedding aggregation of nudge attributes can be formulated as

---

Under review as a conference paper at ICLR 2022
eN., = Em,eN. Og
(3)
Where a (aq, a22 a2))). is an attention weight vector. We calculate the vector from user profile features Iu, of the user u by the following formula:
I (W + b,) 0: EnjER., e7, (W9g. + b,)
(4)
The W, and b, transformation matrix and bias respectively. It will transform the user profile feature 9u into the vector space and then perform the inner product with the embedding vector er, of the attribute type A softmax function is used to compute user relation type preference weights, and then the final nudge attribute embedding nh, is calculated from this equation:
nu. W..N.,, bn
(5)
User profile embedding We also include explicit user features into the user embedding. All user raw data is transformed into user feature vector and then apply a function to get the final representation.
Pa. Wp... + bp
Information Aggregation The last step to get the final embedding of the user u, is by combining user history embedding h. from equation? 2 nudge attribute embedding nu, from equation and user profile embedding Pu, from equation 6 We concatenate all three embeddings and transform by a linear function to get the final user embedding eu.-
On; hu, You, Po.
eu = Wn, + b.
(8)
4.3 MODEL PREDICTION
To calculate the probability that the user u, will click the nudge uj. We use the final user embedding eu; from equation & and nudge embedding e,y of the nudge vy from knowledge graph embedding. We predict a user engagement score from the matching
by
(9) where a is a nonlinear function, similar to the sigmoid function. We can see from the equation 9 the CTR prediction of user u; and nudge Uy Uru; Uz), is dependant on how similar the user embedding eu, is to the nudge embedding Eyg.
Ú( 02) = a (W.e., + b,))
5 CAREGRAPH WITH TEXT EMBEDDING
The CareGraph model supports addition new features such as text embeddings. A basic nudge a short text message, and Here the premise is that text similarity across the broader library nudges can be leveraged to improve the relevance of nudges for target user. In this section, introduce two such extensions of the base model architecture: CareGraph + TextSim and CareGraaph-eetttmmeele
aggregate these embeddings into the model to improve its performance.
CareGraph++exx TextSSi The approach taken in this model stems from our intuitive assumption that if text message of nudge v, is similar to contents in historical clicked nudges of user u, it will be likely that a user u; will also like the nudge Uy. With this assumption, we add text similarity score at the prediction step. The other parts of model are still the same. The text similarity score tuy is be calculated by equation 10
Ene Enudge hist.., dist Foyz)
Lug
(10)
Trudge.hist..
where v, is text embedding of the nudge and dist is distance function of text embedding, such cosine similarity. The CTR prediction score in equation | wiill be replaced with
0 U, v,) = o (w) e ev, + w2tx,)
(11)

---

Under review as a conference paper at ICLR 2022
CarGGraph+TextEmt Another way to include text embedding into the CareGraph model is include them as part of the user embedding eu in equation and and nudge embedding in Eeg-
We first get text embeddings of all nudges reudge_hisl. and aggregate them by simple averaging.
En Emndngge Jist.; Vvn. Iuudge Juisbs
qu:
(12)
t.. Wiqu. + be
(13)
To include text embedding t., into user embedding Cu. we update the equations 7 as follow:
Ou, = ha, 1 Ylu
@ Pu, + tu.
(14)
and then calculate the final Cu in In the equation 9 add text embedding of the nudge 2 into e.y
by
Deu Uz) o oee (Wu(ev, @ )z,)) + b.))
(15)
6 EXPERIMENT SETUP
The data used for this analysis consists of interactions from 283,743 unique users. As mentioned in the description of the Nudges system, the nudge consists on two buttons, one for taking the suggested action and the other for dismissing the Nudge. For the purpose of this analysis user accepting the recommended nudge is considered positive rating or a click. The data has been collected from 283,743 users over a period of 12 months, and have average Click- Click-Through-Rae Rate (CTR) of 16.77%.
6.1 PROCEDURE
In order to determine the performance the proposed model two distinet cases are considered. In the first case we evaluate the improvement in predicting the click behavior across all users and all available Nudges in the data. the second case, the performance of the model evaluated Cold-Start cases. Specifically, the case when new Nudges are added is simulated by leaving one nudge out in the training data and determining the performance in predicting the click behavior for that Nudge. This is a typical situation that is encountered in practice where new content and nudges are created and it is important to direct them to most receptive users in a timely fashion. The performance of the proposed model in both cases is compared to a Deep and Cross baseline model, as described below.
6.1.1 BASELINE MODEL FOR PERFORMANCE EVALUATION
Deep and Cross Network (DNC), similar to Wang et al. (202T) is used as the baseline model to predict the click on the Nudge. The DCN model starts with embedding and stacking layer transform categorical features into a dense vector. A Cross Network and Deep Network work in parallel and their output is combined in the last layer. The Cross Network is composed of cross layers which applied feature crossing at each layer. A cross layer is to calculate this following function:
W1+1 x T + bi + I1 f(x1, Wt; bi) 27
(16)
where æy, 2141 E Rd is the output vector at the layer 1-th and + 1-th layer, respectively. The zo is the embedding vector from the embedding and stacking layer. By adding more cross layers into the cross network, it increases the polynomial degree of the input 2o-
The deep network is standard fulny-connected feed-forward neural network. For each layer, the output of the subsequent layer + 1 calculated by:
h f(Whi + by)
(17)
where W, is weight matrix and is the bias of the layer 1. The final output of the deep network is an embedding vector h where L. denotes the total number of deep layers. The prediction step is performed by concatenating the outputs from the cross network and the deep network and feeding them
7

---

Under review as a conference paper at ICLR 2022
Table I: The AUC results of nudge CTR prediction in a general case
Appeoach
ALC
Deep-und-cross model
0.6421
Caregraph model
0.7142
Caregraph=+TextSe model
0.7156
Caregraph+ Tenegrapp-TeexEnndd TextEmb model
0,7113
to the logit layer. The sigmoid function is used to calculate the final prediction on how likely the user is engage with the recommended nudge.
6.2 TRAINING THE CAREGRAPH MODEL
We trained our knowledge graph embeddings for all nudges, nudge attributes and attribute types (relation types). The embedding sizes in the knowledge graph embedding are set to 100. We use the pretrained Universal Sentence Encoder from Tensorflow Hub fatts::/www.eensffllooror///// to extract the text embeddings.
We use Adam to train both of our recommendation models and the deep-and-cross model by optimizing log loss. We apply hyper-pes hyper-parameters parameters search for knowledge graph embedding size, hidden layer units and L2 regularization. The hidden layer in equation 2 and [ are set to 24 dimensions. For the deep and cross model, the final model architecture is two hidden layers with 240 and 48 dimensions in a deep network and cross layers in the cross network. To evaluate recommendation results with the deep-and-cross model, we use AUC as the evaluation metric.
6.3 RESULTS
We evaluate our proposed model with the baseline model in general case and cold-start situations.
General case This objective of this experiment is to compare recommendation results of our proposed models with the baseline model in a overall case. We split the training, validation and testing data into 60:10:30 respectively. We compared AUC from our three proposed models: CareGraph (section 4.2 CareGraph+TextSinn (section 5) and CareGraph+TextEme (section 5) with the deep-and-cross and-cross model. The results are shown in Table
From the table all CareGraph approaches outperform a deep-and-cross model. We have tried two different model architectures that integrate text information into our CareGraph model. Our initial hypothesis was that the text embeddings would improve recommendation accuracy because similarity could be computed on more underlying data. However, results showed that this was not the case. AUC did not improve in the text-enabled treatments. possible reason for this unexpected result is that the kg-embeddings may have already captured enough information to generate high quality predictions. Further studies of model architectures CareGraph with text may be required to prove that this generalizes to all integrations of text similarity to the model.
Cold start experiments We simulated cold-start cases by using leave-one-our approach. Among 155 nudges, one nudge is selected as testing data and keep the the rest nudges for training. We repeated this process for all 155 nudges and compare all AUC with the baseline. In this experiment, our CareGraph model with the haseline model. For the baseline approach, since the deep-and-cross model cannot learn the nudge embedding for new nudge in the training process, results of the most text-based similarity nudge in the training data for a new nudge. In average, our approach has 61.76% AUC and the baseline approach have 58.17% AUC. Our model outperform the baseline model for 96 nudges out of 155 nudges. The distribution of AUC over all nudges shows in Figure 2
compare
the prediction
Cold start analysis The main objective of using a knowledge graph embedding is to overcome the cold-start and data sparsity problems. A key question in this research is whether knowledge graph embeddings of nudges, nudge attributes, and attribute types capture enough information for new nudge prediction. If we have new nudge that is different from existing nudges, how that will impact to prediction performance?
In the cold-start experiment, we compare the prediction results with a text-based similarity approach. Text-similarity based approach is heavily depends existing nudges in the training dataset. If text message of new nudge has different meaning from the existing nudges, the prediction result can be

---

Under review as a conference paper at ICLR 2022
Text similanty to the neurest midge
approach CareGrrph deep-and-eross model
<<' 0.3
2 0.5
0,5816
0.5822
The distribution of AUC
our wwwpppopodd___oc model
awaaann-rrre andi-crass
13
Tec--kned Deep-1t5-3m8
Danoty
14
ctr pac
bedt stance nearett mudge
Figure 2: Left: The distributions of AUC, and the distribution w.r.t text similarity.
less precise. In our mobile application, it is very likely that we will develop new content. For the knowledge graph embeddings based approach, the embeddings of each nudge preserve connection structures with other entities. Even when we have a new nudge from new service or program, as long as there are shared attributes with existing nudges, the kg-embedding far away from existing nudges in the vector space. Also, by adding our nudge attribute embedding nh:; into the user embedding cu, this embedding will show how this new nudge is similar to user attribute preferences.
new nudge will not be too
in Equation S
In Figure 2 we compare the AUC distributions of CareGraph and the text-based similarity deep-and- cross approaches over the text distance. The r-axis shows the nearest text cosine-similarity of each new nudge existing nudges. A lower distance means new nudge content is different from its nearest existing nudge. At the text distances ranging between 0.4 and 0.6, there is a group of nudges that the baseline approach has lower AUC than our proposed approach. Also, if consider the AUC results of nudges whose nearest text cosine similarity is less than 0.5, our approach improves the AUC by 6.11 17% and 5.24% for greater than equal to 0.5. This result shows that improve AUC more if a new nudge has more different content than existing nudges.
FUTURE WORK AND CONCLUSION
This research produced several avenues for future work, including analysing different methods integrate text-based similarity into the model to improve both regular performance and performance under cold start conditions. To conclude, the primary research question in this work asked whether a knowledge graph can be applied to mitigate cold start problems, in the specific task of recommending a finite set of highly structured health nudge messages. To facilitate this we developed a novel knowledge graph based recommender system, which we called CareGraph, for predicting health nudges to manage chronic discase on a large digital health platform. Our results confirmed that the knowledge graph does benchmark
improve predictive accuracy in cold-start situations, showing 5% improvement model. In addition, a text-similarity based model was evaluated and it was shown that structure in the knowledge graph produced more accurate recommendations than those based text similarity.
REFERENCES
Introduction: Standards of medical care in diabetes- -2021. Diabetes Care, 44(Sppplement 1):S1-S2. diabetes journal
2021. ISSN 0149-5992, doi: 10.2337/de21-Sim
httg:cootett//oe /4/ http://4///ppppppppepmmm
Palakorn Achananuparp and Ingmar Weber. Extracting food substitutes from food diary via distribu- tional similarity, 2016.

---

Under review as a conference paper at ICLR 2022
Antoine Bordes, Nicolas Usunier, Alberto Carcia-Duraie Jason Weston, and Oksana Yakhnenko. Trans- lating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26, 2013.
Asli Celikyilmaz, Dilek Hakkani- Tur, Panupong Pasupat, and Ruhi Sarikaya. Enriching word embed- dings using knowledge graph for semantic tagging in conversational dialog systems. 2015 AAAI Spring Symposium Series, 2015.
Saumil Dharia, Vijesh Jain, Jvalant Patel, Jainikkumar Vora, Shaurya Chawla, and Magdalini Eirinaki. Pro-fit: A personalized fitness assistant framework. In SEKE, 2016.
David Elsweiler, Christoph Trattner, and Morgan Harvey. Exploiting food choice biases for healthier recipe recommendation. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017.
Malay Haldar, Mustafa Abdool, Prashant Ramanathan, Tao Xu, Shulin Yang, Huizhong Duan, Qing Zhang, Nick Barrow- Wiillss,,, Bradley C Turnbull, Brendan M Collins, et al. Applying deep learning to airbnb search. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1927-1935, 2019.
Linmei Hu, Mengmei Zhang, Shaohua Li, Jinghan Shi, Chuan Shi, Cheng Yang, and Zhiyuan Liu. Text-graph enhanced knowledge graph representation learning. Frontiers in Artificial Intelligence, 4, 2021.
Hyunwoo Hwangbo, Yang Sok Kim, and Kyung Jin Cha. Recommendation system development for fashion retail e-commerce. Electronic Commerce Research and Applications, 28:94-101 101, 2018.
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In Twenty-ninth AAAI conference on artificial intelligence, 2015.
Chen Luo, Wei Pang. Zhe Wang, and Chenghua Lin. Hete-cf: Social-based collaborative filtering recommendation using heterogeneous relations. In 2014 IEEE International Conference Data Mining, pp. 917-922, 2014.
Fedelucio Narducci, Cataldo Musto, Marco Polignano, Marco de Gemmis, Pasquale Lops, and Giovanni Semeraro. A recommender system for connecting patients to the right doctors in the healthnet social network. In Proceedings of the 24th International Conference on World Wide Web, WWW '15 Companion, pp. 81-82, 2015.
Paul Resnick, Neophytos lacovou, Mitesh Suchak, Peter Bergstrom, and John Riedl. Grouplens: an open architecture for collaborative filtering of netnews. In CSCW '94; Proceedings of the 1994 ACM conference Computer supported cooperative work, pp. 175-186, New York, NY, USA, 1994, ACM Press. ISBN 0-89791-689-1. URL U /tttp://dxx....// dof org/ 10. 1145/1 92844 4. 92905
Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor. Recommender systems handbook. Springer, New York; London, 2011.
Benjamin Stark, Constanze Knahl, Mert Aydin, Mohammad Samarah, and Karim O. Elish. Betterchoice: A migraine drug recommendation system based on neo4j. In 2017 2nd IEEE International Conference on Computational Intelligence and Applications (ICCIA), 2017.
Aäron Van Den Oord, Sander Dieleman, and Benjamin Schrauwen. Deep content-based music recommendation. In Neural Information Processing Systems Conference (NIPS 2013), volume 26. Neural Information Processing Systems Foundation (NIPS), 2013.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need, In I. Guyon, U. V. Luxburg, S. Bengio, H. Wal- lach, R. Fergus, Vishwanathan, and R. Garnett (eds.), Advances Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL ht httttps::/pprcceednnete http///occeeen.ccc/oo neueeinnss.nuuuuuuueeee neurips Paper..
10

---

Under review as a conference paper at ICLR 2022
Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In Proceedings of the 27th ACM International Conference Information and Knowledge Management, CIKM '8, pp. 417-426. Association for Computing Machinery, 2018a 8a. ISBN 978 155036014..
Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. Dkn: news recommendation. In Proceedings of the 2018 World Wide Web Conference, WWW '18. pp. 1884.1844. International World Wide Web Conferences Steering Committee, 2018b.
Deep knowledge-aware network for
Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. Knowledge graph convolutional networks for recommender systems. corr abs/1904. 12575 (2019). arXiv preprint urXiv: 12575, 2019.
Ruoxi Wang. Rakesh Shivanna, Derek Cheng, Sagar Jain, Dong Lin, Lichan Hong, and Ed Chi. Den v2: Improved deep cross network and practical lessons for web-scale learning to rank systems. In Proceedings of the Web Conference 2021, WWw '21, pp. 1785-1797, New York, NY, USA, 2021. doi: 10.1145/3442381 3450007
Association for Computing Machinery. ISBN 9781450383127.
URL httttps://oo..o////11// 11411114533344 344.044354007777
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.
Bishan Yang. Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv: 2014
Shuo Yang. Lei Zou, Zhongyuan Wang, Jun Yan, and Ji-Rong Wen. Efficiently answering technical questions-a knowledge graph approach. In Thirty-Firs. AAAI Conference on Artificial Intelligence, 2017.
Xiao Yu, Xiang Ren, Yizhou Sun, Quanquan Gu, Bradley Sturt, Urvashi Khandelwal, Brandon Norick, and Jiawei Han. Personalized entity recommendation: A heterogeneous information network approach. In P Proceedings of the 7th ACM International Conference on Web Search and Data Mining, WSDM 14, pp. 283-292. Association for Computing Machinery, 2014. ISBN 9781450323512.
Wen Zhang, Yanlin Chen, Feng Liu, Fei Luo, Gang Tian, and Xiaohong Li. Predicting potential drug-drug interactions by integrating chemical, biological, phenotypic and network data. BMC bioinformatics, 18(1):1-12, 2017a.
Yin Zhang, Min Chen, Dijiang Huang, Di Wu, and Yong Li. idoctor: Personalized and professionalized medical recommendations based on hybrid matrix factorization. Future Gener. Comput. Syst., 66: 30-35, 2017b.
Yiyang Zhang and Jianxin Roger Jiao. An associative classification-bass recommendation system for personalization in b2c e-commerce applications. Expert Systems with Applications, 33(2):357-367, 2007.
Yongfeng Zhang, Qingyao Ai, Xu Chen, and Pengfei Wang. Learning over kww.lelge--beeeeebeeeeeee embeddiing for recommendation. CoRR, abs/ ht5//8833.000 065400640 2018. URL http:// http//axvvvv.../ oww.vv.org//b//1883. 06540
11

---

Under review as a conference paper at ICLR 2022
A
KNOWLEDGE GRAPH ATTRIBUTES AND EXAMPLE
moniaing
nuniien
reducingist
enoduoaeemet
CGM CONIENT
COMINT NUIBIION APPETIER
descriptive
COMPARE
speciic
CWSrEnies
1803
Figure 3: An example of our knowledge graph of nudges
Attribute Type
Description
Example Value Nutrition
Theme
Top diabetes self-management cate- gories such as healthy cating, physi- cal activity, monitoring etc.
Topics
Key topics around which nudges are designed, such as Reducing Risk, Education, Event participation etc. Attribute capture the level of ef- fort needed to complete the sugges- tion action. e.g. Schedule coach- ing session, Read content on the mo- bile application. etc.
Recipe
Action Mode
Read/View content
Action Channel or Surface
Channel or destination for an re- quired action e-g. redirect to web- site, go the app. redirect coach scheduling channel.
Website (redirect)
Tone
Mood of messages e.g. encourage- ment, descriptive etc.
Descriptive
Intent
Intention categories e.g. Broad awareness, Alert, Suggestion e.g. general, specific to group of users or occasion.
Broad awareness
Specificity
General
Table 2: Attribute types, descriptions and example values for the health nudge "In dinner rut? Our meal plan is packed with tasty, colorful, and super-healthy recipes, Want it?"
12


Reviews:
[{"review_id": "hlY4EHmNEor", "reviewer": "Reviewer_r8hu", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\nThe use of knowledge graphs in a recommender system is a timely, trend topic of research, especially in the healthcare context. \n\nWeaknesses\nIt is difficult for me to assess the quality of the work because \n\n(a) the results don't include sufficient recommendations (output) generated from the proposed framework, therefore, it is hard to assess how the proposed approach can be used for a disease or as proposed diabetes care management; \n- On page 12, under the caption of Table 2, an example is given. However, it is not clear regarding how it is used as a recommendation. I would suggest including some supporting examples such as how these nudges can be used to manage a disease in the body of the paper, and supplying information and examples regarding how recommendations are assigned to users (i.e. predicted probabilities).\n\n(b) In Section 6.2 under the training the model, hyperparameters/optimization is not described sufficiently (e.g., hyperparameter tuning in deep neural network and choice of optimal settings based on the comparison of results using different hyperparameter settings (e.g., learning rate, batch size)). \n\n(c) lack of validation approaches for the output. \n- how did authors validate the knowledge inference (output) (such as correctness or safety since they are recommended to patients  with diabetes)?  \n\n", "summary_of_the_paper": "Paper introduces a deep neural network-based recommender that uses the knowledge graph embeddings, called CareGraph to predict health nudges such as content and notifications in a digital health platform to manage a chronic disease. The research question is whether a knowledge graph can be used to mitigate cold start problems to recommend a set of highly structured health nudge messages. ", "summary_of_the_review": "I would suggest a revision for the paper involving a set of results (recommendations) of deep neural network, with clear illustration of input data (ontology of diabetes care). I would also suggest hyperparameter tuning, and comparison of results based on this. Since the recommendations are proposed to manage diabetes care, I would suggest inclusion of a validation approach (e.g., correctness) to make sure that the inferred knowledge can be used as a recommendation and safe for users. \n\nSome minor comments. In general, the relations of knowledge graphs are represented in the form of is-a, part-of, or has-a. In the given example (Fig 3) on page 12 “theme” is stated as a relation, and the ‘nutrition’ (on page 12, Table 2) is its an exemplar value.  Because theme is defined as a relation, so nutrition should not be its instance.  Also, is “encouragement” an instance of a concept (node) or is it a concept?  The defined ontological concepts and their values seem confusing. I think these need to be clarified. These are used in mathematical model of the paper. That would be also nice if development/implementation of the framework platform/software (e.g., python) is specified for readers.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "duNAf9WVq-Y", "reviewer": "Reviewer_ScTh", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The scenario studied in this paper is interesting.\n2. This paper is easy to understand.\n\nWeaknesses:\n1. The technical contribution of this paper is quite limited. Using knowledge graphs to enhance recommendation is a widely studied problem. Using user preferences to select entity neighbors is also a common technique [1][2]. The knowledge graph embedding method used in this work is also the conventional TransE method. Using textual information is also a common problem in the recommendation field, such as news recommendation [3]. Thus, the distinct contributions of this work are quite limited.\n2. Some important details are missing. For example, the construction of the knowledge graph is unclear. The experimental settings such as hyperparameters are not complete.\n3. The compared baselines are too weak and limited. There are a large number of methods for knowledge-aware recommendation. The authors should at least include some of them in the comparison.\n4. The text embeddings given by the pretrained USE model may not be suitable for recommendation. Why do not include raw textual information and jointly learn the text model with recommendation signals (such as DKN)?\n\nMinors:\nSome typos need to be corrected. Just to name a few: \"some of the key problems we face is\", \"individual users\", \"Zhang et al. (2018) extends.\n\n[1] Wang et al. \"Ripplenet: Propagating user preferences on the knowledge graph for recommender systems.\" Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2018.\n[2] Guo et al. \"A survey on knowledge graph-based recommender systems.\" IEEE Transactions on Knowledge and Data Engineering (2020).\n[3] Wu et al. \"Personalized News Recommendation: A Survey.\" arXiv preprint arXiv:2106.08934 (2021).", "summary_of_the_paper": "This paper introduces a knowledge-graph enhanced recommendation method for healthcare platforms. The authors use user profile information to select entity neighbors for user modeling. Experiments on a nudge CTR prediction dataset show some improvements brought by the proposed method.", "summary_of_the_review": "This paper suffers from the issues of insufficient contributions, flawed experiments and missing details. Thus, my recommendation is a rejection.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "VzhjYcvoIIM", "reviewer": "Reviewer_zvjx", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In this paper, the authors developed knowledge graph based recommendation approach for recommending healthcare related nudges to mobile users. The key challenge is claimed as the cold start problem in this domain. To this end, they built a knowledge graph where entities are nudges and predefined nudge attribute values, and links are attribute types. Then TransE was employed to learn nudge and attribute embeddings. Built upon the embeddings, the second step method involves a series of transformations for learning user and nudge embeddings, which were used for computing recommendation score through their similarities. Overall, the proposed method is straightforward, which follows commonly used workflow that uses graphs for recommendation. It has several new designs that are specific to the nudge dataset, such as user embedding based on historical nudge clicks, nudge attributes and demographics. However, these designs are sort of incremental and the proposed technique has limited novelty.\n\nThe proposed method was claimed for a healthcare application, but it seems the technique is general to nudge recommendation. The paper has few discussion on the specificity of the application domain of healthcare, and how the proposed method was designed to accommodate those domain specific challenges.\n\nIn terms of the cold start challenge, from the current paper, it is difficult to understand how could the cold start problem be more severe in this investigated domain than other domains. The paper doesn't highlight and elaborate the challenge. Also, the dataset description has no information about empty users/nudges that are related to the cold start problem.\n\nBecause the investigated problem is specific to one domain, the problem description is important and should be clear. However, the current introduction section is obscure, in how nudges are created, what types of data are included in a nudge, whether the set of attribute values are fixed, what is the rule to set attributes and attribute values. The figure and table in the appendix are important (but doesn't answer all the questions), and are better be moved to the main text.\n\nAs for the technical design, there are some assumptions and simplifications that may not be reasonable. For example, the assumption that users are interested in new nudges that are similar to their historical nudges may not be true, as users may not be interested in duplicate messages. The authors also mentioned the importance of diversity, but it appears the proposed method didn't have explicit designs to encourage diversity of recommended nudges. Also, the average embedding in Eq 2 may be simplified as the temporal information of the historical nudges has not been taken into account. Similarly, the zero embedding in Eq 2 for users without history, and the random selection of nudge attributes for these users are sort of arbitrary. The ratio of such users are unclear from the paper, and the impacts these processing steps for them haven't been discussed. Finally, some designs are not well justified. In Eq 5, it is not clear why the transformation step was included.\n\nThe authors also considered to integrate the texts of nudges in the proposed model, but the experimental results didn't demonstrate the effectiveness of the texts. The reason could be from the complexity of the data, or the simplification of methods to incorporate the text embeddings.\n\nThe experiments remain to be improved. First, only a private dataset was used, thus it is hard to judge how general the proposed method is. In the description of the dataset, important information such as the number of nudges, attributes, attribute values, historical nudges per users, are absent. For comparison, there is only one compared method, but it is unclear why that method was selected as the sole baseline. The evaluation metric only has AUC, other commonly used metrics in recommendation could also be used to provide multiple perspectives of evaluation. In the cold start setting, it is unclear how a new nudge's embedding was learned. Since the knowledge graph was trained to learn all embeddings, does the proposed method re-train the knowledge graph once a new nudge is added?\n\nFinally, the presentation of the paper could be improved as there are grammar errors and typos that make reading not easy. ", "summary_of_the_paper": "This paper investigates the nudge recommendation problem in a mobile healthcare domain, and introduces a knowledge graph based recommendation approach. The proposed method builds a nudge-attribute knowledge graph and uses TransE to infer nudge and attribute embeddings, which are used in a downstream step that leverages embedding similarity for recommendation.", "summary_of_the_review": "The paper provides a generally reasonable framework to handle the nudge recommendation problem, but the proposed method is incremental upon existing techniques. It could be seen as a good exploration of applications of those techniques with proper integration. The paper also remains to be improved in its technical details, presentation and experiments.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "LR73SmsghSr", "reviewer": "Reviewer_qBZ3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength: \n   1. The application is interesting. \n   2. This paper is well organized and easy to follow. \nweakness: \n   1. The technical novelty of the proposed method is limited. \n   2. The experiment is not convincing. The baseline is weak and the experiment lacks an online metric. \n\nsignificance & novelty.\nThe proposed application is interesting and useful for diabetes self-care management. But the technical significance is limited. The technique of knowledge graph based recommendations has been investigated in many areas. The technical novelty is limited. \n\nSoundness. \nThe experiment is not convincing. The paper only compares with DCN that is not a KG model. This paper should compare with other KG models. In addition, in published recommendation system benchmarks, DCN cannot represent the STOA. Moreover, DCN has evolved to DCN-V2. As an offline experiment, only comparing on AUC is not such sound. It is a better way to compare online.   \n\n\n\n", "summary_of_the_paper": "This paper proposes a graph-based recommender system for diabetes self-care management. The proposed method bases on the knowledge graph embeddings techniques. The proposed method shows better performance compared with two baselines on metric AUC. ", "summary_of_the_review": "This paper proposes a graph-based recommender system for diabetes self-care management, which is based on the knowledge graph embeddings techniques. However, the technical novelty and significance are limited. The reviewer tends to reject the paper. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces the CareGraph, a knowledge graph based recommendation approach. 
CareGraph is a deep neural network-based recommender that can be used a mobile healthcare platform
for nudge recommendation. The main motivation is to use the knowledge graph to 
mitigate cold start problems when recommending nudge messages. 

The papers' main strength is the topic of interest. Research on recommending systems in the healthcare context is of great interest.
However, the reviews raised concerns that outweigh the strengths. 
The majority reviewers agree that the work is not ready for publication.
Main concerns focus on weak experimental section and lack of technical details.

I recommend the authors to incorporate all the reviewers' comments and make a 
stronger submission to a future conference!
Decision: Reject