C ARE G RAPH : A G RAPH - BASED R ECOMMENDER S YSTEM FOR D IABETES S ELF -C ARE M ANAGEMENT
Anonymous authors
Paper under double-blind review
A BSTRACT
In this work, we build a knowledge graph that captures key attributes of content and notiﬁcations in a digital health platform for diabetes management.
We propose a Deep Neural Network-based recommender that uses the knowledge graph embeddings to recommend health nudges for maximizing engagement by combating the cold-start and sparsity problems.
We use a leave-one-out approach to evaluate the model.
We compare the proposed model performance with a text similarity and Deep-and-Cross Network-based approach as the baseline.
The overall improvement in Click-ThroughRate prediction AUC for the Knowledge-Graph-based model was 11%.
We also observe that our model improved the average AUC by 5 % in cold-start situations.
I NTRODUCTION
The recent global pandemic has brought with it a permanent shift away from traditional in-person health consultations, towards large digital telehealth platforms that support remote consults.
This research is performed in the context of one such platform that provides a mobile application to help people manage chronic diseases such as diabetes, hypertension and obesity.
As solution designers, some of the key problems we face is to get our users’ attention, foster awareness and encourage them to take actions that help manage their chronic health conditions.
Sending notiﬁcations that nudge users is one efﬁcient way to encourage engagement.
However, user preferences for engaging with these nudges can vary greatly, different users require different persuasion techniques.
Even within a homogeneous messaging context, different tones in messages can appeal to different users.
Recommendation systems can be applied to model preference patterns and predict effective personalized nudge notiﬁcations.
Collaborative recommender systems are well established tools for predicting user preference and have been shown to perform very well, provided there is sufﬁcient information available to model the users preferences, as highlighted by seminal papers in the ﬁeld such as Resnick et al.
For new or esoteric items, or users, there is frequently a lack of sufﬁcient information to make a good prediction.
These conditions, known in the literature as ’cold start’ are the focus of our experiments in this paper.
Speciﬁcally in the context of recommending health nudges, which are created by experts who inherently follow guidelines set by associations such as American Diabetes Association ame (2021).
Our primary research question asks whether a knowledge graph can be applied to mitigate cold start problems, in the speciﬁc task of recommending a ﬁnite set of highly structured health nudge messages.
1. K NOWLEDGE G RAPH - BASED R ECOMMENDATION
Online content and services have undergone a huge volume of growth in recent years.
Accordingly, recommender systems are increasingly relied upon to help people get to the right information at the right time and also in the right way [Ricci et al.
They help customers shorten their times exploring products or services in various applications such as news portals [Wang et al.
(2018b)],
E-commerce [Zhang & Jiao (2007), Hwangbo et al.
(2018)], accommodations [Haldar et al.
(2019)], or music recommendation [Van Den Oord et al.
However, our problems are different from these recommendation applications.
First, in other applications like E-commerce, users can see a list of products and select their interested items.
They can visit websites to search for items of interests anytime they like.
In our nudge notiﬁcations, our users can just see one nudge at a time, and hence we cannot ﬂush a lot of nudges as it will disturb our users and can negatively impact their level of engagement.
Hence, recommending the nudges that match user preferences while also ensuring that there is a diversity in the actions we are recommending, and achieving good performance in as few notiﬁcations as possible is a crucial requirement in this setting.
Also, we frequently need to support the creation of new nudges.
Within this setting, using embeddings from supervised learning methodologies may not be able to solve our problems.
To learn efﬁcient embeddings from those approaches, we require a lot of data for each item.
Organizing and aggregating nudge attributes such as themes, tones or objectives can help the cold-start problem as we can transfer the knowledge of user preferences on nudge attributes to new nudges and help us understand and optimize to user behaviors quickly.
For example, if a user has positive responses to encouragement-tone nudges, we can send more encouragement-tone based messages to this user.
Knowledge graphs (KG) have been applied in various tasks such as search engines [Yang et al.
text classiﬁcation [Hu et al.
and word embedding [Celikyilmaz et al.
They have also been previously introduced to recommendation systems to improve the precision of recommended items and the explainability [Wang et al.
A knowledge graph is a directed heterogeneous graph representing real-world objects and relationship among these objects.
Nodes in a KG represent entities, which can be items or the attributes that describe the item.
The edges represent relations between each entity.
Knowledge graphs give recommendation systems the beneﬁt of enriching the semantic relationships among items [Wang et al.
When we know users’ interests in a KG graph, we can extend the diversity of recommended items from the node connections present in the graph.
Our problem domain is selecting nudge notiﬁcations for our mobile applications that are personalized to individual users preferences.
We have different types of nudge notiﬁcations such as alerting users in regards to their health condition, reminding users to follow their health routine, providing some useful education related to their health, or introducing new services and events to users.
We want to personalize nudges to each user to increase their engagement and reduce over-sending nudges that might disturb the users.
The contributions of this work are summarized as follows:
• We propose a new method using a neural network that combines user attributes with a knowledge graph to overcome the cold-start problem.
• We conduct experiments demonstrating the effectiveness of our new approach, particularly under cold-start conditions.
1. D IGITAL H EALTH S YSTEM FOR D IABETES S ELF -C ARE
This work is based on a system that leverages a Blood-Glucose (BG) monitoring device, and a Mobile
App that is used to provide useful information and suggest actions to users that are relevant and helpful for managing Diabetes.
As a part of this system, users are provided with short text messages, aboutcharacters long.
The text is designed to nudge a user to speciﬁc actions that are relevant to managing the condition, such as, healthy recipes ideas, articles that explain and motivate users to add exercise to their routine, reminders to check and monitor BG levels, talking to expert coaches etc.
The short text is accompanied with two buttons, one of which is used to guide the target user to perform a speciﬁc action (e.g. read content, schedule a call with a coach), and the other can be used to dismiss the recommendation.
In this work we call these recommendations on the Mobile-App, Mobile Nudges (or Nudges for short).
R ELATED W ORK 2. R ECOMMENDER SYSTEMS IN D IGITAL H EALTH
Over the past decade, a rapidly increasing volume of digital information to support clinical decision making has become available to be leveraged by healthcare recommender systems.
Automated health recommender systems have been deployed in various domains.
Some applications aim for improving lifestyle to be healthier through diet [Elsweiler et al.
(2017), Achananuparp & Weber (2016)] or physical activity recommendations [Dharia et al.
One such system, known as Pro-ﬁt [Dharia et al.
(2016)] applies a hybrid approach to personalize workout sessions based on a user’s contextual data and calendar events.
Achananuparp & Weber (2016) proposed a healthier food substitute suggestion system by introducing a food-context matrix and applying Singular Value Decomposition to get the low-dimensional representation of each food item.
The similarity between two food items is measured by cosine similarity.
Narducci et al.
(2015) introduced an HealthNet to personalize doctors and hospitals to users, given the user proﬁle and the health data shared by the community.
The iDoctor system in Zhang et al.
(2017b) provides doctor recommendations by using sentiment analysis from their rating and reviews and Latent Dirichlet Allocation for user preference and doctor features.
The hybrid matrix factorization is applied to predict the doctor rating.
Recommendation systems have also been used for optimizing treatment plan such as drug recommendation [Zhang et al.
(2017a), Stark et al.
Zhang et al.
(2017a) applied the concept of neighborhood-based method in a drug-drug interaction prediction to help reduce unexpected effects from using multiple drugs.
A drug recommendation system for Migraine diseases proposed by Stark et al.
(2017) using a graph database and collaborative ﬁltering approach.
2. K NOWLEDGE G RAPH E MBEDDING
Knowledge graph embedding (KGE) is a process to transform a knowledge graph into low-dimensional continuous vector space which still preserves the network structure information.
The knowledge graph can be represented as a group of triplets, each of which contains two nodes (items or attributes) and the relationship between these nodes.
With these triples KGE will project all entities and relations into a low dimensional vector space that preserves their graph structure in these vector representations.
There are several approaches for building these representations in the KGE setting.
Translation distance models such as TransE [Bordes et al.
(2013)], TransH [Wang et al.
(2014)], TransR
[Lin et al.
and semantic matching models such as DistMult [Yang et al.
(2014)] are some of the popular methods.
2. K NOWLEDGE GRAPH EMBEDDING IN R ECOMMENDATION S YSTEMS
Knowledge graphs have been applied to many recommendation applications.
Wang et al.
(2018a) proposed the RippleNet, an end-to-end recommendation framework that incorporates knowledge graph into recommendation systems.
RippleNet extends user interest of items through links in the knowledge graph which help increasing the diversity.
Wang et al.
(2018b) introduced Deep Knowledge-aware Network (DKN) for news recommendation.
The key component of DKN is knowledge-aware convolutional neural network (KCNN) for incorporating word-level and knowledge-level representations.
Zhang et al.
(2018) extends a collaborative ﬁltering framework to learn over the knowledge graph embedding.
A user-item graph is used, where each connection between nodes depicts how a user interacted with an item, for example ( buy, also bought, also view )
Another method for using a knowledge graph in a recommenation system is path-based approach, which uses the connection patterns in the knowledge graph to generate the recommendation.
Hete-CF [Luo et al.
(2014)], a collaborative ﬁltering recommendation method on Heterogeneous Social Network, which combines different types of meta-paths (user-user, user-item and item-item) and calculates the similarity.
Yu et al.
(2014) introduced HeteRec which represents the connectivity between users and items with meta path-based latent features.
HeteRec recommendation models are further deﬁned into two levels: global (HeteRec-g) and personalized (HeteRec-p) levels.
In the HeteRec-p, users are clustered based on their interests and preferences into subgroups and then learn recommendation models for each user subgroup.
P ROBLEM F ORMULATION
Our nudge recommendation can be seen as a binary classiﬁcation problem where we try to predict the probability that a user will have a positive engagement with a nudge (i.e. accepted the action or suggestion provided by the nudge).
Let U = { u 1 , u 2 , ... } and V = { v 1 , v 2 , ... } denote the sets of users and nudges respectively.
A user-item interactive matrix is deﬁned as Y =
{ y u,v | u ∈ U, v ∈ V } where y u,v = (cid:26) 1 if the user u positively engaged with the nudge v 0 otherwise (1)
We want our model to learn and predict a probability of u i will click on the nudge v j or ˆ y u i ,v j = f ( u i , v j ) .
M ETHODOLOGY
Our approach to nudge recommendation is based on a combination of item and user-based collaborative ﬁltering.
In our model, a user is represented as a triple u ∈ d, i, a where d is the a set of demographic proﬁle information such as age and gender; i is the set of clicks on nudges, and a is the set of attributes of the nudges that were effective in engaging that user.
The objective is to transform users and items (in our case, nudges) into a vector space and make a prediction based on similarity in the latent space.
To learn an embedding of a user proﬁle, we rely on their click history.
This information helps us to passively infer preferences based on how they interacted with nudges in the past.
If a candidate item is similar to a user’s history of clicked nudges, it is likely that a user will also like this nudge too.
However, using only a proﬁle of click history has signiﬁcant limitations and can result in a narrow set of recommended items, lacking in diversity –essentially a similar narrowness problem to that of a traditional content-based recommender system.
To mitigate this problem, which is particularly apparent in cold-start conditions where there are less click data available, we proposed a novel use of “nudge attribute” preferences to user embeddings.
Attributes of recommended nudges effectively extend a user’s preference proﬁle by creating new edges to explore in the item-to-item graph.
Even for a completely new user with no click history, we can infer nudge attribute preferences from the demographic proﬁle until preference data becomes available through click interactions.
The model training occurs in two parts.
We ﬁrst train knowledge graph embeddings to generate nudge, nudge attributes and relation type embeddings.
Next, we construct a deep neural network model to predict the click-through rate (CTR) probability of a user u i on a nudge v j .
The user embeddings e u i (section 4.2.1) are constructed by combining the user click history embeddings
h u i , nudge attribute preference embeddings n u i and user proﬁle embeddings p u i .
To learn nudge attribute preference embeddings, we apply the concept of attention mechanism Vaswani et al.
(2017) to get a set of values for our attention-based network layer.
A nudge prediction is then calculated from a function of e u i and e v j where e v j is an embedding of nudge v j .
4. K NOWLEDGE G RAPH FOR D IABETES S ELF -C ARE
We evaluate our proposed models for CTR prediction on our mobile nudge dataset, which we described in Section 1.2.
The actions suggested to the users in the nudges cover themes such as monitoring their blood glucose level, improving physical activity, adhering to medication etc.
These actions may have different levels of effort to complete.
For example, requesting a healthy recipe might be done directly on the mobile application, while talk to a coach or an expert might require the user to ﬁnd an available time and schedule.
We also have nudges that might recommend the same action but use a different tone in the communication text.
Furthermore, the nudges can also be categorized based on the intent of support the nudges are be providing, such as providing knowledge or requesting preferences through short questions.
We use such categorizations that are used to create new nudges for the users as the attributes for the nudges.
These attributes capture not only the theme and intent of the nudge but also some information around the experience accessing the working on the action recommended by nudge.
Table A shows seven attribute types with descriptions and examples values relating to a nudge “In a dinner rut?
Our meal plan is packed with tasty, colorful, and super-healthy recipes, Want it?” .
These attributes and the associated values have been developed by working with the business and subject matter experts.
Figure 3 shows two example health nudges with their associated, and sometimes overlapping attributes.
The full set of nudges and attributes in our library are used to deﬁne a graph that becomes the initial input to the CareGraph recommender system.
The nudges and the attribute values form the nodes of the graph, as highlighted in Figure 3.
For example, the blood-glucose monotoring nudge and the holiday nutrition nudge both share the attribute that they send a user to a website for additional information, and that they foster broad awareness of the chronic health condition.
More formally, the attribute types are the relations that deﬁne the edges of the graph, i.e. the edge describes how a nudge is related to the attribute.
This can be represented in a form of triples ( h, r, t ) where h is the nudge (head), r is the attribute type (relation) and t is the attribute value (tail).
For example, if a nudge CGM CONTENT BGM COMPARE has monitoring as the theme , then this relationship is represented as CGM CONTENT BGM COMPARE theme −−−→ monitoring , and forms the triplet ( CGM CONTENT BGM COMPARE, theme, monitoring ) .
While this is a useful representation for the structure with which nudges are created and described, it will be beneﬁcial to develop continuous representations that can be used for training a recommender model.
These representations are called knowledge graph embeddings, and are described in Section 2.2.
There are various algorithms that can be used develop these embeddings.
Given the relatively simple structure of the graph and the simplicity of the algorithm we use the standard TransE algorithm
[Bordes et al.
(2013)] in our methodology.
4. C ARE G RAPH M ODEL
The CareGraph model, shown as a ﬂow diagram in Figure 1 is trained in two phases.
We ﬁrst train knowledge graph embeddings for nudges, nudge attributes and attribute types embeddings.
Next we construct a deep neural network model to predict the CTR probability for a target user u i on a nudge v j .
The user embeddings e u i (section 4.2.1) are constructed by combining user click history embeddings
h u i , nudge attribute embeddings n u i and user proﬁle embeddings p u i .
The prediction is calculated from a function of e u i and e v j where e v j is an embedding of nudge v j .
4.2. U SER E MBEDDING
User click history embedding A user’s click history helps us understand which nudges the user liked in the past.
We gather all clicked nudges that the user u i clicked in the past and construct a click history list nudge hist u i where nudge hist u i
= { v k | v k ∈ V, y u
i ,v k = 1 } .
For users who have never clicked on any nudge, their click history will be an empty list.
To represent nudge hist u i as a embedding vector h u i , we use embedding vectors for all v k ∈ nudge hist u from our knowledge graph embeddings and use the equation 2 to aggregate them.
[Equation]
where e v k is an embedding vector of the nudge v k from the knowledge graph embedding.
In the equation 2, we ﬁrst average all embeddings in nudge hist u
i and apply a linear function where W m and b m are transformation weight and bias respectively.
For a user whose click history is empty, the h m will be a zero vector.
Nudge attribute embedding In user click history embeddings, we focus only on nudge embeddings that the user interacted with in the past.
We can leverage nudge attribute nodes to extend user preferences by propagating to neighbor entities around users’ historical clicked nudges.
Since neighbors of nudge entity are nudge attributes entities, we refer to this as the set of attribute embeddings n u i .
From a intuitive motivation that different users have different preference on nudge attributes, for example, a user may be interested in a formal clinical information nudge than a cheerful-tone encouragement nudge, we should use user preference weights on nudge attributes instead of averaging them directly as in equation 2.
Given a set of nudge attributes and their relation types, we apply the concept of attention mechanism Vaswani et al.
(2017) to learn user preference weights of each nudge attribute.
Considering an nudge entity h in the knowledge graph where its connections are represented in the form of ( h, r, t ) , The set of neighbor entities, or nudge attributes, of a user u i is N u
i
=
{ t | h ∈ nudge hist u i , ( h, r, t ) ∈ kg } and the relation types R u i
= { r | h ∈ nudge hist u
i , ( h, r, t ) ∈ kg } .
For a user whose click history list is empty, we randomly select a set of nudge attribute entities.
The embedding aggregation of nudge attributes can be formulated as
[Equation]
Where α = ( α 1 , α 2 , ...
α i ) is an attention weight vector.
We calculate the vector α from user proﬁle features g u i of the user u by the following formula:
[Equation]
The W r and b r a transformation matrix and bias respectively.
It will transform the user proﬁle feature g u into the r vector space and then perform the inner product with the embedding vector e r i of the attribute type r i .
A softmax function is used to compute user relation type preference weights, and then the ﬁnal nudge attribute embedding n u is calculated from this equation:
[Equation]
User proﬁle embedding We also include explicit user features into the user embedding.
All user raw data is transformed into a user feature vector and then apply a function to get the ﬁnal representation.
[Equation]
Information Aggregation
The last step to get the ﬁnal embedding of the user u i is by combining user history embedding h u i from equation 2, nudge attribute embedding n u i from equation 5, and user proﬁle embedding p u
i from equation 6.
We concatenate all three embeddings and transform it by a linear function to get the ﬁnal user embedding e u i .
[Equation]
4. M ODEL P REDICTION
To calculate the probability that the user u i will click on the nudge v j .
We use the ﬁnal user embedding e u i from equation 8 and nudge embedding e v j of the nudge v j from knowledge graph embedding.
We predict a user engagement score from the matching score by
[Equation]
[Equation]
where σ is a nonlinear function, similar to the sigmoid function.
We can see from the equation 9, the
CTR prediction of user u i and nudge v j , ˆ y ( u i , v j ) , is dependant on how similar the user embedding e u i is to the nudge embedding e v j .
C ARE G RAPH WITH T EXT EMBEDDING
The CareGraph model supports addition of new features such as text embeddings.
A basic nudge is a short text message, and we can aggregate these embeddings into the model to improve its performance.
Here the premise is that text similarity across the broader library of nudges can be leveraged to improve the relevance of nudges for a target user.
In this section, we introduce two such extensions of the base model architecture: CareGraph + TextSim and CareGraph + TextEmbedding .
CareGraph+TextSim
The approach taken in this model stems from our intuitive assumption that if a text message of a nudge v j is similar to contents in historical clicked nudges of user u i , it will be likely that a user u i will also like the nudge v j .
With this assumption, we add text similarity score at the prediction step.
The other parts of model are still the same.
The text similarity score t v j is be calculated by equation
[Equation]
where x v is a text embedding of the nudge v and dist is a distance function of text embedding, such as cosine similarity.
The CTR prediction score in equation 9 will be replaced with
[Equation]
CareGraph+TextEmb
Another way to include text embedding into the CareGraph model is to include them as part of the user embedding e u in equation 7 and 8 and nudge embedding in e v j .
We ﬁrst get text embeddings of all nudges in nudge hist u i and aggregate them by simple averaging.
[Equation]
To include text embedding t u i into user embedding e u , we update the equations 7 as follow:
[Equation]
14) and then calculate the ﬁnal e u in 8.
In the equation 9, we add text embedding of the nudge v j into e v j by
[Equation]
E XPERIMENT S ETUP
The data used for this analysis consists of interactions from 283,743 unique users.
As mentioned in the description of the Nudges system, the nudge consists on two buttons, one for taking the suggested action and the other for dismissing the Nudge.
For the purpose of this analysis a user accepting the recommended nudge is considered a positive rating or a click .
The data has been collected from 283,743 users over a period of 12 months, and have an average Click-Through-Rate (CTR) of 16.77%.
6. P ROCEDURE
In order to determine the performance of the proposed model two distinct cases are considered.
In the ﬁrst case we evaluate the improvement in predicting the click behavior across all users and all available Nudges in the data.
In the second case, the performance of the model is evaluated in Cold-Start cases.
Speciﬁcally, the case when new Nudges are added is simulated by leaving one nudge out in the training data and determining the performance in predicting the click behavior for that Nudge.
This is a typical situation that is encountered in practice where new content and nudges are created and it is important to direct them to most receptive users in a timely fashion.
The performance of the proposed model in both cases is compared to a Deep and Cross baseline model, as described below.
6.1. B ASELINE M ODEL FOR P ERFORMANCE E VALUATION
A Deep and Cross Network (DNC), similar to Wang et al.
(2021) is used as the baseline model to predict the click on the Nudge.
The DCN model starts with an embedding and a stacking layer to transform categorical features into a dense vector.
A Cross Network and a Deep Network work in parallel and their output is combined in the last layer.
The Cross Network is composed of cross layers which applied feature crossing at each layer.
A cross layer is to calculate this following function:
[Equation]
where x l , x l +1 ∈ R d is the output vector at the layer l -th and l + 1 -th layer, respectively.
The x 0 is the embedding vector from the embedding and stacking layer.
By adding more cross layers into the cross network, it increases the polynomial degree of the input x 0 .
The deep network is a standard fully-connected feed-forward neural network.
For each layer, the output of the subsequent layer l + 1 is calculated by:
[Equation]
where W l is a weight matrix and b l is the bias of the layer l .
The ﬁnal output of the deep network is an embedding vector h L where L denotes the total number of deep layers.
The prediction step is performed by concatenating the outputs from the cross network and the deep network and feeding them [Table] to the logit layer.
The sigmoid function is used to calculate the ﬁnal prediction on how likely the user is to engage with the recommended nudge.
6. T RAINING THE C ARE G RAPH M ODEL
We trained our knowledge graph embeddings for all nudges, nudge attributes and attribute types (relation types).
The embedding sizes e in the knowledge graph embedding are set to 100.
We use the pretrained Universal Sentence Encoder from Tensorﬂow Hub (https://www.tensorﬂow.org/hub) to extract the text embeddings.
We use Adam to train both of our recommendation models and the deep-and-cross model by optimizing log loss.
We apply hyper-parameters search for knowledge graph embedding size, hidden layer units and L2 regularization.
The hidden layer in equation 2, 5 and 8 are set to 24 dimensions.
For the deep and cross model, the ﬁnal model architecture is two hidden layers with 240 and 48 dimensions in a deep network and 2 cross layers in the cross network.
To evaluate recommendation results with the deep-and-cross model, we use AUC as the evaluation metric.
6. R ESULTS
We evaluate our proposed model with the baseline model in a general case and cold-start situations.
General case This objective of this experiment is to compare recommendation results of our proposed models with the baseline model in a overall case.
We split the training, validation and testing data into 60:10:30 respectively.
We compared AUC from our three proposed models: CareGraph (section 4.2), CareGraph+TextSim (section 5) and CareGraph+TextEmb (section 5) with the deep-and-cross model.
The results are shown in Table 1.
From the table 1, all CareGraph approaches outperform a deep-and-cross model.
We have tried two different model architectures that integrate text information into our CareGraph model.
Our initial hypothesis was that the text embeddings would improve recommendation accuracy because similarity could be computed on more underlying data.
However, results showed that this was not the case.
AUC did not improve in the text-enabled treatments.
A possible reason for this unexpected result is that the kg-embeddings may have already captured enough information to generate high quality predictions.
Further studies of model architectures CareGraph with text may be required to prove that this generalizes to all integrations of text similarity to the model.
Cold start experiments We simulated cold-start cases by using a leave-one-out approach.
Among 155 nudges, one nudge is selected as testing data and keep the the rest nudges for training.
We repeated this process for all 155 nudges and compare all AUC with the baseline.
In this experiment, we compare our CareGraph model with the baseline model.
For the baseline approach, since the deep-and-cross model cannot learn the nudge embedding for new nudge in the training process, we use the prediction results of the most text-based similarity nudge in the training data for a new nudge.
In average, our approach has 61.76% AUC and the baseline approach have 58.17% AUC.
Our model outperform the baseline model for 96 nudges out of 155 nudges.
The distribution of AUC over all nudges shows in Figure 2.
Cold start analysis The main objective of using a knowledge graph embedding is to overcome the cold-start and data sparsity problems.
A key question in this research is whether knowledge graph embeddings of nudges, nudge attributes, and attribute types capture enough information for new nudge prediction.
If we have a new nudge that is different from existing nudges, how that will impact to prediction performance?
In the cold-start experiment, we compare the prediction results with a text-based similarity approach.
Text-similarity based approach is heavily depends on existing nudges in the training dataset.
If a text message of new nudge has different meaning from the existing nudges, the prediction result can be [Figure] less precise.
In our mobile application, it is very likely that we will develop new content.
For the knowledge graph embeddings based approach, the embeddings of each nudge preserve connection structures with other entities.
Even when we have a new nudge from a new service or program, as long as there are shared attributes with existing nudges, the kg-embedding of new nudge will be not be too far away from existing nudges in the vector space.
Also, by adding our nudge attribute embedding n u i in Equation 5 into the user embedding e u i , this embedding will show how this new nudge is similar to user attribute preferences.
In Figure 2, we compare the AUC distributions of CareGraph and the text-based similarity deep-andcross approaches over the text distance.
The x -axis shows the nearest text cosine-similarity of each new nudge to existing nudges.
A lower distance means a new nudge content is different from its nearest existing nudge.
At the text distances ranging between 0.4 and 0.6, there is a group of nudges that the baseline approach has lower AUC than our proposed approach.
Also, if we consider the AUC results of nudges whose nearest text cosine similarity is less than 0.5, our approach improves the AUC by 6.17% and 5.24% for greater than or equal to 0.5.
This result shows that we improve AUC more if a new nudge has more different content than existing nudges.
F UTURE W ORK AND C ONCLUSION
This research produced several avenues for future work, including analysing different methods to integrate text-based similarity into the model to improve both regular performance and performance under cold start conditions.
To conclude, the primary research question in this work asked whether a knowledge graph can be applied to mitigate cold start problems, in the speciﬁc task of recommending a ﬁnite set of highly structured health nudge messages.
To facilitate this we developed a novel knowledge graph based recommender system, which we called CareGraph , for predicting health nudges to manage chronic disease on a large digital health platform.
Our results conﬁrmed that the knowledge graph does improve predictive accuracy in cold-start situations, showing a 5% improvement over a benchmark model.
In addition, a text-similarity based model was evaluated and it was shown that structure in the knowledge graph produced more accurate recommendations than those based on text similarity