Patent AU-2021203258-A1 (AU)
Title: Method and system for updating analytics models that are used to dynamically and adaptively provide personalized user experiences in a software system
Inventor: MASCARO MASSIMO
IPC: G06Q40/02
Filed: 2021-05-20
Abstract: A computer system implemented method for updating analytics models to improve 
delivery of personalized user experiences in a software system, comprising: 
providing, with one or more computing systems, a software system; 
receiving user characteristics data representing user characteristics for each of a plurality 
of users of the software system; 
storing the user characteristics data to memory that is allocated for use by the software 
system; 
training a production analytics model by at least forming a production decision tree 
having rules and parameters governing traversing from a parent node to a child, 
the production decision tree including one or more of a production reach 
probability, a production stop probability, a production user experience option 
and a production user segment; 
training an offline analytics model by at least forming an offline decision tree having 
offline rules and offline parameters governing traversing from a parent node to a 
child, the offline decision tree including one or more of an offline reach 
probability, an offline stop probability, an offline user experience option and an 
offline user segment; 
analyzing the user characteristics data from the memory by applying the user 
characteristics data to the production analytics model, to segment the plurality of 
users into a first plurality of segments of users; 
delivering a plurality of pairs of a plurality of user experience options to the first 
plurality of segments of users, to cause each of the first plurality of segments of 
users to receive at least one of the plurality of pairs of the plurality of user 
experience options, each of the plurality of pairs of the plurality of user 
experience options having a user-preferred user experience option and a test user 
experience option that are provided to the first plurality of segments of users to 
concurrently personalize user experiences with the plurality of user experiences 
and test user preferences for the plurality of user experience options within the 
first plurality of segments of users, the plurality of user experience options being 
represented by user experience options data; 
monitoring user responses data to determine performance metric data for the production 
analytics model, the user responses data representing user responses to receiving
the plurality of pairs of the plurality of user experiences, the performance metric 
data for the production analytics model representing performance metrics of the 
production analytics model; 
applying the user characteristics data to the offline analytics model to segment the 
plurality of users into a second plurality of segments of users at least partially 
based on the user characteristics data; 
determining performance metric data for the offline analytics model at least partially 
based on the user responses to receiving the plurality of pairs of the plurality of 
user experiences, the performance metric data representing performance metrics 
of the offline analytics model; 
comparing the performance metric data for the production analytics model to the 
performance metric data for the offline analytics model; and 
replacing the production analytics model with the offline analytics model, to cause the 
offline analytics model to become the production analytics model, if some of the 
performance metrics of the offline analytics model are greater than at least some 
of the performance metrics data for the production analytics model; 
wherein replacing the production analytics model with the offline analytics model 
includes deploying the offline analytics model into a centralized computing 
system that provides analytics services to one or more additional software 
systems.
4/13 
400 
ACQUIRE USER APPLY USER CHARACTERISTICS 
CHARACTERISTICS DATA AND USER RESPONSES, TO 
DATA 402 THE DELIVERED USER 
EXPERIENCE OPTIONS, TO AN 
OFFLINE ANALYTICS MODEL 412 
APPLY USER CHARACTERISTICS 
DATA TO A PRODUCTION DETERMINE THE STATISTICAL 
ANALYTICS MODEL TO SELECT POWER OF THE OFFLINE 
USER EXPERIENCE OPTIONS ANALYTICS MODEL 414 
FOR USERS 404 
416 
DELIVER USER 
EXPERIENCE OPTIONS STATISTICAL NO 
TO USERS BASED ON POWER > 
THE SELECTED USER THRESHOLD? 
EXPERIENCE OPTIONS 
406 YES 
DETERMINE THE PERFORMANCE 
DETERMINE USER OF THE OFFLINE ANALYTICS 
RESPONSES TO THE MODEL 418 
DELIVERED USER 
EXPERIENCEOPTIONS COMPARE THE PERFORMANCE OF THE 408 PRODUCTION ANALYTICS MODEL TO 
THE PERFORMANCE OF THE OFFLINE 
DETERMINE THE ANALYTICS MODEL 420 
PERFORMANCE OF THE 
PRODUCTION 422 
ANALYTICS MODEL 410 OFFLINE 
ANALYTICS 
MODEL > NO 
PRODUCTION 
ANALYTICS 
MODEL? 
YES 
DEPLOY OFFLINE ANALYTICS 
MODEL TO REPLACE_ 
PRODUCTION ANALYTICS MODEL 
FIG. 4 424