Patent ID: 12248962

DETAILED DESCRIPTION

In the following detailed description, only certain exemplary embodiments of the present invention are shown and described, by way of illustration. As those skilled in the art would recognize, the invention may be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein.

Providers of subscription services to organizational customers (e.g., businesses), such as cloud computing platforms, financial services platforms, and the like may offer a range of different products corresponding to different services offered to its customers. These services may be applicable to different use cases and solve different problems. For example, some businesses may subscribe to cloud computing services to host a website and to store data regarding its customers-depending on the type of data being stored, the access requirements, and usage scenarios. The cloud computing service may offer different mechanisms for storing data, including a relational database service, a key-value store service, a block storage device service (e.g., where the user may choose how to write files to the storage device), a blob store or object store service in association with an identifier (e.g., where a user may store a single file or document), a logging service, and the like. Similarly, a financial service platform may offer services related to invoicing, subscription billing, fraud detection, sales tax compliance, payments management, credit issuing, and the like.

Prospective customers and current customers of these service providers may initially be overwhelmed or confused by the large number of services available, which may slow adoption of services that would provide the most value to those users. The particular services that would be useful to any given user is highly dependent on the customer and their particular needs. In some circumstances, human sales agents conduct sales interviews with customers and prospective customers to understand their needs and to make recommendations based on those understandings.

However, personalized, human sales agents are expensive. In addition, as the number of different services offered by the service provider increases, it becomes increasingly challenging for a salesperson to understand the wide range of options and to understand which products may be useful for a given customer or prospective customer. Accordingly, aspects of embodiments of the present disclosure relate to automatically predicting service product adoption by customers and prospective customers. Some embodiments of the present disclosure relate to automatically computing propensities of prospective customers to adopt different products offered by the service provider. Some embodiments of the present disclosure further relate to automatically computing propensities of current customers to adopt products that they are not currently subscribed to. In addition, some aspects of embodiments of the present disclosure relate to automatically calculating the conditional revenue to be earned if a customer were to adopt a given product, which may be referred to as a conditional product revenue.

These product adoption propensities and conditional product revenue computations may be used to guide the targeted marketing of products to prospective customers and current customers. In some embodiments, the product propensity predictions are used to automatically provide guidance (e.g., displayed on a user interface of a computing system) to sales agents to suggest talking points regarding particular products to be promoted during discussions with a prospective customer. In some embodiments, the product propensity predictions are used to guide the display of target advertisements to prospective customers and current customers (e.g., in the form of targeted e-mails, banner advertisements, and product recommendations in a user interface of a computing system). In some embodiments, the conditional product revenue calculations are used to determine the channels by which the targeted promotions are delivered (e.g., reserving the use of expensive sales agents to combinations of customers and products with high conditional product revenue).

FIG.1Ais a block diagram of a service product adoption prediction system100according to one embodiment of the present disclosure. As shown inFIG.1A, the service product adoption prediction system100includes a plurality of trained models110. The trained models110are configured to take input data120associated with a customer and to output predictions regarding the propensity of the customer to adopt (e.g., purchase or subscribe to) particular products and to output predictions130regarding the conditional product revenue for those products. Types of input data120include, but are not limited to, descriptions of the customer (e.g., in text), customer behavior data (e.g., interactions with a platform), and customer financial data (e.g., volume of transactions, size of transactions, geographic distribution of transactions, etc.).

As noted above, in some embodiments, the predictions generated by the trained models110are provided to product marketing and promotion systems140to generate promotions or targeted advertisements. Examples of product marketing and promotion systems140include, but are not limited to, automatic targeted advertisement generation (e.g., paper and/or electronic targeted mailings, web-based advertisements, and native promotions within a software application such as a native application or a web application) and business intelligence reports (e.g., for guiding sales agents during sales calls), as will be discussed in more detail below.

In some embodiments, the trained models110may be trained based on customer data in a customer data store150collected from current customers of the service provider and their usage activity of the platform. In some embodiments, a model trainer160is configured to retrieve the usage data in the customer data store150and to train or update the trained models110based on the usage data in the customer data store150. In some embodiments, the model trainer160periodically retrains the trained models110based on updates to the usage data in the customer data store150and/or due to changes in the product offerings (e.g., addition or removal of product offerings), as will be discussed in more detail below.

FIG.1Bis a block diagram depicting the use of available information at different stages of a customer or prospective customer relationship with an organization to predict product propensity and conditional revenue and to generate marketing and product promotion messages to customers according to one embodiment of the present disclosure. In the block diagram101ofFIG.1B, customers are shown as progressing between different stages in their relationship with the service provider, including being a prospect or prospective customer104, progressing to be a customer that is onboarding105(e.g., a newly signed-up customer), to a customer that is an active and live customer106of the system provided by the service provider.

During these customer stages103, the types of available information121may differ. For example, at the prospective customer104stage, the service product adoption prediction system may only have access to firmographic data122. This firmographic data122generally includes publicly available information about a given customer (company), such as industry classification (Standard Industry Classification), company size, location, that may be available directly from the company or through government records. In some cases, third parties may collect and publish information regarding companies (e.g., summaries of business including target market and business offerings) where this information may be included in the firmographic data122. In some circumstances, customers may publish their own websites, where those websites include text descriptions of their business. In the case of customers that are publicly traded companies, the firmographic data may include published information regarding the company (e.g., as available in the United States through the Securities and Exchange Commission).

At the onboarding105stage, it is assumed that the customer has completed a sign-up process that requested the customer to provide information describing their business. The responses from the customer may include, but is not limited to, answers to multiple choice questions (e.g., ranges of revenue or transaction volumes), numerical values (e.g., number of employees, number of customers, revenue in the past quarter, etc.), and free-form text descriptions of the business of the customer and/or their reasons for signing up with the service provider, such as a list of current pain points in their business. Accordingly, the available information at the onboarding stage may include both the firmographic data (discussed above) along with the account application data123as provided by the customer.

At the live customer106stage in which the customer is an active user of the software platform, the firmographic data and account application data as discussed above is still available in addition to information collected through customer interactions with the service provider124. These interactions may include customer interactions with the user interface (e.g., web-based application, native desktop application, mobile app, and the like) and customer usage of the services (e.g., growth in usage of computing resources in the case of a cloud computing service or growth in number of transactions processed in a financial service provider).

As noted above, the trained models110B may be used to compute predictions of, for example, a first model111predicting the propensity of a prospective customer104or newly onboarded customer105to adopt various products, a second model112predicting the propensity for a current customer or live customer106to be upsold on a product that the customer is not currently using, and a third model113to compute the conditional product revenue associated with a customer. These models110B may then be used to generate predictions131based on the available information121for a given customer (e.g., prospective customer104, newly onboarded customer105, or live customer106), where these predictions may include product propensity predictions132, product revenue predictions133, and product and bundle recommendations134.

FIG.2Ais a block diagram depicting components of a system210for predicting the propensity of a customer or prospective customer to adopt services offered by a platform according to one embodiment of the present disclosure. The system210may be implemented using one or more computing devices, examples of which are described in more detail below with respect toFIGS.6-8. For example, the input data, intermediate data, and output data (e.g., predictions) computed by the system210may be stored in one or more memory circuits of one or more computing devices, where the intermediate results and the output predictions may be computed based on the data input are computed using one or more processing circuits of the one or more computing devices. These processing circuits may include, but are not limited to, central processing units

(CPUs), graphics processing units (GPUs), neural accelerator units, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and the like. The processing circuits are configured to perform operations according to various embodiments of the present disclosure using program instructions that may be stored in one or more memory circuits (e.g., the same memory circuits that store the input data, output data, and intermediate results, or different memory circuits).

As shown inFIG.2A, the system may be configured to predict the propensity of a customer (e.g., prospective customer or a newly onboarded customer as shown inFIG.1B) to adopt each of a plurality of different products211, illustrated inFIG.2Aas Product1through Product n. Each of the products211may relate to a different service offered by the service provider. For example, in the case of a financial services provider, these may include a product for a customer to manage subscriptions to services that are provided by the customer to its own users, a product for managing the computation, collection, and payment of sales taxes across different jurisdictions, and a product for managing the issuance of credit cards. As another example, in the case of cloud computing services provider, these may include a web app hosting service, a virtual machine service, a database service, a key value store service, a block storage device service, an infrastructure health monitoring service, an alert service, a message queue service, and the like.

As noted above, the system210is configured to compute, for a given customer, a plurality of propensities212corresponding to each of the products211. Each of the propensities212represents a degree of product fit between the product and the given customer (e.g., a likelihood, probability, or other numerical metric). These propensities212may therefore be used to evaluate the likelihood that the given customer will adopt the corresponding product (e.g., become a subscriber or user of that corresponding product).

The inputs to the system210include the data that is available for a given customer. As noted above, for a prospective customer, the system210may have access only to publicly available data describing the customer (e.g., as collected from literature published by the customer such as a website operated by the customer, from third party aggregators of information, and from published descriptions in official sources). In addition, as noted above, in the case of an onboarded customer (who has not yet started using the services), the customer may also provide text-based descriptions of themselves.

FIG.2Bis a flowchart depicting a method230for predicting the propensity of a customer to adopt services offered by a platform according to one embodiment of the present disclosure.

The available text-based data213regarding the customer to be evaluated for propensity to adopt various product may be collected together and supplied to a pre-processor214. At231, the pre-processor applies transformations to the text data using natural language processing (NLP) techniques, such as removing stop words (e.g., words with low semantic value such as “the”, “and”, “a”, “at”, “which”, “that”, “on”, and the like), computing the length of the collected text, removing duplicate chunks of text, and the like.

At233, the pre-processed text is then provided to a language model encoder215, which is configured to generate a customer feature embedding216(or feature vector) of the pre-processed text (e.g., a representation of the text as a vector of numbers) in an embedding space (e.g., or latent space or latent feature space, where similar customers have similar customer feature embeddings). Examples of language models that may be used to perform the embedding of the text into a latent space include, but are not limited to, Bidirectional Encoder Representations from Transformers (BERT), generative pre-trained transformers (GPT), and the like. The language models may be pre-trained or fine-tuned based on the types of text data expected to be presented to the language model encoder215(e.g., text descriptions of the types of companies that are expected to be customers of the service provider).

As noted above, in some circumstances the system may have access to non-textual data217regarding the customer, such as numerical data. These non-textual data217may include data collected from public information provided by the customer (e.g., on a website), from third-party data sources, from public sources, and the like, and may also include data collected directly from the customer during sign-up. At235, in a case where such non-textual data is available, the system210extracts features from this information (e.g., using feature extractor218).

In some embodiments, the feature extractor218converts data into a format suitable for inclusion in the customer feature embedding216(or feature vector). These conversions may include, for example, normalizing input data values into specified ranges and/or applying mathematical operations to the input data values (e.g., converting input values such as revenue or company size to a normalized log scale ranging from0to1), converting multiple choice responses to a one-hot encoding.

At237, the system210generates the customer feature embedding216. In circumstances where such non-textual data217regarding the customer is available, the extracted features from the non-textual data217are combined with the extracted text features (e.g., as extracted using the language model). In circumstances where no non-textual data217is available (e.g., only text data is available), then, in some embodiments, the system210inserts default values for portions of the customer feature embedding that correspond to the non-text features.

At239, the system210supplies the customer feature embedding to a propensity score predictor219, where the propensity score predictor219is trained to compute the plurality of propensities212for the given customer to adopt each of the products211. In some embodiments, the propensity score predictor219is implemented using a neural network. For example, the propensity score predictor219may include one or more fully connected layers (FC layers) of a neural network (e.g., a neural network with a single hidden layer or a deep neural network having more than one hidden layer, where one or more of the hidden layers are fully connected layers). In various embodiments, the propensity score predictor219may be implemented using other trained models such as a gradient boosting with a forest of decision trees (e.g., using XGBoost).

FIG.2Cis a flowchart depicting a method260for training a propensity score predictor to predict the propensity of a customer or prospective customer to adopt services offered by a platform according to one embodiment of the present disclosure. In some embodiments, the method shown inFIG.2Cis performed by the model trainer160as shown inFIG.1A, where the model trainer160is implemented using one or more processing circuits executing instructions stored in one or more memory circuits, where the instructions configure the processing circuits to perform as special purpose devices to perform operations according to embodiments of the present disclosure.

As shown inFIG.2C, at261the model trainer160loads historical customer data for live customers (and/or previously live customers) from the customer data store150as a training data set. This customer data includes information corresponding to the inputs that are to be supplied to the model (e.g., textual data collected from scraping customer websites, from third parties, from published information, and from customer responses to questions during the sign-up process). In addition, the customer data store150may store product usage data for products offered by the service provider since signing up with the service (e.g., which products were adopted and actively used by the customer over a time period, such as being adopted within the first 30 days or first 90 days as a live customer of the service provider). These product usage data serve as labels for the training data. Accordingly, using a method such as the method shown inFIG.2C, the model trainer160trains a statistical model (e.g., a neural network or a fully connected layer thereof, a gradient boosting model, or the like) to predict the labels (e.g., the products) that will be used by a customer based on these input data.

In more detail, an initial statistical model may be provided as an additional input to the model trainer160, where the statistical model may have its parameters (e.g., weights of connections between neurons in a case where the statistical model is a neural network) set to random values (e.g., set using a random number generator) or may have pre-trained parameters that were trained on another data set (e.g., older historical customer data or trained for a different collection of products). At263, the model trainer160computes predictions using the statistical model (e.g., using its current set of parameters). These predictions may correspond to scores representing propensities for a given customer to use or adopt the various products. At265, the model trainer160compares the computed predictions (the outputs of the statistical model) to the labels (e.g., the actual products used by the customers in the training data set) using a loss function to compute loss values.

At267, the model trainer160determines whether the model training process for the statistical model is complete. In some circumstances, this is determined based on whether the accuracy of the statistical model is no longer improving over the previous version of the statistical model based on the previous parameters (e.g., the training of the statistical model has converged), or has improved by less than some threshold amount. In some circumstances, this is determined based on reaching some desired level of accuracy. In some embodiments, this is determined based on reaching a maximum number of training iterations. In some embodiments, this is determined based on a combination of the factors discussed above and may include additional factors.

In a case where training is not complete, at269the model trainer160updates the parameters of the statistical model based on the computed loss. In some embodiments, this is performed using gradient descent and, in the case of a neural network, the parameters of the neural network (the weights of the connections between the layers of neurons) are updated using backpropagation. After updating the parameters of the statistical model, the model trainer160returns to perform another iteration of the training process at263. Different iterations of the training process may use different portions of the training data (e.g., the training data may be broken into batches).

In a case where the model training is determined to be complete, then the trained model (e.g., the trained parameters) are output by the model trainer160, and the trained model may be included in the trained models110shown inFIG.1AandFIG.1B(e.g., after validating the model using validation data taken from the customer data store150).

Accordingly,FIG.2A,FIG.2B, andFIG.2Cdepict some embodiments of methods for training and implementing statistical models to compute the propensity of a prospective customer or a newly onboarded customer to adopt various products offered by a service provider based on training data from the product adoption behavior of other customers of the service provider.

Some aspects of embodiments of the present disclosure relate to further updating the system210to compute propensity scores for additional products (e.g., newly added products) or to remove products (e.g., remove discontinued products). For example, in the architecture shown inFIG.2A, components such as the customer descriptions in the form of available text-based data213, the pre-processor214, the language model encoder215, and the feature extractor218are not affected by the addition or removal of products. Instead, the addition or removal of products merely involves changing the behavior of the propensity score predictor. Accordingly, in a case where the propensity score predictor219is implemented using a neural network (e.g., a neural network having a single, fully connected hidden layer or a deep neural network having multiple hidden layers), changes in the collection of products for which the system210will generate recommendations involves re-training the propensity score predictor219, such as by applying the method260shown inFIG.2C, where the starting point of the statistical model may be the pre-trained weights of the prior model (updated to change the number of outputs or to change the mapping of the outputs to propensities for corresponding products).

In some embodiments of the present disclosure, the statistical model includes a separate, independent statistical model for each product. Accordingly, removing a product from the system (e.g., when a product is discontinued) may be performed by removing the corresponding model for that product from the statistical model (without affecting the computations of propensities for the other products) and adding a product to the system (e.g., when a new product is introduced) may be performed by training a new statistical model for that product (without affecting the computations of the propensities for the other products).

As noted above with respect toFIG.1B, customer of the service provider may progress through various stages including prospective customers, newly onboarded customers, and live customers. Some aspects of embodiments of the present disclosure further relate to modifying the approach described above with respect toFIG.2A,FIG.2B, andFIG.2Cto further apply to training and using product upsell propensity models112as shown inFIG.1Bto compute the propensity of live customers (current users) to be upsold on products (e.g., begin to use products that they had not previously been using).

FIG.3Ais a block diagram depicting components of a system310for predicting the propensity of a customer to adopt services offered by a platform based on historical interactions with the platform (platform usage data) according to one embodiment of the present disclosure. In a manner similar to that described above with respect to the system210shown inFIG.2A, the system310may be implemented using one or more computing devices, examples of which are described in more detail below with respect toFIGS.6-8. For example, the input data, intermediate data, and output data (e.g., predictions) computed by the system310may be stored in one or more memory circuits of one or more computing devices, where the intermediate results and the output predictions may be computed based on the data input are computed using one or more processing circuits of the one or more computing devices. These processing circuits may include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), neural accelerator units, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and the like. The processing circuits are configured to perform operations according to various embodiments of the present disclosure using program instructions that may be stored in one or more memory circuits (e.g., the same memory circuits that store the input data, output data, and intermediate results, or different memory circuits).

As shown inFIG.3A, the system310may include a product upsell propensity predictor311configured to take, as input, a customer feature embedding312and to generate propensity scores313for each of a plurality of different products (e.g., each of products1through n in a manner similar to that shown inFIG.2A).

The customer feature embedding312according to some embodiments of the present disclosure includes customer textual descriptions in a manner similar to that described above, such as a current description of the customer as taken from a corporate website associated (e.g., maintained by) the customer, current information collected from third party resources describing companies, and published public information (e.g., filings with governmental organizations), and may also include updated information provided by the customer (e.g., responses to customer surveys or other requests for updated information regarding the customer). The collected information may also include non-textual data, such as categorical data and numerical data representing various characteristics of the customer (e.g., current revenue, transaction volume, and the like).

As noted above, because the system310further assumes that the customer is a live customer, there is additional available information regarding the customer in the form of usage data regarding interactions between the customer and the service provider.

This platform usage data may include, but is not limited to: reaching milestones (e.g., applying for service, date of first paid transaction, date of launch to public use or “go-live” date, and the like); basic features (e.g., location, customer segment, and the like); application usage information (e.g., interactions with a web application and/or native application, accessing documentation, support requests, and the like); product activity (e.g., stage of interaction, such as evaluation, adoption, live usage, level of revenue through a product, and the like); and interactions with the service provider (e.g., transaction volume through the service provider, subscriptions maintained through the service provider, invoices created through the service provider, and the like). Accordingly, user interactions with the service provider may be collected as features representing the current state of a customer. In various embodiments of the present disclosure, these collected features are processed (e.g., mathematically transformed, such as to a log scale and/or normalized) before being added to the customer feature embedding or feature vector representing the customer.

FIG.3Bis a flowchart depicting a method330for predicting the propensity of a customer or prospective customer to adopt services offered by a platform based on historical interactions with the platform according to one embodiment of the present disclosure. At331, the system310supplies the customer feature embedding312as input to the product upsell propensity predictor311.

As shown inFIG.3A, the product upsell propensity predictor311includes a statistical model315that may be trained as discussed in more detail below. The statistical model computes propensity scores for the customer, as described by the customer feature embedding, to adopt each of the products within some upcoming time period (e.g., a time period such as the next 30 days or a time period such as the next 90 days).

In some embodiments, the statistical model315is implemented using an XGBoost model (e.g., an XGBoost classifier), although embodiments of the present disclosure are not limited thereto and other classifiers may be trained instead, such as a neural network, a deep neural network, a random forest of decision trees, and the like. In some embodiments, the output of the statistical model315may be poorly

calibrated, such that the raw propensity scores output by the statistical model315cannot be reliably interpreted as probabilities. Accordingly, in some embodiments of the present disclosure, the product upsell propensity predictor311further includes a score calibrator317that is used to calibrate the raw propensity scores such that the resulting calibrated propensity scores represent probabilities. In some embodiments, the score calibrator317includes an isotonic regression layer to calibrate the raw propensity scores. The calibrated propensity scores (or “propensity scores”) may then be output and saved to a propensity score data store for further use, such as to generate targeted upsell advertising or promotions to the customer, specifically promoting the products that have high upsell propensity for the customer. In some embodiments, a model tracking data store319is used to store the progress of the development of the statistical model315, such as by storing the parameters (e.g., weights) of the statistical model as the statistical model is trained based on the training data (e.g., as the model is updated based on additional data collected from live customers).

FIG.3Cis a flowchart depicting a method360for training a system to predict the propensity of a live customer to adopt services offered by a platform based on historical interactions with the platform according to one embodiment of the present disclosure.

In some embodiments, the method360shown inFIG.3Cis performed by the model trainer160as shown inFIG.1A, where the model trainer160is implemented using one or more processing circuits executing instructions stored in one or more memory circuits, where the instructions configure the processing circuits to perform as special purpose devices to perform operations according to embodiments of the present disclosure.

As shown inFIG.3C, at361the model trainer160loads historical customer data for live customers (and/or previously live customers) from the customer data store150as a training data set. This customer data includes information corresponding to the inputs that are to be supplied to the model (e.g., textual data collected from scraping customer websites, from third parties, from published information, and from customer responses to questions during the sign-up process) and features extracted from customer usage data as discussed above (e.g., reaching milestones, application usage information, product activity, interactions with the service provider, and the like). In addition, the customer data store150may store product usage data for products offered by the service provider since signing up with the service (e.g., which products were adopted and actively used by the customer over a time period, such as being adopted within the first 30 days or first 90 days as a live customer of the service provider). These product usage data serve as labels for the training data. Accordingly, using a method such as the method shown inFIG.3C, the model trainer160trains a statistical model (e.g., a neural network or a fully connected layer thereof, a gradient boosting model, or the like) to predict the labels (e.g., the products) that will be used by a customer based on these input data.

In more detail, an initial statistical model may be provided as an additional input to the model trainer160, where the statistical model may have its parameters (e.g., weights of connections between neurons in a case where the statistical model is a neural network) set to random values (e.g., set using a random number generator) or may have pre-trained parameters that were trained on another data set (e.g., older historical customer data or trained for a different collection of products). At363, the model trainer160computes predictions using the statistical model (e.g., using its current set of parameters). These predictions may correspond to scores representing propensities for a given customer to use or adopt the various products. At365, the model trainer160compares the computed predictions (the outputs of the statistical model) to the labels (e.g., the actual products used by the customers in the training data set) using a loss function to compute loss values.

At367, the model trainer160determines whether the model training process for the statistical model is complete. In some circumstances, this is determined based on whether the accuracy of the statistical model is no longer improving over the previous version of the statistical model based on the previous parameters (e.g., the training of the statistical model has converged), or has improved by less than some threshold amount. In some circumstances, this is determined based on reaching some desired level of accuracy. In some embodiments, this is determined based on reaching a maximum number of training iterations. In some embodiments, this is determined based on a combination of the factors discussed above and may include additional factors. In a case where training is not complete, at369the model trainer160updates

the parameters of the statistical model based on the computed loss. In some embodiments, this is performed using gradient descent and, in the case of a neural network, the parameters of the neural network (the weights of the connections between the layers of neurons) are updated using backpropagation. After updating the parameters of the statistical model, the model trainer160returns to perform another iteration of the training process at363. Different iterations of the training process may use different portions of the training data (e.g., the training data may be broken into batches).

In a case where the model training is determined to be complete, then the trained model (e.g., the trained parameters) are output by the model trainer160, and the trained model may be included in the trained models110shown inFIG.1AandFIG.1B(e.g., after validating the model using validation data taken from the customer data store150).

In a manner similar to that described above with respect to the system210, the statistical model315may include a single model trained to compute the propensity scores for all products for a given customer feature embedding or a plurality of statistical models, where each statistical model is trained to compute a propensity score for a corresponding product. In the case where a shared model is used for all products, adding or removing products may involve retraining the shared model based on the change in the collection of product offerings. In the case where a separate model is trained for each product, adding or removing products may involve training a separate, new model for the new product and, in a corresponding manner, removing a statistical model corresponding to a discontinued product from the collection of models.

Accordingly,FIG.3A,FIG.3B, andFIG.3Cdepict some embodiments of methods for training and implementing statistical models to compute the propensity of a prospective customer or a newly onboarded customer to be upsold on various products offered by a service provider based on training data from the product adoption behavior of other customers of the service provider and based on platform usage data.

Service providers interested in growing their business may attempt to target prospective customers or upsell current customers on new products based on the expected revenue to be earned from selling a given product to that customer (e.g., the revenue expected to be earned based on the customer newly subscribing to the product). Accordingly, some aspects of embodiments of the present disclosure relate to computing estimates of product value or product revenue conditional on the user going live. Multiplying the conditional estimates by the product propensity scores produce the desired expected product value from users going live. (For example, a customer going live on an expensive product may produce a large amount of revenue, but if the product is a poor fit for the customer needs, the customer is unlikely to subscribe to the product and therefore marketing efforts related to that product may be wasted if the customer propensity score for that product is low.)

FIG.4Ais a schematic depiction of the calculation of per-user rankings of not-yet-adopted products by expected value according to one embodiment of the present disclosure.FIG.4Bis a flowchart depicting a method430for estimating an expected value of a customer adoption of a product according to one embodiment of the present disclosure.

As shown inFIG.4A, a conditional product revenue calculator410(e.g., as one of the models110shown inFIG.1A) takes one or more propensity scores411as one of its inputs. In various embodiments, a conditional product revenue calculator410using one or more computing devices as described in more detail below with respect toFIGS.6-8.

In some embodiments, each of the propensity scores411represents a propensity for a given customer to adopt a corresponding product (e.g., propensity scores411-1,411-2,411-ncorrespond to propensity scores for the user to adopt Product1, Product2, Product n offered by the service provider). At431the conditional product revenue calculator410receives the one or more product propensity scores, where these propensity scores411may be computed, in some embodiments, using a propensity score predictor219or a product upsell propensity predictor311as discussed above with respect toFIGS.2A,2B,2C,3A,3B, and3C.

In a case where the propensity scores411are not already calibrated, at433the conditional product revenue calculator410computes calibrated versions of the propensity scores413(e.g., calibrated propensity scores413-1,413-2, . . . ,413-ncorrespond to probabilities that the user will adopt Product1, Product2, . . . , Product n offered by the service provider), such as by applying an isotonic regression layer (as described above) to the propensity scores411received as input. The calibration at433may be omitted in circumstances where the one or more propensity scores411are already calibrated (e.g., already represent values that can be accurately treated as probabilities). At435the conditional product revenue calculator410multiplies the calibrated propensity scores413by corresponding expected value estimates conditional on product adoption to compute the expected revenue from the customer for each product of the service provider (e.g., for each product offered by the service provider for which the system is configured to compute expected revenue).

For the sake of discussion, the value associated with the user adoption of a product may be defined, in some embodiments, as value as the one-year post-go-live product revenue from customers who reach the go-live milestone for a product within the next 90 days. However, embodiments of the present disclosure are not limited thereto, and these time frames may be chosen to balance between the desire for a longer-term outcome and the availability of product usage data for the products offered by the service provider.

FIG.4Cis a block diagram depicting components of a system450for training a model to estimate conditional product revenue according to one embodiment of the present disclosure. As shown inFIG.4C, the system450receives data customer-level revenue data451from current live customers. The system450also loads additional features453such as those described above with respect to customer feature embedding312shown inFIG.3A, such as customer product activity features453-1, customer characteristics453-2(e.g., customer segment), and customer description embeddings453-3(e.g., based on text descriptions of the customer). Accordingly, in some embodiments, the features453shown inFIG.4Care similar to those of the customer feature embedding312with the further inclusion of corresponding revenue data from that customer on a per-product basis.

During a process of training a statistical model for computing revenue on a per-product basis, a set of training features455-1are taken from the collected features453, where the customer feature embedding312may represent an input to the model and the per-product revenue corresponds to the value being predicted by the model (e.g., the labels). Accordingly, a model training process457(e.g., executed by the model trainer160shown inFIG.1A) trains a statistical model to predict revenue451for each of one or more products based on the customer features453(e.g., without the per-product revenue as an input to the model).

FIG.4Dis a flowchart depicting a method480for training a system to estimate an expected value of a customer adoption of a product according to one embodiment of the present disclosure.

In some embodiments, the method480shown inFIG.4Dis performed by the model trainer160as shown inFIG.1A, where the model trainer160is implemented using one or more processing circuits executing instructions stored in one or more memory circuits, where the instructions configure the processing circuits to perform as special purpose devices to perform operations according to embodiments of the present disclosure.

As shown inFIG.4D, at481the model trainer160loads historical customer data for live customers (and/or previously live customers) from the customer data store150as a training data set. This customer data includes information corresponding to the inputs that are to be supplied to the model (e.g., textual data collected from scraping customer websites, from third parties, from published information, and from customer responses to questions during the sign-up process) and features extracted from customer usage data as discussed above (e.g., reaching milestones, application usage information, product activity, interactions with the service provider, and the like). In addition, the customer data store150may store product usage data for products offered by the service provider since signing up with the service (e.g., which products were adopted and actively used by the customer over a time period, such as being adopted within the first 30 days or first 90 days as a live customer of the service provider). As noted above, the model trainer may further load data regarding revenue from each customer for each of a plurality of products offered by the service provider. These per-customer, per-′product revenue data serve as labels for the training data. Accordingly, using a method such as the method shown inFIG.4D, the model trainer160trains a statistical model (e.g., a neural network or a fully connected layer thereof, a gradient boosting model, or the like) to predict the labels (e.g., the per-product revenue) that will be earned from a customer based on these input data.

In more detail, an initial statistical model may be provided as an additional input to the model trainer160, where the statistical model may have its parameters (e.g., weights of connections between neurons in a case where the statistical model is a neural network) set to random values (e.g., set using a random number generator) or may have pre-trained parameters that were trained on another data set (e.g., older historical customer data or trained for a different collection of products). At483, the model trainer160computes predictions using the statistical model (e.g., using its current set of parameters). These predictions may correspond to scores representing propensities for a given customer to use or adopt the various products. At485, the model trainer160compares the computed predictions (the outputs of the statistical model) to the labels (e.g., the actual products used by the customers in the training data set) using a loss function to compute loss values.

At487, the model trainer160determines whether the model training process for the statistical model is complete. In some circumstances, this is determined based on whether the accuracy of the statistical model is no longer improving over the previous version of the statistical model based on the previous parameters (e.g., the training of the statistical model has converged), or has improved by less than some threshold amount. In some circumstances, this is determined based on reaching some desired level of accuracy. In some embodiments, this is determined based on reaching a maximum number of training iterations. In some embodiments, this is determined based on a combination of the factors discussed above and may include additional factors.

In a case where training is not complete, at489the model trainer160updates the parameters of the statistical model based on the computed loss. In some embodiments, this is performed using gradient descent and, in the case of a neural network, the parameters of the neural network (the weights of the connections between the layers of neurons) are updated using backpropagation. After updating the parameters of the statistical model, the model trainer160returns to perform another iteration of the training process at483. Different iterations of the training process may use different portions of the training data (e.g., the training data may be broken into batches).

In a case where the model training is determined to be complete, then the trained model (e.g., the trained parameters) are output by the model trainer160, and the trained model may be included in the trained models110shown inFIG.1AandFIG.1B(e.g., after validating the model using validation data taken from the customer data store150).

In a manner similar to that described above with respect to the system210, the statistical model459may include a single model trained to compute the conditional product revenue for all products for a given customer feature embedding or a plurality of statistical models, where each statistical model is trained to compute a conditional product revenue for a corresponding product. In the case where a shared model is used for all products, adding or removing products may involve retraining the shared model based on the change in the collection of product offerings. In the case where a separate model is trained for each product, adding or removing products may involve training a separate, new model for the new product and, in a corresponding manner, removing a statistical model corresponding to a discontinued product from the collection of models.

Referring back toFIG.4C, after completing training of the model, a trained statistical model459may be used to perform predictions or estimates of conditional product revenue based on input features or prediction features455-2of a given customer.

The resulting conditional product revenue estimates may then be multiplied by the calibrated product propensities, as shown inFIG.4A, to compute the expected revenue from each product for a given customer. These resulting expected revenue values may then be filtered to remove products that the customer is already subscribed to (e.g., because no estimate of the associated revenue is needed) and may then be sorted based on value to show the highest value potential products to sell or upsell to the customer. As shown inFIG.1A, in some embodiments, the predictions130produced by

the models110are supplied to marketing and promotion systems140. The marketing and promotion systems140

FIG.5is a block diagram of a marketing and promotion system500configured to generate marketing recommendations based on predictions of customer propensity and conditional product revenue according to some embodiments of the present disclosure. As shown inFIG.5, prospective customer product propensities511may be supplied as input to the marketing and promotion system500.

A product filter513filters the available products (e.g., the products offered by the service provider) to identify targeted products based on the prospective customer product propensities511. In some embodiments, the product filter513sorts the products based on their corresponding propensities and selects some number of highest ranking products (e.g., three products having highest propensity) as targeted products for the customer. In some embodiments, the product filter513selects all products satisfying a propensity threshold (e.g., having a propensity score above a threshold value) as targeted products.

Prospective customers may be reachable through various prospective customer contact channels515. These contact channels may include, but are not limited to, emails (e.g., assuming an email address is available, such as where the prospective customer signed up for a mailing list and opted into promotional emails), phone calls (e.g., with live sales agents), paper mailings, banner advertisements on web pages (e.g., targeting based on tracking cookies), and the like. Accordingly, the marketing and promotion system500generates targeted product marketing messages517for the prospective customer based on their accessibility via the corresponding prospective customer contact channels. These generated messages may include, but are not limited to, generating customized images or paper mailings promoting the specific targeted products and explanations as to why those products may be a good fit for the customer, generating sales guidance or scripts or reports for review and use by a human sales agent during discussions with the prospective customer, and the like.

Similar approaches may be applied to onboarded customers. As shown inFIG.5, the marketing and promotion system500may receive onboarded customer product propensities531and apply these onboarded customer product propensities to a product filter533to identify highest propensity products for the onboarded customer. The onboarded customer contact channels535may be substantially the same as the contact channels of a prospective customer contact515, but where contact information may be explicitly available as obtained through a sign-up process (e.g., email addresses and telephone numbers). Furthermore, an additional contact channel is available through user interfaces provided by the service provider, such as native applications (e.g., desktop applications and mobile applications) and web applications or websites operated by the service provider. For example, after completing a sign-up process, recommended products can be shown to a newly logged-in user to suggest ways to get started in their business relationship with the service provider, where these recommended products are identified based on the onboarded customer product propensities. Accordingly, similar targeted product marketing messages537are generated by the marketing and promotion system500and provided to the onboarded customer via corresponding contact channels535.

Similar approaches may further be applied to live customers. As shown inFIG.5, the marketing and promotion system500may receive live customer product propensities and/or expected revenue551(e.g., the marketing and promotion system500may also, or instead, receive expected revenue calculations for the customer for a plurality of products). The live customer product propensities and/or the expected revenue calculations may be supplied to a product filter553to identify targeted products (e.g., products having high propensity and/or high expected revenue).

The live customer contact channels555may be substantially the same as the contact channels of an onboarded customer contact535. For example, after completing a sign-up process, recommended products can be shown to a live customer within a user interface for the service provider (e.g., a dashboard), where the user interface provides a location to show promoted products or recommended products. Accordingly, similar targeted product marketing messages557are generated by the marketing and promotion system500and provided to the live customer via corresponding contact channels555.

In some embodiments, the prospective customer product propensities are computed for a cohort or audience of multiple prospective customers. For example, an advertising service may allow its user to target an audience sharing some collection of characteristics (e.g., accountants working at startup company having fewer than 100 employees). In such circumstances, the customer feature embeddings may be computed based on the descriptions available through the advertising service and the customer feature embedding for this audience is used to predict prospective customer product propensities for that audience in order to generate targeted advertisements for that audience relating to products that the target audience is more likely to adopt.

Accordingly, aspects of embodiments of the present disclosure relate to systems and methods for computing targeted product offerings for customers based on available information such as descriptions of the customers, platform usage data by the customers, and conditional product revenue. Based on the computed target product offerings, targeted advertisements or promotions are automatically presented to the corresponding customers (e.g., prospective customers, newly onboarded customers, and live customers), thereby improving the communication of the benefit of products offered by the service provider (e.g., the potential value of the products to users). This improves the value obtained by customers, who may be able to start using the products sooner than they would otherwise or who may not have known about the availability of such products. This may also improve communications between sales agents and customers by providing information (e.g., talking points) regarding the benefits of the products to those customers.

With reference toFIG.6, an example embodiment of a high-level SaaS network architecture600is shown. A networked system616provides server-side functionality via a network610(e.g., the Internet or a WAN) to a client device608. A web client602and a programmatic client, in the example form of a client application604(e.g., client software for accessing the service platform and through which advertisements or promotions for the platform may be displayed, client software for sales and marketing agents to review reports regarding the targeted products, and client software for operators of the service platform to control the system to compute product propensities and expected revenue for customers or groups of customers), are hosted and execute on the client device608. The networked system616includes one or more servers622(e.g., servers hosting services exposing remote procedure call APIs), which hosts a processing system606(such as the processing system described above according to various embodiments of the present disclosure supporting a service product adoption system) that provides a number of functions and services via a service oriented architecture (SOA) and that exposes services to the client application604that accesses the networked system616where the services may correspond to particular workflows. The client application604also provides a number of interfaces described herein, which can present an output in accordance with the methods described herein to a user of the client device608.

The client device608enables a user to access and interact with the networked system616and, ultimately, the processing system606. For instance, the user provides input (e.g., touch screen input or alphanumeric input) to the client device608, and the input is communicated to the networked system616via the network610. In this instance, the networked system616, in response to receiving the input from the user, communicates information back to the client device608via the network610to be presented to the user.

An API server618and a web server620are coupled, and provide programmatic and web interfaces respectively, to the servers622. For example, the API server618and the web server620may produce messages (e.g., RPC calls) in response to inputs received via the network, where the messages are supplied as input messages to workflows orchestrated by the processing system606. The API server618and the web server620may also receive return values (return messages) from the processing system606and return results to calling parties (e.g., web clients602and client applications604running on client devices608and third-party applications614) via the network610. The servers622host the processing system606, which includes components or applications in accordance with embodiments of the present disclosure as described above. The servers622are, in turn, shown to be coupled to one or more database servers624that facilitate access to information storage repositories (e.g., databases626). In an example embodiment, the databases626includes storage devices that store information accessed and generated by the processing system606such as databases storing descriptions of users, platform usage data, product revenue on a per customer basis, trained models, and the like.

Additionally, a third-party application614, executing on one or more third-party servers621, is shown as having programmatic access to the networked system616via the programmatic interface provided by the API server618. For example, the third-party application614, using information retrieved from the networked system616, may support one or more features or functions on a website hosted by a third-party. For example, the third-party application614may serve as a data source for retrieving, for example, descriptions of customers or other non-text data regarding customers (e.g., prospective customers) may also be accessed by the processing system606.

Turning now specifically to the applications hosted by the client device608, the web client602may access the various systems (e.g., the processing system606) via the web interface supported by the web server620. Similarly, the client application604(e.g., an “app” such as a payment processor app) may access the various services and functions provided by the processing system606via the programmatic interface provided by the API server618. The client application604may be, for example, an “app” executing on the client device608, such as an iOS or Android OS application to enable a user to access and input data on the networked system616in an offline manner and to perform batch-mode communications between the client application604and the networked system616.

Further, while the network architecture600shown inFIG.6employs a client-server architecture, the present disclosure is not limited to such an architecture, and could equally well find application in a distributed, or peer-to-peer, architecture system, for example.

FIG.7is a block diagram illustrating an example software architecture706, which may be used in conjunction with various hardware architectures herein described.

FIG.7is a non-limiting example of a software architecture706, and it will be appreciated that many other architectures may be implemented to facilitate the functionality described herein. The software architecture706may execute on hardware such as a machine800ofFIG.8that includes, among other things, processors804, memory/storage806, and input/output (I/O) components818. A representative hardware layer752is illustrated and can represent, for example, the machine800ofFIG.8. The representative hardware layer752includes a processor754having associated executable instructions704. The executable instructions704represent the executable instructions of the software architecture706, including implementation of the methods, components, and so forth described herein. The hardware layer752also includes non-transitory memory and/or storage modules as memory/storage756, which also have the executable instructions704. The hardware layer752may also include other hardware758.

In the example architecture ofFIG.7, the software architecture706may be conceptualized as a stack of layers where each layer provides particular functionality. For example, the software architecture706may include layers such as an operating system702, libraries720, frameworks/middleware718, applications716(such as the services of the processing system), and a presentation layer714. Operationally, the applications716and/or other components within the layers may invoke API calls708through the software stack and receive a response as messages712in response to the API calls708. The layers illustrated are representative in nature, and not all software architectures have all layers. For example, some mobile or special-purpose operating systems may not provide a frameworks/middleware718, while others may provide such a layer. Other software architectures may include additional or different layers.

The operating system702may manage hardware resources and provide common services. The operating system702may include, for example, a kernel722, services724, and drivers726. The kernel722may act as an abstraction layer between the hardware and the other software layers. For example, the kernel722may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, and so on. The services724may provide other common services for the other software layers. The drivers726are responsible for controlling or interfacing with the underlying hardware. For instance, the drivers726include display drivers, camera drivers, Bluetooth® drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Wi-Fi® drivers, audio drivers, power management drivers, and so forth depending on the hardware configuration.

The libraries720provide a common infrastructure that is used by the applications716and/or other components and/or layers. The libraries720provide functionality that allows other software components to perform tasks in an easier fashion than by interfacing directly with the underlying operating system702functionality (e.g., kernel722, services724, and/or drivers726). The libraries720may include system libraries744(e.g., C standard library) that may provide functions such as memory allocation functions, string manipulation functions, mathematical functions, and the like. In addition, the libraries720may include API libraries746such as media libraries (e.g., libraries to support presentation and manipulation of various media formats such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG), graphics libraries (e.g., an OpenGL framework that may be used to render 2D and 3D graphic content on a display), database libraries (e.g., SQLite that may provide various relational database functions), and the like. The libraries720may also include a wide variety of other libraries748to provide many other APIs to the applications716and other software components/modules.

The frameworks/middleware718provide a higher-level common infrastructure that may be used by the applications716and/or other software components/modules. For example, the frameworks/middleware718may provide high-level resource management functions, web application frameworks, application runtimes742(e.g., a Java virtual machine or JVM), and so forth. The frameworks/middleware718may provide a broad spectrum of other APIs that may be utilized by the applications716and/or other software components/modules, some of which may be specific to a particular operating system or platform.

The applications716include built-in applications738and/or third-party applications740. The applications716may use built-in operating system functions (e.g., kernel722, services724, and/or drivers726), libraries720, and frameworks/middleware718to create user interfaces to interact with users of the system. Alternatively, or additionally, in some systems, interactions with a user may occur through a presentation layer, such as the presentation layer714. In these systems, the application/component “logic” can be separated from the aspects of the application/component that interact with a user.

Some software architectures use virtual machines. In the example ofFIG.7, this is illustrated by a virtual machine710. The virtual machine710creates a software environment where applications/components can execute as if they were executing on a hardware machine (such as the machine800ofFIG.8, for example). The virtual machine710is hosted by a host operating system (e.g., the operating system702inFIG.7) and typically, although not always, has a virtual machine monitor760(or hypervisor), which manages the operation of the virtual machine710as well as the interface with the host operating system (e.g., the operating system702). A software architecture executes within the virtual machine710such as an operating system (OS)736, libraries734, frameworks732, applications730, and/or a presentation layer728. These layers of software architecture executing within the virtual machine710can be the same as corresponding layers previously described or may be different.

Some software architectures use containers770or containerization to isolate applications. The phrase “container image” refers to a software package (e.g., a static image) that includes configuration information for deploying an application, along with dependencies such as software components, frameworks, or libraries that are required for deploying and executing the application. As discussed herein, the term “container” refers to an instance of a container image, and an application executes within an execution environment provided by the container. Further, multiple instances of an application can be deployed from the same container image (e.g., where each application instance executes within its own container). Additionally, as referred to herein, the term “pod” refers to a set of containers that accesses shared resources (e.g., network, storage), and one or more pods can be executed by a given computing node. A container770is similar to a virtual machine in that it includes a software architecture including libraries734, frameworks732, applications730, and/or a presentation layer728, but omits an operating system and, instead, communicates with the underlying host operating system702.

FIG.8is a block diagram illustrating components of a machine800, according to some example embodiments, able to read instructions from a non-transitory machine-readable medium (e.g., a computer-readable storage medium) and perform any one or more of the methodologies discussed herein. Specifically,FIG.8shows a diagrammatic representation of the machine800in the example form of a computer system, within which instructions810(e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine800to perform any one or more of the methodologies discussed herein may be executed. As such, the instructions810may be used to implement modules or components described herein. The instructions810transform the general, non-programmed machine800into a particular machine800programmed to carry out the described and illustrated functions in the manner described. In alternative embodiments, the machine800operates as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine800may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine800may include, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions810, sequentially or in parallel or concurrently, that specify actions to be taken by the machine800. Further, while only a single machine800is illustrated, the term “machine” or “processing circuit” shall also be taken to include a collection of machines that individually or jointly execute the instructions810to perform any one or more of the methodologies discussed herein.

The machine800may include processors804(including processors808and812), memory/storage806, and I/O components818, which may be configured to communicate with each other such as via a bus802. The memory/storage806may include a memory814, such as a main memory, or other memory storage, and a storage unit816, both accessible to the processors804such as via the bus802. The storage unit816and memory814store the instructions810embodying any one or more of the methodologies or functions described herein. The instructions810may also reside, completely or partially, within the memory814, within the storage unit816, within at least one of the processors804(e.g., within the processor's cache memory), or any suitable combination thereof, during execution thereof by the machine800. Accordingly, the memory814, the storage unit816, and the memory of the processors804are examples of machine-readable media.

The I/O components818may include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I/O components818that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones may include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I/O components818may include many other components that are not shown inFIG.8. The I/O components818are grouped according to functionality merely for simplifying the following discussion, and the grouping is in no way limiting. In various example embodiments, the I/O components818may include output components826and input components828. The output components826may include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The input components828may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing instruments), tactile input components (e.g., a physical button, a touch screen that provides location and/or force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

In further example embodiments, the I/O components818may include biometric components830, motion components834, environment components836, or position components838, among a wide array of other components. For example, the biometric components830may include components to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or electroencephalogram-based identification), and the like. The motion components834may include acceleration sensor components (e.g., accelerometer), gravitation sensor components, rotation sensor components (e.g., gyroscope), and so forth. The environment components836may include, for example, illumination sensor components (e.g., photometer), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometer), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., gas sensors to detect concentrations of hazardous gases for safety or to measure pollutants in the atmosphere), or other components that may provide indications, measurements, or signals corresponding to a surrounding physical environment. The position components838may include location sensor components (e.g., a Global Positioning System (GPS) receiver component), altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude may be derived), orientation sensor components (e.g., magnetometers), and the like.

Communication may be implemented using a wide variety of technologies. The I/O components818may include communication components840operable to couple the machine800to a network832or devices820via a coupling824and a coupling822, respectively. For example, the communication components840may include a network interface component or other suitable device to interface with the network832. In further examples, the communication components840may include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. The devices820may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

Moreover, the communication components840may detect identifiers or include components operable to detect identifiers. For example, the communication components840may include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect one-dimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar code, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals). In addition, a variety of information may be derived via the communication components840, such as location via Internet Protocol (IP) geo-location, location via Wi-Fi® signal triangulation, location via detecting an NFC beacon signal that may indicate a particular location, and so forth.

According to one embodiment of the present disclosure, a method includes: extracting text features from one or more text descriptions of a customer using a language model; computing a customer feature embedding based on the text features; computing a plurality of product propensities for the customer by supplying the customer feature embedding to a trained statistical model, the product propensities representing likelihoods that the customer will adopt corresponding products offered by a service platform; identifying one or more targeted products based on the product propensities; and generating one or more targeted product marketing messages to the customer based on the one or more targeted products.

The trained statistical model may include a fully connected layer of a neural network or gradient boosting with a forest of decision trees.

The trained statistical model may be trained based on training data including: one or more training text descriptions of one or more live customers of the service platform; and product usage information for the corresponding products offered by the service platform. The product usage information may include historical data regarding product adoption within a time period.

The method may further include retrieving the one or more text descriptions from one or more of: a website associated with the customer; a third-party data source of company information; and a publication regarding the customer.

The customer may be an onboarded customer of the service platform, and the one or more text descriptions may include an answer to a question presented to the customer during a sign-up process. The method may further include displaying the one or more targeted product marketing messages in a user interface for the customer.

The customer may be a live customer of the service platform, and the method may further include collecting platform usage data from the customer based on interactions between the customer and the service platform.

The method may further include: computing a plurality of expected revenue values for the corresponding products offered by the service platform based on multiplying the product propensities by conditional expected product revenue values of the customer for the corresponding products offered by the service platform. The conditional expected product revenue values may be computed by a second statistical model, the second statistical model trained based on historical data associating customer feature embeddings to customer-level revenue for the corresponding products offered by the service platform.

According to one embodiment of the present disclosure, a computer system includes: a processor; and memory storing instructions that, when executed by the processor, cause the processor to: extract text features from one or more text descriptions of a customer using a language model; compute a customer feature embedding based on the text features; compute a plurality of product propensities for the customer by supplying the customer feature embedding to a trained statistical model, the product propensities representing likelihoods that the customer will adopt corresponding products offered by a service platform; identify one or more targeted products based on the product propensities; and generate one or more targeted product marketing messages to the customer based on the one or more targeted products.

The trained statistical model may include a fully connected layer of a neural network or gradient boosting with a forest of decision trees.

The trained statistical model may be trained based on training data including: one or more training text descriptions of one or more live customers of the service platform; and product usage information for the corresponding products offered by the service platform. The product usage information may include historical data regarding product adoption within a time period.

The memory may further store instructions that, when executed by the processor, cause the processor to retrieve the one or more text descriptions from one or more of: a website associated with the customer; a third-party data source of company information; and a publication regarding the customer.

The customer may be an onboarded customer of the service platform, and the one or more text descriptions include an answer to a question presented to the customer during a sign-up process.

The memory may further store instructions that, when executed by the processor, cause the processor to display the one or more targeted product marketing messages in a user interface for the customer.

The customer may be a live customer of the service platform, and the memory may further store instructions that, when executed by the processor, cause the processor to collect platform usage data from the customer based on interactions between the customer and the service platform.

The memory may further store instructions that, when executed by the processor, cause the processor to: compute a plurality of expected revenue values for the corresponding products offered by the service platform based on multiplying the product propensities by conditional expected product revenue values of the customer for the corresponding products offered by the service platform. The conditional expected product revenue values may be computed by a second statistical model, the second statistical model trained based on historical data associating customer feature embeddings to customer-level revenue for the corresponding products offered by the service platform.

It should be understood that the sequence of steps of the processes described herein in regard to various methods and with respect various flowcharts is not fixed, but can be modified, changed in order, performed differently, performed sequentially, concurrently, or simultaneously, or altered into any desired order consistent with dependencies between steps of the processes, as recognized by a person of skill in the art. Further, as used herein and in the claims, the phrase “at least one of element A, element B, or element C” is intended to convey any of: element A, element B, element C, elements A and B, elements A and C, elements B and C, and elements A, B, and C.

While the present invention has been described in connection with certain exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims, and equivalents thereof.