DIGITAL ENROLLMENT SYSTEMS AND METHODS

A computerized method is provided for responding to a request by a customer to enroll into a digital service. The method includes generating a personalized media clip for presentation to the enrolling customer, which comprises (i) using an artificial intelligence (AI) model to determine a plurality of relevant media objects based on data related to the request and customer data and (ii) forming a randomized composite of the plurality of relevant media objects. The method also includes providing the personalized media clip along with an instruction to the customer to record an audio description of the media clip. The method further includes generating a confidence score that measures a degree of accuracy of the audio description by the customer in relation to the personalized media clip, where enrollment of the customer into the digital service is based on at least the confidence score.

TECHNICAL FIELD

This application relates generally to systems, methods and apparatuses, including computer program products, for responding to a request by a customer to enroll in a digital service.

BACKGROUND

Traditionally, service enrollment of a customer based on voice biometrics is accomplished with a live associate over a telephony communication channel. This form of communication promotes customer thinking voice in order to capture the full color of the customer's utterance, as the thinking voice typically includes hesitation, confirmation, clarity of thoughts, etc. In contrast, service enrollment based on voice biometrics in the digital space/platform is limited to presenting the customer with a predefined one-size-fits-all script so that the customer can read out loud using his/her readout voice, which differs greatly from the thinking voice. Therefore, systems and methods are needed to introduce thinking voice capability in the digital space for facilitating customer enrollment to digital services based on voice biometrics.

SUMMARY

To remedy the above shortcomings in today's market, the present invention provides systems and methods for customer digital enrollment using artificial intelligence (AI) algorithms to invoke a customer's thinking voice without human (e.g., live agent) intervention. In some embodiments, the AI algorithms are used to understand the customer, based on which one or more visuals are selected during runtime to invoke the customer's thinking voice. In some embodiments, the customer's thinking voice is converted to text and analyzed for indicators confirming the thinking voice as well as validating the customer for the purpose of fraud detection. The present invention can be used for digital enrollment to and/or digital procurement of a variety of services such as branch visit, account opening, proactive engagement over the Internet and proactive engagement on mobile devices.

In one aspect, the present application features a computerized method for responding to a request by a customer to enroll into a digital service. The computerized method includes generating, by a computing device, a personalized media clip for presentation to the enrolling customer. Generating the personalized media clip comprises (i) using an artificial intelligence (AI) model to determine a plurality of relevant media objects based on data related to the request and customer data and (ii) forming a randomized composite of the plurality of relevant media objects. The method also includes providing, by the computing device, the personalized media clip along with an instruction to the customer to record an audio description of the personalized media clip and generating, by the computing device, a confidence score that measures a degree of accuracy of the audio description by the customer in relation to the personalized media clip. The confidence score comprises a weighted sum of a plurality of matching scores including (i) a static matching score generated by comparing a text representation of the audio description with a list of one or more predefined keywords, and (ii) an AI score generated by determining whether the text representation describes the randomized composite of the relevant media objects in the personalized media clip. The method further comprises enrolling, by the computing device, the customer into the digital service based on at least the confidence score.

In another aspect, the invention features a computerized means for responding to a request by a customer to enroll into a digital service. The computerized means comprises means for generating a personalized media clip for presentation to the enrolling customer including (i) means for generating and training an artificial intelligence (AI) model to determine a plurality of relevant media objects based on data related to the request and customer data and (ii) means for forming a randomized composite of the plurality of relevant media objects. The computerized means also includes means for providing the personalized media clip along with an instruction to the customer to record an audio description of the personalized media clip and means for generating a confidence score that measures a degree of accuracy of the audio description by the customer in relation to the personalized media clip. The confidence score comprises a weighted sum of a plurality of matching scores including (i) a static matching score generated by comparing a text representation of the audio description with a list of one or more predefined keywords, and (ii) an AI score generated by determining whether the text representation describes the randomized composite of the relevant media objects in the personalized media clip. The computerized means further includes means for enrolling the customer into the digital service based on at least the confidence score.

Any of the above aspects can include one or more of the following features. In some embodiments, each of the plurality of relevant media objects comprises one of a visual image or an audio segment. In some embodiments, the AI model is trained to model relationships between historical request contexts and media objects.

In some embodiments, the data related to the request and the customer data includes one or more of customer demographics information, customer browsing history and interaction history from similar customers.

In some embodiments, the personalized media clip comprises a video segment of a randomized composite of images selected by the AI model. In some embodiments, the randomized composite is formed at runtime as the personalized media clip is presented to the customer.

In some embodiments, the instruction further includes interactive requests asking the customer for one or more physical inputs. The one or more physical inputs can include face capture, expression capture, body movements, or click or drag a visual item.

In some embodiments, the text representation of the audio description is processed before generating the plurality of matching scores. Processing the text representation comprises one or more of tokening the text representation and removing one or more stop words from the text representation.

In some embodiments, the plurality of matching scores further includes a fraud score generated based on fraud analytics of the customer. In some embodiments, the plurality of matching scores further includes a score indicating if the customer is a part of a digital enrollment guest list for the digital service. In some embodiments, the plurality of matching scores further includes a dynamic matching score generated by computing and allocating weights to words in the text representation of the audio description.

In some embodiments, enrolling the customer based on at least the confidence score comprises comparing the confidence score with a predefined confidence level, confirming that a biometric signal associated with the customer matches the customer's biometric print, and allowing customer enrollment if at least one of the confidence score exceeds the predefined confidence level and the biometric signal matches. In some embodiments, the customer is presented with a new personalized media clip if the confidence score is below the predefined confidence level but above a lower confidence threshold indicating a borderline case.

DETAILED DESCRIPTION

FIG.1shows an exemplary diagram of a digital enrollment engine100used in a computing system101for digitally enrolling a customer by invoking the customer's thinking voice, according to some embodiments of the present invention. As shown, the computing system101generally includes at least one client computing device102, a communication network104, the digital enrollment engine100, and one or more databases108.

The client computing device102connects to the communication network104to communicate with the digital enrollment engine100and/or the database108to provide inputs and receive outputs relating to the process of vocally signing a digital document as described herein. For example, the computing device102can provide a detailed graphical user interface (GUI) that allows a user to input enrollment request data and voice samples and display instructions and results using the analysis methods and systems described herein. Exemplary computing devices102include, but are not limited to, telephones, desktop computers, laptop computers, tablets, mobile devices, smartphones, and internet appliances. In some embodiments, the computing device102has voice playback and recording capabilities. It should be appreciated that other types of computing devices that are capable of connecting to the components of the computing system101can be used without departing from the scope of invention. AlthoughFIG.1depicts a single computing device102, it should be appreciated that the computing system101can include any number of client devices.

The communication network104enables components of the computing system101to communicate with each other to perform the process of enrollment of customers to digital services. The network104may be a local network, such as a LAN, or a wide area network, such as the Internet and/or a cellular network. In some embodiments, the network104is comprised of several discrete networks and/or sub-networks (e.g., cellular to Internet) that enable the components of the system100to communicate with each other.

The digital enrollment engine100is a combination of hardware, including one or more processors and one or more physical memory modules and specialized software engines that execute on the processor of the digital enrollment engine100, to receive data from other components of the computing system101, transmit data to other components of the computing system101, and perform functions as described herein. As shown, the processor of the digital enrollment engine100executes a visual processing AI module114, an orchestration module116, and an authentication module118. These sub-components and their functionalities are described below in detail. In some embodiments, the various components of the digital enrollment engine100are specialized sets of computer software instructions programmed onto a dedicated processor in the digital enrollment engine100and can include specifically-designated memory locations and/or registers for executing the specialized computer software instructions.

The database108is a computing device (or in some embodiments, a set of computing devices) that is coupled to and in communication with the digital enrollment engine100and is configured to provide, receive and store various types of data received and/or created for performing voice signature of digital documents, as described below in detail. In some embodiments, all or a portion of the database108is integrated with the digital enrollment engine100or located on a separate computing device or devices. For example, the database108can comprise one or more databases, such as MySQL™ available from Oracle Corp. of Redwood City, California.

FIG.2shows a process diagram of an exemplary computerized method200for digitally enrolling a customer by invoking the customer's thinking voice utilizing the computing system101and resources ofFIG.1, according to some embodiments of the present invention. The method200starts with the orchestration module116of the digital enrollment engine100receiving data related to a request to enroll into a digital service made from a customer via the customer's computing device102(step202). For example, the customer can supply the request and related data from a web-based user interface generated by the orchestration module114and displayed on the customer's computing device102. In some embodiments, the web-based user interface is initiated by the customer via a vendor website to start the desired enrollment request. In some embodiment, the orchestration module116can collect pertinent contextual data related to the enrolling customer, such as customer digital footprint and metadata, demographic data, historical transactional trending data, relationship lifetime data, prediction related to the individual customer and current capability usage from other customers, fraud analysis, market analysis, and/or relationship with third party institutions.

Upon receiving the customer request and related contextual information, the orchestration module116is configured to interact with the visual processing AI module114to generate a personalized media clip for presentation to the enrolling customer (step204). The visual processing AI module114can generate the personalized media clip by (i) first using a trained artificial intelligence (AI) model to determine one or more media objects relevant to the enrolling customer and (ii) forming a randomized composite of the relevant media objects in the media clip that is personalized to the enrolling customer. In some embodiments, the visual processing AI module114trains the AI model to predict relationships between request contexts and media objects. Thus, the trained AI model is configured to generate media objects relevant to a particular enrollment request. In some embodiments, a media object is a visual image or an audio segment.

In some embodiments, the visual processing AI module114uses a hybrid Neural Collaborative Filter algorithm to train the AI model. In some embodiments, the data used to train the AI model includes previously-collected relationship data between customer request contexts and relevant media objects. For example, the training data can comprise at least one of demographics information related to past customers (e.g., existing customer profile data), data related to customer actions across multiple interaction channels (e.g., customer browsing history on other platforms), interaction history data, and device-based feedback such as click stream, live customer interaction, proactive surveys, feedback loop etc. The training data can also include compressive analysis of customers belonging to similar social economic background. In some embodiments, the training data includes recent national/international news that can be utilized to present a relevant theme to the customer for more interactive experience. In some embodiments, the trained model is periodically evaluated, updated and re-trained to take into consideration of more current training data, such as most recent customer data and interaction history data. The resulting trained AI model can be stored in the database108for easy access and retrieval.

In some embodiments, the AI algorithm is implemented in a recommendation system as a hybrid neural collaborative filtering (CF) algorithm. The recommendation system is configured to return a composite visual presentation that is adapted to engage the customer and capture the full color of their thinking voice with no human intervention. This hybrid algorithm combines two or more recommendation strategies in different ways to benefit from their complementary advantages. The hybrid algorithm not only considers user's historical behavior information, but also take into account of the user's context information described above, such as demographics, behavior of customer with similar profile, trust relationships, friend relationships, user tags, time information, location, etc. For example, the training data used in conjunction with the hybrid algorithm can include one or more of visual composites that have been used during previous digital enrollments sessions, spontaneous visual add-ons, NLP that identifies what user has identified and mentioned, customer demographics, behavior of customers with similar profile, trust relationships, friend relationships, user tags, item attributes, time information, location, click stream information, customer phone call records, etc.

In some embodiments, the hybrid algorithm integrates various latent factor models with various users' social relationships, and the results indicate that data dimensions are reduced, recommendation accuracy is improved, and scalability of the recommendation system is enhanced based on these models.FIG.5shows an exemplary data structure500of the hybrid artificial intelligence algorithm used by the digital enrollment engine100ofFIG.1, according to some embodiments of the present invention. As shown, the exemplary data structure500can include an input layer502, an embedding layer504, one or more neural CF layers506and an output layer508, where the output of one layer serves as the input of the next one. The input layer502can include two feature vectors502a,502bdescribing user u and item i, respectively. These feature vectors502a,502bare customizable to support a wide range of modeling of users and items and are context-aware, content-based, and neighbor-based, for example. Above the input layer502is the embedding layer504, which is a connected layer that projects each sparse representation in the input layer (i.e., feature vectors502a,502b) to a dense vector. In the context of latent factor model, the resulting user embedding can be represented as a user latent vector504aand the resulting item embedding can be represented as an item latent vector504b. The user embedding and item embedding504a,504bare then fed into a multi-layer neural architecture, i.e. the one or more neural CF layers506, to map the latent vectors504a,504bto prediction scores. Each layer of the neural CF layers506can be customized to discover certain latent structures of user-item interactions. In some embodiments, the neural CF layers506includes at least one hidden layer X506a, the dimension of which determines the model's capability. The output layer508produces a predicted score (508a). In some embodiments, model training can be performed with the goal of minimizing the pointwise loss between the predicted score508aand its target value Yui (510). Thus, the predicted score can be formulated as the following equation:

where Θout and Θx respectively denote the mapping function for the output layer and x-th neural collaborative filtering (CF) layer, and there are X neural CF layers in total.

After AI model training, the visual processing AI module114can supply contextual data related to the enrolling customer (collected from step202) as inputs to the trained AI model to determine a set of one or more relevant media objects (e.g., visual images and/or audio segments). Based on the relevant media objects obtained, the visual processing AI module114can form a personalized media clip comprising a randomized composite of the multiple relevant media objects and present the personalized media clip to the enrolling customer via a user interface to invoke the customer's thinking voice for the purpose of enrollment/authentication (step206). In some embodiments, the randomization is performed at runtime as the media clip is presented to the customer. As an example, the personalized media clip can be a video segment of a randomized, ad-hoc composite of images selected by the trained AI model. In some embodiments, the user interface additionally provides written instructions to the enrolling customer to record an audio description of the media clip as the media clip is being played to the customer. In some embodiments, in addition to such audio recording, the digital enrollment engine can instruct the enrolling customer to supply other interactive inputs, such as one or more physical inputs, for the purpose of authenticating the customer. Exemplary physical inputs include one or more of face capture, expression capture, specific body movements, and/or click or drag a visual item. In some embodiments, the enrollment process, including feedback/inputs received from the customer, takes place within an augmented reality (AR) or virtual reality (VR) environment. An AR model can utilize a real-world setting while placing objects, images and/or video(s) within the customer's environment for requested descriptions. A VR model can utilize a virtual reality environment while placing objects, images and/or video(s) within the customer's environment for requested descriptions. Exemplary customer feedback within an AR or VR environment includes a hint/nudge, such as a touch, vibration, gesture (e.g., via a device that can capture gestures by finger movement), and/or user mode (e.g., sitting versus standing).

In some embodiments, the visual processing AI module114is configured to also generate a pool of words describing the personalized media clip. First, each select media object can be associated with a set of pre-defined static descriptive keywords prior to runtime, thereby forming a pool of pre-defined static keywords associated with the media clip. At runtime, the visual processing AI module114can determine a randomized order to play these select media objects (i.e., a randomized composite of the media objects) and generate a set of dynamic keywords associated with the personalized media clip. These dynamic keywords can be generated/extracted from a master description of the media clip. Additional words similar to the words in the master description can be determined and added to the pool of dynamic keywords. Further, over time as the media objects in the media clip are displayed to other users in other media clips, user-supplied description of the media objects can be saved as keywords to the pool of dynamic keywords. For example, dynamic keywords can be generated by analyzing previous customer responses to the given media object and identifying frequent commonalities. These commonalities can be selected keywords based on frequency, which can be added to the pool of dynamic keywords improving the AI model. Commonalities can also be between similar customer backgrounds (e.g., age, sex, location, depth and correlations with respect to virtual reality capabilities, etc.) to determine if similar backgrounds yield more commonalities in descriptions. As an example, there are different vocabulary between younger and older customers, which would influence selection of dynamic keywords personalized to the customer's background. Commonalities can further be considered based on depth of field, distance and proximity within an augmented reality (AR) or virtual reality (VR) experience. In some embodiments, determination of these dynamic keywords is accomplished during run time as the personalized media clip is dynamically assembled and played to the enrolling customer. In some embodiments, the personalized media clip, along with its corresponding pools of static and dynamic descriptive keywords, is saved in the database108.

FIGS.3a-cshow a series of exemplary user interfaces for capturing an enrolling customer's thinking voice based on an exemplary personalize media clip displayed to the customer, according to some embodiments of the present invention. As shown in the user interface300ofFIG.3a, the enrolling customer is provided with a personalized video segment302containing at least one image of a train determined from the trained AI model as discussed above with reference to step204of method200. More specifically, this train image is determined by the AI model as being relevant to the enrolling customer, thus providing the customer with a visual for which he/she can describe based on past knowledge and/or related experiences. The user interface300can further display instructions304to the user for recording the customer's voice sample with respect to the video segment302. In some embodiments, the customer can record his/her voice sample after the video is played. In some embodiments, the customer is asked to record his/her voice sample as the video is being played. For example, the instructions304can ask the customer to vocally describe what he/she is seeing in the video segment302by speaking continuous for a specific period of time and select “record” to start recording or “reset” to restart recording. The recording area306of the user interface300can display a record status button306a, a reset button306b, and a countdown clock306cindicating the time period left for voice recording.

The exemplary user interface310ofFIG.3billustrates how an enrolling customer can record his/her voice sample. As shown, upon the customer pressing the record status button306a, the video segment302starts to play. Contemporaneous with the video presentation, the customer can start to vocally describe the content of the video segment302as it is being played, while the digital enrollment engine100records the customer's utterance. While the customer is speaking, a visual que312can appear on the user interface310to confirm that audio is being heard and voice recording is in progress. Further, the countdown clock306ccan be activated to track the customer's recording progress by indicating the time remaining. In some embodiments, the record status button306acan display a “paused” sign to allow the customer to pause the recording and/or the video presentation. As described above, the composite of images in the video segment302can be determined by the digital enrollment engine100during run time from the trained AI model and played to the enrolling customer during the recording session.

FIG.3cshows an exemplary user interface320of when the recording session is completed. As shown, at the end of the recording session, the video segment302is no long played. In some embodiments, the record status button306acan display a “Done” sign and the countdown clock306ccan indicate 0 seconds remaining. Further, an enrollment button322can appear after audio acquisition to allow the customer to actively proceed with the enrollment process. For example, upon the customer clicking the enrollment button322, the digital enrollment engine100can authenticate the customer by ensuring that the recorded vocal description, which captures the context of what the customer uttered, conforms to the content of the video segment302. Alternatively, the customer can activate the reset button306bto repeat the recording session, at which point a new media clip can be generated and presented to the customer.

Referring back to the enrollment process200ofFIG.2, after an audio recording of the enrolling customer is captured by the digital enrollment engine100, the orchestration module116is adapted to transmit the recording to the authentication module118of the digital enrollment engine100for authenticating the customer associated with the digital enrollment request (step208). In some embodiments, authenticating the customer involves the authentication module118calculating a confidence score that generally measures a degree of accuracy between the audio description by the customer and the content of the personalized media clip. This accuracy score is adapted to validate the liveness, authenticity, and accuracy of the digital enrollment process. To calculate this score, the authentication module118can first process the audio recording by generating a text representation of the audio description using a speech-to-text tool, tokening the text representation and/or removing one or more stop words from the text representation. Other processing methods applied to the audio recording include audio compression and/or voice analysis to determine, for example, sentiment, background frequency, pauses, speech cadence, etc. In some embodiments, the confidence score calculated by the authentication module118is a weighted sum of two or more scores including a static word matching score, a dynamic text similarity matching score, a video/audio stamp matching score, a handshake model matching score, and a predictive digital fraud score. The weights assigned to these scores can be the same or different based on their relative importance in the authentication process.

The static word matching score is generated by comparing the text representation of the audio description from the enrolling customer with a list of one or more predefined keywords associated with the media clip to determine the degree at which the audio description captures these predefined keywords. The dynamic text similarity matching score is generated by comparing the text representation of the audio description with a list of one or more dynamic runtime keywords associated with the personalized media clip to determine the degree at which the audio description captures these dynamic keywords. As described above, these predefined static keywords and dynamic runtime keywords associated with the personalized media clip can be generated by the visual processing AI module114and stored in the database108. The video/audio stamp matching score determines the degree with which the customer's audio description correctly captures the order of presentation of the media objects in the media clip. As described above, this order of presentation is randomized and only determined at runtime as the media clip is played to the enrolling customer. Such ad-hoc presentation ensures that the media clip is personalized and unique to the enrolling customer as it captures an appropriate amount of randomness dictated by the trained AI model. Thus the video/audio stamp matching score is adapted to validate the enrolling customer's audio description with respect to the dynamic ordering of media objects in the media clip.

The handshake model matching score indicates if the enrolling customer is a part of a digital enrollment guest list for the digital service requested. Prior to the enrolling request, the customer can be presented with a channel-agnostic invitation (via, for example, email, online account access or mobile app launch) to interact with the digital service enrollment system100. This invitation can have an expiration time after which any interaction is considered invalid. If the customer interacts with the digital service enrollment system100within the set time, the system100can loosely validate the authenticity of the invitation. In some embodiments, the handshake model matching score is a binary score with one score indicating that the invitation is valid and another score indicating that the invitation is invalid or the customer was never a part of a digital enrollment guest list. The predictive digital fraud score can be generated based on performing fraud analytics on the enrolling customer. These fraud analytics can include a composite of external digital fraud intelligence and internal fraud analytics on the customer. For example, fraud data can be received from centralized or external fraud agencies, where the fraud data includes information rooted in (i) digital footprint of the device or the network metadata, and/or (ii) existing list of voice prints that tagged as potential fraudsters within the current agency or indicated by an external agency.

Referring back toFIG.2, after the authentication module118calculates the confidence score and forwards the score to the orchestration module116, the orchestration module116is configured to determine whether to enroll the customer into the digital service based on the confidence score.FIG.4shows an exemplary decision process employed by the orchestration module116of the digital enrollment engine100ofFIG.1to determine whether to enroll a customer, according to some embodiments of the present invention. As shown, the confidence score is first compared with a predefined confidence level (step402). If the confidence score exceeds the predefined confidence level, the orchestration module118interacts with one or more external systems (e.g., vendor systems) to perform additional types of authentication (step404), such as biometrics validation of the customer's voice sample against a stored voice print of the customer. Other authentication techniques can include playback detection, synthetic voice detection and known fraudster list check. The customer is allowed digital service enrollment if the customer passes at least one of the additional authentication checks or the confidence score exceeds the predefined confidence level. In some embodiments, enrollment is allowed if both conditions are satisfied (step406). In some embodiments, if the customer fails any of the authentication checks, the enrollment is considered unsuccessful, and no voice print is registered. In some embodiments, after a number of failed attempts (e.g., three failed attempts), the customer account is completely locked for enrollment.

Alternatively, if the confidence score is below the predefined confidence level, it is determined if the score is in a borderline range, such as below the confidence level but above a lower confidence threshold (step408). If the confidence score does not represent a borderline case, the customer can be presented with a new personalized media clip by repeating steps204-210of process200ofFIG.2(step410). Such an attempt can be made several times. If the customer fails at the end of multiple attempts (step412), the customer can be routed to a live representative for live enrollment processing (step414). Otherwise process proceeds to step404to perform additional types of authentication. However, if the confidence level indicates a borderline scenario (step408), the customer can be challenged with additional verification actions (step416), such as supplying the last four digits of his/her social security number, zip code on file, beneficiaries on file, etc., without presenting a new media clip to the customer. If the customer fails these additional verification actions (step418), the customer can be routed to a live representative for live enrollment processing (step420). Otherwise, process proceeds to step404to perform additional types of authentication.

To provide for interaction with a user, the above described techniques can be implemented on a computing device in communication with a display device, e.g., a CRT (cathode ray tube), plasma, or LCD (liquid crystal display) monitor, a mobile computing device display or screen, a holographic device and/or projector, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse, a trackball, a touchpad, or a motion sensor, by which the user can provide input to the computer (e.g., interact with a user interface element). Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, and/or tactile input.