ENRICHING ARTIFICIAL INTELLIGENCE MODELS DURING DATA CALL FAILURES USING REAL-TIME INTERNET OF THINGS TOKENS

There are provided systems and methods for enriching AI models during data call failures using real-time IoT tokens. A service provider, such as an electronic transaction processor for digital transactions, may provide computing services to users including those for electronic transaction processing. In order to provide computing services, machine learning engines and neural networks may be used to compute scores ingested by computing services for intelligent decisions, predictions, classifications, and the like. The scores may though have inaccuracies and decay, which may be made worse when data fails to load for particular model or network features. As such, the service provider may utilize a framework to enrich scores through computing their entropy as a function of errors and randomness with their decay as a function of inaccuracies over time. IoT tokens may then be used to enrich and provide further accuracy or validity time based corresponding real-time data.

TECHNICAL FIELD

The present application generally relates to machine learning (ML) and other artificial intelligence (AI) models, and more particularly to enriching model output scores to provide more accurate model predictions.

BACKGROUND

Online service providers may provide services to different users, such as individual end users, merchants, companies, and other entities. For example, online transaction processors may provide electronic transaction processing services. When providing these services, the service providers may provide an online platform that may be accessible over a network, which may be used to access and utilize the services provided to different users. During use of these computing services and the processing platforms and services, the service provider may utilize one or more applications, platforms, and/or decision services that implement and utilize ML and other AI (e.g., neural networks (NN), rule-based engines, etc.) models for classifications, predictions, decision-making, and the like during data processing, such as within a production computing environment. For example, an ML model may be used for processing input data for one or more ML features and determining a classification, prediction, decision, or other output. In particular, fraud prediction and risk analysis models may rely on features (e.g., inputs from various downstream systems) in order to compute the fraud likelihood of a particular activity, interaction, or the like, such as a transaction being processed electronically.

There may invariably be a level of entropy associated with the output that lowers the overall accuracy of the computed score, where a lower accuracy fraud model may lead to loss if processing is based on an incorrect score or prediction. Accuracy issues may be caused by several factors, including feature application programming interface (API) data load issues and/or failures from one or more sources, latency in compute time and/or processing, and decay of data (e.g., stale data) being used by the compute function. For example, data call failures, such as a failure of an API to provide or load data, may result in missing data for features of an ML model when generating a score. Further, traditional temporal and time series forecasting is challenging due to issues with variability of temporal data. Conventional temporal and time series forecasting suffers from issues in accuracy due to these challenges and may not adequately consider certain factors and/or temporal data. Therefore, there is a need for more accurate and efficient intelligent systems for classifications and predictions, such as by lifting or enhancing model accuracy of ML and other AI models.

DETAILED DESCRIPTION

Provided are methods for enriching AI models during data call failures using real-time IoT tokens. Systems suitable for practicing methods of the present disclosure are also provided. Such systems and methods may be further used for enhanced model accuracy during risk assessment, fraud detection, user behavior prediction, and the like. As such, AI models, such as ML models and NNs may be improved to provide better and more accurate outputs, longer accuracy lifetime, and more efficient model computations.

An online service provider, such as an online platform providing one or more services to users and groups of users, may provide a platform that allows a user to access and/or interact with the service provider, live agents of the service provider, chatbots or interactive voice response (IVR) systems, and/or other audio and audiovisual endpoints through voice and/or video communications, calls, and sessions. The service provider may allow the user to register an account and/or utilize computing services through various platforms, communications, applications, websites, and/or devices, such as to perform electronic transaction processing and/or otherwise utilize an account for transaction, payment, transfer, and other monetary services. However, during use of the computing services, the user, merchant, and/or an agent or other service assisting the user may perform data processing that invokes ML models and engines, or other AI systems, such as for electronic transaction processing. identification of user information, account details, past communications and sessions including corresponding data files and contents, authentication, and the like. Thus, the service provider may provide intelligent computing services to users through these AI systems.

In this regard, the service provider may process input data that corresponds to different features or variables of ML models or other AI processing engines. For example, features may be associated with measurable datum or other piece of data relevant to a computing task or requested output by an ML model. These ML models may be associated with fraud and risk, such as fraud detection models during electronic transaction processing, and therefore process input user, transaction, merchant, and other data to provide a fraud score or assessment of potential fraud. However, when API endpoints and calls are unresponsive, fail, or do not provide proper data, ML model and other AI system accuracy may fall, and fraud scores may be incorrect or of low accuracy that the predictions and decisions are no longer valuable or correct, which can result in additional uses of computing resources to address. As such, in various embodiments, the service provider may provide a system that computes entropy and decay scores associated with model scores and outputs based on the input data, time of processing, missing data and/or unresponsive APIs and/or calls, and the like. The service provider may utilize IoT tokens for real-time data from an IoT infrastructure to then enrich these models and model scores such that an accuracy and/or time-to-live (TTL) values improves for such scores. Thereafter, secure enriched tokens may be output, which may provide improved model accuracy, TTL, and/or trust in the ML models' scores and decisions.

In order to provide computing and intelligent decision-making services, an online service provider (e.g., an online transaction processor, such as PAYPAL®) may provide account services to users of the online service provider, as well as other entities requesting additional services. A user wishing to establish the account may first access the online service provider and request establishment of an account. An account and/or corresponding authentication information with a service provider may be established by providing account details, such as a login, password (or other authentication credential, such as a biometric fingerprint, retinal scan, etc.), and other account creation details. The account creation details may include identification information to establish the account, such as personal information for a user, business or merchant information for an entity, or other types of identification information including a name, address, and/or other information.

The user may also be required to provide financial information, including payment card (e.g., credit/debit card) information, bank account information, gift card information, benefits/incentives, and/or financial investments. This information may be used to process transactions for items and/or services. In some embodiments, the account creation may be used to establish account funds and/or values, such as by transferring money into the account and/or establishing a credit limit and corresponding credit value that is available to the account and/or card. The online payment provider may provide digital wallet services, which may offer financial services to send, store, and receive money, process financial instruments, and/or provide transaction histories, including tokenization of digital wallet data for transaction processing. The application or website of the service provider, such as PAYPAL® or other online payment provider, may provide payments and the other transaction processing services. However, other service providers may also provide the computing services discussed herein, such as telecommunication service providers. Once the account of the user is established with the service provider, the user may utilize the account via one or more computing devices, such as a personal computer, tablet computer, mobile smart phone, or the like. The user may engage in one or more online or virtual interactions that may be associated with electronic transaction processing, images, music, media content and/or streaming, video games, documents, social networking, media data sharing, microblogging, and the like.

The interactions may require or utilize AI decision-making and prediction services, such as fraud assessment and detection that utilize ML models, NNs, and other AI engines for a predictive framework using enriched model scores and tokens. As such, an AI engine make receive input data for processing by extracting data for model or engine features or variables and processing the data to provide an output score, classification, decision, prediction, or the like. For example, a framework and/or infrastructure for the ML models, engines, and enrichment processes may be provided and/or be implemented as pluggable modules or other software and/or hardware components that may be integrated into real-time and/or batch processing systems for executing data calls. The pluggable modules may therefore include operations to integrate with the downstream services to enrich ML model scores and classifications using IoT tokens and the like. An online transaction processor or other service provider may execute operations, applications, decision services, and the like that may be used to process transactions between two or more users or entities, as well as provide other computing services and automated decision-making.

In order to do so, a service provider may utilize NNs including deep NN (DNN), LSTM, GRU, or other RNN architectures, ML models, and/or other artificial intelligence (AI) systems. Initially, the service provider may train a DNN or other ML model for a predictive output or classification. In order to train the DNN or other ML models, training data for the models may be collected and/or accessed. The training data may correspond to a set or collection of features from some input data records, which may be associated with users, accounts, activities (e.g., past transactions, frauds, etc.), events, external computing services, entities, and the like. In this regard, a feature may correspond to data that may be used to output a decision by a particular node, which may lead to further nodes and/or output decisions by the DNN or another ML model. Training may be done by creating mathematical relationships based on the NN or other ML algorithm to generate predictions, classifications, and other outputs, such as those associated with fraud detection and scores, risk, and the like. The NN or ML model trainer may perform feature extraction to extract features and/or attributes used to train the model from the training data. For example, training data features may correspond to those data features which allow for value determinations and/or outputs by nodes of a model, which may be used in the final predictive output, score, or classification.

Once trained, DNNs and other ML models may be deployed in production environments for intelligent decision-making, classifications, predictions, and other outputs. Data for features may be received and/or collected from various APIs and API endpoints. However, an API may fail and/or an API call may be unresponsive, thereby causing data to fail to load for one or more features. This may be due to source failure, API call errors or failure, latency, data decay, and the like. When such data fails to load or is stale and inaccurate, outputs of the ML models and other AI systems may be inaccurate. As such, the service provider may enrich such models and corresponding output scores using IoT tokens and corresponding real-time data based on entropy and decay score calculations. Thus, in response to detecting an API failure or other data load failure, a model enrichment framework and system may be invoked to improve model accuracy and TTL of output scores.

For example, the initial fraud or other ML model may have a compute with a failure, which relies on less than all of the features. An entropy score calculation may then be performed by the service provider's model enrichment framework and system. The entropy score may seek to identify and assess a randomness or errors in the initial computed function of the model score, such as the inaccuracy introduced by the API or other data load failure. An entropy function may be executed by the model enrichment framework, which may first retry the failed API and/or call that results in the lack of data, or otherwise attempt to retrieve and access the data that failed to load. If successful, the model may be rerun with the data that may result in a difference in scores, which may be used to assess accuracy. However, if the data further fails to load, the entropy function may access and retrieve a past model score for the user and/or user's ID via a cache or offline data snapshot when fraud was last measured for the user. The entropy function may measure the entropy score based on previous activities and differences in activities of the user and system including the behavior of the user from the previous score and system components used with the previous score. For example, the user behavior may include an online shopping history, frequented location, ATM and other payment or merchant terminals and devices used, past merchants, and the like. System components used may include devices and operating systems, Internet service providers (ISPs), geo-location data, hypertext transfer protocol (HTTP) headers and data, and the like.

The model enrichment framework may further compute a decay score of the corresponding data and output score. The decay score may correspond to a measurement or indication of the viability and “freshness” of the score and data, which may correspond to the accuracy of the model score as time passes or increases since initial compute time. In this regard, the decay score may be calculated using input variables and data for the feature's data pulled in using the entropy function and for which the entropy score was calculated, such as those features and corresponding data associated with the dataset, transaction history, financial instrument, user profile and the like. The data decay function may analyze a time decay weighted average for the data, a dynamic decay update of the data, a user channel used for data communication, user device and operating system factors, network latency, the computed entropy score, and/or an overall impact to fraud model accuracy when considering decay caused by failure of data loading for model score output of a particular feature on the model score. The decay score may also be calculated using this data at the overall model level when utilizing all available factors, including IoT tokens.

Thereafter, an IoT system and infrastructure may be used to determine and retrieve IoT tokens for real-time data tracked, monitored, and/or captured by the IoT infrastructure and corresponding devices, servers, and/or sensors. For example, the IoT infrastructure may correspond to a smart city with data monitoring sensors and components for various users and device. The IoT tokens may be used to track and monitor, for particular interactions, incremental metadata for events and activities associated with the IoT tokens having corresponding real-time data for users and/or devices that includes a real or IP address, a location, a real-time captured images, a device identifier, a time of day, and the like. These may be separated into particular entities (e.g., location is a first entity, user image is another entity, etc.) to establish scores to the particular missing data from the failed call or load.

A final reconciled scoring manager may the combine and process all scores to provide an enrichment factor or score to the last model score having the missing data. The final compute score may then have a computed higher accuracy using the last model score with an affected decay or TTL of the new enriched model score based on data decay. An enriched token, such as a secure cryptogram, may be generated for the enriched model score and provided or shared for actionable processing of user requests and other data processing requests that come in from user activities, such as electronic transaction processing. In this manner, a service provider may obtain enhanced model accuracy during failure conditions of data calls and other data loading events. This may improve decision-making and accuracy of decisions and intelligent outputs by models, while also reducing the time taken during model predictions by allowing model computation when APIs and/or calls are unresponsive. Thus, the model enrichment framework may improve the technical field and operations of AI systems.

System100includes a user device110, a service provider server120, and IoT infrastructure140in communication over a network150. User device110may be utilized by users or other entities to interact with service provider server120over network150, where service provider server120may provide various computing services, data, operations, and other functions over network150that may include use of data from IoT infrastructure140. In this regard, user device110may perform activities with service provider server120for account establishment and/or usage, electronic transaction processing, and/or other computing services. Service provider server120may provide intelligent computing services to users through ML models, NNs, and/or other AI systems to user device110. In this regard, to enrich outputs and scores when providing services to user device110, IoT infrastructure140may be used by service provider server120to obtain IoT tokens for real-time data that may change an entropy and decay weight, score, or other factor applied to the output score's accuracy, TTL, and other parameter.

User device110may be implemented as a computing and/or communication device that may utilize appropriate hardware and software configured for wired and/or wireless communication with service provider server120. For example, in one embodiment, user device110may be implemented as a personal computer (PC), a smart phone, laptop/tablet computer, wristwatch with appropriate computer hardware resources, eyeglasses with appropriate computer hardware (e.g., GOOGLE GLASS®), other type of wearable computing device, implantable communication devices, and/or other types of computing devices capable of transmitting and/or receiving data. Although only one user computing device is shown, a plurality of user computing device may function similarly.

User device110ofFIG.1contain an application112, a database116, and a network interface component118. Application112may correspond to executable processes, procedures, and/or applications with associated hardware. In other embodiments, user device110may include additional or different modules having specialized hardware and/or software as required.

Application112may include one or more processes to execute software modules and associated components of user device110to provide features, services, and other operations to a user from service provider server120over network150, which may include account, electronic transaction processing, and/or other computing services and features provided by service provider server120. In this regard, application112may correspond to specialized software utilized by users of user device110that may be used to access a website or application (e.g., mobile application, rich Internet application, or resident software application) that may display one or more user interfaces that allow for interaction with service provider server120, for example, to access an account, process transactions, and/or otherwise utilize computing services. In various embodiments, application112may correspond to one or more general browser applications configured to retrieve, present, and communicate information over the Internet (e.g., utilize resources on the World Wide Web) or a private network. For example, application112may provide a web browser, which may send and receive information over network150, including retrieving website information, presenting the website information to the user, and/or communicating information to the website. However, in other embodiments, application112may correspond to a dedicated application of service provider server120or other entity (e.g., a merchant) for transaction processing via service provider server120. Thus, application112may be used to transmit or provide a request114to service provider server120for data processing, which may utilize IoT infrastructure140during data processing. For example, request114may include a request for a data processing event that uses data from IoT tokens generated by components, devices, and sensors on IoT infrastructure140.

Application112may utilize, process, and/or provide account information, user financial information, and/or transaction histories for electronic transaction processing, including processing transactions using financial instrument or payment card data. Application112and/or another device application may be used to request data processing of request114using data from IoT infrastructure140that may be used to enrich model scores and the like from ML models, DNNs, and the like that provide classifications, predictions, decision-making, and other outputs used by computing services of service provider server120. Prior data requests and other data processing events may be used to train ML and DNN models. The training data may include one or more data records, which may be stored and/or persisted in a database and/or data tables accessible by service provider server120. Thereafter, the DNN or ML model(s) may be deployed in production computing environments, where service provider server120may use such models and networks for intelligent outputs and with computing services. However, during use of models, input data for features may be unavailable as APIs fail or unresponsive, and data calls may not retrieve or access data for particular features of the DNN or ML model. As such, service provider server120may utilize IoT infrastructure140to enrich model scores using IoT data and corresponding ones of IoT tokens146, which may enhance model accuracy, reliability, TTL, and/or trust based on the available data.

Additionally, application112may be used to view the results of processing request114, such as outputs, results, and the like from use of computing services of service provider server120. In this regard, application112may be used for one or more data processing tasks, such as electronic transaction processing. During processing of an electronic transaction, application112may be utilized to enter, view, and/or process items the user wishes to purchase in a transaction, as well as perform peer-to-peer payments and transfers. In this regard, application112may provide transaction processing through a user interface enabling the user to enter and/or view the items that users associated with user device110wish to purchase and submit request114for purchase of the item(s). Thus, application112may also be used by a user to provide payments and transfers to another user or merchant, which may include transmitting request114to service provider server120. Processing of request114may utilize one or more DNNs and/or ML models during processing by computing services (e.g., risk, fraud detection, credit or underwriting, etc.), which may have resulting scores enriched by data and IoT tokens146from IoT infrastructure140.

For example, accounts and electronic transaction processing may include and/or utilize user financial information, such as credit card data, bank account data, or other funding source data, as a payment instrument when providing payment information to service provider server120for the transaction. Additionally, application112may utilize a digital wallet associated with an account with a payment provider as the payment instrument, for example, through accessing a digital wallet or account of a user through entry of authentication credentials and/or by providing a data token that allows for processing using the account. Application112may also be used to receive a receipt or other information based on transaction processing. Further, additional services may be provided via application112, including social networking, media posting or sharing, microblogging, data browsing and searching, online shopping, and other services available through service provider server120.

User device110may further include database116stored on a transitory and/or non-transitory memory of user device110, which may store various applications and data and be utilized during execution of various modules of user device110. Database116may include, for example, identifiers such as operating system registry entries, cookies associated with application112and/or other applications, identifiers associated with hardware of user device110, or other appropriate identifiers, such as identifiers used for payment/user/device authentication or identification, which may be communicated as identifying a user and/or user device110to service provider server120. Moreover, database116may store data used for generating and/or transmitting request114, including digital tokens for digital wallets, accounts, and/or financial instruments and the like.

User device110includes at least one network interface component118adapted to communicate with service provider server120and/or another device or server. In various embodiments, network interface component118may each include a DSL (e.g., Digital Subscriber Line) modem, a PSTN (Public Switched Telephone Network) modem, an Ethernet device, a broadband device, a satellite device and/or various other types of wired and/or wireless network communication devices including WiFi, microwave, radio frequency, infrared, Bluetooth, and near field communication devices.

Service provider server120may be maintained, for example, by an online service provider, which may provide computing services including account and electronic transaction processing services. In this regard, service provider server120includes one or more processing applications which may be configured to interact with user device110to provide computing and customer services using one or more ML, NN, or other AI models and engines. In various embodiments, use of the intelligent computing services may utilize data from IoT infrastructure with correspond decay and entropy of model scores to enrich such scores and provide more accurate model outputs. In one example, service provider server120may be provided by PAYPAL®, Inc. of San Jose, CA, USA. However, in other embodiments, service provider server120may be maintained by or include another type of service provider.

Service provider server120ofFIG.1includes a predictive model platform130, service applications122, a database124, and a network interface component128. Predictive model platform130and service applications122may correspond to executable processes, procedures, and/or applications with associated hardware. In other embodiments, service provider server120may include additional or different modules having specialized hardware and/or software as required.

Predictive model platform130may correspond to one or more processes to execute modules and associated specialized hardware of service provider server120to provide intelligent machine outputs used with computing services provided to users including use of ML models132to generate model scores134that may be consumed and utilized during execution and use of service applications122. Further, predictive model platform130includes score enrichment processes136to generate enriched score tokens138to provide further and enhanced accuracy of model scores134from ML models132based on entropy and decay of model scores134during and after computation with information from IoT infrastructure140. In this regard, predictive model platform130may correspond to specialized hardware and/or software used by a user associated with user device110in conjunction with service applications122for intelligent computing services. Predictive model platform130may receive or detect events that request data processing and intelligent outputs using ML models132, such as model scores134that may be used during computing service provision. The events may be associated with data processing requests, actions, activities, or the like that occur for a user, account, entity, device, or the like. In this regard, predictive model platform130may utilize ML and/or DNN models, such as ML models132having trained layers based on training data and selected ML features or variables, to determine and output model scores134. Using model scores134, determination of entropy and/or decay in model scores134, such as based on API call failures or other lack of data during computation of model scores134, which may be eligible for enrichment using score enrichment processes136. IoT tokens146may be received and/or requested for real-time data144from IoT infrastructure140, which may then be used by score enrichment processes136for enrichment of model scores134. IoT tokens146, as well as real-time data144, may be received in response to API requests or other API calls to IoT infrastructure140and have corresponding API responses from IoT infrastructure140. The resulting enriched score tokens138may include enrichment to model accuracy, TTL of model scores134, and the like, which may be used for further accuracy, reliability, and/or validity length based on the availability of real-time data144for corresponding data that failed to load or be received during computation of model scores134by ML models132.

In this regard, ML models132may initially be trained using training data determined and/or extracted from data tables and/or data records for corresponding features or variables selected for training of ML models132and decision-making during execution of ML models132. For example, ML features or variables may correspond to individual pieces, properties, characteristics, or other inputs for an ML model and may be used to cause an output by that ML model once the ML model has been trained using data for those features from training data. ML models132, once trained, may be used for computation and calculation of model scores134based on ML layers that are trained and optimized. ML models132may be trained to provide a predictive output, such as a score, likelihood, probability, or decision, associated with a particular prediction, classification, or categorization of model scores134.

For example, ML models132may include DNN, ML, or other AI models trained using training data having data records that have columns or other data representations and stored data values (e.g., in rows for the data tables having feature columns) for the features. When building ML models132, training data may be used to generate one or more classifiers and provide recommendations, predictions, or other outputs for model scores134based on those classifications and an ML or NN model algorithm and architecture. Such determination of model scores134may be used with service applications122during the provision of computing services, such as risk, fraud detection, authentication, credit or underwriting, marketing, or the like. The data calls to particular APIs and corresponding API endpoints may incur issues that fail to load data or otherwise fail and be unresponsive, thereby not loading all the data required by ML models132and leading to one or more of model scores134have sub-optimal processing for proper scoring and output.

However, decisions and predictions may still be made and valuable for service applications122, and therefore model scores134may still be useful even when generated without all feature data for the corresponding one of ML models132. The amount of inaccuracy or changes to TTL of model scores134may be assessed using an entropy and decay, which may be applied to model scores134to adjust accuracy and the like. Further, IoT infrastructure140may be availed to provide IoT tokens146, which may increase accuracy, TTL, or other parameters indicating the value or reliability of model scores134. Thus, by utilizing IoT tokens146, score enrichment processes136may generate enriched score tokens138to provide additional accuracy and other benefits to model scores134.

The algorithm and architecture for training ML models132may correspond to DNNs, ML decision trees and/or clustering, and other types of ML architectures. The training data may be used to determine features, such as through feature extraction and feature selection using the input training data. For example, DNN models for ML models132may include one or more trained layers, including an input layer, a hidden layer, and an output layer having one or more nodes, however, different layers may also be utilized. As many hidden layers as necessary or appropriate may be utilized and the hidden layers may include one or more layers used to generate vectors or embeddings used as inputs to other layers and/or models. In some embodiments, each node within a layer may be connected to a node within an adjacent layer, where a set of input values may be used to generate one or more output values or classifications. Within the input layer, each node may correspond to a distinct attribute or input data type for features or variables that may be used to train ML models132, for example, using feature or attribute extraction with the training data.

Thereafter, the hidden layer(s) may be trained with this data and data attributes, as well as corresponding weights, activation functions, and the like using a DNN algorithm, computation, and/or technique. For example, each of the nodes in the hidden layer generates a representation, which may include a mathematical computation (or algorithm) that produces a value based on the input values of the input nodes. The DNN, ML, or other AI architecture and/or algorithm may assign different weights to each of the data values received from the input nodes. The hidden layer nodes may include different algorithms and/or different weights assigned to the input data and may therefore produce a different value based on the input values. The values generated by the hidden layer nodes may be used by the output layer node(s) to produce one or more output values for ML models132that attempt to classify and/or categorize the input feature data and/or data records. Thus, when ML models132are used to perform a predictive analysis and output, the input data may provide a corresponding output for model scores134based on the classifications trained for ML models132.

Layers of ML models132may be trained by using training data associated with data records and a feature extraction of training features. By providing training data to train ML models132, the nodes in the hidden layer may be trained (adjusted) such that an optimal output (e.g., a classification) is produced in the output layer based on the training data. By continuously providing different sets of training data and penalizing ML models132when the output of ML models132is incorrect, ML models132(and specifically, the representations of the nodes in the hidden layer) may be trained (adjusted) to improve its performance in data classifications and predictions, such as outputs of model scores134. Adjusting ML models132may include adjusting the weights associated with each node in the hidden layer.

Score enrichment processes136may be provided to introduce, add, or modify model scores134by effects, loss on classification or outputs, and the like caused by executing ML models132without all of the feature data for a feature set of the corresponding model(s). For example, one of ML models132may have a corresponding feature set where an API failure results in a lack of data or output of the data for a particular feature. As such, the resulting computing score of model scores134relies on data for less than all features, where the missing feature may have no data or a substituted data to account for the API failure. To adjust, score enrichment process may calculate an entropy score as an assessment of the randomness or errors in the initial computed function and score caused by the API failure (e.g., unresponsive or no data API/call or failure to load data from such API/call). However, with the entropy calculation, a model score and/or feature data for a previous model score may be fetched for the given customer or event identifier, such as one for a last score calculation. A decay score may be calculated as a time-based decay associated with the feature pulled in with the entropy calculation. With the entropy and decay scores, one or more of IoT tokens146may be availed to establish correlations with such scores and available real-time data. Score enrichment processes136may process the data together using a formula to calculate enriched score tokens138, which may then be consumed by service applications122during computing service provision.

Service applications122may correspond to one or more processes to execute modules and associated specialized hardware of service provider server120to process a transaction or provide another service to customers, merchants, and/or other end users and entities of service provider server120. In this regard, service applications122may correspond to specialized hardware and/or software used by service provider server120to providing computing services to users, which may include electronic transaction processing and/or other computing services using accounts provided by service provider server120. In some embodiments, service applications122may be used by users associated with user device110to establish user and/or payment accounts, as well as digital wallets, which may be used to process transactions. In various embodiments, financial information may be stored with the accounts, such as account/card numbers and information that may enable payments, transfers, withdrawals, and/or deposits of funds. Digital tokens for the accounts/wallets may be used to send and process payments, for example, through one or more interfaces provided by service provider server120.

In this regard, enriched score tokens138may be generated, which may be processed with corresponding real-time data144as token data126for decision-making, predictions, classifications, and the like during data processing by service applications122. Thus, enriched score tokens138may include data used for authentications, risk analysis, fraud detection, and the like for account access and/or account use. Computing services of service applications122may also or instead correspond to messaging, social networking, media posting or sharing, microblogging, data browsing and searching, online shopping, and other services available through service provider server120. Thus, enriched score tokens138may be used for other computing services based on model scores134for use with users, customers, and other entities.

Service applications122may also be desired in particular embodiments to provide features to service provider server120. For example, service applications122may include security applications for implementing server-side security features, programmatic client applications for interfacing with appropriate application programming interfaces (APIs) over network150, or other types of applications. Service applications122may contain software programs, executable by a processor, including a graphical user interface (GUI), configured to provide an interface to the user when accessing service provider server120via one or more of user device110, where the user or other users may interact with the GUI to view and communicate information more easily. In various embodiments, service applications122may include additional connection and/or communication applications, which may be utilized to communicate information to over network150.

Service provider server120further includes database124. Database124may store various identifiers associated with user device110. Database124may also store account data, including payment instruments and authentication credentials, as well as transaction processing histories and data for processed transactions. Database124may store financial information or other data generated and stored by predictive model platform130. Database124may also include data and computing code, or necessary components for ML models132. Database124may also include training data, input, and/or feature data having data records, as well as data for model scores134and enriched score tokens138that may be processed during provision of computing services by service applications122.

In various embodiments, service provider server120includes at least one network interface component128adapted to communicate with user device110, IoT infrastructure140, and/or other devices or server over network150. In various embodiments, network interface component128may comprise a DSL (e.g., Digital Subscriber Line) modem, a PSTN (Public Switched Telephone Network) modem, an Ethernet device, a broadband device, a satellite device and/or various other types of wired and/or wireless network communication devices including WiFi, microwave, radio frequency (RF), and infrared (IR) communication devices.

IoT infrastructure140may correspond to a network of devices, sensors, applications, and other components that are connected within or in association with one or more real-world locations, such as a city, county, region, or the like. In this regard, IoT infrastructure140includes IoT components142corresponding to one or more websites, applications, devices, sensors, servers and server systems, cloud computing platforms and storages, and other resources that may detect and collect real-time data144and other information associated with the location(s) and provide IoT tokens146for real-time data144to service provider server120in response to executed data calls and requests (e.g., API calls to APIs of IoT infrastructure140). IoT infrastructure140may be provided by a location, city, or the like for detecting real-time data144, which may utilize IoT components142for data collection and processing purposes. In some embodiments, IoT infrastructure140may be hosted, provided by, and/or utilized by a merchant, seller, or the like to advertise, market, sell, and/or provide items or services for sale, as well as provide checkout and payment. As such, IoT components142may be distributed throughout and/or connected with the location for data detection purposes.

As such, real-time data144may correspond to images, video, audio, location-based data, communications, device connections and/or pings to local devices or servers, check-ins or check-outs, processing of transactions, telecommunications, vehicle movements or use, appliance and other device use, and the like. Real-time data144may therefore correspond to IP addresses, geo-locations, real-time/last known pictures of the subject, device identifiers, time of the day for events, etc., as well as metadata for IoT infrastructure140. Real-time data144may be broken down into discrete entities to establish correlations with the scores and data from ML models132of service provider server120. Further, one or more of IoT infrastructure140may correspond to token service providers (TSPs), such as those that may provide and/or utilize network tokens and network token cryptograms for tokenizing real-time data144for provision of IoT tokens146to service provider server120. As such, IoT tokens146may correspond to secure tokens for real-time data144that may be used to retrieve, determine, and validate real-time data144for use in enriching model scores134of ML models132. In some embodiments, the processes, APIs and/or API integrations for data requesting and retrieval by service provider server may be based on one or more operations, software development kits (SDKs), API standards or guidelines, and the like that may be implemented in the corresponding computing service of service provider server120for requesting and receiving IoT tokens146, as well as corresponding data from real-time data144detected by IoT components142.

Network150may be implemented as a single network or a combination of multiple networks. For example, in various embodiments, network150may include the Internet or one or more intranets, landline networks, wireless networks, and/or other appropriate types of networks. Thus, network150may correspond to small scale communication networks, such as a private or local area network, or a larger scale network, such as a wide area network or the Internet, accessible by the various components of system100.

FIGS.2A-2Care exemplary diagrams of data processing for enriching scores and outputs of AI models during data call failures using real-time IoT tokens and corresponding data, according to an embodiment. Diagram200aincludes a system architecture that provides a fraud score or other model compute and output for intelligent computing services when processing a payment attempt202. In this regard, an ML compute system204processing payment attempt202may correspond to one or more of ML models132when executed by predictive model platform130of service provider server120, discussed in reference to system100ofFIG.1.

In diagram200a, a real-time computing environment, according to one embodiment, is shown that is utilized to process payment attempt202and/or other requests and uses of computing services, such as one of electronic transaction processing using an online transaction processor or the like. Prediction of risk, fraud, or the like, as well as authentication and/or authorization, may be required for transaction processing, where ML compute system204may provide such automated and machine intelligence through ML models. In this regard, ML compute system204may include components needed for ML model computes and output scores, which may be enriched to provide further accuracy and confidence. This may be of particular importance with ML computations lacking data for one or more features.

In this regard, ML compute system204includes an initial compute206resulting from execution of an ML model using available feature data, which may be missing data for one or more features for the ML model. The missing data may therefore cause inaccuracies and/or quick decay or low TTL of the resulting output. As such, an entropy management208may add an entropy score to initial compute206to account for inaccuracies, errors, and/or randomness introduced by missing data when running the ML model on the input data. Entropy management208may further access a previous ML score or compute, which may allow for a previous compute and/or data to be used to supplement the missing data. Entropy management208may use an entropy cache218to access past model computes, as well as a smart cache with retrial capabilities220for past data and/or computes. A decay function210further may be added to determine changes to TTL and the like caused by the missing data and/or based on introducing the previous compute and/or data.

To increase accuracy and/or TTL of initial compute206, an IoT secure token212may be availed from an IoT infrastructure and/or system. IoT secure token212may correspond to real-time data from the IoT infrastructure and components, which may assist in improving the confidence in initial compute206, as well as accuracy and/or TTL based on real-world detected data. Using the data, a final reconciled compute score214may be determined, which may correspond to the enriched model score, compute, or other output. Final reconciled compute score214may be tokenized as secure fraud token216, which may be stored with smart cache with retrial capabilities220and made available to computing services for use. Thus, when required for processing, secure fraud tokens216may be accessed. For example, with payment attempt202, a high accuracy fraud score compute222may be output to payment processes224for payment authorization or denial.

Referring toFIG.3B, diagram200bshows entropy that may be calculated using the represented parameters and data for entropy with an entropy formula or calculation, according to one embodiment. In this regard, diagram300bshows an entropy232that may be calculated by entropy management208from diagram200abased on the components listed in diagram200b. As such, entropy232may be used when enriching model scores and outputs, such as when generated final reconciled compute score214.

For entropy232, data for a behavior234associated with behavior parameters236may be used with data for a system238including system parameters240. For example, behavior parameters236may include an online shopping history, frequent locations, frequent ATMs, and/or frequent merchants, although such parameters are merely representative and additional, fewer, or other parameters may also be used. Behavior parameters236may correspond to expected behaviors and therefore may be used in place of missing data and/or to determine deviations from expected behavior and thus errors that may be introduced when data is missing. For system238, system parameters240may include devices and operation system (OS), Internet service provider (ISP), geo-locations, and/or hypertext transfer protocol (HTTP) headers (e.g., data from requests by a client), although such parameters are merely representative and additional, fewer, or other parameters may also be used. As such, system parameters240may indicate expected data for a system making a request, such as payment attempt202, which may be used to determine both expected system data and/or errors from missing data. Additionally, past known data for behavior parameters236and system parameters240may be used to supplement missing data with the last known record.

Referring now toFIG.2C, diagram200cshows decay that that may be calculated using the represented parameters and data for decay with a decay formula or calculation, according to one embodiment. In this regard, diagram300cshows a decay compute252that may be calculated by decay function210from diagram200abased on the components listed in diagram200c. As such, decay compute252may be used when enriching model scores and outputs, such as when generating final reconciled compute score214.

In diagram300c, decay compute252may be generated and determined as a function of a dataset254, a transaction history258, a financial instrument262, and a user profile266. In this regard, dataset254may correspond to the data associated with the request being processing using the intelligent ML model, such as user and/or request data when processing payment attempt202. As such, dataset parameters256may include Visa/Mastercard data, for payment data associated with the request (e.g., payment attempt202), although other card and/or payment data for the user may also be used. Transaction history258may include history parameters260for past transactions, including an order flow of transaction processing orders and flows for processing transactions and declines of specific transactions (including decline reason). Financial instrument262may be associated with instrument parameters264when processing the provided financial instrument, such as stored tokens and stolen financials. Additionally for determining decay compute252, profile parameters268for a user profile266may be used, including any account takeovers (ATOs) for the corresponding account of the user.

FIGS.3A and3Bare exemplary diagrams300aand300bof a decay function used to enrich and adjust time validity values and accuracies of ML model scores, according to an embodiment. For example, diagram300ashows a data decay function302as a function of the listed parameters in a similar manner to diagram200cofFIG.2C. These parameters are then shown as processed using a fraud decay manager322in diagram300b. As such, data decay function302in diagram300amay be computed using fraud decay manager322in diagram300bby predictive model platform130using score enrichment processes136in system100ofFIG.1.

For example, entropy may be associated with model scores, especially where such scores may be computed when missing particular data for one or more features. Further, latency in compute and the like may also introduce errors and therefore inaccuracies. However, stale or old data may also introduce errors, randomness, and inaccuracies that lower reliability and confidence in model scores through decay, which may be computed using data decay function302by fraud decay manager322. As such, when missing data is supplemental with past model scores and/or data, as well as when IoT tokens for real-world data is used, measuring decay as a score that may affect accuracy and/or TTL may be used to enrich scores and provide further accuracy and confidence.

Data decay function302may utilize a time decay weighted average 304 to apply a time-based decay to the data used during model score computation and determination. Additionally, a dynamic decay function306may be used where a 5% decay per unit of time (e.g., second) may be introduced but changed to an additional decay when missing data (e.g., 10% per unit of time with a failure of an API and/or data load in the dataset). A user channel308may introduce decay and add to computation of data decay function302, such as a web, mobile, mobile-web, chat, phone, SMS, etc., channel each having a specific decay based on the data and/or data reliability. User device and OS factors310may also introduce decay and be considered, such as where older devices and/or OS may be more vulnerable, as well as the type of device and/or OS. A network latency312may introduce decay based on the latency of receiving data and any potential slowdown or data transmission. With data decay function302, the computed entropy factor314may also be added, where an overall impact to fraud model accuracy316may the be computed as a function of the aforementioned components.

Referring toFIG.3B, in diagram300b, fraud decay manager322may compute a weighted decay factor336to apply to a model score during enrichment based on decay of corresponding data when computing the model score. As such, fraud decay manager322may be based on a decay coefficient324applied to an initial decay326, a decay coefficient328applied to an initial decay330, and a decay coefficient332applied to an initial decay334, which may be combined into weighted decay factor336. Polynomial weighted decay function338shows the function for utilizing decay coefficients324,328, and332with initial decays326,330, and334in decay equation 340. There, initial values for decay may be computed with decay constants and a time factor to provide an output for weighted decay factor336.

FIG.4is a flowchart400of an exemplary process for enriching AI models during data call failures using real-time IoT tokens, according to an embodiment. Note that one or more steps, processes, and methods described herein of flowchart400may be omitted, performed in a different sequence, or combined as desired or appropriate.

At step402of flowchart400, a request for an ML score during computing service use by a user is received. The request may be received from the user directly, such as a risk or fraud score or the like, or the request may be received from a system, engine, and/or application that may utilize the score during provision of computing services, such as an electronic transaction processing application that may use the risk or fraud score in determining whether to proceed with transaction processing. At step404, the ML score is determined using an ML model of an ML engine and feature data, where the feature data has missing data for one or more model features. For example, the ML model for computation of the score may require data for a feature set, where one or more of the features in the feature set may have data that is unavailable, does not load, or fails to be retrieved. For example, an API may fail or be unresponsive to an API call based on failures, communication or network issues, and the like. As such, the score may be determined but may be missing data that may cause issues in accuracy and/or TTL of the score.

At step406, based on the feature data missing for one or more model features and past scores for the user, an entropy score is computed. For example, the entropy score may correspond to a measure or indicator in the randomness, introduced inaccuracy, and/or errors in the initial computation of the model score. This may be based on the model score having missing data for one or more features, and therefore, the entropy score may be based on past similar API and/or data load failures during model score calculation and/or measured inaccuracies in model score calculation resulting from missing data. With calculation of the entropy score, the system may fetch a past model score for comparison and/or feature data for the past model score, which may be used in place of the missing feature to compute a model score and a difference in accuracy or the entropy caused by the missing data during use of the ML model for score computation.

At step408, a decay score for a decay in the accuracy of the ML score over time is computed. The decay score may correspond to a TTL, which indicates a length of time that the model score is considered to be valid and/or accurate, which allows use of the model score over the TTL before it is considered inaccurate or invalidated pending further computation of a new model score. As such, the decay score may indicate how quickly or how much further the TTL has decayed based on the missing data and/or the past model scores and feature data from the past model scores (e.g., where the older past model scores and/or feature data may be stale and less accurate). As such, the decay score may be associated with the feature pulled in and the model at an overall level based on the length of time the score is considered to be accurate, “live,” or useful for intelligent outputs and computing service use.

At step410, an IoT token for real-time IoT data that changes the TTL of the ML score is determined. An IT infrastructure and/or framework may be associated with a real-world physical location where sensors, devices, servers, and other components may be established to collect, detect, and/or monitor real-world data, which may be used to supplement or replace missing data for the ML model during determination of the model score. For example, where the ML model is associated with fraud, the IoT sensors and system may detect the user at a real-world location to associate the user with the location, a transaction, and/or whether the user is behaving as predicted or in line with the models score. For the real-world data, IoT tokens may be generated, which may be stored and/or accessible. As such, an IoT token for such data may be availed and used to increase or decrease the TTL and/or accuracy of the ML model based on whether the score aligns with such data.

At step412, an enriched ML model score token is generated for the ML score with the IoT token. The enriched ML model score token may correspond to the ML model score that has been enriched or adjusted by the entropy score, the decay score, and real-time data for the IoT token, which may further be associated with the missing data for the corresponding model feature. As such, the additional scores and data added and combined in the enriched model score from the decay and entropy scores with the real-time data and/or IoT token causes the model score to have further parameters allowing for it to be validated and accurate. The enriched model score may be tokenized and consumed by other processors and applications. As such, the enriched model score token allows for further accuracy. At step414, the request is processed using the enriched ML model score. With the added accuracy from enrichment and the additional data applied to the initially calculated model score, a corresponding computing system and/or application may utilize the score to provide more accuracy and/or reliability during computing service provision and use. As such, the request from the user may be processed with higher confidence that the model score is accurate and useful for decision-making, predicting, classifying, and the like.

Computer system500includes a bus502or other communication mechanism for communicating information data, signals, and information between various components of computer system500. Components include an input/output (I/O) component504that processes a user action, such as selecting keys from a keypad/keyboard, selecting one or more buttons, image, or links, and/or moving one or more images, etc., and sends a corresponding signal to bus502. I/O component504may also include an output component, such as a display511and a cursor control513(such as a keyboard, keypad, mouse, etc.). An optional audio input/output component505may also be included to allow a user to use voice for inputting information by converting audio signals. Audio I/O component505may allow the user to hear audio. A transceiver or network interface506transmits and receives signals between computer system500and other devices, such as another communication device, service device, or a service provider server via network150. In one embodiment, the transmission is wireless, although other transmission mediums and methods may also be suitable. One or more processors512, which can be a micro-controller, digital signal processor (DSP), or other processing component, processes these various signals, such as for display on computer system500or transmission to other devices via a communication link518. Processor(s)512may also control transmission of information, such as cookies or IP addresses, to other devices.