High-fidelity data management for cross domain analytics

Systems for providing high-fidelity data management for cross domain analytics may include multiple components. An access management function component may control access to data stored in a data store of a business domain by a user account associated with a search engine domain. A data management function component may select based on at least one of one or more data access privileges for the user account associated with the search engine domain or one or more privacy policies, a data view of multiple data views for viewing the data, and one or more data filters for application to the data. An external API manager component may store in the data store of the business domain cross correlation information that correlates a plurality of machine learning model identifiers of machine learning models of the search engine domain with one or more corresponding business APIs of the business domain.

BACKGROUND

Presently, analytics in the form of machine learning and cognitive networks are commonly used for predictive computation. In the case of search engines, machine learning models are used to send prioritized search results (called “surfacing results” in computer science parlance) to users and other requestors. In the case of businesses, data is collected to make predictions for operations. For example, in inventory management, machine learning models are used to predict when to replenish stock and to predict consumer demand.

Businesses have an interest in being surfaced by search engines and search engines conversely have an interest in providing search result consumers with accurate results. However, businesses store much metadata and contextual data that search engines do not have access to and search engine platforms provide much data and functionality that business stores do not necessarily take advantage of. Accordingly, there is a need to mediate information transfer, and in particular data transfer for the mutual benefit of businesses and search engine platforms alike.

DETAILED DESCRIPTION

This disclosure is directed to techniques for performing high-fidelity management of data that is analyzed by machine learning models in the context of businesses and search engines. Machine learning models, which for purposes of this disclosure include cognitive networks, are summaries of data sets called training data sets. Specifically, a training data set is comprised of data that is representative of a statistically significant number of observations of interest. The training data set is then placed into a series of neural networks that capture the statistical behavior of selected attributes of those observations, called a machine learning model. The machine learning model is much smaller than the training data set and can be used to predict new behavior based on new observations called input data. Specifically, the input data is received, and attribute values corresponding to attribute values in the machine learning model are extracted. Those attributes are then input into the machine learning model, and the value of predicted attributes are then output. Depending on the statistics used, confidence levels can also be calculated and output as well.

Consider an example where a statistically large number of pictures of dogs and cats are collected into a training data set, for purposes of creating a machine learning model to identify whether an image constituting new input data contains a dog or a cat. The training data set is input into a machine learning model generator and the machine learning model is created. The generator may have been programmed to look for specific attributes such as the shape of the snout, the distance between the eyes, and the like. When the generated machine learning model receives and scans a new input data picture, it extracts those attributes, and then the machine learning model generates a probability that a dog or a cat is in the picture. For example, the machine learning model may say that there is an 85% confidence that the input data image is that of a dog.

Businesses make use of analytic applications, also referred to as analytics, for similar reasons. Based on historical data, they make projections for inventory, customer volume, revenue, and the like. Similarly, search engines make use of analytics to improve the accuracy and relevance of search results. However, search engines suffer from not having the most accurate data available about businesses—that is the data owned by the businesses themselves.

By way of example, a search engine can tell whether a user clicked on a link, but the search engine cannot tell whether a customer in fact went to a store and made a purchase—that is information held by the business itself. Similarly, a business cannot tell the volume of interest in its offerings, but data from the search engine would provide this information.

Presently, search engines expose applications programming interfaces (“APIs”) to provide businesses information, and to provide the means of performing search engine optimization (“SEO”). However, presently, while some businesses expose business data via APIs, there is no automation to manage the potential to combine business-owned data and search engine-owned data.

A controller may be used to advantageously manage the combination of data from both the business and the search engine. The controller may include a rules engine, may be implemented as the rules engine, or may be substituted with the rules engine. In various embodiments, the data may include metadata, operational data, and/or contextual data, in which the data may be related to businesses and/or search engines. A first advantage is the management of privacy and security. The laws around privacy and security are constantly in flux and becoming increasingly restricted. As a result, not all business data may be exposed.

A second advantage of the controller is to ensure that machine learning models are developed that improve statistical confidence, rather than defeat the predictive value. The introduction of additional attributes might improve the accuracy of a machine learning model, but only if there is a bona fide correlation between the additional attributes and the predictions. In our dog and cat example earlier, it is possible to program the machine learning model to look at the background color of an image. Barring the unusual, there should be no correlation between the background color of an image and whether the image contains a dog or cat. Accordingly, introducing background color as an attribute would either provide a neutral predictive value or worse would degrade the predictive value of the machine learning model.

Accordingly, a controller can introduce and/or provide the logic necessary to ensure that data received from the search engine APIs and the data shared by the business via their APIs, improve the predictive value of machine learning models. When the data introduced to a machine learning model improves the predictive value, the data model is said to “positively converge” or simply to “converge”.

A third advantage is that a controller has the capability of monitoring what data is shared via the business APIs and quantifying the benefits of the improved machine learning models and data models in general to the business. Accordingly, if a controller exists as a third-party service, the controller may act as a billing and audit model to monetize the services provided by the controller. In this way, such a controller could be the basis for a platform for managing data between a business and a search engine.

Example Configuration for High-Fidelity Data Management

FIG.1is an example configuration100for implementing example embodiments of high-fidelity data management. In various embodiments, such high-fidelity data management may be performed in the context of domains, in which a domain includes the computing assets for an entity. Those assets may exist in the cloud, or outside of the cloud. A business domain102is the set of computing assets for an enterprise104. The enterprise104may have some computing assets dedicated to analytics106and data store108containing its operational data, metadata, etc. Generally, the data in data store108provides training data for data models used by analytics106. Search engine platforms are also businesses and accordingly have their own domains, such as the search engine domain110. Similarly, search engines may have their own analytics112and data stores114.

Because of the volume of data and the computations requirements for generating data models, it is typical that the analytics106and112and data stores108and114reside on the cloud116. The cloud116is generally a set of servers that are disaggregated by a hypervisor that serves up computing and storage on demand in the form of virtual machines. The cloud116is described in greater detail with respect toFIG.2.

Businesses, such as that encapsulated in the business domain102and the search engine domain110may expose their data and receive data through APIs. In the case of a business domain102that wishes to cooperate with a search engine domain110, the business domain may use a controller118. In various embodiments, the controller118may include a rules engine, be implemented as a rules engine, or be substituted with a rules engine. The controller118controls what data and functionality are exposed by controlling the behavior of the business APIs120. The controller118also controls what search engine APIs122are invoked. The controller118is described in further detail with respect toFIG.3. Accordingly, the controller118provides a fully functional two-way data interchange124, not just data exchange or functional invocation solely through the search engine APIs122, with the data interchange124conforming to the business logic of the enterprise104.

Example Environment for High-Fidelity Data Management

FIG.2is a block diagram of an example environment200that includes hardware, software, and communications environment for implementing high-fidelity data management. The functionalities for performing high-fidelity data management are generally hosted on computing devices. Example computing devices include without limitation on the client-side: mobile devices (including smartphones), tablet computers, laptops, desktop personal computers, on-board vehicle navigation devices, and kiosks. Example computing devices on the server-side include without limitation: mainframes, physical servers, and virtual machines. Generally, the computing devices are networked.

A client-side computing device, or client202, may have a processor204, a memory206. The processor may be a central processing unit, a repurposed graphical processing unit, and/or a dedicated controller such as a microcontroller. The computing device may further include an input/output (I/O) interface208, and/or a network interface210. The I/O interface208may be any controller card, such as a universal asynchronous receiver/transmitter (UART) used in conjunction with a standard I/O interface protocol such as RS-232 and/or Universal Serial Bus (USB). The network interface210, may potentially work in concert with the I/O interface208and may be a network interface card supporting Ethernet and/or Wi-Fi and/or any number of other physical and/or datalink protocols. Alternatively, the network interface210may be an interface to a cellular radio.

Memory206is any computer-readable media that may store several software components including an operating system212and software components214and/or other applications216including an internet browser or application integrating internet browsing capability. In general, a software component is a set of computer-executable instructions stored together as a discrete whole. Operating system212and applications216are themselves software components or integrated aggregations of software components. Examples of software components214include binary executables such as static libraries, dynamically linked libraries, and executable programs. Other examples of software components214include interpreted executables that are executed on a run time such as servlets, applets, p-Code binaries, and Java binaries. Software components214may run in kernel mode and/or user mode.

The server-side computing device, or server218, is any computing device that may participate in a network. The network may be, without limitation, a local area network (“LAN”), a virtual private network (“VPN”), a cellular network, or the Internet. The server218has hardware components analogous to the client-side computing device, i.e., client202. Specifically, it will include a processor220, a memory222, an input/output interface224, and/or a network interface226. The memory222may implement an operating system228, server-side software components230, and server-side applications232. Server218differs from client202in that processing power is generally more powerful to handle concurrent processes running and network capacity is greater to communicate with multiple clients202. Server-side software components230may include libraries and run-times (e.g., to run interpreted code). Server-side applications232may include not only web servers (also called “application servers”) and database servers, but also server software providing functionality not typically found in legacy system applications. Example server software may include transaction monitors, single sign-on servers, identity servers, security servers (including access control list (ACL) functionality) and network session managers (which enable multiple concurrent sessions on a legacy system application).

In general, high-fidelity data management may be implemented as a software service on a physical server218. However, such a software service may also be hosted on the cloud116via a cloud service236. Specifically, the cloud service236is comprised of multiple physical computer servers that are disaggregated via a hypervisor. The physical computer servers each may have one or more processors, memory, at least an I/O interface, and/or a network interface. The features and variations of the processors, memory, the I/O interface, and the network interface are substantially similar to those described for the physical server218described above.

A cloud service236includes a hypervisor that can delegate calls to any portion of hardware in the underlying physical serves, and upon request generates a virtual machine from the separate portions of hardware, regardless of a physical server (a process called “disaggregation”). Just as a physical server218, a virtual machine may host not only software applications, components including services, but also virtual web servers238functionality and virtual storage/data store240functionality.

The virtual machines themselves may be further partitioned into containers, which enable the execution of a program in an independent subset of the virtual machine. Software such as Kubernetes, Mesos, and Docker are examples of container management software. Unlike virtual machines which have a delay in startup due to the need for provisioning an entire OS, containers may be generated more quickly and on-demand since the underlying virtual machine is already provisioned.

The cloud service236may embody an abstraction of services. Common examples include service abstractions such as Platform as a Service (“PAAS”), Infrastructure as a Service (“IAAS”), and Software as a Service (“SAAS”).

A Controller for Data Management and Exchange

FIG.3is a block diagram300of an example controller118for controlling the data interchange124between business APIs120and search engine APIs122. The controller118manages several functions and sub-functions (herein “managers”, “data stores”, “ACLs”, “rules engines” and equivalents) comprised of software components implemented via computer-executable instructions and/or computer-readable data. Specifically, it has an access management function304, a data management function306, an external API manager function308, and an administration panel310for an administrator312. The functions304-308and the administration panel310may be implemented via routines, program instructions, objects, and/or data structures that are executed in the environment200to perform particular tasks or implement particular abstract data types.

The access management function304is responsible for determining access to particular APIs and access to data within the business domain102. It is comprised of a user account data store314to store accounts with access to data and/or APIs within the business domain102. In alternative instances, the access management function304may not perform key and account management directly. Instead, the access management function304may delegate such functions to other security infrastructure within the business domain102. The user account data store314at a minimum includes the identifiers for unique accounts to access APIs and/or data.

Access control lists indicate what privileges are associated with each specific account. Accordingly, the access management function304includes a data access control list316and an API access control list. The data access control list316specifies database and data access privileges with respect to data store108within the business domain102. These may be implemented as SQL granted privileges within a relational database management system or equivalents for the data store108within the business domain102. The API access control list318stores the privileges for user accounts in the user account data store314to access the business APIs120.

The data management function306determines the behavior of the business APIs120when accessed by a user account in user account data store314. Specifically, when a request to access data, generally via a business API120is received, the data management function306identifies the accessing user and associated privileges via the access management function304. Based on the received privileges, the view and filter manager320identifies a particular version of data views that may be accessed and additional data filters to apply. The view and filter manager320maps versions of views and filters to apply to user accounts.

In the case of managing views with respect to a relational database, a set of data may expose a limited set of fields by defining a SQL view. Some views may have a large number of fields, and other views may restrict fields. Depending on the account accessing a business API120, the view and filter manager320selects which version of the views is to be accessed.

Furthermore, in a relational database, a SQL view may have additional filters and conditions set on the data thereby limiting the number of data rows returned. Accordingly, depending on the account accessing a business API120, the view and filter manager320selects which filters to apply to the view.

In practice, the view and filter manager320, upon receiving an API request and identifying the accessing account, apply both view and filter management in combination before enabling a return of a result set via the API. In this way, the view and filter manager320can manage what fields and database records are shared from the same API120.

The view and filter manager320acts in concert with the privacy manager322. The privacy manager322interfaces with external automated privacy policies or may store automated privacy policies itself. Specifically, access to data and APIs via the data access control list316and the API access control list318, as well as the application/selection of views and filters in the view and filter manager320are changed upon the recommendation of the privacy manager322. The recommendations are surfaced to the administrator312via the administration panel310. The administrator312may select which changes to perform based on the privacy manager322recommendations. Alternatively, the administration panel310may be set to automatically accept all privacy manager322recommendations. In this way, privacy and cybersecurity remediation may be fully automated

The controller118also exists to ensure that machine learning models (e.g., data models, also referred to as “DM”) dependent on external APIs have data that converge to improving predictive capabilities. The DM-API dependency manager324of the external API manager function308may store a series of identifiers of internal machine learning models from the analytics106within the business domain102cross-correlated with external search engine APIs122. Similarly, the DM-API dependency manager324of the external API manager function308may also store identifiers machine learning models from the analytics112of the search engine domain110as well, where the search engine domain110has configured the sharing of machine learning model information, cross-correlated with the business APIs120. This allows the external API manager function308to track which machine learning models have dependencies on which APIs.

The data model history data store326stores the historical performance of data models. Specifically, it stores the predicted statistical confidence in the data models and, in some cases, the actual predictive performance of the data models. The rules engine328, reviews the history, and based on predetermined thresholds set by the administrator312via the administrator panel310, or based on machine learning models itself, makes recommendations for what changes to APIs access should be made. Specifically, the recommendations from the rules engine328may specify access to additional APIs, removal of APIs as not contributing to the predictive value of the model, or may recommend changes to views and filters applied. These results are presented to the administrator312via the administrative panel310. Upon approval of the administrator312, the access management function304and/or the data management functions306may perform all, or part of, the recommendations of the rules engine328. In some instances, the administrative panel310may be configured to automatically accept recommendations from the rules engine328. In this way, modifications of data and APIs may be fully automated.

Example Processes

FIGS.4and5present illustrative processes400and500for implementing the remote triggering of personal devices to perform actions in response to detected events. Each of the processes400and500is illustrated as a collection of blocks in a logical flow chart, which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions may include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order and/or in parallel to implement the process. For discussion purposes, the processes400and500are described with reference to the configuration100ofFIG.1and the environment200ofFIG.2.

FIGS.4aand4billustrate a flow diagram of an example process400used by a controller to manage data access to data stored in a data store of a business domain in accordance. At block402, the controller118may receive an access request that includes a user account identifier from an application on a computing device, in which the access request is requesting access to data stored in a data store of a business domain. At block404, the controller118may determine based on a data access control list of the business domain whether a user account with the user account identifier has access to the data stored in the data store of the business domain.

At decision block406, if the controller118determines that the user account with the user account identifier has access to the data (“yes” at decision block406), the process400may proceed to block408. At block408, the controller118may determine based on an API access control list of the business domain whether the user account with the user account identifier has access to one or more business APIs of the business domain that provides access to the data stored in the data store.

At decision block410, if the controller118determines based on the API access control list of the business domain that the user account with the user account identifier has access to one or more business APIs (“yes” at decision block410), the process400may proceed to block412. At block412, the controller118may select based on one or more data access privileges for the user account with the user account access identifier a data view of multiple views for viewing the data and one or more data filters for application to the data. In some embodiments, the data access privileges for the user account with the user account access identifier may be stored in the data access control list.

At block414, the controller118may determine based on one or more privacy policies whether to modify the data view selected for viewing. In various embodiments, the one or more privacy policies may be stored in an internal database of the controller118or an external database. At decision block416, if the controller118determines that the data view selected for viewing is to be modified (“yes” at decision block416), the process400may proceed to block418. At block418, controller118may select an alternative view of the multiple data views as specified by the one or more privacy policies for viewing the data. In various embodiments, the selection may be made automatically by the controller118or in response to an input made by an administrator via an administrative interface of the controller118.

At block420, the controller118may determine based on the one or more privacy policies whether to modify the one or more data filters for application to the data. At decision block422, if the controller118determines that the one or more data filters are to be modified (“yes” at decision block422), the process400may proceed to block424. At block424, the controller118may select one or more alternative data filters as specified by the one or more privacy policies for application to the data. In various embodiments, the selection may be made automatically by the controller118or in response to an input made by an administrator via an administrative interface of the controller118.

At block426, the controller118may apply the one or more selected data filters to the data stored in the data store of the business domain to generate filtered data. At block428, the controller118may provide the selected data view of the filtered data to the application on the computing device via the one or more business APIs. For example, the controller118may permit the application on the computing device to access the selected view of the filtered data via the one or more business APIs.

Returning to decision block406, if the controller118determines that the user account with the user account identifier does not have access to the data (“no” at decision block406), the process400may proceed to block430. At block430, the controller118may deny the application on the computing device access to the data stored in the data store of the business domain. Returning to decision block410, if the controller118determines based on the API access control list of the business domain that the user account with the user account identifier does not have access to one or more business APIs (“no” at decision block410), the process400may also proceed to block430.

FIGS.5aand5billustrate a flow diagram of an example process500for controlling the data interchange between business APIs and search engine APIs. At block502, the controller118may store a first plurality of machine learning model identifiers for first machine learning models of a business domain, such as the business domain102, in a data store of the business domain. At block504, the controller118may determine one or more corresponding search engine APIs of a plurality of search engine APIs used by each of the first machine learning models of the business domain to receive data from the search engine domain. For example, the controller118may make the determination based on information the controller118obtained from the analytics106.

At block506, the controller118may store in the data store cross correlation information that cross correlates each of the first plurality of the machine learning model identifiers of the first machine learning models with the one or more corresponding search engine APIs based on the determination.

At block508, the controller118may receive a second plurality of machine learning mode identifiers for second machine learning models of a search engine domain, such as the search engine domain110. In various embodiments, the controller118may receive such identifiers from the search domain. At block510, the controller118may receive additional cross correlation information that correlates the second plurality of machine learning model identifiers with one or more corresponding business APIs of the business domain in which each of the second machine learning models is configured to receive data from the business domain via the one or more corresponding business APIs. In various embodiments, the controller118may receive such information from the search domain. At block512, the controller118may store the second plurality of machine learning model identifiers and the additional cross correlation information in the data store.

At block514, the controller118may determine whether to configure a machine learning model of the business domain or the search engine domain to receive data from one or more newly cross correlated APIs or terminate receiving data from at least one currently cross correlated APIs. In various, the controller118may make such a determination based on performance data for the machine learning model, such as one or more of a historical performance of the machine learning model, a predicted statistical confidence in the machine learning model, or an actual predictive performance of the machine learning model.

At decision block516, if the machine learning model of the business domain or the search engine domain is to receive data from one or more newly cross correlated APIs (“yes” at decision block516), the process500may proceed to block518. At block518, the controller118may store information on a new cross correlation between a machine learning model identifier of the machine learning model in the business domain or the search engine domain and at least one API of a different domain of the business domain or the search engine domain when the machine learning model is newly configured to receive data from the at least one API. For example, the new cross correlation may be for a machine learning identifier of the machine learning model in the business domain to at least one API of the search engine domain, or vice versa.

However, if the machine learning model of the business domain or the search engine domain is not to receive data from one or more newly cross correlated APIs (“no” at decision block516), the process500may proceed to decision block520. At decision block520, if the machine learning model of the business domain or the search engine domain is to terminate receiving data from one or more currently cross correlated APIs (“yes” at decision block520), the process500may proceed to block522.

At block522, the controller118may delete information on an existing cross correlation between the machine learning model identifier of the machine learning model in the business domain or the search engine domain and one or more APIs of a different domain of the business domain or the search engine domain when the machine learning model is configured to terminate receiving data from the one or more APIs. For example, the termination may include terminating an existing cross correlation between a machine learning identifier of the machine learning model in the business domain and one or more APIs of the search engine domain, or vice versa.

Subsequently, the process500may loop back to block514. However, if the machine learning model of the business domain or the search engine domain is not to terminate receiving data from one or more currently cross correlated APIs (“no” at decision block520), the process500may also loop back to block514.

Example Use Cases for High-Fidelity Data Management

There are many use cases for high-fidelity data management as a platform. Depending on the use case, different monetization models may be applied.

Improved Surfacing of Stores

In one use case, present search engines are known to not provide accurate recommendations. For example, a person looking for a coffee shop may be directed to a coffee shop further away than other sources of coffee. This may occur because the search engine is identifying outlets that specialize in coffee solely as candidates for surfacing. This ignores alternative general sources of coffee such as convenience stores.

By providing search engines with data directly from businesses with a full enumeration of services, search engines can then surface not just specialty stores, but also general stores. Examples of data to surface include not only the location and hours of availability of stores, but also an enumeration of services e.g., car washes, fuel/diesel, electric vehicle charging, alcohol, food, and the like.

Machine Learning Model Evolution Matching Data Evolution

Businesses are constantly collecting and changing the collection of data. Many businesses, such as consumer stores have independent mobile applications for users to crowdsource information. Accordingly, as different sources of data are collected as training data, the rules engine328enables data models to improve over time. In this way, machine learning models evolve as data quality evolves.

The administrator312will be aware of changes in the data sources. For example, initial data sources may solely be as received from search engines via search engine APIs122. Then local business data stored in the data store108may be integrated into the training data. Finally, crowdsourced information as collected via mobile applications of users may further be integrated. The administrator312will be enabled by the rules engine328to determine which data to integrate and with which access privileges while ensuring positive data convergence.

White Labeling APIs

Because the controller118resides in the business domain102and is in full control of the enterprise104, additional business APIs120that provide an amalgam of search engine data as exposed through search engine APIs122may be provided. An example may be a business API120that exposes recommendations based on a machine learning model comprised of both search engine domain110and business domain102data.

In other instances, simply sharing data from both the search engine domain110and business domain102in amalgamated sets through a business API120may also be done.

In cases where it is permitted to do so, the business domain102may expose its own APIs that pass-through search engine domain data. This practice is called white-labeling APIs. In this way, a partner of the enterprise104may have a single location and source of APIs.

In some embodiments, the controller118functionality may be resold. The controller118may store a monitor that communicates with the administrative panel310. The administrative panel310may include an application interface that provides access to business APIs120, accesses to data in the data store108, and accepted improvements to machine learning models via the rules engine328. This data may further be stored in an independent data store called an access tracking data store (not shown).

With the data in the access tracking data store, a business may create monetization models to charge for the controller118functionality. The monetization models may include per access or alternatively subscription models. One of the key difficulties in monetization is in demonstrating value. Because the controller118stores improved performance of machine learning models via the data model history data store326, not only can value be demonstrated, but also monetization models can be charged based on the extent of improvement if so desired.

While the embodiments of the high-fidelity data management are described above with respect to a business domain and a search engine domain, alternative embodiments of such high-fidelity data management may be applied to data management between two domains of other types.

Conclusion