Metadata-based information provenance

Techniques for metadata-based information provenance are disclosed. A node in a data provisioning layer receives encrypted payload data to be delivered to a recipient. The node generates provenance metadata that describes at least one action taken by the node with respect to the encrypted payload data. The node transmits the encrypted payload data and the provenance metadata via the data provisioning layer toward the recipient.

BACKGROUND

When transmitting data over a network, the timeliness and accuracy of the transmission are typically important considerations. Various techniques also exist to help ensure that data is transmitted securely and to the intended recipient. However, a recipient (a.k.a. “consumer”) of the data is not always able to confirm that the data received is trustworthy, or determine the path that the data followed in transit from the producer to the consumer. If data is not trustworthy, the recipient may not be able to rely on the data to make important decisions (e.g., targeting decisions in combat situations at the tactical edge, business decisions, medical decisions, and/or other kinds of decisions that depend on the trustworthiness of private, sensitive, or otherwise mission-critical data).

Approaches described in this section have not necessarily been conceived and/or pursued prior to the filing of this application. Accordingly, unless otherwise indicated, approaches described in this section should not be construed as prior art.

TECHNICAL FIELD

The present disclosure relates generally to information provenance in networked computer systems.

SUMMARY

One or more embodiments provide robust provenance trails for payload data, by generating provenance metadata in a data provisioning layer that manages dissemination of the payload data. Provenance trails may describe, for example, data origins and operations performed by intermediate nodes. In addition, one or more embodiments assess the trustworthiness of payload data, based on provenance metadata. Thus, one or more embodiments help ensure that authoritative data sources are used and that the payload data supplied by those sources is trustworthy.

Provenance information and trustworthiness computations, based on provenance metadata as described herein, may be useful in many different contexts. For example, in combat situations, provenance information and trustworthiness computations may allow for mission-critical decisions based on trusted information. Provenance information and trustworthiness computations may also be useful in non-combat situations, such as business or medical settings dealing with the transmission of trade secrets, personally identifiable information, and/or other kinds of private and/or sensitive data. Provenance information may provide insights such as who originally produced the payload data, how the payload data was shaped or otherwise transformed in transit, and/or what path the payload data followed in transit.

In general, in one aspect, one or more non-transitory machine-readable media store instructions that, when executed by one or more processors, cause: a first node in a data provisioning layer receiving encrypted payload data to be delivered to a recipient; the first node generating first provenance metadata that describes at least one action taken by the first node with respect to the encrypted payload data; and the first node transmitting the encrypted payload data and the first provenance metadata via the data provisioning layer toward the recipient. The at least one action taken by the first node with respect to the encrypted payload data may include one or more of receiving the encrypted payload data and/or transforming decrypted contents of the encrypted payload data. The first node may not be authorized to decrypt the encrypted payload data, and generating the first provenance metadata may be performed without decrypting the encrypted payload data. The recipient may be physically located at a tactical edge of a military mission.

The one or more non-transitory machine-readable media may further store instructions that, when executed by one or more processors, cause: a second node in the data provisioning layer receiving the encrypted payload data and the first provenance metadata; and based at least on the first provenance metadata, the second node computing a trustworthiness metric associated with the encrypted payload data. Computing the trustworthiness metric may include applying at least one user-configurable trustworthiness rule to the first provenance metadata.

The one or more non-transitory machine-readable media may further store instructions that, when executed by one or more processors, cause: the first node encrypting the first provenance metadata, using a first encryption policy that is different from a second encryption policy used to encrypt the encrypted payload data.

The one or more non-transitory machine-readable media may further store instructions that, when executed by one or more processors, cause: receiving user input that defines at least part of a provenance metadata policy; and responsive to receiving the user input, configuring multiple nodes in the data provisioning layer to generate provenance metadata according to the provenance metadata policy.

The one or more non-transitory machine-readable media may further store instructions that, when executed by one or more processors, cause: aggregating the first provenance data with a set of provenance metadata associated with multiple payload transmissions; and based at least on the set of provenance metadata, generating a visual representation of one or more provenance paths through the data provisioning layer.

In general, in one aspect, a system includes: a data provisioning layer; and a first node in the data provisioning layer. The node is configured to perform operations including: receiving encrypted payload data to be delivered to a recipient, generating first provenance metadata that describes at least one action taken by the first node with respect to the encrypted payload data, and transmitting the encrypted payload data and the first provenance metadata via the data provisioning layer toward the recipient. The system may further include a second node in the data provisioning layer, configured to perform operations including: receiving the encrypted payload data and the first provenance metadata, and based at least on the first provenance metadata, computing a trustworthiness metric associated with the encrypted payload data.

In general, in one aspect, a method includes: a first node in a data provisioning layer receiving encrypted payload data to be delivered to a recipient; the first node generating first provenance metadata that describes at least one action taken by the first node with respect to the encrypted payload data; and the first node transmitting the encrypted payload data and the first provenance metadata via the data provisioning layer toward the recipient. The at least one action taken by the first node with respect to the encrypted payload data may include one or more of receiving the encrypted payload data and/or transforming decrypted contents of the encrypted payload data.

1. The method may further include: a second node in the data provisioning layer receiving the encrypted payload data and the first provenance metadata; and based at least on the first provenance metadata, the second node computing a trustworthiness metric associated with the encrypted payload data. Computing the trustworthiness metric may include applying at least one user-configurable trustworthiness rule to the first provenance metadata. The first node may not be authorized to decrypt the encrypted payload data, and generating the first provenance metadata may be performed without decrypting the encrypted payload data. The recipient may be physically located at a tactical edge of a military mission.

The method may further include: the first node encrypting the first provenance metadata, using a first encryption policy that is different from a second encryption policy used to encrypt the encrypted payload data.

The method may further include: receiving user input that defines at least part of a provenance metadata policy; and responsive to receiving the user input, configuring multiple nodes in the data provisioning layer to generate provenance metadata according to the provenance metadata policy.

The method may further include: aggregating the first provenance data with a set of provenance metadata associated with multiple payload transmissions; and based at least on the set of provenance metadata, generating a visual representation of one or more provenance paths through the data provisioning layer.

DETAILED DESCRIPTION

FIGS. 1A-1Bare block diagrams of an example of a system100according to an embodiment. In an embodiment, the system100may include more or fewer components than the components illustrated inFIGS. 1A-1B. The components illustrated inFIGS. 1A-1Bmay be local to or remote from each other. The components illustrated inFIGS. 1A-1Bmay be implemented in software and/or hardware. Each component may be distributed over multiple applications and/or machines. Multiple components may be combined into one application and/or machine. Operations described with respect to one component may instead be performed by another component.

In an embodiment, a producer102is configured to transmit payload data (e.g., image data, text data, video data, and/or any other kind of payload data or combination thereof) to a consumer110, across a network122. A data provisioning layer104refers to hardware and/or software that provides an abstraction over the network122, where the network122provides the physical infrastructure and basic networking protocols necessary to transmit data between devices. Specifically, the data provisioning layer104is configured to manage the receipt and dissemination of payload data from the producer102to the consumer110, via the network122. The data provisioning layer104may provide a publish/subscribe interface, whereby the consumer110subscribes to payload data produced by the producer102. The producer102publishes payload data to the data provisioning layer104, which disseminates the payload data across the nodes106,108. Dissemination within the data provisioning layer104may also use a publish/subscribe model between nodes. Nodes with in the data provisioning layer104may be heterogenous, meaning that they are different kinds of devices that communicate using different waveforms. Any combination of heterogenous devices may form a data provisioning layer104, as long as each node supports the necessary protocols to communicate with each other (e.g., to interact with the publish/subscribe interface, to enforce security policies, and/or to generate provenance metadata as needed).

Based on the consumer110's subscription to payload data from the producer102, the consumer110obtains the payload data from the data provisioning layer104(e.g., from node1080). The data provisioning layer104may provide payload data to the consumer110using a push model. For example, node108may notify the consumer110when the subscription has new payload data available. Alternatively, the data provisioning layer104may provide payload data to the consumer110using a pull/polling model. For example, the consumer110may periodically or intermittently query the data provisioning layer104to determine whether the subscription has new payload data available. The data provisioning layer104may use Robust Information Gateway (RIG) technology developed under the Defense Advanced Research Projects Agency (DARPA) Dynamic Network Adaptation for Mission Optimization (DyNAMO) program. Alternatively, the data provisioning layer104may use other technology to receive and disseminate payload data.

In an embodiment, the data provisioning layer104is configured to perform operations described herein for metadata-based information provenance. Specifically, as payload data passes from the producer102to the consumer110, one or more nodes106,108in the data provisioning layer104may be configured to generate provenance metadata that describes one or more actions taken, by the node(s), with respect to the payload data. Examples of operations for metadata-based information provenance, and examples of applications thereof, are described below. Provenance metadata may be expressed as rich metadata that encodes information such as information types and information attributes. For example, provenance metadata may include Resource Description Framework (RDF) literals. In addition, the provenance metadata may be generated using a provenance metadata ontology. For example, provenance metadata may serialized as PROV Ontology (PROV-O) models or using another ontology. As described in further detail below, the provenance data may not be encoded in the payload data itself.

A node in the data provisioning layer104may include various components and/or subsystems. As illustrated inFIG. 1B, a node106may include one or more external interfaces112. An external interface112refers to a set of hardware and/or software that one or more components external to the node106may use to access data and/or functions within the node106.

The external interface(s)112may include an application programming interface (API) that allows access by applications executing outside of the node106(e.g., an application executing at the producer102) to data and/or functions of the node106. The API may operate behind an additional abstraction layer, such as the Open Mission Systems Abstract Service Bus (OMS ASB). Alternatively or additionally, the API may support interactions with a cross-domain solution (CDS). The API may allow an application to continue using the data provisioning layer104, without any changes to the application's code, even when functionality within the data provisioning layer104is changed. For example, metadata-based information provenance features may be programmatically added to a data provisioning layer104, without requiring any changes to the code of an external application that uses the data provisioning layer to publish/subscribe to payload data.

In an embodiment, an application mediation subsystem114is configured to mediate communication between external applications (e.g., requests received via an API as described above) and internal components of the node106. For example, the application mediation subsystem114may include one or more adapters such as OMS Critical Abstraction Layer (CAL) and/or another kind of adapter. The application mediation subsystem114may include a cross-domain solution (CDS) agent configured to communicate with a CDS. The application mediation subsystem114may include one or more parsers (e.g., an extensible markup language (XML) parser and/or another kind of parser) configured to parse data received from external sources. The application mediation subsystem114may include one or more mappers configured to map API requests to internal functions of the node106. For example, the application mediation subsystem114may include an OMS Universal Command and Control Interface (UCI) mapper configured to map UCI requests to internal functions of the node106.

The external interface(s)112may include a monitoring interface. The monitoring interface provides external access to monitoring functions of the node106, such as status information and/or functions associated with operation and configuration of the data provisioning layer104and/or the node106. For example, the monitoring interface may provide access to one or more functions for configuring metadata-based information provenance (e.g., enabling, disabling, and/or changing parameters of one or more provenance functions, such as whether to generate provenance metadata when disseminating payload data within the data provisioning layer104). The monitoring interface may provide access to functions that allow an external component to monitor operation of the data provisioning layer104and/or the node106. For example, the monitoring interface may provide access to one or more functions that allow for visualization of provenance flows through the data provisioning layer104.

The external interface(s)112may include a mission interface. The mission interface provides external access to load a mission template into the node106. The mission template specifies requirements (e.g., Information Exchange Requirements (IER's) and/or another kind of requirement or combination thereof) associated with a military or other kind of objective serviced by the data provisioning layer. For example, the mission template may indicate one or more publish-subscribe relationships and/or one or more service levels for information exchanges (e.g., a minimum service level, an acceptable service level, and/or a maximum service level). The mission interface may further allow a mission template that has already been loaded to be modified or replaced.

The external interface(s)112may include a security interface. The security interface provides external access to one or more functions of a security subsystem116. The security subsystem116is configured to protect the security of data handled by the node106. For example, the security system116may provide identity management, public and/or private key management, security policy management, cryptography, and/or other security features.

In an embodiment, the security subsystem116is configured to use attribute-based encryption (ABE) to secure payload data and/or provenance metadata. Alternatively or additionally, the security subsystem116may use one or more other kinds of security policy. In general, ABE does not require end-to-end sessions or bi-directional exchange of information, making it well suited for securing data in situations where (a) sessions may be difficult to establish and secure, and (b) bi-directional data exchanges may introduce security risks. ABE includes role and content-based cryptographic and granular access control. In role-based access control, attributes describe roles. In content-based access control, attributes describe the content. ABE thus allows for granular policies and high flexibility during deployment. In addition, ABE includes collusion resistance and resilience, because principals cannot collude without combining their keys. Compromise of any number of attribute keys does not further compromise the system. Moreover, ABE includes autonomous authority operations. Each authority has full control over its identity, attribute, and key management services, with no requirement for centralized trust. Autonomous authority operations may be important, for example, in multi-authority (e.g., coalition) environments with varying levels of trust among participants. The security subsystem116may use different security policies to secure payload data and provenance metadata, or may use different security policies for each. Because generating provenance metadata, as described herein, does not require a node to decrypt the payload data, a security policy such as ABE allows the data provisioning layer104to protect the payload data while allowing for the generation of a provenance trail even for data that is encrypted.

One or more of the external interfaces112may be provided by a hypertext transfer protocol (HTTP) web server (not shown) operating in the node106. The web server may be configured to receive requests (e.g., HTTP GET and/or HTTP POST requests) and supply data responsive to the requests. For example, the web server may be configured to serve web pages that include provenance flow visualizations. Alternatively or additionally, one or more of the external interfaces112may use a representational state transfer (REST) protocol and/or may supply data in JavaScript Object Notation (JSON) format. One or more external interface(s)112may use different communication protocols.

The node106may be configured to present data and/or functions, via one or more external interfaces112, in one or more user interfaces124. A user interface124renders user interface elements and receives input via user interface elements. A user interface124may be a graphical user interface (GUI), a command line interface (CLI), a haptic interface, a voice command interface, and/or any other kind of interface or combination thereof. Examples of user interface elements include checkboxes, radio buttons, dropdown lists, list boxes, buttons, toggles, text fields, date and time selectors, command lines, sliders, pages, and forms.

In an embodiment, different components of a user interface124are specified in different languages. The behavior of user interface elements may be specified in a dynamic programming language, such as JavaScript. The content of user interface elements may be specified in a markup language, such as hypertext markup language (HTML), Extensible Markup Language (XML), or XML User Interface Language (XUL). The layout of user interface elements may be specified in a style sheet language, such as Cascading Style Sheets (CSS). Alternatively or additionally, aspects of a user interface124may be specified in one or more other languages, such as Java, Python, Perl, C, C++, and/or any other language or combination thereof.

As noted above, the data provisioning layer104may be an abstraction over a network122. The external interface(s)112may include a network interface configured to communicate with the network122. A data transfer subsystem118may be configured to use the network interface to access the network122, in order to send and receive data, provision data flows, communicate with other nodes in a global namespace, and/or perform other network-related functions. For example, the data transfer subsystem118may be configured to match published data with subscriptions by other nodes and/or consumers. Alternatively or additionally, the data transfer subsystem118may be configured to use the network interface to obtain quality of service (QoS) and/or reachability information about paths and destinations within the data provisioning layer104, to help facilitate the dissemination of payload data from producers to consumers. In an embodiment, the abstraction provided by the network interface allows for cross-waveform communication and/or QoS routing in the data provisioning layer104, even across heterogeneous devices.

In an embodiment, the node106includes a data management subsystem120. The data management subsystem120is configured to manage the contents (e.g., payload data) of messages flowing through the node106. For example, the data management subsystem120may be configured to perform one or more data shaping operations. A data shaping operation modifies (i.e., “shapes”) data into a format that is different than the format in which the data was received. Data shaping may compress, segment, or otherwise modify data to account for network congestion. For example, data shaping may downsample or otherwise transform an image to consume less data, allowing for faster transmission and/or reducing network congestion. Alternatively or additionally, the data management subsystem120may be configured to prioritize messages to account for message urgency (e.g., depending on mission priority), quality of service (QoS) policies, etc.

FIGS. 2A-2Bare a flow diagram of an example of operations for metadata-based information provenance according to an embodiment. One or more operations illustrated inFIGS. 2A-2Bmay be modified, rearranged, or omitted all together. Accordingly, the particular sequence of operations illustrated inFIGS. 2A-2Bshould not be construed as limiting the scope of one or more embodiments.

In an embodiment, a system (e.g., one or more components of the system100ofFIGS. 1A-1B) configures a provenance metadata policy (Operation202). Configuring a provenance metadata policy determines, for a set of possible actions that a node may take with respect to payload data, which action(s) should trigger the generation of provenance metadata. A provenance metadata policy may be partly or wholly user-configurable. Configuring the provenance metadata policy may enable or disable provenance metadata generation for one or more of the following actions: receiving or “ingesting” payload data from a producer; disseminating payload data between nodes in the data provisioning layer; shaping or otherwise modifying payload data; transmitting payload data to a consumer; generating a trustworthiness metric; and/or another action or combination thereof. For example, in some cases, disabling provenance metadata for dissemination between nodes in the data provisioning layer may help avoid network congestion, particularly if network resources are scarce or timeliness is of the essence. On the other hand, less provenance metadata may limit a recipient's ability to assess the trustworthiness of the payload data. Therefore, the specific provenance metadata policy may reflect a subjective compromise between performance and security. In some cases, provenance metadata is always generated upon ingestion and shaping, while provenance metadata for intra-layer dissemination may be user-configurable.

In an embodiment, the system configures a trustworthiness computation rule (Operation204). A trustworthiness computation rule supplies the logic for computing a trustworthiness metric. One or more elements of the trustworthiness computation rule may be user-configurable. For example, a trustworthiness metric may depend on how many hops the payload data took between the producer and the consumer, how many times the payload data was shaped or otherwise transformed in transit, whether the payload data passed through a cross-domain solution (CDS), and/or another factor or combination thereof. A trustworthiness metric may be a numerical value (e.g., a percentage or value on a predetermined scale). For example, the trustworthiness metric may be a scalar value in a range from 0 to 1, where 1 is highly trustworthy and 0 is not trustworthy. For a numerical value, the trustworthiness computation rule may assign weights W1. . . WN (which may be user-configurable) to each factor f1. . . fN (e.g., one or more factors in Table 1). The weights may be normalized to provide a trustworthiness metric in a consistent range (e.g., 0 to 1). The trustworthiness computation rule may use those weights to compute an aggregate function T(F) that represents the trustworthiness of the payload data:
T(F)=W1*f1+W2*f2+ . . .

Alternatively, a trustworthiness metric may be a categorical value that assigns payload data to a categorical “bucket.” For example, the trustworthiness metric may have possible values of high, medium, or low. The trustworthiness computation rule may map combinations of factors (e.g., one or more factors in Table 1) to the different categories. Alternatively, the trustworthiness metric may be any other kind of value that provides a consistent, comparable metric for assessing the trustworthiness of payload data.

In an embodiment, a trustworthiness computation rule is based on one or more factors (some or all of which may a user may be able to enable, disable, or otherwise configure) that relate to the confidentiality, staleness, accuracy, integrity, and/or one or more other qualities of the payload data. Table 1 illustrates some examples of factors that may contribute to a trustworthiness computation rule:

TABLE 1Trustworthiness FactorsFactorMeasureTypeRationalePayload data traversed a CDS?Yes/NoIntegrityOperations taken by a CDS (e.g.,sanitization) may represent a protocol break.Number of hops0 . . . NIntegrityThe risk of corruption increases as data issent over each node in an untrusted network.Payload data is ABE- encrypted?Yes/NoConfidentialityIf data is not ABE-encrypted, it may be readby adversaries and therefore may not betrustworthy for decision-making.Payload data is signed?Yes/NoIntegrityIf data is not signed, it may be corrupted byan adversary.Payload data is timestamped?Yes/NoStalenessData that is not timestamped may be staleand therefore unreliable for decision-making.Payload data has been shapedYes/NoIntegrity andShaping means the data can be corrupted (ifAccuracythe shaping node is compromised) orartifacts may be introduced by a benignshaping algorithm, which may trigger issuesin applications (boundary cases)Geocoordinates have been mappedYes/NoAccuracyMapping one format of geocoordinates tofrom one format to another?another format may introduce fidelity issuesdue to rounding, etc.Authoritative data source?Yes/NoAccuracyData from an unknown/untrusted datasource may not be trustworthy.

As mentioned above, different factors may be assigned different weights to compute a trustworthiness metric. Table 2 illustrates an example of weights assigned to the factors listed in Table 1:

TABLE 2Example of Weighted Trustworthiness FactorsNormalizedFactorMeasureValueWeightPayload data traversed a CDS?Yes/No1 or 00.05Number of hops0 . . . NN/X, where0.05X is themaximumtrustednumberof hopsPayload data is ABE- encrypted?Yes/No1 or 00.1Payload data is signed?Yes/No1 or 00.2Payload data is timestamped?Yes/No1 or 00.1Payload data has been shapedYes/No1 or 00.2Geocoordinates have been mappedYes/No1 or 00.1from one format to another?Authoritative data source?Yes/No1 or 00.2

In some cases, a trustworthiness computation rule may be conceptualized as a symmetrical decision tree, where each enabled factor is evaluated in the same order, regardless of what value (e.g., yes/no or I/O) was determined for the preceding factor. Alternatively, a trustworthiness computation rule may be an asymmetrical decision tree in which certain factors definitively indicate that payload data is trustworthy or untrustworthy, regardless of the other factors. For example, in some cases, payload data that does not come from an authoritative data source can never be considered trustworthy; accordingly, that factor may be placed at the “top” of the decision tree for computational efficiency.

Alternatively or additionally, the system may use machine learning to train a machine learning model based on a labeled set of training data. For example, the training data may include multiple sets of payload data and associated provenance metadata. Each set may labeled with a supervisory signal indicating a trustworthiness metric associated with the payload data. The system may use the machine learning model to assess the trustworthiness of new payload data, based on provenance metadata generated by the system. In practice, some applications will not have enough training data available to rely on machine learning.

In an embodiment, a node in a data provisioning layer receives encrypted payload data (Operation206). The node may receive the encrypted payload data from a producer, another node in the data provisioning layer, a cross-domain solution (CDS), or another source. If the node is receiving the payload data from a source outside the data provisioning layer (i.e., ingesting the payload data into the data provisioning layer), the payload data may be unencrypted as received, and the node may encrypt the payload data before disseminating the encrypted payload data. The node may also obtain and store metadata that describes the source of the payload data.

In an embodiment, the node determines whether data shaping is enabled (Operation208). Data shaping may be enabled on an “always on” basis, i.e., to be performed for all data flowing through the node. Alternatively, data shaping may be enabled on an “as-needed” basis, depending on one or more factors such as network congestion, message priority, quality of service (QoS) agreements, etc. If data shaping is enabled, the node may determine whether it is authorized to decrypt the payload data (Operation210). If the node is not authorized to decrypt the payload data, then the node may be unable to shape the payload data. Alternatively, one or more shaping operations, such as segmenting the payload data into multiple messages, may still be possible. If the node is authorized to decrypt the payload data, or if decryption is not required to shape the payload data, then the shapes the payload data (Operation212). For example, the node may compress, segment, or otherwise modify the payload data.

In an embodiment, the node determines whether provenance metadata is enabled (Operation214). To determine whether provenance metadata is enabled, the node may reference a provenance metadata policy. If the provenance metadata policy indicates that provenance metadata is enabled for an action to be taken by the node with respect to the payload data, then the node generates provenance metadata (Operation216). The provenance metadata describes the action(s) that were taken by the node with respect to the payload data and for which provenance metadata is enabled.

In an embodiment, the provenance metadata includes information that uniquely associates it with the encrypted payload data. For example, a node that ingests the payload data into the data provisioning layer may generate a global unique identifier (GUID) for the payload data. To allow for associating provenance metadata with encrypted payload data without needing to decrypt the encrypted payload data, the GUID may be stored in either an unencrypted format or an encrypted format that is accessible to other nodes in the data provisioning layer. The GUID may be disseminated along with the encrypted payload data. When a node generates provenance metadata associated with the encrypted payload data, the node may store the GUID in the provenance metadata, to logically associate the provenance metadata with the encrypted payload data.

In an embodiment, the node encrypts the provenance metadata (Operation218). The node may encrypt the provenance metadata using the same security policy (e.g., the same attribute-based encryption (ABE) policy or other kind of security policy) that was used to encrypt the payload data. Alternatively, the node may use a different security policy to encrypt the provenance metadata.

In an embodiment, the node determines whether it is the final node before the payload data reaches the consumer (Operation220). If the node is the final node, then the node may compute a trustworthiness metric (Operation222) based on the provenance metadata associated with the payload data. The node delivers the encrypted payload data and the trustworthiness metric to the consumer (Operation224). In this manner, the data provisioning layer may provide the trustworthiness computation as a service to the consumer. Alternatively or additionally, the node may act as a filter for trustworthy payload data. Specifically, the node may evaluate the trustworthiness metric (e.g., by comparing it with a threshold metric that indicates a minimum acceptable level of trustworthiness) and only transmit the payload data to the consumer if the threshold is satisfied. Alternatively or additionally, the node may deliver the encrypted payload data along with the provenance metadata associated with the payload data. The consumer may itself compute a trustworthiness metric based on the provenance metadata.

In some cases, the node may compute updated trustworthiness metrics as additional provenance metadata becomes available (for example, if the data provisioning layer disseminates provenance metadata and payload data in separate messages, which may not arrive at the node simultaneously). The node may compute and provide a trustworthiness metric to the consumer even after the payload data has already been provided to the consumer. The node may release payload data to the consumer even if insufficient provenance metadata is available to compute an initial trustworthiness metric. Alternatively or additionally, in response to receiving additional provenance metadata, the node may compute and provide an updated version of a previously computed and provided trustworthiness metric. When the node provides a trustworthiness metric to the consumer, the node may include a confidence metric that indicates a level of confidence (e.g., high/medium/low, or a value on a numerical scale such as 0 to 1) in the trustworthiness metric. For example, confidence in the trustworthiness metric may be relatively low if the trustworthiness metric is based on a small or incomplete set of provenance metadata, and higher if the trustworthiness metric is based on a large or complete set of provenance metadata.

In an embodiment, if the node is not the final node (i.e., as determined in Operation220), then the node forwards the encrypted payload data and any provenance metadata associated with the payload data (i.e., provenance metadata generated by the node and/or one or more previous nodes that handled the payload data) via the data provisioning layer (Operation226). Another node in the data provisioning layer may receive encrypted payload data and provenance metadata and perform one or more operations described above to continue advancing the message(s) across the network toward the consumer.

In an embodiment, the system aggregates provenance metadata and/or trustworthiness metrics (Operation228) generated over multiple payload data transmissions. For example, the system may store provenance metadata and/or trustworthiness metrics in a database that supports querying across multiple metadata records. Aggregated provenance metadata and/or trustworthiness metrics may be used to generate additional metrics, analyze and/or visualize provenance flows, and/or generate other kinds of insights regarding data dissemination through the data provisioning layer. For example, trustworthiness metrics over time may indicate a trend. New payload data not conforming to the trend may be considered an outlier, and may thus be considered less trustworthy. Such determinations may use statistical analysis (e.g., establishing a trend line and determining whether a new metric falls excessively far from the trend line) or model-based analysis (e.g., modeling data transmission speeds between nodes and determining whether the reported speed of transmission of a particular payload data conforms to the model). In an embodiment, linear regression is used to identify outliers. For each data point in the aggregated data, the system calculates the distance from a fitted line. The system may store information in a stack data structure, remove the top value from the stack, and calculate outlier influence with Cooke's formula:

Di=∑j=1n⁢⁢(Yj^-Y^j⁡(i))2(p+1)⁢σ^2
Alternatively, another technique may be used to identify outliers.

Detailed examples are described below for purposes of clarity. Components and/or operations described below should be understood as examples that may not be applicable to one or more embodiments. Accordingly, components and/or operations described below should not be construed as limiting the scope of one or more embodiments.

FIGS. 3A-3Care block diagrams of examples of messages according to an embodiment.FIG. 3Aillustrates a message300that includes encrypted payload data302, without any provenance metadata. The encrypted payload data302includes the payload304itself, and may also include payload metadata306. Payload metadata306is metadata that describes the payload304. For example, the payload metadata306may include information such as a timestamp indicating when the payload304was created, a timestamp indicating when the payload304was last modified, an author of the payload304, a data size of the payload304, and/or another type of metadata or combination thereof.

When a node generates provenance metadata, the provenance metadata may be included as part of the same message as the payload data, or may be part of a separate message. InFIG. 3B, encrypted provenance data308is bundled into the same message300as the encrypted payload data302. In this example, the encrypted payload data302and encrypted provenance metadata308are disseminated simultaneously, as part of the same message300. InFIG. 3C, the encrypted provenance metadata308is bundled in a separate message310from the encrypted payload data302. In this example, the two messages300,310may be disseminated at the same time or asynchronously. Thus, a node or consumer receiving the message300with the encrypted payload data302may have the option of processing (e.g., decrypting, disseminating, and/or otherwise processing) the encrypted payload data302even if the message310with the encrypted provenance metadata308has not been received. This approach may be useful, for example, if the encrypted payload data302is urgent or otherwise time-sensitive, and where bundling the encrypted provenance metadata308in the message300may compromise the timeliness of the message300(for example, in conditions where network congestion makes it easier to transmit smaller messages). Separating the messages300,310may also be useful in situations where nodes and/or consumers do not make trustworthiness determinations based on provenance metadata, but use provenance metadata to asynchronously analyze provenance flows and/or other network properties.

FIG. 4illustrates an example of metadata-based information provenance according to an embodiment. In this example, a producer400publishes payload data412to a data provisioning layer402. A node404receives the payload data (i.e., ingests the payload data412into the data provisioning layer402) and generates provenance metadata414indicating that the node404received the payload data412. The provenance metadata414may include metadata that identifies the node404, a timestamp when the payload data412was ingested, and/or another kind of metadata.

Node406in the data provisioning layer402receives the payload data412and the provenance metadata414from node404(either in a single message or separate messages, as described above). In this example, node406shapes the payload data412by generating a compressed version416of the payload data. Node406generates provenance metadata418that describes the shaping action. In this example, provenance metadata for dissemination between nodes of the data provisioning layer402is disabled, so the node406does not generate any provenance metadata to indicate receipt of the payload data412.

Node408in the data provisioning layer402receives the original payload data412, the provenance metadata414generated by node404, the compressed payload data416, and the provenance metadata418generated by node406. As described above, node408may receive some or all of these items asynchronously. Because metadata for dissemination between nodes of the data provisioning layer402is disabled, node408does not generate any provenance metadata to indicate receipt of the payload data412or the compressed payload data416.

In this example, node408is the final node in the data provisioning layer402before the payload data412reaches a consumer410. Based on the provenance metadata414generated by node404and the provenance metadata generated by node406, node408computes a trustworthiness metric420. The consumer410receives the payload data412, the compressed payload data416, and the trustworthiness metric420from node408. Alternatively, the consumer410may receive the provenance metadata414,418and compute a trustworthiness metric based on the provenance metadata414,418. As described above, the consumer410may receive some or all of these items asynchronously.

FIG. 5is a block diagram of an example of a system according to an embodiment. Specifically,FIG. 5illustrates an example of a system100in which a producer510and a consumer508are separated by a cross-domain solution504. To disseminate payload data from the producer510to the consumer508, a data provisioning layer may be logically separated into two or more “enclaves” of nodes: data provisioning layer enclave502and data provisioning layer enclave506. Each of the enclaves502,506may include one or more nodes configured to operate as described above. The enclaves502,506may be part of a single data provisioning layer operated by the same entity (e.g., a company, military organization, or other entity), or may be operated by separate entities. When a node in data provisioning layer enclave506receives payload data from the cross-domain solution504, the node may be configured to generate different kinds of provenance metadata, depending on how much information the cross-domain solution504reports. For example, if the cross-domain solution504reports that it shaped or otherwise transformed the payload data in some way, the node may generate provenance metadata that describes the action(s) taken by the cross-domain solution504. Alternatively, if the cross-domain solution504operates as a “black box” and does not report its actions, the node may generate provenance metadata that indicates ingestion of the payload data into the data provisioning layer enclave506. As noted above, the provenance metadata generated may depend on a provenance metadata policy.

FIG. 6illustrates an example of metadata-based information provenance at the tactical edge according to an embodiment. A data gatherer602is configured to gather mission-critical data. In this example, the data gatherer is an imaging device configured to gather payload data that includes target imagery with geo-locating metadata. The data gatherer602publishes the payload data to a data provisioning layer604. In this example, the data provisioning layer604is implemented across aircraft606,608that are configured to communicate with each other (e.g., via line-of-site and/or beyond-line-of-sight waveforms). Aircraft606ingests the payload and generates provenance metadata describing the ingestion. Because the payload data is mission-critical, aircraft606prioritizes dissemination of the imagery and provenance metadata to aircraft608within the data provisioning layer604. Aircraft608computes a trustworthiness metric based on the provenance metadata. Aircraft608supplies the imagery and the trustworthiness metric to a strike asset610(e.g., an airborne or ground asset, which may be manned or unmanned) at the tactical edge. The payload data may also be used to securely task a strike asset610with the capacity (e.g., line of sight, appropriate ordnance, etc.) to strike the target612. The strike asset610compares the trustworthiness metric with a threshold trustworthiness criterion, to determine whether the imagery can be trusted. Alternatively, aircraft608may compare the trustworthiness metric with a threshold trustworthiness criterion; and if the trustworthiness metric indicates that the imagery cannot be trusted, aircraft608may refrain from delivering the imagery to the strike asset610. If the imagery can be trusted, then the strike asset610uses the imagery to locate and strike the target612.

In an embodiment, a system includes one or more devices, including one or more hardware processors, that are configured to perform any of the operations described herein and/or recited in any of the claims.

In an embodiment, one or more non-transitory computer-readable storage media store instructions that, when executed by one or more hardware processors, cause performance of any of the operations described herein and/or recited in any of the claims.

Any combination of the features and functionalities described herein may be used in accordance with an embodiment. In the foregoing specification, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense. The sole and exclusive indicator of the scope of the invention, and what is intended by the Applicant to be the scope of the invention, is the literal and equivalent scope of the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction.

In an embodiment, techniques described herein are implemented by one or more special-purpose computing devices (i.e., computing devices specially configured to perform certain functionality). The special-purpose computing device(s) may be hard-wired to perform the techniques and/or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and/or network processing units (NPUs) that are persistently programmed to perform the techniques. Alternatively or additionally, a computing device may include one or more general-purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, and/or other storage. Alternatively or additionally, a special-purpose computing device may combine custom hard-wired logic, ASICs, FPGAs, or NPUs with custom programming to accomplish the techniques. A special-purpose computing device may include a desktop computer system, portable computer system, handheld device, networking device, and/or any other device(s) incorporating hard-wired and/or program logic to implement the techniques.

For example,FIG. 7is a block diagram of an example of a computer system700according to an embodiment. Computer system700includes a bus702or other communication mechanism for communicating information, and a hardware processor704coupled with the bus702for processing information. Hardware processor704may be a general-purpose microprocessor.

Computer system700further includes a read only memory (ROM)708or other static storage device coupled to bus702for storing static information and instructions for processor704. A storage device710, such as a magnetic disk or optical disk, is provided and coupled to bus702for storing information and instructions.

Computer system700may be coupled via bus702to a display712, such as a liquid crystal display (LCD), plasma display, electronic ink display, cathode ray tube (CRT) monitor, or any other kind of device for displaying information to a computer user. An input device714, including alphanumeric and other keys, may be coupled to bus702for communicating information and command selections to processor704. Alternatively or additionally, computer system700may receive user input via a cursor control716, such as a mouse, a trackball, a trackpad, or cursor direction keys for communicating direction information and command selections to processor704and for controlling cursor movement on display712. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane. Alternatively or additionally, computer system7may include a touchscreen. Display712may be configured to receive user input via one or more pressure-sensitive sensors, multi-touch sensors, and/or gesture sensors. Alternatively or additionally, computer system700may receive user input via a microphone, video camera, and/or some other kind of user input device (not shown).

Computer system700may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware, and/or program logic which in combination with other components of computer system700causes or programs computer system700to be a special-purpose machine. According to one embodiment, the techniques herein are performed by computer system700in response to processor704executing one or more sequences of one or more instructions contained in main memory706. Such instructions may be read into main memory706from another storage medium, such as storage device710. Execution of the sequences of instructions contained in main memory706causes processor704to perform the process steps described herein. Alternatively or additionally, hard-wired circuitry may be used in place of or in combination with software instructions.

The term “storage media” as used herein refers to one or more non-transitory media storing data and/or instructions that cause a machine to operate in a specific fashion. Such storage media may comprise non-volatile media and/or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device710. Volatile media includes dynamic memory, such as main memory706. Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic tape or other magnetic data storage medium, a CD-ROM or any other optical data storage medium, any physical medium with patterns of holes, a RAM, a programmable read-only memory (PROM), an erasable PROM (EPROM), a FLASH-EPROM, non-volatile random-access memory (NVRAM), any other memory chip or cartridge, content-addressable memory (CAM), and ternary content-addressable memory (TCAM).

A storage medium is distinct from but may be used in conjunction with a transmission medium. Transmission media participate in transferring information between storage media. Examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that comprise bus702. Transmission media may also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.

Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor704for execution. For example, the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions over a network, via a network interface controller (NIC), such as an Ethernet controller or Wi-Fi controller. A NIC local to computer system700may receive the data from the network and place the data on bus702. Bus702carries the data to main memory706, from which processor704retrieves and executes the instructions. The instructions received by main memory706may optionally be stored on storage device710either before or after execution by processor704.

Computer system700can send messages and receive data, including program code, through the network(s), network link720and communication interface718. In the Internet example, a server730might transmit a requested code for an application program through Internet728, ISP726, local network722, and communication interface718.

In an embodiment, a computer network provides connectivity among a set of nodes running software that utilizes techniques as described herein. The nodes may be local to and/or remote from each other. The nodes are connected by a set of links. Examples of links include a coaxial cable, an unshielded twisted cable, a copper cable, an optical fiber, and a virtual link.

A subset of nodes implements the computer network. Examples of such nodes include a switch, a router, a firewall, and a network address translator (NAT). Another subset of nodes uses the computer network. Such nodes (also referred to as “hosts”) may execute a client process and/or a server process. A client process makes a request for a computing service (for example, a request to execute a particular application and/or retrieve a particular set of data). A server process responds by executing the requested service and/or returning corresponding data.

A computer network may be a physical network, including physical nodes connected by physical links. A physical node is any digital device. A physical node may be a function-specific hardware device. Examples of function-specific hardware devices include a hardware switch, a hardware router, a hardware firewall, and a hardware NAT. Alternatively or additionally, a physical node may be any physical resource that provides compute power to perform a task, such as one that is configured to execute various virtual machines and/or applications performing respective functions. A physical link is a physical medium connecting two or more physical nodes. Examples of links include a coaxial cable, an unshielded twisted cable, a copper cable, and an optical fiber.

A computer network may be an overlay network. An overlay network is a logical network implemented on top of another network (for example, a physical network). Each node in an overlay network corresponds to a respective node in the underlying network. Accordingly, each node in an overlay network is associated with both an overlay address (to address the overlay node) and an underlay address (to address the underlay node that implements the overlay node). An overlay node may be a digital device and/or a software process (for example, a virtual machine, an application instance, or a thread). A link that connects overlay nodes may be implemented as a tunnel through the underlying network. The overlay nodes at either end of the tunnel may treat the underlying multi-hop path between them as a single logical link. Tunneling is performed through encapsulation and decapsulation.

In an embodiment, a system supports multiple tenants. A tenant is a corporation, organization, enterprise, business unit, employee, or other entity that accesses a shared computing resource (for example, a computing resource shared in a public cloud). One tenant (through operation, tenant-specific practices, employees, and/or identification to the external world) may be separate from another tenant. The computer network and the network resources thereof are accessed by clients corresponding to different tenants. Such a computer network may be referred to as a “multi-tenant computer network.” Several tenants may use a same particular network resource at different times and/or at the same time. The network resources may be local to and/or remote from the premises of the tenants. Different tenants may demand different network requirements for the computer network. Examples of network requirements include processing speed, amount of data storage, security requirements, performance requirements, throughput requirements, latency requirements, resiliency requirements, Quality of Service (QoS) requirements, tenant isolation, and/or consistency. The same computer network may need to implement different network requirements demanded by different tenants.

In an embodiment, in a multi-tenant computer network, tenant isolation is implemented to ensure that the applications and/or data of different tenants are not shared with each other. Various tenant isolation approaches may be used. In an embodiment, each tenant is associated with a tenant ID. Applications implemented by the computer network are tagged with tenant ID's. Additionally or alternatively, data structures and/or datasets, stored by the computer network, are tagged with tenant ID's. A tenant is permitted access to a particular application, data structure, and/or dataset only if the tenant and the particular application, data structure, and/or dataset are associated with a same tenant ID. As an example, each database implemented by a multi-tenant computer network may be tagged with a tenant ID. Only a tenant associated with the corresponding tenant ID may access data of a particular database. As another example, each entry in a database implemented by a multi-tenant computer network may be tagged with a tenant ID. Only a tenant associated with the corresponding tenant ID may access data of a particular entry. However, the database may be shared by multiple tenants. A subscription list may indicate which tenants have authorization to access which applications. For each application, a list of tenant ID's of tenants authorized to access the application is stored. A tenant is permitted access to a particular application only if the tenant ID of the tenant is included in the subscription list corresponding to the particular application.