Patent ID: 12225052

DESCRIPTION OF EXAMPLE EMBODIMENTS

Overview

According to one or more embodiments of the disclosure, a device may determine a compliance status of a communication of a type of data between a first workload and a second workload based on a data compliancy policy and a verified node location of at least one of the first workload and the second workload. The device may send, based on the compliance status of the communication, an instruction for handling the communication to at least one of a node executing the first workload and a node executing the second workload.

Description

A computer network is a geographically distributed collection of nodes interconnected by communication links and segments for transporting data between end nodes, such as personal computers and workstations, or other devices, such as sensors, etc. Many types of networks are available, with the types ranging from local area networks (LANs) to wide area networks (WANs). LANs typically connect the nodes over dedicated private communications links located in the same general physical location, such as a building or campus. WANs, on the other hand, typically connect geographically dispersed nodes over long-distance communications links, such as common carrier telephone lines, optical lightpaths, synchronous optical networks (SONET), or synchronous digital hierarchy (SDH) links, or Powerline Communications (PLC) such as IEEE 61334, IEEE P1901.2, and others. The Internet is an example of a WAN that connects disparate networks throughout the world, providing global communication between nodes on various networks. The nodes typically communicate over the network by exchanging discrete frames or packets of data according to predefined protocols, such as the Transmission Control Protocol/Internet Protocol (TCP/IP). In this context, a protocol consists of a set of rules defining how the nodes interact with each other. Computer networks may be further interconnected by an intermediate network node, such as a router, to extend the effective “size” of each network.

Smart object networks, such as sensor networks, in particular, are a specific type of network having spatially distributed autonomous devices such as sensors, actuators, etc., that cooperatively monitor physical or environmental conditions at different locations, such as, e.g., energy/power consumption, resource consumption (e.g., water/gas/etc. for advanced metering infrastructure or “AMI” applications) temperature, pressure, vibration, sound, radiation, motion, pollutants, etc. Other types of smart objects include actuators, e.g., responsible for turning on/off an engine or perform any other actions. Sensor networks, a type of smart object network, are typically shared-media networks, such as wireless or PLC networks. That is, in addition to one or more sensors, each sensor device (node) in a sensor network may generally be equipped with a radio transceiver or other communication port such as PLC, a microcontroller, and an energy source, such as a battery. Often, smart object networks are considered field area networks (FANs), neighborhood area networks (NANs), personal area networks (PANs), etc. Generally, size and cost constraints on smart object nodes (e.g., sensors) result in corresponding constraints on resources such as energy, memory, computational speed and bandwidth.

FIG.1Ais a schematic block diagram of an example computer network100illustratively comprising nodes/devices, such as a plurality of routers/devices interconnected by links or networks, as shown. For example, customer edge (CE) routers110may be interconnected with provider edge (PE) routers120(e.g., PE-1, PE-2, and PE-3) in order to communicate across a core network, such as an illustrative network backbone130. For example, routers110,120may be interconnected by the public Internet, a multiprotocol label switching (MPLS) virtual private network (VPN), or the like. Data packets140(e.g., traffic/messages) may be exchanged among the nodes/devices of the computer network100over links using predefined network communication protocols such as the Transmission Control Protocol/Internet Protocol (TCP/IP), User Datagram Protocol (UDP), Asynchronous Transfer Mode (ATM) protocol, Frame Relay protocol, or any other suitable protocol. Those skilled in the art will understand that any number of nodes, devices, links, etc. may be used in the computer network, and that the view shown herein is for simplicity.

In some implementations, a router or a set of routers may be connected to a private network (e.g., dedicated leased lines, an optical network, etc.) or a virtual private network (VPN), such as an MPLS VPN thanks to a carrier network, via one or more links exhibiting very different network and service level agreement characteristics. For the sake of illustration, a given customer site may fall under any of the following categories:1.) Site Type A: a site connected to the network (e.g., via a private or VPN link) using a single CE router and a single link, with potentially a backup link (e.g., a 3G/4G/5G/LTE backup connection). For example, a particular CE router110shown in network100may support a given customer site, potentially also with a backup link, such as a wireless connection.2.) Site Type B: a site connected to the network by the CE router via two primary links (e.g., from different Service Providers), with potentially a backup link (e.g., a 3G/4G/5G/LTE connection). A site of type B may itself be of different types:2a.) Site Type B1: a site connected to the network using two MPLS VPN links (e.g., from different Service Providers), with potentially a backup link (e.g., a 3G/4G/5G/LTE connection).2b.) Site Type B2: a site connected to the network using one MPLS VPN link and one link connected to the public Internet, with potentially a backup link (e.g., a 3G/4G/5G/LTE connection). For example, a particular customer site may be connected to network100via PE-3and via a separate Internet connection, potentially also with a wireless backup link.2c.) Site Type B3: a site connected to the network using two links connected to the public Internet, with potentially a backup link (e.g., a 3G/4G/5G/LTE connection).

Notably, MPLS VPN links are usually tied to a committed service level agreement, whereas Internet links may either have no service level agreement at all or a loose service level agreement (e.g., a “Gold Package” Internet service connection that guarantees a certain level of performance to a customer site).3.) Site Type C: a site of type B (e.g., types B1, B2 or B3) but with more than one CE router (e.g., a first CE router connected to one link while a second CE router is connected to the other link), and potentially a backup link (e.g., a wireless 3G/4G/5G/LTE backup link). For example, a particular customer site may include a first CE router110connected to PE-2and a second CE router110connected to PE-3.

FIG.1Billustrates an example of network100in greater detail, according to various embodiments. As shown, network backbone130may provide connectivity between devices located in different geographical areas and/or different types of local networks. For example, network100may comprise local/branch networks160,162that include devices/nodes10-16and devices/nodes18-20, respectively, as well as a data center/cloud environment150that includes servers152-154. Notably, local networks160-162and data center/cloud environment150may be located in different geographic locations.

Servers152-154may include, in various embodiments, a network management server (NMS), a dynamic host configuration protocol (DHCP) server, a constrained application protocol (CoAP) server, an outage management system (OMS), an application policy infrastructure controller (APIC), an application server, etc. As would be appreciated, network100may include any number of local networks, data centers, cloud environments, devices/nodes, servers, etc.

In some embodiments, the techniques herein may be applied to other network topologies and configurations. For example, the techniques herein may be applied to peering points with high-speed links, data centers, etc.

According to various embodiments, a software-defined WAN (SD-WAN) may be used in network100to connect local network160, local network162, and data center/cloud environment150. In general, an SD-WAN uses a software defined networking (SDN)-based approach to instantiate tunnels on top of the physical network and control routing decisions, accordingly. For example, as noted above, one tunnel may connect router CE-2at the edge of local network160to router CE-1at the edge of data center/cloud environment150over an MPLS or Internet-based service provider network in backbone130. Similarly, a second tunnel may also connect these routers over a 4G/5G/LTE cellular service provider network. SD-WAN techniques allow the WAN functions to be virtualized, essentially forming a virtual connection between local network160and data center/cloud environment150on top of the various underlying connections. Another feature of SD-WAN is centralized management by a supervisory service that can monitor and adjust the various connections, as needed.

FIG.2is a schematic block diagram of an example node/device200(e.g., an apparatus) that may be used with one or more embodiments described herein, e.g., as any of the computing devices shown inFIGS.1A-1B, particularly the PE routers120, CE routers110, nodes/device10-20, servers152-154(e.g., a network controller/supervisory service located in a data center, etc.), any other computing device that supports the operations of network100(e.g., switches, etc.), or any of the other devices referenced below. The device200may also be any other suitable type of device depending upon the type of network architecture in place, such as IoT nodes, etc. Device200comprises one or more network interfaces210, one or more processors220, and a memory240interconnected by a system bus250, and is powered by a power supply260.

The network interfaces210include the mechanical, electrical, and signaling circuitry for communicating data over physical links coupled to the network100. The network interfaces may be configured to transmit and/or receive data using a variety of different communication protocols. Notably, a physical network interface210may also be used to implement one or more virtual network interfaces, such as for virtual private network (VPN) access, known to those skilled in the art.

The memory240comprises a plurality of storage locations that are addressable by the processor(s)220and the network interfaces210for storing software programs and data structures associated with the embodiments described herein. The processor220may comprise necessary elements or logic adapted to execute the software programs and manipulate the data structures245. An operating system242(e.g., the Internetworking Operating System, or IOS®, of Cisco Systems, Inc., another operating system, etc.), portions of which are typically resident in memory240and executed by the processor(s), functionally organizes the node by, inter alia, invoking network operations in support of software processors and/or services executing on the device. These software processors and/or services may comprise data transfer compliance process248, as described herein, any of which may alternatively be located within individual network interfaces.

FIG.3illustrates an example architecture300for data compliance according to various embodiments. Application316may be deployed, as are many modern applications, as a set of distributed workloads302. For example, workloads302may be distributed among and run across various nodes (e.g., nodes304) of various public and/or private cloud infrastructures.

Workload engines332may manage the deployment of application316and its corresponding workloads302at the nodes (e.g., node304aand node304b) where the workloads (e.g., workload302aand workload302brespectively) will be deployed. Agents310may communicate with and/or configure workloads302through APIs308. Additionally, workloads302may utilize other nodes (e.g., trusted authority node324aand trusted authority node324b) to provide additional functionality. For example, workloads302may utilize controllers314and/or servers312of authenticating nodes (e.g., trusted authority node324aand trusted authority node324b) to authenticate to one another and establish trusted communications.

With applications increasingly being developed as a set of distributed services (e.g., workloads302) running across a mix of multi-cloud and edge infrastructures (e.g., nodes304) similar to architecture300, enforcement of data compliance has only been made more complex and potential violations made more likely. Handling a mix of data types differentially subject to various regulations, and/or being utilized by geographically dispersed mobile endpoints creates complexities that are not addressed by current data compliance techniques.

The number of laws, regulations, and rules regarding the storage and use of certain types of data are continually increasing across the globe. For instance, the General Data Protection Regulation (GDPR) in Europe places strict regulations on how a user's personal data is collected and shared. These and other regulations have spawned independent efforts across several countries to ensure that online applications comply with specific data regulations at national, federal, or state level, and particularly, those that are cloud-based. This acceleration in data sovereignty regulations is posing complex challenges to the organizations that use or manage that data, since legal obligations and constraints vary from country to country. The challenge is even greater since data compliance requirements are often not limited to data sovereignty obligations. For example, depending on the type of an application, data compliance may demand the amalgamation of other regulations, such as industry-specific regulation (e.g., complying with HIPAA obligations in the healthcare industry in the United States), or organization-specific rules (e.g., on how to deal with confidential data).

Currently, data compliance is largely only being considered at deployment time. For instance, a multinational company (e.g., the application user in this example) may rely on a third-party software vendor to develop an application. The application may need to be deployed and/or used in different countries (e.g., in Russia, Canada, the USA and the UAE). Application developers are neither expected to have knowledge, nor to programmatically deal with, the intricacies of specific data regulations in different regions. However, they do have knowledge on the kind of data being stored, accessed, exchanged, etc. by their application (e.g., through the APIs that they develop). Despite this, developers lack programmatic methods and tooling to facilitate the task of application users or managers to adhere to data compliance rules. This void in existing development, security, and operations (DevSecOps) pipelines and toolchains represents one of the biggest obstacles to planning, instrumenting, and meeting compliance regulations before, during, and after an application is deployed. As a result, data compliance is a problem that is addressed today without programmatic guidance from the organization that knows the most about the application workloads and their corresponding data, that is, the application developer.

As described in greater detail below, a compliance engine306may be implemented to provide configuration, observability, and enforcement of data compliance policies. In order to achieve this role data compliance within the framework of the aforementioned complexities, utilizing compliance engine306to manage data compliance may involve introducing and leveraging a mechanism to identify types of data being used by application316and their applicable data compliance polices.

Beyond introducing the mechanism for identifying types of data and their applicable data compliance polices, utilizing compliance engine306to manage data compliance may also involve introducing a mechanism to monitor and enforce compliance policies regarding the localization and transfer of the various data types. Presently, there are a lack of mechanisms available to implement the data compliance policies to control data transfers between workloads302. In some approaches (e.g., Secure Production Identity Framework for Everyone (SPIFFE) or mutual transport Layer security (mTLS)), each one of the workloads302are provided a cryptographic identity so that it can authenticate to another workload whether in the same cluster or in a different one (e.g., in multi-cluster settings).

For instance, a cloud native workload authentication technique such as SPIFFE may provide a security identity (ID) to each of the workloads302and enable an individual workload to identify and cryptographically authenticate other workloads that it needs to communicate with. In some examples, SPIFFE Verifiable Identity Documents (SVIDs) may be utilized to assign and carry workload IDs. The SPIFFE IDs may be uniform resource identifiers (URIs) (e.g., spiffe://trust_domain/workloadID).

Bundles may be used by a destination workload to verify the ID of a source workload and vice versa. The bundles may be a collection of one or more certificate authority (CA) root certificates that a workload should consider trustworthy. The bundles may contain public key materials such as X.509 and JWT SVIDs.

The IDs and/or bundles may be used and/or authenticated utilizing authenticating nodes324. The IDs and/or bundles may be federated. For example, servers312may identify and authenticate with each other through the federation. Such federated servers may exchange both IDs and bundles and utilize the Agents310to assign secure IDs to workloads302and convey the bundles, so that a workload (e.g., workload302a) can identify and authenticate other workloads (e.g., workload302b).

Agents310may use API308to convey the IDs and bundles to the corresponding workloads. As mentioned above, workload engines332are the ones in charge of spinning up the workloads302at nodes304. Tools, such as Kubernetes, may be used to this end. In such cases, each workload might have assigned a unique (internal) identifier within a workload engine332(e.g., a non-secure identifier assigned through Kubernetes). Such identifiers might need to be communicated to authenticating nodes324(e.g., using API318and controllers314). This information may now be communicated to servers312, which may in turn forward it to Agents310. In this way, Agents310may now identify the workloads spawn up by workload engines332and assign secure IDs to them as well as provide the necessary bundles so that the workloads may now securely identify and authenticate other workloads. More specifically, a source workload (e.g., workload302a) may provide its secure ID to identify and/or authenticate itself to a destination workload (e.g., workload302b) and/or to establish trusted communication with the destination node based on the authentication (AuthN).

While these workload ID and AuthN techniques may be utilized to address the question of “Who am I talking to?” among distributed workloads302, they do not address data authorization or data localization. For instance, the workload ID and AuthN techniques are not able to address the questions of “Given two workload IDs, what categories of data are these workloads authorized to exchange between them?” and/or “Given two workloads, where should these workloads be located such that the transfer of a specific category of data is compliant with existing regulations?”.

Various techniques have been developed to create trustworthy systems and infrastructures (e.g., using a hardware root of trust, secure boot processes, firmware protections, etc.). Further, various infrastructure geolocation techniques including proof of node location may be utilized to establish where trusted infrastructure is located. For example, a geographic location of trusted infrastructure320may be remotely verified utilizing a geolocation verification process322. Geolocation verification process322may be utilized in building a trusted workload and data localization system.

Some techniques may also cover remote attestation procedures to ensure that am application's runtime environment has not been tampered with. However, modern applications are often composed of a set of workloads which are executed in diverse platforms, such as virtual machines (VMs), containers, on bare metal, in serverless environments, etc. Hence, in many cases, the mere identification of such workloads across difference platforms may pose a challenge. For instance, different platforms may use different mechanisms to identify the workloads, therefore, workload-to-workload identification and authentication may be challenging.

As noted above, both trustworthy data authorization and data localization may be aspects implicated in modern data compliance. However, existing cross-platform workload identification and authentication mechanisms remain disconnected from techniques providing trusted infrastructures and proof of location.

For example, there is currently no integration between existing workload ID/AuthN techniques and existing geolocation verification techniques. That is, these techniques are unbonded (e.g., unbonded323a) from one another and are not used together and/or leveraged by one another. Further, utilizing compliance engine306to manage data authorization and drive localization of data based on data compliance policies may involve introducing a mechanism to avoid any unbonding (e.g., unbonded323b) of workload ID and AuthN techniques and/or geolocation verification techniques from compliance engine306.

Thus, existing techniques leave modern applications unable to support the automated identification of data types and their applicable data compliance polices. Modern applications are also rendered unable to support monitoring and enforcement of data compliance policies regarding the localization and transfer of the various data types among distributed workloads of the application. Therefore, modern applications are unable to effectuate data compliance policies at the data transfer level and/or are unable to identify and authorize compliant data transfers even across already authenticated workload IDs subject to various compliance rules and regulations.

Compliant Data Transfers

The techniques herein introduce mechanisms for the automated binding of authenticated workload IDs and specific infrastructure resources with a provable location. Such bindings may be used by a compliance engine to authorize a data transfer between two authenticated workloads, subject to various compliance results and/or regulatory obligations that may apply to the specific category of data to be transferred. These mechanisms may address data compliance requirements, such as GDPR in Europe, which restrict where and how certain types of data are stored, retained, accessed, used, etc.

Illustratively, the techniques described herein may be performed by hardware, software, and/or firmware, such as in accordance with data transfer compliance process248, which may include computer executable instructions executed by the processor220(or independent processor of interfaces210) to perform functions relating to the techniques described herein.

Specifically, according to various embodiments, a device may determine a compliance status of a communication of a type of data between a first workload and a is second workload based on a data compliancy policy and a verified node location of at least one of the first workload and the second workload. The device may send, based on the compliance status of the communication, an instruction for handling the communication to at least one of a node executing the first workload and a node executing the second workload.

Operationally,FIG.4illustrates an example architecture400for data compliance, according to various embodiments. The architecture400may include a data compliance process428. Data compliance process428may be utilized to provide configuration, observability, and enforcement of data compliance rules. Data compliance process428may accomplish these functions utilizing Data Compliance as Code (DCaC).

DCaC may include integrating a data compliance mechanism into the program code of the application. For example, data compliance process428may be utilized to build data compliance into the application development process, supported by automated code annotations, bindings between such annotations and categories of sensitive data, and controls at code, build, and pre-deploy time. Data compliance process428may provide a mechanism whereby application developers proactively assist data teams, application managers, and legal departments with data compliance, while ensuring that developers may remain oblivious to specific regulations, data related obligations, or compliance requirements that organizations might have across different regions.

For example, data compliance process428may include data annotating process402. Data annotating process402may facilitate application developer412automatically adding metadata to program code of an application316during the development of the application316. In various embodiments, this may be performed by automated annotations of data fields in the program code and by the creation of references to such annotations at code-build time. These references to annotated code may be automatically rendered in the form of machine-readable data manifest414.

More specifically, data annotating process402may provide a mechanism for automated annotations of the program code of application316, including classes, application programming interfaces (APIs), and the resulting data at code/build time (e.g., by implementing a Low-Code/No-Code approach supported by software development kits (SDKs)420and tooling418). Application developers may utilize SDKs420and tooling418to automatically label data topics, data producers, data consumers, data processors, data holders, etc. For instance, developers may label certain data by annotating it with a data type identifier. For example, a developer may annotate certain data as “protected-type-1,” or other data as “protected-type-2,” and so on.

SDKs420in data annotating process402may provide a set of predefined data types out-of-the-box, including associations by default to specific categories of sensitive data. Sensitive data may include a type of data that may be considered personal, private, sensitive, confidential, protected, secret, restricted, personally identifiable information (PII), etc. In some examples, sensitive data may include data that is subject to regulation. For example, Table 1 lists examples of predefined protected data types and default associations to some examples of categories of sensitive data.

TABLE 1PROTECTED DATA TYPEDEFAULT ASSOCIATIONprotected-type-1Customer PIIprotected-type-2Employee PII. . .. . .protected-type-23Patient Analysis Results. . .. . .protected-type-41Sales Confidential. . .. . .protected-type-56Restricted HR. . .. . .unprotectedNA
A list of the associations, such as the example illustrated in Table 1, may provide associations by default to several categories of sensitive data, including but not limited to PII, confidential, restricted, and unprotected data. In some embodiments, the set of predefined protected data types might be standardized or rely on an existing taxonomy.

SDKs420in data annotating process402may also provide a mechanism to define and use custom data types in annotating program data of the application316. For example, custom data types may be utilized, which identify protected data types that are not covered by any of those available by default in SDKs420. For example, “custom-type-1” might be a custom data type associated to a category of sensitive data such as “Restricted Employee Poll.” In various embodiments, the generation and/or insertion of the annotations into the program code of the application316may be accomplished by an automated process (e.g., a programmatic identification of data of a particular data type triggering an automated insertion of an annotation of the data as the particular data type, etc.), a partially automated process (e.g., a programmatic flagging of data of a particular data type with a supervised or manual annotation of the data as the particular data type, etc.), and/or a manual process (e.g., a manual flagging of data of a particular data type and/or a manual annotation of the data as the particular data type, etc.).

In various embodiments, associations between protected data types and categories of sensitive data may be assigned and/or instrumented by different organizations and at different moments in time. In some cases, the association between protected data types and categories of sensitive data may be assigned by application developers412at code/build time. This might be the case when the team of application developers412is part of, or develops for, the organization that may use or manage the application316. In such cases, the team of application developers412might have sufficient knowledge about the data and their use, so that they may either use the associations provided by default or create custom ones.

In additional instances, application developers412of application316and/or the users of the application316might belong to different organizations. For example, this may be the case when application developers412are a DevSecOps team that develops an application316that may be used across different organizations, industries, etc. In such cases, application developers412may be unaware of the categories of data that should be assigned by a data team and/or application manager404in another organization (e.g., precisely what data is confidential and what data is not with respect to that organization and its use of the application316). In these instances, application developers412may leverage SDKs420and tooling418to approach data labeling and association in a manner that sidesteps the knowledge deficit while still instilling the functionality. For example, the application developers412may leverage SDKs420and tooling418to automatically add the different classes of protected data type at code and build time (e.g., utilizing predefined and custom protected data types). Additionally, or alternatively, the application developers412may leverage SDKs420and tooling418to automatically insert references in the form of machine-readable descriptions for the protected data types that may be used to generate data manifest414bound to application316at build time.

The protected data type annotations and their corresponding references may be utilized by a data team and/or application manager404in another organization to select and/or create automated associations426between categories of annotated data in the application316(e.g., metadata provided by application developers412) and specific categories of sensitive data (e.g., personal data, private data, sensitive data, confidential data, protected data, secret data, restricted data, PII, etc.). For instance, each protected data type might be bonded to a class of tokens (e.g., JSON Web Tokens with a specific scope), which in turn might represent different categories of sensitive data for a data team and/or application manager404.

In a specific example, an API call for application316may be labeled by application developers412with a data type identifier such as “custom-type-7” at code/build time. The “custom-type-7” labeled API call may attempt to access certain data using its bound token (e.g., “Token 7”) with a scope defined by, for example, a data team and/or application manager404before application316was deployed. From the data team and/or application manager404perspective, the attempt to access this data may translate to a request to access, for instance, “Confidential Partner” data. As such, the data type labels, and their associations may be utilized as an automated data mapping between the programmatic operations of application316and the sensitive data implicated in those operations. In various embodiments, these associations and functionalities may be supported by compliance engine306based on the selection, configuration, and automation of data compliance rules before application316is deployed and/or post-deployment.

In some examples, application developers412, which again may be a DevSecOps team, might opt for a hybrid approach to generating these associations. For example, this may be the case when making some custom associations between data types and categories of sensitive data or using those predefined in the system (e.g., “protected-type-1” to “Customer PII”) might not only be trivial for the application developers412but also may facilitate the task of a data team and/or application manager404in defining associations. However, other associations might not be apparent to application developers412. Hence, certain data in application316may be labeled as “protected types” along with their corresponding machine-readable descriptions in data manifest414, though they may remain unassigned to a specific category of sensitive data, so they can be associated later by a data team and/or application manager404before the application is deployed, or by an automated data lineage, classification, and tagging process at run time (e.g., during the testing phase, that is, before the application is deployed in production).

In some embodiments, a data team and/or application manager404may be provided with a mechanism to change the associations created by application developers412or even associate more than one category of sensitive data to a given data type (e.g., a data team and/or application manager404may associate certain data with both “Employee PII” and “Confidential Data”). Hence, two categories of data compliance policies (e.g., one for “Employee PII” and another for “Confidential Data”) may apply and restrict even further the access to this category of data. In general, a data team and/or application manager404may be able to Create, Read, Update, or Delete (CRUD) any association between the metadata provided by application developers412and categories of sensitive data.

In various embodiments, a data team and/or application manager404may proactively create a set of custom data types. A data team and/or application manager404may provide the set of custom data types to application developers412. Application developers412may then utilize the set of custom data types so that application316is annotated at development based on guidelines (e.g., the set of custom data types, etc.) provided beforehand by the data team and/or application manager404.

In additional embodiments, application developers412and a data team and/or application manager404may collaborate to annotate application316. For example, application developers412and a data team and/or application manager404may iterate in the annotation and association processing in an agile manner. For example, the iteration may be performed as part of a continuous integration/continuous delivery (Cl/CD) pipeline (e.g., at testing, staging, and production).

In some examples, application316may be composed of several services developed with different programming languages. Therefore, application316may utilize different SDKs420. In some instances, the annotation methods and terminology applied to application316may vary depending on the programming language (e.g., usually referred to as attributes in C#, decorators in Python, annotations in Golang, etc.). In such cases, tooling418of data annotating process402may examine the different predefined and custom data types used with different SDKs420, perform checks, and ensure consistency in the annotations and enumeration across the different services at build time. For example, these consistency checks may ensure that a given “custom-type-X” data type identifier represents the same type of data across services programmed using different programming languages even if they were programmed by different developers. Overall, the data annotating process402may provide different degrees of freedom to application developers412, data teams and/or application managers404, and the number of protected data types used, and their corresponding associations may vary depending on the type of application316.

Data annotating process402may, as described above, be utilized in generating automated data references. Specifically, data annotating process402may automatically render a data manifest414bonded to application316at build time. Data manifest414may provide machine-readable descriptions of the predefined and/or custom data types used by application316. A combination of SDKs420and tooling418may facilitate the instrumentation and automation of the program code at build time, including the automated rendering of data manifest414. In some cases, application316may be composed of various containers. Each container may be built and packaged with its own data manifest, such that the final data manifest rendered for application316may be a composition of the individual data manifests. In some cases, application316may include dependencies on external services, such as a MySQL database. Such dependencies may be captured as a dependency manifest. Data fed, processed, retained, or retrieved from these external services may also be annotated and automatically captured in application316data manifest414.

Data annotating process402may, as described above, be utilized for decoupling data compliance from the business logic of application316. For example, SDKs420and tooling418of data annotating process402may provide automated mechanisms for decoupling the configuration, observability, and enforcement of data compliance rules from the business logic of application316. In some instances, application316may be a cloud/edge native application, which may be implemented as a set of workloads composing a service mesh. The decoupling of data compliance from the business logic may be especially relevant for applications of this type, as geographically dispersed and/or variably deployed workloads may implicate increased data compliance complexity.

Various possible embodiments for decoupling data compliance from the business logic of application316may be utilized. For instance, a sidecar model, where the services that implement the business logic of application316are deployed together with sidecar proxies associated to each of those services, may be utilized. The sidecar proxies may be utilized to enforce horizontal functions that are independent of the business logic, such as routing, security, load balancing, telemetry, retries, etc. As such, the sidecars may be well-positioned to decouple, observe, and control data compliance. For example, a combination of distributed data compliance controllers and sidecar proxies may be used to configure, observe, and enforce data compliance rules across different geographies, and distributed multi-cloud and edge infrastructures434.

Instead of, or in addition to, using sidecars, various embodiments may use client libraries, daemons working in tandem with the application-specific services, or sandboxed programs in the OS kernel, e.g., using the Extended Berkeley Packet Filter (eBPF). Further embodiments may use an agentless approach or embed such functionality in Kubernetes itself. In any case, the functionality introduced herein may enable the portability and reuse of observability and enforcement of data compliance functions across not only different applications but also cloud and edge environments.

The above-described data annotating process402may yield a portable annotated application316that is geared with built-in annotations for different types of protected data. In addition, the yielded annotated application316may be structured to operate while remaining agnostic of any state, country, industry, organization-specific regulation and/or data policy requirements that a data team and/or application manager404might have. As a result, data annotating process402may be leveraged as a new model of building applications including DCaC by not only data teams and/or application managers404, but also software as a service (SaaS) providers and others.

Data compliance process428may provide configuration, observability, and enforcement of data compliance rules. As described above, associations426between categories of annotated data in application316and specific categories of sensitive data may be instrumented prior to a deployment of application316. The associations426may be used to control the processing and use of data during and after the deployment of application316. More specifically, compliance engine306may utilize associations426together with current data compliance regulations governing data handling in each region where application316may be used, as well as a specific organization's compliance rules408for/while using application316, to enforce compliance with them. Such controls may apply to data access requests, data storage and retention policies, data processing requirements, etc. of application316both at deploy and execution time, etc.

To this end, data compliance process428may include data compliance regulation repository422. Data compliance regulation repository422may provide a repository of data compliance rules. For example, data compliance regulation repository422may include a repository of industry regulations424which may be applicable to the use of application316. For example, with respect to instances where application316is used by a healthcare provider, data compliance regulation repository422may include industry regulations424such as Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations applicable to handling of data in the healthcare industry. In other examples, data compliance regulation repository422may include a repository of national regulations430which may be applicable to the use of application316. For example, with respect to instances where application316is based in a member state of the E.U., data compliance regulation repository422may include national regulations430such as the GDPR applicable to handling of data in the E.U.

The data compliance regulations included in data compliance regulation repository422may be consumed by a data team and/or application manager404as a service (aaS). Data compliance regulation repository422may support input, expression, collection, approval, visualization, and/or use of data compliance policies covering multiple categories of rules. For example, data compliance regulation repository422may store data compliance policies that are specific to an industry, those that may apply at a national, multi-national, federal, state, and industry levels, etc. For instance, an organization (e.g., a multi-national company) may leverage a data compliance regulation repository422service of a data compliance process428and utilize the regulations already available in data compliance regulation repository422, which may cover regulations across several industries and countries out-of-the-box. An organization may select the target state, country or region, the industry if needed, and select the data compliance regulations that may be applicable at the organizational level (e.g., organization's compliance rules408).

Compliance engine306may offer APIs and a user-friendly user interface (UI) through which a data team and/or application manager404may select and define data compliance requirements. For instance, if application316, which handles Customer PII data, needs to be deployed in British Columbia, Canada, a data team and/or application manager404may simply select “Customer PII→Apply Local Regulation” to constrain the processing, storage, retention, and access to Customer PII data according to the regulations in British Columbia as retrieved from data compliance regulation repository422. To this end, compliance engine306may compute and handle the resulting constraints that apply to Customer PII data in British Columbia transparently to data teams and/or application managers404. More specifically, the set of data compliance constraints may be captured in a machine-readable format from data compliance regulation repository422, and therefore, used by compliance engine306programmatically.

In some examples, compliance engine306may be utilized as a pluggable module working in tandem with one or more of workload engines332, such as Cisco Intersight or any automation tool offered by a hyperscaler, or other cloud and edge providers. Workload engines332may manage the deployment of application316, subject to the rules and constraints provided by compliance engine306.

In various embodiments, compliance engine306may operate either in a push or a pull model. For instance, in a pull model, workload engine332may receive a request to deploy application316in a given region (e.g., a request from a site reliability engineering (SRE) and/or information technology (IT) team410). In such a case, workload engine332may issue a request to compliance engine306, to compute and return data compliance rules and constraints that must be applied for their specific deployment. Alternatively, in a push model, a data team and/or application manager404may select the compliance rules required and a declarative intent for application deployment may be issued from compliance engine306to one or more of workload engines332. Such deployments may involve multi-cluster service meshes, which may run across multi-cloud and edge infrastructures434. In various embodiments, the above-described sidecar proxies in the service mesh may not only provide monitoring and observability of data compliance to compliance engine306but also may receive configuration and compliance updates in real-time436. In additional embodiments, the same functionality may be implemented utilizing client libraries, daemons, eBPF, an agentless approach, or Kubernetes itself. In addition, some embodiments may support the techniques described herein without utilizing a service mesh.

FIG.5illustrates an example architecture500for performing compliant data transfers according to various embodiments. Architecture500may be utilized to enable the automated creation of bindings between workload identifiers and the infrastructure (e.g., a particular one of the nodes304) where the identified workload302runs. In various embodiments, the infrastructure may be infrastructure with a provable location. The bindings may be utilized by compliance engine306to manage data compliance policy enforcement at a data transfer level. For example, compliance engine306may utilize the bindings and/or geolocation verifications to authorize and/or otherwise manage a data transfer between two authenticated workloads302, subject to various compliance rules and/or regulatory obligations that may apply to the specific category of data to be transferred.

Architecture500may comprise infrastructure including nodes304. Nodes304may include computational infrastructure across which an application's workloads302are distributed. Nodes304may be geographically dispersed and/or based/operating on different computing platforms. Nodes304may execute their respective workload302to deliver a specific functionality of the application.

Executing a workload302may include handling or using various types of data. Handling or using a type of data may include sending, receiving, storing, transforming, and/or otherwise using the type of data. The specific type of data being handled or used may be identifiable utilizing the aforementioned DCaC annotations.

Each node may be logically divided into a user space524and a kernel space526. User space524may be conceptualized as the virtual space of a node where workload302and/or workload API308of agent310operate, are accessed, and/or are executed. Kernel space526may be conceptualized as the virtual space where local compliance process522, kernel functions528, and/or local binding process520of agent310operate, are accessed, and/or are executed.

An agent310of each node304may include a workload API308and local binding process520. Workload API308may include a communication and/or control interface with a workload302of that node304. Workload API308may additionally interface with local binding process520of node304.

Each workload302executed at a node304may include a workload ID. A workload ID may include an identifier that uniquely identifies the corresponding workload302. For example, a first workload (e.g., workload302a) of a first node (e.g., node304a) may have a first workload ID and a second workload (e.g., workload302b) of a second node (e.g., node304b) may have a second workload ID that is distinct from the first workload ID. Each workload ID may be a cryptographic identity of its corresponding workload302. In various embodiments, the workload ID may include a security identity provided by a cloud native workload authentication technique such as SPIFFE.

Agent310of each node304may execute local binding process520for node304. Local binding process520may have access to the workload IDs. In some examples, the access may be read-only access to the workload IDs. For example, local binding process520may have read-only access to bundles containing, for instance, other workload IDs and/or certificates.

Local binding process520in each agent310may also have a unique ID. For example, a first local binding process (e.g., local binding process520a) of a first node (e.g., node304a) may have a first local binding process ID and a second local binding process (e.g., local binding process520b) of a second node (e.g., node304b) may have a second local binding process ID that is distinct from the first local binding process ID.

In addition to local binding process520, each node304may also execute a local compliance process522associated with each agent310. Local compliance process522may interface with kernel functions528, which may also interface with local binding process520. Each local compliance process522may also have its own unique ID. For example, a first local compliance process (e.g., local compliance process522a) of a first node (e.g., node304a) may have a first local compliance process ID and a second local compliance process (e.g., local compliance process522b) of a second node (e.g., node304b) may have a second local compliance process ID that is distinct from the first local compliance process ID.

Local binding process520in the agent310may include a mechanism that binds agent310to trusted metrics in the infrastructure node304(e.g., using trust anchors, trusted measurements, etc.) as well as to the local compliance process522. In various embodiments, such functionality might be instrumented through a forward signing interlock technique.

Local binding process520in each agent310may also register itself to a binding process502in a global compliance engine (e.g., compliance engine306). Here, binding process502in compliance engine306may have access to remote verification results. For example, binding process502may have access to remote verification results attesting to and/or proving a geolocation of each node304. For instance, a geolocation verification process322may provide the remote verification results attesting to and/or proving a geolocation of each node304.

In addition, binding process502of compliance engine306may bind a node ID (uniquely identifying the node304) and traces in the remote verification results obtained for the node304with a specific local binding process ID. In turn, local binding process520at each node304may bind local workload IDs and bundles to its own local binding process ID. Thus, local binding process520may securely communicate the local workload IDs and the corresponding bundles managed for each workload302to the binding process502in compliance engine306.

Similarly, binding process502of compliance engine306may bind a workload ID and the corresponding local binding process ID to a specific node ID as well as to a proof of geolocation obtained from geolocation verification process322. A compliance inference module504of compliance engine306may have access to a data compliance rules and constraints repository506. In some examples, rules and constraints repository may include and/or be derived from data compliance regulation repository422as described inFIG.4.

Compliance inference module504may provide the ability to analyze the bundles received by each workload302, infer a workload mesh, and determine which workload-to-workload communications are out of compliance based on the information received from binding process502and data compliance rules and constraints repository506in compliance engine306. In addition, compliance inference module504in compliance engine306may push a data compliance policy to a local compliance process522. The data compliance policy may include an instruction to observe, log, and potentially filter, at a data plane level, all data transfers that are out of compliance.

The trusted exchange of workload IDs and bundles may be supported by a trusted authority node324including servers312and controllers314which may interface with workload engine332to support the trusted exchange of the initial (non-secure) workload identifiers. In some examples, the trusted exchange of workload IDs and bundles may be supported by a trusted authority federation508.

In some embodiments, both local binding process520and local compliance process may be embodied as eBPF processes running in kernel space526. Alternative or additional embodiments may rely on a sidecar model in user space524, a daemon set, or other potential implementations.

In some scenarios, compliance engine306might not have direct access to the nodes304, and therefore, the bindings may be proxied and instrumented through trusted authority node324and/or trusted authority federation508. Likewise, local compliance functions may be proxied and instrumented through trusted authority node324and/or trusted authority federation508. In such scenarios, compliance engine306may interface with external geolocation verification process322as well as with through trusted authority node324and/or trusted authority federation508.

In some embodiments, trusted authority node324and geolocation verification process322might be integrated. In some embodiments, local compliance process522may only observe, log, and notify non-compliant data transfers. That is, enforcement actions may involve a human in the loop and manual activation of enforcement.

In other embodiments, compliance engine306may require the deployment of independent connectors (e.g., in the form of a VM within each trust domain). Such connectors may interface either with the agents310or with the trusted authority nodes324to gain read-only access to the metadata (i.e., the workload IDs, the bundles, trusted metrics, etc.).

FIGS.6A-6Billustrates an example procedure600for performing compliant data transfers according to various embodiments. Procedure600may be executed utilizing components of architecture500illustrated inFIG.5.

A first local binding process (e.g., local binding process520a) may execute at and/or in association with a first infrastructure node. The first local binding process may be associated with a unique first local binding process ID (e.g., local binding process ID602a).

The first local binding process may be executable to create automated bindings for a first workload executing at the first node, bundles associated with the first workload, and/or its own first local binding process ID (e.g., local binding process ID602a) and communicate them to binding process502in compliance engine306. The first workload may be associated with a unique first workload ID (e.g., workload ID604a). A first bundle of the first workload may be associated with a unique first bundle ID (e.g., bundle ID606a). The local compliance process executing at the first node may be associated with a first local compliance process ID (e.g., local compliance process ID608a). In addition, the first local binding process may include trusted metrics and traces610a.

The first local binding process (e.g., local binding process520a) may access the first workload ID and/or may access bundles of the first workload that may contain other workload IDs and/or certificates. The first local binding process may have its own unique ID (e.g., local binding process ID602a), which it may bind to trusted metrics and traces610ain the first node as well as to a local compliance process522aand/or a local compliance process ID608aof local compliance process522aexecuting at the first node. Therefore, the first local binding process may produce a binding including workload ID604a, bundle ID606a, local compliance process ID608a, and/or trusted metrics and traces610abound to local binding process ID602a.

Local binding process520aof the first node may register to binding process502of compliance engine306. Local binding process520amay communicate the binding that it created including local workload ID604a, bundle ID606a, local compliance process ID608a, trusted metrics and traces610a, and/or local binding process ID602ato binding process502of compliance engine306.

Binding process502may access remote verification results proving the geolocation of the first infrastructure node ID associated with the binding. For example, binding process502may obtain the remote verification results through a trusted geolocation verification process322. The remote verification results may include a node ID612of the node being verified.

For example, geolocation verification process322may include a first node ID (e.g., node ID612a) for the first node associated with the first local binding process (e.g., local binding process520a). The geolocation verification results may also include metrics and traces614afor the first node obtained through geolocation verification process322as well as an indication of a geolocation616aof the first node, which may be expressed as latitude and longitude coordinates or as some other expression of location. Binding process502of compliance engine306may bind the node ID612aand traces614ain the verification results for the first node obtained from geolocation verification process322with the specific local binding process ID602aand local compliance process ID608afor the first node.

In various embodiments, binding process502of compliance engine306may, for a first node, bind workload ID604aand a corresponding local binding process ID602ato the node ID612aas well as to the proof of geolocation obtained from geolocation verification process322. As a result, for the first node, binding process502may generate a binding including workload ID604a, bundle ID606a, local compliance process ID608a, local binding process ID602a, node ID612a, trusted metrics and traces614a, and/or geolocation616aof the first node and/or workload.

Likewise, a second local binding process (local binding process520b) executing at and/or in association with a second infrastructure node may perform the functionalities as described above in the context of the first node but for the second node and its workloads. Geolocation verification process322may perform the functionalities described above in the context of the first node for the second node as well.

Binding process502of compliance engine306may perform the functionalities described above in the context of the first node for the second node as well. As a result, binding process502of compliance engine306may generate a binding including workload ID604b, bundle ID606b, local compliance process ID608b, local binding process ID602b, node ID612b, trusted metrics and traces614b, and/or geolocation616bof the second node and/or second workload, in addition to those of the first node and/or first workload.

Compliance inference module504of compliance engine306may analyze the bundles received for each workload. Compliance inference module504may infer the workload mesh from the bindings as well. In addition, compliance inference module504may access data compliance rules and constraints repository506and/or determine which of the data compliance policies in the data compliance rules and constraints repository506are applicable to each workload communication. Compliance inference module504may make such a determination based on a type of data being used (e.g., communicated, sent, received, stored, saved, transformed, handled, etc.) by the workload. The type of data being used may be determined utilizing DCaC annotations of data type and/or their corresponding data category.

Compliance inference module504may determine a compliance status of each workload-to-workload communication involving the workloads based on the information received from binding process502and data compliance rules and constraints repository506in compliance engine306. For example, compliance inference module504may utilize the data compliance rules and constraints repository506and/or those data compliance policies in the data compliance rules and constraints repository506and determine which ones are applicable to each workload communication in order to resolve which workload-to-workload communications are out of compliance when referenced against the information received from binding process502regarding the communication between the workloads.

For example, the data compliance policies of data compliance rules and constraints repository506may specify a geographic location where a type of data may be utilized. As such, a workload-to-workload communication may be determined to be in compliance (e.g., compliance status: compliant) if the geolocation of the source node and the geolocation of the destination node involved in the communication are within the geographical location where the type of data that they are communicating is allowed to be utilized. Conversely, a workload-to-workload communication may be determined to not be in compliance (e.g., compliance status: non-compliant) if the geolocation of the source node and the geolocation of the destination node involved in the communication are not within the geographical location where the type of data that they are communicating is allowed to be utilized.

For example, compliance inference module504may, for a workload-to-workload data communication, utilize the results of binding process502, DCaC elements, and/or corresponding data compliance policies from data compliance rules and constraints repository506to determine a source workload ID620of the communication, a destination workload ID622of the communication, a data tag624of the data included in the communication (e.g., available through metadata added as part of a DCaC process), a data category626of the data included in the communication (e.g., also available as part of the associations involved in a DCaC process), data transfer details628including the workload geolocations, and a compliance status630for the communication.

In various embodiments, compliance inference module504may send, based on the compliance status of a workload-to-workload communication, an instruction632for handling the workload-to-workload communication to at least one of local compliance process522aassociated with the first workload involved in the workload-to-workload communication and/or a local compliance process (not illustrated) associated the second workload involved in the workload-to-workload communication. Sending the instruction632may include pushing a policy and/or instruction to one or more local compliance processes to observe, log and/or potentially filter, at data plane level, all data transfers that are out of compliance.

For example, if the policy and/or instruction implicates enforcement at only a first node executing a first workload (e.g., in instances where the second node is outside a geolocation where the type of data is permitted to be handled), then the policy and/or instruction may be pushed to only the first local compliance process associated with the first node and/or first workload. Conversely, if the policy and/or instruction implicates enforcement at both a first node executing a first workload and a second node executing a second workload (e.g., in instances where both the first node and the second node are outside a geolocation where the type of data is permitted to be handled), then the policy and/or instruction may be pushed to both the first local compliance process associated with the first node and/or first workload and a second local compliance process associated with the second node and/or second workload.

In this manner, procedure600may be utilized to enforce data compliance policies at the data transfer level. That is, techniques are described which allow for different data compliance policies applicable to various types of data and/or in various geolocations to be enforced in an automated and dynamic manner while ensuring that the underlying infrastructure is a trusted actor with authenticated characteristics (e.g., geolocation, etc.) applicable to the data compliance policy. The architecture described with respect to these techniques may also allow for rapid adaptation to evolving data compliance rules and regulations. For example, as data compliance rules and regulations are created and/or modified, they may be rapidly populated to the data compliance rules and constraints repository506so that they are immediately applied to workload communications involving data types implicated by the data compliance rules and regulations without substantial reconfiguration of the system.

FIG.7illustrates an example simplified procedure (e.g., a method) for performing compliant data transfers, in accordance with one or more embodiments described herein. For example, a non-generic, specifically configured device (e.g., device200), may perform procedure700by executing stored instructions (e.g., data transfer compliance process248).

The procedure700may start at step705, and continues to step710, where, as described in greater detail above, a device may determine a compliance status of a communication of a type of data between a first workload and a second workload based on a data compliancy policy and a verified node location of at least one of the first workload and the second workload. The data compliancy policy may specify a geographic location where the type of data may be utilized.

In addition, the device may obtain an association between each of a plurality of workloads of an application service and a corresponding node executing that workload. The device may associate a verified node location to each of the plurality of workloads. The association between each of a plurality of workloads and the corresponding node executing that workload may include a unique ID of an association process that generated the association at the corresponding node. The association between each of a plurality of workloads and the corresponding node executing that workload may include a unique ID of a compliance process to enforce a data compliance policy at the corresponding node. The device may obtain the association between each of a plurality of workloads of the application service and the corresponding node executing that workload from a trusted authority or trusted authority federation.

At step715, as detailed above, a device may send, based on the compliance status of the communication, an instruction for handling the communication to at least one of a node executing the first workload and a node executing the second workload. The instruction may include an instruction to the compliance process to enforce the data compliance policy for the type of data at the node executing the first workload. The instruction may include an instruction to filter the communication at the node executing the first workload. Additionally, or alternatively, the instruction may include an instruction to log the communication.

In various embodiments, the device may analyze bundles received for the first workload and for the second workload and/or infer a workload mesh from the bundles received for the first workload and for the second workload.

Procedure700then ends at step720.

It should be noted that while certain steps within procedure700may be optional as described above, the steps shown inFIG.7are merely examples for illustration, and certain other steps may be included or excluded as desired. Further, while a particular order of the steps is shown, this ordering is merely illustrative, and any suitable arrangement of the steps may be utilized without departing from the scope of the embodiments herein.

The techniques described herein, therefore, may be utilized to provide compliant data transfers. For example, these techniques introduce a mechanism to support trusted binding between authenticated workload IDs and specific infrastructure resources. In addition, these techniques provide a mechanism for provable location of workload IDs that may process, store, retain, and/or distribute data subject to data sovereignty regulations and/or other data compliance constraints. In this manner, the techniques may provide a mechanism for authorization of compliant data transfers across workload IDs, subject to various compliance rules and obligations.

As such, the techniques provide for trustworthy automated data authorization and localization techniques that may be utilized, in combination with DCaC mechanisms, to comply with data regulations. For example, the techniques may provide cross-platform workload identification and authentication mechanisms which are able to incorporate trusted infrastructures and proof of location in a manner that allows a modern application to comply with data regulations.

While there have been shown and described illustrative embodiments that provide compliant data transfers, it is to be understood that various other adaptations and modifications may be made within the spirit and scope of the embodiments herein. For example, while certain embodiments are described herein with respect to using the techniques herein for certain purposes, the techniques herein may be applicable to any number of other use cases, as well.

The foregoing description has been directed to specific embodiments. It will be apparent, however, that other variations and modifications may be made to the described embodiments, with the attainment of some or all of their advantages. For instance, it is expressly contemplated that the components and/or elements described herein can be implemented as software being stored on a tangible (non-transitory) computer-readable medium (e.g., disks/CDs/RAM/EEPROM/etc.) having program instructions executing on a computer, hardware, firmware, or a combination thereof. Accordingly, this description is to be taken only by way of example and not to otherwise limit the scope of the embodiments herein. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the embodiments herein.