Generating service-to-service dependency map from DNS and authentication logs

A distributed computing system is provided, and configured to execute a domain name service (DNS) log analyzer configured to identify a dependency of a first service executed on a first VM at a first server, on a second service executed on a second VM at a second server, via one or more DNS logs of a DNS server. The system is further configured to execute an authentication log analyzer configured to identify a dependency of the first service on a third service executed at a third server, via the one or more token authentication logs of an authentication server. The system is further configured to execute a dependency map generator configured to generate a service-to-service dependency map including the dependency between the first service and the second service, and the dependency between the first service and third service.

BACKGROUND

Modern distributed computing systems include large numbers of servers located at data centers around the world, which communicate with each other across computer networks. Such distributed computing systems offer a plethora of software services that enable developers to develop, configure, deploy, and maintain cloud applications in virtualized environments. A program executed on a particular server can communicate with various AUTHENTICATION of these services executed on other servers, and those services in turn may communicate with other services on yet different servers, and so on. Such communications create dependencies among these services. A failure in one service can affect all services that depend on it. The web of dependencies between such services, and the attendant risks of potential faults due to dependency failures, are exceedingly large in modern distributed computing systems.

SUMMARY

A distributed computing system is provided, and configured to execute a domain name service (DNS) log analyzer configured to identify a dependency of a first service executed on a first virtual machine (VM) at a first server, on a second service executed on a second VM at a second server, via one or more DNS logs of a DNS server. The distributed computing system is further configured to execute an authentication log analyzer configured to identify a dependency of the first service on a third service executed at a third server, via the one or more token authentication logs of an authentication server. The system is further configured to execute a dependency map generator configured to generate a service-to-service dependency map including the dependency between the first service and the second service, and the dependency between the first service and third service.

DETAILED DESCRIPTION

In a modern globally distributed computing system, millions of servers positioned in data centers distributed throughout various geographic regions may generate billions of service-to-service communications per hour. These communications create dependencies between the requesting service and the service responding to the request. As discussed above, challenges exist to determining service-to-service dependencies. These dependencies vary over time, and their fleeting existence makes them difficult to track. Additionally, some services may be hidden and/or omitted in a conventional dependency map created using domain name service (DNS) log data, as the apparent second service may be a pass-through service rather than a destination service. The sheer volume of such requests, as well as privacy and security measures, also make it difficult to track such dependencies. Further complicating matters, virtualization through container architectures or hypervisor architectures, as described herein, can obfuscate the determination of dependent services. Without an accurate service-to-service dependency map, impact analysis and functions of the platform can be compromised. For example, the true cause of a service outage may remain undiagnosed, critical dependencies may go undetermined which can lead to crashed services being brought back online in an incorrect order that generates dependency failures, legally or contractually determined obligations to house data and services in defined geographic regions can be difficult to meet, verifiable compliance reports can prove difficult to generate, and the true scope of software components affected by a computer virus or malware may be remain unknown. Failure to properly determined service-to-service dependencies in the above example scenarios can lead to decreased performance, increased costs to instantiate new virtual machine (VM) instances to respond to demand, downtime, and potential legal or contractual liabilities.

These potential negative effects can be avoided or mitigated if service-to-service dependencies are quickly and accurately identified.

To address these issues, with reference toFIG.1, a distributed computing system10is provided for use in creating a service-to-service dependency map from DNS logs and authentication system logs, according to one example implementation. The distributed computing system10includes an analysis server12configured to communicate over one or more computer networks with managed servers14implementing virtual machines (VMs)16, and with a development and operations (Dev/Ops) server18. The virtual machines16implemented by each server14may be implemented using a containerized architecture20, as shown inFIG.1, or a hypervisor architecture22as shown inFIG.2. Continuing withFIG.1, in the containerized architecture20, a host operating system26is executed on host hardware24(e.g., processor, accelerator hardware, non-volatile storage memory, and volatile memory), along with an instance of a container engine28. The container engine28provides a software interface between the host OS and one or more containers30. Each container30includes respective application programs32, libraries34, binaries and other data used by the applications. In this way, each container30implements a containerized virtual machine16within the container.

Turning briefly toFIG.2, alternatively, server14′ may implement virtualization using a hypervisor architecture22, which includes a hypervisor28A executed on a host OS26A, which in turn is executed on host hardware24A including a processor and memory. The hypervisor28A provides a software interface between the host OS26A and external hypervisor control plane servers, as well as individual virtual machines16′. Each virtual machine16′ includes a guest OS instance35A, as well as libraries34A, binaries, and other data used by applications32A executed within the hypervised virtual machine16′. In some implementations host OS26A may be omitted.

Continuing withFIG.1, when applications are executed in such virtualized environments as the containerized architecture20or the hypervisor architecture22, various services36of the distributed computed system can be called, illustrated schematically as first service36A, second service36B, and third service36C. These services36can be called by the applications, libraries, or binaries executed in the virtual machines16,16′, or by the container engine28/hypervisor28A and/or by the host OS26,26A in response to requests by software components in the virtual machines16,16′. In some cases, a virtual machine16,16′ is instantiated for the purpose of instantiating a particular service36of the distributed computing system, and the application32is a program that implements the service36. It will be appreciated that for each of first through third services36A,36B,36C, a corresponding first through third service instance36A1,36B1,36C1is instantiated on the first through third servers14A,14B,14C, in response to the requests by first through third virtual machines16A,16B,16C. Further, service instances36A1-36C1communicate with each other and other service instances within the distributed computing system10over computer networks. As illustrated by second service36B shown in dashed lines, each service36can include a plurality of service instances executed on servers across the distributed computing network.

In a large deployment of a distributed computing system10, millions of servers14may be provided, and billions of requests per hour may flow between service instances36A-36C. With such a large and complicated flow of communications, a significant technical challenge exists to monitor dependencies among services. Further, as shown inFIG.3, the servers14of computing system10may be located in data centers70located in different geographic regions72. Each of the geographic regions may be subject to different regulatory requirements for the processing and storage of data, including restrictions on the geographic location of data processing and storage of such data. Since data may flow along the dependencies between services, determining that processing of such data is compliant with regulations that restrict geographic scope of data processing and storage in this manner presents a technical challenge. Further, the technical challenges of diagnosing the true origin of a fault in a particular service and generating a recovery schedule for bringing services that depend on each other back online are complicated in such complex distributed computing systems.

Continuing withFIG.1, to address these technical challenges, the distributed computing system10includes an analysis server12. The analysis server12includes a processor38and memory39storing instructions that cause the processor38to execute a domain name service (DNS) log analyzer42. The DNS log analyzer42is configured to identify a dependency of a first service36A executed on a first VM16A at a first server14A of the distributed computing system10, on a second service36B executed on a second VM16B at a second server14B of the distributed computing system10, via one or more DNS logs50of a DNS server48of the distributed computing system10. It will be appreciated that the second service36B can be a pass-through service that passes a request from the first service36A to a third service36C for processing, and returns a response from the third service36C to the first service36A through the second service36B. The DNS request44includes a fully qualified domain name (FQDN)59for a service executed at a destination server14, such as the second service36B executed at the second server14B. In one embodiment, the DNS server48may be configured to be a recursive resolver that resolves the FQDN59to a destination IP address60. The destination IP address60may be either an IPV4 or IPV6 address, for example. The DNS server48can thus fully resolve the FQDN59into the destination IP address60, which is returned to the requesting server14in a DNS response46. The DNS log analyzer42is also configured to identify the destination IP address60associated with a DNS response46to the DNS request44, using one or more DNS logs50of the DNS server48. Because first service36A sends an HTTP request to the second service36B, an FQDN59for the third service36C is not contained in the DNS request44from the first service36A and therefore the destination IP address60of the third service36C is not contained in the DNS response46thereto. Simply put, the first service36A is unaware of the third service36C. It will be appreciated that in addition to the DNS requests44and DNS responses46themselves, metadata such as a source container identifier58or a virtual machine identifier58A (seeFIG.2) are typically sent from the requesting server14to the DNS server48in the DNS requests44and responses46, and these metadata are logged in the DNS logs50, as explained in more detail below with reference toFIG.4.

It will be appreciated that the DNS log analyzer42is capable of identifying a plurality of dependencies of the first service36A, second service36B, and third service36C on a corresponding plurality of other services14with which they directly communicate (rather than through pass through services), using the DNS logs50. As one specific example, the DNS log analyzer42may be further configured to identify a dependency between the second service36B and the third service36C based on the one or more DNS logs. Other dependencies may also be identified by the DNS log analyzer42.

Continuing withFIG.1, distributed computing system10further includes a token generation service51and a token authentication service52, which are a software applications typically executed on an authentication server53or set of authentication servers53and respectively configured to generate and authenticate tokens. Thus, the first service36A requests generation of a token from the token generation service51, which returns the generated token in a response, and stores data related to the token generation in token generation logs54A. The tokens function as access tokens, and can be passed from a requesting service for which they were generated to a destination service to which access is requested. The destination service, such as the third service36C inFIG.4, receives the token of the requesting service, such as the first service36A, and sends the token to the token authentication service52of the authentication server53for authentication in a token authorization request55. If the token authentication service52responds with an affirmative response57confirming the token is valid, then the destination service, such as third service36C can permit access to gated content, for example. The token service52may implement the OAUTH or OAUTH 2.0 authentication protocols, for example. Other protocols may alternatively or additionally be implemented. Information from the token, including an identity of the requesting service14or container30associated with the token and the identity of the service14using the token for authentication, is logged by the token generation service51and token authentication service52in authentication logs54.

Distributed computing system10further includes an authentication log analyzer56, which is a software application executed by the analysis server12configured to analyze data records in the authentication logs54. The authentication log analyzer56is configured to identify various dependencies, including a dependency of the first service36A on the third service36C executed at the third server14C of the distributed computing system10, via the one or more token authentication logs54of an authentication server of the distributed computing system10.

The analysis server12is further configured to execute a dependency map generator62, which is a software application configured to generate a service-to-service dependency map64including the dependency between the first service36A and the second service36B, and the dependency between the first service36A and third service36C. As an option, when computed, the dependency between the second service36B and third service36C may also be included in the service-to-service dependency map64. Of course, multiple other dependencies of the first service36A may also be included, as will be appreciated in view of the examples provided below. The dependency map generator62can be configured to determine the second service36B is a pass-through service by comparing a timeframe of a DNS request44from the second service36B requesting the destination IP address60of the third service36C and a DNS response46to this DNS request44, a timeframe of a DNS request44from the first service36A requesting the destination IP address60of the second service36B, and a timeframe of the authentication request55from the third service and authentication response57from the token authentication service52. These timeframes may be determined by inspecting time stamps61of the DNS request log records50A1and DNS response log records50B1for each DNS request44and DNS response46in the DNS log50and the time stamps61of the token generation log record54A1and authentication request and response log record54B1for each token generation request, authentication request55, and authentication response57in in the authentication logs54.

As described below with respect toFIG.5, the service-to-service dependency map64may be displayed within a graphical user interface (GUI)41on a display40of the computing system10. A number of levels of dependency (e.g., up to first order dependencies, up to second order dependencies, up to third order dependency, all dependencies, etc.) displayed in the GUI can be determined by input from a user.

The dependency map generator62is configured to output the service-to-service dependency map64to a downstream computing program66executed on Dev/Ops server18(or other server) for a processing operation. The downstream computing program66can be a fault diagnosis program66A, recovery program66B, a geographic compliance program66C, or a threat diagnostics program66D, for example. Other types of downstream computing programs66are also contemplated. The downstream computing program66is configured to process the service-to-service dependency map64via a processing operation to thereby generate and output a processing result68. The fault diagnosis program66A can be configured to generate a fault diagnosis68A based on the service-to-service dependency map64as the processing result68. The recovery program64B can be configured to compute a recovery schedule66B for restarting services based on the service-to-service dependency map64as the processing operation. The geographic compliance program64C can be configured to determine a geographic scope66C of a location of servers executing the first service36A and the second service36A (or, all dependent services on a target service) as the processing result. The threat identification program66D can be configured to scan system logs of servers14executing dependent services of the first service36A to identify a presence or effect or a virus or malware on the servers14as the processing operation, to thereby produce a threat identification68D. Additional orders of dependency, discussed below, may also be included in the processing result68. Examples of the fault diagnosis68A, recovery schedule68B and geographic scope of services68C are illustrated inFIGS.6-8, discussed below.

Turning now toFIG.4A-4C, data relationship diagrams are shown, illustrating in detail the steps undertaken by the analysis server12to process the DNS logs50and authentication logs54to generate the service-to-service dependency64, when the source identifier is a source container identifier58(as illustrated) or a virtual machine identifier58A. Beginning withFIG.4A, the DNS log analyzer42is configured to extract DNS records74, which include a DNS request log record50A1from the DNS request log50A and one to N DNS response log records50B1associated with the DNS request log record50A1.

The DNS log analyzer42is configured to identify the dependency of the first service36A on the second service36B at least in part by performing the following operations. First the DNS log analyzer42identifies, within a DNS request log50A of the one or more DNS logs50, a DNS request log record50A1for a DNS request44, the DNS request log record50A1including a source identifier and a source IP address76associated with the first service36A. The source identifier is determined to be associated with the first service36A by identifying a VM16implementing the first service36A on the first server14A that has been assigned the source IP address76. The source identifier can be a source virtual machine identifier58A (SeeFIG.2) or a source container identifier58as shown inFIG.4A. Alternatively, a different form of source identifier may be used. Second, the DNS log analyzer42identifies, within the DNS request log record50A1a destination fully qualified domain name (FQDN)59requested by the DNS request44. Third, the DNS log analyzer42identifies, within a DNS response log record50B1for a DNS response46to the DNS request44, a destination IP address60for the destination FQDN59. A timestamp61may be included in each of the DNS request log50A1and DNS response log50B1. Fourth, the DNS log analyzer42identifies the second service36B as being associated with the destination IP address60, based for example, on an inventory of IP addresses in a service tree data structure maintained by the distributed computing system10.

Continuing withFIG.4A, the computing system of claim1, wherein the authentication log analyzer56is configured to identify the dependency between the first service36A and third service36C, based on one or more authentication log records (such as token generation log record54A1and authentication requestion and response log record54B1) of authentication logs54for an authentication request from the third service36C to authenticate a token63of the first service36A received via the second service36B. As shown inFIG.4A, when a record for an authentication request54B1including a third container ID58associated with the third service36C is identified, which also includes a token63that matches the token63for the first service36A, then a dependency between of the first service36A on the third service36C is established.

The authentication log analyzer56is further configured to identify the dependency of the first service36A on the third service36B at least in part by performing the following operations. First, authentication log analyzer56identifies, within a token generation log54A of the one or more authentication logs54, a token generation log record54A1, the token generation log record54A1including the source container identifier58(or alternatively the FM identifier58A) and a globally unique identifier (GUID) as the token63. The source container identifier58is associated with a source container30implementing a VM16that has been assigned the GUID associated with the token63by the token generation service51. Second, the authentication log analyzer56identifies, in the authentication logs54, a token authentication request55from the second service36B for a token63associated with the first service36B. Third, the authentication log analyzer56identifies, within an authentication request and response log54B of the one or more authentication logs54, an authentication request and response log record54B1for an authentication response57, the authentication request and response log record54B1including a destination identifier (such as a destination container identifier58or destination VM identifier58A) that identifies a VM16implementing the third service36C that requested authentication of the token63in authentication request55, the token63including the GUID assigned by the token generation service to the VM16implemented by the source container30.

FIG.4Billustrates passage of a token63of the first service36A in an HTTP request originating from the first service36A sent to the second service36B. Second service36B, as a pass through service, initiates a separate HTTP request (akin to forwarding the HTTP request of the first service36A) to the third service36C, and passes the token of the first service in that HTTP request to the third service. After authentication request55is sent to the token authentication service52by the third service36C, and an authentication response57indicating validity of the token63is received at the third service36C, then an HTTP response is sent from the third service36C to the second service36B, which in turn causes the second service36B to respond with an HTTP response (for example by passing the HTTP response through) to the first service36A.

FIG.4Cillustrates different dependencies that are included in the service-to-service dependency map64by the dependency map generator62in view of the communication exchange ofFIG.4B. As illustrated, service-to-service dependency map64may include two DNS log derived dependencies and an authentication log derived dependency based on the communication exchange ofFIG.4B. First, the dependency between the first service36A and second service36B, as determined by the dependency identified between the first container ID58and the second service36B destination IP address60using the DNS logs50can be included. Second, the dependency between the second service36B and third service36C, as determined by the dependency identified between the second container ID58and the third service36C destination IP address60using the DNS logs50can be included. Third, dependency of the first service36A on the third service36C, as identified in the authentication logs54derived dependency between the first container ID58and the third container ID58, may be included in the service-to-service dependency map64. In this way, the dependency between the first and third service36A,36C, which would not be discoverable from the DNS logs alone with surety, can be identified and mapped.

FIG.5illustrates an example service-to-service dependency map64created by the computing system10ofFIG.1. Although in the examples described above a first service36A was linked to a second service36B, in the example ofFIG.5, for a particular target service as the first service36A, a plurality of second services36B are identified by the techniques described above as being dependent on the first service36A. As discussed above, the service-to-service dependency map64may be displayed within the GUI41on the display40of the computing system10. The GUI41may include a selector41A, and a number of levels of dependency displayed may be determined by input from a user via the selector41A. Additionally, the displayed service-to-service dependency map indicates a category for each second service upon which the first service is dependent, and the category for each second service can be changed by the user based on optimization function, such as usage, for example. The categories may for example indicate a priority level as determined by usage. It will be appreciated that the viewing and changing of categories is not limited to second services, and that, when additional levels of dependency are displayed, the user may also view and changes the categories assigned to those services. In an alternative configuration, the categories may represent phases in the lifecycle of the first service, such as build time, test time, deployment time, run time, and/or retirement.

FIG.6illustrates a multi-level service-to-service dependency map64A, with a fault diagnosis. Heretofore, first order and second order dependencies have been discussed. First order dependences are illustrated inFIG.6at left, and are similar to the dependencies identified inFIG.5. Each of the first order dependent services is checked for its dependent services using the techniques described above, to identify second order dependent services, i.e., third services. In turn, each of the third services can be analyzed to determine its dependent services, which are third order dependent services, i.e., fourth services, relative to the target service. In this way, a multi-level service-to-service dependency map64A including a plurality of orders of dependency can be generated.

Such a map can be useful for diagnosing the true cause of service outages. For example, an example fault diagnosis68A is illustrated in dotted lines, showing how an error detected in the target service can be traced using the service dependency map through the first order dependent service “Container Mgmt Srvc”, and through the second order dependent service “Service D”, to the third order dependent services “Service G”. As Service G has no dependent services, this service can be determined to be the origin of the fault that caused the error experienced by the target service.

The multi-level service-to-service dependency map64A may be mapped over a lifetime of the identified first service, including build time, test time, deployment time, run time, and retirement. Such a map can be useful for determining services in use that are no longer needed. The deletion unnecessary services may reduce costs associated with executing a service, as well as minimize security issues.

FIG.7illustrates a multi-level service-to-service dependency map64A similar toFIG.6, and shows how it can be used to generate a recovery schedule68B. To correct the fault discussed above in relation toFIG.6, the services can be scheduled for restart in reverse order of dependency. Thus, the third order dependency services that show an error (i.e., Service G) can be scheduled for restarting first, second order dependent services showing an error (i.e., Service D) can be scheduled for restart next, first order dependent services showing an error (i.e., Container Mgmt Srvc) can be restarted next, and finally the target service can be restarted. In this way, at the time of restart, each service will have its dependent services available to it, and post-restart errors will be reduced.

FIG.8illustrates an example of a chart indicating a geographic scope of dependent services68C that can be output by the computing system ofFIG.1. Dependent services of a first service are listed at left inFIG.8, followed by an indication of the geographic region in which the servers instantiating the server instances implementing the dependent services are located. The locations of the dependent services may be ascertained from the inventory records of the services36, which can include geographic location information for each destination IP address59of each service36. The geographic scope of dependent services68C may be used to ensure compliance with regulations governing the geographic location of processing and storage of certain data by the computing system10.

Turning now toFIG.9, a flowchart of a method100according to one example implementation of the present disclosure is shown. Method100can be implemented using the hardware and software described above, or using other suitable hardware and software, as desired. At102, the method100includes identifying a dependency of a first service executed on a first virtual machine (VM) at a first server of the distributed computing system, on a second service executed on a second VM at a second server of the distributed computing system, via one or more domain name service (DNS) logs of a DNS server of the distributed computing system. As indicated at104, the second service can be a pass-through service that passes a request from the first service to a third service for processing. The method may include determining that the second service is a pass-through service by comparing a timeframe of a DNS request from the second service requesting the IP address of the third service and a DNS response to this DNS request, a timeframe of a DNS request from the first service requesting the IP address of the second service, and a timeframe of the authentication request from the third service. Further details of substeps that may be performed to implement step102are discussed below inFIG.10.

At106, the method100includes identifying a dependency of the first service on a third service executed at a third server of the distributed computing system, via the one or more token authentication logs of an authentication server of the distributed computing system. It will be appreciated that the source identifier may be a source container identifier or a virtual machine identifier as described above. Further details of substeps that may be performed to implement step106are discussed below inFIG.11.

At108, method100further includes identifying a dependency between the second service and the third service is accomplished based on the one or more DNS logs. The dependency of the second service on the third service may be determined in a similar manner as the dependency of the first service on the second service is at step102.

At110, method100includes generating a service-to-service dependency map including the dependency between the first service and the second service, and the dependency between the first service and third service. As indicated at111A, the method100may further comprise including the dependency between the second service and third service identified at step108in the service-to-service dependency map. As indicated at111B, method100may further include mapping the service-to-service dependency map over a lifetime of the identified first service, including build time, test time, deployment time, run time, and/or retirement such that the service-to-service dependency includes dependencies from each of these different phases.

At112, method100further includes outputting the service-to-service dependency map and inputting it to a downstream program executed on a server of the distributed computing system. As shown at114, the downstream program can be a recovery program configured to compute a recovery schedule for restarting services based on the service-to-service dependency map as the processing operation. As shown at116, the downstream program can be a fault diagnosis program configured to generate a fault diagnosis based on the service-to-service dependency map as the processing operation. As shown at118, the downstream program can be a geographic compliance program configured to determine a geographic scope of a location of servers executing the first service and the second service as the processing operation. As shown at120, the downstream program can be a threat identification program configured to scan system logs of servers executing dependent services of the first service to identify a presence or effect or a virus or malware on the servers, as the processing operation. Other types of downstream program may also be utilized.

At122, method100further includes performing, via the downstream program, a processing operation, such as the processing operations described above, based on the service-to-service dependency map. At124, method100further includes outputting a processing result of the processing operation. The outputting may include storing the processing result in memory, displaying the processing result on a display, or passing the processing result to a software application for further processing, for example.

As shown inFIG.10, identifying the dependency of the first service on the second service at step102may be accomplished at least in part by performing substeps126-132. At126, the method includes identifying, within a DNS request log of the one or more DNS logs, a DNS request log record for a DNS request. The DNS request log record includes a source identifier associated with the first service and a source IP address associated with the first service. The source identifier can be determined to be associated with the first service by identifying a virtual machine (VM) implementing the first service on a computer that has been assigned the source IP address. The source identifier can be a source virtual machine identifier or a source container identifier, for example. At128, the method includes identifying, within the DNS request log record a destination fully qualified domain name requested by the DNS request. At130, the method further includes identifying within a DNS response log record for a DNS response to the DNS request, a destination IP address for the destination fully qualified domain name. At132, the method includes identifying the second service as being associated with the destination IP address.

Turning now toFIG.11, identifying the dependency of the first service on the third service of step104ofFIG.9may be accomplished at least in part by performing substeps134-140ofFIG.11. At134, the method includes identifying an authentication log record (such as authentication request and response log record54B1discussed above) for an authentication request from the third service to authenticate a token of the first service received via the second service. At136, the method includes identifying, within a token generation log of the one or more authentication logs, a token generation log record that includes the source container identifier and a globally unique identifier (GUID) as the token. The source container identifier is associated with a source container implementing a virtual machine (VM) that has been assigned the GUID associated with the token. At138, the method includes identifying in the authentication logs a token authentication request from the second service for a token associated with the first service. At140, the method includes identifying, within an authentication response log of the one or more authentication logs, an authentication response log record for an authentication response that includes a destination identifier that identifies a VM implementing the third service that requested authentication of the token, the token including the GUID assigned to the VM implemented by the source container.

The systems and methods described above can be used to programmatically and efficiently generate a service-to-service dependency map for services implemented across distributed computing systems with virtualized environments having container or hypervisor architectures. A programmatically generated service-to-service dependency map can be generated for each phase in the lifecycle of an application, such as at build, development, testing, deployment, and retirement phases. Further, the service-to-service dependency map can be kept up to date through periodic automatic updating, to aid in time sensitive fault diagnosis and recovery scheduling. Further, the timing of the service-of-service dependency map can be re-computed with sufficient frequency to aid in regulatory compliance, for example, to geographic restrictions on data storage and processing in certain jurisdictions.

FIG.12schematically shows a non-limiting embodiment of a computing system1200that can enact one or more of the methods and processes described above. Computing system1200is shown in simplified form. Computing system1200may embody any of the servers in distributed computing system10described above and illustrated inFIGS.1and2.

Computing system1200includes a logic processor1202, volatile memory1204, and a non-volatile storage device1206. Computing system1200may optionally include a display subsystem1208, input subsystem1210, communication subsystem1212, and/or other components not shown inFIG.12.

Non-volatile storage device1206includes one or more physical devices configured to hold instructions executable by the logic processors to implement the methods and processes described herein. When such methods and processes are implemented, the state of non-volatile storage device1206may be transformed—e.g., to hold different data.

Volatile memory1204may include physical devices that include random access memory. Volatile memory1204is typically utilized by logic processor1202to temporarily store information during processing of software instructions. It will be appreciated that volatile memory1204typically does not continue to store instructions when power is cut to the volatile memory1204.

Aspects of logic processor1202, volatile memory1204, and non-volatile storage device1206may be integrated together into one or more hardware-logic components. Such hardware-logic components may include field-programmable gate arrays (FPGAs), program- and application-specific integrated circuits (PASIC/ASICs), program- and application-specific standard products (PSSP/ASSPs), system-on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example.

When included, display subsystem1208may be used to present a visual representation of data held by non-volatile storage device1206. The visual representation may take the form of a graphical user interface (GUI).

When included, input subsystem1210may comprise or interface with one or more user-input devices such as a keyboard, mouse, touch screen, camera, and microphone.

The following paragraphs provide additional description of aspects of the present disclosure. One aspect provides a distributed computing system. The distributed computing system may comprise a processor and memory storing instructions that may cause the processor to execute a domain name service (DNS) log analyzer, an authentication log analyzer, and a dependency map generator. The DNS log analyzer may be configured to identify a dependency of a first service executed on a first virtual machine (VM) at a first server of the distributed computing system, on a second service executed on a second VM at a second server of the distributed computing system, via one or more DNS logs of a DNS server of the distributed computing system. The authentication log analyzer may be configured to identify a dependency of the first service on a third service executed at a third server of the distributed computing system, via one or more token authentication logs of an authentication server of the distributed computing system. The dependency map generator may be configured to generate a service-to-service dependency map including the dependency between the first service and the second service, and the dependency between the first service and third service.

In this aspect, additionally or alternatively, the second service may be a pass-through service that passes a request from the first service to a third service for processing. In this aspect, additionally or alternatively, the authentication log analyzer may be further configured to identify a dependency between the second service and the third service based on the one or more DNS logs, and the dependency between the second service and third service may be included in the service-to-service dependency map.

In this aspect, additionally or alternatively, the DNS log analyzer may be configured to identify the dependency of the first service on the second service by: identifying, within a DNS request log of the one or more DNS logs, a DNS request log record for a DNS request, the DNS request log record including a source identifier associated with the first service and a source IP address associated with the first service, wherein the source identifier may be determined to be associated with the first service by identifying a virtual machine (VM) implementing the first service on a computer that has been assigned the source IP address, wherein the source identifier may be a source virtual machine identifier or a source container identifier; identifying, within the DNS request log record a destination fully qualified domain name requested by the DNS request; identifying within a DNS response log record for a DNS response to the DNS request, a destination IP address for the destination fully qualified domain name; and identifying the second service as being associated with the destination IP address.

In this aspect, additionally or alternatively, the authentication log analyzer may be configured to identify the dependency between the first service and third service, based on an authentication log record for an authentication request from the third service to authenticate a token of the first service received via the second service.

In this aspect, additionally or alternatively, the authentication log analyzer may be further configured to identify the dependency of the first service on the third service at least in part by: identifying, within a token generation log of the one or more authentication logs, a token generation log record, the token generation log record including the source container identifier and a globally unique identifier (GUID) as the token, wherein the source container identifier may be associated with a source container implementing a virtual machine (VM) that has been assigned the GUID associated with the token; identifying in the authentication logs a token authentication request from the second service for a token associated with the first service; and identifying, within an authentication response log of the one or more authentication logs, an authentication response log record for an authentication response, the authentication response log record including a destination identifier that identifies a VM implementing the third service that requested authentication of the token, the token including the GUID assigned to the VM implemented by the source container.

In this aspect, additionally or alternatively, the dependency map generator may be further configured to determine the second service may be a pass-through service by comparing a timeframe of a DNS request from the second service requesting the IP address of the third service and a DNS response to this DNS request, a timeframe of a DNS request from the first service requesting the IP address of the second service, and a timeframe of the authentication request from the third service.

In this aspect, additionally or alternatively, the service-to-service dependency map may be output to a downstream computing program for processing, and the downstream program may be selected from the group consisting of a recovery program configured to compute a fault diagnosis program configured to generate a fault diagnosis based on the service-to-service dependency map as the processing operation, a recovery schedule for restarting services based on the service-to-service dependency map as the processing operation, a geographic compliance program configured to determine a geographic scope of a location of servers executing the first service and the second service as the processing operation, and a threat identification program configured to scan system logs of servers executing dependent services of the first service to identify a presence or effect or a virus or malware on the servers as the processing operation.

In this aspect, additionally or alternatively, the computing system may be configured to display the service-to-service dependency map within a graphical user interface, and a number of levels of dependency displayed may be determined by input from a user. In this aspect, additionally or alternatively, the service-to-service dependency map may be mapped over a lifetime of the identified first service, including build time, test time, deployment time, run time, and retirement.

Another aspect provides a computing method for a distributed computing system. The method may comprise identifying a dependency of a first service executed on a first virtual machine (VM) at a first server of the distributed computing system, on a second service executed on a second VM at a second server of the distributed computing system, via one or more domain name service (DNS) logs of a DNS server of the distributed computing system; identifying a dependency of the first service on a third service executed at a third server of the distributed computing system, via one or more token authentication logs of an authentication server of the distributed computing system; and generating a service-to-service dependency map including the dependency between the first service and the second service, and the dependency between the first service and third service.

In this aspect, additionally or alternatively, the second service may be a pass-through service that passes a request from the first service to a third service for processing. In this aspect, additionally or alternatively, the method may further comprise identifying a dependency between the second service and the third service based on the one or more DNS logs; and including the dependency between the second service and third service in the service-to-service dependency map.

In this aspect, additionally or alternatively, identifying the dependency of the first service on the second service may be accomplished at least in part by: identifying, within a DNS request log of the one or more DNS logs, a DNS request log record for a DNS request, the DNS request log record including a source identifier associated with the first service and a source IP address associated with the first service, wherein the source identifier may be determined to be associated with the first service by identifying a virtual machine (VM) implementing the first service on a computer that has been assigned the source IP address, wherein the source identifier may be a source virtual machine identifier or a source container identifier; identifying, within the DNS request log record a destination fully qualified domain name requested by the DNS request; identifying within a DNS response log record for a DNS response to the DNS request, a destination IP address for the destination fully qualified domain name; and identifying the second service as being associated with the destination IP address.

In this aspect, additionally or alternatively, identifying the dependency of the first service on the third service may be accomplished at least in part by identifying an authentication log record for an authentication request from the third service to authenticate a token of the first service received via the second service.

In this aspect, additionally or alternatively, identifying the dependency of the first service on the third service may be at least in part accomplished by: identifying, within a token generation log of the one or more authentication logs, a token generation log record, the token generation log record including the source container identifier and a globally unique identifier (GUID) as the token, wherein the source container identifier may be associated with a source container implementing a virtual machine (VM) that has been assigned the GUID associated with the token; identifying in the authentication logs a token authentication request from the second service for a token associated with the first service; and identifying, within an authentication response log of the one or more authentication logs, an authentication response log record for an authentication response, the authentication response log record including a destination identifier that identifies a VM implementing the third service that requested authentication of the token, the token including the GUID assigned to the VM implemented by the source container.

In this aspect, additionally or alternatively, the method may further comprise determining the second service may be a pass-through service by comparing a timeframe of a DNS request from the second service requesting the IP address of the third service and a DNS response to this DNS request, a timeframe of a DNS request from the first service requesting the IP address of the second service, and a timeframe of the authentication request from the third service.

In this aspect, additionally or alternatively, the method may further comprise outputting the service-to-service dependency map to a downstream computing program for processing, wherein the downstream program may be selected from the group consisting of a recovery program configured to compute a recovery schedule for restarting services based on the service-to-service dependency map as the processing operation, a fault diagnosis program configured to generate a fault diagnosis based on the service-to-service dependency map as the processing operation, a geographic compliance program configured to determine a geographic scope of a location of servers executing the first service and the second service as the processing operation, and a threat identification program configured to scan system logs of servers executing dependent services of the first service to identify a presence or effect or a virus or malware on the servers.

In this aspect, additionally or alternatively, the method may further comprise mapping the service-to-service dependency map over a lifetime of the identified first service, including build time, test time, deployment time, run time, and retirement.

Another aspect provides a computing method for a distributed computing system. The method may comprise identifying a dependency of a first service executed on a first virtual machine (VM) at a first server of the distributed computing system, on a second service executed on a second virtual machine at a second server of the distributed computing system, via one or more domain name service (DNS) logs of a DNS server of the distributed computing system, wherein the second service may be a pass-through service that passes a request from the first service to a third service for processing; identifying a dependency of the first service on the third service executed at a third server of the distributed computing system, via one or more token authentication logs of an authentication server of the distributed computing system; identifying a dependency between the second service and the third service based on the one or more DNS logs; generating a service-to-service dependency map including the dependency between the first service and the second service, the dependency between the second and third service, and the dependency between the first service and third service; outputting the service-to-service dependency map as input to a downstream program executed on a server of the distributed computing system; performing, via the downstream program, a processing operation based on the service-to-service dependency map; and outputting a processing result of the processing operation.