Patent ID: 12242870

DETAILED DESCRIPTION

The following detailed description is directed to technologies for shortening service interruptions. As used herein, a “service interruption” is a period of time in which functionality associated with an application, a site, a service, or some other component is limited and/or is indicated to become limited (e.g., the functionality is unavailable, slow to respond, . . . ). Utilizing technologies described herein, an incident response system is configured to detect incidents associated with a service interruption that impacts and/or may impact customers of a service provider network and provides information, tools, and/or other functionality to allow users of an entity to troubleshoot and mitigate the service interruption.

In some configurations, an incident service executing within a service provider network is used to detect incident(s) that may indicate a service interruption associated with a customer of the service provider network and perform operations to assist in resolving the service interruption. These incidents may involve loss of network connectivity to an application provided by a customer and hosted by the service provider, failure of computing resources utilized by the application, slower response times, and the like. According to some examples, the incident service may identify resources (e.g., computing resources, individuals, . . . ) to triage and remediate the service interruption. For instance, the incident service may provide information to one or more users of a customer experiencing a service interruption to assist in guiding the user(s) to address one or more problems to assist in resolving the service interruption. The information may include one or more actions/tasks to be performed (e.g., re-start a service, configure one or more services, perform some other task(s), . . . ).

In some configurations, the incident service may identify actions to perform and provide one or more runbooks that may include manual tasks and/or automated tasks to assist in resolving the service interruption. Some/all of the actions may be generated by the service provider network based on previous actions performed (by one or more users), and/or from actions determined from other sources (e.g., one or more users). As used herein, a “runbook” identifies one or more predefined procedures and/or actions that are directed at achieving a specific outcome. Generally, a runbook identifies the information to successfully perform a procedure. The runbook may include instructions for the customer to manually perform and/or instructions that are automatically executed.

In some configurations, the incident service may detect an incident associated with a service interruption in response to a triggering event. As used herein, a “triggering event” is an occurrence of one or more specified events/alarms. For example, a triggering event may be identified from an occurrence of one or more conditions within the service provider network (e.g., loss of network connectivity that prevents one or more customers of the entity to access one or more services provided by the service provider network). An alarm may be associated with one or more conditions (e.g., bandwidth, bandwidth and memory, memory, memory plus one or more other conditions, and the like).

When a triggering event is detected that indicates the occurrence of an incident that has customer impact, the incident service may identify and send a message to individuals that are assigned to resolve the service interruption. According to some configurations, after identifying a service interruption, the incident service identifies the individuals associated with the customer to resolve the service interruption in a timely manner. For instance, the incident service may identify one or more subject matter experts to address a particular incident instead of first notifying a user that does not have the knowledge to address the particular incident causing the service interruption.

The incident service may also obtain relevant information from one or more services provided by the service provide network to present to one or more users of the customer experiencing the service interruption. For instance, the incident service may provide a user interface (UI), such as a graphical UI (GUI), that displays information relating to the service interruption in response to an individual assigned to resolve the service interruption selecting an option to display the UI. The UI may be utilized by identified individuals of the customer to perform steps indicated by one or more runbooks, customize a runbook, display information for various metrics (e.g., network connectivity, throughput, processor utilization, memory utilization, storage information, . . . ). In some examples, the UI may also utilize one or more services/components provided by a third party that is not part of the service provider network.

Utilizing the techniques described herein, the incident response system may assist an entity to reduce costs associated with service interruptions by shortening the duration of service interruptions. The incident service may quickly identify incidents associated with service interruptions and provide information to individuals identified to address the incident to resolve the incident in a timely and efficient manner. By resolving service interruptions quickly, computing resources are more efficiently utilized by the service provider network. Additional details regarding the various components and processes described briefly above for shortening service interruptions will be presented below with regard toFIGS.1-8.

It should be appreciated that the subject matter presented herein can be implemented as a computer process, a computer-controlled apparatus, a computing system, or an article of manufacture, such as a computer-readable storage medium. While the subject matter described herein is presented in the general context of program modules that execute on one or more computing devices, those skilled in the art will recognize that other implementations can be performed in combination with other types of program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types.

Those skilled in the art will also appreciate that aspects of the subject matter described herein can be practiced on or in conjunction with other computer system configurations beyond those described herein, including multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, handheld computers, personal digital assistants, e-readers, mobile telephone devices, tablet computing devices, special-purposed hardware devices, network appliances, and the like. The configurations described herein can also be practiced in distributed computing environments, where tasks can be performed by remote computing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote storage devices.

In the following detailed description, references are made to the accompanying drawings that form a part hereof, and that show, by way of illustration, specific configurations or examples. The drawings herein are not drawn to scale. Like numerals represent like elements throughout the several figures (which might be referred to herein as a “FIG.” or “FIGS.”).

FIG.1is a software and network architecture diagram showing aspects of the configuration and utilization an incident response system102for preventing, shortening, and/or minimizing service interruptions. It is to be appreciated that the environment100is merely illustrative and that the various configurations disclosed herein can be utilized in many different types of computing environments.

To provide the incident service130and the other functionality disclosed herein, the incident response system102may include one or more servers110. The servers110can execute software components to provide the services described herein, including incident service130functionality and different available services120provided by a service provider and/or some other entity. The software components can execute on a single server110or in parallel across multiple servers in the incident response system102. In addition, a software component can consist of subcomponents executing on different servers110or other computing devices in the incident response system102. Various components can be implemented as software, hardware, or any combination of the two. In this regard, it is to be appreciated that the incident response system102shown inFIG.1has been simplified for discussion purposes and that many additional software and hardware components can be utilized.

A user122of the incident response system102can utilize the incident service130, via a computing device114or some other input device, to access the incident response system102through a network112. According to some configurations, the computing device114may be configured to understand natural language voice commands and complete tasks for the user, such as tasks related to replication as described herein. As illustrated, a user may interact with the incident service130through a user interface140. In some examples, the user122is a customer of a service provider network.

The computing device114may be one or more devices, such as but not limited to a smart phone, a smart watch, a personal computer (“PC”), desktop workstation, laptop computer, tablet computer, notebook computer, personal digital assistants (“PDA”), electronic-book reader, game console, set-top box, consumer electronics device, server computer, a telephone, a telephone conferencing device, video conferencing device, or any other type of computing device capable of connecting to the network112and communicating with the incident response system102. In other configurations, the computing device114may be configured to communicate with one or more other devices to receive commands from users and/or perform processing related to functionality of the incident response system102.

As illustrated, the computing device114, or some other device or component, may couple with an incident response system102over a network112. The network112may represent an array or wired networks, wireless networks (e.g., Wi-Fi), or combinations thereof. The incident response system102may generally refer to a network-accessible platform implemented as a computing infrastructure of processors, storage, software, data access, and so forth that is maintained and accessible via the network112, such as the Internet. These services may not require end-user knowledge of the physical location and configuration of the system that delivers the services. Common expressions associated with these remote services, such as the incident response system102, include “on-demand computing”, “software as a service (SaaS)”, “platform computing”, “network accessible platform”, and so forth.

As illustrated, the incident response system102may comprise one or more network-accessible resources, such as servers110. These resources comprise one or more processors and computer-readable storage media executable on the processors. In some configurations, the users122may be identified and/or authenticated before interacting with the computing device114that is associated with the incident response system102.

The network112can be a local-area network (“LAN”), a wide-area network (“WAN”), the Internet, or any other networking topology known in the art that connects the user devices to the incident response system102. The user122can use an application (not shown) executing on computing device114that provides user interface140to access and utilize the incident service functionality provided by the servers110. In some examples, the application is a web browser application (not shown). Generally, a web browser application exchanges data with the servers110in the incident response system102using the hypertext transfer protocol (“HTTP”) over the network112. The application might also be a stand-alone client application configured for communicating with the servers110.

The application can also utilize any number of communication methods known in the art to communicate with the incident response system102and/or the servers110across the network112, including remote procedure calls, SOAP-based web services, remote file access, proprietary client-server architectures, and the like. According to some configurations, the application may provide a user interface140that can be utilized by the user122to configure settings associated with the incident service130and/or the computing device114. Typically, a user122interacts with the computing device114using user interface140.

As discussed above, an incident service130is configured to shorten and/or minimize service interruptions that impact access to customer functionality hosted by the service provider network. In some examples, a user, such as user122, may utilize the incident response system102to receive information and perform actions relating to addressing a service interruption. According to some configurations, the user122may view incident data152B, monitoring data154B, runbook data156B, and/or other data, via a user interface, such as user interface140. In some examples, the incident service130provides to a user computing device114incident data152B, monitoring data154B, runbook data156B, and/or other data for display within user interface140, such as a graphical user interface (“GUI”).

Incident configuration data152may include data for configuring one or more metrics/alarms utilized for detecting an incident that is associated with a service interruption. In some configurations, the incident service130may be configured to detect changes in network connectivity associated with a customer. For example, the incident service may utilize a monitoring functionality provided by one or more of the available services120to detect when at least a portion of functionality hosted by the service provider is not accessible by users of the customer. In other examples, the customer may utilize UI140to configure incident detection utilizing one or more UI elements144. For example, an authorized user122may configure various alarms (e.g., a location is unreachable, a component stops working, a storage location is full, memory use is exceeding some predefined value, a computing resource hosting functionality utilized by the customer has experienced a failure, and the like). The user122might also configure the alarms based on other criteria, such as a number of orders within a time period below a specified threshold, a number of uses of particular functionality, and the like.

In some examples, the incident service130may provide recommendations via the UI140for configuring the metrics to monitor and the alarms to set to detect an incident that is associated with a service interruption. For example, the incident service130may provide a list of suggested alarms to the user122to utilize to detect an incident relating to a service interruption. The incident configuration data152may be stored within the data store150as incident configuration data152A, or some other data store, and at least a portion of the incident configuration data152A may be provided as incident configuration data152B for display within the UI140.

As discussed above, the user122may configure different metrics to be monitored. For instance, the user122may configure monitoring of reachability between/to different network resources, monitoring of health metrics of one or more computing resources (e.g., CPU utilization, data transfer, disk usage, memory usage, bandwidth utilized, latency, and the like . . . ). In some configurations, the monitoring data154may include metrics that identify a performance of a computing resource and/or a network element. For example, the incident service130may instruct a monitoring service of the available services120to monitor, collect and store metrics data from various network resources, applications, and services operating in the incident response system102.

The runbook data156includes data relating to one or more runbooks. As discussed briefly above, the incident service130may provide one or more runbooks that may include data identifying manual tasks and/or automated tasks to assist in resolving the service interruption. For example, the runbook data156may include predefined procedures and/or actions that are directed at resolving the service interruption. In some configurations, the incident service130displays a graphical representation of a runbook within the UI140such that the user122may readily identify what actions have occurred, are currently being performed, and other actions yet to be performed.

In some configurations, the incident service130may detect an incident associated with a service interruption in response to a triggering event. For example, in some examples, the incident service130receives data from one or more of the available services120indicating the triggering event. In other examples, the user122may manually trigger an event (e.g., selection of a specified event) to receive current information (e.g., metrics data) related to a specified event. When a triggering event (e.g., loss/reduced network connectivity, loss of computing resources utilized by the customer application, reduced performance by the application, and the like) is detected (e.g., by monitoring one or more metrics associated with the execution of the application) that reflects the occurrence of an incident that has customer impact, the incident service130may obtain relevant information from one or more services provided by the service provide network to present to one or more users of the customer experiencing the service interruption.

For instance, the incident service may provide a user interface (UI), such as a graphical UI (GUI)140that displays information relating to the service interruption. The UI140may be utilized by the customer to perform steps indicated by one or more runbooks, customize a runbook, display information for various metrics (e.g., network connectivity, throughput, processor utilization, memory utilization, storage information, . . . ). In some examples, the UI may also utilize one or more services/components provided by a third party that is not part of the service provider network.

According to some examples, the user interface140includes selectable UI elements144that allow a user122to select, configure, and/or specify different data to be displayed, as well as perform one or more actions (e.g., from a runbook) to assist in addressing a service interruption. For instance, in the example presented inFIG.1, the UI elements144may include UI elements for configuring which incident data152B, monitoring data154B, runbook data156B, and/or other data to display, UI elements144for performing one or more actions (e.g., one or more steps from a runbook), UI elements144for contacting other users regarding the service interruption, and the like.

As discussed above, the incident service130may determine one or more actions to perform based on actions that have been performed to respond to other service interruptions. For example, the incident service130may identify that performing a sequence of actions has resolved a similar service interruption in the past. In some examples, the incident service130may identify previous service interruptions that are similar to a current service interruption for a single customer and/or other customers. For instance, while a particular service interruption may not have occurred for one customer until now, the incident service130may identify that similar service interruptions have occurred for other customers of the service provider network. The incident service130may provide actions that were performed to resolve the similar service interruptions to the customer. These actions may/may not be combined with other actions.

According to some examples, the incident service130may expose an incident Application Programming Interface (API)132. In some configurations, functionality provided by the incident service130may be accessed using the incident API132that may be a Web API. The incident API132might also be used to request data from one or more data stores such as data store150, services120, and/or other applications, and the like. Some exemplary APIs include but are not limited to specifying parameters to monitor within one or more of the services120, specifying what data to display within the UI140, performing one or more actions (e.g., configuring a parameter, rolling back a deployed service/application to a different version, and the like).

In some configurations, the incident service130may access other available services120to obtain data that may be used by the incident service130. For example, the incident service130may access a monitoring service, an event-driven service, and a queue service (SeeFIG.2and related discussion). According to some examples, the incident service130stores data associated with detection of the service interruption as well as resolving the service interruption. For instance, the incident service130may store all or a portion of the monitoring data154, data associated with the actions performed, messages exchanged during the service interruption and/or before/after the service interruption, data associated with reassignment of actions, and the like. In some configurations, the different data that is stored is correlated such that when a user122views data associated with a service interruption, the user122may easily view the relevant data for a particular time (e.g., seeFIG.3E). According to some examples, the incident service130takes a “snapshot” of data, such as monitoring data, periodically (e.g., 1 second, 2 seconds, 10 seconds, . . . ) and stores this data for later use and retrieval. As such, instead of a user122having to search for data relating to a service interruption, the user122may access the relevant data from a single user interface140. Further, unlike other monitoring data that may be discarded, the monitoring data154, as well as other data, may be stored for a specified period of time by the user122, or some other authorized user. Additional details regarding the various processes described above with regard toFIG.1will be provided below with regard toFIGS.2-8.

FIG.2is a software and network architecture diagram showing aspects of an incident response system102that utilizes various services120associated with a service provider network to facilitate shortening and/or minimizing a service interruption. It is to be appreciated that the environment200is merely illustrative and that the various configurations disclosed herein can be utilized in many different types of computing environments.FIG.2is similar toFIG.1but provides more details of the incident response system102.

As illustrated, incident response system102includes incident service130, monitoring service120A, event-driven service120B, messaging service120C, and queue service120D. The incident service130may communicate with the services120using one or more Application Programming Interfaces (APIs), such as incident API132exposed by the incident service130. In some examples, each service may expose one or more APIs (not shown) that can be used by a service, or some other component or application, to access functionality and/or data provided by the service.

In some examples, the incident service130utilizes a monitoring service120A to monitor various metrics associated with one or more of the available services120. For example, the monitoring service120A monitors one or more of the available services120to identify incidents/problems that may indicate a service interruption (e.g., network connectivity, health data, other data), which may correspond to monitoring data154. The monitoring service120A may provide the monitoring data154related to detecting a service interruption to the incident manager210, the incident service130and/or some other computing device or component. The incident service120A, or some other component or user, may configure the monitoring service120A to detect triggering events indicating a service interruption by monitoring metrics/events based on incident data152.

When a triggering event is detected, the incident manager210may attempt to identify one or more users122to which to engage to assist in resolving the service interruption, and/or perform other actions without user interaction. As discussed above, the customer may identify the users122that are assigned to resolve the service interruption. According to some configurations, the incident manager utilizes a messaging service120C to send an electronic message (e.g., a text message, email, a notification via a mobile application, a desktop application, and/or a website, . . . ) to the users assigned to resolve the service interruption.

The incident service may also perform other actions in an attempt to quickly resolve the service interruption. For example, the incident service130may restart a service, device, component, or perform some other action (e.g., adding/replacing a computing resource utilized by the application, changing an amount of memory allocated to the application, . . . ) in a fully or partially automated manner, and possibly without requiring any input from a user associated with the entity experiencing the service interruption. As another example, the incident service130may automatically change a version of an service/application being utilized. In case of service interruptions that cannot be automatically resolved, the incident service130may present detailed incident data152B, and/or other diagnostic information to the user (e.g., using UI140).

According to some configurations, the event-driven service120B is configured to detect a triggering event, such as an alarm that is triggered in response to the monitoring service identifying a change in one or more parameters that are monitored by monitoring service120A. In response to the triggering event, the incident service130may generate a message204that is provided to the messaging service120C and/or the queue service120D for delivery to the user122via the UI140.

The queue service120D can be a managed message queuing service that provides messaging for applications and/or services, such as incident service130. The queue service120D helps to remove the complexity and overhead associated with managing and operating message-oriented middleware and empowers developers to focus on other tasks. As illustrated, queue service120D is configured to store messages utilized by the incident service130to assist in addressing a service interruption. As illustrated, the queue service120D queues messages204generated by the event-driven service120B.

FIG.3Ais a block diagram showing an illustrative graphical user interface300that may be utilized to configure incident detection to identify service interruptions. In some examples, the user122configures alarms to identify incidents associated with service interruptions via a graphical user interface (GUI). In other examples, the user may use a command line interface, or utilize an Application Programming Interface (API). In some instances, the instance service130may provide data for displaying a GUI to a display associated with a user computing device (not shown).

In the example illustrated inFIG.3A, GUI300shows user interface (UI) elements related to identifying and configuring alarms that may be utilized to detect one or more incidents that are associated with a service interruption. More or fewer UI elements144may be included within GUI300. As illustrated, the GUI300includes configuration UI element302that allows a user122to configure alarms306and select from suggested alarms310. The GUI300also shows a search UI element304to enter one or more search terms to locate alarms. According to some configurations, the user122may select incident detection UI element302A to configure one or more alarms that are used as a triggering event to identify a service interruption.

In the current example, the alarms UI element306displays alarms currently being utilized to detect a service interruption. The user122may select one or more of the alarms (e.g., as indicated by the checkmark next to alarm1, alarm2associated with service1) and then edit the alarm using edit alarm UI element308B, or delete the alarm using the delete alarm UI element308C. The user122may also add a new alarm using add alarm UI element308A.

According to some configurations, the display of the GUI300may also include suggested alarms UI element310that includes alarms identified by the incident service130, or some other component, or entity as being related to detecting an incident that indicates a service interruption. In this way, the user122may more easily add relevant alarms without having to search for available alarms. The user122may select one or more of the suggested alarms (e.g., as indicated by the checkmark next to alarm5) and then edit an alarm using edit alarm UI element312B, or delete the alarm using the delete alarm UI element312C. The user122may also add a new alarm using add alarm UI element312A. The user122may use the save UI element314to save changes made to the alarms or exit if no changes are desired.

FIG.3Bis a block diagram showing an illustrative graphical user interface that may be utilized to configure users/groups that may be identified to respond to a detected service interruption. As discussed above, the user122may identify one or more users to be utilized in responding to a detected service interruption.

In the example illustrated inFIG.3B, GUI320shows user interface (UI) elements related to configuring one or more groups. A group may include one or more users. More or fewer UI elements may be included within GUI320. As illustrated, the GUI320configuration UI elements302that allows a user122to configure the users to respond to a service interruption, a search UI element304to enter one or more search terms to locate users. A group UI element322may also be provided that displays group information. The group information may include information about users122within a group (e.g., names, office locations, phone numbers, availability information, . . . ). The edit group UI element324B when selected is used to edit group information for a selected group. For instance, a user122may select a group (e.g., group2as indicated by the checkmark) to edit information about the group. The user122may delete a group using the delete group UI element324C or add another group using the add group UI element324A. The user122may use the save UI element314to save changes made to the alarms or exit if no changes are desired.

FIG.3Cis a block diagram showing an illustrative graphical user interface that may be utilized to configure runbook data156to display to respond to a detected service interruption. As discussed above, the user122may identify one or more runbooks to utilize when responding to a service interruption.

In the example illustrated inFIG.3C, GUI330shows user interface (UI) elements related to configuring runbook data156. More or fewer UI elements may be included within GUI330. As illustrated, the GUI330configuration UI elements302that allows a user122to configure runbook information utilized to respond to a service interruption, and a search UI element304to enter one or more search terms to locate runbook information. An actions UI element332may also be provided that displays one or more actions to perform for a particular step within the runbook. In the current example, the actions UI element332illustrates that actions to perform for step3of the runbook336. The edit action UI element334B when selected is used to edit the action(s) to perform for the selected step. For instance, a user122may select a step (e.g., step3as indicated by the larger border and hashing fill) to edit information about the action(s) for step3. The user122may delete an action using the delete action UI element334C or add another action using the add action UI element334A. The user122may use the save UI element314to save changes made to the alarms or exit if no changes are desired.

GUI330also illustrates a runbook UI element336that presents the action items/tasks that are associated with a runbook. As illustrated, the runbook UI element336include a list of 8 action items. More or fewer action items may be included within a runbook. In some configurations, the different action items are selectable.

FIG.3Dis a block diagram showing an illustrative graphical user interface that may be utilized to configure data to display to respond to a detected service interruption. As discussed above, the user122may identify various information to utilize when responding to a service interruption.

In the example illustrated inFIG.3D, GUI340shows user interface (UI) elements related to configuring reports and/or data that may be displayed in the UI140when responding to a service interruption. More or fewer UI elements may be included within GUI340. As illustrated, the GUI340configuration UI elements302that allows a user122to configure the UI140utilized to respond to a service interruption, and a search UI element304to enter one or more search terms to locate information associated with content included in the GUI340. The timeline UI element342may also be provided that displays a timeline of actions taken during responding to a service interruption. As actions are performed, the incident service130may update the timeline UI element342.

The chat box UI element344may be provided that shows messages exchanged while responding to the service interruption. In this way, users assigned to respond to the service interruption may see messages and other relevant information within a same UI140instead of having to utilize a variety of different applications/programs. The metrics UI element346shows monitoring data154that is associated with the alarm(s) that caused the triggering event. For example, the monitoring data may include network connectivity data when an alarm was triggered indicating a loss of network connectivity. The tasks UI element348may be provided to display the list of tasks that have been completed/as well as tasks that have not been completed. The runbook UI element336illustrates the steps within the selected runbook. In the current example, the runbook UI element336shows that step4is the current step to utilize.

FIG.3Eis a block diagram showing an illustrative graphical user interface that may be utilized to view a selected time within different metrics data to respond to a detected service interruption. As discussed above, the user122may identify various information to utilize when responding to a service interruption.

In the example illustrated inFIG.3E, GUI350shows user interface (UI) elements related to viewing reports and/or data within the UI140when responding to a service interruption. More or fewer UI elements may be included within GUI350. As illustrated, the GUI350configuration UI elements302that allows a user122to configure the UI140utilized to respond to a service interruption, and a search UI element304to enter one or more search terms to locate information associated with content included in the GUI340. The timeline UI element352may also be provided that displays a timeline that includes different times associated with responding to a service interruption. As illustrated, the timeline UI element352includes different times (T1, T2, . . . . TN) that a user may select. In the current example, the user has selected to view data associated with time T4as indicated by indicator3544.

According to some examples, in response to selection of a time on the timeline UI element352, the metrics UI element346may be updated to display monitoring data154at the selected time. For instance, monitoring data154A shows a graph that has an indicator356A at the selected time T4, monitoring data154B shows a graph that has an indicator356B at the selected time T4, monitoring data154C shows a graph that has an indicator356C at the selected time T4, monitoring data154D shows a graph that has an indicator356D at the selected time T4, and monitoring data154N shows a graph that has an indicator356N at the selected time T4.

In this way, users assigned to respond to the service interruption may see the monitoring data154at the same point in time within the UI140instead of having to manually select different views for the specified time. As discussed above, the metrics UI element346shows monitoring data154that is associated with the alarm(s) that caused the triggering event. For example, the monitoring data154may include network connectivity data when an alarm was triggered indicating a loss of network connectivity.

FIGS.4and5are flow diagrams showing illustrative routines400, and500, for shortening and/or minimizing service interruptions, according to examples disclosed herein. It should be appreciated that the logical operations described herein with respect toFIG.4,FIG.5, and the other FIGS., can be implemented (1) as a sequence of computer implemented acts or program modules running on a computing system and/or (2) as interconnected machine logic circuits or circuit modules within the computing system.

The implementation of the various components described herein is a matter of choice dependent on the performance and other requirements of the computing system. Accordingly, the logical operations described herein are referred to variously as operations, structural devices, acts, or modules. These operations, structural devices, acts, and modules can be implemented in software, in firmware, in special purpose digital logic, and any combination thereof. It should also be appreciated that more or fewer operations can be performed than shown in the FIGS. and described herein. These operations can also be performed in parallel, or in a different order than those described herein. Some or all of these operations can also be performed by components other than those specifically identified.

FIG.4is a flow diagram showing an illustrative routine400for shortening and/or minimizing service interruptions. At410, a user122configures incident detection to identify a service interruption. As discussed above, the incident service130may utilize incident configuration data152to configure one or more triggering events to identify a service interruption. According to some examples, the incident configuration data152is utilized by a monitoring service120A to configure monitoring of one or more available services120. For example, the incident configuration data152may specify that a triggering event occurs in response to a value of one or more metrics (e.g., network connectivity metrics, health metrics, . . . ) indicating an occurrence of a service interruption. For instance, the user122may configure monitoring of reachability between/to different network resources, monitoring of health metrics of one or more computing resources (e.g., CPU utilization, data transfer, disk usage, memory usage, bandwidth utilized, latency, and the like . . . ). In some configurations, the monitoring data154may include metrics that identify a performance of a computing resource and/or a network element.

At420, monitoring is performed. As discussed above, the incident service130may utilize a monitoring functionality provided by one or more of the available services120, such as monitoring service120A, to detect when at least a portion of functionality hosted by the service provider network is not accessible by users accessing functionality associated with the customer. In some configurations, the incident service130may instruct the monitoring service120A to monitor, collect, and store metrics data from various network resources, applications, and services operating in the data store150, or some other memory, associated with the incident response system102.

At430, a decision is made as to whether a service interruption has occurred. As discussed above, the incident service130may receive a notification of a triggering event from another service120, or some other component that indicates occurrence of a service interruption. In other examples, the incident service130may receive or access monitoring data154to determine whether a service interruption has occurred. When a service interruption has not occurred, the routine returns to420. When a service interruption has occurred, the routine moves to440.

At440, one or more users that may be utilized to assist in resolving the service interruption are determined. As discussed above, the incident service130may access data, such as incident configuration data152, and/or other data, to determine individuals associated with the entity to resolve the service interruption. In some examples, the user122may identify one or more users to assist in resolving the service interruption utilizing the UI140.

At450, actions are generated that may be performed to assist in resolving the service interruption. As discussed above, in some examples, the incident service130may generate and/or access runbook data156that includes data relating to one or more runbooks. In some examples, the runbook data156identifies manual tasks and/or automated tasks to assist in resolving the service interruption. For instance, the runbook data156may include predefined procedures and/or actions that are directed at resolving the service interruption.

At460, data relating to the service interruption is provided. As discussed above, the incident service130may provide data, such as incident configuration data152, monitoring data154, runbook data156, and/or other data for display within the UI140such that the user122may readily identify what actions have occurred, are currently being performed, and other actions yet to be performed. In some examples, the incident service130transmits an electronic message to the identified users that are assigned to resolve the service interruption that indicates the occurrence of the service interruption.

At470, one or more actions are performed when determined. As discussed above, a user may utilize the UI140to perform an action (e.g., by entering or selecting a command from the runbook), and/or the incident service130may automatically perform a command. For instance, the incident service130may restart one or more processes/services in an attempt to address the service interruption.

FIG.5is a flow diagram showing an illustrative routine500for performing one or more actions to shorten and/or minimize service interruptions, according to examples disclosed herein.

The routine500begins at510, where actions are displayed that are directed at resolving the service interruption. As discussed above, the incident service130may provide data, such as runbook data156B, for display within a UI140. In some examples, the runbook data156is displayed within the UI140as a graph of different ordered actions.

At520, an action to perform is identified. As discussed above, an action may be manually performed and/or automatically performed by the incident service130. In some examples, the user122identifies an action to perform by making a selection of a UI element144(e.g., a box with the desired action).

At530, the action is performed. As discussed above, the incident service130may cause the action the action to be performed in response to the selection, or in response to some other event or condition.

At540, the display is updated to reflect the performed action. As discussed above, the incident service130may update the UI140to reflect the performed action (e.g., changing an indictor within the UI140). In some examples, the incident service130changes a display of an action of the runbook to reflect the performed action.

At550, a decision is made as to whether there are more actions to perform. As discussed above, when the incident manager160determines that there are more actions, the process500returns to530. When there are not more actions to perform, the process flows to560.

At560, data related to the resolving of the service interruption may be provided. As discussed above, the incident service130may provide data relating to the actions performed, when actions were performed, messages associated with resolving the service interruption between users assigned to resolve the interaction, and the like.

FIG.6is a system and network diagram that shows an illustrative operating environment for the configurations disclosed herein that includes an incident response system102that can be configured to provide the functionality described above. As discussed above, the incident response system102can execute network services that provide computing resources for implementing the functionality disclosed herein. The computing resources implemented by the incident response system102can be data processing resources, such as virtual machine (“VM”) instances, data storage resources, networking resources, data communication resources, network services, and other types of resources.

The computing resources utilized can be general-purpose or can be available in a number of specific configurations. For example, data processing resources can be available as physical computers or VM instances in a number of different configurations. The VM instances can be configured to execute applications, including web servers, servers, media servers, database servers, some or all of the network services described above, and/or other types of programs. Data storage resources can include file storage devices, block storage devices, and the like. The incident response system102can also include and utilize other types of computing resources not mentioned specifically herein.

As also discussed above, the computing resources provided by the incident response system102are enabled in one implementation by one or more data centers604A-604D (which might be referred to herein singularly as “a data center604” or collectively as “the data centers604”). The data centers604are facilities utilized to house and operate computer systems and associated components. The data centers604typically include redundant and backup power, communications, cooling, and security systems. The data centers604can also be located in geographically disparate locations. One illustrative configuration for a data center604that can be utilized to implement the technologies disclosed herein will be described below with regard toFIG.8.

The users can access the services provided by the incident response system102over a network602, which can be a wide area communication network (“WAN”), such as the Internet, an intranet or an Internet service provider (“ISP”) network or a combination of such networks. For example, and without limitation, a computing device600operated by a user or other user of the incident response system102, such as the computing device114, can be utilized to access the incident response system102by way of the network602. It should be appreciated that a local-area network (“LAN”), the Internet, or any other networking topology known in the art that connects the data centers604to remote users and other users can be utilized. It should also be appreciated that combinations of such networks can also be utilized.

FIG.7is a computing system diagram that illustrates examples for a data center604that can be utilized to implement the incident service130, other available services120, and the other functionality disclosed herein. The example data center604shown inFIG.7includes several server computers702A-702F (which might be referred to herein singularly as “a server computer702” or in the plural as “the server computers702”).

The server computers702can be standard tower, rack-mount, or blade server computers configured appropriately for providing various types of computing resources710for implementing the functionality disclosed herein. As mentioned above, the computing resources710provided by the data center604can be data processing resources such as VM instances or hardware computing systems, data storage resources, database resources, networking resources, and others. Some of the servers702can also be configured to execute network services712A-712-E, respectively, capable of instantiating, providing and/or managing the computing resources710A-710E.

The data center604shown inFIG.7also includes a server computer702F that can execute some or all of the software components described above. The server computer702F can also be configured to execute other components and/or to store data for providing some or all of the functionality described herein. In this regard, it should be appreciated that components or different instances of the services can execute on many other physical or virtual servers in the data centers604in various configurations.

In the example data center604shown inFIG.7, an appropriate LAN708is also utilized to interconnect the server computers702A-702F. The LAN708is also connected to the network602illustrated inFIG.6. It should be appreciated that the configuration of the network topology described herein has been greatly simplified and that many more computing systems, software components, networks, and networking devices can be utilized to interconnect the various computing systems disclosed herein and to provide the functionality described above. Appropriate load balancing devices or other types of network infrastructure components can also be utilized for balancing a load between each of the data centers604A-604D, between each of the server computers702A-702F in each data center604, and, potentially, between computing resources710in each of the data centers604. It should be appreciated that the configuration of the data center604described with reference toFIG.7is merely illustrative and that other implementations can be utilized.

FIG.8shows an example computer architecture for a computer800capable of executing program components for implementing the functionality described above. The computer architecture shown inFIG.8illustrates a conventional server computer, workstation, desktop computer, laptop, tablet, network appliance, e-reader, smartphone, or other computing device, and can be utilized to execute any of the software components presented herein.

The computer800includes a baseboard802, or “motherboard,” which is a printed circuit board to which a multitude of components or devices can be connected by way of a system bus or other electrical communication paths. In one illustrative configuration, one or more central processing units (“CPUs”)804operate in conjunction with a chipset806. The CPUs804can be standard programmable processors that perform arithmetic and logical operations necessary for the operation of the computer800.

The CPUs804perform operations by transitioning from one discrete, physical state to the next through the manipulation of switching elements that differentiate between and change these states. Switching elements can generally include electronic circuits that maintain one of two binary states, such as flip-flops, and electronic circuits that provide an output state based on the logical combination of the states of one or more other switching elements, such as logic gates. These basic switching elements can be combined to create more complex logic circuits, including registers, adders-subtractors, arithmetic logic units, floating-point units, and the like.

The chipset806provides an interface between the CPUs804and the remainder of the components and devices on the baseboard802. The chipset806can provide an interface to a RAM808, used as the main memory in the computer800. The chipset806can further provide an interface to a computer-readable storage medium such as a read-only memory (“ROM”)810or non-volatile RAM (“NVRAM”) for storing basic routines that help to startup the computer800and to transfer information between the various components and devices. The ROM810or NVRAM can also store other software components necessary for the operation of the computer800in accordance with the configurations described herein.

The computer800can operate in a networked environment using logical connections to remote computing devices and computer systems through a network, such as the network808. The chipset806can include functionality for providing network connectivity through a NIC812, such as a gigabit Ethernet adapter. The NIC812is capable of connecting the computer800to other computing devices over the network808. It should be appreciated that multiple NICs812can be present in the computer800, connecting the computer to other types of networks and remote computer systems.

The computer800can be connected to a mass storage device818that provides non-volatile storage for the computer. The mass storage device818can store an operating system820, incident programs822for providing functionality associated with the incident response system102, user interface140, and data, which have been described in greater detail herein. The mass storage device818can be connected to the computer800through a storage controller814connected to the chipset806. The mass storage device818can consist of one or more physical storage units. The storage controller814can interface with the physical storage units through a serial attached SCSI (“SAS”) interface, a serial advanced technology attachment (“SATA”) interface, a fiber channel (“FC”) interface, or other type of interface for physically connecting and transferring data between computers and physical storage units.

The computer800can store data on the mass storage device818by transforming the physical state of the physical storage units to reflect the information being stored. The specific transformation of physical state can depend on various factors, in different implementations of this description. Examples of such factors can include, but are not limited to, the technology used to implement the physical storage units, whether the mass storage device818is characterized as primary or secondary storage, and the like.

For example, the computer800can store information to the mass storage device818by issuing instructions through the storage controller814to alter the magnetic characteristics of a particular location within a magnetic disk drive unit, the reflective or refractive characteristics of a particular location in an optical storage unit, or the electrical characteristics of a particular capacitor, transistor, or other discrete component in a solid-state storage unit. Other transformations of physical media are possible without departing from the scope and spirit of the present description, with the foregoing examples provided only to facilitate this description. The computer800can further read information from the mass storage device818by detecting the physical states or characteristics of one or more particular locations within the physical storage units.

In addition to the mass storage device818described above, the computer800can have access to other computer-readable storage media to store and retrieve information, such as program modules, data structures, or other data. It should be appreciated by those skilled in the art that computer-readable storage media is any available media that provides for the non-transitory storage of data and that can be accessed by the computer800.

By way of example, and not limitation, computer-readable storage media can include volatile and non-volatile, removable and non-removable media implemented in any method or technology. Computer-readable storage media includes, but is not limited to, RAM, ROM, erasable programmable ROM (“EPROM”), electrically-erasable programmable ROM (“EEPROM”), flash memory or other solid-state memory technology, compact disc ROM (“CD-ROM”), digital versatile disk (“DVD”), high definition DVD (“HD-DVD”), BLU-RAY, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information in a non-transitory fashion.

As mentioned briefly above, the mass storage device818can store an operating system820utilized to control the operation of the computer800. According to examples, the operating system comprises the LINUX operating system or one of its variants. According to another configuration, the operating system comprises the WINDOWS® SERVER operating system from MICROSOFT Corporation. According to further configurations, the operating system can comprise the UNIX operating system or one of its variants. It should be appreciated that other operating systems can also be utilized. The mass storage device818can store other system or application programs and data utilized by the computer800.

In examples, the mass storage device818or other computer-readable storage media is encoded with computer-executable instructions which, when loaded into the computer800, transform the computer from a general-purpose computing system into a special-purpose computer capable of implementing the configurations described herein. These computer-executable instructions transform the computer800by specifying how the CPUs804transition between states, as described above. According to examples, the computer800has access to computer-readable storage media storing computer-executable instructions which, when executed by the computer800, perform the various processes described above with regard toFIGS.1-8. The computer800can also include computer-readable storage media for performing any of the other computer-implemented operations described herein.

The computer800can also include one or more input/output controllers816for receiving and processing input from a number of input devices, such as a keyboard, a mouse, a touchpad, a touch screen, an electronic stylus, or other type of input device. Similarly, an input/output controller816can provide output to a display, such as a computer monitor, a flat-panel display, a digital projector, a printer, or other type of output device. It will be appreciated that the computer800might not include all of the components shown inFIG.8, can include other components that are not explicitly shown inFIG.8, or can utilize an architecture completely different than that shown inFIG.8.

Based on the foregoing, it should be appreciated that technologies for shortening service interruptions have been described herein. Moreover, although the subject matter presented herein has been described in language specific to computer structural features, methodological acts, and computer readable media, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features, acts, or media described herein. Rather, the specific features, acts, and media are disclosed as example forms of implementing the claims.

The subject matter described above is provided by way of illustration only and should not be construed as limiting. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure. Various modifications and changes can be made to the subject matter described herein without following the example configurations and applications illustrated and described, and without departing from the true spirit and scope of the present invention, which is set forth in the following claims.