Bookmarks and performance history for network software deployment evaluation

Described are automatically generated software deployment bookmarks that maintain information regarding events and/or triggers with respect to a network software deployment operation. The bookmarks are accessible and useable for evaluating the progress and state of the deployment. For example, a bookmark may be generated when deployment to a client machine succeeds, when a software deployment package is created and/or distributed, and for network component and/or membership state changes. A set of bookmarks may be used to generate a curve indicative of the deployment progress over time, and may be compared to previous progress of deploying other network software to client computer systems that is based on maintained performance history data.

BACKGROUND

Information technology (IT) administrators need to monitor the deployment of software, including programs and patches, with respect to the deployment's state and progress. This includes checking whether a given deployment is on track or out of control, and/or is proceeding efficiently or not. In general, this is accomplished by the administrator analyzing the status messages reported by client machines, and is based upon the administrator's experience.

In a large enterprise, typically there are many computing machines, of various types, including desktops, laptops and mobile devices. Each of these machines typically reports to a central configuration server (e.g., a System Center Configuration Manager, or SCCM server) or the like through log streams (e.g. status messages or status events). As a result, in order to determine the progress of deployment and/or the significant events that occurred or are occurring with respect to that deployment, the administrator needs to manually analyze a substantial amount of raw data.

SUMMARY

Briefly, various aspects of the subject matter described herein are directed towards a technology by which network components and/or user interface interaction automatically generate bookmarks that maintain information regarding events and/or triggers with respect to a network software deployment operation. The bookmarks are maintained in a data store, and are accessible and useable for evaluating the progress and state of the network software deployment operation as well as for use in troubleshooting and diagnosing various issues.

Examples of bookmarks that may be generated include a bookmark corresponding to when a software deployment package is created, a bookmark corresponding to when a software deployment package is distributed to a distribution point, a bookmark corresponding to when a distribution policy is generated or changed, and/or a bookmark corresponding to an object change that affects deployment task status. Other examples include a bookmark corresponding to when a network component starts, stops or restarts, a bookmark corresponding to when membership of a collection of network members changes, a bookmark corresponding to when a network component is added, removed, updated or refreshed, and/or a bookmark corresponding to information regarding a maintenance task.

In one aspect, the set the bookmarks, which each record data regarding a network software deployment operation, may be accessed and used to generate a visual representation (e.g., a curve) indicative of progress over time of deploying network software to client computer systems. Performance history data related to one or more previous network software deployment operations may be accessed and used to generate a visual representation indicative of previous progress of deploying other network software to client computer systems.

DETAILED DESCRIPTION

Various aspects of the technology described herein are generally directed towards automatically recording network software deployment-related (e.g., events and trigger) information, in data referred to as bookmarks, throughout a network software deployment operation. In general, the bookmarks record and save the significant events and/or triggered information that affect the deployment, and bind the events with the time that they happened. Via the bookmarks, an administrator is able to evaluate (e.g., monitor and check) what happened and/or is happening in the deployment process, and thereby efficiently find any deployment-related issues, as well as help troubleshoot those issues.

Another aspect is related to performance history, which in general collects the raw data as the history of a deployment, and automatically generates performance reports and/or graphs via data aggregation and calculation. The automatic collection and analysis of the history helps an administrator to easily and efficiently evaluate a deployment progress, including the deployment progress relative to previous, similar deployments.

It should be understood that any of the examples herein are non-limiting. Indeed, an architecture is described as one example of how the various aspects of bookmarks and performance data history may be implemented, but this is only one suitable example architecture, and bookmarks and/or performance data history are not limited to any particular implementation/architecture. As such, the present invention is not limited to any particular embodiments, aspects, concepts, structures, functionalities or examples described herein. Rather, any of the embodiments, aspects, concepts, structures, functionalities or examples described herein are non-limiting, and the present invention may be used various ways that provide benefits and advantages in computing and networking in general.

FIG. 1shows a general example architecture into which bookmark and deployment performance data operations may be implemented;FIG. 2represents a general ordering of operations related to bookmarks. In general, an administrator creates a software deployment package comprising program software or a patch for distribution. When this occurs, an activity101for distribution (e.g., corresponding to a deployment specified on an administrator console102) may be generated by a provider104, as generally represented via the circled numeral one (1) inFIG. 2. For example, the administrator console102may include a user interface106to specify the distribution task, as well as view alerts108; the console may be coupled to the provider104via Windows Management Instrumentation (WMI) technology110. The activity101is maintained in a suitable data store, such as a database112.

An activity generally corresponds to information of a distribution task, including package, advertisement of the package to clients, and deployment information. Various factors related to distribution may include package size, number of distribution points, whether the deployment is mandatory or optional, and whether the deployment is scheduled or not (as well as scheduled in UTC time or local time). Other information may specify whether the deployment is to run from a distribution point or from the local machine, and retry behavior.

Each activity is an instance that maintains an association with the actual distribution task instance, as well as maintaining a historical summary of a distribution task's progress in a consistent manner for various types of distribution tasks. An activity also may be used to associate the administrator's expectation with a checkpoint of the progress, and evaluate the on-track or late status, and/or healthy or unhealthy status with the checkpoint. An activity also may track the progress of each step of the distribution process, as long as the status message provides information about which step resulted in the message.

Once created, the software deployment package is then distributed to a distribution point113, which is typically a file share server used for client machines1141-114nto get the package. The management point116generates new policy (e.g., an offer or advertisement) that describes the related information, such as to inform the clients that the new software package is ready, and thus may alternatively be referred to as (or work with) a policy provider, which may be implemented as a systems management server (SMS) executive. At some point thereafter, the client machines1141-114nconnect to the management point116to obtain the policy and know that there is a new patch/software program package update available for installation.

After retrieving the information from the policy, each client machine (e.g.,1141) connects to get the deployed software for installation. Each client machine sends a status message to a status manager118of the distribution point113to report the results, namely success or failure.

In general, a bookmark operation adds a bookmark120(e.g., a record in the database112) to save the information for every significant step, such as when the package is created, when the package is distributed to the distribution point113, when the policy is generated, when the components start, stop or restart, the success or failure, and so forth. Whenever an administrator changes an object (e.g., a package, advertisement, software update deployment) via the UI106, and the change affects the deployment task status, an appropriate bookmark will be logged via the provider104.

By way of example, consider that the administrator creates a package and attempts deployment to ten-thousand machines, and finds that thirty-five percent failed. After checking the bookmarks, the administrator may determine that the package was successfully created, and distributed to seven distribution point servers, with failure reported on three of the distribution point servers. The administrator may then check and fix the failed distribution point servers and redistribute the software to these three servers. The client machines coupled to those distribution point servers may then load and install the software at a later appropriate time.

Other components represented inFIGS. 1 and 2include a collection evaluator119, which in general updates collection members (systems/network components that are involved in a deployment) by a schedule or by a UI trigger. Note that systems are regularly added to a network, while others become unhealthy, and so forth. The collection evaluator119adds a bookmark (e.g.,1201,FIG. 2) into the database112for such member-related updates. Note that the policy provider obtains collection member change information and when there is a change, updates the policy for every member, as this affects deployment; the policy provider also adds a corresponding bookmark into the database112. Thus, status changes of site components and the provider are logged as a bookmark. For a site component, only a general message may be logged (e.g., component online/offline). APIs may be implemented for other components to implement bookmark features.

Thus, one of the steps that results in a bookmark being recorded include content distribution, from a site server to the distribution points. Others include the policy distribution of the target policy to the clients, retrieval of the policy by a client from the site server, starting the service/content download from the distribution points to a client, and installation/execution of the program on a client. With some or all of these bookmarks, and possibly others, a checkpoint model may be built, e.g., to determine whether a deployment is on track or late with respect to the final execution result for policy distribution, and/or with respect to the final copy result for content distribution.

The distribution manager122copies a package source to a distribution point, and updates and refreshes the distribution point. Its status affects package activity status.

The offer summarizer124is an existing component (e.g., thread) that captures distribution status messages in order to accumulate a real time summary123, e.g., for maintaining in the database112. The offer summarizer124also captures distribution status messages to populate details as to the per client, per advertisement statuses into the database112. The offer summarizer124may integrate client health data126with the real time summary, e.g., by periodically synchronizing the real time summary with the state system summary; the client health summary127may be maintained in the database112. The healthy or unhealthy state of deployment may be based on the error messages received during the distribution.

The state system128is an existing component (e.g., thread) that parses, replicates and executes state messages, as well as executes scheduled summary tasks (e.g.,240,FIG. 2). Facts244about the tasks may be generated. Note that advertisement and deployment summary data are updated by summary tasks running in state system128. The state system128may work with new package distribution summary tasks, integrate the client health data with the summary, and maintain a history of the summary.

The administrator may set one or more checkpoints, e.g., to set one or more expected execution success ratios and/or dates for on-track/late evaluation. The administrator may also set an error message threshold. The delivery scope evaluator130is a thread component that may be triggered by a summarization task in the state system. In general, the delivery scope evaluator130evaluates the distribution status from the summary against the checkpoints, associates bookmarks with activities, and raise notifications of distribution status changes. There may be an evaluation as to when new facts244have been copied, and/or to raise a warning or error when the latest actual progress is beyond a baseline.

A database notification monitor246(FIG. 2) triggers the distribution manager122to handle the changes that results when a package is added/updated or a distribution point is added/updated to a package. The database notification monitor246can also run maintenance tasks.

Also shown for completeness inFIG. 1are alerts160, such as managed by an alert manager162, and performance history data164. Performance history data164may be used as described below with reference toFIG. 6, e.g., as part of an automatically generated graph.

A bookmark library may be provided that enables other components to generate a bookmark. For example, a “GenerateBookmark( . . . )” function call or the like may be provided for this purpose.

FIG. 3comprises a dataflow diagram summarizing some of the various operations. Note that the numbers before a decimal point indicates the entity performing the operation related to a bookmark. Thus, inFIG. 3, “1.x” indicates that a UI action adds bookmark dataflow; “2.x” indicates when site components add bookmark dataflow; “3.x” indicates maintenance task dataflow; and 4.x indicates a UI Get bookmark dataflow 5.1 also represents the delivery scope evaluator130updating an activity-to-bookmark association.FIG. 3also shows some example bookmark-related maintenance tasks that may be implemented to maintain bookmark data, to maintain activity data and/or to maintain an activity fact.

FIGS. 4 and 5, along with the following tables, provides additional details of bookmarks and activities in one example implementation.

DS_BookmarkTypes440is a table that holds the targets that the bookmark feature support. The data may be filled in the site server setup process.

FIG. 5shows bookmark related classes in one implementation. CBookmark550is a class in base object (baseobj), and provides a memory map for a bookmark object. CActivityBookmarkMap552is a relationship class in base object that describes which bookmarks are related to an activity. CBookmarkSource554is a class in basesrv; bookmark related operations with the database are implemented in this class. The API Bookmark.dll556is a separate binary for the AdminConsole. UI codes can invoke the API GetFormattedMessage( ) to get a bookmark localized message. Note that if the UI wants to retrieve a large number of messages, GetFormattedMessage( ) provided poor performance; better performance is obtained by calling GetResourceHandle( ) to get the resource handle to the message binary file, and then calling GetFormattedMessageFromHandle( ). Win32 FreeLibrary( ) is invoked to free the resource handle.

As can be seen, a bookmark captures information including change events, and operates to associate those events with one, several or all activities. Bookmarks provide an administrator with awareness of context/environment changes of given distribution, and may be used to find a potential root cause of a deployment issue.

A bookmark may be maintained with respect to changing the members of a collection, a policy change, or adding, removing updating or refreshing a distribution point with respect to a package. A bookmark also may be maintained for updating an advertisement schedule, retargeting an advertisement to a collection, or updating a schedule. Still other bookmarks record a distribution point management component change, a management and monitor components of inbox change, a database monitor component change, a status message component change, an SMSEXEC main thread change, or a management point component change, for example.

Further, bookmarks can be prioritized, may be grouped and may be marked as being positive or negative with respect to a result. The user interface may filter the set of bookmarks into a subset based on priority, based on groups and/or based on their results.

The performance history aspects are directed towards obtaining the current deployment data, such as package size, the number of clients to which deployment is needed, and so forth. During the deployment process, the performance history component collects the current data (e.g., in real time) and compares that data to the historical data.

FIG. 6shows one way bookmarks may be used to provide an automatically generated chart660. The curve between the light and dark-shaded areas comprises a performance history curve, and each triangle with exclamation point (“warning”) corresponds to a bookmark that occurred during the deployment.

The black line is a curve representing the actual deployment progress. If this curve is in the lightly shaded area, deployment is on track, while if in the darker shaded area, something wrong has happened, suggesting troubleshooting is needed. For example, around 6:00 PM on 1/16, the deployment curve enters the darker area, and there are two bookmarks around that timestamp. The administrator may select the time range, e.g., the medium-shaded area selected by using a mouse/pointer, to view the bookmark information in this time range. For example, as can be seen in the description of the two bookmarks, the administrator finds that the SCCM executive was paused at that time, which is the root cause of the problem. When the SCCM executive was later restarted, deployment resumed normally.

Bookmarks along with performance history thus help the administrator focus on the main/significant events, troubleshoot issues with high efficiency, and/or automatically evaluate the health of the entire deployment process. Automatic computations and the like help the administrator to estimate when the current deployment will complete, as well as to evaluate the current deployment progress, such as whether it is on track or delayed relative to the history of similar past deployments.

As can be seen, with the bookmark information, an administrator does not have to check the status messages reported by the client machines and/or analyze the raw data manually, which are inefficient operations. With the performance history, the administrator has an estimate of the time usage and progress, and can easily determine whether a deployment task is on track or delayed, without relying on personal experience. Bookmarking and performance history thereby significantly assist with any needed troubleshooting.

Exemplary Operating Environment

FIG. 7illustrates an example of a suitable computing and networking environment700on which the examples ofFIGS. 1-6may be implemented. The computing system environment700is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing environment700be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary operating environment700.

With reference toFIG. 7, an exemplary system for implementing various aspects of the invention may include a general purpose computing device in the form of a computer710. Components of the computer710may include, but are not limited to, a processing unit720, a system memory730, and a system bus721that couples various system components including the system memory to the processing unit720. The system bus721may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus.

The system memory730includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM)731and random access memory (RAM)732. A basic input/output system733(BIOS), containing the basic routines that help to transfer information between elements within computer710, such as during start-up, is typically stored in ROM731. RAM732typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit720. By way of example, and not limitation,FIG. 7illustrates operating system734, application programs735, other program modules736and program data737.

The computer710may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,FIG. 7illustrates a hard disk drive741that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive751that reads from or writes to a removable, nonvolatile magnetic disk752, and an optical disk drive755that reads from or writes to a removable, nonvolatile optical disk756such as a CD ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The hard disk drive741is typically connected to the system bus721through a non-removable memory interface such as interface740, and magnetic disk drive751and optical disk drive755are typically connected to the system bus721by a removable memory interface, such as interface750.

The drives and their associated computer storage media, described above and illustrated inFIG. 7, provide storage of computer-readable instructions, data structures, program modules and other data for the computer710. InFIG. 7, for example, hard disk drive741is illustrated as storing operating system744, application programs745, other program modules746and program data747. Note that these components can either be the same as or different from operating system734, application programs735, other program modules736, and program data737. Operating system744, application programs745, other program modules746, and program data747are given different numbers herein to illustrate that, at a minimum, they are different copies. A user may enter commands and information into the computer710through input devices such as a tablet, or electronic digitizer,764, a microphone763, a keyboard762and pointing device761, commonly referred to as mouse, trackball or touch pad. Other input devices not shown inFIG. 7may include a joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit720through a user input interface760that is coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). A monitor791or other type of display device is also connected to the system bus721via an interface, such as a video interface790. The monitor791may also be integrated with a touch-screen panel or the like. Note that the monitor and/or touch screen panel can be physically coupled to a housing in which the computing device710is incorporated, such as in a tablet-type personal computer. In addition, computers such as the computing device710may also include other peripheral output devices such as speakers795and printer796, which may be connected through an output peripheral interface794or the like.

The computer710may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer780. The remote computer780may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer710, although only a memory storage device781has been illustrated inFIG. 7. The logical connections depicted inFIG. 7include one or more local area networks (LAN)771and one or more wide area networks (WAN)773, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

When used in a LAN networking environment, the computer710is connected to the LAN771through a network interface or adapter770. When used in a WAN networking environment, the computer710typically includes a modem772or other means for establishing communications over the WAN773, such as the Internet. The modem772, which may be internal or external, may be connected to the system bus721via the user input interface760or other appropriate mechanism. A wireless networking component such as comprising an interface and antenna may be coupled through a suitable device such as an access point or peer computer to a WAN or LAN. In a networked environment, program modules depicted relative to the computer710, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,FIG. 7illustrates remote application programs785as residing on memory device781. It may be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

An auxiliary subsystem799(e.g., for auxiliary display of content) may be connected via the user interface760to allow data such as program content, system status and event notifications to be provided to the user, even if the main portions of the computer system are in a low power state. The auxiliary subsystem799may be connected to the modem772and/or network interface770to allow communication between these systems while the main processing unit720is in a low power state.

CONCLUSION