JOB PLAN VERIFICATION

A job plan verification system (100) may include a job analysis module (104) to receive job details for jobs to be executed, and a job history management module (105) to receive prior job execution data. A facts creator module (107) may generate planned execution object instances for the jobs to be executed based on the job details and the prior job execution data. A verification module (108) may generate a verification report (110) for the jobs to be executed based on rules for job management and the planned execution object instances.

DETAILED DESCRIPTION OF EMBODIMENTS

For simplicity and illustrative purposes, the principles of the embodiments are described by referring mainly to examples thereof. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. It is apparent that the embodiments may be practiced without limitation to all the specific details. Also, the embodiments may be used together in various combinations.

A job plan verification system is described herein and provides a user with the ability to verify a job roster in a very flexible and automated manner before jobs are actually executed. The system provides verification of job schedules for a defined planning scope using a flexible rule-based approach. As a result, the service quality is improved while the labor time and effort for the job plan maintenance is minimized.

As described in greater detail below, the job plan verification system may generally receive data from an IM system that performs job scheduling and job execution. The IM system may include an IM application that includes job details such as assigned resources and defined schedules. The IM system may also include IM data about prior job executions. The job plan verification system may include a job analysis module to collect job details such as assigned resources and defined schedules from the IM application. A job history management module may collect data about prior job executions from the session history of the IM application, which feeds into the IM data. Based on this historical data, the estimated job duration may be calculated. An environment data module may collect environment specific data related to, for example, network topology, server capacity, and capacities of connections used by the job resources (e.g. devices). Job data, which includes the schedule information collected by the job analysis module and the job history management module, and environment data may be used by a facts creator module to generate planned execution object instances within a defined planning scope. Alternatively, the facts creator module may use just the job data to generate planned execution object instances within the defined planning scope. Each planned execution object instance may represent one execution of a job within the planning scope having a defined starting date, time and duration. The generated planned execution object instances and the job details collected by the job analysis module may be asserted into a rule-based verification module. The verification module may be executed to generate a verification report that provides details about the different jobs, possible conflicts and resource utilization.

The job plan verification system provides for detection of job resource conflicts before they actually happen. In other words, the job plan verification system simulates and models future job executions and possible conflicts within a defined planning scope. The rule-based verification of job plan schedules offers a flexible mechanism to automatically check existing job schedules for resource conflicts within a defined future time period. New constraints may be readily added by defining additional rules. This automated job schedule verification gives the user the opportunity to prevent possible resource conflicts proactively before they actually happen during job execution.

As discussed above, the job plan verification system may also be used to simulate what-if scenarios. For example, the positive effect of adding a new device on an anticipated future resource conflict may be simulated. The failure of one or more resources may also be simulated. Furthermore, the addition of new jobs may be simulated in order to ensure that no resource conflicts are generated. Certain time periods (e.g. year end processing) may be verified a long time ahead. For example, the job plan verification system may be used to determine which jobs would be affected by a failure, for example, within the next 24 hours or at another time period. Reports such as the expected device allocation, server load or network utilization may be readily generated for a specified future time period using the same, rule-based approach. The job plan verification system thus provides flexibility by the rule language to check for resource conflicts, certain priority conditions and to create reports on future job executions. These aspects also facilitate adaptation to new applications or extension with new rules by providing a new set of rules. A user may react on expected issues at a convenient time before job processing, rather than out of necessity based on job processing conflicts with existing jobs.

FIG. 1illustrates a job plan verification system100, according to an embodiment. The system100may generally receive data from an IM system101that performs job scheduling and job execution. The IM system may include IM application102that includes job details such as assigned resources and defined schedules. Jobs may include, for example, data archiving. The IM application102may thus keep data on jobs that have been scheduled. The IM system may also include IM data103about prior job executions (i.e. execution histories of jobs). The job plan verification system100may include a job analysis module104to collect job details such as assigned resources and defined schedules from the IM application102. The modules and other components of the system100may include machine readable instructions, hardware or a combination of machine readable instructions and hardware. For example, the job analysis module104may extract the names of jobs, including, for example, the resources that are used by the jobs. The job analysis module104may also extract information on the devices for performing the job (i.e. for backing up data with devices such as tapes, disks etc.). A job history management module105may collect data about prior job executions from the session history of the IM application102which may be fed into the IM data103. Based on this historical data, the estimated (i.e. expected) job duration may be calculated. The schedule information collected by the job analysis module and the job history management module may then be combined as job data, or alternatively, may be directly fed to a facts creator module107as described below. An environment data module106may collect environment data111related to, for example, the network topology, server capacity, and capacities of the connections used by the job resources (e.g. devices). The job data and environment data111may be used by the facts creator module107to generate planned execution object instances within a defined planning scope, described in further detail below. Alternatively, the facts creator module107may use just the job data to generate planned execution object instances within the defined planning scope. The job data may describe the jobs including their schedules, expected duration, and resources that are being utilized. Based on the job data and the environment data111, facts may be generated by the facts creator module107. Facts may be in the form of object instances which represent the different jobs and the representations may be presented to a verification module108. Each planned execution object instance may represent one execution of a job within the planning scope having a defined starting date and time and duration. The generated planned execution object instances and the job details collected by the job analysis module may be asserted into the rule-based verification module108. The verification module108may obtain rules for job management from the rule set109. The rule set109may be divided into different groups for conflict management and resource utilization analysis of jobs as described below. Upon execution, the verification module108may generate a verification report110that provides details about the different jobs, possible conflicts and resource utilization.

Referring toFIG. 1, using the system100, the job verification process may begin by defining a planning scope. This planning scope may begin at the current date or at any future date, and end <n> days later. To begin, the job analysis module104may collect job details, such as assigned resources and defined schedules from the IM data103. The job history management module may then collect data about prior job executions from the session history of the IM application102that may be fed into the IM data103. Based on this historical data, the estimated job duration may be calculated (e.g. using the average duration plus a buffer).

FIG. 2illustrates how planned executions instances may be generated for each job. For example, referring toFIG. 2, the estimated job duration is calculated for Jobs 1, 2 and 4, with the jobs being collectively designated120. InFIG. 2, history data, simulated data and the planning scope are respectively designated121,122and123. The shaded boxes inFIG. 2generally represent job executions that either actually happened in the past or are planned in the future. For example, Job 1 includes two job executions, Job 2 includes 1 job execution etc. The repetition of the shaded boxes inFIG. 2also represents the recurrence of a job within the planning scope123. For example, Job 1 recurs twice, whereas Jobs 2 and 3 occur once within the planning scope123. It should be noted that the history data121may differ from planned job execution objects generated based on the current job schedule and job duration taken from the history data. For example, a schedule which was valid for the history data may not be relevant for generation of the simulated data122. The currently defined schedule from the IM system101may be used to generate the planned execution objects in the planning scope. Thus the interval of historic job executions may look different than the intervals valid in the planning scope. In the example provided, it is assumed that the job schedules were not changed. Therefore the history data121and the simulated data122show the same intervals. The simulation aspect of the system100as described herein may be used for jobs that recur in the future. The shaded areas124under history data show times at which the history data121is available. The job timings for new jobs are shown as shaded areas125(also designated planned execution objected instances) under the simulated data122. Possible conflicts in schedule are shown at126, for example, for Jobs 1 and 3. For the possible conflicts for Jobs 1 and 3, a user may address the conflicts by modifying the schedule or allocating additional resources. Additionally, the verification report110generated by the system100may propose similar methods (e.g. general schedule and resource changes) and provide related specifics (e.g. proposed timing changes, resources needed) for addressing conflicts. If no job history data is available (e.g. for new jobs), a heuristic may be used in order to estimate the duration of the job. For example, for Job 3, since no historic data is available, a heuristic may be used for estimating the job duration. For example, by determining the amount of data needed to be backed up, the type of devices being used etc., based on such factors, the time needed for the backup can be estimated to thus estimate job duration. The schedule information collected by the job analysis module104and the job history management module105may then be used by the facts creator module107to generate the planned execution object instances125within the defined planning scope. Each planned execution object instance may represent one execution of a job within the planning scope having a defined starting date, time and duration.

The generated planned execution object instances and the job details collected by the job analysis module104may then be asserted into the rule-based verification module108. The facts creator module107may then create additional helper objects. Helper object instances may provide additional information to the rule system that is not included in the planned execution objects themselves or are derived from them. Thus the helper object instances may be related to the planned execution object instances and may be used by the rules to perform validations. The facts creator module107may create plan exclusion object instances (e.g. days excluded from the job schedule) based, for example, on the job schedule definition provided by the job analysis module104. Environment specific object instances (e.g. server capabilities) may be created based on the data provided by the environment data module106. Thereafter, the verification module108may then execute different types of rules from rule set109according to the rule flow illustrated inFIG. 3. Referring toFIG. 3, the rule set109may include three groups of rules defined, namely, initialization rules140, validation rules141and reporting rules142. The rules within each group may have different priorities in order to control their execution sequence. The initialization rules140may prepare the fact base for the validation and reporting phase. Initialization rules may also be used to generate helper object instances that make it easier to check for specific conditions. An initialization rule may be used to generate device allocation objects that show at which time period a specific device is allocated to a job. The device allocation object instances may contain the device name, and may be used by a rule to verify specific conditions (i.e. in the conditional part of the rule). The initialization rules may also be used to clean up the data (e.g. duplicates can be removed, where depending on the scheduling mechanism certain jobs may be listed multiple times). Functionality may be implemented from the information management application. Further, excluded job executions may be removed. For example, assuming a schedule occurs every Friday at a certain time, but it is explicitly stated that a job cannot happen on a specific date, such constraints can be extracted from the job definitions so that the job analysis module104may create exclusion facts which are then used by the initialization phase to remove these job executions from the verification module108so that they are not considered anymore.

The validation rules141may perform the actual verification of the job schedules against constraints. Validation rules may check for all types of potential resource conflicts, as long as the data is made available to the rule system. For example, the validation rules141may check for device conflicts, job internal schedule conflicts, and if there are too many jobs active on one server in parallel. The validation rules141may also check for device allocations that are acceptable, or if jobs are within a certain backup window, for example, to make sure that certain backups happen during a certain time window (e.g. between 8 pm and 8 am on weekdays).

The reporting rules142may be used to generate reports based on the facts available. For example, based on the device allocation object instances generated by an initialization rule, a reporting rule may calculate the estimated device utilization within the defined planning scope. Using this data, a free device slots report may be generated, helping administrators schedule new jobs accordingly. For example, the reporting rules142may create device allocation statistic, for example, stating that a device is utilized a certain percentage within a planning scope. This may also facilitate determination of which devices are used at capacity and which are not used at all to allow the unused devices to be used for future jobs.

Referring toFIG. 4, the conflict between Jobs 1 and 3 may be determined by a rule160as illustrated. The rule160may check for device allocation objects that have the same device name, different job names and finally, overlapping planned execution intervals.

The rule-based definition of resource conflicts and other conditions provides several options. For example, the application data model may be available in a JAVA object structure using, for example, standard getter and setter methods. These data objects may be asserted as facts to the system100before the rules are executed, for example, if the rule system would have access to network connectivity data it could verify if certain jobs running in parallel on devices attached to the same network connection would generate a potential resource issue on the corresponding network connection. Even cross-application resource conflicts may become detectable if job schedule data would be shared with the system100. For example, a source for such data may include HEWLETT PACKARD'S Universal Configuration Management Database software (UCMDB). The CMDB software may automatically maintain accurate, up-to-date information on the relationships between infrastructure, applications, and business services.

Depending on the length of the planning scope, the number of jobs and the schedule intervals, several planned execution objects may be generated, occupying a large amount of memory406(seeFIG. 9). A large verification job may be partitioned into smaller jobs, for example, for cloud based processing. For example, a job may be partitioned based on time. Therefore, in order to limit resource consumption, larger planning scopes may be partitioned into smaller planning scopes which are verified consecutively or in parallel. For jobs that cross scope boundaries, all affected scopes of such jobs may be considered. For example, a planned execution abject instance starting in Interval 1 and ending in Interval 2 may have to be evaluated within both intervals.

FIG. 5shows how the planning interval (i.e. planning scope)123may be segmented into 7 day intervals at180. This approach provides for scalability and for cloud based processing. The rule-based operation of the system100provides for efficient checking of the current configuration of an environment.

In addition to the verification of already scheduled jobs, the system100may also be used to simulate the effect of modifications to existing job schedules or additions of new job schedules to the environment. This may also include the simulation of resource outages. For example, in case of a device failure, a report may be generated showing all planned job executions within the next 24 hours that will be affected. The planning scope may also be implemented as a kind of moving window having a starting date defined somewhere in the future. For example, if combined with a user interface utilizing, for example. GANTT CHARTS to visualize the job roster, a user may verify specific time periods in the future, for example when the year-end processing is done and therefore resource conflicts are more likely.

With regard to the foregoing rule based approach, the approach may be extended to additional resources. For example, constraints may be placed on application servers to limit the number of jobs that can be handled in parallel (e.g. 10 jobs in parallel). In this regard, a user can be preemptively warned if greater than a predetermined number of jobs (e.g. greater than 10 jobs) would be running in parallel.

An example of an implementation of the system100with HEWLETT PACKARD'S Data Protector is shown inFIG. 6.

Referring toFIG. 6, the implementation may include a PERL script and a JAVA application. The PERL script may extract all necessary job and schedule information from the HEWLETT PACKARD Data Protector DB and from backup and schedule configuration files. This data may then be stored into an ORACLE DB using a data schema. The JAVA application may be the job plan verification system100. The JAVA application may take the job data object instances from the ORACLE DB using, for example, HIBERNATE, determine the estimated job durations, and generate the planned execution objects as described above. Then all the data objects may be asserted as facts into, for example, the DROOLS EXPERT rule-based system. After all the planned execution and job data is asserted into the working memory of DROOLS EXPERT, a set of rules may be executed on the planned execution data. Different sets of rules may be used for initialization, verification, reporting and debugging. In a first step, an initialization rule may generate device allocation object instances. After that, the verification rules may be executed, generating a report showing potential resource conflicts within the current configuration.FIG. 7shows a sample output of the implementation example of the system100.FIG. 7shows a conflict with the device V_g2u0025c_LAN—100.13, which would be needed by the jobs V1_BackupServers_I and V1_BackupServers_F at the same time. As another result of the job verification, a XML report may be created. This report may be converted into other formats (e.g. html) using, for example, XSL transformations. The rule set may be loaded by the DROOLS EXPERT rule engine from a text file during startup and may be extended or updated as needed.

FIG. 8illustrates a method300for job plan verification, according to an embodiment. The method300is described with respect to the job plan verification system100shown inFIG. 1by way of example and not limitation. The method300may be performed by other systems.

At block301, using the system100, the job verification process may begin by defining a planning scope. This planning scope may begin at the current date or at any future date and end <n> days later. In an example, the verification may be performed just for all planned job executions within this defined planning scope, and all planned job executions outside of the planning scope may be ignored.

At block302, the system100may receive data from the IM system101that performs job scheduling and job execution. Referring toFIG. 1, the IM system may include the IM application102that includes job details such as assigned resources and defined schedules. The IM application102may thus keep data on jobs that have been scheduled. The IM system may also include the IM data103about prior job executions (i.e. execution histories of jobs). The job analysis module104may collect job details such as assigned resources and defined schedules from the IM application102. For example, the job analysis module104may extract the names of jobs, including, for example, the resources that are used by the jobs. The job analysis module104may also extract information on the devices for performing the job (i.e. for backing up data with devices such as tapes, disks etc.).

At block303, the job history management module105may receive data about prior job executions from the session history of the IM application102which may be fed into the IM data. Based on this historical data, the estimated (i.e. expected) job duration may be calculated.

At block304, the schedule information collected by the job analysis module and the job history management module may then be combined as job data.

At block305, the environment data111from the environment data module106may be obtained and asserted into working memory. For example, the working memory may be the memory406(seeFIG. 9) area used by the verification module108. All facts may be asserted into the working memory such that the facts are made visible to the verification module108. The environment data may include data related to, for example, the network topology, server capacity, and capacities of the connections used by the job resources (e.g. devices).

At block306, information from the job data may be used by the facts creator module107to generate planned execution object instances within a defined planning scope. Alternatively, information from the job data and the environment data may be used by the facts creator module107to generate planned execution object instances within the defined planning scope. The job data may describe the jobs including their schedules, expected duration, and resources that are being utilized. Based on this data, facts may be generated by the facts creator module107.

At block307, facts in the form of object instances which represent the different jobs and the representations may be presented to the verification module108. Each planned execution object instance may represent one execution of a job within the planning scope having a defined starting date and time and duration. The generated planned execution object instances and the job details collected by the job analysis module may be asserted into the rule-based verification module108.

At block308, the verification module108may obtain rules from the rule set109. The rule set109may be divided into different groups as described above. For example, referring toFIG. 3, the rule set109may include three groups of rules defined, namely, initialization rules140, validation rules141and reporting rules142.

At block309, upon execution, the verification module108may generate a verification report110that provides details about the different jobs, possible conflicts and resource utilization.

4. Computer Readable Medium

FIG. 9shows a computer system400that may be used with the embodiments described herein. The computer system400represents a generic platform that includes components that may be in a server or another computer system. The computer system400may be used as a platform for the system100. The computer system400may execute, by a processor or other hardware processing circuit, the methods, functions and other processes described herein. These methods, functions and other processes may be embodied as machine readable instructions stored on computer readable medium, which may be non-transitory, such as hardware storage devices (e.g., RAM (random access memory), ROM (read only memory), EPROM (erasable, programmable ROM), EEPROM (electrically erasable, programmable ROM), hard drives, and flash memory).

The computer system400includes a processor402that may implement or execute machine readable instructions performing some or all of the methods, functions and other processes described herein. Commands and data from the processor402are communicated over a communication bus404. The computer system400also includes a main memory406, such as a random access memory (RAM), where the machine readable instructions and data for the processor402may reside during runtime, and a secondary data storage408, which may be non-volatile and stores machine readable instructions and data. The memory and data storage are examples of computer readable mediums. The memory406may include modules420including machine readable instructions residing in the memory406during runtime and executed by the processor402. The modules420may include the modules104-108of the system100shown inFIG. 1.

The computer system400may include an I/O device410, such as a keyboard, a mouse, a display, etc. The computer system400may include a network interface412for connecting to a network. Other known electronic components may be added or substituted in the computer system400.

While the embodiments have been described with reference to examples, various modifications to the described embodiments may be made without departing from the scope of the claimed embodiments.