Abstract:
A method and system for predicting performance and capacity of an information technology (IT) system before the IT system is built, where the predicting uses a database of performance statistics measured for reusable software modules. A performance simulation modeling tool receives a set of performance statistics of a test computer. The set of performance statistics is associated with a set of software modules included in a library of reusable software modules. The set of software modules is required to build a target IT system. The set of performance statistics is received from a database of performance statistics resulting from individual executions on the test computer of each reusable software module in the library. The performance simulation modeling tool predicts computing resources required by the target IT system. The computing resources prediction follows the receipt of the set of performance statistics and precedes the target IT system being built.

Description:
FIELD OF THE INVENTION 
     The present invention relates to a method and system for predicting system performance and capacity, and more particularly to a technique for generating a set of performance statistics associated with a library of reusable software modules and utilizing a performance simulation modeling tool to accurately predict system performance and capacity prior to the system being built. 
     BACKGROUND OF THE INVENTION 
     Building an information technology (IT) system requires developing a system architecture that meets specified functional and non-functional requirements. With respect to meeting non-functional requirements, known IT system building techniques are costly, lead to excessively long development cycles, and are marked by performance and stability problems. Conventional risk mitigation strategies for non-functional requirements validation during the development cycle of an IT system are listed below:
         Building and testing the IT system before deployment. This approach is the most accurate, but is also excessively expensive because all of the hardware and software must be acquired and the system must be built prior to an assessment of the non-functional requirements.   Testing in a test environment. This strategy is very costly and time-consuming if a production-like test environment is built. If shortcuts are used to build the test environment, such as building less than a full end-to-end system, using stubbed out interfaces, using emulators or simulators, reducing the test scope, etc., then the risk mitigation and the accuracy of the validation is negatively impacted.   Building and testing prototypes. This approach is also expensive and requires at least some of the hardware that the target system is supposed to use. Very often, prototype testing emphasizes functional requirements testing at the expense of non-functional requirements testing.   Capacity and performance simulation modeling. This approach requires having performance data for the system available in order to calibrate the model. Therefore, using this approach requires either having the system available to test or having a test or prototype system available to generate system performance statistics. If actual or test data is unavailable, then the accuracy of the modeling results decreases. In cases in which the modeling approach is used iteratively as more system performance statistics become available, this approach becomes time-consuming. This approach may also require acquiring new hardware before the system performance is validated. Often final results are not available by the time the hardware for the new system must be purchased. Finally, initial estimates may be deficient because of inaccuracies.   Paper and pencil estimation. Although being the fastest and cheapest of the listed conventional methods of estimating predicted performance or required capacity of a system, this approach is the least accurate method and suffers from a number of other drawbacks. This approach is less useful if the system being built is complex because the more complex the system, the more difficult it becomes to keep track of all system components.
 
Thus, there exists a need to overcome at least one of the preceding deficiencies and limitations of the related art.
       

     SUMMARY OF THE INVENTION 
     The present invention provides a computer-implemented method of predicting system performance and capacity using a database of performance statistics measured for reusable software modules, comprising: 
     receiving, by a performance simulation modeling tool of a computing system, a set of performance statistics of a test computer, wherein the set of performance statistics is associated with a set of reusable software modules included in a library of reusable software modules, wherein the set of reusable software modules is required for building a target information technology (IT) system, and wherein the set of performance statistics is received from a database of a plurality of performance statistics resulting from a plurality of individual executions on the test computer of each reusable software module of the library; and 
     predicting, by the performance simulation modeling tool, subsequent to the receiving the set of performance statistics and prior to the target IT system being built, a plurality of computing resources required by the target IT system. 
     A system, computer program product and a process for supporting computing infrastructure that provides at least one support service corresponding to the above-summarized method are also described and claimed herein. 
     Advantageously, the present invention provides an IT system performance and capacity prediction technique that provides inexpensive, accurate and quick predictions of computing resources required for a target IT system prior to the acquisition of any computing hardware for building the target IT system. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram of a system for predicting system performance and capacity using a database of performance statistics measured for reusable software modules, in accordance with embodiments of the present invention. 
         FIG. 2  is a block diagram of a system for generating the database used in the system of  FIG. 1 , in accordance with embodiments of the present invention. 
         FIG. 3A  is a flow diagram of a process for generating the database used in the system of  FIG. 1 , in accordance with embodiments of the present invention. 
         FIG. 3B  is a flow diagram of a process for adding performance statistics of customized middleware to the database used in the system of  FIG. 1 , in accordance with embodiments of the present invention. 
         FIG. 4  is a flow diagram of a process of predicting IT system performance and capacity using the database used in the system of  FIG. 1 , in accordance with embodiments of the present invention. 
         FIG. 5  is a block diagram of computing systems that include components of the system of  FIG. 1  and that implement the processes of  FIG. 3A ,  FIG. 3B  and  FIG. 4 , in accordance with embodiments of the present invention. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     Overview 
     The present invention generates a set of component-level performance statistics (a.k.a. performance characteristics) associated with a library of reusable software modules (a.k.a. software components) and stores these performance statistics in a database in permanent storage. The component-level performance statistics in the database are used to estimate full (i.e., overall) performance characteristics of complex IT systems before the IT systems are built. The software modules in the library are reusable as building blocks of the IT systems. 
     Performance &amp; Capacity Predication System 
       FIG. 1  is a block diagram of a system for predicting system performance and capacity using a database of performance statistics measured for reusable software modules, in accordance with embodiments of the present invention. System  100  includes a test environment computer  102  (a.k.a. test computer) that includes reusable software modules  104  that can be executed on test computer  102  and a performance &amp; capacity database generator  106 . In one embodiment, test environment computer  102  is one of multiple computers which comprise a test bed for generating performance statistics for each of the software modules  104 . The functionality of performance &amp; capacity database generator  106  is discussed below relative to  FIGS. 3A and 3B . 
     Software modules  104  can be used to build target IT systems with shorter development times because most of the software components of the target IT systems already exist in a predefined library. This functional reusability of software modules  104  is leveraged by the present invention to generate performance statistics that characterize each module&#39;s contribution to overall performance of a completed target IT system. 
     Further, system  100  includes input messages  108  that are input to reusable software modules  104  and a database  110  that includes performance statistics for individual software modules that are output from test environment computer  102 . The performance statistics in database  110  include, for example, transaction or message response times, CPU levels, disk I/O utilization levels, message type, message volume, etc. 
     Still further, system  100  includes a performance simulation modeling tool  112  that resides on a computing system (not shown in  FIG. 1 ; see  FIG. 5 ), input to tool  112  that includes a repository  114  that includes a standard set of benchmarks for computer configurations and a repository  116  that includes a projected transaction mix defining workload scenarios for a target IT system, and output  118  from tool  112  that includes predicted computing resource requirements, transaction response times, input/output (I/O) activity and network message volumes. 
     A target IT system&#39;s design can be modeled in performance simulation modeling tool  112  using performance statistics determined in earlier tests. Tool  112  allows a system modeler to specify target hardware with which to model the target IT system. A user specifies projected message volumes and a projected transaction mix for the target IT system as input to tool  112 . The output of tool  112  is a quick, cheap and highly accurate prediction of the target IT system&#39;s performance and resource utilizations. Performance simulation modeling tool  112  is, for example, a modeler in the HyPerformix IPS suite offered by HyPerformix, Inc. of Austin, Tex. The functionality of tool  112  is also discussed below relative to  FIG. 4 . 
     Set of benchmarks  114  is any set of standard computer performance rating benchmarks such as SPEC CPU2000, which is a CPU component benchmark suite provided by the Standard Performance Evaluation Corporation (SPEC). 
       FIG. 2  is a block diagram of a system for generating the database used in the system of  FIG. 1 , in accordance with embodiments of the present invention. System  200  includes test environment computer  102  that includes a reusable software module J (i.e., a software module  204 ) and performance &amp; capacity database generator  106 . Further, system  200  includes an input message type K (i.e., an input message type  208 ) that is a type of a message input to test environment computer  102 . Still further, system  200  includes performance statistics  210  for software module J for input message type K. Performance statistics  210  are stored in database  110  (see  FIG. 1 ) along with identification of which test computer configuration, software module, message type and message volume was used in the test. Thus, database  110  (see  FIG. 1 ) is a repository of performance characteristics for each of the reusable software modules  104 . 
     In system  200 , the J is an index that denotes an individual software module of software modules  104  (see  FIG. 1 ). Each individual software module J (i.e., software module  204 ) needs to be evaluated in test environment computer  102  separately. The K of input message type  208  is an index that denotes individual message types for software module J (i.e., software module  204 ). There may be 0 to n message types associated with a single software module J. Thus, if there are multiple message types for software module J (i.e., software module  204 ) then each message type K is run through software module J (i.e., software module  204 ) in the test environment of computer  102 . The number of message types may vary for different software modules. For example, one software module may have 4 input message types, whereas another software module may have only 1 input message type. 
     For each combination of input message type K and software module J, an execution of the software module is performed and the performance statistics for the execution are determined, captured and stored. For example, consider a case of a library with 2 software modules. If module  1  has 6 message types and module  2  has 4 message types, then the total number of evaluations of performance statistics is 10 (i.e., 6 evaluations+4 evaluations). That is, there is one evaluation for each of the 6 message types of module  1  (i.e., 6 evaluations) and one evaluation for each of the 4 message types of module  2  (i.e., 4 evaluations). 
     Generating Performance &amp; Capacity Database 
       FIG. 3A  is a flow diagram of a process for generating the performance &amp; capacity database used in the system of  FIG. 1 , in accordance with embodiments of the present invention. The process of generating the performance &amp; capacity database begins at step  300 . In step  302 , test environment computer  102  (see  FIG. 1 ) obtains reusable software modules  104  (see  FIG. 1 ) from a predefined library of reusable software modules. In step  304 , the reusable software modules  104  (see  FIG. 1 ) are run individually on test environment computer  102  (see  FIG. 1 ). 
     In step  306 , performance &amp; capacity database generator  106  (see  FIG. 1 ) determines and captures component-level performance statistics of computer  102  (see  FIG. 1 ) for each of the individual runs of software modules  104  (see  FIG. 1 ) in step  304 . The component-level performance statistics determined and captured in step  306  characterize the contribution of each of software modules  104  (see  FIG. 1 ) to the overall performance of a target IT system. As used herein, a target IT system is an IT system that is to be built using a set of reusable software modules, but is not yet built. 
     In step  308 , performance &amp; capacity database generator  106  (see  FIG. 1 ) stores the component-level performance statistics determined in step  306  in database  110  (see  FIG. 1 ). In step  310 , performance &amp; capacity database generator  106  (see  FIG. 1 ) stores identifiers in database  110  (see  FIG. 1 ) to associate each of the reusable software modules  104  (see  FIG. 1 ) with performance statistics stored in step  308  and with a configuration of test environment computer  102  (see  FIG. 1 ) that was used to execute the software module in step  304 . An association to the configuration of computer  102  (see  FIG. 1 ) is necessary because performance characteristics are a function of the configuration. In one embodiment, the identifiers stored in step  310  are associated with a set of performance statistics determined in step  306  and identify which software module, message type and message volume was used in the test execution of the software module in step  304 . The process of generating a performance &amp; capacity database ends at step  312 . 
     The database generated by the process of  FIG. 3A  is used by a performance prediction technique such as capacity and performance simulation modeling or paper and pencil estimation to accurately predict overall IT system performance before the IT system is built The same database (i.e., database  110  of  FIG. 1 ) may be reused for predictions of overall performance of multiple IT systems. The prediction of overall IT system performance is described below relative to  FIG. 4 . 
       FIG. 3B  is a flow diagram of a process for adding performance statistics of customized middleware to the database used in the system of  FIG. 1 , in accordance with embodiments of the present invention. The process of  FIG. 3B  starts at step  320  with middleware of the target IT system being customized instead of being available in a predefined library of reusable software modules  104  (see  FIG. 1 ). In step  322 , test environment computer  102  (see  FIG. 1 ) executes the customized middleware. 
     In step  324 , performance &amp; capacity database generator  106  (see  FIG. 1 ) determines and captures performance statistics that characterize the middleware&#39;s contribution to the overall performance of the target IT system. 
     In step  326 , performance &amp; capacity database generator  106  (see  FIG. 1 ) stores the performance statistics determined in step  324  in database  110  (see  FIG. 1 ). In step  328 , performance &amp; capacity database generator  106  (see  FIG. 1 ) stores identifiers in database  110  (see  FIG. 1 ) to associate the middleware with the performance statistics stored in step  326  and with a configuration of test environment computer  102  (see  FIG. 1 ) that was used to execute the middleware in step  328 . The process of adding performance statistics of customized middleware to a performance &amp; capacity database ends at step  330 . 
     After the process of  FIG. 3B  stores the performance statistics for the customized middleware of the target IT system in database  110  (see  FIG. 1 ), the stored middleware performance statistics are used as needed by performance simulation modeling tool  112  (see  FIG. 1 ). The overall performance of the target IT system is the composite of the performance statistics of the target IT system&#39;s middleware and software modules. 
     Predicting System Performance &amp; Capacity 
       FIG. 4  is a flow diagram of a process of predicting IT system performance and capacity using the database used in the system of  FIG. 1 , in accordance with embodiments of the present invention. The IT system performance and capacity prediction process begins at step  400 . In step  402 , performance simulation modeling tool  112  (see  FIG. 1 ) obtains relevant performance statistics from database  110  (see  FIG. 1 ). The relevant performance statistics obtained in step  402  are a subset of all the performance statistics included in database  110  (see  FIG. 1 ). The relevancy of the performance statistics obtained in step  402  is based on their being associated with the execution of all software modules and middleware required to build the target IT system. 
     In step  404 , performance simulation modeling tool  112  (see  FIG. 1 ) obtains the target IT system&#39;s projected workloads and projected transaction mix from repository  116  (see  FIG. 1 ). In step  406 , performance simulation modeling tool  112  (see  FIG. 1 ) obtains benchmark ratings for computer configurations from repository  114  (see  FIG. 1 ). As one example, if tool  112  (see  FIG. 1 ) is HyPerformix IPS, the default benchmark rating is SPECInt2000. 
     In step  408 , performance simulation modeling tool  112  (see  FIG. 1 ) generates and outputs a prediction of the target IT system&#39;s computing resource requirements, transaction response times, I/O activity and network message volumes. Performance simulation modeling tool  112  (see  FIG. 1 ) generates the prediction in step  408  by utilizing the performance statistics obtained in step  402 , the projected transaction mix and workloads obtained in step  404  and the benchmark ratings obtained in step  406  in one or more predefined mathematical algorithms built into tool  112  (see  FIG. 1 ). For example, tool  112  (see  FIG. 1 ) is HyPerformix IPS, which includes proprietary mathematical algorithms that (1) receive as input the performance statistics of step  402 , the transaction mix and workloads of step  404  and the benchmark ratings of step  406  and (2) output the aforementioned prediction of step  408 . The prediction outputted in step  408  is used to generate precise recommendations as to what computing resources (e.g., computer hardware, network bandwidth) are sufficient to build the target IT system while avoiding over-specifying the computing resources. The process of  FIG. 4  ends at step  410 . 
     Computing System 
       FIG. 5  is a block diagram of computing systems that include components of the system of  FIG. 1  and that implement the processes of  FIG. 3A ,  FIG. 3B  and  FIG. 4 , in accordance with embodiments of the present invention. A first computing system  500  generally comprises a central processing unit (CPU)  502 , a memory  504 , an input/output (I/O) interface  506 , a bus  508 , I/O devices  510  and a storage unit  512 . CPU  502  performs computation and control functions of computing system  500 . CPU  502  may comprise a single processing unit, or be distributed across one or more processing units in one or more locations (e.g., on a client and server). 
     Memory  504  may comprise any known type of data storage and/or transmission media, including bulk storage, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), a data cache, a data object, etc. Cache memory elements of memory  504  provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution. Storage unit  512  is, for example, a magnetic disk drive or an optical disk drive that stores data. Moreover, similar to CPU  502 , memory  504  may reside at a single physical location, comprising one or more types of data storage, or be distributed across a plurality of physical systems in various forms. Further, memory  504  can include data distributed across, for example, a LAN, WAN or storage area network (SAN) (not shown). 
     I/O interface  506  comprises any system for exchanging information to or from an external source. I/O devices  510  comprise any known type of external device, including a display monitor, keyboard, mouse, printer, speakers, handheld device, printer, facsimile, etc. Bus  508  provides a communication link between each of the components in computing system  500 , and may comprise any type of transmission link, including electrical, optical, wireless, etc. 
     I/O interface  506  also allows computing system  500  to store and retrieve information (e.g., program instructions or data) from an auxiliary storage device (e.g., storage unit  512 ). The auxiliary storage device may be a non-volatile storage device (e.g., a CD-ROM drive which receives a CD-ROM disk). Computing system  500  can store and retrieve information from other auxiliary storage devices that may include database  110 . Such auxiliary storage devices can include a direct access storage device (DASD) (e.g., hard disk or floppy diskette), a magneto-optical disk drive, a tape drive, or a wireless communication device. In another embodiment, database  110  is included in storage unit  512 . 
     Memory  504  includes program code for performance &amp; capacity database generator  106 . Further, memory  504  may include other systems not shown in  FIG. 5 , such as an operating system (e.g., Linux) that runs on CPU  502  and provides control of various components within and/or connected to computing system  500 . 
     A second computing system  600  generally comprises a CPU  602 , a memory  604 , an I/O interface  606 , a bus  608 , I/O devices  610  and a storage unit  612 , which have the functionality, capabilities and implementation options of CPU  502 , memory  504 , I/O interface  506 , bus  508 , I/O devices  510  and storage unit  512 , respectively, as described above. Computing system  600  can retrieve information from a storage device that includes database  110 . In one embodiment, database  110  is included in storage unit  612 . Memory  604  includes program code for performance simulation modeling tool  112 . 
     In an alternate embodiment (not shown), computing system  500  is utilized to implement the processes of  FIG. 3A ,  FIG. 3B  and  FIG. 4  without requiring computing system  600 , storage unit  612 , I/O devices  610  and any of the components internal to computing system  600 . In this alternate embodiment, performance &amp; capacity database generator  106  and performance simulation modeling tool  112  are both included in memory  504 . 
     Furthermore, the invention can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code  106  and/or program code  112  for use by or in connection with a computing system  500  and/or a computing system  600  or any instruction execution system to provide and facilitate the capabilities of the present invention. For the purposes of this description, a computer-usable or computer-readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. 
     The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, RAM  504 , ROM, a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read-only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD. 
     Any of the components of the present invention can be deployed, managed, serviced, etc. by a service provider that offers to deploy or integrate computing infrastructure with respect to the method of predicting system performance and capacity using a database of performance statistics measured for reusable software modules. Thus, the present invention discloses a process for supporting computer infrastructure, comprising integrating, hosting, maintaining and deploying computer-readable code into a computing system (e.g., computing system  500  or computing system  600 ), wherein the code in combination with the computing unit is capable of performing a method of predicting system performance and capacity using a database of performance statistics measured for reusable software modules. 
     In another embodiment, the invention provides a business method that performs the process steps of the invention on a subscription, advertising and/or fee basis. That is, a service provider, such as a Solution Integrator, can offer to create, maintain, support, etc. a method of predicting system performance and capacity using a database of performance statistics measured for reusable software modules. In this case, the service provider can create, maintain, support, etc. a computer infrastructure that performs the process steps of the invention for one or more customers. In return, the service provider can receive payment from the customer(s) under a subscription and/or fee agreement, and/or the service provider can receive payment from the sale of advertising content to one or more third parties. 
     The flow diagrams depicted herein are provided by way of example. There may be variations to these diagrams or the steps (or operations) described herein without departing from the spirit of the invention. For instance, in certain cases, the steps may be performed in differing order, or steps may be added, deleted or modified. All of these variations are considered a part of the present invention as recited in the appended claims. 
     While embodiments of the present invention have been described herein for purposes of illustration, many modifications and changes will become apparent to those skilled in the art. Accordingly, the appended claims are intended to encompass all such modifications and changes as fall within the true spirit and scope of this invention.