Patent Publication Number: US-7917625-B1

Title: Predictive processing resource level control

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     Not Applicable. 
     STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH 
     Not Applicable. 
     BACKGROUND OF THE INVENTION 
     The present invention relates in general to minimizing electrical power consumption of pooled computer resources, and, more specifically, to turning pooled resources on and off in an efficient manner that maintains quality of service (QoS) from a service delivery network. 
     Service delivery networks for providing telecommunication and/or data processing services such as web pages or electronic multimedia typically use pooled computer resources in a co-located facility or data center. Computing grids employ a large number of pooled computer resources that may be allocated on request to users within a group. A primary difficulty faced by service providers utilizing pooled computing resources relates to electrical power consumption and heat production. As computer processors have become increasingly powerful, they have also become increasingly power hungry and hot. In response, microprocessor chip manufacturers have taken steps to limit the power consumption and heat production using processor power management schemes such as variable clock rates and by developing multi-core processor chips. Some manufacturers are working on shutting down portions of a processor when they are not needed so that power consumption and heat production can be reduced. 
     While processor chip improvements are helpful, a great deal of electrical power continues to be wasted in many applications. Demands for capacity from a particular service can vary significantly during a day (e.g., up to 500% variations). Resources that are not utilized or are underutilized waste power and create unnecessary heat by operating the associated microprocessors and also the support components and peripherals associated with the microprocessor or computer. 
     Computing resources such as standalone computers and servers or individual processing blades in blade servers or clusters are known which are capable of being remotely stopped (i.e., powered down) and restarted. Attempts have been made to match active resource levels of pooled resources to the actual demand for services by activating only those resources necessary to meet the current demand. Such systems suffer a serious drawback, however, in that the restart time after a computer or blade has been shut down is sufficiently long that computer processing performance of the system lags behind the increase in demand. 
     Quality of Service (QoS) relates to the obligation of a service provider to maintain performance levels in accordance with certain guaranteed criteria. For example, transmission rates and/or error rates at a guaranteed level may be needed for purposes of transmitting video and/or multimedia data. Under fluctuating load conditions, when increased resources become necessary QoS levels may be adversely affected by the inherent time delays associated with restarting halted resources. The lead-time for adding additional capacity from halted resources in a pool includes time for starting associated hardware, loading an operating system, and loading the appropriate applications run by the resource. Thus, prior art attempts to reduce electrical power consumption have not been appropriate in the context of a service provider that needs to guarantee a particular QoS. 
     SUMMARY OF THE INVENTION 
     The present invention has the advantage of reducing electrical power consumption and heat generation without impacting quality of service (QoS) while dormant resources are re-started. 
     In one aspect of the invention, a method is provided for controlling the activation of pooled resources in a network, wherein the pooled resources include remotely startable and stoppable computing resources, and wherein starting of each of the computing resources requires a respective startup time. A resource utilization model is established including alert events for providing respective resource activation levels representing either an increase or a decrease in the resource utilization model expected to occur after the respective alert events. Occurrence of an alert event is detected. If the detected alert event corresponds to an increase, then selected ones of the pooled resources are started. If the detected alert event corresponds to a decrease, then an actual load level is checked, and selected ones of the pooled resources are stopped if the actual load level is less than the respective resource activation level corresponding to the detected alert event. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram of a network architecture for delivering network services with reduced power consumption according to the invention. 
         FIG. 2  is a block diagram showing an alternative configuration of pooled resources including blade servers. 
         FIG. 3  is a plot showing one embodiment of a relationship between a network usage model and a resource activation level to support QoS. 
         FIG. 4  is a table showing alert events in a resource utilization model. 
         FIG. 5  shows a management and control resource in greater detail. 
         FIG. 6  shows a preferred method of the invention. 
     
    
    
     DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
     Referring now to  FIG. 1 , a plurality of pooled resource servers  10 - 13  receive electrical power from a power source  14  and are cooled by a cooling system  15  (which may include a room air conditioning system). A management and control resource  16  makes the computing services of pooled resource servers  10 - 13  available to remote users via network  17 . A router  18  and a firewall  19  are used to couple management and control resources  16  to network  17 . Management and control resource  16  performs a load balancing function for directing service requests received from network  17  to the pooled resources in a distributed manner. In a preferred embodiment of the present invention, management and control resource  16  remotely controls the on or off states of pooled resources  10 - 13  such that excess resources are not activated and yet QoS levels are maintained by activating resources prior to expected load increases in a predictive manner. 
     As described below, predicted load variations are determined using a resource utilization model maintained within management and control resource  16 , for example. The model includes alert events which provide respective resource activation levels representing either an increase or a decrease in resource utilization expected to occur shortly after the respective alert events. Alert events typically comprise timed events such as times of day on specified days of the week or special dates (e.g., holidays). Alert events may also comprise predetermined occurrences happening outside the pooled resource system controlled by management and control resource  16 . For example, pooled resources  20  providing related services may experience a significant load variation which is communicated to management and control process  16  as a predetermined occurrence. Pooled resources  20  are coupled to a router  21  via a firewall  22  and its own local management and controlled resource  23 . Router  21  is coupled to router  18  providing a communication path to management and control resource  16  for signaling the predetermined occurrence. 
     Pooled resource servers  10 - 13  in  FIG. 1  may comprise separate computing devices (i.e., standalone machines) installed in a data center as part of a cluster or a grid, for example. Management and control resource  16  typically resides on a separate machine. The resource utilization model and its activation/deactivation control function may be incorporated into resource  16  or may be incorporated into one of the servers  10 - 13 .  FIG. 2  shows an alternative service delivery architecture utilizing blade servers. A blade is a thin circuit board based processing system. A rack or other enclosure  25  contains a main backplane  26 , a power backplane  27 , and cooling fans  28  which are shared amongst a plurality of blades. A management and control blade  30  and an Ethernet switch blade  31  are interconnected with backplanes  26  and  27 . Ethernet switch blade  31  is coupled to a network via a router and a firewall as shown in  FIG. 1 , for example. Incoming packets delivered to Ethernet switch blade  31  are examined by management and control blade  30  as is known in the art and distributed to selected ones of pooled server blades  32 - 36 . Each pooled server blade is remotely is startable and stoppable under control of management and control blade  30 . As in the case of pooled resource servers  10 - 13  in  FIG. 1 , pooled server blades  32 - 36  require a respective startup time in order to perform a power-on self-test, boot up an operating system, and load any applications. Although the startup time for a server blade may be somewhat less than a separate server machine, the time required would still significantly impact QoS levels if a blade were only started when the currently activated blades become insufficient to handle current traffic loads. 
     In order to insure that sufficient pooled resources are active and available to respond to expected loads without creating unnecessary power consumption, a resource utilization model and a resource activation level are utilized as shown in  FIG. 3 . A resource utilization model is represented in  FIG. 3  by a curve  40  plotting a load level over time. Thus, during particular times of day, days of the week, and special dates such as holidays, a service provider can statistically predict an anticipated load level. A curve  41  shows a target minimum level of activated resources in order to meet anticipated load level demands, preferably maintaining a small cushion of excess capacity to handle small variations in peak load. As the usage model  40  varies between substantially steady state values throughout a day, alert events  42 - 47  are defined in order to track the variations in predicted usage. 
       FIG. 4  shows one preferred representation of the resource utilization model wherein alert events defined in terms of time of day, day of week, special dates, and predetermined occurrences provide a corresponding resource activation change as an increase or decrease and a target resource activation level in terms of units of computing capacity (which may be represented as an absolute number of resources or a percentage of full capacity). For simplicity in  FIG. 4 , it is assumed that 100 units of capacity are available. Various increases and decreases are shown during a typical weekday to follow commercial activity, for example. Different load levels are predicted during weekends and during a holiday (e.g., representing a higher than normal usage of telecommunication services on Mother&#39;s Day). Also shown is a predetermined occurrence of more than two other resource pools experiencing load over their resource utilization models, resulting in an increase of resource activation by an amount based on a percentage of the excess load being experienced by the other resource pools. Those skilled in the art will appreciate that many other types of occurrences and methods for defining capacity changes are possible. 
       FIG. 5  shows a management and control resource  50  of the present invention in greater detail. A controller  51  is connected to a packet buffer  52  which receives packets from a network and provides packets out to identified pooled resources. Controller  51  is coupled to a load balancing process  53  for determining appropriate distribution of load levels as is known in the art. Load balancing process  53  monitors the overall load level and the load levels of individual pooled resources. Controller  51  is coupled to the pooled resources for remotely starting and stopping the resources either in response to the resource utilization model or in response to actual load levels that may exceed the model. A resource utilization model  54  includes alert events that are provided to a comparison block  55 . For timed alert events, a clock/calendar  56  is coupled to comparison block  55  such that comparison block  55  detects occurrences of each particular alert event and signals controller  51  of the corresponding increase or decrease to be made in the activated resource level. Comparison block  55  also includes an occurrence input  57  responsive to predetermined occurrences such as reports of excess load levels from other pooled resources. 
     A preferred method of the present invention is shown in  FIG. 6 . The management and control resource initializes the resource utilization model in step  60  by setting appropriate alert events. The method of the invention then waits for the occurrence of an alert event in step  61 . Once an alert event occurs, a check is made in step  62  to determine whether the alert event is for an increase in resources. If so, then a check is made in step  63  to determine whether the resource capacity associated with the alert event is greater than the current load (as determined by the load balancing process). If the current load is already greater than the alert capacity, than the variance is logged in step  64  (i.e., indicating usage already over the predicted level) and a return is made to step  61  to await the next alert event. If the load level associated with the alert is greater than the current load, then a check is made in step  65  to determine whether additional resources are available. If not, then the insufficiency is logged in step  66  and a return is made to step  61 . If additional resources are available, then the additional resources are started up in step  67  and a return is made to step  61 . 
     If the alert event in step  62  relates to a decrease, then a check is made in step  68  to determine whether excess capacity is in fact running (i.e., whether the resource activation level identified by the alert event is greater than the current load). If there is not excess capacity, then the variance is logged in step  69  and a reconfigurable retry alert event may be set in step  69 . For example, the alert event can be deferred by a predetermined amount of time. If excess capacity is found in step  68 , then it is shut down in step  70 . If the actual load is intermediate between the current available resources and the predicted load, then an intermediate amount of resources may be shut down. If an intermediate amount of resources are shut down, then a retry alert event may also be created to attempt to reduce available resources to the predicted level after a predetermined amount of time. 
     In response to logged variances from the current model, an adjustment step  71  may be periodically executed. Conventional statistical methods can be used to adjust the times or capacity levels associated with alert events.