Responded to Jim Rogers feedback

The ability of a processor to self-regulate its P-state based on demand for specific functional units is always going to be more effective than a job scheduler deeming a job to be I/O-bound. Most modern processors already do this quite well.

These assets have very clear depreciation schedules, and the modest cost savings in terms of electricity consumption due to some of these methods may not (or frequently will not) outweigh the capital investment cost in the computer.  I.e. If <site> spent $100M for a computer that will remain in production for 60 months, then the apparent benefit of power capping, etc can easily be outweighed by lost productivity of the consumable resource.
-----------------------
First, his comment about the P-States is saying that the processors themselves (CPU or GPU accelerators) recognize when they are being used and automatically adjust themselves to "clock down" when underutilized.  This means that their idle power draw drops pretty low (say, below 30 Watts) but when they have tasks placed onto them they automatically ramp up to full draw.  He uses this notion to make a mental leap that says "therefore, all of the coarse-grained work to try to limit power draw is not very useful".  I don't necessarily agree with this.  The reason I don't agree is biased: it's what I work on in my research.  What he's not considering are heterogeneous environments where you have a mix of machine types in a single site.  If I have the opportunity to run a program on Machine Type 1 or Machine Type 2, it's quite possible that it will experience different performance on the two machine types as well as different power draw-down characteristics.  A smart scheduler would observe this and figure which one gives the best performance:power ratio.  Jim's experience is in "practical" environments like ORNL where there is only homogeneous resources.  Accordingly, he is just concerned with pushing as many jobs onto resources as fast as possible, so his logic about letting the processors idle themselves makes sense from his point of view.

Second, his comment about the opportunity cost that is lost by running your entire machine at a slower rate in response to a utility provider's request is exactly what the comment I put in the Opportunities section means.  That is to say, the cost of running a machine like Titan consists of things like: cost of renting the physical space where the machine resides, amortized cost of the machine over its lifetime, cost of the personnel needed to run the machine, and somewhere in all of that is the cost of electricity.  His experience says that the cost of electricity is such a small fraction of the overall cost of running the machine that he would NEVER run his machine at less than 100% of its capacity due to short-changing all of the other more expensive factors.  In my opinion, he is exactly right about this perspective.  I tried very hard to hedge about this in the Opportunities section, but I think it is a major flaw in the whitepaper.  One of the big issues I have is that we have a lot of people with research backgrounds working on the paper and not a lot of people with direct operations backgrounds, and I am not very confident that researchers appreciate the practical issues like Jim is raising here.


So, I think we should consider very carefully what he's saying.  He's essentially saying that demand-response is a dead-end issue to him.  Now, others in the demand-response group have indicated their facilities really do want this capability.  If that's the case, we should hopefully have that message from the Jim Rogers equivalent at those sites to support saying in the paper that even though sites like ORNL may not have a current need, other sites do have a need for a demand-response strategy.