Energy expense is becoming an increasingly dominant part of the operating costs of high-performance computing (HPC) systems. Compounding the problem is the fact that electricity prices vary dramatically at different times of the day. To address these issues, researchers from Argonne National Laboratory and the Illinois Institute of Technology have developed a smart, power-aware job scheduling approach for HPC systems based on variable energy prices and job power profiles.
Rather than allocating jobs one by one, the novel scheduler makes decisions on a group or “window” of jobs. A standard model is a used to determine which items to include in a group while meeting certain constraints – in this case, selecting a subset of jobs such that their total power consumption is no more the allotted power budget, while the maximizing the number of nodes allocated. Typically, those jobs with high power consumption demands are run during the off-peak electricity price period.
The power-aware scheduling project had three targets: reducing energy costs, affecting system utilization only minimally, and preserving fairness. For the first two targets, the researchers evaluated real workloads from the IBM Blue Gene/P over a six-month period. Three different ratios of on-peak to off-peak pricing were used to explore dynamic electricity pricing, and three different power ranges were used to investigate the approximate power consumption of jobs.
The third target proved more problematic. “Balancing fairness and system performance is critical,” said Narayan Desai, an experimental systems engineer in Argonne’s Mathematics and Computer Science Division. “Since our new power-aware approach could disrupt job execution by selecting any job from within the scheduling window, we devised a new metric that reflects the extent of disruption when measuring fairness.”
Experimental results demonstrated that power-aware scheduling can reduce energy costs by up to 25%. “For HPC centers such as the Argonne Leadership Computing Facility, the energy savings would translate into over $250,000 per year,” said Desai.
Arguably, this savings comes at the expense of impact on system utilization during both the on-peak and off-peak price periods. But since the overall system utilization does not change much on a daily basis, the slight loss to system utilization is considered acceptable.
The researchers also observed that some jobs will be pushed to off-peak times. The intention is that less energy be consumed during peak periods. In particular, jobs having high power consumption may be delayed, but the delay is limited to a day.
For further information, see
Z. Zhou, Z. Lan, W. Tang, and N. L. Desai, “Reducing energy costs for IBM Blue Gene/P via Power-Award job scheduling,” in Proceedings of the 17th Workshop on Job Scheduling Strategies for Parallel Processors (JSSPP 2013), Boston, May 2013.