  Modern computing devices employ High-Bandwidth Memory (HBM) to meet their
memory bandwidth requirements. An HBM-enabled device consists of multiple DRAM
layers stacked on top of one another next to a compute chip (e.g. CPU, GPU, and
FPGA) in the same package. Although such HBM structures provide high bandwidth
at a small form factor, the stacked memory layers consume a substantial portion
of the package's power budget. Therefore, power-saving techniques that preserve
the performance of HBM are desirable. Undervolting is one such technique: it
reduces the supply voltage to decrease power consumption without reducing the
device's operating frequency to avoid performance loss. Undervolting takes
advantage of voltage guardbands put in place by manufacturers to ensure correct
operation under all environmental conditions. However, reducing voltage without
changing frequency can lead to reliability issues manifested as unwanted bit
flips. In this paper, we provide the first experimental study of real HBM chips
under reduced-voltage conditions. We show that the guardband regions for our
HBM chips constitute 19% of the nominal voltage. Pushing the supply voltage
down within the guardband region reduces power consumption by a factor of 1.5X
for all bandwidth utilization rates. Pushing the voltage down further by 11%
leads to a total of2.3X power savings at the cost of unwanted bit flips. We
explore and characterize the rate and types of these reduced-voltage-induced
bit flips and present a fault map that enables the possibility of a
three-factor trade-off among power, memory capacity, and fault rate.
