Before revision:  The LMs were trained on the Summit supercomputer , using 5616 GPUs  and one TPU Pod , using V3-512 cores.
After revision:  The LMs were trained on the Summit supercomputer at Oak Ridge National Laboratory (ORNL), using 936 nodes (total 5616 GPUs and one TPU Pod , using V3-512 cores.