Before revision:  The LMs were trained on the Summit supercomputer at Oak Ridge National Laboratory (ORNL), using 936 nodes (total 5616 GPUs ) and one TPU Pod (V3-512 or V3-1024).
After revision:  The LMs were trained on the Summit supercomputer using 5616 GPUs ) and one TPU Pod (V3-512 or V3-1024).