Before revision:  The LMs were trained on the Summit supercomputer , using 5616 GPUs  and one TPU Pod , using V3-512 cores.
After revision:  The LMs were trained on the Summit supercomputer , using 5616 GPUs and one TPU Pod ( V3-512 cores.