lightning_mnist_tpu Colab notebook uses a GPU instead of a TPU
https://colab.research.google.com/drive/1-_LKx4HwAxl5M6xPJmqAAu444LTDQoa3#scrollTo=BHBz1_AnamN_
I am assuming that is not intended.
Notebook works fine after changing the runtime to TPU and the train params accordingly