Before revision:  Transformer and BERT modelsoperate in their first layers preferably in the global averaging regime, while they operate in higher layers in metastable states.
After revision:  Transformer and BERT modelsoperate in their first layers preferably in the global averaging regime, while they operate in higher layers partial averaging via metastable states.