transformers upper bound