loosen the lower bound of transformers