Decrease lower dependency bound for transformers