update bounding box vertices according to transform