Do not offset bounding box if not held by player