Take dragging-status into account when moving to frame