[JENKINS-17067] Cancelling the deliver