Make sure that archive jar is deleted