Don't remove when raising Alreadyfetched (s3-proxy)

c52a5f157ad8db95258a67f6622b12a4eeca6274