Remove from the proxy list if we can't insert it