Improve the remove of SendResponseListener