To remove Autoptimize you have to delete it from the plugin page and deleting all the files in the caches.