Implement RemoveFailed config option