Add the processing which remove notifier at skiplist_destroy function