Remove the element from list when adding to queue