Fix critical warnings when starting a new game

Never try to remove the same source twice