Nasty data structure to keep track of priority for each path