ENH: Minor performance improvement in load scene

Use set instead of vector for storing unique IDs