possibly they should even be std::maps, where the key is just an integer so we can remove and add without breaking the source_id.