Build out a label -> sibling label map on startup.