Can you use Multimap instead? That would simplify some logic.