Revert "Use std::map<K, V> rather than std::map<K, std::unique_ptr<V>>"

Seems libstdc++ on some buildbots is lacking std::map::emplace, which is
weird... reverting while I look into it.

This reverts commit r222937.