Revert some of the last changes

Travis uses g++ 4.6 which doesn't support map.emplace yet.