Switch a vector<pair<const T &, const U &>> to a vector<pair<const T *,
const U *>>. Even in C++11 it doesn't seem this is valid as vector's
emplace support requires move assignment, and there is no way to move
assign a reference.

The real motivation however is that this fixes the build of lld with
libstdc++ 4.6.