Revert "[cpp] Keep separate key map to ensure that references are held.  Fix toString"

This reverts commit 83d1dbbbb5b3ae415ac3ebde20bf6aee2180cf71.