Fix use of pointers when parsing.

Using append, the whole array may get reallocated, which means
that existing pointers into the array become stale. Switch to
indices.