Use vectors of shared_ptr rather than plain pointer.

Taking a bit of a gamble that this won't break elsewhere.