What's stopping them from being borrowed pointers now?

(It would be good to either have a fixme pointing to an issue, or a longer description about why these aren't using borrowed pointers, so that someone can look at the code/bug tracker to work out if `*` can be changed to `&` or not.)