I think this needs to take `mut self` to be sound, otherwise I believe it can hand out multiple mutable pointers to `T`