Remove usages of to_unsafe_ptr