Make sure we're freeing all unsafe pointers

[ci skip]