Remove the remaining raw pointers

Fixes #195