Remove ref_ptr in favor of std::unique_ptr