Fix some unecessary smart pointer usage