I'm a dumbass, removed reference counter class and just used shared_ptr for matrix data. (best part is I was using shared_ptr to maintain references to the reference counter class)