Fix performance bug.

Make smart pointers less smart and more efficient.