I think this has a performance hit. While profiling code, I realized a similar mistake I had made in SVDPP. Profiling showed me how the amount of garbage created in each iteration was far more.