Object lifetime was the issue, if you refer to things on the C++ side by reference you have to make sure they don't get gc'd on the R side\!