Eliminate reference to ublas matrix