Adds the off-heap dense matrices and primitive sparse