Do not clone nmatrix if already there