Matrix: do not create a new block for the matrix given invalid indexes.