We now pass in the maxDirectionChanges to BuildMatrixFor