Remove most of the dependence on row index