simplify handling of scoremat parameter
Avoid subsetting big matrix by names

* fetching names is expensive so cache and then use numeric
  indices
* I noticed ~ 20x speedup on tests datasets