I think you want an `Either[List[Matrix...], (Long, Matrix)]` so you can be sure that when you don't have a list (don't use the empty one) you always have the (Long, Matrix) and the reader and compiler can see the same.