Address some of the Matrix2 TODOs

Merge only AFTER #605 which this is based on.