Fix matrix allocs, parallel_mult up to allgathers