improve matmul result type. closes #4807, closes #4796