convert matrices to uint8, improve error metrics