Add build matrix.

Testing current and previous numpy.