Only install python3-numpy with CI