Remove dependency on numpy

Closes #17.

The dependency was merely a convenience and functionally not necessary. By
removing it we gain pypy compatibility and make the package more light-weight
in general.