Remove unnecessary dependencies

Removed the dependency on numpy since that was totally unused. Also
cleaned code and pruned unnecessary pieces.