Refactor

A PR with a lot of breaking changes. Check a new README.rst to get an overview. 

@tpeng could you check it and merge?