Reorganize dependencies

Remove dependency to sphinx for tests. This speeds up travis builds a
bit