Worker refactoring to get rid of state where everything depends on everything else

This is pool of changes that changes all APIs that depend on worker, although I do plan to split them into separate pull requests, it's still maybe worth if @ochameau will start looking into it.

P.S.: I'm afraid I did not managed to do everything I promised @ochameau other day, also I could not get internet access where I would be able to push to git so doing it only now.