5.0.0

This is just a copy of the 4.1.0 (#340) branch, since there are so many changes here I think we should release it as 5.0.0-b1 first.
There has been a lot of discussion in the old PR at #340