Picked a lot of solver improvements from develop
Merged crawler changes from develop branch