remove unnecessary not null assertion
extract page loader, create crawler exception, and some other design changes + logging