extended simple scrapping to cope with different styles of URLs