added nokogiri
fixing user agent
fixing bundle
fixing nokogiri
fixing nokogiri
added several features and possibly fixed url extraction