Package: google-search-results-serpwow
Google Search Results in PythonThis Python package allows you to scrape and parse Google Search Results usingSerpWow. In addition toSearchyou can also use this package to access the SerpWowLocations API,Batches APIandAccount API.InstallationYou can install google-search-results-serpwow with:$pipinstallgoogle-search-results-serpwowand upgrade with:$pipinstallgoogle-search-results-serpwow--upgradeDocumentationWe have included examples here but full SerpWow API documentation is available at theAPI Docs:Search API DocsLocations API DocsAccount API DocsBatches API DocsYou can also use theAPI Playgroundto visually build Google search requests using SerpWow.ExamplesSimple ExampleExample ResponseGetting an API KeySearching with a LocationSearching Google Places, Google Videos, Google Images, Google Shopping and Google NewsReturning results as JSON, HTML and CSVRequesting mobile and tablet resultsParsing ResultsPaginating results, returning up to 100 results per pageSearch example with all parametersLocations API ExampleAccount API ExampleBatches APISimple ExampleSimplest example for a standard query "pizza", returning the Google SERP (Search Engine Results Page) data as JSON:fromserpwow.google_search_resultsimportGoogleSearchResultsimportjson# create the serpwow object, passing in our API keyserpwow=GoogleSearchResults("API_KEY")# set up a dict for the search parametersparams={"q":"pizza"}# retrieve the search results as JSONresult=serpwow.get_json(params)# pretty-print the resultprint(json.dumps(result,indent=2,sort_keys=True))Example ResponseA snapshot (shortened for brevity) of the JSON response returned is shown below. For details of all of the fields from the Google search results page that are parsed please see thedocs.{"request_info":{"success":true},"search_metadata":{"id":"20c8e44e9cacedabbdff2d9b7e854436056d4f33","engine_url":"https://www.google.com/search?q=pizza&oq=pizza&sourceid=chrome&ie=UTF-8","total_time_taken":0.14},"search_parameters":{"q":"pizza"},"search_information":{"total_results":1480000000,"time_taken_displayed":0.45,"query_displayed":"pizza","detected_location":"Ireland"},"local_map":{"link":"https://www.google.com/search?q=pizza&npsic=0&rflfq=1&rldoc=1&rlha=0&rllag=53350059,-6249133,1754&tbm=lcl&sa=X&ved=2ahUKEwiC3cLZ0JLgAhXHUxUIHQrsBC4QtgN6BAgBEAQ","gps_coordinates":{"latitude":53.350059,"longitude":-6.249133,"altitude":1754},"local_results":[{"position":1,"title":"Apache Pizza Temple Bar","extensions":["American-style pizza-delivery chain"],"rating":3.6,"reviews":382,"type":"Pizza","block_position":1}]},"knowledge_graph":{"title":"Pizza","type":"Dish","description":"Pizza is a savory dish of Italian origin, consisting of a usually round, flattened base of leavened wheat-based dough topped with tomatoes, cheese, and various other ingredients baked at a high temperature, traditionally in a wood-fired oven.","source":{"name":"Wikipedia","link":"https://en.wikipedia.org/wiki/Pizza"},"nutrition_facts":{"total_fat":["10 g","15%"],"sugar":["3.6 g"]}},"related_searches":[{"query":"apache pizza","link":"https://www.google.com/search?q=apache+pizza&sa=X&ved=2ahUKEwiC3cLZ0JLgAhXHUxUIHQrsBC4Q1QIoAHoECAUQAQ"}],"organic_results":[{"position":1,"title":"10 Best Pizzas in Dublin - A slice of the city for every price point ...","link":"https://www.independent.ie/life/travel/ireland/10-best-pizzas-in-dublin-a-slice-of-the-city-for-every-price-point-37248689.html","domain":"www.independent.ie","displayed_link":"https://www.independent.ie/.../10-best-pizzas-in-dublin-a-slice-of-the-city-for-every-p...","snippet":"Oct 20, 2018 - Looking for the best pizza in Dublin? Pól Ó Conghaile scours the city for top-notch pie... whatever your budget.","cached_page_link":"https://webcache.googleusercontent.com/search?q=cache:wezzRov42dkJ:https://www.independent.ie/life/travel/ireland/10-best-pizzas-in-dublin-a-slice-of-the-city-for-every-price-point-37248689.html+&cd=4&hl=en&ct=clnk&gl=ie","block_position":2}],"related_places":[{"theme":"Best dinners with kids","places":"Pinocchio Italian Restaurant - Temple Bar, Cafe Topolisand more","images":["https://lh5.googleusercontent.com/p/AF1QipNhGt40OpSS408waVJUHeItGrrGqImmEKzuVbBv=w152-h152-n-k-no"]}],"pagination":{"current":"1","next":"https://www.google.com/search?q=pizza&ei=fRZQXMKqL8en1fAPitiT8AI&start=10&sa=N&ved=0ahUKEwiC3cLZ0JLgAhXHUxUIHQrsBC4Q8NMDCOkB"}}Getting an API KeyTo get a free API Key head over toapp.serpwow.com/signup.Searching with a locationExample of a Google query geo-locating the query as if the user were located in New York.fromserpwow.google_search_resultsimportGoogleSearchResultsimportjson# create the serpwow object, passing in our API keyserpwow=GoogleSearchResults("API_KEY")# set up a dict for the query (q) and location parameters# note that the "location" parameter should be a value# returned from the Locations APIparams={"q":"pizza","location":"New York,New York,United States"}# retrieve the search results as JSONresult=serpwow.get_json(params)# pretty-print the resultprint(json.dumps(result,indent=2,sort_keys=True))Searching Google Places, Google Videos, Google Images, Google Shopping and Google NewsUse thesearch_typeparam to search Google Places, Videos, Images and News. See theSearch API Parameters Docsfor full details of the additional params available for each search type.fromserpwow.google_search_resultsimportGoogleSearchResultsimportjson# create the serpwow object, passing in our API keyserpwow=GoogleSearchResults("API_KEY")# perform a search on Google News, just looking at blogs, ordered by date, in the last year, filtering out duplicatesparams={"q":"football news","search_type":"news","news_type":"blogs","sort_by":"date","time_period":"last_year","show_duplicates":"false"}result=serpwow.get_json(params)print(json.dumps(result,indent=2,sort_keys=True))# perform a search on Google Places for "plumber" in Londonparams={"search_type":"places","q":"plumber","location":"London,England,United Kingdom"}result=serpwow.get_json(params)print(json.dumps(result,indent=2,sort_keys=True))# perform an image search on Google Images for "red flowers"params={"q":"red flowers","search_type":"images"}result=serpwow.get_json(params)print(json.dumps(result,indent=2,sort_keys=True))Returning results as JSON, HTML and CSVSerpWow can return data in JSON, HTML and CSV formats using theget_json,get_htmlandget_csvmethods. For CSV results use thecsv_fieldsparam (docs) to request specific result fields.fromserpwow.google_search_resultsimportGoogleSearchResultsimportjson# create the serpwow object, passing in our API keyserpwow=GoogleSearchResults("API_KEY")# set up a dict for the query (q) and location parameters# note that the "location" parameter should be a value# returned from the Locations APIparams={"q":"pizza","location":"New York,New York,United States"}# retrieve the Google search results as JSONresult_json=serpwow.get_json(params)# retrieve the Google search results as HTMLresult_html=serpwow.get_html(params)# retrieve the Google search results as a CSVresult_csv=serpwow.get_csv(params)Requesting mobile and tablet resultsTo request that SerpWow renders the Google search results via a mobile or tablet browser use thedeviceparam:fromserpwow.google_search_resultsimportGoogleSearchResultsimportjsonserpwow=GoogleSearchResults("API_KEY")# set up the mobile paramsparams_mobile={"q":"pizza","device":"mobile"}# set up the tablet paramsparams_tablet={"q":"pizza","device":"tablet"}# set up the desktop params (note we omit the "device" param)params_desktop={"q":"pizza"}# retrieve the mobile search resultsresult_mobile_json=serpwow.get_json(params_mobile)# retrieve the tablet search resultsresult_tablet_json=serpwow.get_json(params_tablet)# retrieve the desktop search resultsresult_desktop_json=serpwow.get_json(params_desktop)Parsing resultsWhen making a request via theget_jsonmethod a standard Pythondictis returned. You can inspect this dict to iterate, parse and store the results in your app.fromserpwow.google_search_resultsimportGoogleSearchResultsimportjson# make a simple query, returning JSONserpwow=GoogleSearchResults("API_KEY")result=serpwow.get_json({"q":"pizza"})# determine if the request was successfulsuccess=result["request_info"]["success"]ifsuccess:# extract the time taken and number of organic resultstime_taken=result["search_metadata"]["total_time_taken"]organic_result_count=len(result["organic_results"])# printprintstr(organic_result_count)+" organic results returned in "+str(time_taken)+"s"Paginating results, returning up to 100 results per pageUse thepageandnumparameters to paginate through Google search results. The maximum number of results returned per page (controlled by thenumparam) is 100 (a Google-imposed limitation) for allsearch_type's apart from Google Places, where the maximum is 20. Here's an example.fromserpwow.google_search_resultsimportGoogleSearchResultsimportjson# request the first 100 resultsserpwow=GoogleSearchResults("API_KEY")params={"q":"pizza","page":1,"num":100}result_page_1=serpwow.get_json(params)# request the next 100 resultsparams["page"]=2result_page_2=serpwow.get_json(params)# pretty-print the resultprint"Page 1"printjson.dumps(result_page_1,indent=2,sort_keys=True)print"Page 2"printjson.dumps(result_page_2,indent=2,sort_keys=True)Search example with all parametersfromserpwow.google_search_resultsimportGoogleSearchResults# create the serpwow object, passing in our API keyserpwow=GoogleSearchResults("API_KEY")# set up a dict for the search parameters, retrieving results as CSV (note the csv_fields param)params={"q":"pizza","gl":"us","hl":"en","location":"New York,New York,United States","google_domain":"google.com","time_period":"custom","sort_by":"date","time_period_min":"02/01/2018","time_period_max":"02/08/2019","device":"mobile","csv_fields":"search.q,organic_results.position,organic_results.domain","page":"1","num":"100"}# retrieve the search results as CSVresult=serpwow.get_csv(params)printresultLocations API ExampleTheLocations APIallows you to search for SerpWow supported Google search locations. You can supply thefull_namereturned by the Locations API as thelocationparameter in a Search API query (seeSearching with a locationexample above) to retrieve search results geo-located to that location.fromserpwow.google_search_resultsimportGoogleSearchResultsimportjson# create the serpwow object, passing in our API keyserpwow=GoogleSearchResults("API_KEY")# set up a dict for the location query parametersparams={"q":"mumbai"}# retrieve locations matching the query parameters as JSONresult=serpwow.get_locations(params)# pretty-print the resultprint(json.dumps(result,indent=2,sort_keys=True))Account API ExampleTheAccount APIallows you to check your current SerpWow usage and billing information.fromserpwow.google_search_resultsimportGoogleSearchResultsimportjson# create the serpwow object, passing in our API keyserpwow=GoogleSearchResults("API_KEY")# get our account inforesult=serpwow.get_account()# pretty-print the resultprint(json.dumps(result,indent=2,sort_keys=True))Batches APITheBatches APIallows you to create, update and delete Batches on your SerpWow account (Batches allow you to save up to 15,000 Searches and have SerpWow run them on a schedule).For more information and extensive code samples please see theBatches API Docs.