Package: google-shopping-scraper-api
Google Shopping ScraperAs part ofE-Commerce Scraper API,Google Shopping Scraperextracts timely e-commerce data in raw HTML or structured JSON format. The scraper offers a maintenance-free data collection infrastructure that automates the bulk of underlying processes, from sending HTTP requests to data parsing.The underlying measures, such as proxies, ensure considerably fewer CAPTCHAs and IP blocks. The scraper supports localized results from almost any locale worldwide (195 countries) with country-level and postal code targeting.Additionally, the scraper can automate recurring scraping and parsing jobs throughScheduler, load dynamic websites that use JavaScript for rendering content, and retrieve results via the API or directly to Google Cloud Storage or Amazon S3 storage bucket.How it worksThere are various page types we can scrape and parse on Google Shopping. You can either provide us with a fullURLor a few input parameters via specifically built data sources (e.g.Search,Product,Product Pricingso we can form the URL on our end.OverviewBelow is a quick overview of all the available datasourcevalues we support with Google Shopping.SourceDescriptionStructured datagoogleSubmit any Google Shopping URL you like.Depends on the URL.google_shopping_searchSearch results for a search term of your choice.Yes.google_shopping_productProduct page of a product ID of your choice.Yes.google_shopping_pricingList of offers available for a product ID of your choice.Yes.URLThegooglesource is designed to retrieve content from various Google Shopping URLs. Instead of sending multiple parameters and letting us form and scrape Google Shopping URLs, you can provide us with a URL to the required Google Shopping page. We do not strip any parameters or alter your URLs in any other way.This data source also supports parsed data (structured data in JSON format), as long as the URL submitted links to a page that we can parse.Query parametersParameterDescriptionDefault ValuesourceData source.More info.googleurlDirect URL (link) to Google page-user_agent_typeDevice type and browser. The full list can be foundhere.desktoprenderEnables JavaScript rendering.More info.callback_urlURL to your callback endpoint.More info.-geo_locationThe geographical location that the result should be adapted for. Using this parameter correctly is extremely important to get the right data. For more information, read about our suggestedgeo_locationparameter structureshere.-parsetruewill return parsed data, as long as the URL submitted is for Google Search.-- required parameterPython code exampleIn this example, we make a request to retrieve a Google Shopping Search result for keywordadidas, as seen in New York, USA.importrequestsfrompprintimportpprint# Structure payload.payload={'source':'google','url':'https://www.google.com/search?tbm=shop&q=adidas&hl=en','geo_location':'New York,New York,United States'}# Get response.response=requests.request('POST','https://realtime.oxylabs.io/v1/queries',auth=('user','pass1'),json=payload,)# Instead of response with job status and results url, this will return the# JSON response with results.pprint(response.json())Code examples for other languages can be foundhere.Shopping SearchThegoogle_shopping_searchsource is designed to retrieve Google Shopping search results.Query parametersParameterDescriptionDefault ValuesourceData source.More info.google_shopping_searchdomainDomain localizationcomqueryUTF-encoded keyword-start_pageStarting page number1pagesNumber of pages to retrieve1localeAccept-Languageheader value which changes your Google Shopping page web interface language.More info.-results_languageResults language. List of supported Google languages can be foundhere.-geo_locationThe geographical location that the result should be adapted for. Using this parameter correctly is extremely important to get the right data. For more information, read about our suggestedgeo_locationparameter structureshere.-user_agent_typeDevice type and browser. The full list can be foundhere.desktoprenderEnables JavaScript rendering.More info.-callback_urlURL to your callback endpoint.More info.-parsetruewill return parsed data.-context:nfprtruewill turn off spelling auto-correction.falsecontext:sort_bySort product list by a given criteria.rapplies default Google sorting,rv- by review score,p- by price ascending,pd- by price descendingrcontext:min_priceMinimum price of products to filter-context:max_priceMaximum price of products to filter-- required parameterPython code exampleIn this example, we make a request to retrieve the first4pages of Google Shopping search for the search termadidas, sorted by descending price and minimum price of$20.importrequestsfrompprintimportpprint# Structure payload.payload={'source':'google_shopping_search','domain':'com','query':'adidas','pages':4,'context':[{'key':'sort_by','value':'pd'},{'key':'min_price','value':20},],}# Get response.response=requests.request('POST','https://realtime.oxylabs.io/v1/queries',auth=('user','pass1'),json=payload,)# Print prettified response to stdout.pprint(response.json())Code examples for other languages can be foundhere.Shopping ProductThegoogle_shopping_productsource is designed to retrieve Google Shopping product page for a specified product.Query parametersParameterDescriptionDefault ValuesourceData source.More info.google_shopping_productdomainDomain localizationcomqueryUTF-encoded product code-localeAccept-Languageheader value which changes your Google Shopping page web interface language.More info.-results_languageResults language. List of supported Google languages can be foundhere.-geo_locationThe geographical location that the result should be adapted for. Using this parameter correctly is extremely important to get the right data. For more information, read about our suggestedgeo_locationparameter structureshere.-user_agent_typeDevice type and browser. The full list can be foundhere.desktoprenderEnables JavaScript rendering.More info.callback_urlURL to your callback endpoint.More info.-parsetruewill return parsed data.-- required parameterPython code exampleIn the code example below, we make a request to retrieve the product page for product ID5007040952399054528from Google Shopping oncomdomain.importrequestsfrompprintimportpprint# Structure payload.payload={'source':'google_shopping_product','domain':'com','query':'5007040952399054528',}# Get response.response=requests.request('POST','https://realtime.oxylabs.io/v1/queries',auth=('user','pass1'),json=payload,)# Print prettified response to stdout.pprint(response.json())Code examples for other languages can be foundhere.Product PricingThegoogle_shopping_pricingsource is designed to retrieve pages containing lists of offers available for a product ID of your choice.Query parametersParameterDescriptionDefault ValuesourceData source.More info.google_shopping_pricingdomainDomain localizationcomqueryUTF-encoded product code-start_pageStarting page number1pagesNumber of pages to retrieve1localeAccept-Languageheader value which changes your Google Shopping page web interface language.More info.-results_languageResults language. List of supported Google languages can be foundhere.-geo_locationThe geographical location that the result should be adapted for. Using this parameter correctly is extremely important to get the right data. For more information, read about our suggestedgeo_locationparameter structureshere.-user_agent_typeDevice type and browser. The full list can be foundhere.desktoprenderEnables JavaScript rendering.More info.callback_urlURL to your callback endpoint.More info.-parsetruewill return parsed data.-- required parameterPython code exampleIn the code example below, we make a request to retrieve the product pricing page for product ID5007040952399054528from Google Shopping ongoogle.com.importrequestsfrompprintimportpprint# Structure payload.payload={'source':'google_shopping_pricing','domain':'com','query':'5007040952399054528',}# Get response.response=requests.request('POST','https://realtime.oxylabs.io/v1/queries',auth=('user','pass1'),json=payload,)# Print prettified response to stdout.pprint(response.json())Code examples for other languages can be foundhere..If you have questions or concerns about Google Shopping Scraper or associated features, get in touch via (support@oxylabs.io) or through the live chat on ourwebsite.