Research indices using web scraped price data
Source: data.gov.uk (2016-05-23)

Web scraping is a tool for extracting information from the underlying HTML code of websites. ONS has been conducting research into these technologies and, since May 2014, has been scraping prices from the websites of three retailers. Last year, ONS released two updates that constructed experimental price indices from the data. In this release, we provide updates to the experimental indices, and an analysis of the different methods used to clean and classify the data.

Cookies on ons.gov.uk

Cookies are small files stored on your device when you visit a website. We use some essential cookies to make this website work.

We would like to set additional cookies (/cookies) to remember your settings and understand how you use the site. This helps us to improve our services.

Accept additional cookies

Reject additional cookies

Manage settings (/cookies)

You have accepted all additional cookies. You have rejected all additional cookies. You can change your cookie preferences (/cookies) at any time. Hide

Beta

This is a new service – your feedback (/feedback) will help us improve it.

/releases/researchindicesusingwebscrapedpricedatamay2016update

(/)

English (EN) | Cymraeg (CY) (https://cy.ons.gov.uk/releases/researchindicesusingwebscrapedpricedatamay2016update)
- Release calendar (/releasecalendar)
- Methodology (/methodology)
- Media (/news)
- About (/aboutus)
- Blog (https://blog.ons.gov.uk/)
- Menu (#nav-primary)
- Search (#nav-search)
- Home (/)
- Business (/businessindustryandtrade/business)
- Changes to business (/businessindustryandtrade/changestobusiness)
- Construction industry (/businessindustryandtrade/constructionindustry)
- IT and internet industry (/businessindustryandtrade/itandinternetindustry)
- International trade (/businessindustryandtrade/internationaltrade)
- Manufacturing and production industry (/businessindustryandtrade/manufacturingandproductionindustry)
- Retail industry (/businessindustryandtrade/retailindustry)
- Tourism industry (/businessindustryandtrade/tourismindustry)
- Economic output and productivity (/economy/economicoutputandproductivity)
- Environmental accounts (/economy/environmentalaccounts)
- Government, public sector and taxes (/economy/governmentpublicsectorandtaxes)
- Gross Domestic Product (GDP) (/economy/grossdomesticproductgdp)
- Gross Value Added (GVA) (/economy/grossvalueaddedgva)
- Inflation and price indices (/economy/inflationandpriceindices)
- Investments, pensions and trusts (/economy/investmentspensionsandtrusts)
- National accounts (/economy/nationalaccounts)
- Regional accounts (/economy/regionalaccounts)
- People in work (/employmentandlabourmarket/peopleinwork)
- People not in work (/employmentandlabourmarket/peoplenotinwork)
- Armed forces community (/peoplepopulationandcommunity/armedforcescommunity)
- Births, deaths and marriages (/peoplepopulationandcommunity/birthsdeathsandmarriages)
- Crime and justice (/peoplepopulationandcommunity/crimeandjustice)
- Cultural identity (/peoplepopulationandcommunity/culturalidentity)
- Education and childcare (/peoplepopulationandcommunity/educationandchildcare)
- Elections (/peoplepopulationandcommunity/elections)
- Health and social care (/peoplepopulationandcommunity/healthandsocialcare)
- Household characteristics (/peoplepopulationandcommunity/householdcharacteristics)
- Housing (/peoplepopulationandcommunity/housing)
- Leisure and tourism (/peoplepopulationandcommunity/leisureandtourism)
- Measuring progress, well-being and beyond GDP (/peoplepopulationandcommunity/wellbeing)
- Personal and household finances (/peoplepopulationandcommunity/personalandhouseholdfinances)
- Population and migration (/peoplepopulationandcommunity/populationandmigration)
- Taking part in a survey? (/surveys)
- English (EN) | Cymraeg (CY) (https://cy.ons.gov.uk/releases/researchindicesusingwebscrapedpricedatamay2016update)
Search for a keyword(s) or time series ID Search
- Home (/)
- Release calendar (/releasecalendar)
- Published (/releasecalendar?release-type=type-published)
Release

Research indices using web scraped price data: May 2016 update

Release date: 23 May 2016 9:30am

Contents
- Summary (#summary)
- Publications (#publications)
- Data (#data)
- Methodology (#methodology)
- Contact details (#contactdetails)
- You might also be interested in (#links)
Summary

Web scraping is a tool for extracting information from the underlying HTML code of websites. ONS has been conducting research into these technologies and, since May 2014, has been scraping prices from the websites of three retailers. Last year, ONS released two updates that constructed experimental price indices from the data. In this release, we provide updates to the experimental indices, and an analysis of the different methods used to clean and classify the data.

Publications

Research indices using web scraped price data: May 2016 update (/economy/inflationandpriceindices/articles/researchindicesusingwebscrapedpricedata/may2016update)

Web scraping is a tool for extracting information from the underlying HTML code of websites. ONS has been conducting research into these technologies and, since May 2014, has been scraping prices from the websites of three retailers. Last year, ONS released two updates that constructed experimental price indices from the data. In this release, we provide updates to the experimental indices, and an analysis of the different methods used to clean and classify the data.

Data

Research indices using web scraped price data - Appendix tables (/economy/inflationandpriceindices/datasets/researchindicesusingwebscrapedpricedataappendixtables)

This workbook presents experimental price indices that are constructed from ONS web scraped data from three grocery retailers since May 2014. These are early analysis using experimental techniques to help us develop our statistical methodology and are not comparable with headline estimates of inflation. We would strongly caution against their use in economic modelling and analysis.

Methodology

Imputing Web Scraped Prices (/economy/inflationandpriceindices/methodologies/imputingwebscrapedprices)

Contact details

Name

Tanya Flower

Email

xxx@xxx.xx (mailto:xxx@xxx.xx)

Phone

xx-xxxx-xxxx (tel:+44%20%280%291633%20455171)

You might also be interested in
- Research indices using web scraped price data: interactive (https://onsvisual.github.io/dvc344/project/index.html)
Is this page useful?

Report a problem with this page (/feedback)

How should we improve this page?

If you'd like us to get back to you, please add your name and email address below.

Name (optional)

Email (optional)

Send feedback I don't want to provide feedback

Footer links

Help
- Accessibility (/help/accessibility)
- Cookies (/cookies)
- Privacy (/help/privacynotice)
- Terms and conditions (/help/termsandconditions)
About ONS
- What we do (/aboutus/whatwedo)
- Careers (https://careers.ons.gov.uk)
- Contact us (/aboutus/contactus)
- News (/news)
- Freedom of Information (/aboutus/transparencyandgovernance/freedomofinformationfoi)
Connect with us
- X (https://twitter.com/ONS)
- Instagram (https://www.instagram.com/officefornationalstatistics/)
- Facebook (https://www.facebook.com/ONS)
- LinkedIn (https://www.linkedin.com/company/office-for-national-statistics)
- Consultations (https://consultations.ons.gov.uk/)
- Discussion forums (https://www.statsusernetwork.ons.gov.uk/)
- Email alerts (https://public.govdelivery.com/accounts/UKONS/subscribers/new)
All content is available under the Open Government Licence v3.0 (http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/) , except where otherwise stated