Developing and Enhancing Worldwide 

Energy Data

Duncan Millard, Chief Statistician & Head of the Energy Data Centre, IEA

IEA
© IEA 2017

Overview

The IEA and its Energy Data Centre

Work programmes with countries and regions 

Energy Data Progress against Challenges and the potential of Digitalisation
The IEA and its Energy Data Centre
© IEA 2017

The IEA

Formed in 1973 in wake of oil embargo with mission to promote member country energy security –

autonomous agency of the Organisation for Economic Cooperation and Development (OECD)

30 member countries

Asia Pacific: Australia, Japan, Republic of Korea and New Zealand

Americas: United States, Canada, Mexico (since Feb 2018)

Europe: Austria, Belgium, Czech Rep, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, 

Ireland, Italy, Luxembourg, Netherlands, Norway, Poland, Portugal, Slovak Republic, Spain, Sweden, 

Switzerland, Turkey and United Kingdom

European Commission also participates in the work of the IEA

Chile is in the process of accession to become members of the IEA

Brazil, China, India, Indonesia, Morocco, Singapore and Thailand are countries in Association

Decision-making body: Governing Board - Consists of member country representatives

Under the Governing Board, several committees are focusing on each area

Secretariat: Staff of around 260, mainly energy experts and statisticians

Modernisation and Open door policy started in 2015
© IEA 2017

Total primary energy supply (TPES) by region

 0

2 000

4 000

6 000

8 000

10 000

12 000

14 000

16 000

1990 1995 2000 2005 2010 2015

World TPES from 1990 to 2015 

by region (Mtoe)

Americas Europe Asia Africa Oceania Bunkers1

Source: IEA, Key World Energy Statistics, 2017
© IEA 2017

World total primary energy supply by fuel

6,101 Mtoe 13 647Mtoe

Source: IEA, Key World Energy Statistics, 2017
© IEA 2017

Data collection at the IEA: the road to global outreach

• IEA collects data from 150 countries, on a voluntary basis for all non-OECD countries

NON-OECD

IEA

OECD Non IEA

Association

Rest
© IEA 2017

OECD

• 5 annual Joint questionnaires

• Annual collection on Energy efficiency, 

RD&D and forecasts

• Quarterly questionnaire on prices and 

taxes

• Monthly questionnaire 

• Electricity

• Oil and gas (M-2)

• Oil and gas (JODI) (M-1)

• Emergency questionnaires

Non-OECD

• Voluntary

• Joint IEA/UNECE/Eurostat 

questionnaires used by some and 

growing

• Network of statistics contacts in 120 

countries

• Cooperation with international 

organisations

Data Collection by the IEA – source: direct from countries

Common objective for all - to improve timeliness and detail of data sent to IEA

http://www.iea.org/statistics/resources/questionnaires/
© IEA 2017

What Support Does The IEA Provide?

Continuous 

improvement 

in data 

quality

Manuals

Training

Capacity 

Building

Open 

communication 

with country 

sources

Uniformity 

across 

agencies
© IEA 2017

Energy Statistics Manuals and guidance

Available in 10 languages Available in 4 languages

Over 170 country practices
© IEA 2017

IEA Statistics Training and Capacity Building Overview in 2017- 2018

The IEA directly trained 496 statisticians worldwide and over 4,500 online

Association countries

Accession countries

IEA

183

25

224

39

40

40

People trained/continent

People trained/country

20

3

39

32

90
© IEA 2017

Online Training: The IEA Online School Program 2017

9 webinars to cover the range of energy statistics, http://www.iea.org/training/ieaonlinestatisticstrainingprogramme/ieaonlinestatisticsschool2017/

(aiming to make available in other languages: French, Spanish, Arabic, Chinese, Russian) 

Module Content Webinar Date of release Views

Prices Both on data collection 

and analysis of data.

Energy prices (data 

reporting and use)

16/08 1 009

Data analysis

Dedicated to energy 

analysts.

Energy efficiency 23/08 956

Energy balances 06/09 1 107

CO2 13/09 243

Data reporting

Primarily for our 

contacts to be trained in 

how to use the Joint 

Questionnaires.

Renewables 23/08 240

Coal 20/09 300

Electricity and heat 27/09 246

Oil 04/10 294

Gas 11/10 321
© IEA 2017

Fundamental of energy statistics in Chinese

IEA stats videos in Chinese launched on 25 May 2018 in China as part of training event on renewables 

and energy efficiency available on NBS website
Work programmes with countries and regions
© IEA 2017

• India – assistance to develop energy balances, expend knowledge, data 

collection, data consistency

• China – renewables and energy efficiency

• Indonesia – balances and energy consumption

• Mexico – data in a new market structure

• Africa – energy consumption and consistency with energy balances

• South America – international data consistency

Directed workplans
© IEA 2017

India

Ongoing cooperation for many years, but detailed IEA support from 2018 :

Fact finding mission in March 2018, discussion on the current state of India’s energy statistics and means to 

improve with 14 ministries.

Strong commitment from all on cooperation to improve data, all seeing the benefit to India and the IEA in 

so doing.

Two new cross ministry groups being established, on IEA recommendation, to better coordinate work on 

data and data sharing across ministries with IEA as a member in an advisory role.

• Ministry of Statistics and Program Implementation (MoSPI), responsible for the energy balance

- MOSPI would share with IEA the forms used to collect data from ministries, so we can advise on changes needed to build a full

balance. 

- This would be complemented by a training course during 2018.

- Cooperation with NSSO (survey office of MoSPI) will be explored to expand data collection and improve demand side data. 

• PPAC agreed to join our Petro-Chemical working group, which is working to improve the reporting of this key sector – they would be the first non-IEA member to join.

• Work agreed to unify classifications of products and industries and link them to international definitions.
© IEA 2017

• An MoU was signed in March between BPS (stats office) and ESDM

- a significant development in improved sharing and access to data and also opens up the route for 

ESDM to include questions in BPS surveys, IEA recommendations from the 2017 mission.

• Meeting with senior advisor to the Minister from strategic planning at ESDM to discuss 

data needed for sensible energy efficiency policy options:

- Focus on the use of the data that exists but is not shared yet (including administrative data) and 

small additions to surveys that could significantly improve knowledge on why energy is used. 

• Very successful cross ministry meeting on energy end use data, but need to make this 

cross ministerial cooperation part of normal work.

- Real discussions between the ministries helping them identify data overlaps and sources of data. 

• New plans to measure the CV of coal data on export (covers 80% of production) from 

2018 third quarter. 

Indonesia 
© IEA 2017

• Association strengthened the relationship with China National Bureau of Statistics 

(NBS) and make significant progress in understanding their energy data system:

- Training events in May 2016 (Beijing), that taught to 100 Province statisticians, and key 

meetings with China Electricity Council and Tsinghua University

- Fact-finding mission in March 2017 in the Zhejiang region, gave a deep understanding of 

data collection and validation at all geographical levels as well as the companies’ work to 

comply with it

- Secondee from NBS to EDC – work on classification comparisons

- International workshop on renewables and energy efficiency statistics (Beijing), attended by 

100 Province statisticians and 10 neighboring countries – May 2018. 

- Launch on-line training in Chinese

• Since China joined Association, improvements on data affected:

- The estimation of biomass used by residential

- Rail consumption of coal adjusted for the 1971-2016

- Improved timing in responses to questions and ability to raise more questions

China
© IEA 2017

South America

• Agreement from Olade and the 6 countries to support then to develop a new questionnaire for reporting to Olade that will be consistent with the IEA forms and will mean consistent data is reported to the IEA by these 

countries. And separate agreement to support Brazil to do the same.

• All done by taking time to discuss unique data flows and process in South America and showing how IEA 

reporting process did work for them.

Africa

• The IEA focus is on quality, comparability, and alignment with internationally agreed definitions and 

methodologies, and expand its individual coverage beyond 30 covered , work with other organisations on the 

ground (AFREC, IRENA, IAEA, etc), particularly on training and capacity building activities.

- 2017: 6 training events with AFREC , 2 on energy balances and statistics 4 on energy efficiency data

- 2018 : 4 workshops done or planned, combining both topics and country focal points.

- Developed a simplified questionnaire to collect data on energy end use by households with countries that 

is now being used and findings discussed at workshops

- New contacts are identified and relationship with the IEA enhanced increasing likelihood of additional 

countries in the next two years.

Regional assistance
© IEA 2017

 Support between training events 

 Developing tools for data collection at country level

 Improving knowledge of energy statistics

 Improving practical knowledge – data validation etc

 Data Governance and data sharing

 Demand side data

 Linking data to policy

 International consistency

Key areas for future work
Energy Data Progress against Challenges and 

the potential of Digitalisation
© IEA 2017

The need for energy statistics

 Energy underpins all economic activity (output and transport)

 To ensure adequate security and understand risk to supply – be able to understand all flows and 

ability to model the future

 Energy systems are transforming

 Better data helps improve decision making

 Required to identify cost effective steps for each country’s Clean Energy Transitions

 Provides clear understanding for investors and business

 Understanding energy use allows for efficiency, greater output at lower cost

 Design, monitor and evaluate policies
© IEA 2017

Addressing current challenges to energy data

 Challenges exist, including

 Cooperation from business in supplying timely data

 Confidentiality concerns that prevents data being provided

 Gaps in the data e.g.: stocks (levels and change), trade breakdowns, non-energy use

 Incomplete coverage (e.g. of all forms or power generation) 

 Liberalization of energy markets – one company to many

 Accuracy – e.g. the need for good CV data

 These problems can be solved and many are taking action – need to share practice and work with 

international guidance and international organisations
© IEA 2017

The new challenges to energy data

 Increasing desire for detailed data on energy end use to make informed policy choices

 Energy use data creates understanding of the service needed – the why energy is used, not just the what

 Need detailed energy data and related activity data

 G20 Energy End Use initiative co-led by France and the IEA

 Learning from each other, sharing tools and new ideas, developing new approaches

 Off grid generation

 bring energy to millions of homes, but real issues for measurement

 Irena leading work to utilize customs and other data to make informed estimates of generation based on mass on PV 

cells imported

 Wider outreach also highlighted many other forms of small scale local renewables not covered in official data

 Increasing development of producers being consumer for electricity – how to measure self use
© IEA 2017

World total final consumption by sector

Industry 

38%

Transport 

23%

Residential 

24%

Commerce 

and public 

services

8%

Agriculture/

forestry 

3%

Nonspecified 

(other) 

4%

Industry 

37%

Transport 

29%

Residential 

22%

Commerce 

and public 

services

8%

Agriculture/

forestry 

2%

Nonspecified 

(other) 

2%

1971 2015

4 244 Mtoe 9 384 Mtoe

Source: IEA, World Energy Balances, 2017
© IEA 2017

Maximizing new opportunities

 Transparency 

 Transparency of data builds confidence

 Increases engagement and informs debate

 Data informs all markets: data vacuums cause uncertainty and increase volatility

 JODI

 Open data

 Increasing drive to open data and make available to all – provides business opportunities and supports academic 

work

 Computer power and development in treating confidentiality are making it easier to release larger data sets 

 Digitalization

 A potential game changer for energy data:

 Already increasing use of GIS

 Data matching (e.g. UK’s NEED matching data to understand energy efficiency)

 Significant potential with the appropriate regulatory/legal framework
© IEA 2017

Electricity: power flows in a non-digitalized system
© IEA 2017

Electricity statistics in a non-digitalized system
© IEA 2017

Electricity: power flows in a digitalized system
© IEA 2017

Electricity statistics: in a digitalized system?
© IEA 2017

Digitalization – one example to improve data?

 1st phase?

 Business have more timely data (e.g. digital twin of power stations, direct monitoring of networks)

 Official data still dependent on collection from business (monthly) and consumers (annual with large lags)

 2nd phase?

 Automatic flows of data from digital systems in homes and businesses to statistical offices?

 Could it happen?

 Official stats do not need time critical data, aggregation over time periods (e.g. 1 hour) or time lags (weeks) still a real 

benefit

 Detail data could have a longer lag of months

 Need legislation to maximize the benefit of using data whilst ensuring data protection
© IEA 2017

Support from international organisations

 International Recommendations for Energy Statistics

 Developed by over 20 organisations and agreed by the UNSC as a guide to producing comprehensive and comparable energy 

statistics

 Full product classification with definitions

 Energy balance as the framework to understand energy

 http://unstats.un.org/unsd/energy/ires/IRES_Whitecover.pdf

 Regional cooperation between Agencies, especially in Latin America

 Common data collection

 Increasing efforts to harmonize data collection under common terms, long experience with IEA, EU, APEC – very positive work 

now underway with OLADE and IEA

 Training and capacity building

 Manuals in multiple languages

 On-line – the IEA web videos (being translated)

 Coordination of training 
© IEA 2017

The IEA has organised 10 training events and co-organised 20 others since 2015 

IEA Statistics Training Events in collaboration with other organisations

Organised by IEA only

In collaboration with IRENA

In collaboration with AFREC

In collaboration with APEC

In collaboration with UNSD

In collaboration with ASEAN

In collaboration with JODI

In collaboration with G20

In collaboration with EU4Energy

6
© IEA 2017

Conclusions

 Enhancing the comprehensiveness of energy data will lead to better decisions.

 Countries can further support this by:

 being open with their data and their methodology;

 work with the IEA and others to build up tools that all countries can use; and 

 support the efforts of all international organisations in capacity building.

 Resources and their allocation are essential to establish and maintain a national energy statistics system: 

 how energy data are prioritised vs other statistical areas; 

 how synergies can be exploited between energy and other policy areas; 

 what priority is given to the monitoring component within the policy formulation process; and 

 how resources are allocated across areas within the energy domain. 

 Data governance and proactive regulation are needed to address any current weaknesses in data collection systems and 

to ensure that the maximum benefit can be achieved from digitalisation whilst maintaining confidentiality

 Data need to be comparable, timely, regular, complete and made available in a transparent way
© IEA 2017