COMMISSION STAFF WORKING DOCUMENT EVALUATION of Regulation (EU) No 376/2014 on the reporting, analysis and follow-up of occurrences in civil aviation
Date: 2021-02-17
Institutions: Directorate-General for Mobility and Transport, European Commission
Subjects: accident prevention, air safety, civil aviation, data collection, disaster resilience, disclosure of information, exchange of information, transport accident

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 17.2.2021  SWD(2021) 30 final  COMMISSION STAFF WORKING DOCUMENT  EVALUATION  of  Regulation (EU) No 376/2014  on the reporting, analysis and follow-up of occurrences in civil aviation {SWD(2021) 31 final}  
 
 
Table of contents 

1. 

INTRODUCTION ................................................................................................................................ 4 

1.1.  PURPOSE AND SCOPE .................................................................................................................... 4 

1.1.1. PURPOSE OF THE EVALUATION ................................................................................................ 4 

1.1.2. SCOPE OF THE EVALUATION ..................................................................................................... 5 

2. 

BACKGROUND TO THE INTERVENTION ..................................................................................... 5 

2.1.  TRANSITION  TOWARDS  A  PROACTIVE  AND  EVIDENCE-BASED  AVIATION 

SAFETY MANAGEMENT SYSTEM............................................................................................... 5 

2.2.  DESCRIPTION  OF  THE  INTERVENTION  LOGIC  AND  OBJECTIVES  OF  THE 

INITIATIVE ......................................................................................................................................... 6 

2.3.  BASELINE AND POINTS OF COMPARISON ................................................................................ 10 

3. 

IMPLEMENTATION / STATE OF PLAY ........................................................................................ 10 

4.  METHOD............................................................................................................................................ 13 

4.1.  METHODOLOGY AND SOURCES OF INFORMATION .............................................................. 13 

4.2.  LIMITATIONS AND ROBUSTNESS OF FINDINGS ..................................................................... 16 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS .......................................... 16 

5.1.  EFFECTIVENESS .............................................................................................................................. 16 

Establishment of mandatory and voluntary reporting systems .............................................. 17 

Occurrence collection rates ................................................................................................... 18 

Occurrence reporting flow ..................................................................................................... 19 

Quality and completeness of data .......................................................................................... 20 

Protection of information source and use of information for safety purposes ...................... 22 

“Just Culture” ........................................................................................................................ 23 

5.2.  RELEVANCE ..................................................................................................................................... 29 

5.3.  EFFICIENCY...................................................................................................................................... 32 

5.4.  COHERENCE ..................................................................................................................................... 33 

5.5.  EU ADDED VALUE .......................................................................................................................... 36 

6. 

CONCLUSIONS ................................................................................................................................. 37 

ANNEX 1: PROCEDURAL INFORMATION ............................................................................................ 40 

ANNEX 2: STAKEHOLDER CONSULTATION ....................................................................................... 43 

ANNEX 3: METHODS AND ANALYTICAL MODELS .......................................................................... 57 

1 

 
 
Glossary 

Term or acronym 

Meaning or definition 

ADREP 

Accident/Incident Data Reporting Programme 

AMC 

ANS 

AOC 

 ATC 

ATM  

CAA 

CAG 

CAT 

DG JUST 

DG MOVE 

EASA 

EC 

ECCAIRS 

Acceptable Means of Compliance 

Air Navigation Services 

Air Operator Certificate  

Air Traffic Control 

Air Traffic Management 

Civil Aviation Authority 

Collaborative Analysis Group 

Commercial Air Transport  

Directorate-General for Justice and Consumers  

Directorate-General for Mobility and Transport 

European Union Aviation Safety Agency 

European Commission 

European  Coordination  Centre  for  Accident  and  Incident 
Reporting Systems 

ECR 

European Central Repository  

ECR-ECCAIRS  

European Central Repository for Occurrences 

ECR-SRIS 

ENCASIA 

EPAS 

ERCS 

EU 

FDM 

FTE 

GDPR 

European Central Repository for Safety Recommendations 

European  Network  of  Civil  Aviation  Safety  Investigation 
Authorities 

European Plan for Aviation Safety 

European Risk Classification Scheme 

European Union 

Flight Data Monitoring  

Full-time Equivalent 

General Data Protection Regulation 

2 

 
 
GM 

ICAO 

IFR 

iSTARS 

ITS 

JRC 

MOR 

NoA 

RMT 

SCM 

SIA 

SMS 

SMTeB 

SSP 

UAS 

VOR 

Guidance Material 

International Civil Aviation Organisation 

Instrument Flight Rules 

Integrated Safety Trend Analysis and Reporting System 

Interrupted Time Series 

Joint Research Centre 

Mandatory Occurrence Report 

Network of Analysts 

Rulemaking task 

Standard Cost Model 

Safety Investigation Authority 

Safety Management System 

Safety Management Technical Body 

State Safety Programme 

Unmanned Aircraft System 

Voluntary Occurrence Report 

3 

 
1. 

INTRODUCTION 

1.1. 

Purpose and scope 

1.1.1.  Purpose of the evaluation 

This Commission Staff Working Document presents the ex-post evaluation of Regulation 
(EU)  No  376/2014  on  the  reporting,  analysis  and  follow-up  on  occurrences  in  civil 
aviation1  carried  out  to  support  the  preparation  of  the  Commission’s  Report  on  the 
implementation  of  this  Regulation  pursuant  to  Article  24  of  Regulation  (EU)  No 
376/20142.  It  assesses  whether  the  main  objectives  of  this  Regulation  have  been 
achieved, in particular whether the Regulation improved aviation safety by ensuring that 
relevant  safety  information  relating  to  civil  aviation  is  reported,  collected,  stored, 
protected, exchanged, disseminated and analysed. 

This Staff Working Document evaluates whether the Regulation contributed to reducing 
the number of aircraft accidents and related fatalities. It compares the initial expectations 
with the current situation. The evaluation highlights the improvements made in the area 
of  occurrence  reporting  and  analysis  thanks  to  the  Regulation  and  identifies  some 
deficiencies  and  possible  gaps.  The  assessment  takes  account  of  the  following  criteria: 
relevance,  effectiveness,  efficiency  and  the  added  value  of  the  EU  intervention.  It  also 
reviews the internal coherence of the Regulation as well as its coherence with other EU 
aviation safety regulations.3 

The main objective of this evaluation is to determine whether the Regulation has had the 
expected effect on safety and whether it has contributed to the reduction of accidents and 
related fatalities in aviation throughout the EU.  

1   The Implementing Regulation (EU) 2015/1018 also forms part of this assessment because it is directly 
related.  Commission  Implementing  Regulation  (EU)  2015/1018  of  29  June  2015  laying  down  a  list 
classifying occurrences in civil aviation to be mandatorily reported according to Regulation (EU) No 
376/2014  of 
the  European  Parliament  and  of  the  Council,  OJ  L  163,  30.6.2015,  p.  1, 
http://data.europa.eu/eli/reg_impl/2015/1018/oj.  

2    Regulation (EU) No 376/2014 of the European Parliament and of the Council of 3 April 2014 on the 
reporting,  analysis  and  follow-up  of  occurrences  in  civil  aviation,  amending  Regulation  (EU)  No 
996/2010 of the European Parliament and of  the  Council and repealing  Directive 2003/42/EC of the 
European Parliament and of the Council and Commission Regulations (EC) No 1321/2007 and (EC) 
No 1330/2007, OJ L 122, 24.4.2014, p. 18, http://data.europa.eu/eli/reg/2014/376/oj.  

3     In particular Regulation (EU) No 2018/1139 of the European Parliament and of the Council of 4 July      
2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety 
Agency,  and  amending  Regulations  (EC)  No  2111/2005,  (EC)  No  1008/2008,  (EU)  No  996/2010, 
(EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the 
Council,  and  repealing  Regulations  (EC)  No  552/2004  and  (EC)  No  216/2008  of  the  European 
Parliament  and  of  the  Council  and  Council  Regulation  (EEC)  No  3922/91,  OJ  L  212/1,  and  
Regulation (EU) No 376/2014 of the European Parliament and of the Council of 3 April 2014 on the   
reporting,  analysis  and  follow-up  of  occurrences  in  civil  aviation,  amending  Regulation  (EU)  No 
996/2010 of the European Parliament and of  the  Council and repealing  Directive 2003/42/EC of the 
European Parliament and of the Council and Commission Regulations (EC) No 1321/2007 and (EC) 
No 1330/2007, OJ L 122/18. 

4 

 
 
                                                           
1.1.2.  Scope of the evaluation  

The evaluation  covers the period from  2015 to  2019. All  EU Member States4 and their 
stakeholders  were  consulted  through  the  public  consultation,  online  survey  and  cost-
benefit questionnaire. Except for one Member State, Member States were represented by 
at  least  one  stakeholder.  Non-EU  countries,  including  Norway  and  Switzerland,  were 
also consulted, as well as other third countries not directly affected by the Regulation (i.e. 
Turkey, Canada, United States of America, Japan, Montenegro, and Albania). 

2.  BACKGROUND TO THE INTERVENTION 

2.1.  Transition  towards  a  proactive  and  evidence-based  aviation  safety 

management system 

Regulation  (EU)  No  376/2014  is  applicable  as  from  15  November  2015.  It  aims  to 
prevent aircraft accidents and related fatalities through feedback and lessons learnt from 
occurrences.  An  occurrence  is  a  safety-related  event  which  endangers  or  which,  if  not 
corrected or addressed, could endanger an aircraft, its occupants or any other person. The 
Regulation  seeks  to  ensure  that  relevant  safety  information  relating  to  civil  aviation  is 
reported,  collected,  stored,  protected,  exchanged,  disseminated  and  analysed.  The 
industry  and  the  competent  authorities  have  to  take  safety  action  in  a  timely  manner 
based on the analysis of the information collected. To ensure the continued availability of 
safety  information,  the  Regulation  provides  rules  on  confidentiality  of  the  reporting 
source  and  on  the  appropriate  use  of  information.  To  this  end,  the  Regulation  grants, 
under certain conditions, the protection to the persons reporting the occurrences and other 
persons  mentioned  in  occurrence  reports  and  ensures  that  the  information  is  used  for 
safety improvement purposes. Finally, the Regulation ensures that aviation safety hazards 
are identified and appropriate risk-mitigation actions are taken at industry, national and 
Union levels. 

This  proactive  and  evidence-based  approach  is  implemented  by  the  relevant  aviation 
safety authorities of Member States, by organisations as part of their safety management 
system and by the European Union Aviation Safety Agency (EASA). 

Regulation  (EU)  No  376/2014  is  complemented  by  an  implementing  regulation 
establishing a list of occurrences in civil aviation to be mandatorily reported.5  

Before the establishment of the occurrence reporting system in Europe, the advancements 
in  aviation  safety  relied  mainly  on  lessons  learned  from  the  investigation  of  aircraft 
accidents.  However,  whilst  the  ability  to  learn  lessons  from  accident  investigation  is  a 
crucial  element  in  ensuring  aviation  safety  and  preventing  the  accidents  from 
reoccurring,  such  reactive  approach  had  shown 
in  bringing  safety 
improvements, notably in the context of continuous air traffic growth (which in 2014 was 

limits 

its 

Including the United Kingdom that was EU Member State during this period.   

4  
5   Commission Implementing Regulation (EU) 2015/1018.   

5 

 
                                                           
expected to almost double by 2030). On the other hand, the experience had shown that 
accidents  are  often  preceded  by  safety-related  incidents  and  deficiencies  revealing  the 
existence of safety hazards. Safety information is therefore an important resource for the 
detection  of  potential  safety  hazards.  Consequently,  the  reactive  system  of  accident 
investigation should be complemented by a proactive system, which uses other types of 
safety information to achieve effective improvements in aviation safety.  

At  the  European  level,  the  transition  towards  a  more  proactive  aviation  safety 
management system  started with  the adoption of Directive 2003/42/EC6 which required 
each Member State to set up a mandatory occurrence reporting system (MORS). Member 
States had to collect, store, protect and disseminate between themselves information on 
certain  civil  aviation  incidents.  The  Directive,  however,  did  not  include  provisions 
related to the analysis of the data collected, for the benefit of aviation safety. Considering 
other reporting requirements existing in parallel, both at the European and international 
level, the adoption of Directive 2003/42/EC also  led the Member States to adopt rather 
divergent approaches. These diverging approaches resulted, for instance, in the existence 
of multiple occurrence databases at the European level.  

the  quality, 

the  analysis  and 

The  clarification  of  reporting  obligations  had  to  be  addressed  in  order  to  improve  the 
accessibility, 
the  exchange  of  occurrence-related 
information. Additionally, the  Impact Assessment for the proposal of a new Regulation 
on occurrence reporting7 envisaged that the future legislative should address the growing 
importance  of  a  “Just  Culture”8,  encouraging  reporting  of  essential  safety-related 
information. 

2.2.  Description of the intervention logic and objectives of the initiative 

The  2012  Impact  Assessment  for  the  proposal  for  the  occurrence  reporting  regulation 
identified  the  following  four  main  problem  drivers  related  to  the  previous  legal 
framework established by Directive 2003/42/EC: 

  The collection of occurrences was not optimal 

At the time, the Impact Assessment concluded that the optimal collection of the safety-
related information had not been achieved due to the following reasons: 

o  The  scope  of  reporting  differed  between  the  Member  States,  which  resulted  in 

discrepancies in Member States reporting levels 

6   Directive 2003/42/EC of the European Parliament and of the Council of 13 June 2003 on occurrence 

7  

8  

reporting in civil aviation, OJ L 167, 4.7.2003, p. 23, http://data.europa.eu/eli/dir/2003/42/oj . 
‘Commission Staff Working Paper: Impact Assessment Accompanying document to the Proposal for a 
Regulation  of  the  European  Parliament  and  the  Council  on  occurrence  reporting  in  civil  aviation’, 
SWD/2012/0441 final, https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:52012SC0441 . 
‘just  culture’  means  a  culture  in  which  front-line  operators  or  other  persons  are  not  punished  for 
actions,  omissions  or  decisions  taken  by  them  that  are  commensurate  with  their  experience  and 
training,  but  in  which  gross  negligence,  wilful  violations  and  destructive  acts  are  not  tolerated. 
Regulation (EU) No 376/2014, Art 2(12).  

6 

 
                                                           
o  Individual  reporters  were  afraid  of  repercussions  for  submitting  occurrence 

reports  

o  No  obligation  to  establish  voluntary  reporting  schemes  was  laid  down  in  the 

Directive 

o  There  was  an  insufficient  clarity  in  reporting  obligations  and  in  the  flow  of 

information  

Consequently,  the  specific  objective  of  the  Regulation  is  to  ensure  that  all  occurrences 
which  endanger  or  would  endanger  aviation  safety  are  collected  and  are  providing  a 
complete and clear picture of safety risks in the EU and its Member States. To achieve 
this objective, following operational objectives were set:  

o  Establishment of the mandatory reporting systems 

o  Establishment of the voluntary reporting systems  

o  Reaching  a  higher  occurrence  collection  rate  through  harmonisation  and 

classification of reporting obligations  

o  Clarifying and development of the legislative requirements on “Just Culture” 

o  Clarifying the flow of information  

To achieve these operational objectives, the Regulation sets out the specific provisions on 
the  mandatory  and  voluntary  reporting,  establishes  a  regime  on  the  protection  of 
information source and sets the related requirements for the Member States to ensure the 
protection of reporters and it provides for the specific rules facilitating the exchange of 
safety information between the stakeholders.  

  Suboptimal data integration, low quality and incomplete data  

High  quality  safety-related  data  is  crucial  for  identifying  and  assessing  safety  hazards 
posed  by  occurrences.  The  Impact  Assessment  identified  low  data  quality  and 
incompleteness of data provided in the occurrence reports as two main causes of the data 
integration,  needed  for  comprehensive  understanding  of  potential  safety  deficiencies, 
being  suboptimal.  To  remedy  the  situation,  the  specific  objective  was  set  for  the 
Regulation  to  make sure that occurrence reports  stored in  the national  databases and in 
the European Central Repository (ECR) are complete and contain high quality data.  

Establishment  and  management  of  the  ECR,  standardised  data  entry  processes, 
establishment of mandatory data fields and establishment of quality checking processes 
are the operational objectives leading to the achievement of this specific objective.  

  Legal  and  organisational  obstacles  for  ensuring  adequate  access  to 

information contained in the ECR 

The regime established by the Directive on the access to the data contained in the ECR 
led to a situation where due to the requirement to de-identify the data contained therein, 

7 

 
Member State authorities and EASA did not have an access to crucial occurrence data. At 
the same time, Member States would also refrain from uploading the data into the ECR 
due to  fears that such data could  be used by other stakeholders for other  purposes than 
safety (e.g. benchmarking9).  

Consequently, the specific objective of the Regulation is to ensure that all safety-critical 
information  stored in the ECR is  accessed adequately  by the competent  authorities and 
that  they  are  used  strictly  for  safety-improvement  purposes.  Hence,  at  the  operational 
level, the Regulation provides for the granting of access rights to the ECR information to 
appropriate  safety  authorities  and  establishes  rules  on  confidentiality  when  data  is 
transferred to the interested parties and safeguards against misuse of data.  

The Regulation therefore contains the detailed rules on the access to the ECR, granting 
the full online access to the ECR to any entity entrusted with regulation of civil aviation 
safety or any  safety investigation authority  within the Union  and  further  specifying the 
conditions  under  which  the  interested  parties  listed  in  Annex  II  to  the  Regulation  can 
request  the  information  contained  in  the  ECR.  The  Regulation  lays  down  the 
requirements  on  confidentially  and  protection  of  data  gathered  via  the  occurrence 
reporting systems. 

  Lack  of  occurrence  analysis  at  Member  State  level  and  at  European  level 

and of appropriate corrective and preventive actions  

Even where the data was collected under the previous legal framework, the Directive did 
not specify how the information collected should be used in order to contribute to safety 
improvements. Therefore, the Regulation aims at ensuring that reported occurrences are 
effectively analysed, that safety hazards are identified and addressed, where relevant, and 
that the safety effectiveness of actions taken is monitored.  

To achieve this objective, following operational objectives were set:  

o  To analyse occurrence data and to identify actual or potential safety hazards 

o  To adopt preventive or corrective actions where appropriate 

o  To oversee the effectiveness of the actions taken  

o  To create a common EU risk classification scheme for classifying occurrences  

To this end, the Regulation contains the provisions on the occurrence analysis and follow 
up  at  both  Member  States  and  EU  levels.  The  Regulation  also  provides  for  the 
development  of  the  European  risk  classification  scheme  and  confers  upon  the 
Commission the necessary delegated and implementing powers to define the scheme in 
the EU legislation and to set the arrangements for its implementation.  

9   Comparing  their  respective  safety  performances  and  using  such  data  for  commercial  and  marketing 

purposes.  

8 

 
                                                           
Figure 1: 

 Intervention logic of Regulation (EU) No 376/2014 

9 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2.3.  Baseline and points of comparison  

Baseline  analysis  was  prepared  as  part  of  the  Support  Study  to  this  evaluation.10  The 
baseline  analysis  was  built  on  the  qualitative  analysis  conducted  in  the  2012  Impact 
Assessment by using quantitative methods to forecast how the selected indicators would 
have evolved without the Regulation (EU) No 376/2014 in place. The methodology used 
is the Interrupted Time Series (ITS) methodology, relying on comparison of time-series 
data  before  and  after  intervention,  built  upon  the  basic  assumption  that  the  outcome 
variable would not be altered if there were no intervention. The baseline scenario for the 
selected  indicators  was  forecasted  through  time-series,  capturing  the  historic  data 
evolution of these indicators themselves and other independent variables with time, and 
projecting  this  same  behaviour  into  the  future  as  per  the  actual  independent  variables 
evolution. In the issue at hand, time-series were used to forecast selected indicators from 
2015 up to 2019, based on their historic evolution (from 2009 to 2015) and the evolution 
of other variables. After that, the Baseline results were compared with the ones actually 
recorded to derive conclusions on the intervention impact.  

To model the counterfactual scenario under which the Regulation had not been adopted, 
the study analysed a number of parameters and their evolution over a period of time. The 
parameters included the recorded growth in the level of reporting, the levels of reporting 
per  type  of  operation,  stakeholder  and  Member  State,  as  well  as  percentage  of 
completeness  of  ECR  fields,  the  amount  of  civil  aviation  accidents  and  incidents,  and 
other  variables  such  as  traffic  movements  or  a  number  of  safety  recommendations  and 
safety actions.    

In the counterfactual scenario, due to the increasing air traffic in the EU, the occurrence 
reporting  would  have  continuously  increased  at  a  modest  pace  for  some  stakeholders. 
However, since the entry into force of the Regulation higher number of occurrences than 
forecasted  is  reported.11  Such  improvement  in  occurrence  reporting  is  evidenced  across 
all aviation domains which are subject to reporting requirements.  

10   ‘Ex-post  evaluation  of  regulation  (EU)  no  376/2014  on  the  reporting,  analysis  and  follow-up  of 
occurrences in civil aviation – Technical Annexes’ (‘Evaluation Support Study - Technical Annexes’), 
https://op.europa.eu/en/publication-detail/-/publication/8d56315a-3b5f-11eb-b27b-01aa75ed71a1, 
Annex 9. 

11   This  conclusion  was  reached  by  comparing  the  actual  number  of  the  occurrences  reported  since  the 
Regulation’s  entry  into  force  with  the  forecasts  of  the  reporting  in  the  counterfactual  scenario.  The 
number  of  occurrences  in  the  Evaluation  Baseline  has  been  forecasted  using  relevant  background 
variables  such  as  traffic  movements.  By  doing  this,  time-varying  confounders  (seasonable  trends  or 
specific  events),  which  may  interfere  with  the  forecast  results  have  been  taken  into  account  in  the 
models. Ibid.  

10 

 
 
                                                           
Without  the  Regulations’  provisions  on  data  quality,  the  level  of  completeness  of  the 
occurrence reports in  the ECR would be considerably lower. Thanks to  the Regulation, 
occurrence reports are approximately 10% more complete.12  

Finally,  while  impossible  to  conclude  with  certainty,  the  increased  reporting  may  have 
contributed to the overall improvement of aviation safety at the European level especially 
by  improving  safety  awareness,  identification  of  safety  risks  and  increased  safety 
promotion. This might have had an indirect effect on the reduction of the total number of 
accidents compared to the counterfactual scenario.  

On  a  more  general  level,  without  the  Regulation,  the  voluntary  reporting  systems  in 
organisation  established  in  the  EU  Member  States  would  have  not  existed.  The 
Regulation allowed also  for the protection of identity of the reporters of occurrences in 
civil aviation.  

3. 

IMPLEMENTATION / STATE OF PLAY 

The  provisions  on  the  establishment  of  mandatory  and  voluntary  reporting,  set  out  in 
Articles 4 and 5 of the Regulation, were fully implemented by both the Member States 
and  EASA.  In  practice,  this  means  that  each  Member  State  has  established  both 
mandatory  and  voluntary  reporting  mechanisms  at  the  level  of  its  aviation  authority. 
Similarly,  EASA  has  a  functioning  mandatory  reporting  system  in  line  with  the 
requirement set out in Article 4(4) and voluntary reports can be submitted directly to the 
Agency and the Member States via the European Reporting Portal.13  

Some  gaps  have  been  identified  in  the  implementation  of  these  requirements  by  the 
organisations. While some organisation were already subject to the mandatory reporting 
system requirement prior to  the Regulation’s entry into force, some other organisations 
had  to  develop  it.  This  resulted  in  some  organisations  reporting  less  than  others. 
Similarly, in the field of voluntary reporting, some organisations indicated that they did 
not  have  a  voluntary  reporting  system  in  place.  Finally,  some  organisations  do  not 
distinguish  between  voluntary  and  mandatory  reporting  systems,  having  only  one 
reporting system in place.  

The Regulation lays  down the requirements for  the collection, storage and exchange of 
information  in  Articles  6  and  9.  Those  requirements  were  implemented  in  full  by  the 
organisations  as  well  as  by  the  Member  States  and  EASA.  Member  States  established 
mechanisms for the exchange of information with other Member States and with EASA, 
and for the transfer of such information into the ECR.  

12   A  “completeness  percentage”  indicator  has  been  defined  to  quantify  to  which  extent  each  of  the 
mandatory reporting fields are used. This indicator mainly captures the average percentage of reports 
in which each specific field has been used / informed (i.e. is not null). An analysis has been conducted 
on  the  attribute  usage  trends  within  the  ECR  with  respect  to  the  common  mandatory  data  fields  as 
listed in Regulation 376/2014, over the ten-year period 2009-2019. Ibid, p. 205.  

13   https://www.aviationreporting.eu/AviationReporting/. 

11 

 
                                                           
The  Regulation  also  lays  down  specific  requirements  for  the  management  of  the  ECR, 
standardised  data  entry  processes  and  quality  checking  processes.  Majority  of  those 
provisions  were  fully  implemented  by  the  organisations,  Member  States  and  EASA. 
Nonetheless, some organisations do not have in place a formal quality checking process 
as  required  by  Article  7(3).  In  addition,  the  development  of  a  common  European  risk 
classification  scheme  was  implemented  partially.  While  the  Commission  developed  the 
scheme by the deadline of 15 May 2017, as envisaged in Article 7(5), it still has to define 
the  scheme  in  accordance  with  Article  7(6).  The  definition  of  ERCS  together  with  the 
arrangements for its implementation is ongoing.  

Articles 10, 11 and 12 stipulate the arrangements for access to the information stored in 
the  ECR.  Those  provisions  are  fully  implemented  by  the  Commission,  which  is 
responsible  for  the  processing  data  stored  in  the  ECR  and  by  the  Member  States  (so 
called  “points  of  contact”)  who  assist  the  Commission  in  relation  to  the  requests  for 
information stored in the ECR.  

The  implementation  of  the  Regulation’s  provisions  related  to  “just  culture”  and  the 
protection and appropriate use of the source of information was closely analysed. There 
are few shortcomings in the implementation of those provisions. While it was found that 
all  Member  States  and  majority  of  the  organisations  put  in  place  processes  to  ensure 
anonymization  of  occurrence  reports,  the  issue  with  traceability  –  i.e.  the  possibility  to 
infer the persons concerned in an occurrence report based on the circumstances described 
in the report – remain, especially in the smaller organisations with less employees.  

Article  16(11)  lays  down  an  obligation  for  the  organisations  to  adopt  internal  “just 
culture”  rules. The available information is  not  conclusive on whether this requirement 
has been fully implemented by all organisations.  

The most significant shortcoming in the implementation of the Regulation was found in 
relation to the obligation laid down in Article 16(12) for the Member States to designate 
a body responsible for the implementation of the “just culture” principles. Based on the 
reviews  of  the  reports  on  the  functioning  of  the  “just  culture  bodies”  submitted  by  the 
Member  States  in  accordance  with  Article  16(13)  and  additional  information  received 
from  citizens,  the  Commission  identified  11  Member  States  that  failed  to  designate  a 
“just  culture  body”  responsible  for  the  implementation  of  paragraphs  6,  9  and  11  of 
Article 16. As a consequence, the Commission  opened infringement  cases and sent  out 
10  Letters  of  Formal  Notice  and  one  Reasoned  Opinion  to  the  non-compliant  Member 
States.14 The Commission also sent out a number of EU Pilots to the Member States that 
stated in their reports to have designated a “just culture body” but where there were still 
doubts as to whether such body has the competencies to discharge the function envisaged 
in the Regulation.  

14   The Reasoned Opinion was closed as the State provided the evidenced that the Just Culture Body was 
designated  with  the  necessary  competences  to  execute  its  function  under  Article  16(12)  of  the 
Regulation. The rest of the cases are still pending.   

12 

 
                                                           
Similarly, there has been a limited progress in the implementation of Article 15(4) of the 
Regulation,  which  aims  to  ensure  the  cooperation  of  the  competent  authorities 
responsible  for  the  collection,  evaluation,  processing,  analysis  and  storage  of  the 
occurrence  reports  and  the  judicial  authorities  in  the  Member  States  via  advanced 
arrangements.  It  appears  that  only  limited  number  of  Member  States  put  such 
arrangements in place and some of those arrangements do not comprehensively cover the 
details of the cooperation.  

Finally, the Regulation also lays down in Article 13 rules on the analysis of occurrence 
reports  and  on  the  adoption  of  preventive  or  corrective  actions.  EASA  and  the 
organisations across the EU fully implemented that provision. The Member States do use 
the  analysis  of  the  occurrence  reports  to  identify  potential  or  actual  safety  hazards. 
However,  some  Member  States  upload  reports  into  the  ECR  that  are  relevant  to  other 
Member  States,  without  duly  informing  them.  Consequently,  the  concerned  Member 
States are not aware of those occurrence reports and cannot make use of the information 
contained therein.  

When  it  comes  to  the  adoption  of  preventive  or  corrective  actions,  those  are  fully 
implemented  by  EASA  and  Member  States  who  take  preventive  actions  identified 
through the analysis of the occurrences. At the organisations side, the situation depends 
on  the  maturity  of  the  organisation’s  occurrence  reporting  and  analysis  systems. 
Organisations  with  more  mature  systems  are  capable  of  identifying  and  acting  upon 
potential  safety  hazards,  other  organisations,  with  less  mature  systems,  identify  less 
safety hazards and take less preventive or corrective actions.  

4.  METHOD  

4.1.  Methodology and sources of information 

A  support  study  was  carried  out  by  an  external  contractor  to  provide  input  to  this 
evaluation.15 In accordance with the Terms of Reference, the study was structured along 
eight consecutive Tasks, as depicted in the figure below: 

15   ‘Ex-post  evaluation  of  regulation  (EU)  no  376/2014  on  the  reporting,  analysis  and  follow-up  of 
occurrences  in  civil  aviation  –  Final  Report’  (‘Evaluation  Support  Study  –  Final  Report’), 
https://op.europa.eu/en/publication-detail/-/publication/dabeabc8-3b60-11eb-b27b-01aa75ed71a1 .  

13 

 
                                                           
Figure 2: Support Study Methodology Structure  

The methodology for this support study was based on: 

  Desk-based  research,  including  a  range  of  legislative  documents  as  well  as 
previous studies and other relevant reports that were coded and analysed to form 
the  basis  of  the  study  and  informed  the  assessment  of  the  current  state  of 
implementation;  

  Stakeholder  consultation  via  an  online  survey,  cost-benefit  questionnaires,  in-
depth interviews, Member State experts and industry stakeholders meetings, and 
an  open  public  consultation  to  supplement  secondary  evidence  with  views  and 
opinions from the wide range of stakeholders; and  
In-depth  analysis  of  the  primary  and  secondary  data  collected,  including  an 
assessment  of  the  state  of  implementation,  an  analysis  of  the  baseline,  a  cost 
assessment,  and  five  analytical  case  studies  presenting  results  for  a  range  of 
thematic areas within civil aviation.  

 

All  findings  and  results  were  triangulated  and  synthesised  to  provide  evidence-based 
answers to the evaluation questions. 

The Study was structured according to an evaluation matrix, presented in Annex 3. This 
matrix operationalised the study’s evaluation questions, setting them out against the sub-

14 

 
 
questions,  indicators  and  judgement  criteria  that  will  be  used  to  answer  them.  It  was 
developed  on  the  basis  of  an  in-depth  understanding  of  the  Regulation’s  intervention 
logic, depicting what it aims to achieve and how, as presented in Section 2.2 above.  

The  field  research  consisted  of  a  public  consultation  as  well  as  a  range  of  targeted 
consultation activities. Seventeen stakeholder groups16 were consulted in the context of 
the study by different means. 

  An  online  survey  was  launched  on  13  November  2019.  The  survey  remained 
open for three weeks and formally closed for responses on 9 December 2019. It 
was  targeted  to  a  range  of  different  stakeholder  groups,  including  national 
competent  authorities,  safety  investigation  authorities,  EASA,  but  also  all 
different industry sectors, and associations representing staff in the aviation field. 
The  online  survey  included  questions  on  the  effectiveness,  relevance,  and  EU 
added value of Regulation (EU) No 376/2014, as well as few questions pertaining 
to  the  degree  to  which  the  Regulation  has  been  implemented  in  the  Member 
States.  A  total  of  67  respondents  fully  completed  the  online  survey,  and  32 
respondents partially completed it (i.e. started completing the survey but did not 
answer  all  questions).  Considering  the  efforts  made  to  reach  as  broad  a 
respondent  group  as  possible,  including  through  reminders/follow-ups,  this  is 
considered a decent number of respondents.  

  Targeted interviews aimed to complement the questionnaires by collecting more 
details  on  why  stakeholders  hold  certain  opinions,  as  well  as  gather  additional 
evidence in relation to evaluation questions for which qualitative data was judged 
to be an important source. A total of 40 interviews were planned out of which 39 
were  effectively  carried  out  with  a  sample  of  key  stakeholders  per  main 
stakeholder type.  
In  terms  of  expert  meetings,  three  workshops  were  organised,  two  with 
competent  authorities  (September  and  October  2019  in  Cologne,  Germany)  and 
one  with  the  aviation  industry  representatives  (November  2019  in  Brussels, 
Belgium), with the aim to gather data to  feed into the answers to the evaluation 
questions.  

 

  Finally, the Commission launched an open public consultation17 on 7 November 
2019  to  gather  the  opinions  on  the  Regulation  from  the  broader  stakeholder 
community,  specifically  relating  to  the  relevance,  effectiveness,  coherence, 
efficiency  and  EU  added  value  of  the  Regulation,  and  on  specific  questions 

16   European  Union  staff  (European  Commission,  JRC,  EASA);  national  competent  authorities,  safety 
investigation authorities, aviation industry, commercial aviation operators, aviation providers, aviation 
personnel,  general  aviation,  UAS,  just  culture  bodies,  judicial  authorities,  research  centres  and 
passengers.  

17   The open public consultation was organised by the European Commission and consisted of an online 
questionnaire published on a dedicated Commission webpage. It was accessible to the general public 
and  any  person  interested  in  the  topic  could  have  submitted  a  contribution.  The  online  survey, 
mentioned above, was organised by the consultant and distributed among the pre-selected stakeholder 
groups (including national competent authorities,  safety  investigation authorities, EASA, but also all 
different industry sectors, and associations representing staff in the aviation field).  

15 

 
                                                           
related  to  the  concept  of  “just  culture”.  The  public  consultation  was  open  for 
responses from the general public until 30 January 2020. A total of 198 responses 
were received18, 

Furthermore, in order to complement the study with factual information, desk research on 
several  related  issues  was  carried  out.  The  aim  of  the  desk  research  was  to  collect, 
organise  and  analyse  relevant  information  from  relevant  secondary  sources.  This 
included statistical  data,  legislative documents and their related implementing acts,  and 
relevant  reports  and  studies,  with  the  intention  of  collecting  both  quantitative  and 
qualitative  evidence  to  complement  the  primary  evidence  collected  through  field 
research. 

  EU Legislation 
  ECCAIRS cases 
  EASA documentation 
  Existing studies and literature. 
  Five  thematic  case  studies,  namely  Air  traffic  control,  commercial  air  transport 
(CAT) operations of large aeroplanes and related design and production, general 
aviation  manufacturer of Non-Complex aeroplanes, Unmanned Aircraft Systems 
(UAS) with a Maximum Take-Off Mass below 150 kg and ground handling. 

The  proposed  sample  of  case  studies  was  well  balanced  in  order  to  cover  instances  of 
occurrence  reporting,  analysis  of  the  reported  occurrences,  its  quality  as  well  as  the 
follow-up  actions  taken  by  the  relevant  stakeholders.  The  case  studies  illustrate  the 
instances of both voluntary and mandatory occurrence reporting and were selected to also 
provide  insights  into  exchange  and  dissemination  of  information,  confidentiality,  and 
protection  of  the  information  source.  The  objective  of  the  case  studies  was  to  take 
findings  from  stakeholder consultations (including surveys and interviews) and provide 
an insight on the impacts of the Regulation in specific operational scenarios. 

The  data  collected  was  used  to  respond  to  the  evaluation  questions.  All  the  analytical 
findings constitute the basis for the assessment on how the Regulation has scored on the 
evaluation criteria. Each of these criteria was addressed through evaluation questions, as 
presented in the evaluation question matrix presented in Annex 3.  

In the Commission, an Inter-Service Steering Group provided advice and monitored the 
progress  of  the  exercise.  Being  composed  of  members  from  different  Commission 
services and having the necessary mix of knowledge and experience, the Steering Group 
brought  together  a  range  of  different  perspectives  and  provided  the  necessary  input,  in 
particular where the evaluation touched different policy areas.  

18   Company/business  organisation,  EU  citizen,  non-EU  citizen,  Other,  Business  association,  Trade 
Union, Academic/research institution, NGO, Environmental organisation, Consumer organisation. 

16 

 
 
 
                                                           
4.2.  Limitations and robustness of findings 

Even though the evaluation was designed to ensure the robustness of the data supporting 
the  findings  presented  in  this  Staff  Working  Document,  some  limitations  to  the 
robustness of certain data were identified while drafting the support study.  

First, the responses from the online survey came mostly from the competent authorities 
in the Member States (34 out of 99), which biased the results of the survey towards that 
stakeholder  group.  To  mitigate  this  bias,  the  results  of  the  stakeholder  groups  were 
analysed separately, where relevant.  

Secondly, there was a limited response rate to the cost-benefit questionnaires, especially 
from  the  industry  stakeholders.  This  made  the  assessment  of  costs  incurred  by  the 
industry brought by the Regulation more difficult.  

Thirdly, the contractor initially encountered some difficulties in scheduling the targeted 
interviews with the selected representatives from various stakeholder groups, in essence 
due  to  a  lack  of  availability  by  the  persons  and/or  the  entities  targeted  for  such 
interviews. The impact of such situation on the data gathered was however mitigated by 
extending the period in which the interviews were scheduled.   

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

This  section  provides  the  analysis  and  the  results  for  the  five  evaluation  criteria  of 
effectiveness, efficiency, relevance, coherence, and EU added value. 

The  findings  presented  are  based  on  the  results  from  desk  research,  as  well  as  results 
obtained through stakeholder consultations.   

5.1.  Effectiveness 

This  subsection  assesses  whether  the  Regulation  has  been  effective  in  achieving  the 
intended objectives and in particular the improvement of aviation safety in Europe. It in 
particular  assesses  how  the  specific  objectives  and  the  operational  objectives  of  the 
intervention have been implemented. 

Figure 3: Steps in the occurrence reporting, analysis and follow-up process 

17 

 
 
The  first  specific  objective  of  Regulation  (EU)  No  376/2014  is  to  ensure  that  all 
occurrences that endanger or would endanger aviation safety are collected and are 
providing a complete and clear picture of safety risks in the EU and in the Member 
States.  

Establishment of mandatory and voluntary reporting systems 

Requirements for mandatory occurrence reporting already existed prior19 to the adoption 
of  Regulation  (EU)  No  376/2014  and  they  did  not  therefore  need  to  be  re-established. 
The  Regulation  has  however  filled  gaps  by  developing  the  common  EU  standards  for 
reporting,  across  different  industry  domains  and  Member  States.  As  such,  the 
Regulation’s requirements are broader in terms of scope than of its predecessor Directive 
2003/42/EC.  

Mandatory  and  voluntary  occurrence  reporting  systems  are  indeed  in  place  at  the 
Member  State  level,  at  EASA  level  and  in  the  majority  of  concerned  organisations.  In 
addition, the European Reporting Portal was established by the Commission services.20 

The majority of Member States21 do not distinguish between the two types of reports in 
their respective databases, as they  rather prefer  encouraging organisations to  submit all 
reports to them, regardless of their status. As a consequence, it is  difficult to assess the 
number  of  voluntary  reports  submitted,  as  there  is  no  easy  way  to  distinguish  between 
MOR and VOR stored in the ECR.  

The Study concluded that there are no major concerns in terms of coverage of who has to 
report. Moreover, EASA noted that the Regulation omits ‘declared organisations’.  

A  related  problem  for  EASA  is  that  it  also  oversees  organisations  established  in  third 
countries,  which  are  subject  to  reporting  requirements  under  Regulation  2018/1139  but 
not  Regulation  376/2014,  which  raises  questions  in  terms  of  a  level  playing  field  and 
what  creates  a  burden  on  EASA  who  has  to  define  ‘equivalent’  occurrence  reporting, 
analysis and follow-up requirements for those organisations.   

19   Directive 2003/42/EC and Implementing rules to the EASA Basic Regulation 
20   The  European  Reporting  Portal  (ERP)  is  an  online  portal  through  which  aviation  professionals  can 

report on occurrences directly to their competent authority.   

21   14  out  of  39  participants  in  the  second  workshop  with  the  Member  States  authorities  indicated          
that their Member State does not distinguish between mandatory and voluntary reporting in its national 
database. 

18 

 
 
 
                                                           
Occurrence collection rates  

An  expected  outcome  from  the  effective  implementation  of  Regulation  (EU)  No 
376/2014  is  the  increased  number  of  reports  submitted  and  thus  a  greater  amount  of 
aviation safety information. One of the key objectives of Regulation (EU) No 376/2014 
is  to  increase the available data on occurrences. This  objective has been met. The  total 
number  of  occurrence  reports  stored  in  the  ECR  has  increased  significantly  since  the 
entry  into  force  of  the  Regulation.  In  2015,  there  were  just  over  200,000  occurrence 
reports in the ECR in the given year. As shown in the Figure 4 below, in 2019, the total 
number of reports transferred to the ECR in one year reached 291,458.22 Cumulatively, 
the  number  of  occurrence  reports  contained  in  the  ECR  more  than  doubled  during  this 
period: from 1,125,264 reports at the end of 2014 to 2,548,578 reports in 2019.23  

Figure  4:  Number  of  records  in  the  ECR  (source:  JRC  overview  of  ECR-ECCAIRS 
content) 

As  shown  in  the  Figure  5  below,  there  would  have  been  an  increase  in  the  amount  of 
collected  occurrences  even  in  the  counterfactual  scenario,  without  the  Regulation  in 
place.  This  would  have  been  directly  correlated  with  the  annual  traffic  increase. 
However, since entry into force of the Regulation, the collection rate has been increased 
significantly compared to the expected trends. This therefore suggests that the Regulation 
facilitated the increase in the occurrence reporting across the EU.  

22   JRC data provided by the European Commission. 
23   ECR data extract provided by EASA.  

19 

 
 
                                                           
Figure 5: Number of occurrences reports issued in the evaluation baseline at European 
level24 

However,  according  to  the  views  expressed  by  stakeholders  in  the  online  survey,  no 
conclusion can be made on whether all relevant occurrences are reported and whether the 
reported occurrences provide a complete and clear picture of safety risks across the EU.25 
This can in part be attributed to the difficulty to measure whether some occurrences pose 
an immediate, significant safety risk and hence whether they should be reported.  

in  understanding  safety 

There  is  a  divergence  in  the  views  expressed  by  the  Member  States  and  the  industry 
stakeholders.  The  authorities  in  the  Member  States  believe  that  there  has  been  a  clear 
improvement 
they 
acknowledge that it is likely that not all occurrences are reported, they have seen a clear 
improvement, not just in numbers but also in the quality of the occurrence reports. They 
believe  that  the  picture  of  safety  risks  is  adequate.  Conversely,  the  stakeholders 
representing organisations or associations stated that not all occurrences are reported and 
hence the general understanding of safety risk might not be fully adequate.  

reporting.  Although 

through 

risks 

Occurrence reporting flow  

The  reporting  flow  (i.e.  who  reports  to  whom  and  when)  is  well  understood  by  all 
stakeholders involved, although some issues have been identified. Sometimes the details 
of occurrences are uploaded to the ECR by one Member State with no notification to the 
Member  States  concerned.  However,  there  is  a  general  expectation  that  the  new 
ECCAIRS 2 software will help on this aspect. 

Furthermore, the industry stakeholders complained about the lack of transparency on the 
processing  of  occurrence  reports  after  they  have  been  submitted  to  the  competent 

24   ‘Evaluation Support Study – Technical Annexes’, Annex 9, p.237.  
25   45  out  of  71  stakeholders  believe  that  all  occurrences  which  endanger  or  could  endanger  aviation 
safety  are  collected,  and  41  out  of  71  believe  that  the  occurrence  collected  provide  a  complete  and 
clear picture of safety risks in the EU.  

20 

 
 
                                                           
authority and stored in the ECR. 23 out  of 25 industry representatives in  the workshop 
expressed  their  view  that  it  is  not  very  clear  or  not  clear  at  all  what  happens  with  the 
occurrence  reports  once  they  are  submitted  to  the  competent  authorities.  The  industry 
also expressed disappointment on the lack of a subsequent feedback from the authorities.  

Furthermore,  some  stakeholders  also  expressed  the  view  that  the  reporting  timelines 
provided in the Regulation are too tight. The 72-hour time limit set out in the Regulation 
seems in particular to be an issue for the air navigation service providers and commercial 
air transport operators. However, it should be noted that in exceptional circumstances the 
Regulation gives possibility to submit the occurrence reports after the 72-hour time limit.  

The second specific objective of Regulation (EU) No 376/2014 is to ensure that the 
occurrence reports stored in the national databases and the ECR are complete and 
contain high quality data.  

Quality and completeness of data 

Regulation  (EU)  No  376/2014  underlines  the  importance  of  high-quality  and  complete 
data. Overall, there has been a considerable improvement in the quality and completeness 
of the occurrence data contained in the ECR as well as in the national databases. This has 
been  achieved  thanks  to  the  key  requirements  mandated  in  the  Regulation  such  as  a 
standardised  data  entry  process,  minimum  information  to  be  included  and  quality 
checking processes.  

An important EU contribution is the ECR26 that stores all occurrence reports collected in 
the  EU.  Member  States  and  EASA  are  required  to  transfer  to  the  ECR  all  information 
relating  to  safety  stored  in  their  respective  databases  in  a  timely  manner.  The  national 
data is integrated in the ECR and checked in accordance with quality criteria.  

The  ECR  consists  of  two  physical  databases  –  the  European  Central  Repository for 
Occurrences (ECR-ECCAIRS)  and 
for  Safety 
Recommendations (ECR-SRIS).  Access  to  the  ECR is  subject  to  strict  rules  and  mainly 
limited to the European competent authorities27.  

the European  Central  Repository 

Throughout the period under review, the European Commission’s Joint Research Centre 
(JRC) provided the operational services related to the ECR-ECCAIRS. As identified by 
the JRC, the ECR-ECCAIRS is fed with data originating from any of the 31 contributing 
States28  as  well  as  from  EASA  on  a  scheduled  basis,  with  a  maximum  timespan  of  1 
month.29  

26   JRC, ECCAIRS portal, https://eccairsportal.jrc.ec.europa.eu/index.php?id=164.  
27   Regulation (EU) No 376/2014, Art 10(4). 
28   28 MS plus Switzerland, Norway and Iceland. 
29   Regulation  (EU)  No  376/2014,  Art  9(1),  “Occurrence  reports  shall  be  transferred  to  the  European 

Central Repository no later than 30 days after having been entered in the national database.” 

21 

 
                                                           
Following  a  change  of  strategy  and  after  an  impact  study30,  the  Commission  services 
decided  to  replace  the  current  ECCAIRS  platform  by  a  fully  re-engineered  platform 
called  ECCAIRS  2,  and  to  transfer  the  management  of  the  ECCAIRS  system  from  the 
JRC to EASA as of 2020.  

ECCAIRS 2 was released in October 2020 and it is expected to solve many of the issues 
reported by competent authorities in terms of user-friendliness, system performance and 
training. It is a web/cloud-based solution with a user-friendly interface. It will integrate 
the  local  national  repositories  (without  duplication/copying)  into  a  regional  view  and 
integrate into the Data4Safety (D4S)31 programme of the Commission/EASA, subject to 
the strict rules on protection of information contained in the ECR imposed by Regulation 
(EU)  No  376/2014  and  applicable  data  protection  rules.  ECCAIRS  2  will  allow  the 
competent  authorities  to  exploit  advanced  analytics,  insights  and  auto-improved  data 
quality.  ECCAIRS  2  is  likely  to  have  a  further  positive  impact  on  the  effectiveness  of 
Regulation 376/2014 as a whole.  

To furthermore ensure the operational objective as regards the completeness of the data 
contained  in  the  occurrence  reports,  Annex  I  to  the  Regulation  sets  out  a  list  of 
mandatory  data  fields  to  be  filled  when  uploading  an  occurrence  report.  The  list  of 
mandatory  data  fields  can  be  updated  by  means  of  delegated  acts,  thus  giving  the 
Commission  the  flexibility  to  adapt  those  data  fields,  when  appropriate  and  to  reflect 
developments that may occur in the aviation sector. There are diverging opinions about 
that list. Some stakeholders think that not all mandatory data fields are relevant for all the 
reporters or reporting organisations. Furthermore, both organisations and Member States 
noted  that  the  requirement  of  completing  the  mandatory  data  fields  has  produced  an 
additional workload for both reporters and authorities. This is not the view of EASA, as 
it considers that all eight (8) common mandatory data fields are relevant, however those 
should be completed only if applicable.  

Quality checking procedures should be implemented for checking the data quality of the 
occurrence reports thus avoiding inconsistencies and completing missing parts. Based on 
the collected evidence32, it can be concluded that quality checking processes have at least 
partially  been  implemented  by  organisations  and  competent  authorities,  and  they  are 
implemented  in  the  ECR  with  feedback  being  sent  on  the  success/failure  to  the 
submitting  body.  It  cannot  be  concluded  with  certainty  that  this  was  as  a  result  of  the 
Regulation  as  these  processes  already  existed.    In  terms  of  resources,  there  are  strong 

30   Bearing Point, ‘Impact Assessment of ECCAIRS transition from JRC to EASA’, 12 June 2017.  
31   Data4Safety  (also  known  as  D4S)  is  a  data  collection  and  analysis  programme  that  will  support  the 
goal  to  ensure  the  highest  common  level  of  safety  and  environmental  protection  for  the  European 
aviation system. It aims at collecting and gathering not only occurrence data, but also flight data (i.e. 
data  generated  by  the  aircraft  via  the  Flight  Data  Recorders),  surveillance  data  (air  traffic  data), 
weather data, and more. 

32   This is evidenced by the views expressed by the Member State authorities that collect and analyse the 
occurrence reports, as gathered for the purposes of the support study. This is further underlined by the 
outcomes  of  the  work  conducted  by  a  dedicated  subgroup  on  data  and  taxonomy  of  the  Network  of 
Analysts. See Evaluation Support Study – Final Report, p. 29-31.  

22 

 
                                                           
concerns  in  the  Member  States  about  the  resources  needed  to  do  this  task,  as  it  is 
performed mostly manually. The use of digital tools (e.g. ECCAIRS 2, D4S) would help.  

The third specific objective of Regulation (EU) No 376/2014 addresses the 
competent authorities’ adequate access to the safety-critical information contained 
in the occurrence reports, and the strict use of such information for safety-related 
purposes, only.  

Protection of information source and use of information for safety purposes  

Regulation (EU) No 376/2014 mandates for all organisations and competent authorities 
in  the  Member  States  to  take  the  necessary  measures  to  ensure  the  appropriate 
confidentiality  of  the  details  of  occurrences  obtained  by  them  (i.e.  mandatory  and 
voluntary  reports,  as  well  as  the  data  taken  from  the  ECR)  and  to  process  this  data  in 
accordance  with  applicable  European  rules  on  data  protection.33  It  requires  that 
competent authorities in the Member States as well as EASA, do not record any personal 
details in their respective databases to prevent a transfer of personal data to the ECR.  

Based  on  the  evidence  collected  for  this  evaluation34,  it  can  be  concluded  that 
organisations and all competent authorities in the Member States have internal processes 
in place to ensure that occurrence reports are anonymised before they are transferred to 
interested parties, notwithstanding the high workload this creates. Despite these efforts, 
mistakes happen and occasionally personal data is transferred due to a human error when 
inputting the occurrence data into the ECR.  

With  regard  to  the  operational  objective  that  the  information  contained  in  the  ECR  is 
used  solely  for  the  improvement  of  the  safety  situation  in  aviation  and  not  for 
apportioning blame and liability, the Regulation sets out a possibility for the competent 
aviation  authorities  to  conclude  advance  administrative  arrangements  with  their 
counterpart  judicial  authorities  setting  out  the  terms  of  using  the  occurrence  data  in 
judicial  proceedings  while  seeking  “to  ensure  the  correct  balance  between  the  need  for 
proper  administration  of  justice,  on  the  one  hand,  and  the  necessary  continued 
availability of safety information, on the other”.35 However, almost no Member State has 
been  able  to  conclude  such  advance  arrangements  with  their  respective  judicial 
authorities. In fact, only seven Member States have some type of arrangement in place.36 

33   The  Regulation  makes  reference  to  Directive  95/46/EC,  which  is  the  predecessor  of  what  is  now 
known  as  the  GDPR  (General  Data  Protection  Regulation  (EU)  2016/679).  As  such,  since  this 
Directive was repealed by the GDPR, the new GDPR rules automatically apply. 

34   ‘Evaluation Support Study – Final Report’, p.42.  
35   Regulation (EU) No 376/2014, Art 15(4).  
36   ‘Evaluation Support Study – Final Report’, p. 44.  

23 

 
 
                                                           
Hindering  factors  are  the  lack  of  guidance  as  to  the  content  or  format  of  such  an 
arrangement  or  lack  of  resources  and  the  fact  that  those  arrangements  are  only 
facultative. 

Despite  the  lack  of  advance  administrative  arrangements,  the  information  available 
suggests that so far the data contained in the ECR is used strictly for safety improvement 
purposes only. This is one of the objectives to which the Regulation has contributed the 
most. 

“Just Culture” 

Regulation  (EU)  No  376/2014  is  the  first  legal  act  to  incorporate  “just  culture” 
provisions, thus ensuring its application across all the domains of the aviation at industry 
as  well  as  at  the  competent  authority  level.  The  Regulation  requires  organisations  to 
establish specific rules and procedures for the application of the Just Culture principles to 
the reporting of occurrences, thus  ensuring that  the reporters will not  be subject  to  any 
form  of  punishment  or  prejudice  for  the  occurrences  reported,  unless  their  conduct 
amounts to gross negligence or wilful misconduct. This should motivate the reporting at 
every level of the organisation.  

From  the  evidence  collected,  there  are  indications  that  such  rules  are  being  formally 
established  by  the  concerned  organisations,  however  the  evidence  is  inconclusive  as 
regards the actual compliance with those rules by the organisations. Some representatives 
interviewed during the stakeholder workshop of November 2019 spoke of few instances, 
which appeared to be in direct contradiction to the “just culture” principles.  

At  the  national  level,  the  Member  States  are  also  required  to  put  in  place  measures 
ensuring the protection of reporters and ensuring that the reporters will not be prosecuted 
in  relation  to  the  information  shared  in  the  occurrence  report,  unless  their  conduct 
amounts  to  gross  negligence  or  wilful  misconduct.  To  ensure  such  protection,  the 
Member  States  are  required  to  designate  a  body  specifically  responsible  for  the 
implementation of these “just culture” principles, as contained in paragraphs 6, 9 and 11 
of Article 16. However, only a limited number of Member States have complied with this 
requirement,  which  led  the  Commission  to  opening  a  number  of  infringement 
proceedings. It should also be added that many industry stakeholders are unaware of the 
existence of a “just culture” body in their State, with only 4 out of 31 participants in the 
industry  workshop  answering  positively  when  asked  about  their  awareness  of  the 
existence of such body.  

At  the  industry  level,  the  European  social  partners,  members  of  the  sectoral  social 
dialogue committee for civil aviation (CANSO, ETF, ATCEUC), have jointly agreed on 
the  “ATM  Just  culture  tool  box”.37  It  targets  staff  and  managers  within  organisations 
providing air traffic management / air navigation services and sets guiding principles for 
implementing Just Culture and fostering a healthy and open reporting culture.  

37   https://ec.europa.eu/social/BlobServlet?mode=dsw&docId=11897&langId=en.  

24 

 
                                                           
Ensuring that the all reported occurrences are effectively analysed, safety hazards 
are  identified  and  addressed,  where  relevant,  and  the  safety  effectiveness  of  such 
actions is monitored, is the fourth specific objective of the Regulation.  

Analysis of occurrences and identification of actual or potential hazards 

The  Regulation  set  outs  the  obligation  for  the  organisations,  Member  States  and  for 
EASA  to  develop  processes  to  analyse  the  information  contained  in  the  occurrence 
reports  in  order  to  identify  safety  hazards  associated  with  those  occurrences. 
Furthermore, the competent authorities and EASA are mandated to exchange and analyse 
safety information contained in the occurrence reports at the EU level.  

Whereas  the  Study  concluded  that  organisations,  generally  speaking,  do  have  such 
processes in place, it should be noted that majority of the organisations already had such 
processes  in  place  before  the  Regulation,  as  part  of  the  requirements  related  to  safety 
management system (SMS) contained in the implementing  rules in  the  area of aviation 
safety.38 Nonetheless, there are organisations (e.g. ground handling), which subsequent to 
the Regulation’s entry into force had to introduce the processes to analyse occurrences. 
However, the study noted the problems experienced by such organisations in setting up 
such processes as the provisions of the Regulation in this regard are rather general.  

Additionally, many industry stakeholders noted that the timeline of 30 days for providing 
the  preliminary  analysis  of  the  occurrences  to  the  competent  authority  is  too  short. 
Moreover, the Regulation also sets a deadline of three months from the initial report for 
the  organisations  to  submit  their  final  analysis.  The  majority  of  industry  stakeholders 
indicated  that  this  deadline  is  too  tight.  The  Regulation  allows  however  to  extend  that 
deadline if this is justified.39  

Similar  to  organisations,  the  competent  authorities  in  the  Member  States  all  have 
processes  in  place  to  analyse  occurrences.  However,  Member  States  differ  in  their 
opinions  as  regards  the  usefulness  of  such  analyses  and  their  content.  Some  authorities 
believe that the Regulation is not sufficiently clear on this aspect.  

Consequently, there are  differences  in  the degree to  which analyses are carried out  and 
the depth of these analyses. This in turn can have an impact on the identification of safety 
hazards  and  on  appropriate  follow  up  actions  being  taken  in  a  timely  manner.  Other 
reasons  for  those  differences  derive  also  from  a  lack  of  resources  (in  terms  of  time, 
budget and skills) to carry out the analytical work.  

38   Regulation (EU) 1139/2018 and its implementing rules. 
39   Regulation (EU) No 376/2014, Art 13(4).  

25 

 
 
                                                           
The  authorities  noted  also  the  “imbalance”  between  the  requirements  for  such  analysis 
and  the  degree  of  details  included  in  the  requirements  for  reporting,  collecting, 
processing and storing occurrences. This imbalance leads to a high workload associated 
with  the  collecting  and  processing  of  occurrence  reports  and  leaves  little  time  for  the 
analysis and follow-up.  

At  the  EU  level,  there  are  two  streams  of  analysis:  EASA’s  analysis  of  occurrences 
reported  directly  to  the  Agency  by  organisations  for  which  EASA  acts  as  a  competent 
authority  (based  on  occurrences  stored  in  EASA’s  own  database)  and  a  collaborative 
analysis  done  together  with  Member  States.  Those  analyses  result  in  the  publication  of 
the  EASA  Annual  Safety  Reviews  and  in  the  development  of  Safety  Risk  Portfolios 
(SRPs),  which  are  used  in  the  publication  of  the  European  Plan  for  Aviation  Safety 
(EPAS).  

The  EASA  Annual  Safety  Review  is  produced  by  the  Safety  Intelligence  and 
Performance Department of EASA  and is  based  on the data contained both  in  EASA’s 
own  Occurrence  Database  as  well  as  data  contained  in  the  ECR.  As  part  of  its  risk 
management process, the Agency also defines and implements appropriate actions, which 
are subject to a continuous monitoring.  

EASA  works  with  a  range  of  collaborative  partners  through  two  main  platforms:  the 
Network  of  Analysts  (NoA)40  and  the  Collaborative  Analysis  Groups  (CAGs)41.  The 
NoA  is  set  up  by  Regulation  376/2014  and  pulls  together  data  from  different  Member 
States to get a more general picture of safety hazards faced by the EU as a whole. The 
CAGs are expert groups set up by EASA that are responsible for analysing the safety of 
European  aviation  for  specific  aviation  domains.  They  review  available  safety 
information,  arrange  in  depth  safety  analyses  and  identify  emerging  issues.  They  also 
monitor  the  safety  performance  of  their  domain  and  provide  feedback  on  the 
effectiveness  of  actions  taken.  Both  systems  are  complementary  because  they  provide 
analysis  at  different  levels  and  they  contribute  to  the  Safety  Risk  Portfolios  and  to  the 
safety actions which may be adopted in the EPAS.  

Adoption and monitoring of preventive and corrective actions  

40   The NoA comprises EASA, the European Commission, EASA Members States and an observer. It was 
initially formed as a voluntary network to support the analysis of safety data for the European Aviation 
Safety  Plan,  which  is  the  predecessor  to  the  EPAS,  in  2011.  The  implementation  of  EU  Regulation 
376/2014  has  formalised  the  role  of  the  NoA  and  underscored  the  importance  of  safety  analysis  in 
supporting the EPAS and improvement of aviation safety in Europe. The NoA currently meets twice a 
year, once in Spring and once in Autumn.  

41   The CAGs consist of groups of industry stakeholders of EASA’s regulatory partners. Each CAG meets 
up to three times per year. There are currently six different CAGs: (1) aerodrome and ground handling; 
(2) air traffic management; (3) balloons; (4) commercial air transport aeroplanes; (5) general aviation; 
(6)  human  factors.  Source:  https://www.easa.europa.eu/easa-and-you/safety-management/safety-risk-
management 

26 

 
                                                           
At  the  EU  level,  the  analyses  are  carried  out  by  EASA  in  collaboration  with  the 
authorities in the NoA and CAGs. The outcomes of those analyses drive the development 
of safety actions for the EPAS and allow the setting of safety actions and priorities, not 
only  for  EASA,  but  also  for  all  the  different  actors  in  the  aviation  supply  chain.  The 
Regulation  has  therefore  contributed  largely  to  the  development  of  analyses  at  the  EU 
level.  

It  is  difficult  to  conclude  with  certainty  to  what  extent  organisations  implemented 
preventive  or  corrective  actions  as  a  result  of  the  Regulation  or  whether  those  actions 
derive  from  the  pre-existing  SMS  requirements.    Although  the  analyses  are  not 
necessarily  resulting  in  a  safety  action,  they  always  contribute  to  the  greater 
understanding  of  the  potential  safety  risks.  In  addition,  the  analyses  also  contribute  to 
further staff training, safety awareness and other safety-related purposes.42  

The study identified that the high workload and unclear definition of the term “analysis” 
has resulted in a limited number of corrective and preventive actions by Member States.  

Regarding  the  monitoring  of  the  actions,  the  study  confirmed  that  Member  States  and 
organisations put in place such monitoring. It appears, however, that those processes are 
largely based on the pre-existing SMS requirements and Regulation (EU) No 376/2014, 
did  not  change  them.  This  is  because  the  latter  does  not  concretely  suggest  how  the 
monitoring of actions should be conducted, limiting itself to establishing the obligation to 
conduct such monitoring.  

There  is  a  certain  lack  of  clarity  on  the  side  of  national  authorities  responsible  for  the 
oversight of the Regulation. While the Regulation allows for the Member States to assign 
one  or  more  competent  authorities  responsible  for  establishing  a  mechanism  to 
“independently  collect,  evaluate,  process,  analyse  and  store  details  of  occurrences”. 
However, the Regulation does not provide an explicit mandate to a competent authority 
nominated  under  the  Regulation  to  oversee  the  implementation  of  the  Regulation. 
Furthermore,  in  some  Member  States  the  competent  authority  under  Article  6(3)  is  a 
Safety  Investigation  Authorities  (SIA),  or  another  independent  body.  Those  bodies 
however  are  often  not  empowered  to  conduct  oversight  at  the  level  comparable  to  the 
CAA  (who  have  an  experience  in  the  certification,  oversight  and  enforcement  in  the 
context of national regulations and Regulation (EU) 2018/1139).  

42   At the workshop with industry stakeholders, less than half of the participants indicated that they had 
implemented preventive or corrective actions as a result of analysis carried out. However, almost all 
participants  indicated  that  they  use  the  analyses  for  other  purposes,  such  as  training  of  personnel  or 
issuing safety recommendations. ‘Evaluation Support Study – Final Report’, p. 53.  

27 

 
 
                                                           
At  the  EU  level,  part  of  the  EPAS  concerns  the  monitoring  of  the  implementation  of 
EPAS  actions.  The  implementation  of  safety  actions  identified  through  the  domain-
specific  analysis  carried  out  through  the  EASA  CAGs  is  directly  monitored  by  the 
relevant group. For the actions where Member States are not competent and where EASA 
acts  as  a  competent  authority,  feedback  on  the  implementation  is  regularly  provided 
during advisory body meetings. 

It can be concluded that organisations, Member States and EASA put in place processes 
for the analysis and follow-up of occurrences. Whereas the Regulation introduced some 
changes  in  the  areas  where  such  requirement  did  not  exist,  it  nevertheless  appears  that 
some stakeholders would welcome improvements in terms of cataloguing and monitoring 
the  corrective  and  preventive  actions  adopted  as  the  result  of  the  occurrence  reports’ 
analyses.43  

Reduction in the number of aircraft accidents and fatalities 

The general objective of the Regulation is to contribute to the reduction in the number of 
aircraft accidents and fatalities through a proactive approach resulting from the specific 
objectives detailed in the preceding sections.  

It is difficult to measure the degree to which the Regulation has contributed to an actual 
decrease  in  the  number  of  accidents  and  fatalities  in  civil  aviation,  both  quantitatively 
and qualitatively.  

Figure  5  shows  the  evolution  of  key  statistics  from  commercial  aircraft  accidents  and 
serious incidents of the European operators recorded by EASA. Between 2010 and 2018 
the number of accidents has decreased in average terms. The decrease is more evidenced 
in  the  rate  of  accidents  by  1  million  flights  (high  severity).  The  rate  of  accidents  has 
continuously decreased since 2014, while the rate of serious incidents  stabilised after a 
peak  in  2016.  That  peak  was  not  an  actual  increase  in  incidents  but  rather  a  result  of 
technical adjustments and improvements in the classification.  

43   84% of the participants in the workshop with the national authorities indicated that their Member State 
has a  system in place to  monitor or follow-up safety actions. However, the stakeholders interviewed 
complained  about  the  lack  of  detail  in  the  Regulation  on  how  to  monitor  the  follow-up  actions  and 
what can be done to address inappropriate safety actions. ‘Evaluation  Support Study – Final Report’, 
p. 53-56.  

28 

 
 
                                                           
Figure 6: Annual safety-related occurrences in total numbers and rate per 1M 
movements (Source: EASA Annual Safety Reviews (2019, 2018, 2017) 

After the entry into force of the Regulation, there has been a decrease in the accident rate, 
from  3.5  accidents  per  1  million  flights  and  1.3  accidents  per  1  million  flights  post-
intervention (see Figure 6).  

Figure 7: Number of civil aviation accidents in the evaluation baseline (Source: results 
from the analysis of the baseline) 

The  support  study  concluded  that  the  Regulation  contributed  to  the  increase  of  the 
number  of  occurrence  reports  stored  in  the  ECR,  which  in  turn  might  have  had 
contributed  to  the  improvement  of  safety  awareness  and  identification  of  safety  risks. 
Consequently,  Regulation  (EU)  No  376/2014  has  likely  helped  to  reduce  the  total 
number  of  accidents.  Although  it  is  almost  impossible  to  model  the  exact  number  of 

29 

 
 
 
accidents and other safety related occurrences in the absence of the Regulation, it can be 
concluded that, to some extent, the Regulation has contributed to that reduction.   

Conclusions on Effectiveness 

The Regulation  has made a significant  step towards achieving its specific objectives.  It 
has  led  to  improvements  in  the  reporting  across  the  stakeholder  groups  and  across  the 
EU.  There  has  also  been  an  improvement  in  the  completeness  and  quality  of  the 
occurrence  reports.  However,  the  associated  high  workload  related  to  filling  in  the 
mandatory data fields and the lack of feedback from the authorities affect to some extent 
the motivation to report.  

There has been an increase in the number of reports stored in the ECR, suggesting that 
the Regulation has been effective in meeting its objectives in the area of data integration. 
The  complexity  of  the  use  of  the  previous  version  of  the  ECCAIRS  software  and  the 
noted gaps in  the flow of information  between the Member States should be solved by 
the new and improved ECCAIRS 2 software.  

As  regards  the  Regulation’s  objectives  related  to  the  protection  of  information  sources 
and  the  “just  culture”,  the  Regulation  provides  an  adequate  level  of  protection  of  the 
reporters. However, the Regulation has not produced yet the desired progress in terms of 
advance  administrative  arrangements  being  concluded  with  the  judicial  authorities  and 
the “just culture” bodies in the Member States.  

Likewise,  whilst  the  Regulation  caters  for  improved  occurrence  analysis  and  follow-up 
requirements, nevertheless no improvements in terms of analysis and follow up seem to 
be  noted.  The  evaluation  has  not  revealed  any  issues  with  the  regulatory  provisions 
themselves, but points to some difficulties with the availability of adequate resources for 
such analysis.    

In  addition,  the implementation  of Regulation  (EU) No 376/2014  and its Implementing 
Regulation  2015/1108  has  caused  increased  workload  for  both  organisations  and 
competent authorities (the increased requirements for quality, content and timing of the 
occurrence reporting). 

Finally, no firm conclusions can be drawn as to whether the Regulation has succeeded in 
achieving  its  overall  objective  to  contribute  to  the  reduction  of  aircraft  accidents  and 
related fatalities. Nonetheless, the data presented above show the decrease in the amount 
of  accidents  when  compared  to  the  increasing  level  of  traffic.  Due  to  the  improved 
collection and analysis of the safety data from the occurrence reports and the elaboration 
and implementation of relevant corrective and preventive measures, it can be concluded 
that the Regulation  has  to some extent contributed to  this  decrease in  the accident rate, 
along with other safety measures put in place.  

30 

 
5.2.  Relevance 

The  core  objective  of  the  Regulation  is  to  ensure  that  the  information  relevant  to 
occurrences  which  endanger  or  could  endanger  aviation  safety  is  reported,  collected, 
processed, stored, and analysed with a view to identifying safety hazards that can then be 
adequately  addressed  through  preventive  or  corrective  actions.  This  proactive  and 
predictive approach is increasingly important as there are fewer fatal accidents in the EU 
than  globally44  making  the  traditional  reactive  approach  of  lessons  learnt  from  such 
accidents unsuitable for the continuous maintenance of high levels of aviation safety.  

The aviation sector has experienced important developments since the entry into force of 
the  Regulation.  In  view  of  the  emerging  technology  advancements,  as  well  as 
advancements in digitalisation, the study assessed whether Regulation is relevant and fit-
for-purpose,  notably  given  the  upcoming  rapid  proliferation  of  the  Unmanned  Aircraft 
Systems (UAS) and the growing importance of cybersecurity.  

Unmanned Aircraft Systems  

Thanks  to  the  recent  amendment  to  the  Regulation  brought  by  Regulation  (EU) 
2018/1139,  reporting  of  occurrences  and  other  safety-related  information  involving 
unmanned  aircraft  falls  within  its  scope45.  However,  in  case  of  occurrences  and  other 
safety-related  information  involving  unmanned  aircraft    for  which  a  certificate  or 
declaration  is  not  required  pursuant  to  the  Regulation  (EU)  2018/1139,  the  Regulation 
(EU)  No  376/2014  only  applies  as  long  as  the  occurrence  results  in  a  fatal  or  serious 
injury to a person or it involves aircraft other than unmanned aircraft.  

As  said,  the  occurrences  involving  the  UAS  fall  within  the  scope  of  the  Regulation  as 
amended by  Regulation  (EU) 2018/1139. However,  given the novel  nature of the UAS 
operations and the recent entry into force of the specific regulatory framework regulating 
such  operations  there  might  be  a  need  to  adjust  the  list  of  mandatory  data  fields  in  the 
Annex  to  the  Regulation  to  reflect  the  specificities  of  UAS  operations.  An  issue  to  be 
closely monitored once reporting of occurrences with UAS aircraft advances.  

The  Regulation  however  offers  relatively  simple  solutions  in  the  possibilities  to  update 
both  the  list  of  occurrences  to  be  mandatorily  reported  and  the  Annex  containing  the 
mandatory data fields in order to make the Regulation more fit for this type of operations. 
It follows that the Regulation continues to be relevant in the area of UAS operations.  

Cybersecurity  

Cybersecurity is  an important  new challenge in  the aviation  safety sector. The growing 
risks emanate from the use of new technologies and new devices connected to the aircraft 
systems  and  for  which  there  are  currently  no  guarantees  that  there  won’t  be  any 

44   EASA,  ‘Annual Safety Review 2018’, p. 16, 

https://www.easa.europa.eu/sites/default/files/dfu/218639_EASA_ASR_MAIN_REPORT_2018.pdf  

45   Regulation (EU) 2018/1139, Art 136. 

31 

 
                                                           
interferences46. It is expected that in the context of further smart and sustainable mobility 
developments, new technologies will continue to develop.  

Though the cyber aspect is not directly addressed in Regulation (EU) No 376/2014, nor 
in  its  Implementing  Regulation  2015/1018,  it  is  possible  to  address  such  aspects  under 
the mandatory occurrence reporting categories of ‘meteorological conditions or security-
related occurrences’47, ‘failure of ATM system security’48 or ‘aerodrome security related 
occurrences’49. 

Cybersecurity should figure as one of the Key Risk Areas in the common European Risk 
Classification  Scheme  (ERCS),  currently  under  development.  It  is  expected  that, 
similarly  to  UAS,  a  possible  review  of  the  mandatory  data  fields  in  light  of  potential 
cybersecurity threats will be needed when more experience is gathered. 

Relevance to other recent developments 

Some EU legislation adopted after the entry into force of Regulation (EU) No 376/2014 
has introduced a number of provisions that might impact Regulation (EU) No 376/2014. 
This  concerns  Regulation  (EU)  2018/1139,  the  General  Data  Protection  Regulation 
(GDPR)50, Regulation (EU) 2018/1725 on the protection of natural persons with regard to 
the processing of personal data by the Union institutions, bodies, offices and agencies51 
and  the  Directive  (EU)  2019/1937  on  the  protection  of  persons  who  report  breaches  of 
Union  law  (the  “Whistleblowing  Directive”)52.  In  particular,  the  Whistleblowing 
Directive  contains  a  number  of  provisions  detailing  the  manner  in  which  reporting 
channels should be set-up, and how breaches of the Regulation should be followed-up as 
well  as  measures  of  protection  of  reporting  persons.  Nevertheless,  none  of  the  above 
instruments affect the continuing relevance of Regulation (EU) No 376/2014.  

Furthermore,  no  concerns  have  been  identified  in  relation  to  the  environmental 
developments that might impact the continued relevance of the Regulation.  

Market and social developments have also been considered. The air transport sector has 
grown by 26% between 2014 and 2018 in terms of the number of passengers transported 

46   https://www.easa.europa.eu/newsroom-and-events/news/air-issue-14-cybersecurity-aviation-editorial. 
47   Regulation (EU) 376/2014, Art 4(1)(a)(viii).  
48   Implementing Regulation (EU) 2015/1018, Annex III, 2(7).  
49   Implementing Regulation (EU) 2015/1018, Annex IV, 1.3 (2), Annex IV, 2.1(9). 
50   Regulation  (EU)  2016/679  of  the  European  Parliament  and  of  the  Council  of  27  April  2016  on  the 
protection of natural persons with regard to the processing of personal data and on the free movement 
of  such  data,  and  repealing  Directive  95/46/EC  (General  Data  Protection  Regulation),  https://eur-
lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679&from=EN.  

51   Regulation (EU) 2018/1725 of the European Parliament and of the Council of 23 October 2018 on the 
protection of natural persons with regard to the processing of personal data by the Union institutions, 
bodies, offices and agencies and on the free movement of such data, and repealing Regulation (EC) No 
45/2001 and Decision No 1247/2002/EC, OJ L 295, 21.11.2018, p. 39, 
http://data.europa.eu/eli/reg/2018/1725/oj.  

52   Directive (EU) 2019/1937 of the European parliament and of the Council of 23 October 2019 on the 

protection of persons who report breaches of Union law. 

32 

 
                                                           
and its growth is expected to continue with a commensurate rise in competitiveness and 
commercial  pressures  to  reduce  costs.  This  could  have  an  impact  on  the  employment 
patterns (e.g. such as hiring aircrews via intermediaries or as self-employed or so-called 
"pay-to-fly"). Under such context people might feel less safe to report or sharing safety 
experiences  being  afraid  that  such  reporting  might  put  their  jobs  at  risk. Therefore,  the 
concept of “Just Culture” introduced by the Regulation is even more relevant. 

Conclusions on relevance 

Even  in  the  light  of  the  recent  developments  in  the  aviation  sector,  involving  the  rapid 
increase  in  the  UAS  operations  and  the  increasing  threats  related  to  cybersecurity,  the 
Regulation  continues  to  be  relevant  and  able  to  capture  those  trends.  Where  some 
adjustments to the Regulation’s provisions could be desirable, e.g. the updates to the list 
of mandatorily reported occurrences to better reflect the UAS occurrences or mandatory 
data  fields  to  gather  the  UAS-specific  data,  the  Regulation  offers  enough  flexibility  to 
introduce those changes in a timely and efficient manner.  

The Regulation also continues to be relevant in a broader socio-economic context. With 
its rules on the “just culture”, it offers an adequate level of protection to the reporters to 
foster reporting even in the increasingly more competitive environment.  

5.3.  Efficiency 

The  global  average  annual  financial  impact  of  Regulation  (EU)  No  376/2014  for  the 
European aviation industry as a whole has been estimated to be around €15.3 million in 
2018.53  The  apparent  increase  in  costs  due  to  Regulation  (EU)  No  376/2014  is  mainly 
caused  by  a  sustained  increase  in  the  workload  associated  with  its  implementation, 
mostly the increasing number of occurrence reports collected.  

Over  2014–2020  period,  the  administrative  burdens  borne  in  the  implementation  of 
Regulation  vary  significantly  across  Member  States,  industry  key  players  and  years, 
ranging from €7.32 million in 2015 to more than €15 million in 2018.  

The total annual cost of Regulation (EU) No 376/2014 for national competent authorities 
is estimated to be €7.7 million. There is, however, significant variation between Member 
States and years, ranging from estimated annual costs of €50,000 to more than €400,000. 
Over  time,  the  total  costs  incurred  by  Member  States  due  to  Regulation  (EU)  No 
376/2014  have  increased  from  about  €2  million  in  2014  to  almost  €8  million  in  2019, 
indicating  a significant  increase resulting  from  the need to  process  the large volume of 
reports.  As  a  result,  the  average  increase  in  workload  is  estimated  at  3  full-time 
employees per Member State.  

53  Methodology for the cost assessment is further described in Annex 3 to this CSWD.  

33 

 
                                                           
The annual costs incurred for the exchange of information, analysis of occurrences and 
dissemination  are  estimated  to  represent  the  largest  share  of  total  costs  for  competent 
authorities  in  the  Member  States  (53%),  closely  followed  by  the  collection  and 
processing of occurrences (47%). 

EASA  has  experienced  an  annual  cost  increase  of  €700,000  as  a  result  of  the 
implementation  of  the  Regulation,  merely  in  collecting  and  processing  of  occurrences 
(63%),  analysis  of  occurrences,  exchange  of  information  and  dissemination  (16%).  In 
addition,  infrastructure  costs  relative  to  ECCAIRS  updates  to  accommodate  occurrence 
reporting developments have been estimated as €100,000 per year.54 

On  the  benefits  side,  the  Regulation  has  introduced  a  benefit  of  approximately  €66.48 
million over the 2015-2019 period. The model employed, taking account of all monetised 
costs and benefits for the 2014-2019, results in a cost-benefit ratio of 1.14, indicating a 
positive return on investment.  

Conclusions on efficiency  

Calculations  made  in  the  context  of  the  Support  Study  show  that  the  benefits  derived 
from  the  Regulation  outweigh  the  implementation  costs  incurred  by  the  various 
stakeholders.  It  follows  that  the  Regulation  has  been  efficient  in  bringing  considerably 
more benefits than costs.  

5.4.  Coherence 

The evaluation  did  not  identify  any  contradiction, overlap, duplication or inconsistency 
relative to the regulatory provisions set out in Regulation (EU) No 376/2014. The various 
components of the Regulation complement each other to achieve its objectives.  

Similarly, no major issues were identified in terms of coherence with its Implementing 
Regulation 2015/1018. 

In terms of external coherence, the Regulation has been assessed against the following 
EU Regulations: 

-  Regulation (EU) 996/201055 

No  major  inconsistencies  or  a  lack  of  coherence  has  been  identified.  However,  safety 
investigation authorities  raised concerns that due to  the lack of clarity between the two 
reporting  systems  serious  incidents  and  accidents  would  be  reported  only  to  the 
competent authority under Regulation (EU) No 376/2014. Recital 2 to Regulation (EU) 
No  376/2014  explains  that  this  Regulation  should  not  interfere  with  the  process  of 

54   ‘Evaluation Support Study – Technical Annexes’, Annex 10, p. 288-289.  
55   Regulation (EU) No 996/2010 of the European Parliament and of the Council of 20 October 2010 on 
the  investigation  and  prevention  of  accidents  and  incidents  in  civil  aviation  and  repealing  Directive 
94/56/EC, OJ L 295, 12.11.2010, p. 35, http://data.europa.eu/eli/reg/2010/996/oj.  

34 

 
                                                           
accident and incident investigations managed by national safety investigation authorities 
as  defined  in  Regulation  (EU)  No  996/2010.  In  the  event  of  an  accident  or  a  serious 
incident, notification of the occurrence is also subject to Regulation (EU) No 996/2010. It 
seems therefore, that there is a need for better coordination between the relevant national 
authorities,  so  that  the  serious  incidents  and  accidents  are  not  reported  only  to  the 
competent authority under Regulation (EU) No 376/2014. 

-  Regulation (EU) 2018/113956  

Regulation  (EU)  2018/1139  and  its  implementing  rules  establish  obligations  on  certain 
organisations  to  establish  occurrence  reporting  systems  in  the  context  of  their  safety 
management systems. A safety management system (SMS) aims to mitigate safety risks 
in  a  proactive  way  before  they  result  in  aviation  accidents  and  incidents.  For  this,  it  is 
important to identify those risks and control them in a systematic and regulated way.  

When Regulation (EU) No 376/2014 was proposed, as reflected in its fourth recital,57  a 
need  to  avoid  duplication  of  reporting  efforts  was  highlighted.  The  Impact  Assessment 
for Regulation (EU) No 376/2014 identified however a need for a specific regulation that 
would cover all occurrence reporting requirements,  while at  the same time establishing 
firm rules protecting the reporters and fostering the ‘just culture. At the time Regulation 
(EU)  No  376/2014  was  adopted  it  covered  areas  for  which  EASA  had  no  parallel 
competence. That situation has changed over time with the extension of EASA’s areas of 
activities, although it has not yet produced any parallel coverage. 

The  large  majority  stakeholders  considered  that  both  Regulations  are  consistent  and 
coherent, and no specific problems were highlighted.   

EASA highlighted some practical difficulties. Although there are no legal constraints, it 
is  often  very  difficult  to  maintain  and  update  a  coherent  and  effective  set  of  safety 
management  and  oversight  systems  when  (1)  the  personal  and  material  scope  of  those 
legal  acts  vary,  and  (2)  the  related  occurrence  reporting  provisions  are  spread  across 

56   Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common 
rules  in  the  field  of  civil  aviation  and  establishing  a  European  Union  Aviation  Safety  Agency,  and 
amending  Regulations  (EC)  No  2111/2005,  (EC)  No  1008/2008,  (EU)  No  996/2010,  (EU)  No 
376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, 
and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and 
of  the  Council  and  Council  Regulation  (EEC)  No  3922/91,  OJ  L  212,  22.8.2018,  p.  1, 
http://data.europa.eu/eli/reg/2018/1139/oj.  

57   In its recital (4), it is stated that “Existing legislative acts of the Union, in particular Regulation (EC) 
No 216/2008 of the European Parliament and of the Council and its implementing regulations, impose 
obligations  on  certain  organisations  to  establish  occurrence  reporting  systems  in  the  context  of  their 
safety  management  systems.  Compliance  with  Regulation  (EC)  No  216/2008  and  its  implementing 
regulations  should  not  exempt  [own  emphasis]  organisations  from  compliance  with  this  Regulation. 
Likewise,  compliance  with  this  Regulation  should  not  exempt  organisations  from  compliance  with 
Regulation (EC) No 216/2008 and its implementing regulations. However, this should not give rise to 
two parallel reporting systems, and Regulation (EC) No 216/2008, its implementing regulations, and 
this Regulation should be seen as complementary”. 

35 

 
                                                           
various  legal  acts,  thus  rendering  their  understanding  difficult  and  hence  possibly 
hampering effective safety management. 

Although  the  majority  of  stakeholders  did  not  raise  any  issue  in  terms  of  coherence, 
consistency  or  duplication  resulting  from  the  co-existence  of  occurrence  reporting 
provisions in those acts, a compilation of the existing Regulations into one act could be 
contemplated. It has to be born in mind however that certain fundamental aspects dealt 
with by Regulation (EU) No 376/2014 do not fall under EASA’s remit.  

Whereas  the  coexistence  of  various  regulatory  provisions  dealing  with  occurrence 
reporting in different EU legislative acts has not been identified as a major hurdle or an 
impediment in the evaluation, a consolidation of those existing regulatory provisions by 
creating a one-stop-for all system which still guarantees protection of reports and a Just 
Culture provisions could be considered.  

-  Other relevant EU legislation 

As  regards  coherence  of  Regulation  (EU)  No  376/2014  with  other  legislation,  the 
assessment  included,  primarily,  a  range  of  legislative  acts  in  the  field  of  aviation 
security58, where no inconsistencies, duplications or concerns were identified.  

As regards employment and social matters, no relevant EU legislation has been identified 
that would require coherency assessment.  

As  regards  protection  of  personal  data,  the  coherence  review  concluded  that  there  are 
potential  inconsistencies  between  the  General  Data  Protection  Regulation  and  the  data 
protection  rules  specific  to  EU  institutions  and  agencies  contained  in  Regulation  (EU) 
2018/1725  on  the  one  hand  and  Regulation  (EU)  No  376/2014  on  the  other  hand.  The 
relevant  competent  authorities, organisations, EASA and the Commission while storing 
data  on  the  occurrences  must  comply  with  the  respective  requirements  laid  down  in 
Regulations  (EU)  2016/679  and  (EU)  2018/1725.  This  means,  that  if  a  report  includes 

58   This included:  

(1)  Regulation (EU) No 300/2008 of the European Parliament and of the Council of 11 March on 

common rules in the field of civil aviation security and repealing Regulation (EC) No 2320/2002;  

(2)  Commission Regulation (EC) No 272/2009 of 2 April 2009 supplementing the common basic 
standards on civil aviation security laid down in the Annex to Regulation (EC) No 300/2008 of 
the European Parliament and of the Council;  

(3)  Commission Regulation (EU) No 1254/2009 of 18 December 2009 setting criteria to allow 

Member States to derogate from the common basic standards on civil aviation security and to 
adopt alternative security measures;  

(4)  Commission Regulation (EU) 2016/2096 of 30 November 2016 amending Regulation (EU) No 

1254/2009 as regards certain criteria to allow Member States to derogate from the common basic 
standards on civil aviation security and to adopt alternative security measures;  

(5)  Commission Regulation (EU) No 18/2010 of 8 January 2010 amending Regulation (EC) No 
300/2008 of the European Parliament and of the Council as far as specifications for national 
quality control programmes in the field of civil aviation security are concerned;  

(6)  Commission Implementing Regulation (EU) 2015/1998 of 5 November 2015 laying down 

detailed measures for the implementation of the common basic standards on aviation security. 

36 

 
                                                           
personal data, a relevant consent for the processing of such data as required by the GDPR 
must be obtained from all persons identified in the report.   

However,  when  it  comes  to  removal  of  personal  data,  the  Regulation  distinguishes 
between  two  forms  of  such  removal.  On  the  one  hand,  when  uploading  occurrence 
reports into the national databases59 and subsequently transferring them into the ECR, or 
when disseminating the information within the organisation concerned60, the Regulation 
requires  the  disidentification  of  the  data.  Disidentified  information  is  defined  as 
“information arising from occurrence reports from which all personal data such as names 
or addresses of natural  persons have been removed”.61 However, such information may 
still  contain  other  contextual  information,  which  may  lead  to  the  identification  of  the 
persons  involved  in  the  reported  occurrence.  Such  data  might  however  be  essential  for 
the meaningful reporting, the analysis and a follow-up of occurrences.  At the same time, 
disidentified  information  can  still  contain  personal  data  and  hence  compliance  with  the 
GDPR rules must be ensured.  

On  the  other  hand,  the  Regulation  requires  a  full  anonymization62,  i.e.  removal  of  all 
personal data when it comes to access to the occurrence data by the interest parties listed 
in Annex II to the Regulation or when the occurrence data are published by the Member 
States either as individual reports63 or in the annual safety reports64.  

Finally, the Regulation’s coherence with Directive (EU) 2019/1937 on the protection of 
persons who report breaches  of Union law was  also  assessed. The Directive lays down 
rules  for  common  minimum  standards  to  ensure  the  establishment  of  internal  and 
the  adequate  protection  of  reporting  persons 
external  reporting  channels  and 
(“whistleblowers”). The  Directive explicitly states that it shall  complement the existing 
rules on the reporting of occurrences under Regulation (EU) No 376/2014, which at the 
same time constitute breaches of European law. Consequently, once transposed in 2021, 
the  Directive  is  anticipated  to  have  some  impacts  on  the  regime  established  by  the 
Regulation.  

The  Directive  establishes,  amongst  other  provisions,  specific  details  on  how  reporting 
channels should be set-up, how reports should be followed-up, including an obligation to 
provide  feedback  to  the  reporting  person.  This  is  expected  to  complement  the 
Regulation’s regime and potentially partly solve some of the shortcomings related to the 
lack  of  feedback  from  the  competent  authorities  as  identified  in  the  section  on 
effectiveness above.  

The Directive also provides for a number of enhanced measures that the Member States 
shall introduce to protect the reporters and penalties for breaching the confidentiality of 

59   Regulation (EU) no 376/2014, Art. 16(3).  
60   Ibid, Art 16(2).  
61   Ibid, Art 2(6).  
62   Ibid, Art 11(4).  
63   Ibid, Art 13(12).  
64   Ibid, Art 13(11).  

37 

 
                                                           
the reports as well as for a retaliation against the reporter, which could, in some aspects, 
complement the Regulation’s own regime on the protection of reporters.  

In conclusion, the Regulation is both coherent with and complemented by the Directive. 

Conclusions on coherence 

The  Regulation  is  internally  coherent  and  contains  no  overlaps,  contradictions  or 
inconsistencies.  Overall,  the  Regulation  is  coherent  with  other  European  legislation  on 
aviation safety. The Regulation’s reporting requirements however partially overlap with 
the requirements contained in Regulation 2018/1139 and its implementing rules. Further 
analysis  on  the  impacts  of  removing  those  overlaps,  possibly  through  a  consolidation 
process, could be considered.  

The Regulation is also coherent with other relevant EU legislation.  

5.5.  EU Added Value 

The Regulation has been found to bring EU added value in the following areas: 

First, the Regulation has standardised the approach to occurrence reporting and analysis 
throughout  the  EU,  unifying  criteria  and  leading  to  increased  rates  of  reporting  and 
higher  quality  reports.  Results  from  the  baseline  analysis  show  a  clear  improvement  in 
the  reporting  rates  and  completeness  of  reports.  The  Regulation  has  created  a  common 
framework  at  the  EU  level,  which  has  facilitated  the  harmonisation  of  occurrence 
reporting,  analysis  and  follow-up  actions  across  Member  States  and  across  all  industry 
domains. 

Second,  the  EU  involvement  has  enabled  the  pooling  of  information  and  experience  at 
the  European  level.  This  improves  coordination  among  Member  States,  EASA  and  the 
Commission, for example via the Network of Analysts. All Member States are involved 
in the proceedings and have full access to the ECR data. It is expected that the exchange 
of  safety  information  will  improve  with  the  help  of  modern  and  efficient  e-tools  (e.g. 
ECCAIRS  2,  Data4Safety).  The  Regulation  has  enabled  an  EU-wide  understanding  on 
aviation safety that could not have been achieved by the Member States themselves and 
has in turn contributed to the further maturation of the EPAS. 

Third, the Regulation has supported the development of the “Just Culture” and helped to 
increase  the  awareness  of  the  importance  of  occurrence  reporting  for  safety,  which  in 
turn led to an increase in the number of occurrence reports submitted. 

Finally, with the upcoming replacement of the current ECCAIRS platform with a more 
modern  and  efficient  software  platform  developed  and  managed  by  EASA  and  the 
adoption  of  a  common  risk  classification  scheme  as  required  by  Regulation  (EU)  No 
376/2014 further improvements in the collection and analysis of the occurrences could be 
expected.  

38 

 
On  the  basis  of  all  those  elements  it  can  be  concluded  that  the  EU  added  value  of 
Regulation (EU) No 376/2014 is very high. 

6.  CONCLUSIONS 

The  Regulation  has  met  all  evaluation  criteria.  As  the  Regulation  is  a  part  of  a  wider 
aviation safety framework, it is difficult to isolate its effects. Nonetheless, the evaluation 
suggests  that  the  Regulation  has  contributed  to  the  overall  improvement  of  aviation 
safety in the EU.  

While it  is  impossible to draw  conclusions on the exact  impact  of Regulation (EU) No 
376/2014  on  the  reduction  of  accidents  and  fatalities  in  civil  aviation,  there  are  some 
indications that the Regulation played a positive role in decreasing aviation accidents in 
Europe.  

The Regulation  has made a significant  step towards achieving its specific objectives.  It 
has  been  particularly  successful  in  increasing  the  number  of  occurrences  reported  and 
collected  and  it  had  a  positive  impact  on  the  level  of  reporting.  Nevertheless,  some 
shortcomings relating to the big amount of mandatory data fields or too short reporting 
deadlines  were  identified,  which  could  have  led  to  an  increased  workload  both  at 
organisational and Member State levels and might have had an impact on the quality and 
completeness of the reports.  

The  Regulation  has  contributed  to  better  collection  and  storage  of  the  occurrence 
reporting. There has been a considerable improvement in the quality and completeness of 
the occurrence data contained in the ECR and national databases. This however implies a 
considerable  workload  for  the  involved  stakeholders  and  especially  for  the  authorities. 
Digital  processing  offered  by  the  recently  introduced  ECCAIRS  2  software  should 
mitigate some of those difficulties.  

Concerning data protection and exchange of information, as occurrence reporting uses no 
IT  technology  which  would  ensure  full  disidentification  of  the  reports,  personal  details 
occasionally  are  registered  in  the  national  databases  and  the  ECR.  In  this  regard  better 
enforcement of GDPR requirements is needed, and as of December 2021, the competent 
authorities should also ensure the confidentiality of within the obligations established by 
the Whistleblowing Directive. The evaluation  concluded that access to the ECR is well 
protected and that ECR data have been used for safety purposes only.   

The  Regulation  however  only  partially  achieved  its  objectives  with  regard  to  the 
protection  of  reporters  and  the  "just  culture”.  Only  a  fraction  of  Member  States 
established advance administrative arrangements with their respective judicial authorities 
to  manage  the  access  to  and  the  use  of  occurrence  reports  contained  in  their  national 
databases.  Moreover,  a  number  of  Member  States  failed  to  designate  a  “just  culture 
body”  with  the  necessary  competences  to  implement  its  duties  and  guarantees  that 
reporters are not punished for reporting. To address this problem the Commission opened 
several infringement proceedings against Member States that are lagging behind.  

39 

 
At the EU level, the Regulation has facilitated the analysis of occurrence reports through 
the  standardisation  of  requirements  what  resulted  in  the  increase  of  quality  of  reported 
data.  The  annual  publication  of  the  EPAS,  including  key  risk  areas  and  actions  to 
mitigate safety concerns is a good example. At a national level, however, the evaluation 
has identified an apparent lack of analysis and follow up capabilities. A lack of expertise, 
both  in  terms  of  adequate  resources  and  procedures  seem  to  be  the  root  causes. 
Furthermore,  industry  stakeholders  have  highlighted  the  lack  of  feedback  from  the 
competent authorities on the analysis of the reported occurrences.  

The  Regulation’s  main  deficiency  is  the  lack  of  clarity  on  the  competent  authority 
responsible for the oversight of the entire Regulation. The Regulation allows the Member 
States to nominate up to three ‘competent authorities’ to run the process from collection 
to analysis, with no further obligations on oversight. Some of those authorities seem not 
to be properly equipped to oversee the follow up actions.  

In  terms  of  relevance,  the  evaluation  shows  that  Regulation  (EU)  No  376/2014  and  its 
objectives  remain  relevant  also  in  face  of  the  latest  developments  in  aviation,  allowing 
for taking account of the upcoming increase of UAS traffic.  

Regulation  (EU)  No  376/2014  has  resulted  in  costs  for  both  competent  authorities  and 
organisations, but the identified benefits clearly outweigh the costs incurred. 

In  terms  of  coherence,  Regulation  (EU)  No  376/2014  is  fully  coherent  internally  and 
overall,  it  is  consistent  with  other  EU  instruments,  notably  with  Regulation  (EU) 
2018/1139,  Regulation  (EU)  No  996/2010  on  accident  and  incident  investigation, 
Regulation  (EU)  2016/679  on  the  protection  of  natural  persons  with  regard  to  the 
processing of personal data and on the free movement of such data, Regulation (EU) on 
the  protection  of  natural  persons  with  regard  to  the  processing  of  personal  data  by  the 
Union institutions, bodies, offices and agencies and on the free movement of such data 
and  the  Directive  (EU)  2019/1937  on  the  protection  of  persons  who  report  breaches  of 
Union law. The interaction between Regulation (EU) 2018/1139 and Regulation (EU) No 
996/2010 warrant further consideration whether reporting and analysis obligations should 
be streamlined into one  set of obligations. 

Finally, based on the evidence analysed, it can be concluded that the Regulation brought 
a  clear  added  value  in  the  area  of  occurrence  reporting,  analysis  and  follow-up.  It  has 
contributed to a clear improvement in reporting rates, completeness of reports, amount of 
safety-relevant data available across the EU as well as to the protection of reporter and 
introduction of “just culture” principles.  

Regulation  (EU)  No  376/2014  has  fostered  the  coordination  amongst  Member  States, 
EASA and the Commission via different groups (e.g. Network of Analysts). Exchange of 
safety information is possible via the access to the European Central Repository, set up 
by the EU by all Member States’ authorities (including safety investigation authorities). 

40 

 
Annex 1: Procedural information 

1.  LEAD DG, DeCIDE PLANNING/CWP REFERENCES 

Lead DG:  

DG MOVE  

Planning reference:   PLAN/2019/5269 

2.  ORGANISATION AND TIMING 

Article 24 of Regulation (EU) No 376/2014 establishes the legal obligation to submit an 
evaluation report on the implementation of this Regulation to the Parliament and to the 
Council. 

The  evaluation  was  launched  in  April  2019  by  establishing  an  inter-service  steering 
group (ISSG). The first meeting of the ISSG took place on 12 April 2019 and discussed 
the detailed planning of the evaluation exercise as well as the content of a draft roadmap 
outlining  the  procedural  and  methodological  steps  to  be  taken.  The  roadmap  was 
published for feedback for a period of four weeks. The roadmap was finalised on 22 May 
2019. 

The Commission launched a call for tenders for a support study on “Ex-post evaluation 
of Regulation (EU) No 376/2014 on the reporting, analysis and follow-up of occurrences 
in  civil  aviation”.  A  specific  contract  No  MOVE/E4/SER/2019-422/SI2.811176  was 
signed  with  an  external  consultant  Ramboll  Management  Consulting  A/S  on  23  May 
2018  under  the  framework  contract  No  MOVE/A3/2017-257.  The  external  consultant 
delivered the first draft of the support study on 24 February 2020. The final delivery of 
the final report was sent on 12 May 2020 and the ISSG subsequently approved the final 
report and it was published by the Commission. 

The  evaluation  study  is  based  on  the  assessment  of  the  effectiveness,  efficiency, 
coherence,  relevance  and  EU  added  value  of  Regulation  996/2010,  and  reviews  the 
objectives of the original regulatory intervention and the performance of the intervention 
as  compared  to  the  initial  expectations  and  the  current  situation.  The  study  also 
determines whether there are overlaps with other safety-related regulatory tools. 

Based on the answers received in the OPC and based on the support study produced by 
the external consultant, the Commission proceeded with drafting this Commission Staff 
Working Document (CSWD).  The ISSG was consulted on the draft CSWD on 16 and 30 
September 2020. 

3.  EXCEPTIONS TO THE BETTER REGULATION GUIDELINES 

No exceptions to the Better Regulation Guidelines. 

41 

 
 
 
 
4.  CONSULTATION OF THE RSB (IF APPLICABLE) 

N/A 

5.  EVIDENCE, SOURCES AND QUALITY 

The evaluation of Regulation 376/2014 was based on the intervention logic of Regulation 
(EU) No 376/2014 and a comprehensive analytical framework comprising the evaluation 
questions and their respective judgement criteria, indicators and information sources. 

The  data  collection  tools  used  to  gather  the  relevant  information  consisted  of  desk 
research, targeted online survey, targeted cost-benefit questionnaires, targeted interviews, 
expert meetings, public consultation and five case studies. 

legislation,  reports  and  other 
The  documents  reviewed  consisted  of 
communications,  ENCASIA 
ICAO  documents,  advanced  arrangements 
concluded  between  the  SIAs  and  national  judicial  authorities,  safety  recommendations, 
documents  and  reports  by  the  national  investigation  authorities,  and  other  sources, 
including academic articles and publications. 

the  EU 

reports, 

The targeted online survey covered a representative selection of stakeholders involved in 
the or affected by the reporting, analysis and follow-up of occurrences in  civil  aviation 
including national competent authorities, safety investigation authorities, EASA, but also 
all different  industry sectors, and associations  representing staff in  the aviation  field.  A 
total  of  67  respondents  fully  completed  the  online  survey,  and  32  respondents  partially 
completed it. 

The targeted cost-benefit questionnaire dissemination followed the same approach as that 
used  for  the  online  survey.  In  total  17  respondents  completed  the  targeted  cost-benefit 
questionnaire. 

Targeted interviews were conducted with a range of different stakeholder groups, and a 
wide  geographical  scope,  namely  with  European  Commission  officials  and 
representatives  of  the  following  organisations:  ICAO,  EUROCONTROL,  EASA, 
competent authorities in EU Member States, Safety Investigation Authorities, just culture 
bodies  /  judicial  authorities,  commercial  aircraft  operators,  manufacturers,  maintenance 
industry, airports / ground handling operators, aviation personnel, General aviation / non-
commercial  operations  /  UAS,  ANSPs,  ATC  service  providers,  research  centre,  and 
passengers. A total of 39 targeted interviews were carried out. 

In  the  context  of  the  expert  meetings,  the  study  team  facilitated  two  workshops  and 
attended  a  third  one  as  an  observer  with  support  from  EASA  and  the  European 
Commission. The first workshop was a full day workshop with competent authorities; the 
second workshop was a half-day workshop with the Network of Aviation Safety Analysts 
(NoA);  and,  the  third  workshop  was  a  full-day  workshop  with  a  range  of  industry 
stakeholders. 

42 

 
In  addition,  the  European  Commission  launched  an  online  public  consultation  on  7 
November  2019  to  gather  the  opinions  of  the  broader  stakeholder  community  on 
Regulation 376/2014. A total of 198 responses were received. 

Five  case  studies  were  conducted  on  the  application  of  the  Regulation  376/2014.  The 
case studies were selected based on the criteria of the relevance vis-à-vis the evaluation 
questions,  falling  within  the  scope  of  the  Regulation,  being  related  to  Member  States 
compliance  and  based  on  the  diversity  and  the  availability  of  sufficient  amount  of 
details/information. 

Then,  an  assessment  of  the  state  of  implementation  of  the  Regulation  376/2014  was 
through  desk-based  research,  stakeholder 
carried  out  based  on  data  collected 
consultations,  data  collection  tools,  the  online  survey  and  interviews.  This  information 
was  then  triangulated  to  enable  a  judgement  to  be  made  on  the  extent  to  which  the 
requirement has been implemented. 

Furthermore,  an  analysis  of  the  evaluation  baseline  was  carried  out.  In  particular,  the 
baseline  analysis  covered  the  effectiveness  of  the  Regulation,  particularly  pertaining  to 
the  extent  to  which  the  Regulation  has  impacted  the  levels  of  safety  in  the  EU.  The 
methodology used concerns the Interrupted Time Series (ITS) methodology, which was 
considered the strongest  quasi-experimental method to derive causal inference from the 
data available. 

Finally,  a  cost-assessment  was  carried  out.  The  targeted  cost-benefit  questionnaires 
formed the basis of this assessment, supplemented by estimations where needed to fill the 
gaps. The methodology that was applied for this assessment is the Standard Cost Model 
(SCM), which allows to address the costs and burdens of Regulation 376/2014 for both 
Member States (competent authorities) and industry stakeholders. 

43 

 
 
Annex 2: Stakeholder consultation 

Introduction 

The objective of this synopsis is to provide an overview of the results of the stakeholder 
consultation  carried  out  in  order  to  evaluate  the  Regulation  376/2014  on  the  reporting, 
analysis and follow-up of occurrences in civil aviation. It includes a general analysis of 
the several methods used to reach the stakeholders, which were: 

  An online survey launched on the 13th November 2019 and extended until the 9th 

December 2019; 

  A cost-benefit questionnaire launched on the 13th November 2019 and extended 

until the 9th December 2019; 

  Targeted interviews; 
  Two workshops with competent authorities held on the 19th September 2019 and 

30th October 2019; 

  A workshop with industry stakeholders held on the 25th November 2019; 
  An open public consultation launched on the 7th of November 2019 until the 30th 

of January 2020. 

The goal was to assess to what extent the aim of the Regulation was achieved, through an 
independent and evidence-based investigation. It also gave the possibility to stakeholders 
to  express  themselves  and  share  their  opinion  on  the  progress  made  in  accident  and 
occurrence reporting in civil aviation since the Regulation entered into force. 

Consultation activities 

Online survey 

Launched  on  the  13th  November  2019,  the  survey  was  targeted  to  a  range  of  different 
stakeholder  groups,  including  national  competent  authorities,  safety  investigation 
authorities,  EASA,  but  also  all  different  industry  sectors,  and  associations  representing 
staff  in  the  aviation  field.  A  total  of  67  respondents  fully  completed  the  online  survey, 
and  32  respondents  partially  completed  it.  Stakeholders  were  often  unresponsive  to  the 
launch  of  the  survey  and  consequently  the  deadline  had  to  be  extended  and  reminders 
sent.  No  answers  were  collected  from  three  groups:  judicial  or  law  enforcement 
authorities, EFTA surveillance authority, and aerodrome operators. All other groups were 
represented,  at  least  by  a  few  respondents  (Figure  1).  The  respondents  were  from  28 
Member States (Belgium, Croatia, Cyprus, Czechia, Denmark, Estonia, Finland, France, 

44 

 
 
 
 
 
Germany,  Greece,  Hungary,  Ireland,  Italy,  Latvia,  Lithuania,  Luxembourg,  Malta, 
Netherlands,  Norway,  Poland,  Portugal,  Romania,  Slovakia,  Slovenia,  Spain,  Sweden, 
Switzerland and United Kingdom).  

Figure 1: Distribution of the respondents to the targeted survey over the 
stakeholder groups (N=99). 

The  online  survey  included  questions  on  the  effectiveness,  relevance,  and  EU  added 
value  of  Regulation  376/2014,  as  well  as  a  few  questions  pertaining  to  the  degree  to 
which the Regulation has been implemented in the Member States. 

Cost-benefit questionnaire 

Launched  on  the  13th  November  2019,  the  cost-benefit  questionnaire  was  targeted  to 
national authorities on one hand, and industry stakeholders on the other.  

On  the  authority  side,  a  total  of  14  completed  questionnaires  were  returned  (Figure  2), 
covering 14 different Member States  On the industry side, however, only 3 completed 
questionnaires were returned: from a ground handling service provider, an aircraft parts 
maintenance organisation, and an aviation manufacturer. 

45 

 
 
 
 
[CELLRANGE] 

[CELLRANGE] 

Civil Aviation Authority

Safety Investigation
Authority

Figure 2: Distribution of the national authorities that responded to the cost-benefit 
questionnaire over the role within the framework of Regulation 376/2014 (N=14). 

The  cost-benefit  questionnaire  contained  questions  pertaining  to  the  efficiency  of 
Regulation 376/2014, including specific questions regarding the costs incurred as a result 
of the Regulation, and how these costs compare to the benefits achieved.  

Targeted interviews 

A total of 40 targeted interviews were carried out in the context of the evaluation support 
study. They cover a range of different stakeholder groups, and a wide geographical scope 
(Table 1). 

Some  of  the  interviews  conducted  were  group  interviews  involving  two  or  more 
stakeholders,  so  the  total  number  of  organisations  covered  is  higher  than  the  foreseen 
number  of  interviews.  Issues  encountered  in  scheduling  interviews  (e.g.  because  of  the 
Christmas holidays, and aviation professionals having limited time to talk to us) caused 
delays compared to the initially foreseen timeline. However, the deadline was extended 
to mitigate this, and the foreseen number of interviewees was attained. Overall, the study 
team  judges  the  sample  consulted  to  be  representative  of  the  breadth  of  stakeholders 
concerned by Regulation 376/2014. 

Table 1: Overview of stakeholders interviewed. 

Stakeholder  
ICAO  
EUROCONTROL  
European Commission  
EASA  
Competent authorities in EU Member States  
Safety Investigation Authorities (SIA) 
Just culture bodies / Judicial authorities   
Commercial aircraft operators  

Planned 
1 
1 
2 
2 
5 
3 
2 
4 

Conducted 
1 
1 
2 
2 
5 
3 
2 
4 

46 

 
 
 
 
Stakeholder  
Manufacturers  
Maintenance industry  
Airports / ground handling operators  
Aviation personnel  
General aviation / non-commercial operations / UAS  
ANSPs 
ATC service providers  
Research centre  
Passengers  
Total 

Planned 
2 
2 
3 
3 
3 
3 
2 
1 
1 
40 

Conducted 
2 
2 
3 
3 
3 
3 
2 
1 
1 
40 

The  targeted  interviews  aimed  to  complement  the  questionnaires  by  collecting  more 
details on why stakeholders hold certain opinions, as well as gather additional evidence 
in  relation  to  evaluation  questions  for  which  qualitative  data  was  judged  to  be  an 
important source. 

First workshop with competent authorities 

A first workshop with competent authorities was held at EASA premises in Cologne on 
19th  September  2019,  after  the  SMTeB  meeting.  Hence,  it  was  attended  by 
representatives of the Member States’ competent authorities, some of which are SMTeB 
members.   

A total of 17 Member States were present at the meeting and six non-EU Member States 
(Moldova, Norway, Serbia, Montenegro, Iceland, and Switzerland), and EASA.   

The first workshop was a full day workshop with competent authorities, which aimed to 
gather insights into: 

1.  the effectiveness, efficiency and coherence of Regulation 376/2014; 
2.  reporting and collecting;   
3.  data protection, confidentiality and just culture; 
4.  exchange of information, dissemination and actions. 

The  authorities  were  split  up  into  discussion  groups  (ensuring  a  balanced  geographical 
and gender representation in each group), where they discussed each of the above topics, 
following  a  list  of  predetermined  questions.  After  two  discussion  sessions,  participant 
rapporteurs were asked to present their key findings in plenary. 

Second workshop with competent authorities 

The  second  workshop  with  competent  authorities  was  held  on  30th  October  2019  in 
Cologne.  

47 

 
 
 
 
It  constituted  of  a  half-day  workshop  with  members  of  the  network  of  analysts  (NoA). 
The  topics  covered  were  therefore  primarily  related  to  analysis  and  follow  up.  
Considering  the  limited  time  and  space  available,  live  audience  polling  software  was 
used  to  gather  data  from  participants,  specifically  covering  questions  on  four  broad 
topics: 

1.  storage;  
2.  risk classification;  
3.  analysis;  
4.  actions and dissemination. 

The results of given questions were discussed by a participant panel, after which the floor 
was  opened  to  additional  comments  from  all  other  participants.  This  data  was 
subsequently  analysed  in  Excel,  and  completed  with  notes  taken  by  a  member  of  the 
study team who attended the workshop as an observer. 

Workshop with industry stakeholders 

The third workshop  was a full-day  workshop and was held  on 25th  November 2019 in 
the Albert Borschette conference centre in Brussels. The study team, with support from 
EASA and the European Commission, facilitated the workshop.  

A  range  of  different  industry  stakeholders  were  invited  to  attend,  to  get  as  broad  a 
stakeholder  coverage  as  possible  (Table  2).  However,  it  proved  difficult  to  get  some 
groups  on  board,  e.g.  large  airlines,  airports,  large  aircraft  manufacturers,  ground 
handling service providers, maintenance organisations. Yet, a decent balance was struck, 
both  in  terms  of  geographical  coverage  and  organisations  types,  although  many  were 
represented by their respective associations. 

Table 2: Stakeholder groups present at industry workshop. 

Sector  
ANSPs 

ATC 

Operators  

General aviation   
Airports  
Ground handling service provider  
Maintenance organisation  
Manufacturer 

Organisation  
DFS (Germany)  
ENAIRE (Spain)   
PANSA (Poland) 
ENAV (Italy)  
FerroNATS (Spain) 
A4E (Association – airlines) 
EHA (Association – helicopters) 
Luxaviation Group  
Brussels Airport  
ASA World (Association) 
Lufthansa Technik 
Dassault  
ATR  
GAMA Europe 

48 

 
 
Sector  

Aviation personnel  

International/regional organisations  

Organisation  
ASD (Association)  
ECA (Association – pilots)  
EBAA (Association – operators)  
IFATSEA 
– 
electronics) 
IFATCA (Association – ATC)  
EUROCONTROL  
IATA 

(Association 

safety 

The workshop consisted of live audience polling using the software “Poll Everywhere”, 
followed by  comments  by  the predetermined participant  panel,  and comments  from  the 
audience.  Questions covered four topics:  

1.  mandatory and voluntary occurrence reporting;  
2.  risk classification, analysis and follow-up;  
3.  data protection and just culture;  
4.  general issues related to Regulation 376/2014. 

The results of given questions were discussed by a participant panel, after which the floor 
was  opened  to  additional  comments  from  all  other  participants.  This  data  was 
subsequently analysed in Excel, and completed with notes taken by the study team. 

Public Consultation 

The European Commission designed  and carried  out  a Public Consultation (PC) on the 
Evaluation of Regulation (EU) No 376/2014 on the reporting, analysis and follow-up of 
occurrences  in  civil  aviation.  The  PC  was  launched  on  the  7th  of  November  2019  and 
remained  open  for  responses  until  the  30th  of  January  2020.  A  total  of  198  responses 
were received (Figure 3).  

Figure 3: Contribution per stakeholder 

49 

 
 
 
 
 
 
The  Public  Consultaiton  consisted  of  21  questions  and  the  structure  of  this  report  is  as 
follows: 

1.  Profiling questions 
2.  Relevance 
3.  Effectiveness 
4.  Coherence 
5.  Just Culture 
6.  Efficiency 
7.  EU Added Value 
8.  Additional Comments 

The  answers  were  analysed  in  Excel  and  used  to  answer  the  evaluation  questions 
contained in the evaluation matrix. 

The questionnaire opened with a general question relating to the relevance of Regulation 
376/2014.  The  respondents  were  asked  for  their  views  on  whether  the  process  of 
occurrence reporting is  an appropriate method  for collecting safety related information. 
Overall, the majority of respondents answered that they “strongly agree” while just a few 
responded “disagree”. It is worth noting that while “strongly disagree” was an option in 
this question, it gained zero responses (Figure 4).  

Figure 4: PC Question on Relevance 

With regards to the collection and analysis of occurrence information, the majority of the 
respondents  were  in  agreement  that  the  collection  and  analysis  of  occurrence  reports 
leads to a clear picture of aviation safety risks (Figure 5). In addition, the majority agree 
that  the  occurrence  reporting  logic  of  the  Regulation  is  effective  in  achieving  an 
evidence-based  aviation  system.  On  the  other  hand,  regarding  the  completeness  of  the 
data, there was not a clear majority, however, the largest support was gathered from the 
participants  that  agreed  that  the  Regulation  ensures  that  all  safety  related  information 
which has the potential to endanger aviation safety is collected. 

50 

 
 
 
 
Figure 5: PC Question on Effectiveness 

Concerning  the  action  on  safety  information,  the  participants,  although  without  a  clear 
majority, mostly agreed that industry organisations or public authorities take appropriate 
actions based on the analysis of the occurrence received. Also, the number of respondents 
who agree with the opinion that the Regulation ensures that safety action can be taken in 
a  timely  manner  was  the  highest,  though  without  a  clear  majority.  In  addition,  the 
majority  of  the  respondents  agree  that  the  Regulation  has  improved  aviation  safety  by 
ensuring  that  relevant  safety  information  is  reported,  collected,  protected,  analysed  and 
disseminated.  Moreover,  the  majority  agree  with  the  view  that  annual  safety  reviews 
from Member States and EASA are both informative and provide a view of the types of 
aviation safety. 

In relation to coherence with other legislations, the respondents were asked to highlight 
their  agreement  or  disagreement  with  three  statements  that  referred  to  the  coherence 
between  Regulation  376/2014  and  other  regulations  and  organisational  rules.  The 
statement that gathered the most support was that the Regulation was coherent with other 
EU  related  legislation.  On  the  second  statement,  there  was  a  similar  finding  that  the 
majority  of  the  respondents  were  in  agreement  that  the  Regulation  is  coherent  with 
international aviation-related regulations. Out of the three statements, the statement that 
‘the Regulation is coherent with other regulations derived from social policies’ had more 
of a variety of responses (Figure 6). 

51 

 
 
 
Figure 6: PC Question on Coherence 

Regarding the encouraging reporting while retaining the need for sufficient information, 
there was a  great  variety of  opinions on whether the Regulation  stimulates and ensures 
consistency and a holistic approach to safety management. When asked to elaborate on 
why  they  disagreed  with  the  statement,  the  most  salient  theme  was  that  the  Regulation 
does not act as an “efficient driver” with respect to safety management. Moreover, there 
was  a  strong  proportion  of  the  respondents  that  agreed  with  the  statement  that  the 
Regulation ensures a good balance between encouraging reporting and the need to have 
sufficient information. 

Concerning  the  reporting  culture  and  protection  of  information,  the  respondents  were 
asked if they agreed or disagreed that the Regulation is able to foster a “just culture” and 
positive reporting culture with  respect  to  their national  legislative frameworks. Overall, 
the vast majority of the respondents are in agreement that the Regulation is able to foster 
a  just  culture  (Figure  7).  In  addition,  a  strong  majority  are  in  agreement  that  the 
Regulation ensures that there is enhanced protection for persons mentioned in occurrence 
reports. With regards to the current provisions of the Regulation in terms of ensuring that 
information  reported  is  used  for  safety  improvement  purposes  only,  the  majority  of 
respondents answered that they agree.  

52 

 
 
 
Figure 7: PC Question on Just Culture 

When asked about the existence of a ‘just culture’ body in Member States, the majority 
of the respondents were in agreement that they are aware of the existence of a just culture 
body in their Member State. 

Two  main  questions  were  asked  which  directly  relate  to  the  alignment  of  reporting 
methods and other safety reporting obligations. First, the respondents were asked if the 
types of occurrences that are required to be reported on are well aligned with other safety 
reporting obligations. There is a variety of responses and no clear majority from any of 
the  respondents  with  a  large  proportion  either  in  disagreement  or  neither  agree  nor 
disagree  (Figure  8).  In  the  second  question,  there  is  a  divide  in  opinion  on  where 
regulatory burdens have been brought about as a result of the Regulation. To explore the 
factors behind the decision from the people who responded “agree” or “strongly agree”, a 
follow-up  question  allowed  for  participants  to  elaborate  further.  An  example  of  an 
element  which  imposes  these  issues  are  the  number  of  mandatory  fields  that  reporters 
need  to  fill  in  when  completing  an  occurrence  report.  The  other  themes  that  emerged 
included  the  lack  of  coordination  both  within  the  Regulation  and  with  other  reporting 
obligations and issues regarding the lack of feedback and increased costs. 

Figure 8: PC Question on Efficiency 

The final section looks at the added value of the Regulation which compares the existing 
situation  with  a  hypothetical  scenario  in  which  the  Regulation  does  not  exist.  The 

53 

 
 
 
 
 
analysis  then  identifies  the  changes,  which  can  reasonably  be  argued  to  be  due  to  the 
Regulation being in place, as compared to the scenario where only national measures by 
the EU Member States are in place. This section starts with the question if the Regulation 
has  put  provisions  in  place  to  harmonise  information  storage  and  risk  classification. 
There is no clear majority for any of the responses. The largest response came from the 
respondents who agreed that the Regulation had put provisions in place (Figure 9). Then, 
the  respondents  were  asked  to  identify  if  they  agree  or  disagree  that  the  Regulation 
includes  a  list  of  occurrences  to  be  mandatorily  reported  by  the  different  types  of 
reporters. In particular, the respondents were asked to answer whether they thought this 
classification had improved the clarification on occurrences to be reported. The majority 
of  responses  were  in  agreement  that  there  was  an  improvement.  Moreover,  the  vast 
majority of responses think that stopping the Regulation would have a negative effect on 
aviation safety. 

Figure 9: PC Question on EU Added Value 

Results of consultation activities 

Stakeholders have provided feedback, which has helped evaluate the Regulation, identify 
the  issues  and  have  also  provided  with  some  suggestions  for  improvement.  These 
suggestions  came  mainly  through  the  workshop,  the  focus  groups  and  the  open  public 
consultation.  In  general,  participants  have  considered  that  although  there  have  been 
improvements, there are still shortcomings to be addressed. 

Relevance 

During  the  consultation  activities,  there  was  an  overall  agreement  that  the  Regulation 
376/2014’s  objectives  continue  to  be  relevant  today,  as  there  will  always  be  a  need  to 
reduce  the  number  of  aircraft  accidents  and  fatalities  through  a  proactive,  preventative 
approach  to  aviation  safety  at  EU  level  based  on  information  stemming  from  daily 
occurrences.  However,  it  was  mentioned  that  the  Regulation  does  not  cover  reporting 
requirements for individual UAS operators, and there is still a lack of clarity in the UAS 
domain in terms of what needs to be reported on. 

54 

 
 
 
 
Effectiveness 

There  was  an  overall  agreement  through  the  consultation  activities  that  the  Regulation 
376/2014 has ensured that more occurrences are reported and collected than in the past, 
contributing to a more complete picture of aviation safety than was previously the case 
by ensuring that occurrence reporting is  done more frequently across  different  domains 
and Member States. That being said, its impact in this regard has been greater in industry 
domains/Member States which did not have a mature reporting system in place prior to 
the inception of the Regulation. 

Despite  such  increasing  trends,  the  effectiveness  of  the  reporting  process  has  been 
hampered  by  a  number  of  factors,  notably  the  time  and  effort  associated  with  the 
extensive requirements for reporting in Regulation 376/2014. This includes the long list 
of  mandatory  data  fields  which  are  not  all  relevant  to  all  reporting  organisations,  and 
timelines for reporting and analysis which cannot always be met. As a result, there is a 
lack of voluntary reporting on safety hazards or other occurrences which are not part of 
the MOR system, primarily because of a lack of clarity on what constitutes “other safety 
related  information”,  and  the  limited  resources  available  which  instead  are  spent  on 
compliance  with  the  extensive  requirements  for  reporting  on  mandatory  occurrence 
types.   

During  the  consultation  activities,  it  was  concluded  that  the  Regulation  376/2014 
adequately  sets  out  what  occurrences  need  to  be  reported  on  in  principle,  pursuant  to 
Article 4(1) and Implementing Regulation 2015/1018, although it would be appropriate 
to conduct a thorough review with input from aviation professionals to ensure any new 
occurrence types are covered. 

There  has  been  a  considerable  improvement  in  the  quality  and  completeness  of  the 
occurrence  data  contained  in  the  ECR  and  national  databases  as  a  result  of  Regulation 
376/2014 which put in place a standardised framework for both mandatory and voluntary 
reporting.  However,  there  are  still  gaps  in  the  quality  and  completeness  of  reports 
contained  in  the  ECR  because  of  different  approaches  to  quality  checking  and  data 
transfer  (e.g.  Member  States  not  ‘matching’  duplicate  reports),  which  complicates  the 
analysis at EU level. 

Other  conclusion  consisted  on  Regulation  376/2014  being  partly  effective  in  ensuring 
data  protection.  The  organisations  concerned  by  occurrence  reporting  and  competent 
authorities alike have processes in place to ensure occurrence reports are disidentified so 
that  no  personal  information  is  recorded  in  their  (national)  databases  or  in  the  ECR. 
However, these processes were not necessarily established as a result of the Regulation. 

Many  Member  States  have  not  yet  designated  a  ‘just  culture’  body.  Even  among  those 
who have, only six Member States have put the necessary rules in place to ensure they 
are  empowered  to  carry  out  the  activities  stipulated  in  Article  16(6),  (9)  and  (11)  of 
Regulation 376/2014. 

55 

 
The analysis of reported occurrences takes place at three different levels – the level of the 
organisations,  at  national  level  and at  EU level,  but  it is  not working optimally in  each 
case,  leading  to  limited  identification  of  safety  risks.  Overall,  there  is  always  room  for 
improvement,  e.g.  by  taking  into  account  exposure  data  in  the  analysis,  rather  than 
focusing on occurrence data alone. 

Moreover,  industry  stakeholders  are  widely  of  the  opinion  that  the  authorities  are  not 
providing  sufficient  feedback  on  the  outcome  of  their  analysis,  which  makes  them 
believe that the data they provide is not being analysed and falls into a ‘black hole’ (the 
ECR).  Authorities  are  not  sufficiently  disseminating  the  results  of  their  analysis  which 
acts as a disincentive to reporting. 

Competent  authorities  are  required  by  the  Regulation  to  oversee  the  implementation  of 
actions  taken  by  organisations  for  which  they  act  as  competent  authority,  but  it  is  not 
clear  how  they  should  act  in  case  of  a  lack  of  compliance.  Beyond  monitoring  the 
implementation  of actions, there is  no requirement  for competent  authorities to  oversee 
the correct implementation of the Regulation more broadly. 

Efficiency 

During  the  consultation  activities,  it  was  concluded  that  the  Regulation  376/2014  has 
resulted  in  costs  for  both  competent  authorities  and  organisations,  but  based  on 
conservative assumptions, the benefits identified outweigh the costs incurred at least to a 
limited extent. 

Coherence 

Based on the results of the consultation activities, the Regulation 376/2014 did not reveal 
major  issues  in  terms  of  internal  coherence,  except  for  a  number  of  inconsistencies 
related to the use and denomination of certain definitions in the Regulation. 

The analysis of the external coherence of Regulation 376/2014 with EU legislation in the 
EU  air  transport  acquis  revealed  a  few  key  issues.  Stakeholders  criticised  that  the 
existence of different legal acts covering related requirements for the reporting, analysis 
and follow-up on occurrences in civil aviation leads to confusion and complexity. 

EU added value 

During  the  consultation  activities,  it  was  concluded  that  the  Regulation  376/2014 
provides clear EU added value through its contributions to effective safety management 
at EU level compared to what likely could have been achieved without EU intervention 
or at different levels of intervention. However, this added value has not been felt to the 
same  extent  across  Member  States  and  industry  domains  –  depending  on  the  degree  to 
which a mature occurrence reporting culture was already in place. 

56 

 
 
Use of consultation results 

These consultation activities served as field research in order to provide the Commission 
with  the  stakeholders  view  on  the  achievements  of  the  Regulation  376/2014,  and  to 
determine if the aims of the regulation have been accomplished. It collected relevant data 
on  the  stakeholders  opinions,  which  provide  insight  on  the  shortcomings  of  the 
Regulation.  

It  was concluded that the Regulation  requirements  have brought  about  higher rates  and 
more quality on reporting, analysis and follow-up of occurrences in civil aviation, and a 
consequent  improvements  in  aviation  safety.  Nonetheless,  further  improvement  is 
possible.  The  stakeholders  consultation  and  the  OPC  provided  valuable  feedback  from 
the  stakeholders  which  is  helpful  in  finding  where  the  shortcomings  are  and  possible 
solutions  for  how  to  tackle  them.  The  final  results  are  a  rich  source  of  material  to 
understand the achievements of the Regulation 376/2014. 

57 

 
 
 
Annex 3: Methods and analytical models 

As  a  general  principle,  the  methodology  used  during  the  evaluation  respects  the 
principles  of  objectivity,  reliability  and  evidence  based  assessment,  and  complies  with 
the requirements of the Better Regulation Guidelines. Where relevant, tools proposed in 
the Better Regulation "Toolbox" have been taken into account and made use of.  

The  following  five  criteria  have  been  applied  throughout  the  evaluation  of  Regulation 
(EU) No 376/2014 on the investigation and prevention of accidents and incidents in civil 
aviation  and  shall  allow  to  determine  to  what  extent  the  original  objectives  of  this 
Regulation  have  been  met:      effectiveness,  efficiency,  relevance,  coherence,  and  EU 
added value. 

The  following  evaluation  questions  have  been  used  to  carry  out  the  tasks  done  in  the 
context of the evaluation and reflection each criterion used: 

Effectiveness 

1)  To what extent have the operational objectives of the Regulation been achieved? To 

what extent have the specific and general objectives been met? 

2)  What actual observed effects (positive and negative impacts) has the Regulation had? 

To what extent has the Regulation contributed to these effects? 

3)  What were the main drivers and hindrances to the effectiveness of the Regulation? If 
there  are  significant  differences  between  Member  States,  or  between  industry 
domains, please provide examples and explain in which ways and to what extent. 
4)  Which  unexpected  or  unintended  effects  (positive  or  negative)  have  occurred  as  a 
result  of  the  implementation  of  the  Regulation,  if  any  and  what  factors  have 
influenced those achievements? Please specify the effects for each stakeholder. 

5)  Does the Regulation meet the current scope of aviation activity in EU Member States 
and  EASA?  Are  all  organisations  and  persons  adequately  addressed  in  the 
Regulation? 

6)  Does the Regulation adequately and consistently empower Member States and EASA 

to oversee the implementation of the Regulation in organisations? 

Efficiency 

7)  To  what 

extent 

costs, 
enforcement/implementation  costs  and  administrative  costs)  of  the  Regulation 
proportional with the achieved its benefits? Who bears these costs? 

compliance 

regulatory 

costs 

(i.e. 

the 

are 

8)  Has  the  Regulation  resulted  in  unnecessary  regulatory  burdens  or  inefficiencies?  If 
so,  what  are  the  reasons  for  this?  Is  there  a  potential  for  the  reduction  of  the 
regulatory costs for the main actors or for simplification? 

9)  Could  the  same  results have  been  achieved  at  a  lower  cost?  Could  the use  of  other 

policy instruments or mechanisms have provided for better efficiency? 

Relevance 

58 

 
 
10) To  what  extent  are  the  objectives  and  scope  which  were  identified  at  the  time  of 
adoption still adequate in the current context and how do they still correspond to the 
problems  and  needs  of  the  aviation  safety  system  that  is  currently  in  place?  Please 
take into account the technological, market, political, social, environmental and legal 
developments such as digitalisation, big data and artificial intelligence, the growth in 
air traffic and its implications for air safety, the increasing complexity and dynamics 
of the aviation system as well as any other matters that may be relevant. 

11) Which issues that arose after the adoption of the Regulation require further attention 

in view of the objectives pursued?  

Coherence 

12) Are the requirements and provisions set out in the Regulation coherent and consistent 
the  differences,  overlaps  or 

with  one  another?  If  not  entirely,  what  are 
inconsistencies? 

13) To  what  extent  is  the  Regulation  coherent  and  consistent  with  other  EU  regulatory 
instruments under the air internal transport acquis, such as Regulation (EU) 996/2010 
on the investigation and prevention of accidents and incidents in civil aviation, or the 
safety management and occurrence reporting requirements set out in the EASA Basic 
Regulation  and  its  Implementing  Rules?  If  not  entirely,  what  would  be  the 
differences, overlaps or contradictions or inconsistencies? 

14) To  what  extent  is  the  Regulation  coherent  and  consistent  with  the  relevant  EU 
legislation on safety, security, employment and social matters? Does a legal gap exist 
in these cases? 

EU Added Value 

15) What is the added value resulting from EU intervention in regulating the occurrence 
reporting  system?  Could  the  same  results  be  achieved  at  international,  national  or 
regional level without EU intervention? 

16) What  would  be  a  consequence  of  stopping  or  withdrawing  the  existing  EU 

intervention? 

The intervention logic present below is a visual representation of the main objectives and 
the  causal  chain  of  the  intervention  behind  the  Regulation  996/2010.  The  intervention 
logic links the following main elements:  

General  objectives  Specific  Objectives    Operational  Objectives    Inputs   
Outputs Results  Impacts 

59 

 
 
 
 
60 

 
 
 
 
 
 
 
Methodology applied for cost assessment  

The  cost  assessment  focused  on  the  costs  incurred  by  the  industry,  Competent 
Authorities  and  EASA  to  collect,  transfer,  store  and  disseminate  information  on  safety 
related  occurrences.  The 
the 
implementation  of the Regulation, including direct  compliance costs and administrative 
burdens. 

term  “costs”  refers  direct  costs  resulting  from 

The  main  difference  between  most  compliance  costs  and  administrative burdens  is  that 
the  latter  only  occur  due  to  the  legislative  requirements  of  providing  and  collecting 
information  while  compliance  costs  are  incremental  costs  (i.e.  non-business  as  usual) 
costs to the target group of complying with regulation. It could be argued that Regulation 
(EU)  No  376/2014  mainly  introduced  information  requirements  to  Member  States  and 
the  industry  and  thus,  the  costs  incurred  can  be  mainly  classified  as  administrative 
burdens. Following a similar reasoning, the 2012 Impact Assessment for the Regulation 
proposal  did  not  take  into  consideration  compliance  costs  and  considered  all  costs  as 
administrative burdens.   

At the same time, these costs can be further divided into additional categories as captured 
in the table below: 

Definition 

Implementation 
costs 

Cost categories 
Direct labour costs  The costs of staff time devoted to completing the activities required 
to  achieve  regulatory  compliance.  Only  the  costs  of  staff  directly 
involved  in  undertaking  these  activities  should  be  included:  the 
costs of staff supervision/management are included in the overhead 
cost  category.  Direct  labour  costs  include  two  main  elements:  the 
cost of wages paid and non-wage labour costs. 
The  costs  regulated  entities  incur  in  familiarising  themselves  with 
new  or  amended  regulatory  compliance  obligations,  developing 
compliance strategies and allocating responsibilities for completing 
compliance-related  tasks.  In  large  part,  therefore,  they  are  short-
term one-off costs. 
Those costs incurred by businesses whenever they need to purchase 
items  of  capital  equipment  to  comply  with  a  regulation.  This  can 
include both machinery and software.  
The  incremental  costs  incurred  in  changing  some  of  the  material 
inputs used in the production process in order to ensure regulatory 
compliance  (thus,  they  are  sometimes  called  “input  costs”).  They 
are therefore ongoing costs. 

Equipment costs 

Material costs 

Key methodologies for cost assessment 

The  methodology  that  inspired  the  approach  chosen  for  the  cost  assessment  is  the 
Standard Cost  Modelling  (SCM), which allows  to  address the  costs and  burdens of the 
Regulation for both Member States and industry stakeholders. It allows the quantification 
of actual costs (i.e. time costs) resulting from the Regulation requirements (for Member 
States  and  industry  stakeholders  at  MS  level),  to  enable  an  assessment  of  the  extent  to 
which the Regulation is deemed efficient. 

61 

 
The  SCM  in  its  original  format  was  designed  for  the  assessment  of  administrative 
burdens  of  Member  States,  individuals  and  organizations.  However,  the  following 
features were used to reinforce and complete the approach chosen:  

Estimation of benefits derived from a potential reduction on the number of accidents and 
serious incidents 

Administrative Burdens are estimated on the assumption that legislation is complied with 
for 100%, despite the fact that in practice, not all stakeholders fully comply with  all of 
the  requirements.  While  this  assumption  has  been  held  for  the  great  majority  of 
information  obligations,  whenever  there  was  certainty  that  a  specific  information 
requirement is not met it was not considered. 

Input information and extrapolation of costs and benefits 

Eighteen cost-benefit questionnaires provided the core evidence base for cost assessment 
analysis. These questionnaires provided both quantitative and qualitative information to 
shed more light on the actual costs incurred as a result of the Regulation as  well as the 
benefits achieved.  

Cost-benefit  questionnaires  were  sent  to  all  possible  key  stakeholders  and  targeted  to 
national authorities on one hand, and industry stakeholders on the other. The cost-benefit 
questionnaire  contained  questions  pertaining  to  the  efficiency  of  Regulation  376/2014, 
including  specific  questions  regarding  the  costs  incurred  as  a  result  of  the  Regulation, 
and how these costs compare to the benefits achieved.  

On  the  authority  side,  a  total  of  fifteen  (15)  completed  questionnaires  were  returned, 
covering  fourteen  (14)  different  Member  States  and  EASA.  On  the  industry  side, 
however,  only  three  completed  questionnaires  were  returned:  from  a  ground  handling 
service  provider,  an  aircraft  parts  maintenance  organisation  and  an  aviation 
manufacturer.  

Complementing the cost-benefit questionnaires,  a set  of interviews  was  conducted with  
stakeholders  from  various  operational  environments  and  contextual  settings.  For  the 
qualitative  study  questions  these  interviews  provided  strong  evidence  base  for  the 
analysis  of  the  Regulation’s  efficiency.  However,  in  terms  of  quantitative  information, 
data obtained from interviews was too limited to be used for building the basis to draw 
overarching conclusions for all EU Member States. 

Given the challenges in data collection and the sometimes-heterogeneous responses from 
Member  States,  all  this  data  was  standardized,  validated  from  a  data  consistency 
perspective,  thus  deleting  potential  outlier  answers  and  checked  against  desk-research 
gathered data. Data gaps for cost driver data had to be set out by relevant assumptions, 
contrasted through interviews or data available in relevant literature.  

Furthermore, extrapolation of quantitative information (personnel cost, training cost etc.) 
was challenging. Firstly, cost drivers were selected to enable extrapolation from a limited 

62 

 
set of cases to EU level. Number of occurrences integrated into the ECR and the market 
size  (number  of  organizations)  were  used  as  the  main  cost  drivers.  Secondly,  data 
collection  was  not  always  possible  at  the  desired  level  of  granularity  due  to  a 
combination  of  factors  such  as  non-availability  of  data,  low  data  quality,  different 
reporting practices, etc. Consequently, data gaps had to be filled using methods such as 
cross-country extrapolation or contextual indicator comparison.  

63