EUROPEAN COMMISSION Brussels, 10.1.2018 SWD(2018) 3 final COMMISSION STAFF WORKING DOCUMENT MID-TERM EVALUATION

of the Regulation (EU) No 250/2014 of the European Parliament and of the Council of 26 February 2014 establishing a programme to promote activities in the field of the protection of the financial interests of the European Union (Hercule III programme) and repealing Decision No 804/2004/EC Accompanying the document Commission report to the European Parliament and the Council {COM(2018) 3 final}



Table of contents 1.Introduction 2.Background to the intervention 3.Implementation / state of Play 4.Method 5.Analysis and answers to the evaluation questions 6.Conclusions Annex 1: Procedural information Annex 2: Synopsis report of stakeholder consultation in the context of the Study Annex 3: Methods and analytical models 1.Introduction Purpose and scope This report is a Commission Staff Working document (SWD) that accompanies the mid-term evaluation of the Hercule III programme. The evaluation of the Commission was supported by an external study. The mid-term evaluation is required by Article 13, paragraph 2(a) of the regulation that establishes the Hercule III programme (‘the Regulation') which provides that “The Commission shall carry out a thorough evaluation of the programme and present to the European Parliament and to the Council by 31 December 2017, an independent mid-term evaluation report on the achievement of the objectives of all the actions, results and impacts, the effectiveness and efficiency of the use of resources and its added value to the Union, in view of a decision on the renewal, modification or suspension of the actions; the mid-term evaluation report shall additionally address the scope for simplification, internal and external coherence of the programme, the continued relevance of all objectives of the programme, as well as the contribution of the actions to the Union’s priorities of smart, sustainable and inclusive growth; it shall also take into account evaluation results on the achievements of the objectives of the Hercule II programme". The scope of the mid-term evaluation, as set out in the evaluation roadmap, is represented by all interventions supported under the Hercule III programme up to June 2017 as well as the preparatory and implementing activities undertaken by the stakeholders for these interventions. It also covers those applications submitted by potential beneficiaries which remained unsuccessful. The evaluation assesses six evaluation criteria – relevance of all the objectives of the programme; internal and external coherence of the programme; effectiveness in attaining it objectives and expected results; the efficiency of the use of resources; the programme's added value to the EU; and sustainability i.e. the effects of Hercule III actions after their completion, as well as the contribution of the actions to the Union’s priorities of smart, sustainable and inclusive growth. The time period covered by the evaluation is the first half of the seven year period that Hercule III programme lasts for, starting on 1 January 2014 (when the Regulation entered into force) until June 2017. 2.Background to the intervention

2.1 Description of the intervention The Hercule programme is an instrument specifically dedicated to supporting the protection of the financial interests of the European Union by fighting irregularities, fraud and corruption. It is administered by the European Anti-Fraud Office (OLAF) within the Commission. The first Hercule programme was established in 2004 and was succeeded by Hercule II (2007-2013). The current Hercule III programme (2014-2020) was adopted in 2014 on the basis of a proposal tabled by the Commission in 2011. The regular reporting by Member States on irregularities and suspected fraud cases, detected during the implementation of the budget on the basis of Article 325 TFEU, shows that the financial impact throughout the years remains at levels that require the Union to keep developing activities in order to strengthen the protection of the Union's financial interests. The general objective of the Hercule III programme is 'to protect the financial interests of the Union thus enhancing the competitiveness of the Union's economy and ensuring the protection of taxpayers' money' , whereas the programme's specific objective is 'to prevent and combat fraud, corruption and any other illegal activities affecting the financial interests of the Union' . In addition, the programme lists five operational objectives centred around: ·strengthening cross-border and multi-disciplinary cooperation; ·facilitating the exchange of information, experiences and best practices; ·providing technical and operational support to national authorities; ·fighting specifically organised fraud, especially cigarette smuggling and counterfeiting; ·promoting comparative law analysis . The Commission manages a budgetary envelope of EUR 104.9 million over the period 2014-2020 for the Hercule III programme, with an annual average of around EUR 15 million. The eligible beneficiaries of the programme are national or regional administrations of Member States and research bodies, as well as educational institutes and non-profit-making entities that have been established and operate in a Member State for at least a year. The programme provides financial support for three types of actions : 1. "Technical Assistance" (at least 70% of the programme's budget): support aimed at providing specific knowledge, equipment and information technology tools to national authorities as well as providing specific databases and IT tools facilitating data access and analysis. 2. "Training" (maximum 25% of the programme's budget): support to the organisation of targeted specialised training, risk analysis workshops, conferences and legal studies focused on the protection of the Union's financial interests. 3. "Any other action" (not more than 5% of the budget). The programme is implemented by means of grants following calls for proposals, public procurement contracts following calls for tender and administrative arrangements concluded with the Joint Research Centre (JRC) for the development of specific tools. The programme also provides for the reimbursement of costs incurred by representatives from a limited number of non-EU countries participating in training actions such as conferences or seminars. The co-financing rate for grants does in principle not exceed 80% of eligible costs. 2.2 Intervention logic Annual Work Programmes (AWP) ensure that the general, specific and operational objectives of the Hercule III programme are implemented in a consistent manner. They outline the expected results, the methods of implementation and their total amount. With regard to grants, the AWPs include the actions financed, the selection and award criteria and the maximum co-financing rate. AWPs are adopted by a Financing Decision of the Commission. The preparation of an AWP involves a consultation of the main stakeholders within OLAF to ensure that new requirements of OLAF's partners are taken into account. A draft version of the AWP is subsequently sent to the members of the COCOLAF and the AFCOS , for consultation and additional observations. On the basis of these consultations, modifications may be made. The next step in the adoption of the AWP is an internal consultation within the Commission in order to ensure that the proposed activities do not overlap with activities funded under other Union programmes, such as Customs 2020 or the Internal Security Fund (ISF) activities. The Commission starts the implementation by preparing the call for proposals (e.g. for technical assistance, conferences, legal training and studies) once the Financing Decision is adopted and the appropriate financial resources are made available. Subsequently, the submitted proposals are evaluated according to the selection and award criteria. The successful proposals receive appropriate financial support for the relevant actions.

OLAF submits an annual Implementation Report to the Parliament and the Council with the main results achieved and the relevant insights in terms of consistency and complementarity with other EU programmes, as required by Article 13(1). This report is annexed as a Commission Staff Working Document to the annual Article 325 report. The Committee on Budgetary Control (CONT) of the European Parliament drafts its opinion on the Article 325 report by the end of the year and the Commission takes account of these observations in the elaboration of the next AWP. Within OLAF, the programme is managed by a small sector and staff in the budget Unit (8 FTE in total). Figure 1: Intervention logic: 2.3 Baseline and points of comparison Hercule III was based on an Impact Assessment (IA) that identified five key drivers linked to nine problems that should be addressed under the programme. The problems and the drivers are interconnected as can be seen in Figure 2. Figure 2: Hercule III drivers and problems The IA also found that the positive impact of the Hercule II programme was not fully reached at the time, e.g. because some Member States were not able to co-finance Hercule II projects, especially where procurement of technical equipment was concerned. Notably on the eastern and southern fringes of the EU, the areas affected were also those potential or actual weak points where there is the greatest common interest in strengthening the control of the EU’s borders against smuggling. In response, an important change introduced by the Hercule III programme was to increase the maximum rate of funding in co-financed activities from 50% to 80% . The Commission’s evaluation of the Hercule II programme concluded that the programme achieved its objectives. However, the evaluation report also identified some areas for improvement: more staff exchanges, access to databases, information exchange in the procurement area, beneficiaries' reporting and creating synergies between Commission services. Some of these have already been taken on board or addressed in Hercule III. For example, cross-border cooperation, including staff exchanges are encouraged in calls for proposals for Hercule III subsidies. Regarding synergies between Commission services please see also page 19. 3.Implementation / state of Play 3.1 Administrative implementation As detailed in section 2.1, Hercule III actions are divided along the following categories: ·provision of technical assistance for the competent authorities of the Member States; ·organisation of targeted specialised trainings, risk analysis training workshops, and conferences; ·as well as an open-ended other actions cluster not falling into the previous two categories. The actions are funded under grant agreements or procurement. Over the period 2014 to 2016, the overall committed budget amounted to EUR 41 million, of which 75% was distributed via grant agreements. 391 applications were submitted over that period and 125 grants were awarded, while 1700 participants took annually part, on average, in events funded under Hercule III. As explained in section 2.2, the programme is implemented on the basis of AWP that are adopted by a Financing Decision of the Commission . The preparation of an AWP for a given year (n) starts in the autumn of the preceding year (n-1) by an internal consultation within OLAF as well as consulting the main stakeholders. Following these consultations the AWP is adopted. Once the Financing Decision is adopted and the appropriate financial resources are made available, the Commission starts the implementation activities as announced in the AWP. During the spring of the following year (n+1), the Commission drafts the annual overview mentioned in Section 2.2 to be annexed to the Article 325 report. The European Parliament adopts its resolution in spring of the year after (n+2). Therefore, a full cycle for the preparation of, implementation and reporting on an AWP lasts almost 2.5 years.

3.2 Monitoring The Regulation establishing Hercule III provides that the achievement of the specific objective of the programme , "to prevent and combat fraud, corruption and any other illegal activities affecting the financial interests of the Union" has to be measured by four key performance indicators (KPI). Beneficiaries of the actions have to report on the results achieved. The modalities of this reporting differ according to the type of action (technical assistance and training). In the case of contracts for the purchase of access to databases, the control over the results of the actions takes a different form: the Commission receives detailed information on the use of the databases by the different users. The findings based on the beneficiaries' reports in both the technical assistance and the training reports are included in the annual overviews on the implementation of the programme and are used as input for the estimation of the KPI. 4.Method 4.1. Procedural aspects In line with Article 13(2) of Regulation 250/2014 which mandates an independent mid-term evaluation, OLAF used an outside contractor to perform a study in support of the Commission evaluation (the ‘external study’) . The external study was carried out between February and June 2017. This study is a key contribution to this SWD, in combination with other sources (they are referenced as appropriate in the relevant sections). The main elements used for the external study were desk research and targeted consultations of stakeholders through interviews and surveys. Further information on the external study methodology can be found in Annex 3. The external study expresses the views of the external contractor and, unless stated otherwise in this staff working document, the Commission neither endorses these findings nor can it be held responsible for its content. An Inter-Service Steering Group (ISSG) of Commission services was set up to follow the evaluation process. The ISSG reviewed the roadmap, the terms of reference for the external contract, the inception and the end-of–fieldwork reports. The ISSG was also closely involved in the preparation of this SWD. 4.2. Data collection This SWD answers the questions identified in the evaluation roadmap. These questions were further detailed in the terms of reference for the external contract and operationalised by the evaluator using judgement criteria and indicators. The main data sources it relied upon to reply to the evaluation questions are: Primary data; data that were collected specifically for the purpose of the evaluation via: ·interviews with Commission staff; national institutions active in fraud prevention and the protection of the EU's financial interests, as well as beneficiaries of actions funded under the programme; ·online surveys with beneficiaries of actions funded during the first two years of the programme; unsuccessful applicants; participants in events funded under the programme, as well as users of services procured under the programme, in particular database users. Secondary data; data gathered by examining: ·public sources, such as the text of the Regulation and supporting documents (the Commission proposal as adopted in 2011, together with the Impact Assessment); Hercule III AWPs; Hercule III Annual Implementation Reports; Article 325 Reports; the final evaluation report and interim reports of the Hercule II programme, and legal acts in relation to other EU programmes, such as Customs 2020, Fiscalis 2020 or the Internal Security Fund; ·documentary evidence on submitted applications for grants (application forms) and awarded contracts and grants (Final Technical Reports, Final Financial Reports and Final Implementation Reports). The evaluator consulted 574 stakeholders of the programme. In-depth interviews were held with 49 civil servants (16 staff members of the Commission and 33 from national administrations). In addition, 56 beneficiaries of the programme were interviewed (71% of all beneficiaries), 67 unsuccessful applicants (25% of all rejected applicants), 321 participants in events (27% of participants to whom a questionnaire was sent) and 112 users of databases and services (31% of users to whom a questionnaire was sent). Quantitative information on the implementation of the AWPs, such as the budget and the number of grants/contracts, broken down by type of action and Member State were collected from the overviews on the implementation of the AWPs. 4.3. Limitations The life cycle of a programme like Hercule III is such that it takes some time, at the beginning of the programme, to produce results. This generates some limitation to the input available for a mid-term evaluation. The data collection and the interviews were mostly carried out in 2017. By that time, only partial results of actions funded under the new programme were available as these actions were funded in the first two years of the programme (2014 and 2015). This limitation was particularly felt for Technical Assistance (TA) actions. 13 out of 25 TA actions surveyed for the evaluation were still ongoing at the moment of gathering data. In addition, for the 13 completed actions, most beneficiaries reported having only recently received the equipment, thus putting them in no position to provide meaningful feedback on the outcome and impact of the actions. In other words, at the moment of performing the evaluation, the independent evaluator had access only to a limited number of reports with information on results and the contribution the results made to the achievement of the programme's general, specific and operational objectives.

The timing for the external study therefore resulted in some limitations to the tools that could be used and led the evaluator to rely to a significant extent on the opinions expressed by beneficiaries, applicants, participants, and Commission staff. However, the interviewees arguably account for a representative sample of the programme's stakeholders, as described above in section 4.2. It cannot be excluded that there is a bias in the opinions expressed by the stakeholders, be it positive for those whose application was successful and carried out some Hercule-funded actions, or negative for those stakeholders whose application was rejected. Additionally, the evaluation of some training activities under grants, such as conferences or training sessions, could only be carried out via the grant beneficiary who organised the event and who had access to the personal data of the participants required for the transmission of the survey questionnaire(s). As not all these grant beneficiaries kept records of the personal data, it was not possible to contact all the participants to training and conferences under grants. However, this had only a minor impact on the overall response rates since it related to events whose aggregated budget amounted to less than 10% of the financial support made available for training actions. The external study found that beneficiaries face difficulties providing the data to measure the KPI a), b) and c) of Article 4 of the Regulation. The collection of data to measure these indicators is reportedly the most burdensome aspect of the reporting phase. The reports to feed the KPIs therefore provided only a relatively limited input into the mid-term evaluation of the programme. The definition of the KPIs in the Regulation allows only for results that, in practice, are too general and would require to be further broken down to be fully used. This should however be addressed by the successor programme in the next MFF period. There is no indication, however, that the above limitations would invalidate the result of the external study. 5.Analysis and answers to the evaluation questions 5.1 Relevance This section focuses on the relevance of the programme for the protection of the financial interests of the Union. The questions are based on the Better Regulation Guidelines examples and were adapted for the purposes of the evaluation by the ISSG. EQ1: To what extent have the specific and operational objectives of the Hercule III programme proven to be relevant for the general objective of the protection of the financial interests of the EU? On the basis of the external study, it appears that the achievements of the specific and operational objectives of the programme were relevant to achieving its general objective. The latest Article 325 report shows that the estimated financial impact of irregularities and fraud is hardly declining over the last years (see Figure 3). The Article 325 report reports on all (fraudulent) irregularities detected by the Member States. This decline is not exclusively linked to the Hercule programmes. The fluctuations in reported irregularities and related financial impact can be also due to the level of reporting and (lack of) measures taken to address the irregularities. Hercule III helps to boost the operational and administrative capacity of Member States’ customs and other authorities. In that regard substantial successes were reported in relation to smuggled and counterfeit cigarettes and tobacco: seizures were made with the help of equipment and training funded under the programme.



Figure 3: Number of reported irregularities and amount: expenditures Source: External study, contractor's elaboration on Article 325 reports. The policy problem in connection with fraud and irregularities to the EU's financial interests that was prevailing at the time of the preparation of the Regulation, as reflected in the IA, is still prominent. Therefore, considering that the Regulation deviates from the IA findings only on procedural matters , the specific and operational objectives of the programme could be expected to be still relevant for the general objective of the protection of the financial interests of the EU. This is confirmed by the stakeholders’ perception as presented in the bar chart below. The same is corroborated by the conclusions in final technical reports of actions funded under grant agreements, where all beneficiaries, who submitted the final technical reports, reported on how the results contributed to the specific and general objectives (see the answers to the next evaluation question). Figure 4: To what extent does each operational objective contribute to the PFI and, more specifically, to preventing and combating fraud, corruption and other illegal activities against EU financial interests, including cigarette smuggling and counterfeiting? (Average evaluation; number of respondents)

Scale: (1) not at all; (2) to a limited extent; (3) to some extent; (4) to a high extent; or (5) to the fullest extent. Respondents who replied “Do not know/No opinion” or “Not Applicable” are not included. Source: On-line survey and interviews with institutions, beneficiaries and unsuccessful applicants. External study. This bar chart also shows that the stakeholders perceived that the operational objective "promoting comparative law analysis and supporting academic analysis of strategic legal issues" was relevant, yet less than the other operational objectives. It should be observed that this action is very specific to a small group of beneficiaries (research and educational institutes and not for profit organisations), which explains why most other stakeholders have seen this operational objective as less relevant. EQ2: To what extent have the activities of the Hercule III programme proven to be relevant for achieving its operational and specific objectives? The activities of Hercule III have been relevant to the achievement of the programme’s operational and specific objectives. During the course of the external study, the stakeholders largely agreed to this as also corroborated by findings in the final technical reports submitted by grant beneficiaries. One important explanation for this result is that this parameter is assessed during the evaluation of applications for grant. Applicants must demonstrate to the Commission how the action is expected to contribute to the programme's general, specific and operational objectives. Nonetheless, two actions were largely perceived as less relevant to the achievement of the general objective. These are: "purchase of services to store and destroy seized cigarettes and other counterfeit goods" and "purchase of services to carry out chemical analysis of samples from tobacco and cigarette seizures." The Commission received only one application in 2014 and 2015 for grants for the destruction of seizures. Regarding the latter objective, the Commission asked, under the programme, the Joint Research Centre (JRC) to set up laboratory facilities for the chemical analysis of samples from seized tobacco and cigarettes. The overall use of this facility has so far been low, possibly because it is not yet sufficiently known amongst the operational staff responsible for processing the seizures. To address this issue, the Commission is organising in 2017-2018 training sessions in the Member States for operational staff, financed under Hercule III. For the same reasons identified under EQ1 above some stakeholders did not find of particular relevance the actions geared towards building and disseminating scientific and legal knowledge in the field of the protection of the financial interests of the EU, as this action is specific to the actual small group of beneficiaries. While emphasising the need to streamline the number of eligible actions to avoid dispersion of funds on too many activities, some stakeholders, suggested additional activities that would contribute to the relevance of the programme. These include trainings on big data analysis, activities in connection with detecting and analysing digital evidence, research in the field of digital forensics, multidisciplinary research into determinants of fraud affecting the EU financial interests, and risk analysis and profiling of passengers to enhance detection of smuggled goods. The external study also found that, in view of the parallel increase of irregularities on the expenditure side and the decrease on the income side , more emphasis could be put on activities in connection with the expenditure side. 5.2 Coherence This section examines to what extent the activities within the programme are coherent (internal coherence) and to what extent the programme's activities are coherent with activities undertaken by other EU programmes (external coherence) in order to detect any overlaps and to identify synergies. EQ3: What are the synergies between and within the different types of actions under the programme and with other EU supported measures, programmes and actions, such as Customs 2020 or Fiscalis 2020? The internal coherence of the programme is ensured by (i) the allocation of the budget for the period 2014-2020, (ii) the control performed by the Commission on possible overlaps and repetitions on the basis of information provided in the applications, (iii) the Regulation’s requirement that applications for equipment be accompanied by other activities, e.g., trainings, and (iv) discussions and coordination between Member States on the occasion of high level conferences financed by Hercule ensuring some actions which are particularly relevant to the protection of the financial interests of the EU (PFI) are performed irrespective of grant applications. I.There is an internal coherence element in the programme between the different categories of actions provided for in the Regulation (see Section 2.3). The financial limits that are set for each category (at least 70% of the budget for technical assistance actions, maximum 25% for training actions, both anti-fraud and legal training actions, and up to 5% for other actions) are a guarantee that this coherence will be respected, by balancing the actions where there is more demand and other actions that are particularly relevant to the PFI (e.g., conferences on specific PFI topics or forensics training). The financial implementation of the AWPs shows that this indicative allocation has been complied with . II.Applicants for grants have to indicate in their applications whether or not similar requests have been made in the past and during the evaluation of the grant applications, allowing the Commission to closely monitor that actions do not overlap: conferences proposed in training applications should not coincide or address overlapping topics, whereas requests for funding for expensive technical equipment are checked to ensure that similar equipment is not already in use by the same services or in the same geographical area. The accepted applications for grants do indeed comply with this obligation. III.The Regulation provides in Article 10(3) that the purchase of equipment shall not be the sole component of a grant agreement. This provision is implemented by requesting applicants to ensure that technical assistance actions contain training for staff to operate the equipment or to include activities enabling the exchange of relevant information and best practices with other law enforcement services or peers in neighbouring Member States.

IV.High-level conferences arranged by OLAF ensure that some key topics for the PFI (e.g. fight against cigarette smuggling, communication activities on fraud prevention, fraud in structural funds, cooperation in anti-fraud activities) are discussed and coordinated at the EU level by relevant national authorities. This allows a coherent approach to the PFI in general and the programme in particular across Member States, irrespective of the specific actions for which national authorities decide to apply for. The contractor carried out an analysis of the regulations establishing the programme and Customs 2020, Fiscalis 2020, ISF Police and the Justice programme and the most recent AWPs. Although each programme pursues very different general objectives, interactions can be detected between specific and operational objectives. However as regards external coherence, the external study found that the overlap between the Hercule programme and other mentioned programmes is very limited, essentially thanks to the Hercule programme’s requirements and the Commission internal consultation mechanisms. As explained above, the selection process allows detecting overlaps, on the basis that applicants have to indicate whether or not they received grants for similar actions from other EU programmes. Some equipment procured under Hercule III can be used for purposes other than the protection of the EU financial interests. A scanner installed at a border will, for instance, not only detect smuggled cigarettes but also illicit products such as drugs or trafficked persons as these might be transported in vehicles that are checked by the scanner procured under Hercule III. This would not however weaken the coherence of the programme, and could even be seen as complementing the specific objective of the ISF programme to achieve "a uniform and high level of control of the external borders" and thus creating synergies. The Commission internal coordination process allows preventing overlaps at several levels. As testified during the interviews with EU staff, few overlaps exist between EU programmes, such as Customs/Fiscalis 2020, or the ISF (see the bar chart below, Figure 5). The Commission inter-service consultation process to which draft AWPs are submitted ensures complementarity of the programmes. Additionally, during the preparation of the AWP of Hercule, Commission services responsible for the management of Customs/Fiscalis 2020 and the ISF are closely involved in order to prevent any overlaps from occurring and to ensure that existing synergies can be strengthened or that new synergies can be identified. The AWP is also submitted to different working groups of Member States' representatives to collect information and feed-back to ensure that the proposed AWP addresses existing or new needs and to avoid overlaps with other EU and national programmes. Lastly, at the stage of launching calls for proposal, representatives of other Commission services are closely involved in the elaboration of the call documents as well as in the examination of applications in order to ensure, as far as possible, that overlaps do not occur and that synergies with other EU programmes are strengthened. This is supported by the views of the participants to the online survey who mostly replied that they, for the most part, were unaware of other programmes and that those who were, indeed detected synergies, and only limited overlaps, between Hercule and other EU programmes (Figure 5). Although more synergies between Commission services could be achieved, this needs to be carefully balanced against inflating workload, creating extra layers and generating delays. If fighting against corruption and VAT fraud would become more central in Hercule III, the external study advises to maximise synergies with other programmes managed by DG TAXUD and DG HOME, which touch upon these two issues that can harm the EU's financial interests. Figure 5: Replies to the question: "To what extent does the programme have synergies and/or overlaps with other EU-funded programmes (you are familiar with)?" Scale: (1) not at all; (2) to a limited extent; (3) to some extent; (4) to a high extent; or (5) to the fullest extent. Respondents who replied “Do not know/No opinion” or “Not Applicable” are not included. Source: On-line survey and interviews with institutions, beneficiaries and unsuccessful applicants. External study. 5.3 Effectiveness This section focuses on the extent to which the programme has attained its objectives and its expected results. EQ4: To what extent have the overall intervention logic/strategy of the programme and the actions contributed to the achievement of the objectives of the Hercule III programme? Over the period covered by the mid-term evaluation, the programme financed actions which, because of their nature or objectives clearly contributed to achieving the objectives of the programme (e.g. providing co-financing for a scanner purchase in order to identify smuggled tobacco products). In terms of measuring this contribution, this is indeed corroborated by the external study as far as anti-fraud and legal training actions are concerned, as well as by the analysis of the final reports for grants. Because of the data limitations highlighted in Section 4.3, the external study is inconclusive on this point as far as technical assistance is concerned. Under training actions, 65 events were organised for training, conferences or seminars. The events covered a wide range of different topics, such as training for customs staff, digital forensic training and training on fraud prevention or general issues in relation to the protection of the financial interests of the Union. On the basis of the final reports and in particular of the results of the user satisfaction survey, the external study (see Figure 6 below) showed that the expected outcomes, such as networking possibilities or the acquisition of new skills and competences have been largely achieved. However, only a few actions involved staff exchanges between national administrations, although the evaluation of the Hercule II programme recommended that more personal contacts between administrations, e.g. through staff exchanges, would improve the impact of the programme. Figure 6: To what extent did the event funded by the programme contribute to the achievement of the following outcomes? (Average evaluation; number of respondents)

Scale: (1) not at all; (2) to a limited extent; (3) to some extent; (4) to a high extent; or (5) to the fullest extent. Note: Number of informed respondents in brackets. Differences in number of respondents across dimensions are due to respondents selecting the “Do not know/No opinion” or “Not Applicable” answer. Source: Online survey with participants in events. External study. Technical assistance grants were awarded to applications covering four different topics: investigation tools (cameras, monitoring devices for vehicles or mobile communications, adapted vehicles, etc.), detection equipment (scanners, sniffer dogs), Automated Number Plate Recognition Systems (ANPRS) and the destruction of seizures. Most of the budget was spent on investigation tools (54%), whereas 28% was spent on detection tools. ANPRS actions received 15% of the budget and the destruction of seizures (one grant only) amounted to less than 2%. Taken account of the relatively recent signature of the first grant agreements at the beginning of 2015, only few results were available at the start of the evaluation . Technical assistance actions also include procured access to databases for use by law enforcement agencies during PFI related operations, laboratory facilities for the examination of samples from tobacco seizures as well as the development of specific IT-tools for data-mining and data analysis purposes. The latter provides substantial input to Member States' customs for risk analyses to identify shipments at risk. The external study found that the users of these services considered them as effective, as shown in the bar chart below. The examination of the available 13 Final Technical Reports on technical assistance actions did not provide sufficient input for the outcome indicators, at the level of results. As indicated earlier, this is due to more than half of sampled actions being still ongoing during the evaluation period while many others were concluded very recently, thus unable to provide more outcome indicators. Yet, they provided anecdotal examples of achievement. For instance, the deployment of a wide range of investigation and detection equipment enabled the seizure of more than 21 tonnes of cigarettes during more than 150 different operations. In addition, drugs, weapons and alcohol were seized as well. The use of the ANPRS led to the discovery of 10 tonnes of drugs. While such equipment contributes to successful operations, it must be borne in mind that it is used in combination with other tools and does not allow attributing all the results only to the programme. During the mid-term evaluation, the Commission continued receiving final technical reports and final implementation reports related to actions funded under Hercule II, which are comparable to the reports required by Hercule III. These reports also contained interesting information on the results achieved with specific technical equipment. The aggregated prevented losses to the national and Union budget in these reports represent an amount that is substantially higher than the Hercule II budget. It is likely to be even higher because the beneficiaries are not required to report on a systematic basis. The reporting period is mostly limited to a period covering 12-15 months and results beyond that period are not reported to the Commission. Regarding the evaluation of the perception of the stakeholders, the external study found that the programmes’ actions and their results indeed contribute to the programme's objectives. Overall, across all respondents’ groups, even the action that is seen as contributing the least to the programmes’ objectives, i.e. the promotion of comparative law analysis and the support to academic analysis of strategic legal issues, still scores relatively well (between 3 and 4 on a scale from 1 to 5 - Figure 7). However, the external study did also note that more should be done on the expenditure side of the EU budget, which is an area currently receiving fewer grants. Figure 7: Replies to the question: "To what extent did the action funded by the programme contribute to the achievement of the following objectives?" Scale: (1) not at all; (2) to a limited extent; (3) to some extent; (4) to a high extent; or (5) to the fullest extent. Respondents who replied “Do not know/No opinion” or “Not Applicable” are not included. Source: On-line survey and interviews with institutions, beneficiaries and unsuccessful applicants. External study. EQ5: To what extent have these objectives been achieved through the Hercule III programme’s interventions and to what extent have other factors played a role? Fraud, corruption and irregularities to the EU budget unfold in a multi-factor environment where the EU and national regulatory frameworks, political will, judicial capacity, and criminal behaviours inter-play, among others. The achievement of the objectives of the programme is therefore heavily influenced by these factors. As an illustration, the positive impact of the procurement of scanners to identify smuggled tobacco products will be usually stronger in a country where the judiciary fight against the criminal groups organising such smuggling is a priority and vice versa. However, it is impossible to measure the interplay between these other factors and Hercule's interventions in the achievement of the objectives of Hercule III. The programme's stakeholders have identified a number of specific factors as influencing the achievement of the programme's objectives. The most important factor hindering the achievement of the objectives, with no significant differences in ranking among all stakeholder categories, is the ability of fraudsters to “adapt quickly to new circumstances at the EU level, operate cross-border and exploit the weakest points in the external borders”. The other identified factors are: -“the large number and varied nature of competent national and regional authorities” leading to an uneven PFI; -“differences in incentives and capacities between MS”; -“divergent application of criminal law and penal sanctions” across EU MS; -“pressure on public finance” requiring more cost-efficient solutions. A few stakeholders, not statistically representative, have identified some other obstacles, including: weak political willingness to protect the EU financial interests; rising scepticism vis-à-vis EU institutions; poor knowledge about fraud affecting the EU financial interests; limited financial envelope of the programme and limited financial resources available at MS level to protect the EU financial interests; and lack of skilled personnel and modern technical tools to combat and prevent fraud. Consideration might be given to the impact of these factors during the final evaluation of the Hercule III programme. 5.4 Efficiency This section examines whether the results of the actions have been achieved at a reasonable cost.

EQ6: To what extent have the desired effects been achieved at reasonable costs? Cost-effectiveness is inherent to the implementation of technical assistance actions: they mostly consist of the acquisition of equipment that has to be purchased by public procurement procedures where the "best value for money" principle is used to select equipment. Nevertheless, due to the data limitation (see section 4.3), further quantification was not possible. The external study found that the costs per participant for trainings are broadly aligned with those recorded by other EU-funded programmes. The average beneficiary of the programme spent approximately 18 person-days to complete the application phase, 4.4 person-days for the grant-awarding phase and 13 person-days for the reporting phase. The external study found that this is in line with other EU funded programmes. This translates into the average applicant spending approximately EUR 4 500 for transaction costs, i.e.: all costs incurred for the acquisition, monitoring and reporting (for more detail per type of action, see Figure 8 below). The external study also established the average cost per participant to anti-fraud training at EUR 467 and that to legal training at EUR 694. This cost is broadly in line with other EU-funded programmes. No such analysis could be done for technical assistance because of limited availability of data. The external study demonstrated that regulatory costs linked to the application phase yield “value for money”. The beneficiaries of anti-fraud training activities were expected to get about EUR 12 for each euro spent in preparing a proposal (the expected return on investment). For legal training proposals this amount was around EUR 7 for each euro spent. While beneficiaries of technical assistance grants were expected to get almost EUR 67, as the average grant requested was much greater (Figure 8).



Figure 8: Costs/benefits analysis Administrations / Research and educational institutes and non-profit-making entities Regulatory costs Benefits / Return on investment for each euro invested in the application process if grant is awarded Anti-fraud training activities EUR 2 195 - Application phase EUR 467 - Grant award and contracting phase EUR 1 830 - Reporting phase EUR 4 492 - Total EUR 12.4 Legal training activities EUR 3 235 - Application phase EUR 886 - Grant award and contracting phase EUR 2 900 - Reporting phase EUR 7 021- Total EUR 7.3 Technical assistance EUR 1 839 - Application phase EUR 486 - Grant award and contracting phase EUR 1 312 - Reporting phase EUR 3 637 - Total EUR 66.8 Source: External study EQ7: Could the same effects have been achieved with lower costs if procedures had been simpler, involving less administrative burden and/or efficient implementation mechanisms had been applied? On the basis of the external study’s findings, the application process does not appear to be too cumbersome for the majority of stakeholders (approximately 70%). 30% of applicants expressed reservations on the application process and 33% of beneficiaries found the reporting process to be cumbersome. Among those, some made suggestions for improvement which, according to the external study, would generate some substantial savings (see footnotes 56, 57 and 58). If their suggestions were accepted, the external study estimates that savings of about EUR 2 000 per action would materialise (see Figure 9 below). Some regulatory obligations are, however, inherent to EU procurement and cannot be simplified given the current legal framework. Therefore, only some of the suggestions of the applicants could be implemented. Almost all respondents who considered the grant awarding and contracting phase cumbersome suggested what could be done for making the application phases less cumbersome. However, each suggestion was provided by a limited number of consulted stakeholders. Figure 9: Potential savings Administrations / Research and educational institutes and non-profit-making entities Potential Savings Application EUR 1 100 Grant award and contracting EUR 350 Reporting EUR 700 Total EUR 2 150 Source: External study In 2017, the Commission services have started using an electronic management system for the submission, processing and management of grant applications under the programme (‘e-grant system’). The objective of this system is to streamline the process and alleviate the burden of the applicants. The improvements brought about by the e-grant system will address a large number of these suggestions, for instance by digitising the whole process and introducing the use of digital signatures. The e-grant system is expected to noticeably alleviate the administrative burden and related time and resources spent on the applications in particular.



5.5 EU added value This section covers the programme's added value at the level of the European Union, compared with no intervention at EU level in this field. EQ8: Has the programme allowed delivering results that could not, or to a lesser extent, be achieved by interventions undertaken at national or regional level?

The external study found that the stakeholders almost unanimously recognised the added value of the programme (see in particular Figure 10), in all its components, and its capacity to generate unique results and benefits that could not be generated at national or regional levels. The primary explanation is the lack of budget to fund similar actions at national or regional levels. Absent the programme, the actions provided for by Hercule would probably not exist. Additionally, from the point of view of the Commission and Member States’ services, Hercule III funded interventions generate better results and more benefits than comparable national/regional interventions would. Figure 10: Would the results and benefits of actions funded by the programme also be generated in the absence of the programme? (% of respondents) Source: Online survey and interviews with institutions and beneficiaries. External study. Importantly also, the programme contributes to a better protection of EU external borders and to strengthening long-lasting cross-border cooperation, cross-border exchange of information and cross-border exchange of best practices (see Figure 11 below). This confirms the added-value of a programme at the level of the EU. Some beneficiaries, although not a statistically significant number of them, suggested that there is more potential in centralised (common) purchases of technical equipment as this would reduce costs for national authorities and would ensure cross-border use of that equipment.

Figure 11: To what extent did the actions funded by the programme lead to cross-border cooperation, exchange of information and/or best practices and/or common use of equipment? (Average evaluation; number of respondents) Scale: (1) not at all; (2) to a limited extent; (3) to some extent; (4) to a high extent; or (5) to the fullest extent. Source: Online survey and interviews with institutions and beneficiaries. External study. The majority of the funded actions are expected to generate benefits in terms of exchange of information and good practices between Member States as well as cross-border cooperation, although to a lesser extent in the case of technical assistance actions (see Figure 12). Figure 12: Percentage of actions enabling cross-border cooperation Anti-fraud training Legal training Technical assistance Total Exchange of information and best practices 100% 88% 45% 71% Cooperation with other MS 76% 94% 24% 54% Note: For technical assistance, results are based on information sourced from application forms. For legal training, 37.5% of the results are based on information sourced from application forms. For anti-fraud training, 4% of the results are based on information sourced from application forms. Source: External evaluator’s elaboration on application forms and available reporting documents. Overall, these results are coherent with the IA which had identified that there were inadequate national solutions to a number of problems within the scope of the programme (e.g. insufficient information sharing between authorities or inadequate sharing of best practices). EQ9: Does the intervention at the EU level provide added value in terms of the efficient use of financial resources as compared to a possible intervention at national level? The external study found that the financial resources are more efficiently used through the programme than national interventions would, for instance by generating cost savings or securing better commercial terms (e.g. lower searching costs to find the most adequate equipment and lower purchasing costs thanks to volume discount). Furthermore, in their feedback 67% of beneficiaries and 85% of relevant stakeholders from the Commission and national services mentioned that Hercule III enables more efficient use of resources, as well as improved planning, monitoring and quality standards due to transparency and reporting requirements. 5.6 Sustainability

This section examines the likelihood that the programme's actions will continue generating positive effects after their completion. EQ10: To what extent are the (positive) effects of the intervention likely to last after the intervention has ended? Overall, it appears that the effects of the programme last after the intervention has ended. The external study found that the beneficiaries of the programme are of the view that the Hercule III funded actions will, or are likely to, continue producing benefits after their completion. Equipment purchased under technical assistance are used for several years after their acquisition, while training actions contribute to acquiring skills and building networks likely to be used over a longer period of time. The beneficiaries of training actions are asked whether they will keep on performing the action funded by the programme after the EU’s financial support has come to an end for the purposes of the Final Technical Report. The external study analysed 30 out of 41 of available reporting documents for training actions funded by grants in 2014 and 2016. The analysis showed that more than 50% of the reviewed training actions will not continue in the absence of EU funding. However, this seems to address the sustainability of the action itself, rather than that of its effects. The experience with Hercule II also provides some interesting input into the sustainability question that is also relevant to the Hercule III programme. The sustainability of the results under grants awarded under the Hercule II programme was promoted by covering maintenance as well as training costs to enable staff to operate the equipment acquired under technical assistance grants. Beneficiaries were asked to submit a final implementation report one year after the closing date of the project to demonstrate its effectiveness and the sustainability of the outcomes. As regards training, seminars and conferences, the participants were generally confident that the acquired skills and competences can be put into practice over time: sustainability was achieved in this respect too. This was particularly true for participants in digital forensic training. 5.7 Contribution to smart, sustainable and inclusive growth The external study found that the contribution of Hercule III to the five headline targets identified by the Europe 2020 strategy for smart, sustainable and inclusive growth is not direct. However, protection of EU financial interests allows for: i) securing a larger amount of financial resources to pursue EU initiatives that aim to achieve the headline targets (revenues); and ii) ensuring that such resources are spent to foster smart, sustainable and inclusive growth rather than illegally diverted to other ends (expenditures). In addition, the increased protection of EU external borders stemming from Hercule III actions contributes to creating a level playing field between EU businesses and their competitors based in third countries, which is considered another key aspect to ensuring the timely achievement of Europe 2020’s targets. 6.Conclusions On the basis of the answers to the evaluation questions, together with the Commission's findings in relation to the results of the Hercule III programme, the following conclusions can be drawn with regard to the evaluation criteria. Relevance Against the background that the key drivers and problems identified in the impact assessment (see Figure 2) for the Commission's proposal for the Hercule III programme are still prevalent today, in particular the overarching fraud phenomenon, the external study demonstrated that the specific and operational objectives continue to be relevant for the protection of the EU’s financial interests. A few specific actions listed under Article 8 of the Regulation (e.g. “purchase of services to carry out chemical analysis of samples from tobacco and cigarette seizures” and “purchase of services to store and destroy seized cigarettes and other counterfeit goods”) were found to be less relevant to fulfilling their mission in relation to the PFI. This finding can largely be explained by the fact that these actions target a narrow group of possible participants and, consequently, the bulk of the participants do not see their relevance. To address some of these issues, the Commission is organising in 2017-2018 training sessions in the Member States for operational staff, financed under Hercule III, to increase awareness of the laboratory facilities for the chemical analysis of samples from seized tobacco and cigarettes. Some beneficiaries suggested a few ideas of possible additional operational objectives which would improve the relevance of the programme, in particular in the areas of cross-border cooperation between Member States, cooperation with non EU partners, as well as new technological developments. To what extent these ideas are shared by the other stakeholders was not tested within the external study. Their value is therefore limited. However, they are coherent with the Commission’s experience in implementing the programme and the direction towards which the Commission has started steering the programme over the last few years. Coherence The evaluation found that overall both the internal and external coherence of the programme were ensured. The programme’s internal coherence between its different components is guaranteed by the financial limits which the Regulation sets for each of them, by balancing the actions where there is more demand and other actions that are particularly relevant to the PFI (e.g., conferences on specific PFI topics or forensics training). Internal coherence is also guaranteed by the possibility to detect overlaps between applications at the selection stage, as well as the requirement that the purchase of equipment be complemented by technical assistance activities (eg, trainings to use the equipment). External coherence of the programme is ensured by the Commission internal consultation mechanisms that allow preventing duplications and overlaps with other EU programmes. Such consultation takes place essentially not only at the stage of elaboration of the AWP, but also during the selection of applications for grants. These conclusions are supported by the overall perception of coherence by stakeholders, as evidenced by the external study. Although more synergies with other Commission programmes might be possible, this needs to be carefully balanced against generating delays and inflating workload. Effectiveness The actions financed under Hercule III contributed to the achievement of the programme’s general, specific and operational objectives. The programme financed actions which, because of their features, clearly contributed to achieving the objectives of the programme, for instance when funding the procurement of investigation and detection tools (eg, devices and animals to carry out inspections, monitoring devices, scanners) or supporting trainings in, for example, the digital forensic area. A large number of training events were funded and these events covered a wide range of topics relevant to the PFI. The external study concludes, on the basis of the final reports of the actions and beneficiaries’ surveys, that those actions generated results that were largely in line with the expected outputs. Nonetheless, only few actions involved staff exchanges between national administrations and few actions involved international participation. The external study did also note that more should be done on the expenditure side of the EU budget, which is an area that has received relatively fewer grants. With regards to technical assistance, the timing for the evaluation, i.e. early in the life of the programme, made it challenging to further consolidate the above findings about the effectiveness of the programme.

Among the many external factors interfering with the achievement of the programme’s objectives, the ability of fraudsters to “adapt quickly to new circumstances at the EU level, operate cross-border and exploit the weakest points in the external borders” was perceived as the most important. Such external factors tend to harm the PFI and require continued efforts by national authorities to combat and prevent fraud. Efficiency The external study shows that the costs generated by the programmes’ training actions are in line with other EU programmes, be it in terms of the quantity of person-days necessary at the different stages (preparation of an application, signing of the grant agreements, and reporting), of the transaction costs for the acquisition and implementation of actions under grant agreements, or the average costs per participant in training actions. Cost-effectiveness is inherent to the implementation of technical assistance actions as they mostly consist of the acquisition of equipment that has to be purchased by public procurement procedures where the "best value for money" principle is used to select equipment. However, the results obtained following the deployment of the equipment during PFI related operations is difficult to assess for lack of data. On the basis of the external study’s findings, the application and reporting stages do not appear to be too cumbersome for a large majority of stakeholders. Some of the critical applicants made a number of suggestions that are currently in the process of being addressed. However, only part of these suggestions could be addressed, as some regulatory obligations are inherent to EU financial and procurement rules and cannot be simplified given the current legal framework. The Commission has started using a system for the electronic submission of applications for grants and for the management of grant agreements. The improvements brought about by the electronic management system will address a large number of the applicants’ suggestions, for instance by digitising the whole process and introducing the use of digital signatures. The electronic system is expected to noticeably alleviate the administrative burden and related time and resources spent on the applications in particular. EU added value The external study’s results demonstrated that the benefits from the programme would not materialise without the support of Hercule III or equivalent EU funding. The actions allowed for a better protection of the Union's external borders. In addition, the programme is an enabler for cross-border cooperation, in particular cross-border exchanges of information and best practices. The training actions facilitated networking opportunities between the participants to the events thus contributing to strengthening cross-border cooperation. The stakeholders almost unanimously recognised the added value of the programme and its essential contribution to generating benefits. The external study also revealed that the programme's transparency and reporting requirements contribute to the improvement of the beneficiaries’ planning, monitoring and quality standards and to the quality of the beneficiaries' financial management in general. Sustainability There is a consensus among beneficiaries that the actions funded under the programme will continue producing benefits, even after their completion. This was particularly relevant for technical assistance actions where the deployment of equipment will continue generating results beyond the closing date of a grant agreement, but also applied to training actions, where the beneficiaries were generally positive about the lasting effect of networking and exchanges of information and best practices. Nonetheless, a substantial number of beneficiaries indicated that they would discontinue the activities funded under the programme, if it were to be terminated. This finding suggests that the discontinuation of the programme would negatively impact the overall protection of the financial interests of the Union. Contribution to smart, sustainable and inclusive growth The evaluation found that the programme contributes indirectly to EU’s strategic goal for smart, sustainable and inclusive growth, in particular by protecting the financial resources that are used to achieve the priority’s goals (revenues) and ensuring that such resources are spent to foster smart, sustainable and inclusive growth rather than illegally diverted to other ends (expenditures). The increased protection of EU external borders supported by the programme also contributes to creating a level playing field between EU businesses and their competitors based in third countries. Annex 1: Procedural information This annex provides procedural information. It explains how the evaluation was managed both in terms of organisation and time. It provides information about external expertise used. A1.1Lead DG: European Commission, European Anti-Fraud Office (OLAF). A1.2Organisation: The evaluation roadmap was published in August 2016 and sets out the purpose and scope of the exercise. The roadmap presented the questions to be addressed under the five categories of effectiveness, efficiency, relevance, coherence and EU added value. An Inter-Service Steering Group (ISSG) was set up to support the evaluation. The tasks of the ISSG were to check the key elements of the external study, to support and monitor the evidence gathering and stakeholder consultation process, to review all external deliverables for the evaluation as well as the SWD and to assist with the quality assessment of the consultant's evaluation report.

The ISSG was composed of the Secretariat-General of the Commission, the Legal Service and DGs TAXUD, JUST, HOME and BUDG. The Commission contracted out an external study in support to the evaluation (see below), which provided the basis for this SWD. A1.3Timetable: Date Description June 2016 Establishment of the Steering Group 5 July 2016 1st Steering Group Meeting: Evaluation roadmap and ToR 09 September 2016 Specifications for study contract to support the evaluation sent to the framework contractor 25 October 2016 2nd Steering Group Meeting: Kick off meeting with the external contractor Award of a study contract and signature of the contract with CEPS on behalf of the consortium 12 December 2016 3rd Steering Group Meeting: Inception report meeting February - June 2017 Targeted stakeholder consultation –surveys and interviews 30 June 2017 4th Steering Group Meeting: End-of-fieldwork meeting 4 September 2017 5th Steering Group Meeting: External Draft final report meeting 24 October 2017 6th Steering Group Meeting: 1st draft Staff Working Document January 2018 Adoption of the report accompanied by the SWD Source: OLAF A1.4Exceptions to the Better Regulation Guidelines: Given the scope of the Regulation and the well-defined group of stakeholders, targeted consultations were considered as an appropriate and proportional means of consultation rather than an open public consultation. Considering that the public at large could not be considered as directly impacted by the provisions of the Regulation, responsible for their application, or possessing specific evidence needed for the evaluation, an open public consultation was not carried out. A1.5Consultation of the Regulatory Scrutiny Board: The evaluation has not been selected for scrutiny by the Regulatory Scrutiny Board. A1.6External Expertise: An external independent study is the basis for the conclusions presented in this document. A contract was signed on 25 October 2016 with Economisti Associati-led consortium (external contractor) . The external contractor carried out targeted consultations for the evaluation as explained in Annex 2. Annex 2: Synopsis report of stakeholder consultation in the context of the Study This annex provides a synopsis of the stakeholder consultation that was carried out for this evaluation by an external contractor as part of the external support study. It presents the main steps and findings of the consultation of interested parties and stakeholders. It shows that the stakeholder consultation process complied with the Commission's stakeholder consultation principles and minimum standards taking into account proportionality as required by the Better Regulation Guidelines . A1.1Feedback on evaluation roadmap The evaluation roadmap was opened for feedback during the course of the evaluation. No feedback was received in this period. A1.2Open public consultation

Regulation 250/2014 established the Hercule III programme to promote activities in the field of the protection of the financial interests of the European Union. The scope of this mid-term evaluation covers the interventions supported under the Hercule III programme to date as well as the preparatory and implementing activities undertaken by the stakeholders for these interventions. The groups of stakeholders it targets and impacts are well-defined, i.e. for the most part administrators and law enforcement officers at national and regional level. The public at large cannot be considered as directly impacted by the provisions of the Regulation or responsible for their application or possessing specific evidence that would usefully contribute to the evaluation of the application of the Regulation. Therefore, an open public consultation was not carried out and stakeholders were consulted in targeted consultations as explained below. A1.3Targeted consultations – overview of stakeholders consulted This section presents an overview of different stakeholder groups consulted via in-depth interviews and a survey. A total of 574 stakeholders were consulted through interviews and survey, comprising 16 officials of Commission or national services, 56 beneficiaries (i.e. 71% of all beneficiaries), 67 unsuccessful applicants (i.e. 25% of all unsuccessful applicants), 321 participants in events (i.e. 27% of all participants invited to reply) and 112 users of services (i.e. 31% of all users invited to reply). Whereas 49 stakeholders (i.e. all officials from institutions and 33 beneficiaries) were interviewed, the remaining stakeholders participated in the online surveys. European Commission and national officials Furthermore, exploratory interviews were conducted with several OLAF officials and with officials from DG BUDG, DG JUST, DG TAXUD. These interviews were additional to the 16 interviews with the Commission and national services conducted during the Fieldwork phase, which were divided as follows: five interviews with OLAF officials (Hercule Sector, Internal Auditor, units B1, C3, and D4); two interviews with officials from other DGs (DG JUST and DG TAXUD); nine interviews with officials from national institutions (eight AFCOS which are also members of the Advisory Committee for the Coordination of Fraud Prevention (COCOLAF) ; and one central customs authority) in Austria, Bulgaria, France, Germany, Italy, Lithuania, Poland, Romania and Slovenia. Beneficiaries In total, 56 beneficiaries from 18 Member States were consulted , i.e. 71% of those that were awarded a grant either in 2014 or 2015 ( Figure 1). Figure 1. Consulted beneficiaries by Member States and category of action (number of respondents) Unsuccessful applicants All entities that applied without success to the Hercule III calls for proposals in 2014, 2015 and 2016 were invited to participate in the online consultation. 67 respondents from 21 Member State completed the survey, i.e. 25% of all unsuccessful applicants during the first three years of the programme ( Figure 2) . Almost 50% of respondents were located in Poland and Romania; this is in line with data registered in the total population, as 121 out of 267 (45%) unsuccessful applications during the first three years of the programme were submitted by Polish and Romanian entities. Figure 2. Consulted unsuccessful applicants by Member States and category of action (number of respondents) Events participants 312 events participants from 25 Member States and nine from third countries completed the online survey (Figure 3).

Figure 3. Consulted participants by country of origin (number of respondents)

Unit: Number of respondents. Note: 321 total respondents. Source: Authors’ elaboration on online survey with participants in events. About one-third of the respondents took part in events arranged in a different MS from their country of residence ( Figure 4). The three largest groups of participants by type of organisation included customs officials, police officials and academic audiences ( Figure 5). Interestingly, many respondents had participated in more than one type of event in the context of the same action funded by Hercule III ( Figure 6). The reason is that several actions combined different types of event to achieve the expected results. Respondents were aged mainly between 25 and 54, with women representing less than 40% of the sample ( Table 1). Figure 4. Participants in events held in the same Member State and in a different Member State Note: 321 respondents. Figure 5. Consulted participants by type of organisation Note: 321 respondents. Figure 6. Consulted participants by type of event attended Note: 321 respondents; each respondent may have participated in more than one event in the context of the same action funded by Hercule III. Table 1. Consulted participants by age category and gender Age category Gender Total Female Male Prefer not to disclose Below 25 1.3% 1.3% 0.0% 2.5% 25-34 9.1% 9.7% 0.6% 19.4% 35-44 14.4% 22.8% 0.3% 37.5% 45-54 9.1% 18.4% 0.3% 27.8% 55-64 4.4% 7.2% 0.0% 11.6% 65 0.0% 1.3% 0.0% 1.3% Total 38.1% 60.6% 1.3% 100.0%

Source: External study For confidentiality reasons, relevant contact details for users of services procured by Hercule III were not provided to the external contractor; hence, OLAF sent the invitations to complete the online survey to users. Against this background, based on information provided by OLAF, 352 users were invited to complete the survey, which was eventually completed by 112 respondents (32% response rate; Table 2). Users were mainly based in Germany, the UK, Belgium and Italy (Figure 7). Almost 70% of respondents are users of statistics and IT tools; 30% are users of databases (e.g. NTELX, GTI, GRS, D&B reporting system, SEASEARCHER, GTA); only 6% are users of services to carry out chemical analysis of samples from tobacco and/or cigarette seizures (Figure 8). Table 2. Share of total population represented by consulted beneficiaries, unsuccessful applicants, participants in events and users of services Type of stakeholder Total sample Total population Sample / population Total beneficiaries 56 79 71% Beneficiaries TA (2014-15) 25 38 66% Beneficiaries AFT (2014-15) 21 25 84% Beneficiaries LTS (2014-15) 10 16 63% Total applicants 67 267 25% Applicants TA (2014-16) 56 174 32% Applicants AFT (2014-16) 5 61 8% Applicants LTS (2014-16) 6 32 19% Participants (2014-15) 321 1,194* (3,400)** 27%* (9%)** Users (2014-16) 112 352* 32%* Note: *Participants/users invited to complete the online survey. **Participants in events funded during the first two years of the programme according to the Annual Implementation Reports. As regards the type of organisation, the lion’s share of respondents belongs to the customs category (Figure 9). No information is available with regard to the total population of users; hence, no conclusion can be drawn about the share of the population represented by the sample. Figure 7. Consulted users by country and type of service (number of respondents) Note: 112 total respondents. Figure 8. Consulted users by type of service Note: 112 respondents. Figure 9. Consulted users by type of organisation Note: 112 respondents. Annex 3: Methods and analytical models

This annex provides a description of the methodological approach to the evaluation taken by the external contractor that supported this SWD. It summarises the main methodological elements and cross-references to the more detailed methodological annex of the external report supporting this SWD. A1.4Logic of intervention The intervention logic developed in the context of the evaluation includes a detailed description of the needs and problems that the programme is supposed to address, the objectives that the programme is expected to achieve, the activities that the programme intends to implement, the expected results of the programme and the logical links between these various components. A comparison between the expected results (i.e. how the programme was intended to work) and the actual results stemming from the intervention under analysis (i.e. how the programme worked in reality) enables a mid-term evaluation based on evidence. Against this background and in line with the Better Regulation Guidelines, the following components of the intervention logic were identified and described: ·The rationale for the intervention -Needs and problems -Objectives (general, specific, operational) ·The intervention -Inputs/activities ·The expected results of the intervention -Outputs (expected) -Outcomes (expected) -Impacts (expected) A1.5Evaluation Framework The evaluation framework serves the purpose of guiding the mid-term evaluation and includes the following items: ·evaluation criteria; ·evaluation questions; ·judgment criteria; ·indicators;

·data sources; ·data collection methods. The six evaluation criteria are generally used to evaluate EU policies and are enshrined in the 2015 “Better Regulation Guidelines” and the “Better Regulation Toolbox” . These criteria are made more specific through evaluation questions. Judgment criteria, expressed in the form of statements are used to answer the various evaluation questions. They can be then confirmed and/or rejected by the research. A comprehensive set of indicators serve the purpose of applying judgment criteria and have been chosen according to the RACER framework . Data sources are the sources of evidence on which the external study relies to apply the judgment criteria and answer the evaluation questions. Data collection method is the approach adopted to gather the required evidence. A1.6Data collection Primary data, i.e. data that were collected specifically for the purpose of the Assignment via: -interviews with: i) Commission staff; ii) national institutions active in fraud prevention and PFI; and iii) beneficiaries of actions funded by the programme; and -online surveys with: i) beneficiaries of actions funded during the first two years of the programme; ii) unsuccessful applicants; iii) participants in events co-financed by Hercule III; and iv) users of services procured by Hercule III. Secondary data, i.e. data gathered by consulting: -public available sources including, inter alia, the Regulation and accompanying material (e.g. the IA), AWPs, Annual Implementation Reports, Article 325 reports, final evaluation reports and interim reports of the Hercule II programme and legal texts establishing other programmes (e.g. Customs 2020, Fiscalis 2020, Internal Security Fund); and -official documentary evidence on both submitted proposals (i.e. application forms) and awarded contracts and grants (i.e. feedback from grant beneficiaries in the form of Final Technical Reports, Final Financial Reports and Final Implementation Reports (limited to technical assistance actions)). A1.7Evaluation challenges and limitations This section describes the main challenges encountered and the associated limitations of the evaluation results and mitigation measures put in place. In doing so it provides an overall assessment of the robustness of the methodology applied and the reliability of the available data. ·The population coverage ensured by the consulted national institutions (Member States receiving more than 60% of grants awarded in 2014 and 2015, plus two MS that did not receive any grant) and beneficiaries (71% of the total population) allows to draw robust conclusions for all evaluation questions. ·However, the shares of total population represented by unsuccessful applicants (25%), participants in events (27% of invited participants, 9% of total population) and users of services (32% of invited users) are lower. Therefore, more caution is required when interpreting conclusions based on these surveys. In order to ensure the highest quality of the findings: i) all evaluation questions are addressed by combining feedback from more than one stakeholder category; ii) whenever possible, data and information collected from stakeholders are compared with evidence retrieved from application forms and reporting documents in order to ascertain consistency across data sources consulted to perform the evaluation. ·The timing of the mid-term evaluation is compliant with Article 13 of the Regulation establishing the Hercule III programme, which requires the Commission to present an Independent Mid-Term Evaluation Report to the European Parliament and the Council by 31 December 2017. Against this background, the evaluation is confined to actions funded during the first two years of the programme (2014 and 2015). As regards 2016, the analysis is limited to applications received in the context of calls for proposals; in fact, no action co-financed by Hercule III grants for 2016 calls was completed at the moment of conducting fieldwork activities. Some actions covered by the 2014 and 2015 budget were still ongoing during fieldwork; hence, their impact can only be recorded to a limited extent. Due to the unexpected high number of applications, some delays in awarding the grants and finalising the funded actions were registered. In addition, based on information retrieved from application forms, half of the technical assistance actions funded in 2014 and 2015 were expected to last more than 12 months. These data limitations impinge on the assessment of both outcomes and impacts: -With regard to outcomes, 13 out of 25 technical assistance actions surveyed for this Assignment were still ongoing at the moment of gathering data and information to assess the effectiveness of the programme. In addition, for completed actions, most beneficiaries argued that they only recently received the devices, tools and systems co-financed by Hercule III. As a consequence, whereas most of the beneficiaries of technical assistance actions provided output indicators (e.g. type and number of items purchased with Hercule III funds), evidence on outcome indicators (e.g. successful operations, number of arrests, convictions seizures, etc.) is scant. Therefore, limited conclusions can be drawn about the alignment between actual and expected outcomes of technical assistance actions. -Impacts occur in the long-term and cannot be captured by a mid-term evaluation. In principle, in the long-run, the Hercule III programme is expected to improve the PFI, thus ensuring the competitiveness of the European economy and the protection of taxpayers’ money. It is too early to measure such impacts, especially considering that the first actions funded by the programme were only implemented in 2015. ·Available primary and secondary data and information mainly allow evaluating actions funded via grant agreements, which represent 75% of the overall commitments in 2014 and 2015. The evaluation of databases, IT tools and analyses of cigarettes and tobacco samples funded under procurement is performed on the basis of feedback provided by respondents to the online survey with users; hence, all TA actions have been assessed. It was not possible to collect evidence from the contractors directly due to confidentiality, data protection and contractual reasons in case of the evaluation of conferences, digital forensics training, studies and other actions procured by Hercule III. Therefore, OLAF contacted the contractor to distribute the links to the online surveys and to request additional information where necessary (e.g. satisfaction surveys). Such actions represent less than 10% of the overall commitments in the first two years of the programme. Therefore, the Assignment covers the bulk of the programme.