Source document: https://docs-lawep.s3.us-east-2.amazonaws.com/1698060647824.pdf (Georgia)
Terms of Reference (ToR) For Final Evaluation of the Project - EU4Youth: Social Entrepreneurship Ecosystem Development (SEED) Program for Green Growth in Borderline Communities Project Number: ENI/2019/412-395 Location: Project Period: Project Budget: Georgia (Gori, Ninotsminda, Akhmeta municipalities), Armenia (Lori, Tavush and Shirak regions) December 19, 2019 – April 19, 2023 EUR 1,666,625 Name of implementing agency: CENN Name of co-applicants: Kakheti Regional Development Foundation (Georgia); Green Lane NGO (Armenia) Table of Contents 1.
2.

3.
Background and Context ............................................................................................................................................ 3 Scope .......................................................................................................................................................................... 5 Evaluation Questions ................................................................................................................................................. 5 4.
Design and Approach ................................................................................................................................................. 6 5.

Workplan.................................................................................................................................................................... 7 6.
Deliverables................................................................................................................................................................ 8 7.

8.

9.
9.

The Evaluation Team .................................................................................................................................................. 8 Coordination/Responsibility ...................................................................................................................................... 9 Submission of Offers .................................................................................................................................................. 9 10.
Checklist for the submission of offers ....................................................................................................................... 10 2 1.
Background and Context The project “Social Entrepreneurship Ecosystem Development (SEED) Program for Green Growth in Borderline Commu- nities” regards social entrepreneurship (SE) as a means to contribute to wider social and environmental benefits of the local communities in the target regions of Armenia and Georgia and uses SE as a pathway to successful transition to work of disadvantaged rural youth.
The overall objective of the project is to develop the entrepreneurial potential of young people from the EaP region in the field of social entrepreneurship, with a view to contributing to social cohesion, employment, inclusion and reduction of inequalities.
The specific objective of the action is to facilitate youth employability in Georgia and Armenia by capacity building, promoting sub-national, national and cross-border cooperation actions, supporting the establishment of favourable ecosystems for SE and green innovation as tools for social transformation.
Outcome 1: Multi-stakeholder platforms (Caucasus, Geo & Arm, and sub-nationals) on SE&GI are established and effectively promote creation of enabling environment for SE&GI of youth in the South Caucasus region: A1.1.
Mobilization of the stakeholders at the national and local levels for promotion of SE&GI; establishment of multi- stakeholder platforms (Caucasus, national and locals) and mechanisms for effective coordination and cooperation; organization of issue-based regular online and physical coordination events and meetups (with emphasis on engage- ment of young women, youth and disadvantaged groups, local change makers, activists, tech-makers, public servants, social entrepreneurs, private sector, scientists) to raise awareness, establish partnerships (i.e.
PPPs) and advocate SE&GI; A1.2.

Review of the existing national regulatory frameworks for implementing SE and the European regulations and best practices promoting SE; development of recommendations for creating enabling environment for SE in Georgia and Armenia, with a particular focus on women, youth and disadvantaged groups living in vulnerable borderline com- munities; active advocacy of the recommendations on effective transition of youth groups to labour market using mod- ern and IC tools; A1.3.
Organization of a Caucasus regional Conference & Expo on Youth Social Entrepreneurship and Green Innovation (SE&GI).

Outcome 2: Disadvantaged youth from target regions of Georgia and Armenia (especially, girls, various vulnerable groups – PwDs, LGBTQI, minorities, IDPs, ecomigrants) are empowered for effective transition to work through ca- pacity development: A2.1.
Elaboration and provision of the knowledge materials on SE with a specific focus on green innovation and green growth (circular economy, sustainable resources management, climate resilient communities, inclusive rural develop- ment, social equality); A2.2.

Development of an education program to strengthen local educational institutions.

Facilitation of access of local youth to existing e-learning platforms e.g.

Khan Academy courses on SE&GI; A2.3.


Mobilization of the rural youth from vulnerable groups and identification of young potential social entrepreneurs; A2.4.

Knowledge and skills development of the target groups via trainings and capacity building activities including sharing of international expertise on the best practices in SE&GI; A.2.5: Organization of Social Innovation Camps (transboundary) to share experience and knowledge.
Outcome 3: Benefits of innovative SE solutions are demonstrated through implementation of pilot social initiatives in Georgia (1) and Armenia (1), seed funding for the young SEs’ ideas, creating trans-boundary social innovation links, and implementing cross-border cooperation initiatives on SE&GI: 3 A3.1.

Implementation of pilot projects on SE&GI (1 project in Armenia by Green Lane and 1 project in Georgia by CENN) for disadvantaged youth; A3.2.

Creation of Youth Opportunity Fund.


Implementation of Caucasus Social Innovation Competitions (under the- matic challenges on green innovation: circular economy, climate-smart solutions, green cities and transport, sustainable resources management, tourism) for identification of innovative SE solutions and synergies at the national and trans- boundary levels.
Implementation of the Youth Social Entrepreneurship GREENcubator Program for provision of men- toring to young social entrepreneurs, which will give them the knowledge and skills to obtain SE funding and access production resources.

Provision of seed funding to young social innovators for implementation of projects and creation of trans-boundary synergies with other programmes led by civic, public, private and academic sectors; A3.3.
Organization of study tours - exchange visits of Georgian and Armenian youth (SE competition winners, SE project trainees, young leaders, social change-makers and young entrepreneurs) to explore and learn about the best social entrepreneurship practices; A3.4.

Creation of “Idea Bank” digital repository of SE&GI – a portfolio of innovative project ideas by newcomer social innovators – that can be supported by investors, venture capitalists, state and/or successful entrepreneurs in the future.
Outcome 4: Awareness is increased and benefits of SE for youth transition to work are recognized, links are estab- lished and networking is active among the stakeholders to upscale best practices and ensure sustainability of the project: A.4.1.
Launching extensive communication and awareness raising campaigns to create clear understanding of the con- cept of SE with particular focus on GI and circular economy (CENN’s E-network (30,000 subscribers), series of vlogs on successful SEs, GREENovator YouTube Channel); A4.2.

Organization of job fairs and career orientation events (JobFest) with a special focus on girls and women, youth and disadvantaged groups; A4.3.
Promoting success stories from projects via organization of site visits and media tours; A4.4.

Addressing information gaps, creating new economic opportunities and reaching disadvantaged youth by provid- ing access to internet and technologies for young people, with a specific focus on women and girls in the remote vul- nerable regions through the establishment of Rural Innovation Knowledge Hubs (6).
These actions will result in the following impact: Transboundary multi-stakeholder cooperation effectively promoted SE in the South Caucasus region and creation of an enabling environment for its further development; SE awareness is raised amongst stakeholders; disadvantaged youth are empowered in target regions through SE capacity building and implementation of SE&GI initiatives in Georgia and Armenia.

1.
1.

Purpose and Objectives Overall purpose of the project evaluation is to determine the fulfilment of project objectives during the entire project.

The objectives of the project evaluation are to assess effectiveness, efficiency, coherence, relevance, EU added value and sustainability of the project and identify related good practice and challenges, as well as related facilitating and hindering factors.
The evaluation should also give analysis on both positive and negative changes resulted from the intervention, and how intended or unintended these changes were.

The overall objective of project evaluation is to elaborate lessons learned in relation to effectiveness and sustainability based on the findings and issue evidence-based, actionable recommendations for future programming and implementation.

The project shall be evaluated based on understanding of the following concepts: 1.
Effectiveness: The extent to which the development intervention’s objectives were achieved, considering their relative importance and timeline of the project.

Note: Also used as an aggregate measure of (or judgement of) the merit or worth of an activity, i.e.

the extent to which an intervention has attained, or is expected to attain, its major relevant objectives efficiently in a sustainable fashion and with a positive institutional developmental impact.

2.
2.

Efficiency: A relationship between the resources used by an intervention (including funds, expertise, time, etc.)

and the changes generated by the intervention (which may be positive or negative).

3.

Relevance: The extent to which the objectives of a development intervention are consistent with beneficiaries’ needs and problems, the country’s requirements, global priorities and EU policy goals and priorities; 4 4.
Coherence: Assessment on how well or not different actions work together, including existing or desired syn- ergies to improve overall performance, tensions e.g.

objectives which are potentially contradictory, and ap- proaches which are causing inefficiencies.

5.

EU Added Value: Assessment of the changes due to the EU intervention, over and above what could reasonably have been expected from national actions by the countries.

6.
6.

Sustainability: The continuation of benefits from a development intervention after development assistance has been completed.

The probability of continued long-term benefits.

7.

Replication: The possibility and practical recommendations for scale-up activities.

2.

Scope The evaluation will cover activities that have taken place since the beginning of the project until the time of the evalu- ation (December 19, 2019 – April 19, 2023).
The evaluation will start during the final stage of the project implementation (February-March, 2023) and be finished by May 19, 2023.

The evaluation should provide quantitative and qualitative analysis of the initiative’s efforts in both countries.

The final evaluation will be based on EU Better Regulation Guidelines & Toolbox1.

3.

Evaluation Questions It is expected that the selected evaluation team prepares detailed interview guides corresponding to each target group, reviewed and approved by CENN.
The general evaluation questions are provided below: Effectiveness • • • To what extent has the intervention achieved the intended objectives?

To what extent have the target groups been reached?

To what extent was gender mainstreaming ensured throughout the planning and implementation phases of the project?

Efficiency • To what extent are the costs associated with the intervention proportionate to the benefits it has generated?

What factors are influencing any particular discrepancies?
How do these factors link to the intervention?

To what extent have human resources been used economically?

• • Were there any alternatives for achieving the same results with fewer inputs/funds?

Relevance • • • To what extent was the intervention relevant?

To what extent have the (original) objectives proven to have been appropriate for the intervention in question?
To what extent do respective strategic goals and project targets correspond with the basic principles of EU (including Youth and Education Package, EU Youth Strategy, EU Youth Strategy etc.)?

Coherence • • • To what extent is this intervention coherent with other interventions which have similar objectives?

To what extent is coordination and complementarity with other relevant initiatives ensured?

To what extent is the intervention coherent internally?

(E.g.
(E.g.

are the expected results/outputs of the project consistent with the outcome, immediate impact and overall goal/impact (as part of the analysis of the logframe matrix and the presentation of the theory of change and its underlying assumptions)?
EU Added Value 1 https://ec.europa.eu/info/sites/default/files/better-regulation-toolbox.pdf 5 • What is the additional value resulting from the EU intervention(s), compared to what could reasonably have been expected from the countries acting at national and/or regional levels?

• What would be the most likely consequences of stopping or withdrawing the existing EU intervention?

Sustainability • • • To what extent will activities and results continue after donor support has ended?
To what extent does the intervention reflect on and take into account factors which, by experience, have a major influence on sustainability (i.e.

economic, ecological, social and cultural aspects)?

To what extent did the project strengthen local ownership and leadership?

Replication To what extent can the program be replicated?

What should be replicated and what not?

Why?

• • What will be the added value and increased impact of replication?
• • What are the minimum necessary resources needed for replication?

• What are the risks associated with replication?

Looking back - what would we do differently?

Looking forward – what should we do differently next time?

4.

Design and Approach The evaluation follows a mixed-methods approach using non-experimental design.

Hybrid (on-site and remote where applicable) or fully online methods are preferred.
The methods suggested are a document review, secondary data anal- ysis, key informant interviews, focus group discussion(s) and surveys of specific target groups, such as RIK Hub benefi- ciaries and sub-grant holders.

The triangulation of data, sources and methods should be ensured as much as possible in order to promote credibility and use of evaluation results.
The evaluation design, approach and methods should clearly show how data collection will be human rights based, foster environmental sustainability, and gender sensitive.

In ad- dition, all data needs to be disaggregated by sex, region, age, stakeholder group (duty bearers vs. rights holders).

Under the overall guidance of CENN, the selected partner will be responsible for providing the following tasks: 1.
To design the qualitative and quantitative study methodology (including sample designs) in close consultations with CENN team and based on EU Better Regulation Guidelines & Toolbox evaluation criteria 2.

To develop and pilot quantitative and qualitative research instruments 3.

To conduct surveys representative of relevant target groups (as listed below) 4.

To conduct the Key Informant Interviews and Focus Group(s) for collecting data from direct project stakehold- ers as listed below 5.
To conduct desk study and review relevant secondary data for measuring achievement of intended project targets; 6.

To analyse the collected quantitative and qualitative data (including the secondary data obtained from desk study) and submit the first draft report in English language to CENN 7.

To incorporate feedback in the report and submit the final report in English to CENN.
The main stakeholders and beneficiaries who are expected to be involved in the evaluation process are listed below (indicating approximate number of people in each category).

Selection and justification of each method appropriate for the given group of stakeholders/beneficiaries should be provided in the technical proposal.

The suggested methods with indication of maximum people are given below: Key Informant Interviews (KIIs) Central authorities of Georgia and Armenia (approx.
10) • • Authorities of target regions (approx.

6, local municipalities/regions) • Representatives of donor organization (approx.

4) • Representatives of other interventions with similar objectives (approx.

2).

Focus Group discussions • • • Private sector (including large companies and SMEs) (approx.

10 people) Social Enterprises (approx.

20 people) Educational institutions (approx.

20) 6 • Members of multi-stakeholder platforms (approx.

15 organizations) • CSOs (approx.

20 people).


Mixed methods (quantitative survey representative of the sample and Focus Groups for in-depth context) Youth beneficiaries of capacity building activities (more than 3500) • Rural Innovation Knowledge (RIK) Hubs (approx.

1200 members) • • Beneficiaries empowered in job transition (more than 500) • • Beneficiaries of GREENcubator program (approx.

50 people) • Beneficiaries of university programs (approx.
30 people) • Beneficiaries of Caucasus Social Innovation Competitions (27 sub-grants).

Social/Green Enterprises (more than 60) Project team to be interviewed: • • Project team in Georgia (approx.

5 people) Project team in Armenia (approx.

3 people).

5.

Workplan The process of the evaluation should follow the next steps: Kick-off and Inception Phase (March, 2023) The first phase comprises of the following actions/deliverables by the evaluation team: a.
Desk research and a study/desk review of documents and data (portfolio analysis) provided by the project and documents identified by the evaluation team (other literature/evaluative evidence) b.
Focus group discussion with project team focusing on project theory c. Review of existing evaluations and studies as a basis for enhancing the project theory d. Draft inception report (10 pages plus annexes) that must contain at least the following points: • Detailed presentation of the methods to be applied during data collection • Proposed evaluation time plan, including field visits • • Final reconstructed project theory Evaluation Matrix (showing how each question will be answered through the use of indicators, data col- lection tools etc.)
Interview and survey outlines (in annex) • • Bibliography (in annex) • Proposed structure of replication strategy Incorporation of the written comments into the draft inception report and submission of the final inception report.

e. f. A workshop for presentation of inception report results.

Data Collection and Interview Phase (April, 2023) The main components of the second phase are: a.

Data collection as agreed upon in the inception report.
Any changes to the inception report need written per- mission from CENN b.

Key informant Interviews, Focus Group Discussions, survey data, as applicable.

Data Analysis and Submission of Report (May, 2023) a.

Data analysis and preparation of triangulation matrix.
b. Submission of Draft Evaluation Report that must contain at least the following points: Executive Summary Introduction • • • Background and Context Analysis Evaluation Design and Approach • o Methodological Approach o Data Collection and Analysis Tools o Limitations, Risks and Mitigations Measures 7 • Findings • Conclusions • Recommendations • Annexes a. Submission of Draft replication strategy (in a separate document) b. Incorporation of the feedback (or else non-incorporation of feedback based on a sound justification) by the evaluation team and submission of the final draft report and replication strategy (in separate documents) to CENN c. Providing a PPT presentation summarizing main findings of the evaluation d. Holding of a concluding workshop, including the presentation of results and conclusion as well as a discussion e. on refining the recommendations by the evaluation team Incorporation of the written comments by the evaluation team and submission of the final version of the report to CENN for coherence screening.
6.

Deliverables The selected evaluation team is expected to produce the following deliverables: 1.

Draft and final inception report, including detailed study methodology, the planned evaluation approach and design, structure of the reports, a work plan with timeframe and draft data collection instruments 2.

Survey data and Interviews/Focus Group Discussions including transcripts 3.

Feedback matrices to the draft reports 4.

The draft and final evaluation report, including an executive summary 5.
Analysis of project replication strategy (in a separate document) 6.

A PPT presentation summarizing main findings of the evaluation 7.

A workshop for presentation of inception report results and a concluding workshop.

All deliverables must be in English (the data collection instruments should additionally be in Georgian and Armenian, as needed).

7.
7.

The Evaluation Team The evaluation should be conducted by a team of evaluators, including at least two national evaluators from each coun- try (Georgia and Armenia).

The evaluation team should include different gender.

The team composition should be de- tailed and explained in the technical offer, together with a division of tasks among all team members and the added value of each member.
The team leader should have the following qualifications: • • Track record in leading evaluations during the last 5 years (at least three evaluations conducted), proven by at least one such evaluation annexed to the offer Solid experience in developing methodologies and research instruments, including survey development in line with international academic standards and guidelines proven by at least one writing sample (published or un- published, to be annexed to the offer) • Graduate degree in social science, environmental studies or related field • At least 5 years of experience in designing and implementing donor-funded projects in the field of social/sus- tainable development and/or environment in Georgia and/or Armenia • At least 5 years of experience in evaluating donor-funded projects in the field of social/sustainable develop- ment • At least 5 years of experience in working with multi-stakeholders: government, civil society, community-based organizations, and UN/multilateral/bilateral institutions Sound understanding of rural development, sustainable development and participatory decision-making pro- cess in the regions of Georgia and Armenia Proven knowledge of Social and Green Entrepreneurship is a strong asset Familiarity with the political, economic and social situation in Georgia and Armenia Language qualifications: Fluency in English; fluency in Georgian and Armenian • • • • 8 • Previous experience in working with international organizations would be a strong asset.
The other team member(s) should have the following qualifications: • Graduate degree in social sciences or humanities and/or senior academic experience • At least 5 years of experience in leading the design and conducting surveys, writing study reports • Proven experience with survey development and conduct, proven by at least one writing sample • At least 5 years of proven work experience on project/program evaluations with the OECD/DAC and EU Better Regulation Guidelines & Toolbox evaluation criteria; previous evaluations in the rural development/environ- mental field in Georgia and Armenia will be an asset • At least 3 years of experience in conducting research on rural development issues • • • Sound gender expertise and knowledge of evaluation methods needed for gender-specific analysis Proven knowledge of Social and Green Entrepreneurship Proven familiarity with or work experience in Georgia and Armenia; evaluation experience in Georgia and Ar- menia Language qualifications: Fluency in English, Georgian and Armenian.
• 8.
Coordination/Responsibility The evaluation team will be provided with: Project documents (including annexes, revisions and budget) • • All progress reports • • • • Final report (draft report will be prepared during the evaluation) Previous evaluations and relevant data collection results of the project Information on other projects closely connected with the project to be evaluated List of contacts containing all persons involved in the implementation of the project (with function, task, con- tact data and information on language skills) as well as all local partners (including representatives of the target group, as far as feasible) • All other relevant documents prepared by the project (e.g.
guidebooks, recommendations etc.).

Ms. Mariam Khergiani, project manager will be the contact person for this evaluation.

9.

Submission of Offers The proposal must include the following components provided below.

Incomplete applications will not be considered.

The weight of each award criteria is given below: Award criteria Weight of award criteria in % 1.

CV of maximum 4 pages for each person, high- 20% lighting relevant experience 2.

One sample of previous similar work (in English) 10% 3.
The technical proposal with a description of the understanding of the assignment, the suggested expert/s, evaluation design, data collection and analysis methods to be used, including risks and mitigation strategies, suggested workplan, divi- sion of work and detailed timeframe.

50% 4.

The financial proposal, with detailed cost break- down (please note that budget should be ex- cluded VAT) 20% Please send any questions as well as your complete offer (1.

CVs in a separate pdf and 2.


All other documentation) to the following email address: mariam.khergiani@cenn.org no later than April 10, 2023, 23:59, Tbilisi time.

9 10.

Checklist for the submission of offers Before sending your proposal, please check that each of the following cri- teria HAVE BEEN MET IN FULL AND TICK THEM OFF Tick the items off below Yes No 1.

CVs in a separate pdf file 2.

One sample of previous similar work 3.


Technical proposal/methodology including workplan and time- line (please refer to the table above) 4.

Financial proposal preferably in MS Excel file 5.

Any other documentation that can be used to prove legal status in Georgia/other countries 10