Comments 93% achieved. The target for this indicator was revised upwards in the 2013 restructuring to reflect progress to date and revised expectations. Indicator 11 Number of subprojects involving researchers residing abroad participating in collaborative research projects (cumulative) Value 0 10 37 54 Date 2009 2013 12/31/2015 Comments Surpassed (146% achieved). These subprojects aimed at promoting collaboration between Colombian researchers and the Colombian diaspora. The target for this indicator was revised upwards in the 2013 restructuring to reflect progress to date and revised expectations. Indicator 12 Number of evaluations for the convocatorias corresponding to the Labor Insertion and Diaspora pilots completed. Value 0 2 2 Date 2009 12/31/2015 Comments 100% achieved. The two studies provide a process evaluation of the two pilots, and sum up the main results from these investments. Component 3. Strengthening COLCIENCIAS’ capacity to promote research and innovation; investment for research and innovation. Indicator 13 Number of proposals financed for innovation-management capacity building subprojects Value 0 1.3 100 128 Date 2009 2013 12/31/2015 Comments Surpassed (128% achieved). This indicator was originally formulated as "Ratio of proposals for innovation management capacity building subprojects received, to proposals financed". It was revised in the 2013 restructuring because the ratio of proposals received to those financed depends on a number of factors (for instance, quality of proposals submitted, amount of financing available, etc.) that do not reflect COLCIENCIAS’ capacity. The new indicator measures the number of proposals actually financed under the Innovation Management activity. Nevertheless, the achievement of the indicator as originally formulated be of 5.6 as of December 2015. Indicator 14 Number of R&D and innovation subprojects in priority areas for which Project resources have been committed (cumulative) Value 0 10 2 5 Date 2009 2013 12/31/2015
Indicator 18 Number of citizens directly participating in COLCIENCIAS’ citizen participation activities financed by the Project (excluding Ondas and mass media) Value 0 75,000 8,500 27,512 Date 2009 2013 12/31/2015 Comments Surpassed (324% achieved). The wording of this indicator was changed in the 2013 restructuring from “benefitting” to “participating in” since the former concept is less straightforward to measure. The target was revised down to 8,500 to reflect the actual targeted number of participants for activities as planned in FY13 and FY14. The total number of participants at Project completion includes 25,000 participants from "A Ciencia Cierta", and 2,512 participants from the Social Appropriation pilot program (see Annex 2 for more details on these activities). Indicator 19 Annual number of document downloads from COLCIENCIAS’ Web page (not including documents related to for proposals) Value 18,000 39,500 n.a. 87,086 Date 2009 2013 12/31/2015 Comments Surpassed (210% achieved). This indicator was dropped in the 2013 restructuring. It was replaced by intermediate indicator 20 that better captures COLCIENCIAS’ improvement in reaching Colombian society and increasing awareness science, technology and innovation. Indicator 20 Annual number of unique visits to COLCIENCIAS’ web page Value n.a. 429,000 4,735,851 Date 2013 12/31/2015 Comments Surpassed (1104% achieved). The actual number of unique visitors registered at Project completion includes visitors in the period September- December 2015, as COLCIENCIAS installed the tool to measure this indicator (Google Analytics) only by the 08/24/2015. Assuming a constant number of visitors for each 4-month period, COLCIENCIAS' web page could have achieved as much as 18,943,404 unique visitors in the whole year. Indicator 21 Development of social innovation policy document Value No Yes Yes Date 2013 12/31/2015 Comments 100% achieved. This document served as input for the development of the CONPES (national strategy/policy document).
Efficacy Overall Efficacy Substantial Substantial Strengthen COLCIENCIAS’ capacity to promote human capital for the knowledge economy, R&D and innovation High High Raise awareness of STI in the Colombian society Modest Substantial Efficiency Modest Modest Rating Moderately Satisfactory Moderately Satisfactory Overall outcome rating Rating value 4 4 Disbursement weight 40% 60% Weighted value 1.6 2.4 4 Final Rating Moderately Satisfactory 3.5 Overarching Themes, Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development A proactive effort was made to get the most vulnerable population involved in the STI activities financed by the Project. The proportion of Ondas beneficiaries from the regions with the highest concentration of indigenous and afro-Caribbean population out of total beneficiaries went up from 26 percent in 2010 to 31 percent in 2014. As of 2015, 7 percent of Project beneficiaries were children and adolescents from indigenous communities. The Social Appropriation program did not explicitly target the most vulnerable population by design, but de facto ended up focusing on rural communities, indigenous populations and other ethnic minorities. Similarly, the A Ciencia Cierta contest, open to the whole population, rewarded several proposals submitted by indigenous communities. Importantly, this program was also able to reach areas inflicted by Colombia’s armed conflict. Notably, these activities had the effect not only of empowering indigenous communities and increasing their awareness about the benefits from science and innovation, but they also offered a public platform to increase awareness in the Colombian society about indigenous culture and traditions, specific problems and resources. It was one the first examples in the region to explicitly recognize local community knowledge as an equally important contributor to address STI challenges. Many of the innovations developed under pilot programs had a clear focus on social development, focusing on topics such as health (the development of low-cost point-of-care diagnostic for tuberculosis; a prototype tool for evaluating motor rehabilitation of patients with neurological diseases; the use of DNA sequencing technology for the diagnosis of hereditary cardio diseases), environment protection (protection of hydro resources, biodiversity), social development of marginalized groups (a new model of agricultural production combined to social support to families previously employed in cocaine
Kristine Ivarsdotter Senior Social Specialist LCSSO Ignacio del Busto Mellado Junior Professional Associate Juan C. Belausteguigoitia Safeguards Specialist OPSPF Dianna M. Pizarro Senior Social Development Specialist Glenn S. Morgan Safeguards Specialist OPSPF Juan Carlos Vasquez Arancibia Environmental Specialist GFM04 Nicole Lucilia Amaral Junior Professional Associate LCSHE Judith Marcano Williams Consultant LCSHH Sara Troiano Consultant GED04 ICR Consultant Antonella Novali Program Assistant GED04 Beverly Dixon Program Assistant Elsa Coy Program Assistant LCCCO Jagannath Griffiths Palma E T Temporary GED04 (b) Staff Time and Cost Stage of Project Cycle Staff Time and Cost (Bank Budget Only) No. of staff weeks USD Thousands (including travel and consultant costs) Lending FY10 81 383 Total: 81 383 Supervision/ICR FY11 70 210 FY12 45 138 FY13 39 157 FY14 35 143 FY15 20 129 FY16 15 102 224 879 Annex 5. Beneficiary Survey Results N/A Annex 6. Stakeholder Workshop Report and Results N/A
Annex 10. Causal Linkages between the PDO, Revised Key Indicators, and the Project's Components (according to Review during the ICR Stage) Outcome Output (Intermediate outcome) Key Indicators and Targets Intermediate Indicators and Targets Additional Indicators (collected at ICR stage, figures reflect actual values) Specific activities financed (inputs) Strengthen capacity to promote human capital for the knowledge economy, research and development and innovation Institutional strengthening of the STI National System 1 - 60% of COLCIENCIAS annual investment budget committed by the end of the second quarter of the calendar year - 3 instruments designed or redesigned and approved by COLCIENCIAS’ Board of Directors (dropped in 2013) - Lessons learned summary completed based on the results of the evaluations (dropped in 2013) - 80 trained according to a revised competencies model - 10 management using the Balanced Scorecard - Revised COLCIENCIAS’ convocatoria processes - 8 research papers, short policy papers, or technical papers completed by the internal policy unit -420 beneficiaries who have completed short course 1.11 as ratio of performance of COLCIENCIAS programs financed by this Project relative to the average performance of STI programs in Colombia, as per scores obtained in the Functional Analysis (World Bank, 2016) - Optimization of the convocatorias process - Reorganization of COLCIENCIAS' organizational structure - Balanced Card system - Development of the Policy Unit - Design of a revised competencies and training - Short courses to regional STI stakeholders Development of capital for science and technology 2 - 30 new ideas or innovations developed by Labor Market Insertion, Diaspora, Innovation Management, Open Innovation and Priority Area program - 1.2 million students covered yearly by Ondas - Design of monitoring and evaluation framework for Ondas - 29 doctoral graduates participating in the Insertion programs - $CO 1.7 pesos invested by the private sector for each CO$ 1 invested by COLCIENCIAS under the Labor Market Insertion program -48% of PhDs by the same firm participating in the program - Inserción laboral - Diaspora -
activities reflected in components 2 and 3 - 37 subprojects involving researchers residing abroad participating in collaborative research projects - 2 evaluations for the convocatorias corresponding to the Insertion and Diaspora pilots completed - 16 scientific articles produced the Diaspora program -23 graduate thesis supported under the Diaspora program Investment for research and innovation 3 - 100 proposals financed for management capacity building subprojects - 2 R&D and innovation subprojects in priority areas for which Project resources have been committed - US$ 4 million of project resources committed to finance R&D and innovation subprojects in there is at least one firm participating (dropped in 2013) - 3 evaluations for the calls-for corresponding to the R&D and innovation pilots completed - 93% of the beneficiary firms thinks the capacity building received was useful/very useful (Innovation Management) + 113% in the participating budget allocation to STI investment of firms with respect to the pre-program levels (Innovation Management) + 44% in the number of employees with PhD with respect to the pre-program levels (Innovation Management) + 68 % in the number of employees assigned to STI activities with respect to the levels (Innovation Management) + 87 % in the number of STI projects financed with respect to - Priority areas - Innovation Management
the pre-program levels (Innovation Management) Raise awareness of science, technology and innovation in the Colombian society Social Dissemination of Science, Technology and Innovation and Institutional Communication 4 - 2674 grant applications for R&D and innovation subprojects received yearly by COLCIENCIAS (Dropped in 2013) - 76% of the population agrees that Colombia creates Science and 73% that it creates Technology - Monitoring and evaluation framework for social dissemination of STI defined - 8,500 citizens directly participating in COLCIENCIAS’ citizen participation activities financed by the Project - 39,500 document downloads from COLCIENCIAS’ Web page per year (dropped in 2013) - 429,000 unique visits to COLCIENCIAS’ web page per year - 2,512 citizens participating to Social Appropriation - 25,000 citizens participating to A Ciencia Cierta - 50% increase in the number of citations of COLCIENCIAS and/or its activities in the media - 59,737 followers in COLCIENCIAS Facebook page -100,403 followers in COLCIENCIAS Twitter account - Social Appropriation - A Ciencia Cierta - Institutional communication