3 – The monitoring and evaluation framework to be adopted by will be closely aligned with the ToC as outlined in Figure 5 and Figure 33. For each level of the ToC, the framework outlines relevant performance indicators, the expected baseline, evaluation questions that have emerged, the identification of parties responsible for data collection, key risks to the achievement of the outcomes, targets to be sought, the reporting that would be associated therewith as well as the use of the information provided. An initial framework to be finetuned by the IM looks as follows: Figure 50: Illustrative monitoring and evaluation framework for theory of change - Inputs Program 1.Desired result Performance indicators 4.Evaluation questions of parties 6.Risks 7.Target 8.Reporting 9.Use Provision of capital to investee Funding available for investment Capital raised and set aside for investment purposes Zero What does funding for such an activity look like, and how best to manage this internally? Secretariat; trustee; IM (i) Failure to secure sufficient funding; (ii) higher cost structures associated with operations of EMIIF; (iii) of available funds to investments n/a Quarterly reporting to SAG and FAS Secretariat and SAG, to determine relevant interventions and report on success Provision of TA support to SMEs through investee Funding available to support TA activities Capital raised and set aside for TA purposes Zero Secretariat; trustee; IM; investee funds Under-allocation of available funds to TA n/a Quarterly reporting to SAG and FAS Secretariat and SAG, to determine relevant interventions and report on success Provision of TA support to investee Funding available to support TA activities Capital raised and set aside for TA purposes Zero Secretariat; trustee; IM Set-up of EMIIF and governance structures; appointment of trustee and IM (i) Establishment of entities, contractual arrangements, governance structure; (ii) appointment of counterparties (i) Relevant legal agreements in place; (ii) decision-making to establish governance structures; (iii) meetings and engagements of relevant stakeholders Zero What are the biggest challenges in completing set-up? Secretariat (i) Failure to secure internal decision-making and approvals ii) Failure to procure relevant services Target structures and appointments Monthly reporting to SAG and FAS SAG and FAS, to monitor progress and determine need for
Figure 51: Illustrative monitoring and evaluation framework for theory of change - Outputs Program logic 1.Desired result 2.Two-three Performance indicators 4.Evaluation questions of parties 7.Target 8.Reporting 9.Use Investments made Funding disbursed to investee funds i) Overall amount committed to and # of investee funds which focus exclusively on early-stage SMEs ii) Amount committed to and # of investee which focus exclusively on early-stage SMEs iii) Amount committed to and # of investee funds which have first-time fund managers Zero Can the EMIIF structure be deployed effectively and the relevant funding disbursed as well as support delivered within a reasonable timeframe? trustee; IM i) Failure to have sufficient capital allocated ii) Failure to originate, successfully recommend or close investments n/a Quarterly reporting to SAG and FAS SAG and FAS, to monitor progress and determine TA funding disbursed Funding disbursed to fund TA activities i) Amount committed to investee in terms of TA funding allocation ii) # of investee funds receiving direct TA support (including type of support) iii) # of hours of training sessions held on impact management and gender lens investing Zero trustee; IM; investee funds n/a Quarterly reporting to SAG and FAS IM actively managing portfolio Ongoing active activities i) Frequency of engagements with investee funds ii) Qualitative reporting on nature and content of engagements n/a trustee; IM i) IM not remunerated sufficiently ii) IM not sufficiently skilled and equipped n/a Annual reporting to SAG and FAS Training provided to investee Improved understanding through training delivered i) # of of training programs delivered ii) Categories of training delivered Zero IM i) Availability of relevant from DFAT and/or other service providers ii) Availability of resources to fund training n/a Quarterly reporting to SAG and FAS Dissemination of information across DFAT… Quality information shared with other stakeholders Information pieces (e.g. memorandii, reports, presentations) made available Zero Do the necessary resources and capabilities exist to allow for positive engagement within DFAT? Secretariat n/a Annual reporting to SAG and FAS … through fora and other means Delivery of fora and other channels to share information Number and types of fora and other channels/events organised Zero Secretariat i) Insufficient resources committed to Secretariat ii) Insufficient attention devoted to preparation of information and organisation of relevant channels. n/a
Figure 53: Illustrative monitoring and evaluation framework for theory of change – End of program outcomes Program 1.Desired result 2.Two-three Performance indicators 4.Evaluation questions of parties 6.Risks 8.Reporting 9.Use Funded SMEs operationally sustainable Funded SMEs are financially sustainable and profitable i) # of SMEs receiving TA support from investee funds with improved business practices ii) Financial performance of portfolio SMEs (e.g., business revenues, profits, sales) iii) # and % of SMEs able to secure follow-on capital following EMIIF’s investee funds initial investments Unknown at design stage How effective is EMIIF in achieving SME growth? Secretariat; IM; investee funds i) External market risks ii) Insufficient financing and other support by investee funds iii) Underperformance of investee SMES due to internal shortcomings n/a Annual reporting to SAG and FAS SAG and FAS, to monitor progress and evaluate long-term value of EMIIF initiative Increased adoption of gender-equality and impact management by SMEs Increased levels of adoption of relevant practices and approaches by funded SMEs i) # of SMEs incorporating impact management and gender-friendly policies (e.g., flexible working arrangements) ii) # and % of SMEs that are certified by environmental regulations Unknown at design stage How effective is in facilitating a mindset shift among SMEs? Secretariat; IM; investee funds i) Failure by investee funds and/or TA providers to impart sufficient understanding of implicit business value ii) Lack of buy-in from underlying SMEs n/a Annual reporting to SAG and FAS id. Demonstrated track-record of investee funds Proven, verifiable of investment performance by investee funds i) Amount and # of fund managers (investee funds) receiving follow-on investments ii) # of investments successfully exited SMEs) iii) Performance of investment portfolio, based on metrics Unknown at design stage How effective is a structure like EMIIF in developing sustainable Secretariat; trustee; IM i) External market risks ii) Poor performance of investee fund management team iii) Underperformance of investee SMEs due to internal failures n/a Quarterly reporting to SAG and FAS id.
Demonstrated financial and development benefits of impact management and gender lens investing Positive financial and impact benefits evidenced through relevant metrics and reporting i) No. of investee with improved investment processes ii) No. of investee funds with improved risk management strategies, incl. social, cultural and environmental factors Unknown at design stage Can the benefits of adopting approaches be proven? Secretariat; IM; investee funds i) Ineffective training not allowing for effective adoption ii) Poor implementation by underlying businesses n/a Annual reporting to SAG and FAS id. Increased funding flows through EMIIF Increased size and scale of EMIIF i) # of additional initiatives launched through EMIIF ii) Size of additional funding committed through EMIIF Zero Can EMIIF successfully attract and deploy funding from other Divisions? Secretariat i) Lack of support from country ii) Insufficient effort devoted by Secretariat iii) Underperformance of EMIIF fund n/a Annual reporting to SAG and FAS id. Information and insights are integrated to inform all DFAT activities Integration of learnings and insights into all DFAT activities Qualitative reporting on broader dissemination of information and integration of learnings Zero How best can EMIIF support DFAT’s broader PSD/PSE objectives? Secretariat i) Lack of support and engagement by other DFAT departments ii) Information provided not effective in creating learnings n/a Annual reporting to SAG and FAS id. Figure 54: Illustrative monitoring and evaluation framework for theory of change – Long-term outcomes Program logic 1.Desired result 2.Two-three Performance indicators questions of parties 6.Risks 7.Target 8.Reporting 9.Use
and introducing greater levels of interest from and investors? Investee attracting additional capital Increased amounts of funding being made available to SMEs i) Amounts co-invested in investee ii) Amount and # of investee funds and broader industry receiving follow-on capital Unknown at design stage IM; investee funds i) External factors affecting of financing ii) Poor performance of investment portfolios iii) Failure to effectively disseminate learnings n/a Annual reporting to SAG and FAS id. Broader adoption of gender lens investing and impact management in the impact investing market Increased adherence and improved adoption of relevant factors within impact investment i) # of SME funds adopting impact management and/or gender lens investing approaches ii) Use of tools and guidelines in the wider industry Unknown at design stage How effective is EMIIF in facilitating a mindset shift within the broader market? IM i) Absence of proven impact and benefits from EMIIF and similar initiatives ii) Lack f support and buy-in from impact investors n/a Annual reporting to SAG and FAS id. Investment-based approach broadly adopted as complementary approach in DFAT Entrenchment of investment-based approach within DFAT as an organisation i) Overall support offered for relevant activities ii) # of initiatives adopting an investment-based approach, and amount of funding committed thereto Limited How well can an approach be integrated within DFAT? i) Lack of support and engagement by DFAT departments ii) Information provided not effective in creating interest iii) Inherent underperformance of EMIIF n/a Annual reporting to SAG and FAS id. Industry-wide recognition of benefits of collaboration Broader recognition of relevant approach within overall landscape i) # of new organisations undertaking similar initiatives, potentially informed directly by EMIIF ii) # of initiatives seeking to collaborate with EMIIF Unknown at design How effective is EMIIF in influencing industry thinking? i) Lack of support and engagement by other DFAT departments ii) Information provided not effective in creating interest iii) Underperformance of EMIIF n/a Annual reporting to SAG and FAS id.