E VA LUAT ION R E P OR T

Executive summary
Palestinian Territoriesand Jordan
Scaling up Media and Information Literacy and supporting
emerging women journalists

dw-akademie.com

BackgroundIn 2021, the Palestinian Territories andJordan were ranked 132 and 129, respectively, out of 180 countries in the World
Press Freedom Index. In both places, authorities restrict free access to information, especially in the digital realm, and intimidate journalists for political reasons.
Over half of the respective populations
are under 25 years old. Particularly in Jordan, this demographic has surged in recent years due to the settlement of Syrianrefugees, mainly in the Northern regions
and the capital, Amman. For young people,
social media is by far the most importantsource of information and means of communication. Just like everywhere aroundthe world, false information and hatespeech abound on social media with young
people particularly exposed. In addition,
there is stark gender inequality in the Palestinian media landscape. Even thoughmany more women than men graduate
from journalism programs, only very few
women work as journalists, let alone asmedia executives.
The DW Akademie country project underevaluation, funded by the Federal Ministry for Economic Cooperation and Development (BMZ), aims to address thesechallenges with two project components,

one for the Palestinian Territories (alsoaddressing Palestinians in Israel) and one
for Jordan. The two components work toward a common project objective to support “young people—especially young
women in the Palestinian Territories andalso refugees in Jordan—to use and produce media independently and on equal
footing.” Both components aim to scale up
the promotion of Media and Information
Literacy (MIL) among youth, in particular
by integrating MIL into school curriculaand extracurricular activities. The Palestine component also aims to strengthen
gender equality in the media.
The objective of the evaluation is to enableorganizational learning at DW Akademie,
support strategy development for the next
project phase (planned for 2023-2025) andprovide accountability to the BMZ. The
evaluation covered the ongoing project
phase running from January 1, 2020, to December 31, 2022. An impact study conducted as a part of the evaluation also included
activities from previous project phases.
The evaluation was kicked off in December
2021 and concluded with a change workshop with the main stakeholders on April
8, 2022. The assessment was based on thestandard six OECD/DAC criteria. In addition,

it focused on the cross-cutting criterion ofadaptive ­capacity. Each of the criteria wasbroken down into a set of specific evaluation questions which form the basis for theconclusions presented below.
The evaluation combined two evaluation
approaches, namely a contribution analysis of the whole project and a qualitative
ex-post impact assessment of MIL activities in the Palestinian Territories. The
impact assessment considered activities
from the current as well as two previous project phases (2014-2016 and 20172019). The evaluation relied on qualitative
methods, including document analysis,
24 semi-structured individual or groupinterviews (mainly remote), 7 focus group
discussions with a total of 50 high school
students (34 female, 16 male), and an interactive Theory-of-Change workshop withthe project teams. Initial data collection
coincided with the publication (on February 7, 2022) of the report of an external
investigation into allegations of antisemitism against individual staff members of
DW and individual partner organizations.
Its publication led to delays in data collection which, however, did not affect the validity of the evaluation results.

Project evaluation results
Relevance: Both project components are
consistent with the policy objectives andstrategies of the BMZ and the German
federal government. They adequately address the needs and priorities of the project partners who confirmed that activities
are planned jointly and that DW Akademie
regularly includes the partners’ ideas in
the advancement of the project strategy.
The Jordan component, for example, isclosely aligned with the national MIL strategy, published in 2019.
Both project components reacted in atimely and adequate fashion to changes in the broader economic, political,
and social context. The Palestine component faced the triple challenge of theCOVID-19 pandemic (since 2020), the flaring up of hostilities between Palestinians
and Israeli armed and security forces(since May/June 2021), and the graduallyshrinking spaces for civil society. It reactedwith changes to the project strategy andspecific measures to address these developments, all in close coordination withpartner organizations. In addition to challenges and delays caused by the COVID-19
pandemic, the Jordan component wasmainly affected by high personnel turnover both at the main partner organization
and all levels of government. Overall, therelevance of the project is rated as “comprehensively fulfilled.”
Coherence: For the purposes of this evaluation, internal coherence was defined as
the complementarity of the project activities to the partner’s own efforts and as
the synergies among the partners of eachproject component. The activities of bothcountry components are c­ omplementary

to the partners’ efforts. However, bothcomponents did not fully exploit the potential for cooperation and synergies amongpartners. For example, although the Palestine component had identified synergies
among partners as a strategic priority forthe current project phase, progress in creating these was successively hamperedby pandemic-related travel restrictions, areplacement of the project team, the worsening security situation, and uncertainties
and delays of some project activities after
the publication of the external investigation report.
External coherence was rated rather highfor the Jordan component. The work on
integrating MIL into the national school
curriculum and promoting it in youth centers fills a niche hitherto not addressed byinternational media development in Jor-dan. At the same time, activities fundedby other donors now also make use of theMIL training materials and the MIL trainer pool created together with the Jordan
Media Institute. As for the Palestine component, many avenues for more synergies
with other donors (such as UNESCO) exist,
in particular cooperation on a flagship MIL
center in the West Bank, to be built by thepartner organization Pyalara. However,the Palestine component did not fully develop its external coherence for the reasons already cited above. Overall, the coherence of the project is rated as “partiallyfulfilled.”
Effectiveness: The project set itself very
ambitious quantitative outcome indicators. In the judgment of the evaluators,
based on a thorough analysis of monitoring data and the project’s Theory ofChange, both indicators will most likely
not be reached by the end of the project
period. The main reasons for this are delays in the achievement of four of the six
underlying output targets, which in turnstem from the rescheduling of activities
necessitated by the pandemic and the reshuffle of the project team—both of whichwere outside the control of the project
staff. Although the contribution analysis
showed some minor inconsistencies in the
Theory of Change, these mainly result in a
possible underestimation of goal achievement and do not compromise the overallsoundness of the Theory of Change. It istherefore likely that many of the project’s
indicators will be reached at a later stage(e.g. during a possible follow-up project).
Pursuant to its focus on adaptive capacity,
the evaluation specifically looked at how
flexibly and effectively the project dealt
with emerging needs and changing contexts. Apart from the external challenges
addressed under the relevance criterion,
the reactions to two developments proved
to be important for the implementation of
the project. First, the project adapted very
swiftly and effectively to the challenges
presented to teambuilding and effective
collaboration by the pandemic. Therefore,
the DW Akademie project team was able
to keep the negative impact on its internal
process to a minimum. Second, the publication of the external investigation reportin February 2022 caused a major disruption of the project’s Palestine component.
The evaluation shed some light on possible

reasons for why DW Akademie handled thereport’s publication in a way that was met
with criticism by the Palestinian partnerorganizations. Overall, the effectivenessof the project is rated as “partially fulfilled.”
Efficiency: The assessment of this criterion
focused on the internal efficiency of administrative processes at DW Akademie, therole division between the project teams andthe partners, and the degree to which the
project’s responses to various crises werecost sensitive. Two evaluation questions
regarding DW Akademie’s regionalization
strategy and the funding modalities of BMZ
projects, which are also discussed in the efficiency section, did not factor into the efficiency assessment as they lie outside theproject’s sphere of influence.
The evaluation found that the efficiency ofinternal processes at DW Akademie has improved in recent years due to an increase inadministrative staff and an enhanced cooperation between DW Akademie’s contractand financial administration (PAD) and theprogram divisions. A remaining challenge
cited by partners and project staff alike isthe time needed to finalize both long andshort-term contracts. Efforts by the project teams and PAD to assist partner organizations in fulfilling the administrativerequirements by DW Akademie are largely
seen as helpful and effective.
In addition, the evaluation found that
there is some degree of disagreement
between DW Akademie and the partners
about the role division regarding the regional MIL network and the MIL center in
the West Bank. However, this does not result in inefficiencies but rather highlights
the need for a strategic decision by DW
Akademie as to which role it wants to playin both contexts in the future. Whereas athorough cost-benefit analysis was outside the scope of the evaluation, the qualitative assessment did not find any signs ofcost-insensitivity. All in all, the efficiency ofthe project is rated as “overall fulfilled.”

Impact: The first part of the impact assessment pertains to the whole projectand is based on the contribution analysis. As the outcome targets had not yetbeen reached (see above), the evaluators
mainly assessed the impact the projectis likely to make presuming the outcome

is realized at a later stage. As previous
phases of the Palestine component have
had a major impact by helping to integrate MIL into the school curriculum andpromoting extracurricular MIL activities
in selected schools, the planned MIL center in the West Bank will very likely play animportant role in scaling up MIL education
through extracurricular activities. Likewise, the work of the Jordan component
to integrate MIL into the national school
curriculum has the potential to improve
MIL skills among youth nationwide. Theevaluation found no unintended overarching effects of the Jordan component.
As for the Palestine component, it has had
the unintended positive consequence thatthe use of online tools promoted by DW
Akademie through their partners during
the COVID-19 pandemic will make it easier to include final beneficiaries from theGaza strip in future activities. The tensions
and challenges in the relationship with thepartner organizations that followed fromthe publication and handling of the external investigation report stands as a negative unintended consequence.
The second part of the impact assessment
considers the long-term effects of the project line that aims to strengthen the MIL
skills of high-school students in the Palestinian Territories and has been implemented by the partner organization Pyalaraover three project phases (2014-2016,2017-2019 and 2020-2022). The qualitative
impact study found evidence that interviewed beneficiaries acquired and/or improved their skills in accessing informationand media, analyzing and critically reflecting on the acquired information, acting on
their learnings regarding MIL and creating
media content themselves. Overall, thestudents rated the extracurricular activities as very useful and confirmed that they
had contributed to positively changing
their use of the internet and social media.
All in all, the impact of the project is rated
as “overall fulfilled.”
Sustainability
The assessment of sustainability focusedon the capacities of partners to sustain theoutcomes of the project without long-term
support from DW Akademie and on theproject’s contributions to strengthening
these capacities. In both project components, the efforts with the highest potential for sustainability are those focused on

promoting MIL in various forms in the public school system as there is a high stakeholder buy-in, continued political support,
and a willingness among partners to continue these activities. High staff turnoverat partner organizations, as well as limited
financial means to continue activities after
disengagement from DW Akademie, posethe greatest threats to the sustainability ofboth components.
The project was successful in strengthening the resilience and organizational
capacity of the partner organizations tosome degree. Positive examples are theprovision of psycho-social support andcapacity building in physical and digital
security for partners in the Palestinian
Territories and the building of capacities
in project steering, networking, and stakeholder management in Jordan. At the timeof writing, it was unclear how seriously
the publication of the investigation report
would affect the sustainability of project
activities in the Palestinian Territories.
Overall, the sustainability criterion is rated
as “partially fulfilled.”

DW Akademie

is Deutsche Welle’s center forinternational media development,journalism training and knowledgetransfer. Our projects strengthen thehuman right to freedom of expression
and unhindered access to information.
DW Akademie empowers people
worldwide to make independent
decisions based on reliable facts andconstructive dialogue. DW Akademie isa strategic partner of the German
Federal Ministry for Economic
Coopera­tion and Development. We
also receive funding from the Federal
Foreign Office and the European Union
and are active in approximately 60
developing countries and emerging
economies.

CONTAC TDr. Laura Moore
Head Research and Evaluation
laura.moore@dw.com
Berlin, Germany
DWAkademie@dw_akademie
DWAkademie
dw.com/newsletter-registration
dw.com/mediadev

The objective of the external evaluation is to provide a constructive assessment of the project results. It does not necessarily align with theperspective and planning of DW Akademie. Nevertheless, all findings were discussed in order to allow lessons learned to be incorporated into
the development of future strategies.