This project is fundedby the European Union

E VA LUAT ION R E P OR T 2 02 1

Executive summary
UkraineSupport to the National Public Broadcaster of Ukraine

dw-akademie.com

Background
The “Support to the National Public Broadcaster of Ukraine” project was implemented from 2017 to 2021 by a consortium of
three organizations – lead applicant DW
Akademie, co-applicant BBC Media Action(BBCMA), both respected organizations
with expertise in the field of media development, and the engineering consultancy
firm NIRAS as a second co-applicant. The
project is one of the largest projects to befinanced from the general budget of theEuropean Union, within the framework of
the “Commission Implementing Decisionof 26 July 2016 on Special Measure 2016
for Anti-Corruption and Support to Key
Reforms in favor of Ukraine.” Co-funding
was provided by the German Federal Foreign Office. The project aim and key objective was to promote the national publicbroadcaster of Ukraine (UA:PBC) throughthe establishment of a functioning digital
multiplatform newsroom and a training
center, the Public Media Academy (PMA).
As a result of the project, UA:PBC was supposed to improve its effectiveness andcreate a permanent foundation for quality, objective, and unbiased news content
for all segments of the population (television, radio, and new media both on thenational and regional levels) by meetingthe standards of a modern public broadcaster and improving its ability to produce
high-quality news programs, by providing
multiple viewpoints and news stories thatare shot and edited using professionaland modern production standards. Overall, the improved news products were expected to lead to the greater acceptanceof UA:PBC by the public, thus contributing
to the improvement of the information
environment in Ukraine and a higher understanding of the role of public service

broadcasting in the country.
The project also sought to create a permanent basis for quality, objective, andunbiased news through the involvementof a wide range of international and local
experts, in order to assure the provisionof organizational development support
to ensure the sustainable developmentof UA:PBC. Experts from the leading
non-governmental
Ukrainian
media
watchdog Detector Media supported the
project team in conducting media monitoring and the analysis necessary to evaluate UA:PBC’s progress in reaching its
key objectives of improved capacity andoutput.
The purpose of this final project evaluation was to provide an unbiased assessment of the extent to which the project
achieved its objective. The evaluation
methodology was grounded on the outcome-based approach through which theevaluation team explored the extent towhich the project activities produced anticipated outputs, and the extent to which
produced outputs generated outcomesleading to the accomplishment of the
project objective.
The initial evaluation design included desk
research, and data collection and analysis. The evaluation itself was expected
to be completed by the end of April 2022.
However, due to Russia’s full-scale invasion of Ukraine on the 24 February 2022,
the evaluation team suggested amending
the approach to account for the changing
circumstances. Thus, the evaluation design was expanded to ensure the evaluation captured all available direct and indi-

rect evidence of the project results, theirimpact and sustainability. In addition to
the initially envisaged review of the project documents, the evaluation team suggested expanding the desk research toinclude a review of open source studies
and available reports that provide information on UA:PBC’s rating and image, as
well as the quality of the produced content. Fieldwork was considered optional
and as such the evaluation team could
proceed if operational conditions allowed
for arranging key informant interviews
with UA:PBC staff and representatives ofother relevant stakeholders. In the end,
the evaluation included: expanded desk
research, which covered the project documentation review as well as searches andreviews of other relevant studies, reports,
and data; fieldwork, comprised of key informant interviews with the project team,the representatives of UA:PBC and other stakeholders; and an online survey of
UA:PBC producers and editors. Additionally, with the approval from the EU Delegation, the project director suggested an
extension of the evaluation term until theend of 2022 to ensure the largest amountof information on the project’s results andimpact possible. The evaluation activities
thus took place from February to November 2022, while the final evaluation report
was prepared and submitted in early December 2022. This allowed the evaluators
to not only engage in sufficient observations regarding the impact of the projectand its sustainability over the additional
period, but also to consider UA:PBC’s performance in wartime.

Project Evaluation Results
The evaluation found that the project accomplished all anticipated results to theextent possible, allowing UA:PBC to keep
moving toward strengthening its position in the Ukrainian media landscapeas an independent, unbiased media outlet addressing the informational needsof various segments of the population
through the production and delivery ofhigh-quality news content.

Relevance. The project was a timely opportunity to support UA:PBC's transition
from state-owned media company topublic service broadcaster. The transition and reform process required not
only improvements to the broadcaster’s
technical capacity, changes at the organizational and management level, whichwere crucial for its continued consistentand efficient development, and build-ing capacity at various levels, including
management, production, editing, andjournalistic output. The project outputs,
their impact and sustainability proved its
timeliness and relevance.
Coherence. The project was in line with
Ukraine's efforts to comply with the 2014
EU-Ukraine Association Agreement interms of ensuring freedom of expression

and a stable, impartial media environment. UA:PBC, established as the public
service broadcaster in early 2017, wasconsidered a platform for reliable, independent, and objective reporting, andthus able to support Ukraine in fulfilling
the Agreement’s requirements. The efforts of DW Akademie and BBCMA werecomplemented by a further set of donor
projects implemented within the same
timeframe and aimed at supporting thereform process and capacity building inareas not envisaged within this project.
Among these complementary initiatives,
it is worth mentioning a project funded by the Swedish International Development Cooperation Agency, throughwhich UA:PBC was able to create and promote its official website and develop thecapacity of its regional branches, whichis also crucial for the harmonization of
the reform process across the entire organization.
Effectivity. The project outputs, such as
the modern multimedia digital newsroom, revised organizational structure,
re-organized news production workflow,
and the establishment of the Public Media Academy – an in-house learning hub– provided UA:PBC with a solid basis forsteady progress toward the improvement of its effectiveness and capacity
to generate quality, objective, and unbiased news content for all segments ofthe population beyond the project end
date. UA:PBC and other stakeholders
considered the creation of the convergent newsroom as a flagship output,
which became a jumping-off point for
accelerating further internal transformations and a support during an extremely
challenging time.
Efficiency. Despite delays related toboth the construction of the newsroom
and the COVID-19 pandemic, the project
team and UA:PBC effectively managed
the project’s implementation throughregular meetings, addressing the needs
revealed by the public broadcaster’s
analysis, finding the most feasible solutions for challenges as they occurred. The
partnership between DW Akademie and
BBCMA demonstrated efficient teamwork and concordance. Each was able toleveraged their expertise, which resulted
in the balanced development of organi-zational capacity in terms of workflows,
improved technical capacity thanks to
new equipment, and staff able to follow
new workflows and produce quality news
content. The project team’s engagementin the coordination of the assistance to
UA:PBC from other donors allowed the
project to avoid double funding the same
segments of the reform process.
Impact. The evaluation showed that
outputs produced have the potential tofurther support the UA:PBC’s mission to
become a strong, independent publicbroadcaster. In fact, there is some evidence that the instigated changes are
facilitating the development of systemic change. The public broadcaster succeeded in increasing its audience reachthrough digital communication channels. For instance, the official UA:PBC
Telegram channel (launched in 2019) has
more than 300,000 subscribers as of the
final evaluation report submission date.
While such audience numbers may seemsmall in comparison to other Ukrainian
national media outlets, there has beenslow but sustainable growth and an increase in the level of trust in UA:PBC asa source of information since 2017. It is
worth mentioning that the newsroom
was officially launched in late 2021, andonly a few months later, Ukraine was atwar, which made it very complicated tofollow the desired development strategies as planned. In terms of TV audiences, there was a significant increasein 2021, from 1.97 million unique users
in 2020 to 4.5 million unique users andthirteen million views in 2021. While thenewsroom was still in the process of creation, improvements in the performanceof journalists, producers, and editors
was visible through the improved qualityof the content produced, from compliance with most of journalistic standards
to an improved visual presentation andquality.
In addition, the newsroom became a central point for the progress of UA:PBC operations toward cost-efficiency. Currently, the premises of the newsroom host
Kyiv-based teams, which were previously located in a 24 story building known
as the “pencil”, maintenance of which
drained the lion’s share of the broadcaster’s budget, especially for heating. As of

8 November 2022, UA:PBC officially put
the “pencil” building out of service.
The ability of UA:PBC to provide staff
capacity building through the establishment of the PMA positively influenced
UA:PBC’s image as an employer, as well
as allowing it to address its own mosturgent needs in terms of acquiring new
skills or upgrading existing ones. For instance, since the beginning of Russia’s
invasion of Ukraine, the PMA adjusted its
training agenda to build the capacity ofjournalists to work under wartime conditions.
Sustainability. The sustainability ofthe produced outputs was proven immediately through the wartime conditions. For instance, the team managed
to relocate quickly to Lviv and continue
delivery of news content in the framework of a national tele-marathon. Later,
the team came back to the newsroom
in Kyiv and thus to relative operational
normality. During such turbulent andthreatening times, UA:PBC preserved all
approaches, methods and equipment
delivered during the project’s implementation. Considering the UA:PBC’s
previous capacity, including in terms oftechnical skills, one can state that without the progress made in response to
the project’s implementation, the publicbroadcaster would likely been unable tocope with the stressful and challenging
operational environment of 2022. The
evaluation found that, even in wartime,
the broadcaster’s team was looking foropportunities to develop and improve its
performance.

Lessons learned
Both the project team and UA:PBC managed to adapt the course of the project tochanges in the operational environment
caused by the COVID-19 pandemic, the reduced engagement of one of the consortium members, the resulting redistribution of roles and responsibilities between
the remaining partners, and legal challenges related to the construction of thenewsroom.
Both UA:PBC and DW Akademie gained thecapacity to manage and implement tasks
outside of their key expertise. The UA:PBC staff who were actively engaged in theconstruction of the newsroom not only
gained an opportunity to create a space

DW Akademieof their dreams, but were also involved in
specific workflows and processes related
to construction and renovation. Once thesituation in Ukraine allows, they will beable to use this experience in the buildingof regional hubs. As project implementer,
DW Akademie can use the experiences
gained in construction and renovation asa unique asset in the future to avoid possible pitfalls, and strengthen its project crisis management and resilience to specificstresses.
Building partnership with local partners
based on trust is important for ensuring
ownership over the implemented changes
and their sustainability for the future.

Selected Recommendations
To ensure assessment consistency, as
well as the ability to track evidence-based
progress on performance improvement
within specific domains, it is important to
use the same assessment methodology
over the entire course of the project. Whenplanning further organizational development activities, the members of the consortium can use available organizational
development assessment tools, adapting
them to the area in question, to streamline
and simplify the process of organizational
development planning and progress assessment (for example, the media-specificorganizational capacity assessment (OCA)
tool developed by Internews, or any otherstructured tool, including one developedby the implementers themselves). The application of such a tool provides a structure
for the needs assessment, implementationof any required changes and the analysis of
any results. UA:PBC is a unique beneficiaryin the Ukrainian media landscape, as it hasreceived the support of numerous donors,
with an emphasis on assistance for organizational development. It would therefore

be beneficial for UA:PBC to adapt the organizational capacity assessment approach,which would allow the team to track its
progress over a longer time-span, ensure
proper documentation of the processes
and any progress, as well as offer donors
a uniform approach and reporting mechanism.
Knowledge management should be an important component not only of the project
itself, but of UA:PBC’s overall learning process. In this case, the human resource department could play a key role in crafting astrategy and ensuring that there are trained
professionals who can provide top and middle management with mentoring, support
and training. A review of the experience
gained and its documentation, including
processes and lessons learned, should bepart of any strategic planning and strategy
reviews as well as one of the key elementsof staff performance reviews.

is Deutsche Welle's center forinternational media development,journalism training and knowledgetransfer. Our projects strengthen thehuman right to freedom of expression
and unhindered access to information.
DW Akademie empowers people
worldwide to make independent
decisions based on reliable facts andconstructive dialogue.
DW Akademie is a strategic partnerof the German Federal Ministry
for Economic Cooperation andDevelopment. We also receive funding
from the Federal Foreign Office andthe European Union and are active inmore than 70 developing countries
and emerging economies.

CONTAC T
Dr. Laura Moore
Head Research and Evaluation
laura.moore@dw.com
Berlin, Germany

DWAkademie@dw_akademie
DWAkademie
dw.com/newsletter-registration
dw.com/mediadev

Text edited by DW Akademie.

ThisobjectiveThe
publicationof thehasexternalbeen produced
evaluation
withistheto provide
assistancea constructiveof the European
assessment
Union andof thetheproject
Germanresults.
FederalIt does
Foreign
notOffice.
necessarily
The contentsalign withof thethisperspective are
publication
andtheplanningsole responsibilityof DW Akademie.
of Deutsche
Nevertheless,Welle andall findings
can in nowere
way discussed
be taken to
inreflectorder totheallowviewslessonsof the European
learned toUnion
be incorporated
or the German
into
the development
FederalForeign Office.
of future
Thestrategies.
objective of the external evaluation is to provide a constructive assessment of the project results.