Paraguay ECT Final Report

i

Office of 

Technical

Assistance: 

Rwanda 

Revenue 

Program

Program Evaluation 

Report

Final Submission

Aug 1, 2023
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

i

Table of Contents

I. EXECUTIVE SUMMARY .................................................................................................. 1

KEY FINDINGS AND CONCLUSIONS .............................................................................................. 1

KEY RECOMMENDATIONS............................................................................................................ 3

II. EVALUATION OVERVIEW.......................................................................................... 5

PROGRAM DESCRIPTION .............................................................................................................. 5

EVALUATION PURPOSE ................................................................................................................ 6

EVALUATION QUESTIONS ............................................................................................................ 7

EVALUATION METHODOLOGY ..................................................................................................... 7

DATA COLLECTION ...................................................................................................................... 8

QUALITY ASSURANCE.................................................................................................................. 8

LIMITATIONS................................................................................................................................ 9

III. FINDINGS AND CONCLUSIONS................................................................................. 9

PROGRAM COMPONENT 1: KEY PERFORMANCE INDICATORS (KPIS)......................................... 10

PROGRAM COMPONENT 2: COMPLIANCE IMPROVEMENT PLANNING.......................................... 13

PROGRAM COMPONENT 3: TAX DEBT COLLECTIONS ................................................................. 15

PROGRAM COMPONENT 4: SPECIALIZED SECTOR AUDIT............................................................ 22

PROGRAM COMPONENT 5: COMPUTER-ASSISTED AUDIT........................................................... 25

PROGRAM COMPONENT 6: GENERAL AUDIT CAPACITY............................................................. 28

PROGRAM COMPONENT 7: TAX DISPUTE RESOLUTION .............................................................. 32

PROGRAM COMPONENT 8: INTERNAL AUDIT ............................................................................. 36

PROGRAM COMPONENT 9: RISK MANAGEMENT (INFORMATION TECHNOLOGY RISK 

MANAGEMENT).......................................................................................................................... 39

EVALUATION QUESTION 3 ......................................................................................................... 44

IV. PROGRAM LEVEL RECOMMENDATIONS........................................................... 44

PROGRAM DESIGN CONSIDERATIONS......................................................................................... 44

STAFFING AND ONBOARDING CONSIDERATIONS........................................................................ 46

V. ANNEXES ........................................................................................................................... 48

ANNEX A. LIST OF DOCUMENTS REVIEWED .............................................................................. 48

ANNEX B. KEY INFORMANTS INTERVIEWED .............................................................................. 49

ANNEX C. EVALUATION TEAM .................................................................................................. 50

ANNEX D. EVALUATION TEAM’S RESPONSE TO THE OTA REVENUE TEAM’S COMMENTS ......... 51

ANNEX E. OTA RESPONSE TO “OFFICE OF TECHNICAL ASSISTANCE: RWANDA REVENUE 

PROGRAM EVALUATION REPORT” ............................................................................................. 52
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

ii

This report was prepared by Bixal

August 1, 2023

Bixal Solutions, Inc. 

3050 Chain Bridge Road, Suite 305

Fairfax, VA 22030 

info@bixal.com 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

iii

ACRONYMS & ABBREVIATIONS 

CAA Computer-Assisted Audit

CAS Computer Audit Specialist

CIP Compliance Improvement Plan

DTD Domestic Tax Division

EO Enforcement Officer

EQ Evaluation Question

FATAA Foreign Aid Transparency and Accountability Act

IA Internal Audit

ICTD International Center for Tax and Development

IRS Internal Revenue Service

IT Information Technology

KII Key Informant Interview

KPI Key Performance Indicator

Logframe Logical Framework

M&E Monitoring and Evaluation

MIS Management Information System

OTA Office of Technical Assistance

QRS Quality Review Staff

RRA Rwanda Revenue Authority

TA Technical Assistance

TADAT Tax Administration Diagnostic Tool

TIATA Treasury International Affairs Technical Assistance

USAID United States Agency for International Development
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

1

I. EXECUTIVE SUMMARY

From 2015–2020, the U.S. Treasury’s Office of Technical Assistance (OTA) engaged with the 

Rwanda Revenue Authority (RRA) to increase domestic resource mobilization in support of 

Rwanda’s economic and development priorities. OTA focused on building technical capacity in 

nine program components: 1) key performance indicators (KPIs) and reports; 2) compliance 

improvement planning; 3) debt collection; 4) specialized sector audit; 5) computer assisted audit; 

6) general audit; 7) tax dispute resolution; 8) internal audit; and 9) risk management, including 

information technology (IT) risk management. Over the course of the program, OTA provided 

technical assistance by periodic deployment of short-term expert advisors who worked closely 

with RRA counterparts. 

In compliance with the Foreign Aid Transparency and Accountability Act (FATAA) 

requirements, Bixal1conducted a third-party, summative, program evaluation of OTA’s Rwanda 

Revenue Authority Program, under contract to OTA. The evaluation of the OTA/Rwanda 

revenue sector program is meant to provide OTA with in-depth, third-party, retrospective 

insights of its activities in Rwanda and lessons learned for future programming. 

The qualitative evaluation methods included a desk review of key project documents and 22 key 

informant interviews (KIIs) with the OTA advisors and RRA counterparts. 

The evaluation focused on the following evaluation questions (EQs):

EQ 1. Which of the expected project outcomes were achieved? 

EQ 2. Which of the achieved outcomes have been sustained up to the present?

EQ 3. OTA took a new approach to program design in Rwanda by involving 

counterparts in the design and writing of key operational manuals. Did OTA’s new 

approach contribute to the sustainability of the outcomes, and if so, how? 

EQ 4. For any expected outcomes that were not achieved, which factors hindered 

success?

Key Findings and Conclusions

PROGRAM COMPONENT 1: SUPPORT FOR THE DEVELOPMENT OF KEY 

PERFORMANCE INDICATORS (KPIs)

While RRA had already adopted some KPIs (related to audits) at program inception, OTA’s 

assistance was limited to some training for audit and debt staff. Further development of the KPI 

component was limited because RRA was planning to implement a new Management 

1 www.bixal.com
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

2

Information System (MIS) system that will support expanded collection and reporting of KPI 

data. The evaluation team found that data already collected by RRA were not currently utilized

to their full extent—data were used at the managerial level but not at a corporate or strategic 

level.

PROGRAM COMPONENT 2: COMPLIANCE IMPROVEMENT PLANNING

(TAXPAYER)

OTA’s engagement with this component was limited because it was discovered that RRA 

already had a compliance improvement plan (CIP) in place. However, OTA initiated an idea that 

RRA commission a taxpayer behavior study, which was commissioned late in the program cycle 

(2020). OTA assisted with updating the CIP as informed by the study findings.

PROGRAM COMPONENT 3: TAX DEBT COLLECTIONS (ARREARS)

OTA provided overall guidance on debt policies to be incorporated into RRA’s tax debt manual, 

including extending the payment period for delinquent accounts. However, the evaluation team 

concluded that OTA’s advice in this case may lead to increased delinquencies, as some taxpayers 

may take advantage of the extended leniency and withhold tax payments. They also concluded 

that OTA’s support did not help RRA address a growing issue with the lack of write-offs that 

distorts RRA’s accounting. The RRA needed to revise its write-off approval processes before 

they could apply OTA’s technical advice which led to delays. Finally, OTA introduced the use 

of a timesheet to track “productive” time (engaged in cases) and “unproductive” time (meetings, 

travel, etc.) of auditors. However, the timesheet was not well received and is only used to

measure auditors’ performance rather than improve the efficiency of RRA’s audit department.

PROGRAM COMPONENT 4: SPECIAL (BUSINESS) SECTOR AUDITS

OTA provided specialized training to a select group of RRA’s auditors on the construction sector 

businesses. However, RRA deployed the trained auditors back to their previous general-sector 

audit work and, as a result, the newly trained auditors lost the ability to apply and develop their 

newly acquired skills to the construction sector.

PROGRAM COMPONENT 5: COMPUTER ASSISTED AUDITS

While providing some training to prepare key RRA staff for future adoption of computer-assisted 

audit practices, OTA did not achieve most of its objectives for this component because RRA did 

not have an MIS or software in place to support computer-assisted audits.

PROGRAM COMPONENT 6: GENERAL AUDIT CAPACITY

OTA’s key achievement under this component was supporting RRA to produce a high-quality,

detailed audit manual. This manual has been digitized by RRA and is frequently updated to 

maintain its utility and relevance. Moreover, RRA has since initiated the development of other 

procedures manuals without inputs from OTA. In addition, OTA introduced enhancements for 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

3

RRA’s quality review process for audits that remain updated and useful. However, the 

evaluation team concluded that OTA’s recommended enhancements to the audit manual may 

have been too detailed and included too many prescriptive procedures. While the inclusion of 

detailed audit cases was intended to eliminate ambiguity across a wide range of audit cases, 

OTA’s approach should have been to create high-level guides and build capacity of the auditors 

on how to make decisions.

PROGRAM COMPONENT 7: TAX DISPUTE RESOLUTION

OTA helped draft new dispute resolution policies, advised on legal support for consistent, fair 

and effective dispute processes, and provided training to enforcement staff on improved 

procedures. However, despite these outputs, RRA’s dispute resolution process remains largely 

the same because not all affected departments were consulted in this process. More importantly,

there was inadequate outreach and education to taxpayers to raise awareness of the changed 

resolution process.

PROGRAM COMPONENT 8: INTERNAL AUDIT

OTA reviewed RRA’s internal audit procedures, recommended revisions including enhanced 

reporting, and developed a process strategy for addressing RRA’s backlog in aged, unaddressed 

past internal audit recommendations. However, OTA did not follow up with internal audit 

training or support for related interdepartmental communication practices that would capitalize 

on the assessment activities and support RRA to engage internal audit practices more fully.

PROGRAM COMPONENT 9: RISK MANAGEMENT (INFORMATION 

TECHNOLOGY RISK MANAGEMENT)

OTA assessed RRA’s physical and IT risk environment, supported RRA in developing a risk 

register used to identify and manage risks, and provided risk management training to appropriate 

RRA staff on a regular basis. Follow-up activities included building security enhancements, 

using staff key cards to control access to different building floor sections, and developing an IT 

security plan for equipment, password, and network security practices. However, the evaluation 

team found that security measures that protect taxpayer confidential records remains weak, 

indicating that IT security remains at risk. Likewise, a comprehensive risk management plan for 

taxpayer information and records that includes risk reduction and disaster recovery planning is 

needed.

Key Recommendations

While OTA did make progress in achieving some outputs and outcomes at the component level, 

the evaluation team found that the design and approach to implementation could be strengthened. 

This section contains high-level recommendations that relate to Rwanda but may also be 

applicable to OTA’s work more broadly. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

4

PROGRAM DESIGN CONSIDERATIONS

Theory of Change 

A theory of change (or similar) should have been created at an early stage of the project. The 

process to create the theory of change would have helped the participants to better identify the 

development needs and to obtain a common understanding of a feasible way forward. The 

process should have followed a common process, using standard formats and terminology. 

Benchmarking at Program Inception

The theory of change could also serve as the basis for establishing a monitoring framework to

measure progress towards outputs and outcomes over the course of the project implementation. 

Baselines and targets should be included to set expectations and to serve as a point of 

comparison. 

Systems Approach 

The evaluation team recommends that OTA advisors take more time to understand the 

counterpart’s organization, business processes, and priorities prior to developing a workplan. 

This would ensure the organization is ready for the activity and is able to implement the changes 

and sustain them after the OTA engagement. This system’s approach would also ensure OTA 

advisors understand the interdependencies and can address barriers to specific changes.

STAFFING AND TRAINING CONSIDERATIONS

Selection of Advisors

Many OTA advisors were very senior but lacked practical day-to-day operations expertise to 

translate recommendations to frontline workers to implement the reforms. OTA should consider 

the composition of the teams of advisors to encompass a wider range of experiences and 

backgrounds. While OTA engaged a local project coordinator to provide communications, 

logistics, and administrative support for the program, the evaluation team recommends that OTA 

also needs a higher-level technical role with specific country knowledge and capable of working 

across multiple program component areas. 

Orientation Package

The evaluation team also identified that an orientation package for the intermittent advisors is a 

missing component of training. It should have included information from RRA staff on 

organizational structure and identified issues, business processes, and information on Rwandan 

cultural and their political and legislative process. Technical assistance would be more effective 

if local cultures were understood.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

5

II. EVALUATION OVERVIEW

Program Description

Since 2015, the U.S. Department of the Treasury’s OTA has provided technical assistance in 

public financial management to Rwanda using a resident-based project coordinator and 

intermittent expert advisors who worked closely with counterparts in RRA. The goal of the 

project has been to support RRA's efforts to increase domestic resource mobilization in support 

of Rwanda's economic and development priorities by building its capacity in nine areas:

1. KPIs and reports.

2. Compliance improvement planning.

3. Debt collection.

4. Specialized sector audit.

5. Computer assisted audit.

6. General audit.

7. Tax dispute resolution.

8. Internal audit.

9. Risk management, including IT risk management.

At the time this project began, RRA was reasonably well developed, compared to its regional 

counterparts, and had motivated, educated employees. Third-party Tax Administration 

Diagnostic Tool (TADAT) assessments were conducted in both 2015 and 2019.

2 These 

assessments identified some deficiencies to be addressed by RRA. Some of the main issues 

identified were:

• RRA’s strong performance of promoting voluntary compliance was outweighed by the 

relatively weak execution of audits and debt management functions, resulting in a high 

rate of tax payment arrears.

• RRA had an established risk management structure but lacked the depth needed for a 

comprehensive compliance improvement strategy, and it faced challenges to identify and 

mitigate corporate risks, especially those in the IT sector.

• The cumbersome dispute resolution process required high-level internal approvals of 

relatively minor individual taxpayer objections, causing undue delays. 

• Auditors did not have the specialized skills required to audit specialized sectors, such as 

the construction industry, or engage in complex computer-assisted audits.

2

Internationally recognized tax administration assessments conducted by the TADAT organization.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

6

• Internal auditors were not trained to conduct high-quality audits, and there was no 

overarching process to monitor departments’ responses to audit recommendations. 

• RRA did not use KPIs to monitor organizational progress.

According to the OTA advisors, these assessments informed the annual work plans, which in 

turn constituted the basis for the drafting of the statements of work that outlined the activities for 

OTA advisors engaged with RRA. Advisors’ intermittent visits typically lasted two weeks. The

visits included following up on the advice given during an earlier visit, discussions with RRA 

staff, new assessments of RRA capacities and operations, and the presentation of new or updated 

advice to the senior management at RRA.

A Program Manager who also served as an advisor coordinated OTA’s activities and provided 

technical oversight and produced monthly and annual reports.

At the close of the program, the OTA team provided RRA’s Commissioner General with future 

recommendations. The recommendations focused on RRA’s Domestic Tax Division (DTD) in 

the areas of quality assurance, human resources, administration, and training. The OTA team 

also drafted a six-month action plan to implement the recommendations. The registers and the 

action plan were included with the final Closing Memo to the Commissioner General. 

Evaluation Purpose

OTA commissioned Bixal to carry out a summative program evaluation. The evaluation 

activities took place during January–July 2023, with a field visit by the evaluation team to 

Kigali, Rwanda, during the period of March 23–April 9, 2023. 

This summative program evaluation was designed to serve two purposes. The first is for OTA to 

comply with the Foreign Aid Transparency and Accountability Act (FATAA) of 2016.3 The 

second is to provide OTA with in-depth, third-party, retrospective insights of its project activities 

in Rwanda. Some aspects of OTA’s experience in Rwanda may be extrapolated to other current 

or future projects in Rwanda or other countries to inform adaptation, improve projects, discover 

ways to reduce the time to achieve project outcomes, identify new ways to effect positive 

change, and document project achievements and success.

3 Foreign Aid and Transparency and Accountability Act of 2016, Public Law 114-191 

https://www.congress.gov/114/plaws/publ191/PLAW-114publ191.pdf
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

7

Evaluation Questions

Bixal’s evaluation team collaborated with OTA’s revenue and RRA teams to develop the 

following evaluation questions (EQs). The EQs provided the foundation for the remaining 

design elements of the evaluation, such as methodology and approaches to data collection and 

analysis. 

EQ 1. Which of the expected project outcomes were achieved? 

EQ 2. Which of the achieved outcomes have been sustained up to the present?

EQ 3. OTA took a new approach to program design in Rwanda by involving 

counterparts in the design and writing of key operational manuals. Did OTA’s

new approach contribute to the sustainability of the outcomes, and if so, how? 

EQ 4. For any expected outcomes that were not achieved, which factors hindered 

success?

The evaluation findings found in Section III are grouped by OTA’s nine program components, 

each of which includes multiple objectives. For each program component objective, the report 

addresses findings, conclusions, and recommendations for EQ 1, EQ 2, and EQ 4. Note that 

EQ 3, a high-interest line of inquiry for OTA, is focused on assistance with developing 

operational manuals and is broken out and discussed explicitly at the end of Section III. The 

evaluation team determined that EQ 3 findings are integral to several program components, so 

EQ 3 is largely addressed in context together with the other EQs.

Evaluation Methodology 

The evaluation methods included a desk review of nearly 350 project documents provided by 

OTA, followed by individual interviews of OTA’s advisors, their counterparts, and other key 

respondents. The overall evaluation design was developed using a theory-based approach with 

OTA’s program logical framework (logframe) as a starting point, which divided the program into 

nine project components. The evaluation team produced an evaluation framework (see Annex 

A) that mapped specific areas of inquiry for each of the nine project components. The evaluation 

team used this to develop detailed interview guides that were tailored for each KII. The KIIs 

provided multiple perspectives and sources that support findings related to the EQs. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

8

Data Collection

OTA and RRA shared vital project documents with the evaluation team. Among these

documents were the terms of reference for the program, the program logframe, workplans, trip 

reports, statements of work, monthly reports, project narratives, project performance reports, a 

debt management manual, closing memo, the end of project review, and an evaluation 

coversheet. Annex B contains a detailed list of the documents used by the evaluation team. The 

team reviewed each document to provide context for the program, as well as to elicit findings to 

add to the base of evidence.

The desk review was complimented by 22 KIIs, 18 with RRA officials and 4 with the OTA 

advisors. Of the 22 interviews, 15 were conducted in person and seven were conducted using 

virtual technology. A detailed list of key respondents is included in Annex C. 

DATA ANALYSIS PROCESS 

The evaluation team conducted qualitative analysis of KII data on an ongoing basis, concurrent 

with interviews, to ensure quality in real time and to identify findings and trends in a timely 

manner. Interview notes were completed by the team members and voice recordings were made 

with the respondent’s consent. A selection of KIIs were also transcribed to cross-check and 

validate for coding. 

KII notes and transcriptions were coded and analyzed using ATLAS.ti, grouping similar data 

under different categories and themes. Thus, the evaluation team could locate, retrieve, and 

combine the textual data that corresponded to a category of interest. These data were further 

triangulated and contextualized with the findings from the desk review.

When the data collection was nearly complete, the evaluation team members convened for a joint 

analysis. The evaluation team discussed potential findings, conclusions, and trends. Insights and 

ideas from this process were shared in a presentation of preliminary findings to OTA for vetting 

and discussion before the final evaluation report was drafted. 

Quality Assurance

The evaluation team followed standard quality assurance best practices that included the 

following:

1. Notetakers uploaded their notes at the end of each day of data collection. Notes were

reviewed to ensure completeness and comprehension and to identify emerging themes. 

2. The evaluation team met at the end of each day of data collection in Kigali. The team 

discussed what they heard and observed to ensure consistency in collecting and reporting 

data.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

9

3. The ATLAS.ti application enabled the team to check for accuracy and completeness of 

qualitative data as they were entered and coded. 

4. The joint analysis and presentation of preliminary findings provided further opportunity 

to cross-check and validate data and findings.

The evaluation team took steps to ensure data security, including using secure communication 

platforms, storage of information on secure and password-protected sites, and deletion of 

recording and transcription files after the completion of the study.

Limitations

In general, the evaluation team faced very few limitations. Most KIIs could be carried out as 

planned and sufficient documentation was provided for a comprehensive desk review. However, 

a few circumstances limited the data collection and consequently the analysis:

• Lack of program framework, action plan, coherent documentation

• The Logframe was designed very late into the project and did not include sufficient 

indicators that could help measure progress over the life of the project.

• For some respondents, it was difficult to remember, in detail, the activities that OTA had 

carried out and to what activities OTA advisors had contributed. 

• Some staff had left RRA and were not available for interviews. 

• In most cases, KIIs with Rwandan counterparts were conducted offsite, outside their offices.

This was due to both the lack of quiet meeting spaces, a result of ongoing construction in 

RRA facilities, and the counterparts’ frequently stating their preference for interview privacy 

away from their colleagues. While this had some advantages for candid discussions with the 

evaluation team, it also limited opportunities for direct observation of RRA’s workspace and 

environment.

• The structure of the workplans changed from year to year and were not fully compatible with 

the program logframe. This made longitudinal tracking of achievements difficult.

III. FINDINGS AND CONCLUSIONS

Section III presents each of the evaluation findings for OTA’s nine program components for the 

RRA program. Section IV discusses additional conclusions and recommendations at a systems

level and considers OTA’s overall program.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

10

Program Component 1: Key Performance Indicators (KPIs) 

COMPONENT 1 OBJECTIVE 

Component 1 focused on assisting RRA’s leadership in adopting and using KPIs to monitor 

and manage organizational performance, set expectations, and allocate resources in key tax 

administration functions, especially in audit and debt management. The planned project 

activities included: 1) designing KPIs; 2) identifying data sources; 3) training staff on collecting 

KPIs; and 4) training on how to analyze KPI data.

In assessing this objective, the evaluation team considered that KPIs need to be developed 

carefully to align with activities, provide baselines and targets, identify data sources, collect data,

and, finally, analyze and communicate data to decision makers to contribute to the operation of 

the program. 

Table 1 summarizes the findings for Component 1, and a detailed discussion follows the 

summary.

Table 1: Component 1 Outcomes—Activities and findings

Expected Outcome 1.1: RRA develops and regularly collects KPI data. RRA leadership uses 

KPI data to measure the efficiency/effectiveness of core functions (i.e., audit and debt 

collections), set expectations for organizational performance, and allocate resources in 

accordance with organizational goals.

Activities Findings

1.1.1: Assist RRA Planning and 

Research Director to identify 

and design RRA KPIs

• RRA was collecting some KPI data manually, mostly 

focused on the number and types of audits.

• OTA assisted RRA to update and further develop an 

existing KPI template.

• RRA updated KPI measures with OTA’s support.

• Counterparts reported satisfaction with OTA’s assistance

and that department managers did continue to use KPIs. 

However, KPIs were not used at the corporate or 

strategic level. 

1.1.2: Identify data sources to 

enhance KPIs 

• OTA assisted RRA to identify relevant data; however,

they did not have the appropriate MIS in place to begin 

collecting the information.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

11

1.1.3: Train specialized audit, 

debt, and planning and research 

staff to collect KPI data 

• Some audit and debt staff were trained in the use of 

KPIs, but this was on a limited scale. 

• This activity was delayed because the MIS needed to be 

developed prior to the training.

1.1.4: Train management to 

analyze and use KPIs

• OTA was not able to train management staff on KPIs, as 

the MIS system for collecting more strategic level KPIs 

was not yet in place. 

COMPONENT 1 FINDINGS

EQ 1: Which of the expected project outcomes were achieved?

Outcomes Status: Partially achieved

The outcomes for Component 1 were partially achieved. The evaluation team found that RRA 

has developed KPIs and regularly collects KPI data at the managerial level. These data were

collected manually and tracked in an Excel document. The information collected was minimal 

and, during the tenure of the OTA project, RRA leadership did not use KPI data to set 

expectations for organizational performance or to allocate resources. KPI data continue to be 

used only to perform basic revenue functions (e.g., measuring the efficiency and effectiveness of 

the core functions of audit and debt collections) but do not inform higher-level decision-making. 

EQ 2: Which of the achieved outcomes have been sustained up to the present?

Outcomes Sustained: Monthly KPI reports

The evaluation team found, through counterpart interviews, that RRA continues to produce five 

monthly KPI reports that were supported by OTA advisors, so this practice is sustained. This 

information can inform day-to-day operations; however, without a broader organizational effort, 

there continued to be missed opportunities.

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success? 

Hindering Factors: 1) No single, coordinated technical assistance action plan; 2)

Lack of organizational level MIS for KPI data collection and analysis

The evaluation team found significant differences regarding the formulation of the achieved 

objectives as reported in the End of the Project Review Document compared to the objectives for 

each program component in the project logframe document. This indicated that OTA did not 

follow a single, coordinated technical assistance action plan among the multiple advisors to 

guide activities and track progress across the duration of the program.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

12

Additionally, the evaluation team learned from counterpart interviews that RRA lacked an MIS

with the functionality to collect and consolidate KPI data across the organization. Due to this, 

OTA was unable to provide training to staff on how to produce usable analysis that leadership

could use to set expectations for organizational performance and allocate resources in accordance 

with organizational goals. 

COMPONENT 1 CONCLUSIONS

OTA provided relevant advice to RRA on Component 1, but RRA lacked relevant IT 

infrastructure to execute fully. 

The evaluation team found that RRA had been using some KPIs to track audits prior to OTA’s 

assistance and RRA was interested in further developing additional indicators and using data to 

inform management decisions. Unfortunately, RRA lacked an organization-wide MIS to collect 

and process KPI data. OTA provided relevant advice and RRA retained one Oracle contractor to 

make improvements to their MIS. Overall, this component is still in progress toward OTA’s 

anticipated outcomes indicated in the program logframe. Training did not happen as planned as

the MIS was not yet in place.

COMPONENT 1 RECOMMENDATIONS

Continue improvements in reaching and sustaining outcomes by adopting a formal approach to 

KPI technical assistance; forming a dedicated unit of technical experts focused on knowledge

management and implementation; collaborative use of a plan/tracker; clearly defining future 

steps to achieving outcomes and measuring progress.

A formal approach by OTA to delivering technical assistance on the development and use of 

KPIs could have produced more substantial and sustainable outcomes. For example, when RRA 

requested support on KPIs, securing agreement on the use of KPIs from all management levels at 

RRA would have achieved the expected outcome under Component 1. 

The formation of a dedicated unit composed of technical experts to track, manage, and 

implement OTA’s advice and continue implementing the new knowledge gained from OTA’s 

support may have provided better and more sustained outcomes for RRA. RRA maintained their 

own internal tracker, but we recommend using an agreed-upon action plan/tracker between 

counterparts and OTA for any future programming.

Structuring the work plan, activities, outputs, and outcomes in a way that is clearly defined and 

can be measured over time is key to making this sustainable. The new templates are a step in 

this direction, but the need remains for further buy-in at the different levels of the organization, 

training to operationalize, and a system to ensure accurate analysis and use.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

13

Program Component 2: Compliance Improvement Planning

COMPONENT 2 OBJECTIVE 

OTA’s objective was to improve voluntary taxpayer compliance by identifying and 

mitigating barriers. Project activities included assisting RRA to develop a comprehensive CIP 

that addresses different steps of the process including taxpayer registration, tax filing, payment, 

and mitigating accuracy in reporting/filing. OTA also advised RRA on how to identify and

mitigate revenue risks. 

Table 2 summarizes the findings for Component 2, and a detailed discussion follows the 

summary.

Table 2: Component 1 Outcomes—Activities and findings

Expected Outcome 2.1: RRA to research taxpayer behaviors and attitudes to understand barriers to 

voluntary compliance in the areas of registration, filing, and payment. RRA adopts and implements a 

CIP to increase taxpayer registration and mitigate filing and payment accuracy risks. 

Activities Findings

2.1.1: Develop compliance 

research plan

• OTA suggested RRA conduct a study to research taxpayer 

behaviors. RRA conducted the study with help from the 

International Center for Tax and Development (ICTD) in 2020.

OTA did not contribute directly to the research.

2.1.2: Develop a CIP • RRA already had a well-developed CIP. OTA contributed to 

the plan by recommending the audit teams be better focused 

and trained before attempting to work in the field, but this 

activity was limited in scope.

Expected Outcome 2.2: RRA identifies areas that need additional focus on an ongoing basis and 

dedicates resources to mitigation activities.

2.2.1: Assist RRA to develop 

ongoing monitoring 

methodology

• Since the CIP was not operationalized until after the ICTD 

study, OTA was not able to complete this activity. The 

evaluation team found that identification and mitigation of 

compliance risks through the CIP was not being monitored. 

OTA provided guidance for developing risk reports. However, 

senior management had not used the risk reports to inform their 

annual workplans or measure performance against the plan. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

14

COMPONENT 2 FINDINGS

EQ 1: Which of the expected project outcomes were achieved?

Outcome Status: Partially achieved

Overall, OTA’s expected outcomes under this component were partially achieved. At the 

suggestion of OTA, RRA engaged ICTD in 2020 to conduct research on taxpayer behaviors and 

attitudes—to better understand barriers to voluntary compliance in the areas of registration, 

filing, and payment. Since this study was not conducted until near the end of OTA’s program in 

2020, there was not enough time within OTA’s program cycle to incorporate the findings.

However, the evaluation team learned from counterpart interviews that RRA recently used the 

findings from the ICTD study to update and implement a revised CIP to increase taxpayer 

registration and mitigate filing and payment accuracy risks. OTA helped identify some areas 

that needed additional focus and, in a final memorandum, recommended that RRA dedicate 

additional resources to the mitigation of the risks that arise from taxpayer noncompliance. 

EQ 2: Which of the achieved outcomes have been sustained up to the present?

Outcomes Sustained: Improved voluntary taxpayer compliance

Counterparts reported to the evaluation team that voluntary taxpayer compliance has improved,

although RRA is not measuring taxpayer compliance against an established baseline, and 

quantitative data as evidence for improved compliance was not available.

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success? 

Hindering Factors: Sequencing and timing

OTA’s contributions were limited due to the sequencing and timing of events. RRA needed data

to design effective activities. While RRA acted on that recommendation, research was not 

available until after the OTA program cycle.

COMPONENT 2 CONCLUSIONS

Resource allocation decisions need to factor in country context and best practices for optimal 

allocation (e.g., number of auditors per case)

OTA’s contributions to this Component 2 were minimal as RRA already had a CIP and a process 

to update and revise it in place. Nonetheless, OTA suggested conducting a study researching 

taxpayers’ behaviors that could inform and improve compliance behaviors. Since this study was 

only conducted in 2020, OTA’s engagement was over prior to the findings being available. 

For better resource allocation, OTA recommended a reduced number of auditors assigned to each 

case (lowered from a team of two auditors per case to only one auditor per case). The 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

15

recommendation was adopted by RRA without the necessary investigation. In our view, RRA’s

practice of assigning two auditors per case was providing a safeguard against corruption and 

malpractice that often arises when there is only one auditor for a case. Corruption mitigation 

plays a significant role in the allocation of resources to the revenue authorities in most 

developing economies and OTA might have discussed with RRA to investigate the logic behind 

the choice of two auditors per case. We iterate this gap under program-level recommendations,

which highlights the need for OTA to better align expertise and advice with the best practices 

suitable to the country context and consider the reasons that lead RRA to making some of these 

operational decisions.

COMPONENT 2 RECOMMENDATION 

For increased efficiency, stage efforts to complete joint planning, target and goal setting, and 

monitoring framework prior to start of project activity

Prior to initiating development work of this type, joint planning should be completed before the 

engagement—this should include a desk review and interviews. This planning process would 

help identify data gaps and the need for further research, which could be done prior to the start of 

the OTA engagement. This would save valuable time and increase efficiency of the OTA 

advisors. 

In addition, in line with the first component, establishing a baseline and setting up targets and

milestones to achieve those targets would help quantify gains due to OTA’s technical assistance 

efforts.

Program Component 3: Tax Debt Collections

COMPONENT 3 OBJECTIVE

OTA’s objective for this project component was to guide the development of standardized

procedures for increasing RRA collection of established tax debts. The planned project 

activities included revising tax debt policies and procedures, developing a case selection process, 

updating the procedural manual, training employees in the use of the manual, designing 

management information reports, training staff to analyze management information reports, 

designing timesheets, and training management in the analysis of timesheet data. Due to the 

relatively large scope of OTA’s activities under this component, the findings are categorized into 

three outcome areas under Component 3.

COMPONENT 3; OUTCOME 3.1 FINDINGS

Table 3 summarizes the findings for Component 3; Outcome 3.1 and a detailed discussion 

follows the summary.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

16

Table 3: Component 3; Outcome 3.1—Activities and findings

Expected Outcome: 3.1

RRA adopts and implements revised tax debt policies, procedures, and manual to select cases with 

a higher likelihood of collectability, resulting in increased revenues from collections.

Activities Findings

3.1.1: Assist RRA in 

revising tax debt policies 

and procedures, including 

write-offs of uncollectible 

accounts, installment 

agreements, improving 

data on tax arrears.

• OTA provided overall support on debt policies, including 

revising the tax debt manual and guiding RRA to put in place 

standard operating procedures.

• Of note, under advisement by OTA, RRA extended the 

maximum possible repayment period for tax debtors from 12 

months to 36 months.

• Support for the write-off of uncollectible debts did not

materialize and support for improving data on tax arrears was 

minimal.

3.1.2: Assist RRA to 

develop a tax debt case 

selection process 

• OTA supported RRA to revise or streamline procedures for case

selection and case management; this led to a higher likelihood of 

debt collectability. 

• OTA recommended a matching program that would offset debts 

against government contract payments, but it was not 

implemented because the e-Tax case management system was 

not sophisticated enough to accept the changes. This suggestion 

was recently submitted to the Ministry of Finance and is 

awaiting approval.

3.1.3: Draft updates to the 

Debt Management 

Procedures Manual 

• OTA’s most significant contribution was the support provided by 

advisors to develop and update the debt management manual. 

• RRA continues to have out-of-date debt write-off procedures in 

its policies. The process is cumbersome and involves approvals 

from the top level of government. 

3.1.4: Conduct training for 

debt management 

employees in use of the 

revised manual

• OTA provided brief training to the enforcement officers and 

managers in the use of the revised debt manual. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

17

EQ 1: Which of the expected project outcomes were achieved?

Outcome 3.1 Status: Partially achieved

Outcome 3.1 was partially achieved. OTA assisted RRA in streamlining the audit and 

debt management processes for increased efficiency. One of the major achievements under this 

activity was the reduced level of involvement by the senior management in audit cases and their

approvals. This delegation and streamlining of functions resulted in reduced interactions 

between managers and taxpayers in the audit and debt departments, which led to more efficient 

processes. 

OTA developed a memo with recommendations for the RRA legal department to consider 

increasing the use of legal actions to recover money from the largest debtors. This includes

garnishment actions, seizure and sale of assets, and other significant enforcement measures that 

better utilize already available tools. OTA recommended improvements to tax arrears 

management data as well to inform RRA’s process.

EQ 2: Which of the achieved outcomes have been sustained up to present?

Outcomes Sustained: Streamlining of the audit and debt management process and use 

of updated collections manual

RRA has maintained changes to the audit and debt process that made it more efficient. RRA 

does not have benchmark data prior to the changes, which will make it difficult to quantify the 

specific resulting impacts on revenue. The Rwandan Parliament amended the law to lower the 

penalty and interest rates on arrears to encourage delinquent taxpayer compliance. This action 

will ensure application; however, it still needs to be incorporated into policy at RRA. 

Based on feedback from interviews, the collections manual is being used by RRA staff in its

updated form.

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success?

Hindering Factors: Approval delays and OTA’s broad scope

Some of the changes that OTA recommended are not yet enacted due to high-level decisionmaking. For example, the e-Tax case management system changes have been submitted to the 

Ministry of Finance but are still awaiting approval. Additionally, some tasks did not get OTA’s

focus given the amount of work in this area. These have been addressed in the final 

Memorandum of Recommendations to Commissioner General of RRA for further follow up.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

18

OUTCOME 3: COMPONENT 3.1 CONCLUSIONS

Component 3.1: The support of OTA was well received and many of the recommended changes

have already been adopted or are underway. Additional recommendations for future work are

included in the OTA Memorandum of Recommendations to the Commissioner General. Several 

areas for improvement were identified, including 1) delinquent tax payment deadlines, 2) 

collections officer capacity building, 3) joint development of a comprehensive collections 

approach, 4) ending automatic assessment of non-filers, and 5) guidance for best practices in 

arrears balance management. 

Overall, OTA’s support was well received in this area and RRA adopted many of the 

recommended changes or is in the process of doing so. Due to the large number of issues to be 

addressed, OTA recommends continued work, which is detailed in the Recommendations Memo. 

Despite OTA’s advice to RRA, the evaluation team concludes that extending delinquent payment 

periods is not considered best practice. OTA’s recommendation has been adopted, but the 

evaluation team concluded that OTA may have misunderstood the root causes of taxpayers’

delinquencies. An efficient collections operation requires several enforcement stages because

delinquent accounts become less likely to collect over time. OTA recommended policy changes 

that assumed delinquent taxpayers needed more time to pay; however, the evaluation team 

concludes that extensions of delinquent payments to 36 months contributed to noncompliance. 

Arrears become increasingly more difficult to collect as they aged, eventually becoming

uncollectable. Devoting efforts to educate taxpayers and enforce timely payment of tax 

obligations in a consistent way is widely considered best practice. 

The evaluation team also concludes that OTA’s support to develop RRA’s collection manual is 

only part of a capacity building strategy that, to be fully effective, should have included 

specialized training in improved investigative techniques that enable collections officers to work 

through the best strategies and solutions for the many variations in circumstances and challenges 

they encounter. RRA’s manual should describe how the business process works in any given 

sector, describe the necessary documentation, and provide guidance for where to look for assets 

and income. Rather than simply following prescribed solutions to anticipated situations, the 

manual should enable critical thinking for auditors based on principles and critical questions for

each situation. The evaluation concludes, based on multiple counterpart interviews, that RRA’s 

officers face limitations to investigate taxpayer's arrears properly, particularly where 

circumstances vary beyond the limited cases found in the manual.

OTA also advised RRA to prioritize collections of easy case collections. However, the 

evaluation team concludes that, based on evidence from counterparts, this resulted in more 

complex cases to be almost entirely neglected, resulting in an increase in aged arrears. Instead of 

selecting easy cases, OTA needed to work with RRA to develop a more comprehensive 

collections approach that included enforcement interventions at each stage of the collections 

process that would provide sequential enforcement and increased consequences as cases age. 

Every case should see the same initial level of effort applied as the case moves up the 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

19

enforcement ladder. The easy resolution cases will be finalized early and the more complex will 

continue up the ladder. OTA’s advice to focus on the easiest cases effectively allows 

enforcement staff to pick the cases on which they wish to work, reducing incentives for resolving 

all but the least revenue-recovering, more consequential, cases and generally erode a culture of 

taxpayer compliance.

The evaluation team also concludes that OTA should have advised against RRA’s policy of 

unrestrained, automatic assessments for non-filers. Automatic assessments should stop after one 

or two late payments to confirm that the taxpayer’s business is active, as counterparts reported 

many temporary and short-lived businesses remain on RRA’s registry, contributing to higher 

arrears rates. 

Finally, the evaluation team concludes that OTA should have provided guidance regarding 

policies for write-offs or the setting aside of uncollectible debts that reflect best practices. Writeoffs provide a critical step in collections cycle, without which total arrears balances grow without 

the appropriate enforcement action being taken and distort the Government of Rwanda’s revenue 

accounting. OTA did not appear to assist RRA with arrears balance management best practices 

through generally accepted, standard write-off policies.

COMPONENT 3: OUTCOME 3.1 RECOMMENDATIONS

OTA needs to use a more systematic approach to planning engagements.

It is critical that OTA advisors are current on country context and best practices in the field of tax 

administration. It is an evolving field; as such, it is critical that OTA advisors recommend

changes that advance RRA’s practices. The conclusions section outlines a number of examples 

where OTA guidance may not have been as efficient or effective due to a lack of understanding 

of RRA’s larger business process and context. See Section IV for further information.

COMPONENT 3; OUTCOME 3.2

Table 4 summarizes the findings for Component 3.2, and a detailed discussion follows the 

summary.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

20

Table 4: Component 3 Outcome 3.2—Activities and findings

Expected Outcome: 3.2 

RRA adopts and implements management information reporting and debt management reporting to 

track and analyze trends in arrears over time. RRA management adjusts policies/procedures as needed 

to facilitate debt collection.

Activities Findings

3.2.2: Assist RRA in 

designing management 

information reports

• RRA had some management information in place, produced 

manually in Excel spreadsheets. OTA supported the revision and 

improvement of these reports. For example, OTA assisted with 

refinement of the Top 25 arrears case reports and the debt 

management report.

• RRA revised, adopted, and implemented some of the tax debt 

policies suggested by OTA. However, RRA was primarily focused 

on tax debt processes and procedures. Industry sector notes 

(construction sector only) were produced to increase efficiency

when working with this key sector.

• RRA was already tracking one KPI for the aging of debts when the 

OTA program started. 

3.2.2: Train an RRA analyst 

to conduct trend analyses 

and develop monthly debt 

management analytical 

reports for management

• Training was completed for a number of staff, but few managers 

were included in the sessions. Most trainings were single sessions, 

not components of a systematic training approach.

• Training materials were developed by OTA but were not 

complemented by use of a train-the-trainer model.

EQ 1: Which of the expected project outcomes were achieved? 

Outcomes Status: Partially achieved

The outcomes under outcome 3.2 were partially achieved. Full achievement of Component 3.2 

would have included training managers and building in a more systematic implementation, such 

as a train-the-trainer model, to ensure training would continue to be replicated. 

EQ 2: Which of the achieved outcomes have been sustained up to the present? 

Outcomes Sustained: Report revision and construction sector industry notes

The evaluation team has found that OTA’s assistance with the revision of the Top 25 reports has 

been carried forward and continues to be used. Respondents noted that construction industry 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

21

sector notes have been continued but they have not been developed for the other sectors, such as 

telecoms, banking, etc.

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success? 

Hindering Factors: Time restraints prevented training

RRA was not trained to analyze data because of time restraints at the end of the project.

COMPONENT 3; OUTCOME 3.3 FINDINGS

Table 5 summarizes the findings for Component 3.3, and a detailed discussion follows the 

summary.

Table 5: Component 3; Outcome 3.1—Activities and findings

Expected Outcome 3.3

RRA analyzes data from timesheets to understand trends in productivity and assign cases to match 

enforcement officer skills with caseloads.

Activities Findings

3.3.1: Assist RRA in designing 

timesheets for enforcement 

officers (EOs)

• OTA supported the adoption and implementation of timesheets 

for debt management as requested by RRA leadership.

However, OTA did not socialize this change or do any change 

management, which led to poor adoption by staff.

3.3.2: Train Debt Management 

managers and EOs on timesheets

• Training for RRA to analyze and use data from timesheets for 

management decisions did not happen due to time constraints.

EQ 1: Which of the expected project outcomes were achieved?

Outcomes Status: Partially achieved

Outcome 3.3 was partially achieved. OTA introduced the concept of timesheets and the 

importance of tracking staff time for accountability and task management. RRA put timesheet 

protocols in place and is using them to monitor individual auditors’ performance.

EQ 2: Which of the achieved outcomes have been sustained up to the present?

Outcomes Sustained: Timesheet protocols and practices 

RRA has maintained the timesheet protocols and practices, but adaptation continues to be limited 

in scope and not systematic across the organization.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

22

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success?

Hindering Factors: Incomplete timesheet use

Timesheets are not used by all staff or fully completed; therefore, RRA is not able to use and 

analyze timesheet data to understand trends in productivity across the organization. Ideally, this 

information would be used to identify efficiencies and inform resource allocation, but data are

not sufficient for this level of analysis.

COMPONENT 3; OUTCOME 3.3 CONCLUSIONS

Timesheet use is not consistent across RRA and may not fully meet operational needs.

Timesheets were introduced by OTA, but never fully operationalized. While the use of 

timesheets is a best practice, OTA advisors should begin the process by understanding RRA’s 

operational process and ensuring the timesheets met their business needs. This would have made 

the tool more useful and alleviated some of the resistance by staff to using the timesheet.

COMPONENT 3; OUTCOME 3.3 RECOMMENDATIONS

Train OTA advisors in change management principles and employ this approach when

introducing and operationalizing new tools or processes, such as timesheets. 

OTA advisors should be trained in change management principles in the future to ensure they are 

taking a more holistic approach when introducing new tools or processes. Any organizationallevel change should use a change management approach to ensure the new process meets the 

needs of the organization and to gain buy-in by staff and managers. All levels of staff should 

also be consulted to ensure the practice is applied and used throughout the organization.

Program Component 4: Specialized Sector Audit

COMPONENT 4 OBJECTIVE

The objective of this component was to strengthen RRA’s capacity to audit construction 

sector businesses and improve taxpayer compliance. The project components focused on 

strengthening RRA’s ability to understand the sector’s business environment and regulations, 

building them into their own audit policy and procedures. The expected outcome was to increase 

the number of cases to be upheld in appeals as the result of RRA auditors using a completed and 

more accurate audit manual.

COMPONENT 4 FINDINGS

Table 6 summarizes the findings for Component 4, and a detailed discussion follows the 

summary.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

23

Table 6: Component 4—Activities and findings

Expected Outcome 4.1

RRA adopts and implements a construction sector section of the audit manual that complies with the 

sector’s policies, procedures, laws, and regulations. RRA staff use audit manuals to conduct audits that 

are complete and accurate, which are more likely to be upheld when appealed, leading over time to 

increased voluntary tax compliance.

Activities Findings

4.1.1: Assist auditors in 

reviewing the laws and 

regulations pertaining to the 

construction sector and propose 

revisions to the sector audit 

policies and procedures

• OTA supported auditors to review the laws and regulations 

pertaining to the construction sector and develop sector-specific 

notes.

4.1.2: Assist in drafting a new 

construction sector section of the 

audit manual

• OTA helped adopt and implement a construction sector section

of the audit manual that complies with the policies, procedures, 

laws, and regulations specific to the sector.

4.1.3: Train designated managers 

and staff in the use of the 

construction sector manual

• OTA provided some training materials and trained designated 

managers and staff in the use of the construction sector section 

of the audit manual. However, trained staff were moved to do 

general audits right after receiving construction sector 

training. 

4.1.4: Mentor RRA auditors and 

management on conducting 

construction sector audits

• OTA provided on-the-job training and provided some live-case 

reviews (one case was examined); however, OTA fell short on 

securing commitment from RRA management to keep the 

trained auditors in the same section. 

EQ 1: Which of the expected project outcomes were achieved?

Outcomes Status: Achieved

OTA provided comprehensive support for all aspects of the component including devising and 

updating the special sector audit manual, sector notes, and providing experiential training using a 

live case. In general, the evaluation team found that the support provided in drafting the sector 

notes was very useful and appreciated by the auditors at RRA.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

24

EQ 2: Which of the achieved outcomes have been sustained up to the present?

Outcomes Sustained: Use of construction sector audit notes and updated manual 

section

RRA still uses and regularly updates the sector notes and revised construction sector audit 

manual.

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success? 

Hindering Factors: Transfer of trained sector auditors to nonspecialized audits

After OTA provided the construction sector audit training, RRA management transferred the 

trained auditors back to general, nonspecialized audit work. As a result, according to several 

counterparts, OTA’s training has not been used.

COMPONENT 4 CONCLUSIONS

The range of construction sector audit support and advice of OTA was well received, gains were 

made in all aspects of construction sector audits. Efforts are needed to keep auditors with 

specialized training assigned to work in their new area of specialization.

This is one of the components where OTA provided comprehensive advice and their support 

covered all aspects of the construction sector audit activity ranging from devising and updating 

the special sector audit manual, sector notes, demonstration through a live case, and training. 

OTA’s support on this component remained available consistently through the program. The 

counterparts were very pleased with the model of support provided by OTA. However, the 

auditors trained in construction were pulled away for work in other sectors immediately after 

they completed their construction sector training. Here, OTA could have used the leverage and 

sought assurance from the senior management that the trained auditors will be allowed to 

practice their newly acquired skills. 

COMPONENT 4 RECOMMENDATIONS 

For future programs, direct and frequent meetings of OTA advisors and audit (technical) 

managers, along with the user of a wider range of cases to demonstrate techniques during 

training, can improve direct knowledge transfer. Securing an RRA leadership commitment to 

retain auditors in the department of their (sector) training will also help sustain knowledge 

transfer and future training.

OTA advisors worked only on one live case during the training on the construction sector. It 

would have been more useful if OTA had demonstrated audit techniques by using a variety of 

cases during the training to give a wider perspective. Feedback from respondents was that the 

OTA advisors should meet directly and more frequently with the audit (technical) managers
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

25

instead of the senior managers. This would provide for more direct knowledge and skills transfer

to audit managers that are more involved in the day-to-day operations. 

RRA would have seen greater benefits from OTA efforts if there was a commitment on their part 

to train and deploy special sector auditors in a more efficient manner. RRA has a culture of 

shifting employee resources at will and this practice could have been addressed in the areas 

where OTA provided assistance, such as the construction sector audits. This could have been 

done by securing an agreement from RRA leadership to keep the auditors in the same department 

for which they received training until they cement their expertise and then train more auditors for 

more sustainable outcomes.

Program Component 5: Computer -Assisted Audit

COMPONENT 5 OBJECTIVE

The objective of this component was to enhance the capacity of RRA auditors to conduct 

complex audits and streamline the audit process in high-risk compliance sectors, such as 

telecommunications and banking, using a Computer-Assisted Audit (CAA) program.

Component activities focused on training Computer Audit Specialists (CAS), educating 

taxpayers on the relevant filing requirements that enable RRA to use computer-based audits, and 

teaching auditors how to use audit data. 

Table 7 summarizes the findings for Component 5, and a detailed discussion follows the 

summary.

Table 7: Component 5—Activities and findings

Expected Outcome 5.1

RRA operationalizes and appropriately staffs a CAA Manager and a CAS. RRA adopts and 

implements the CAA audit manual to analyze large datasets of taxpayer substantiation and efficiently 

and effectively identify issues and adjust or verify tax returns. 

Activities Findings

5.1.1: Advise RRA management 

on recruiting and staffing CAA 

and CAA manager

• OTA suggested the idea of CAA and advised RRA 

management on recruiting, staffing, and software needs 

necessary for CAA management.

5.1.2: Train CAS auditors on data 

management, analysis, 

presentation skills, and other 

fundamental CAA skills

• OTA developed some training materials for CAA and trained 

three auditors. However, the team of three auditors who 

received the training are no longer on staff. RRA is currently 

training new CASs.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

26

• Due to fast staff turnover, RRA wanted OTA to train all 

auditors on CAA; however, OTA did not have the resources 

to do so.

5.1.3: Identify sources for online 

and classroom training in 

advanced Excel and IDEA Data 

Analysis Software for the CAS 

auditors

• The evaluation team did not find evidence, such as training 

materials or sources for online or classroom training, for this 

activity. 

5.1.4: Assist RRA in developing a 

new CAA section of the audit

manual

• OTA assisted in adopting and implementing the CAA manual 

to analyze large datasets of taxpayer substantiation.

5.1.5: Train CAA manager in 

CAS group management 

responsibilities and how to 

conduct CAA case reviews

• OTA did not provide any training to the CAA manager in 

CAS group management. 

5.1.6: Assist the CAA manager in 

reviewing the development of the 

CAS auditors and, as needed, 

provide on-the-job training for 

CAA

• This output was only partially achieved as CAA had not 

actually started during OTA’s program. RRA is currently 

preparing their officers so they can launch a formal CAA 

program.

5.2.1: Assist RRA in developing 

guidance to taxpayers on 

maintenance of electronic records 

for use in taxpayer audit

• RRA has not yet issued any guidance or materials to 

taxpayers to educate them on how to maintain electronic 

records for use in CAAs. 

• Furthermore, OTA suggested revising the tax law and 

procedures (the legislation was passed just a few weeks prior 

to the evaluation team’s visit). The new law requires 

taxpayers to file returns in an approved format, which enables 

RRA to use the information and data from the return for CAA

purposes.

COMPONENT 5 FINDINGS

EQ 1: Which of the expected project outcomes were achieved?

Outcomes Status: Partially achieved

The expected outcomes for Component 5 were only partially achieved. OTA was able to 

introduce CAA as a concept and begin initial steps of implementation, including the training of 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

27

staff. However, CAA is a complicated and resource-heavy endeavor. Full implementation will 

take time and both human and financial resources beyond the scope of the current OTA project.

EQ 2: Which of the achieved outcomes have been sustained up to the present?

Outcomes Sustained: Not achieved, prerequisite infrastructure needed

This component cannot be considered achieved or sustainable because of prerequisites

that must be in place for operationalizing CAA—including software that can process CAA data

and expertise to use the CAA tool. In addition, regulations that allow RRA to perform CAAs 

were only recently passed in 2023 after the program ended. The passage of the regulations 

demonstrates buy-in by the Government of Rwanda to this process. RRA expects to conduct 

further training to implement CAA now that a legal framework is in place. 

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success? 

Hindering Factors: Prerequisite infrastructure needed

As mentioned above, RRA does not yet have the relevant infrastructure to support the 

outcomes—the software and systems and the expertise to carry out such audits. The needed

legislative changes were enacted after the program ended. While OTA successfully introduced

CAA, systematic barriers prevented undertaking or achieving most of the workplan activities.

COMPONENT 5 CONCLUSIONS

Technical advice on CAA was provided by OTA before RRA had the established infrastructure 

necessary to fully adopt and operationalize a CAA system. Further technical assistance on 

establishing CAA, from taxpayer education to computing infrastructure, will be needed.

CAA is generally a very useful concept. This concept was recognized by the OTA team when 

they recommended setting up a dedicated CAA team in RRA. However, the evaluation team 

concludes that RRA had neither the capacity nor the systems in place to utilize technical advice 

on CAAs. OTA provided technical advice that would have been useful (and needed) only after

RRA had established infrastructure to fully adopt a CAA system.

There will be continued work by RRA to implement the CAA that may require ongoing technical 

assistance. This initiative requires a significant investment in taxpayer education for this type of 

program to be successful. Implementation of a CAA program also requires software that 

communicates with other RRA systems to be able to interpret information provided by taxpayers. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

28

COMPONENT 5 RECOMMENDATIONS

A whole-package approach to CAA in Rwanda, which includes sequential planning and is 

informed by a thorough understanding of the local context, is key to effectively introducing and 

operationalizing CAA. 

CAA needs to be seen as an all-encompassing program from RRA and needs to be planned 

accordingly. The recommendations made by OTA should therefore have included the wholepackage approach with an outline and sequencing of the necessary steps to be taken. 

Due to high staff attrition, RRA wanted OTA to train all auditors for CAA. However, OTA did 

not agree to the proposal and provided CAA training only to a couple staff members who have 

since left RRA. This component highlights another aspect where close attention to the local 

context rather than prescribing/following standard practice would be beneficial.

Program Component 6: General Audit Capacity

COMPONENT 6 OBJECTIVE

The objective of this component was to strengthen RRA’s audit quality by establishing 

Quality Review Staff (QRS), documenting procedures and standards for audit quality, and 

developing a staffing plan and organizational structure to assure and expand adherence to 

standardized good audit practices. Better quality audits were anticipated to result in accurate 

assessments, allowing for better targeted collections and reducing the burden on the tax authority 

in resolving disputes via the formal dispute resolution process.

Table 8 summarizes the findings for Component 6, and a detailed discussion follows the 

summary.

Table 8: Component 6—Activities and findings

Expected Outcome 6.1

RRA establishes a QRS team with qualified employees, selected in accordance with stipulated position 

descriptions and selection criteria. QRS personnel understand and execute their duties using defined 

roles and responsibilities. RRA establishes a QRS team that operates a fully developed QRS that 

improves audit case quality and reduces taxpayer appeals.

Activities Findings

6.1.1A: Assess baseline audit 

quality

• The evaluations assessed that this activity had not happened. 

6.1.1B: Review current audit 

organizational structure and 

• OTA supported the establishment of a QRS team with qualified 

employees, selected in accordance with stipulated position 

descriptions and selection criteria.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

29

recommend establishment of a 

QRS team

6.1.2: Assist RRA in creating 

position descriptions and 

selection criteria for QRS 

auditors

• OTA helped QRS personnel understand and execute their duties 

using defined roles and responsibilities and set up a fully 

developed QRS that improves audit case quality and reduces 

taxpayer appeals.

6.1.3: Assist RRA in developing 

plans and procedures for quality 

control in the audit process

• OTA supported the development of a quality review plan, a set 

of procedures and audit techniques for audit quality control.

• OTA supported digitization of the audit manual, which 

enhanced usability of the manuals. OTA also helped RRA set 

up a process to update the audit manual.

Expected Outcome 6.2

RRA adopts and implements timekeeping, standardized audit reports, tax computation worksheets, and 

audit planning. Auditors apply good practices for audit to their cases, identify weaknesses in the audit 

process and within individual audits, provide constructive feedback through an established procedure, 

and initiate a corrective follow-on process for audit quality improvement. This leads to progressively 

improved audit outcomes. 

Activities Findings

6.2.1: Assist RRA in developing 

audit tools including 

timekeeping, standardized audit 

reports including a tax 

computation worksheet, and 

audit planning

• OTA supported adoption and implementation of timekeeping, 

standardized audit reports, tax computation worksheet, and 

audit planning. More specifically, OTA:

o Provided a template that RRA has modified and used to 

produce a risk-based annual audit plan that includes case 

selection and case management.

o Supported the development of procedures and standards for 

audit quality.

o Provided a timesheet template to help managers gauge 

auditors’ performance.

o Developed a staffing plan and organizational structure to 

assure and expand adherence to standardized good audit 

practices.

• Most of the advice provided by OTA was reported to be useful. 

The audit manual jointly developed with support from OTA is 

maintained and being used by RRA.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

30

• Although the review team did not inspect the audit manual, 

advisors referenced its size and complexity.

• The timesheet was specifically noted as having limited utility, 

and it is only being used for distinguishing direct and indirect 

time spent on a case by the auditor.

6.2.2: Train employees and 

managers in the use of the audit 

tools

• OTA developed training materials, and a limited amount of

training was given to audit staff.

COMPONENT 6 FINDINGS

EQ 1: Which of the expected project outcomes were achieved? 

Outcomes Status: Partially achieved

Most of the outcomes identified were achieved to some degree. The quality review process was 

enhanced, audit reports and computation worksheets were developed, an audit manual was 

improved, and audit planning templates were developed. 

EQ 2: Which of the achieved outcomes have been sustained up to the present?

Outcomes Sustained: Audit manual use and updates, quality review process

improvements

The audit manual has been fully adopted by RRA and continues to be updated regularly. The 

quality review process that was enhanced and the risk-based audit plan template developed by 

OTA continues to be used and was reported to be effective in staff interviews. 

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success? 

Hindering Factors: Change management not used to introduce timesheets

The introduction of timesheets has been a contentious issue and limited in use. These control 

measures were introduced without prior groundwork to socialize the change and explain the 

utility. This change would have allowed RRA to link individual audit case times to the e-Tax 

system so audit results as well as time and level of effort factors could be uploaded to risk 

management and to performance management units. This information would have been useful 

when planning the workloads for coming periods. Unfortunately, this process was not explained 

to staff and most felt like it was an intrusive level of oversight.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

31

COMPONENT 6 CONCLUSIONS

The OTA support in producing a high-quality, audit manual and introducing enhancements for 

RRA’s quality review process for audits were key successes. These tools and OTA’s advice could 

have been improved by a better understanding of the RRA context—producing higher-level, less 

prescriptive guides and providing advice that reflected RRA’s organizational situation.

Overall, OTA provided support that was relevant and well received under this component. 

The evaluation team found that OTA advisors were able to assist in getting several tools in place 

within RRA including standardized audit reports, tax computation worksheets, and audit 

planning. Unfortunately, the feedback from those interviewed was that the operations manual, 

while useful, was very prescriptive and cumbersome to use effectively. An effective audit 

manual should include tools to guide a decision-making process and be a complement to 

classroom training rather than include detail on each case.

OTA supported changes to the delegation of authority, which led to fewer levels of approvals 

required to process an audit case and resulted in faster processing procedures and time spent on 

each case. 

The most significant achievements include setting up QRS and a quality review process. OTA 

helped establish a constructive feedback procedure to assist auditors to improve their audit skills. 

OTA dug into the issue of sending untrained auditors into the field and made recommendations 

accordingly. 

OTA also recommended reducing the number of field auditors for each case from two to one. 

This is an example of not fully understanding RRA’s business needs and the policy was soon 

reversed due to potential for corruption with a single auditor. It is also good practice to pair a 

new auditor with a well-trained person to provide a better opportunity for on-the-job mentoring 

and closing knowledge gaps. 

Some other initiatives by OTA, including the suggestion to move desk audit under the 

enforcement (previously being done by the general audit department), seemed to have missed 

RRA’s needs. Desk audit activities are linked directly to the main audit function and should be 

used as a training ground for new audit staff. The strategy to mix the unit into enforcement 

would need to be reviewed. 

As in some other components, OTA did not establish a baseline for audit quality and there were 

no quantitative means to assess if audit quality had improved. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

32

COMPONENT 6 RECOMMENDATIONS

It is important for OTA advisors to invest the time to understand the organization, business 

practices and rationale, and the local context to better facilitate the introduction of appropriate 

new procedures. To gain insight into the organization’s day-to-day processes and needs and to 

improve uptake of suggested changes, advisors should both work with staff at all levels and use a

project-wide change management approach. 

As discussed in the conclusion section, while OTA has made some progress, there were also 

many oversights. When designing future programming, the evaluation team recommends that

OTA advisors take the time to understand the organization and their business process, including 

the rationale for that current structure. This would avoid missteps in introducing new procedures 

counter to the organization’s requirements. Any proposed modifications should also be 

accompanied by a more holistic change management approach—from senior leaders down to 

more junior-level staff. As in the case of timesheets, even if there is buy-in at the leadership 

level, the change will not work if staff do not understand the intent and are brought into the 

process at an earlier stage. Finally, working with staff at all levels of the organization can help to 

understand the day-to-day needs and the practical application of any new tools. 

Program Component 7: Tax Dispute Resolution

COMPONENT 7 OBJECTIVE

The objective of this component was to improve voluntary compliance among taxpayers by 

increasing fairness and transparency in dispute resolution. The planned project activities 

focused on 1) reforming the tax dispute resolution process; and 2) creating a feedback loop 

between dispute resolution personnel and audit staff.

Table 9 summarizes the findings for Component 7, and a detailed discussion follows the 

summary.

Table 9: Component 7—Activities and findings

Expected Outcome 7.1

RRA understands the gaps between their current tax dispute resolution process and good practices and 

takes steps towards improvement. RRA creates an action plan to improve audit quality, and, over time, 

reduce the number of audit disputes.

Activities Findings

7.1.1: Review and recommend 

improvements to the current tax 

dispute resolution process, 

• OTA recommended improvements to the tax dispute process, 

including changing the interest rate and penalty rates. OTA

provided a template for reference, which RRA ultimately 

adopted.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

33

including the objections and 

appeals process

• OTA also advised on the legislative changes required to enable 

the new process, which RRA ultimately secured approval 

through Parliament. Regulations still need to be put into place 

to operationalize the change.

• Once RRA gained approval, OTA assisted them to create an 

action plan to improve audit quality with the aim of reducing

the number of audit disputes. The evaluation team was not able 

to quantify fiscal gain or reduction in the number of appeals as 

no baselines were previously established. 

• OTA also suggested setting up an appeals tribunal (an 

international best practice); however, this was not adopted and

RRA continues to have the administrative review process. 

7.1.2: Deliver a workshop on 

international practices in tax 

disputes

• Provided a 1-day training on audit procedures and tax law to 

the audit staff.

Expected Outcome 7.2

RRA forms and regularly holds working group meetings to provide feedback to auditors regarding the 

outcome of the dispute resolution process. Auditors use feedback from dispute resolution personnel to 

improve audit quality, and, over time, reduce the number of audit disputes. 

Activities Findings

7.2.1: Develop a working group 

composed of dispute resolution 

staff and auditors

• OTA conducted a workshop on understanding the gaps between 

their current tax dispute resolution process and good practices.

• A working group was set up to discuss the outcomes of appeals 

to enhance auditors’skills.

Expected Outcome 7.3: 

Tax dispute resolution staff attend auditor training and understand the audit division's policies and 

procedures and use the information to achieve fairer and more accurate results in dispute resolution.

Activities Findings

7.3.1: Recommend inclusion of 

dispute resolution staff in auditor 

training

• Dispute resolution staff did not receive any audit training from 

the OTA although RRA had some internal training for dispute 

resolution staff on audit procedures and would discuss appeal 

outcomes with the auditors. 

• The dispute resolution staff provided feedback to the auditors 

through the working group. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

34

Expected Outcome 7.4: 

RRA creates and publishes taxpayer audit dispute resolution information on their website. Taxpayers 

have access to accurate information regarding their rights in disputing the results of an audit with the tax 

administration.

Activities Findings

7.4.1: Assist RRA to develop tax 

dispute resources for inclusion on 

the RRA website

• OTA did not provide any support for taxpayer education in

reference to the appeals process. 

• The RRA website does not have any new resources on dispute 

resolution other than a small section on legislative changes that 

is difficult to navigate for an average taxpayer. Taxpayers 

continue to receive information about next steps through an

assessment notice in the mail. Mailing information to 

taxpayers was also the procedure prior to OTA’s program. 

COMPONENT 7 FINDINGS

EQ 1: Which of the expected project outcomes were achieved?

Outcomes Status: Partially achieved

The outcomes under this component were partially achieved. OTA advised RRA on

improvements to the tax dispute process, including changing the interest rate and penalty rates, 

which were adopted. Additionally, OTA assisted RRA to propose and ultimately pass the 

needed legislative changes. Policy changes are needed to fully operationalize this 

recommendation, but the legislation paves the way for this to happen. OTA established a 

feedback loop between the audit staff and dispute resolution personnel through a cross-agency 

working group; however, they were unable to reform the dispute resolution process. 

EQ 2: Which of the achieved outcomes have been sustained up to the present?

Outcomes Sustained: Feedback mechanism, policy changes

The legislation changes solidified modifications for the tax dispute process, which makes it 

sustainable. RRA will still need to translate this legislative change into a policy that is more 

operational, but that is in process. The internal process changes, such as the working group, are 

still in place. There is continued use of feedback loops between the audit staff and dispute 

resolution staff. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

35

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success?

Hindering Factors: Readiness for dispute resolution process, limited scope of training, 

The evaluation team observed that RRA was not at the point to begin instituting a dispute 

resolution process. This is a complex process that requires changes at multiple levels and more 

intensive capacity building. Multiple KII participants made observations on the mode and 

quantity of training provided by OTA—under this component, a one-day (or shorter) training on

audit, tax law, and procedures was provided by the advisor—and many at RRA felt that this

training was insufficient or was not provided to enough staff to affect change. The general 

perception was that additional staff training on all aspects is needed. 

COMPONENT 7 CONCLUSIONS

A focus on developing the tools for taxpayer education on dispute resolution would have been 

more beneficial for RRA.

While the internal process changes were mostly achieved, a major component was not addressed. 

Taxpayer education and facilitation is critical to increasing tax compliance. Tax codes and 

procedures are complicated, and any effort spent to simplify tax procedures must be clearly 

communicated to the public to increase compliance. Auditors should be trained to discuss any 

proposed assessment with taxpayers, give them an opportunity to submit new evidence, and then 

work with the taxpayer to evaluate the new evidence. An investment by tax administration in 

developing and establishing a dispute resolution process, which includes an initial screening (to 

identify a frivolous dispute) and offers opportunities to resolve cases before going to a court, can 

reduce the number of tax disputes. RRA could have benefited from more support from OTA in 

developing such materials to make the operation of the legal framework fairer for taxpayers.

Information provided in a format that is easily understandable by the taxpayers on their 

obligations like the assessment amount, timeframe to pay, and mode of payments will have a 

direct impact on disputes. Tax disputes often arise due to several factors—including limited 

taxpayer’s understanding of the tax legislation or a lack of communication between the auditor 

and taxpayer, a difference of opinion on taxability issues, a strategy by the taxpayer to delay 

payment of assessment by disputing it, or avoidance of payment due to a simple disagreement 

with assessment. No such dissemination of information or facilitation to taxpayers was 

supported by OTA. 

COMPONENT 7 RECOMMENDATIONS

Assist organizations to identify the root causes of problems/issues to determine the appropriate 

solutions to address the issues.

OTA should assist RRA, or other organizations in the future, to understand the root causes of the 

problem to ensure the solution is appropriate. In this case, educating the taxpayer would have 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

36

had a more significant impact in dropping the number of appeals than changing internal 

processes. If taxpayers understand the process and the reason for the assessment, they are more 

likely to comply, resulting in a reduced number of appeals. Both OTA and RRA should have 

seen this as an opportunity to change taxpayer behavior by mounting an education and awareness 

program first. Once the root causes of the issue were addressed, then the training of staff would 

have been more effective. 

Program Component 8: Internal Audit

COMPONENT 8 OBJECTIVE

The objective of this component was to improve RRA’s performance through enhancing 

the effectiveness of its internal audit (IA) function. The planned project activities focused on 

1) revising and implementing an internal audit manual, 2) training staff in internal audit 

methodology, and 3) improving RRA departments' responses to internal audit recommendations 

to improve internal and managerial controls.

An internal audit unit is a quality control mechanism for all functional aspects of the revenue 

administration, starting from the initial registration of the taxpayers to their tax assessment and 

exercise of enforcement functions and leading to the final phase of debt write-off of the 

unrecoverable debts and taxpayer accounts. Information learned about effectiveness and 

correctness of every step of the process should be fed back into the organization to create an 

environment that seeks to improve processes and efficiency. 

Table 10 summarizes the findings for Component 8, and a detailed discussion follows the 

summary.

COMPONENT 8 FINDINGS

Table 10: Component 8—Activities and findings

Expected Outcome 8.1:

RRA leadership approves and implements standardized internal audit forms and updated IA

procedures. The revised procedures and standardized internal audit reports reflect findings and actions 

that improve consistent application of laws, regulations, and procedures. RRA management monitors 

the progress of departments in utilizing IA findings to mitigate risks, optimize operations, and reduce 

revenue leakages in a timely fashion. 

Activities Findings

8.1.1: Review existing IA 

procedures, reports, and 

• OTA reviewed RRA’s IA procedures, reports, and inventory of 

existing cases and identified gaps in IA procedures. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

37

inventory, and identify gaps in 

audit procedures

8.1.2: Review RRA departments’

processes for acknowledging and 

implementing recommendations 

resulting from IA

• OTA reviewed departmental processes for acknowledging and 

implementing recommendations resulting from IA. 

• RRA was already using standardized forms; OTA assisted in 

updating and improving the process. 

8.1.3: Revise IA procedures • OTA reviewed procedures and made recommendations on 

revising or updating some of the IA procedures and 

standardized internal audit reports. This improved the 

consistent application of laws, regulations, and procedures. 

• RRA continues to use the revised forms and procedures.

8.1.4: Develop processes and 

implement a plan to address 

RRA’s inventory of aged IA 

recommendations

• RRA IA department had a backlog of departmental functions 

recommendations, which were not yet implemented. OTA 

assisted RRA to develop processes and implement a plan to 

address RRA’s inventory of aged IA recommendations. OTA 

suggested that RRA develop a tracking system for the 

inventory of aged IA cases. RRA has not yet implemented 

that recommendation. 

Expected Outcome 8.2: 

IA liaisons are assigned from each department, bridging the knowledge gap between IA and the various 

departmental functions. Liaisons communicate as needed to collaborate to identify and resolve issues 

across functional areas.

Activities Findings

8.2.1: Brief RRA leadership on 

the role and value of establishing 

Audit Liaisons and create an 

Audit Liaison selection process

• OTA briefed RRA leadership on the role and value of 

establishing IA liaisons. RRA initiated a more formal 

approach to embedding an IA liaison in each functional 

department. 

• IA liaison (a role previously filled by informal focal persons)

continued to be assigned from each department, bridging the 

knowledge gap between IA and the various departmental 

functions. They communicated as needed and collaborated to 

identify and resolve issues across functional areas. 

8.2.2: Provide training to audit 

liaisons on their role

• There was no formal training, training materials, or guidance 

on communicating with other departments provided by the 

OTA, because the IA liaisons were experienced in the role. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

38

EQ 1: Which of the expected project outcomes were achieved?

Outcomes Status: Partially achieved

The outcomes under this component were partially achieved. OTA’s recommendations on

revising or updating some internal audit procedures and standardized internal audit reports were 

put into use by RRA. Additional recommendations for revisions of internal audit procedures to 

assist with the backlog of cases are not yet fully implemented. An internal audit manual was 

developed with the assistance of OTA. While RRA already had informal focal persons from 

various departments for internal audit purposes, OTA helped formalize the IA liaison positions.

EQ 2: Which of the achieved outcomes have been sustained up to the present?

Component 8 Outcomes Sustained: Updated audit manual in use, IA liaison roles

formalized

Gains in Component 8 have largely been sustained. RRA continued to use the audit manual with 

updated procedures and put a process in place to update the manual to maintain its relevance.

Although some divisions have had turnover of individuals, the IA liaisons have remained a 

position within RRA. Given this turnover, there will be a need for RRA to continue training to 

maintain quality and consistency across divisions.

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success? 

Component 8 Outcomes Hindering Factors: Training not needed for IA liaisons

Prior to the program, RRA already had staff acting as informal focal persons for the IA. While 

OTA assisted with formalizing the role of IA liaisons, training was not needed for this 

experienced cohort, which already held meetings to identify issues, collaborate, and address 

problems across their functional areas. Similarly, OTA did recommend working group meetings 

but did not develop an internal communications plan or supported RRA to set up an internal 

communication process.

COMPONENT 8 CONCLUSIONS

Sustained progress was made in updating forms, tools, and procedures for the IA and in 

formalizing the IA liaison role. To support these gains, a formal training program is needed, 

particularly considering the high staff turnover rate in RRA. Establishing a tracking system for 

the audit recommendations backlog system and uptake of new recommendations is similarly 

important for future progress. 

OTA made significant progress with the proposed activities in Component 8. They were able to 

update many of the forms and tools needed to perform IAs and to standardize the process for 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

39

their implementation. Additionally, they formalized the IA liaisons positions within the 

divisions as the main points of contact. These were all positive changes and largely sustained by 

RRA. While progress was made, there was no formal training program initiated as part of the IA

effort given the experience of current staff. This training will need to be established to sustain 

change over time, given the high turnover within RRA. RRA also was not able to implement a 

system to track action on the backlog of audit recommendations. Establishing a backlog tracking 

system will also be important in terms of ensuring the IA function serves its purpose. The 

administration of RRA needs to better embrace the value of the internal audit function and see 

the efforts as a measurement tool for the performance of the revenue function. More effort could 

have been proposed by OTA in the areas of backlog reduction, all-staff training, and tracking IA 

recommendations after they have been delivered to the operations team. 

COMPONENT 8 RECOMMENDATIONS

For future programs, invest in understanding and scoping the organizational needs in order to 

develop a sequential plan for based on organizational capacity for implementing and sustaining 

changes in the IA process.

The workplan may have been overly ambitious given the limited time and resources. As stated 

in other areas, it is important for OTA to look at organizational needs more holistically and 

ensure RRA, or other organizations in the future, are capacitated not only to implement the 

change but also to maintain it. OTA may need to be more targeted in future projects to ensure 

activities/functions can be implemented, rather than trying to address a broad number of changes 

within an organization at one time.

Program Component 9: Risk Management (Information 

Technology Risk Management)

COMPONENT 9 OBJECTIVE

The objective of this component was to assist RRA to implement a risk management 

process to support the operations and strategic objectives of RRA and to enhance the 

physical security of RRA headquarters. The planned project activities included reviewing the 

risk management process, training management on risk management, developing a methodology 

for risk registers, developing a physical security plan, and mitigating physical and IT risks.

Table 11 summarizes the findings for Component 9, and a detailed discussion follows the 

summary.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

40

COMPONENT 9 FINDINGS

Table 11: Component 9—Activities and findings

Expected Outcome 9.1

RRA develops and implements a corporate risk register for identifying and mitigating compliance risks. 

RRA effectively identifies corporate risks and prioritizes resources according to their magnitude. 

Activities Findings

9.1.1: Review the risk 

management process and 

framework at RRA for 

identification of gaps in corporate 

risk registry

• OTA supported the risk management department to identify a 

significant number of potential corporate risks and to prioritize 

resources to mitigate according to their magnitude. 

• RRA continues to update the corporate risks in the register to 

capture emergent risks and develop mitigation plans.

• While RRA has made significant progress in systematizing its 

risk management process and mitigation, some areas remain to 

be addressed, such as the safety of taxpayer information.

9.1.2: Provide training to middle 

and senior management on risk 

management practices

• OTA provided training on risk management to both middle 

and senior management and communicated with the risk 

management staff on a regular basis to reinforce the training.

9.1.3: Assist RRA in developing a 

methodology for identifying and 

rating corporate risks in risk 

registers

• OTA reviewed the risk registers and worked with RRA on a 

process to prioritize resource allocation for mitigation.

Expected Outcome 9.2

RRA implements safety and security improvements, reducing risk to employee safety and RRA 

facilities and property, and protecting taxpayer information. 

Activities Findings

9.2.1: Develop a physical security 

improvement plan based on risks 

of physical security at RRA and 

their potential impact on 

personnel, equipment, and 

operations

• For the physical security improvement plan, OTA developed 

processes based on identified risks that enhanced RRA’s 

physical security through:

o Improving physical security by implementing a colorcoded card system for staff to limit access to specific 

floors. 

o Revising staff code of conduct, which includes antiharassment policies.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

41

o Implementing a background check requirement for new 

staff recruitment.

o Sensitizing staff on security protocols.

o Requiring visitors to always wear a visitor’s pass while 

inside RRA.

o Limiting taxpayers’ entry/access to the first floor of the 

RRA building unless accompanied by an RRA employee.

o Placing security guards in key access areas.

o Developing fire drills and evacuation procedures.

9.2.2: Provide guidance on 

mitigating risks outlined in 

security improvement plan 

• RRA developed a risk mitigation plan and a business 

continuity plan.

Expected Outcome 9.3

RRA adopts an IT plan allowing them to mitigate urgent threats to its information security and puts in 

place a longer-term plan to allocate resources to meet its IT needs.

Activities Findings

9.3.1: Assess and identify priority 

IT security needs, including IT 

risks and mitigations

• RRA reinstituted its IT strategy and a steering committee that

currently meets regularly to monitor the performance of IT 

projects and to make informed decisions on future IT 

investments in alignment with RRA’s strategic plans. 

• OTA supported RRA to develop and adopt an IT plan allowing 

RRA to mitigate urgent threats to its information security

department. The IT plan included:

o Revised IT policy, which included care and control of 

official IT equipment.

o User policies that included password protections and

network protection to control security breaches, data 

loss, fraud, and errors in RRA operations.

o USB ports disabled on the official computers and 

laptops.

• RRA implemented a longer-term plan to allocate resources to 

meet its current IT needs.

• RRA adopted an IT training plan ensuring that IT staff receive 

training commensurate with their responsibilities, as well as in 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

42

critical areas (such as cybersecurity). The additional training 

allowed RRA IT staff to carry out their duties more effectively 

and respond to ongoing and changing threats.

• While significant progress has been made in securing IT 

equipment and infrastructure, gaps remain in information 

security that will need to be prioritized.

EQ 1: Which of the expected project outcomes were achieved?

Outcomes Status: Key outcomes achieved

The evaluation team found that OTA and RRA successfully achieved key outcomes in 

Component 9. RRA was able to develop robust risk mitigation procedures, including 

identifying, recording, and mitigating risk. They were able to prioritize resource allocation using 

this information and minimize the risks identified. RRA was able to take concrete actions with 

the support of OTA to specifically identify risks related to physical and IT security. As noted in 

the table above, changes were made to ensure the physical safety of the staff, including badging 

protocols, limiting access of visitors within the building, and posting security guards at key 

access points. Similarly, RRA was able to reinstitute an organization-wide IT strategy that had 

not been fully implemented. This allowed RRA to monitor the IT security risk more proactively

through a comprehensive IT plan and a steering committee, which met regularly to assess and 

prioritize risks and resources.

EQ 2: Which of the achieved outcomes have been sustained up to the present?

Component 9 Outcomes Sustained: Risk management, strengthening security

RRA has largely been able to maintain outcomes achieved in Component 9 related to risk 

management, including the changes instituted to strengthen physical and IT security. The 

evaluation team noted that the changes RRA made were done in a holistic manner, which 

included both the changes to policy and procedures and the concomitant training and 

sensitization of staff on these changes. This also involved a process to prioritize and reallocate 

resources as necessary.

EQ 4: For any expected outcomes that were not achieved, which factors hindered 

success?

Component 9 Outcomes Hindering Factors

OTA advisors recommended the establishment of an emergency command center to better 

coordinate and respond to risks efficiently. This would be additional to the measures already put 

into place mentioned above in Table 11, under Outcome 9.3. The resources for the emergency 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

43

command center were made available at the time of the evaluation, but the unit has not yet been 

established. 

While RRA made significant progress on physical and IT security, cyber or information security,

including protecting confidential taxpayer’s information, remains weak. The RRA prioritized 

risk areas where they could make progress quickly—“the low hanging fruit.” Information 

security is more complex and will take time, capacity, and funding to address.

COMPONENT 9 CONCLUSIONS

The main contributions of OTA in risk management centered on the physical security of RRA

equipment and building. Due to time and resource constraints, OTA did not take on 

development of a complete risk management program outside of addressing physical security. A 

robust risk management program that considers incorporating risk analysis, hazard mitigation,

and disaster recovery planning across the agency, is a significant need for RRA. 

The purpose of a gap analysis is to flag potential risks and constraints to every administrative 

process within a tax authority. The risk management group is a key component of a compliance 

improvement plan that identifies factors that impede or otherwise can impair the operations and 

effectiveness of the revenue administration. 

Risk management often requires the intake of significant amounts of data to identify potential 

strengths and weaknesses and the outcome of this exercise involves entering relevant, clean, and 

accurate data into the analysis for accurate forecasting of risks. OTA primarily focused on the 

physical security aspects of the risk program for RRA, which included the physical security of 

the building, requiring employees to wear badges, and defining protocols around when and 

where taxpayers can access the tax authority. OTA did not have the time or sufficient resources 

to support a complete risk management program for the RRA. It appears that OTA did not work 

on, or help with, strategizing and planning for other risk areas relating to taxpayer information or 

other tax functions, for example, tax compliance. Confidentiality of taxpayer documents was 

improved to an extent—more in terms of equipment theft rather than regarding confidential 

taxpayer information. Hence, OTA’s core support around risk management remained central to 

only the physical security of the equipment and building. 

COMPONENT 9 RECOMMENDATIONS

An organization-wide risk analysis, particularly for IT, and the development of a robust 

management program across all departments is a pressing need for RRA.

Risk management has links to every other function in RRA and while the physical security issues 

were important, they overshadowed the importance of an effective disaster recovery plan for 

RRA’s records and data. The Risk Management project by OTA was very effective in mitigating 

risk in physical space and developing risk registers and employee safety measures. The review 

team was left with the impression that RRA accounting software was still very much at risk from 

hacking threats and from general disasters, yet the protection and recovery plans were vague. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

44

The risk management team needs to complete a full analysis of risk in the IT function and 

develop a mitigation plan as soon as possible. 

Evaluation Question 3

The focus of EQ 3, regarding a new approach to program design that involved counterparts in 

operational manual development, was a high interest activity for OTA at the time this evaluation 

was designed. Although prior sections discuss aspects of OTA’s support for developing 

operational manuals, it is worth highlighting the positive effects of this effort.

EQ 3. OTA took a new approach to program design in Rwanda by involving 

counterparts in the design and writing of key operational manuals. Did OTA's 

new approach contribute to the sustainability of the outcomes, and if so, how? 

Previously, OTA used a predetermined set of manuals to apply to each project. The manuals

were adapted by OTA advisors for each new project, with only minimal involvement of their 

counterparts. OTA’s new approach, which included close collaboration with their Rwandan 

counterparts for operational manuals development, had several positive effects. The evaluation 

team found that by employing this approach, OTA obtained buy-in from their counterparts, 

promoted ownership of the product, and created a positive environment for updates and 

revisions. 

OTA worked with RRA counterparts to develop the RRA audit manual. Component 3 highlights

some quality issues with the audit manual, nevertheless, involving RRA counterparts in both the 

design and writing of the manuals contributed not only to their relevance but also to the 

counterparts’ acceptance of end product. This approach also laid the foundation for RRA to 

continue to design and write additional manuals.

IV. PROGRAM LEVEL RECOMMENDATIONS 

While Section III includes specific recommendations for each of OTA’s nine program 

components at the activity level, this section provides a discussion of recommendations focused

at a higher, overall program level. The recommendations focus on two overarching areas: 

program design and ensuring appropriate staffing and training of OTA advisors.

Program Design Considerations

Based on the evaluation findings and conclusions, the evaluation team recommends that OTA 

consider the following three suggested changes in project approach and design.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

45

THEORY OF CHANGE 

The evaluation found that both the project logframe document, work plans, regular reports, and 

the End of Program Review (May 11, 2022) did not consistently follow a common framework in 

terms of outcomes, activities, and outputs—and sometimes not even in the overall project 

components and objectives. Consequently, the alignment of tasks to strategic plans and the 

creation of feedback loops were weak. While work plans were developed with some ownership 

from the technical senior managers at RRA, the evaluation team concluded that this ownership 

should have been built-in at a much earlier stage of the project and should also have included a 

wider range of RRA staff cadre. 

A theory of change (or similar) should be created at an early stage of the project. The 

process to create the theory of change would have helped the participants better identify the 

development needs and to obtain a common understanding of a feasible way forward. The 

theory of change development should follow a common process, using standard formats and 

terminology that will be further used throughout the project planning and monitoring process. 

BENCHMARKING AT PROGRAM INCEPTION

In addition, if the theory of change had been created, the basis for establishing a monitoring 

framework for measuring and maintaining accurate, factual records of outputs and outcomes 

over the course of the project implementation would have been in place. Baseline data and 

progress indicators would have been part of that framework. 

Collecting benchmarking data from the counterparts should take place at the inception/ 

start of a new component. This will a) build capacity at the counterparts to collect, synthesize, 

and report data and b) help OTA quantify its impact. We also suggest articulating some of the 

outputs and outcomes with more measurable/quantifiable indicators. Using outputs such as

“review the policy” or “provide training” lack the specific information needed to assess 

usefulness or specific activities undertaken.

The completion of tasks at the output and outcome level needs to be evidenced by facts and data

in RRA reporting. The evaluation team therefore recommends that OTA should advise and 

build capacity of RRA to ensure proper data collection, including establishing baselines for 

any production-related component. Collection and comparison of data are vital for 

determining progress.

SYSTEMS APPROACH

The evaluation team recommends that OTA advisors invest more time to understand the 

counterpart’s organization, business processes, and priorities prior to developing a 

workplan. By developing a robust understanding of the organization’s context, OTA advisors 

can understand the organization’s readiness for the activity, ability to implement recommended 

changes, and potential to sustain changes after the OTA engagement—factors critical to the 

short- and long-term success of the program.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

46

As noted in the components section, a number of outcomes were not achieved or not sustained

because OTA advisors did not understand the interdependencies. For example, OTA introduced 

a series of internal changes to increase taxpayer compliance but did not address the most critical 

aspect of taxpayer education. If taxpayers are not aware of or do not understand the changes 

meant to streamline the process, ultimately there will not be an increase in taxpayer compliance.

In another case, during the development of the KPIs, the department managers and staff received

some training, but senior managers were not included in KPI training, and they do not use KPIs

to inform higher-level decision-making. 

The evaluation team recommends that OTA advisors develop an enhanced task plan for 

each component they are responsible for implementing, including who needs to be trained, 

what reports need to be generated, and what actions need to be taken to maintain the 

sustainability of the operation. KPI work was further delayed by the lack of a management 

information system to collect data. Using a systems approach and working from a more robust 

understanding of the organization could have allowed OTA to focus on eliminating these barriers 

to progress and sequence activities to maximize impact.

To that point, the evaluation team observed that OTA helped RRA make progress in collecting 

outstanding arrears, but focused on process, including reducing the time to begin collection and 

appropriately staffing to resolve each case in a timely manner. However, the evaluation team 

believes that the project should have suggested that RRA develop a more comprehensive method 

to resolve the wider issues. For example, RRA arrears balance included a large number of 

nonactive registrants who discontinued business years ago. 

OTA should work with RRA, through its Single Project Implementation Unit that enables

follow-up activities to ensure the continuity of technical support and consistent interaction,

especially when providing intermittent or virtual support.

Staffing and Onboarding Considerations

SELECTION OF ADVISORS

Many OTA advisors came from a background in the Internal Revenue Service (IRS) at the 

executive level and have since retired from the IRS. Candidates’ seniority and years of 

experience make them attractive to OTA, but they often lack practical day-to-day operations 

expertise to translate recommendations to the frontline workers who implement the reforms. In 

addition, the field continues to evolve and the practices used during their tenure at IRS may no 

longer be appropriate. 

For example, much of OTA’s work was focused on the senior management level of RRA, but 

these capacity building efforts did not sufficiently trickle down to the required levels. For 

example, OTA suggested that RRA introduce a timesheet to capture staff time and effort spent 

on audits. An outdated template of a timesheet from the Internal Revenue Service was

introduced with very little adaptations or consultations. The process to introduce any complex 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

47

system change requires a significant organizational communication effort and collaboration with 

other relevant departments (such as Payroll, Human Resources, and Risk Management). The 

timesheet program was put together for RRA without these necessary considerations and 

consequently, the timesheet had limited use and was not trusted by employees who saw it as a 

control measure.

For future OTA projects, the evaluation team therefore recommends OTA consider the 

composition of the teams of advisors to encompass a wider range of experiences and 

backgrounds. 

Several counterparts and advisors reported that the program did not establish a primary, local 

technical contact for the major activities under the various components at RRA and 

communications between the advisors and counterparts were not always clear, which led to some 

gaps in activity. While OTA engaged a local project coordinator to provide communications, 

logistics, and administrative support for the program, the evaluation team recommends that 

OTA provide, in addition to or instead of the local project coordinator, a local person 

embedded among the counterparts with a higher-level technical role capable of working 

across multiple program component areas. 

ORIENTATION PACKAGE

The evaluation team recommends that all OTA advisors receive a thorough orientation package. 

The orientation package should include information from RRA staff on organizational structure, 

identified issues, business processes, and other information to get the OTA advisors prepared for 

the work. This insight provided by employees would have been a valuable timesaving tool. 

Orientation on Rwandan cultural and their political and legislative processes should have been 

provided or accommodated within the team using a local advisor. Several respondents suggested 

that technical assistance would be more effective if the local culture were understood.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

48

V. ANNEXES

Annex A. List of Documents Reviewed

Terms of reference—Rwanda Revenue Authority and U.S. Department of the Treasury—Office 

of Technical Assistance (OTA)

Project Logframe, OTA, 2019

Workplans for 2015–2016, 2017, 2018, 2019–2020, OTA

Monthly reports, OTA (59 documents)

Statements of work, OTA (66 documents)

Trip reports, OTA (82 documents)

Project narratives, OTA (6 documents)

Project performance reports, OTA (5 documents)

Closing memo, OTA

Evaluation coversheet, OTA 

Establish an RRA Future State Vision, OTA, March 2016

OTA’s Summary of Future State Vision Offsite, OTA, March 19, 2016

OTA—Revenue Administration Program, Rwanda End-of-Project Review, George Dodd and 

Leslie Lehrkinder, May 11, 2022

Debt Management Manual, RRA, Domestic Taxes Department

Taxpayer Perceptions on the RRA Appeal System, Strategy and Risk Analysis Department, 

Research and Policy Analysis Section, Theonille Mukamana, August 2020
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

49

Annex B. Key Informants Interviewed

NAMES REDACTED

RWANDA REVENUE AUTHORITY—PHYSICAL INTERVIEWS

Assistant Commissioner for Charge of Registration, Filing and Payment 

Compliance Improvement Planning Officer

Director of Effective Management of RRA Risk Registers

Professional in Charge DTD Performance Management

Single Project Implementation Unit Coordinator

Assistant Commissioner in Charge of Research, Planning, M&E Division

Tax Appeals Management and Dispute Resolution Officer

Assistant Commissioner in Charge of Human Resource Division

Institutional Review Officer/ Implementation of RRA New Operating Model

Director of Change Management and Development

Professional in Charge of Security and Safety

Audit Quality Review Officer

Deputy Commissioner General

Audit Support Officer

Assistant Commissioner in Charge of Debt Management Division

RWANDA REVENUE AUTHORITY—VIRTUAL INTERVIEWS

Commissioner General

Commissioner for Domestic Taxes

Commissioner for Internal Audit and Integrity

OTA—VIRTUAL INTERVIEWS

Advisor

Advisor

Program Manager and Advisor 

Advisor

AMERICAN CHAMBER OF COMMERCE IN RWANDA

Board President
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

50

Annex C. Evaluation Team 

Steve Crout spent 25 years with the Canadian Government in Employment Insurance, Goods 

and Service Tax, Investigations, and Income Tax Administrations. He started international work 

in 2001 and has worked in developing and post-conflict countries, covering all functions of the 

revenue administration. Steve resides in Western Canada.

Uzma Ashraf Barton is a Public Financial Management and Economic Policy expert with more 

than fifteen years of experience in more than 12 countries working on public finance, tax policy 

and administration, legal and financial market development, governance reform implementation, 

and capacity building and research. Her academic research is widely published, and her book,

Rethinking Finance, (Kluwer, 2018) explores issues of systemic financial risks and suggests 

sustainable financing options. Ms. Barton has a PhD and an LLM in Financial Law.

Joseph Kotun is a Director of Monitoring, Evaluation, and Learning at Bixal. He has more than 

25 years of technical experience in international development, focused on evaluation. Before his 

current role at Bixal, he worked extensively as an evaluation Team Leader for contractors with 

USAID, MCC, USDA, U.S. Department of State, World Bank, DFID, Asian Development Bank, 

and other private sector and NGO clients. He has a master’s degree in international development 

and management from the School for International Training Graduate Institute and a Bachelor of 

Science degree in Earth and Biological Sciences from McGill University. 

Michael Tatone is a Senior International Evaluation Project Manager at Bixal. He has 13 years 

of technical experience in evaluation and research, managing and providing technical support for 

evaluations and impact studies in conflict and violence in Colombia, Brazil, and Chicago.

Previous roles have included Research Manager at the University of Chicago and various Project 

Manager and Researcher roles in academia and NGOs. He has a Master of Political Science 

degree from Universidad de Los Andes in Colombia and a Bachelor of Arts degree in Political 

Science from University of Wisconsin-Eau Claire. 

Mats Alentun is a senior evaluation, program planning, and good governance expert with a 

strong international background. He has 15 years of technical experience in evaluation and 

evaluability and 20 years of technical experience in international development. His professional 

experience includes six long-term assignments in four developing countries (Mozambique, 

Zimbabwe, Tanzania, and Bangladesh), various short-term missions in Africa, Southeast Asia,

and the Balkans, as well as top positions in seven Swedish central government agencies. He has

a Master of Economic Sciences degree from the Swedish School of Economics and Business 

Administration in Helsinki, Finland. He currently resides in Kigali, Rwanda. 
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

51

Annex D. Evaluation team’s response to the OTA Revenue 

Team’s Comments

The evaluation team received the OTA Revenue Team’s written response to the evaluation report 

on July 17, 2023. Below is the evaluation team’s response: 

Thank you for taking the time to review and respond to the Evaluation Report for the Rwanda 

RRA project. Bixal and its partners strive to ensure accurate and objective findings and 

conclusions when completing any evaluation and appreciate the feedback provided to ensure this 

is achieved. While the evaluation team appreciates OTA’s perspective, the evaluation also 

gathered input from a variety of people and resource documents to support its conclusions. The 

purpose of the evaluation is to provide OTA with in-depth, third-party, retrospective insights of 

its project activities. 

It is generally acknowledged that some of the findings presented to the client may not be of a 

favorable nature but are meant to provide learnings and advance the process of technical 

assistance in Rwanda and to other countries.

The comments from the OTA Revenue Team focused on three areas:

1. Inaccuracies: the evaluation team have gone through all the points presented by the OTA 

Revenue Team and updated the report as appropriate. Not all the points were 

incorporated because the report may reflect a different point of view than that of the OTA 

Revenue Team based on the evaluation team’s access to a larger body of data.

2. Missing facts: many of these points in the OTA Revenue Team’s comments were

providing new information about the project in Rwanda which the evaluation team did 

not have when formulating its findings. The evaluation team did request all relevant 

information be provided to ensure the evaluation team would be able to conduct a

comprehensive review. The new information could not be incorporated into the findings 

of this report.

3. Contradictory statements: As with category number one, the evaluation team has gone 

through each point and updated the report if required. In many cases, the evaluation team 

feels that the report is accurate, and any contradictory statements may be a result of a

difference of definition or viewpoint.

As an evaluation team, we take the concerns of the OTA advisors seriously. The evaluation team 

did meet with the OTA Revenue Team on Aug 26, 2023, to discuss the issues further in a session 

separate from the presentation to the Management Team of OTA. Our goal is to make the 

evaluation process a learning event for everyone.
Office of Treasury Assistance: Rwanda Revenue Authority Program Evaluation Draft Final Report

52

Annex E. OTA Response to “Office of Technical Assistance: 

Rwanda Revenue Program Evaluation Report ”

OTA confirms receipt of the Program Evaluation Report by Bixal (dated August 1, 2023) related 

to OTA’s technical assistance engagement with the Rwanda Revenue Authority (RRA) from 

2015 to 2020. OTA provided a written response to the evaluation report on July 17, 2023, and 

OTA management met with Bixal representatives on July 28, 2023, to discuss the report’s 

findings and recommendations. The dialogue was both informative and insightful and OTA 

appreciates that Bixal incorporated technical corrections provided by the program. 

OTA recognizes the importance of independent, third-party project evaluations in compliance 

with the Foreign Aid Transparency and Accountability Act (FATAA). OTA also appreciates the 

opportunity for program learning presented by the evaluation and its findings and 

recommendations. Since the completion of the project with the Rwanda Revenue Authority in 

2020, OTA has taken significant steps to strengthen important aspects of the technical assistance 

project lifecycle, particularly as it relates to project design and monitoring. These changes 

address several of the recommendations included in the evaluation report, including development 

of logical frameworks, improved project scoping and problem definition, and systematized 

monitoring of project progress against defined outputs and outcomes. The program has also 

started to design outcome indicators for each project. Taken together, OTA believes that these 

actions will improve the program’s ability to achieve impactful and sustained reforms and 

counterpart capacity. Again, OTA appreciates the results and lessons gained from this 

evaluation and will ensure that they will be integrated, as appropriate, into future technical 

assistance projects.