Program Evaluation Summary

Rural Generalist Program Japan (2017-2019)

Background
Medical practitioners are concentrated in urban
areas, and the Japanese specialist-driven
approaches in medical education and public healthhave resulted in disproportionate distribution of
medical services in different parts of the country.

Geography
anddemographics•More than 400 islands•Increasing elderly populations•Increasing chronic disease•Challenging retrieval and patienttransport mechanisms

Workforce•Maldistribution of doctors
between urban and rural•Lack of pathway dedicated torural/remote areas•Working in Island Health not seen
as attractive career•No political drive and funding toincrease rural doctors

Context
specifictrainings•Competencies for rural/remote
generalists differ from urban
hospitalists•Mismatch between the skills of
the doctors and the health needsof the community•Lack of training opportunities

Of the 10 million Japanese people residing in ruraland remote areas, more than 50% of the population
are elderly. Their communities are facing social,economic and political difficulties with a shrinking
labor force, health inequalities and disappearanceof cultural heritage. Figure 1 outlines the issues andchallenges in rural and remote medicine in Japan.
There has been an effort by the Japanese
government to increase the number of ruralgeneralists to overcome such situations. However,
the intricate balance is required in developing aneffective program which focuses on connectedness
with local communities, understanding local culture
and provision of specialist care.
Rural Generalist Program Japan was born out ofsuch concerns and strives to train and develop a
network of rural generalist who contribute toremote medicine in Japan.

Figure 1. Issues and Challenges in rural and remote medicinein JapanCurrent Training System for Rural Generalists in JapanA new board-certification system started in 2018 by the Japanese Medical Specialty Board (JMSB) and ‘General
Practice’ was added as the 19th major clinical discipline in Japan. Certification of family physicians/generalpractitioners is currently managed by the Japan Primary Care Association (JPCA), but has been in the process oftransition from JPCA to JMSB. The total number of JPCA-certified family physicians was only 672 (0.2%) of311,205 physicians in 2018.
Training system for general medicine/family
medicine under JMSB is heavily based onthe hospitalist model. Rural Generalist
Program Japan is considered as the only
training model in Japan that has beendeveloped to educate and train physicians
to serve in the rural and remote
Figure 2. General Practice Training in Transition
communities.
Presented at the Board Meeting at the Australian College of Rural and Remote Medicine (ACRRM) on February 22, 2020.
Prepared by Manabu Saito, Director, Rural Generalist Program Japan (RGPJ), GENEPRO LLC, Japan and Takara Tsuzaki,
Interdisciplinary Ph. D in Evaluation, Western Michigan University, USA.

-1-Overview – Rural Generalist Program Japan
The program was developed to ensure it was tailored to regional and local contexts, focussing on provision of abroad scope of clinical care with a unique combination of abilities and aptitude to respond to the communityneeds. This brought together various members of the health care team including General Practitioners or Family
Physicians (GPs/FPs) with community-based primary care roles, hospitalists, emergency physicians and a rangeof consultant medical specialists.
The purpose of the Rural Generalist Program Japan wasdefined as:
1. To improve access to health care and health outcomes forrural and remote areas in Japan
2. To increase rural medical workforce
3. To increase in appeal and generate interest in doctors
pursuing career in rural medicine
4. To introduce the concept of rural generalism andestablishment as a specialty
The RGPJ is a 15-month training program which includes both
local and distance supervision of registrars, on-site training at
training hospitals, webinars and elective placements as
illustrated in Figure 3. There are three components of registrar
training: 1) onsite domestic training in a rural or remotehospital; 2) webinars featuring online discussions and lecturesby rural generalists; 3) elective training.

The program is certified by the Australian College of Rural andRemote Medicine (ACRRM). Registrars who complete training
requirements and the assessment program are awarded the"Certificate of Rural Medicine" jointly by ACRRM and RGPJ.

Webinars•Weekly webinars in
Medical Knowledge andSkill Training•English language skills
improvement/Training

ElectiveTraining•Up to 3 months(Australia, Nepal, USA,Mongolia, Japan etc.)

Figure 3: Overview of the Rural Generalist Program
Japan (RGPJ)Registrars

Training Hospitals

Registrars have substantial prior experiences. A prerequisite for entry to the RGPJ training is completionof two years of postgraduate internship andcommitment to rural/remote medicine. The medianage for registrars in Cohort 1, 2 and 3 was 35 years
old with the median postgraduate years of 8. Most
had achieved specialist status, including Emergency
Medicine. Table 1 is the summary of registrar
demographics for Cohort 1-3.

Registrars have undertaken training at six hospitals.
GENEPRO’s training hospitals are selected based on
the following criteria:• rurality/population,• deep and acute understanding of issues
surrounded the rural medicine• dedication to solving workforce shortages• commitment to training young doctors as future
investment/collaborative endeavour rather thantreating them as temporary human resources attheir hospital.

12

DomesticTraining•12 months at Training
Hospital•Clinical visits ( 4 times peryear)•On-site Workshop (twiceper year)

Cohort andcommencing
year

Registrars
Commencing

Median Age Range atcommencement

Post Graduate Registrars CompletingYear atcommencement(median)
4-13 (9)6 (5M/1F)

1 – 2017

7 (6M/1F)37

2 – 2018

12 (10M/2F)

31

3-25 (6)10 (8M/2F)

3 – 2019

6 (3M/3F)38

5-13 (9.5)

6 (3M/3F) *Total

2535

822

Table 1: Registrar demographics*Cohort 3 is expected to complete in June 2020.

Figure 4 : Training Hospitals

Presented at the Board Meeting at the Australian College of Rural and Remote Medicine (ACRRM) on February 22, 2020.
Prepared by Manabu Saito, Director, Rural Generalist Program Japan (RGPJ), GENEPRO LLC, Japan and Takara Tsuzaki,
Interdisciplinary Ph. D in Evaluation, Western Michigan University, USA.

-2-Program Evaluation
Evaluation Purpose
Program evaluations was conducted to inform improvements (formative)
and provide impartial and independent evidence for accountability anddecision making (summative). In order to understand the extent to which
the overall objectives were met, the evaluation incorporatedorganizational and managerial aspects. The scope of this evaluation
included; needs assessment; lessons learnt from the first two cohorts ofregistrars; organizational and management systems to support theregistrars; and refining and improving the training program.
The first program evaluation took place in 2019, with the main focus onregistrars from cohort 1 and 2. Other key stakeholders included directors
at training hospitals, administrative staff, mentors and learning advisors.
Since most stakeholders were Japanese, surveys and interviews wereconducted in Japanese.
Program evaluation is important for setting future directions andproducing lessons learned for further improvement. However, given theprogram is in its infancy and the number of registrars are still small,there may well be lack of consensus, but this data and analysis from thefirst program evaluation is expected to inform program development
and subsequent evaluations for the future.

Evaluation Questions•

What effect, if any, does RGPJ
have on the growth ofregistrars?

•What effect, if any, does RGPJ
provide competencies andconfidence in registrars?

•

What, if any, changes occur inregistrars’ awareness ofdifferent socioeconomic
circumstances in rural andremote settings?

•

What, if any, effect does RGPJ
have on registrars’ behavioral
and aspirational changes andtheir future prospect?

Evaluation Process & Approach
The approach for this program evaluation was non-experimental and relied on mixed-methods. The evaluation
and research methods applied were primarily qualitative, with a quantitative element limited to descriptivestatistics. The approach had participatory characteristics and aimed at contributing to the learning/actionoriented approach. The evaluation approach was also utilization focused in that there was clarity from theonset as to who the evaluation was for and how the results would be disseminated and used.

1. Survey administration (Typeform with video introduction). The overall response rate was 75％.
QCP elective training survey response rate was 100%
2. Face-to-Face and online interviews (60 min X 14 respondents)
3. Transcription and data analysis (mixed-methods coding with DeDoose)

Presented at the Board Meeting at the Australian College of Rural and Remote Medicine (ACRRM) on February 22, 2020.
Prepared by Manabu Saito, Director, Rural Generalist Program Japan (RGPJ), GENEPRO LLC, Japan and Takara Tsuzaki,
Interdisciplinary Ph. D in Evaluation, Western Michigan University, USA.

-3-

Data Snapshot
The following data collection and analysis methods were used to collect data for RGPJ program evaluation.
- Document review: program documents, Handbook, academic journal, policy papers, conference presentations,newspaper articles, and program reports.
- Surveys administered to registrars and supervisors at the teaching hospitals in Japan- Key informant interviews: Program Director, management staffs, registrars (Cohort 1 and 2), and supervisors at
the teaching hospitals

Membership to Medical Associations and Societies
Japanese SocietyofAnesthesiologists
4%

Japan Society of
Obstetrics andGynecology 4%

The Japanese
Circulation
Society 4%

Australian Collegeof Rural andRemote Medicine(ACRRM) 4%
The JapaneseSociety ofInternal Medicine
30%Japanese Societyof Emergency
Pediatrics 4%Japan PediatricSociety 4%
The Japanese
Associationfor Infectious
Diseases 4%
The Japanese
UrologicalAssociation
4%

Japan PrimaryCareAssociation
23%

Japanese SocietyofTravel Medicine
4%Japanese
Association forAcute Medicine
11%

Presented at the Board Meeting at the Australian College of Rural and Remote Medicine (ACRRM) on February 22, 2020.
Prepared by Manabu Saito, Director, Rural Generalist Program Japan (RGPJ), GENEPRO LLC, Japan and Takara Tsuzaki,
Interdisciplinary Ph. D in Evaluation, Western Michigan University, USA.

-4-Findings

Advocacy, Research andPolicy

Career aspirations andconfidence
RGPJ program stimulated and reaffirmed registrars’interests in rural/island generalist practice.

Building public awareness and engaging in researchand policies are integral part of developing better
rural generalist practice in Japan.

“In Japan, remote/rural/island medicine is supported“In Japan, medical decisions are often left on theshoulders of the medical professionals and doctors aresolely responsible for making important decisions. Thiscan be overwhelming. Building awareness andeducation in the general public is important.”

primarily by the dedication of a few enthusiasticdoctors.”“I had felt that it would be extremely difficult for
female doctors with children to continue engaging inremote, rural and island medicine; however, I now[after the elective training] see some hopes that it ispossible. My experiences through the elective training
reaffirmed that there are ways to secure and retain
medical professionals in rural Japan.”“I learned about the importance of having a bigger
picture and system approach in rural medicine. Thereare issues that can be tackled on site, but some arerelated to policy issues. Australia taught me different
ways of looking at issues and problems we are facing inJapan.”“Research is an integral part of the general practice. I
now understand the importance of research whileengaging in practice. I want to learn how to doresearch especially where research is neglected -- ruraland remote areas in Japan.”

Leadership andManagement

Lessons-Learnedtransferred to Japan

Developing competencies in both individual andteam levels are essential to bring the rural generalist
practice to a higher level in Japan.

Registrars are contributing their learnings andexperiences in their homeland.

“I learned about the importance of standardization of“Physicians are always leaders in hospitals as well ascompetencies and guidelines, supporting system ofpatients transfer, securing of continuous training
opportunities, and assuring sufficient income andholiday for rural generalists.”

in communities. In remote areas, it is important to be
humble and have accurate understanding of his/hercompetencies.”“I was impressed by how QCP was managed underclear philosophy and vision, rather than replying on
the efforts of individual hospitals and [ad hoc] efforts
by medical doctors or staffs.”“I work as a GP in the General Practice Department
in a small town and continue providing the necessary
medical care in the community. At the same time, I
initiated a new GP training program for medicalstudents and residents based on my learning andexperience [through RGPJ and elective training].”“It may take a long time, but I have come to believe
more strongly that systematic approach is necessary
in Japan.”Presented at the Board Meeting at the Australian College of Rural and Remote Medicine (ACRRM) on February 22, 2020.
Prepared by Manabu Saito, Director, Rural Generalist Program Japan (RGPJ), GENEPRO LLC, Japan and Takara Tsuzaki,
Interdisciplinary Ph. D in Evaluation, Western Michigan University, USA.

-5-Recommendations
The following themes emerged as recommendations for program improvement. Roles and perspectives ofregistrars and supervisors at training hospitals are key predictors for success. As identified below, there weremultiple (sometime contesting) perceptions of success and various definitions around rural/remote/island
medicine and rural generalist medicine.
Communication and relationship building among stakeholders are crucial as they have direct implications for
program delivery, recruiting, and public relations.
Articulation of vision and objectives→ Clarifying what “Providing the best training program for Japanese rural generalism” means for each stakeholders,
what qualifications they need to have, and what components would contribute to these qualifications.
What do we measure? What do we value?
→ Increase in quantitative values (#s) or qualitative values for registrars/training hospitals?
Timeline. When do we know that we are on the right track?
→ Immediate outcome/impact on the program level?
Long-term effect on the policy level and educational reform in Japan?
Individual capacity? Collective impact?
→ Immediate outcome/impact that can be measured and assessed on individual level (competencies) ?
collective capacity development as a team of rural generalists?

Next StepLessons Learned: Program Director

1

Rural workforce development with focus on
quality, safety and efficiency

2

RGPJ as leadership development program –“beyond” generating #s of rural generalists
in Japan3

4

Clear focus of the program and areas ofimprovement- Human relations, communication withstakeholders etc.
Increased accountability and transparency- For registrars and training hospitals inJapan- For accrediting institutions in Australia

Develop assessment tools and explore
evaluation approaches to evaluate program
impact and training outcomes for RGPJ longitudinal study and cost-analysis

Empowerment in every aspect ofmanagement and programing- Multiple definitions of success and goals
based on different values- Changes in perspectives and behaviors

Engage in international research andcomparative studies in rural generalist
medicine

Evidence-based public relations anddevelopment of communication tools

Presented at the Board Meeting at the Australian College of Rural and Remote Medicine (ACRRM) on February 22, 2020.
Prepared by Manabu Saito, Director, Rural Generalist Program Japan (RGPJ), GENEPRO LLC, Japan and Takara Tsuzaki,
Interdisciplinary Ph. D in Evaluation, Western Michigan University, USA.

-6-

Annex: ProfileTakara Tsuzaki, MSc. is an evaluation
professional with 15 years of experience inproviding strategic planning and execution,
policy research, fund raising, public relations
and monitoring and evaluation in four sectors:
academic institutions, public sector, privatesector, and the nonprofit sector. She has
worked extensively in the fields of international
development, community development,
economic development, social impact andmedical education at UN-HABITAT, National
Graduate Institute for Policy Studies (GRIPS),
Goldman Sachs Japan and Kurume University
Graduate School of Medicine.

Manabu Saito M.D.
CEO, GENEPRO LLC.
Director of Rural Generalist Program JapanFJEMA (Board Certified Fellow of Japanese
Emergency Medicine Association)FJPCA (Board Certified Senior Member of theJapanese Primary Care Association)
Ph.D. Candidate in Medical Education in
Juntendo University School of Medicine
Dr. Manabu Saito is an Emergency Physician anda General Practitioner in Japan. Graduated from
Juntendo University School of Medicine in 2000,
Dr. Saito worked as an emergency physician atOkinawa Urasoe General Hospital andTokunoshima Island Tokushukai Hospital in
Kagoshima Prefecture, and has extensive
experience in home care and remote island
medicine.

Ms. Tsuzaki is currently enrolled in the Ph.D.
Program in Interdisciplinary Evaluation atWestern Michigan University and pursuing her
research in evaluation theories from bothWestern and Eastern philosophies, sociocultural and language translatability issues in
evaluation, and evaluative thinking ineducation. Her academic qualifications include
MSc. in Non-governmental Organisational
Management, Social Policy at London School ofEconomics and Political Science and B.A. inAnthropology and International Studies at
Macalester College. She has lived and worked
in India, Nepal, Switzerland, the UK, and theUSA, and fluent in Japanese and English.

In 2015, he established GENEPRO, a joint venture
aimed at eliminating the shortage of doctors on
rural areas and remote islands. In April 2017,
Rural Generalist Program Japan (RGPJ) waslaunched to support medical training in rural
areas and remote islands in Japan. Training
scheme and curriculum of RGPJ have adopted
the best practices and lessons-learned fromaround the world to meet the needs of Japanesecontext in rural, remote and island medicine.

takara.tsuzaki@wmich.edu

saito@genepro.org-7-