International Journal of General Medicine

Dovepress
open access to scientific and medical research

O riginal R esearch

Open Access Full Text Article

The hospital educational environment andperformance of residents in the General Medicine
In-Training Examination: a multicenter study
in JapanThis article was published in the following Dove Press journal:
International Journal of General Medicine
26 July 2013
Number of times this article has been viewed

Taro Shimizu 1
Yusuke Tsugawa 2,3
Yusuke Tanoue 4
Ryota Konishi 5
Yuji Nishizaki 6
Mitsumasa Kishimoto 7
Toshiaki Shiojiri 8
Yasuharu Tokuda 9
Hospitalist Division, Departmentof Medicine, Nerima Hikarigaoka
Hospital, Tokyo, Japan; 2Divisionof General Medicine and Primary
Care, Beth Israel Deaconess Medical
Center and Harvard Medical School,Boston, MA, USA; 3Center for Clinical
Epidemiology, St Luke’s Life Science
Institute, 4Department of Vascularand Oncological Surgery, Hospitalof Tokyo University, 5Departmentof General Internal Medicine, Kanto
Rousai Hospital, 6Departmentof Cardiology, Juntendo University
School of Medicine, 7Divisionof Rheumatology, St Luke’s
International Hospital, Tokyo, Japan;
8Asahi Chuo Hospital, Chiba, Japan;
9Department of Medicine, Tsukuba
University Mito Kyodo GeneralHospital, Mito City, Ibaraki, Japan
1

Background: It is believed that the type of educational environment in teaching hospitals may
affect the performance of medical knowledge base among residents, but this has not yet been
proven.
Objective: We aimed to investigate the association between the hospital educational environment and the performance of the medical knowledge base among resident physicians in Japaneseteaching hospitals.
Methods: To assess the knowledge base of medicine, we conducted the General Medicine InTraining Examination (GM-ITE) for second-year residents in the last month of their residency.
The items of the exam were developed based on the outcomes designated by the Japanese
Ministry of Health, Labor, and Welfare. The educational environment was evaluated using thePostgraduate Hospital Educational Environment Measure (PHEEM) score, which was assessedby a mailed survey 2 years prior to the exam. A mixed-effects linear regression model wasemployed for the analysis of variables associated with a higher score.
Results: Twenty-one teaching hospitals participated in the study and a total of 206 residents(67 women) participated and completed the exam. There were no residents who declined toparticipate in the exam. The mean GM-ITE score was 58 (standard deviation 8.4). The mixedeffects linear regression analysis showed that a higher PHEEM score was associated with ahigher GM-ITE score (P = 0.02). Having a department of general medicine, and hospital location in a provincial community (versus an urban setting), were also shown to have a significant
relationship with the higher score (P = 0.03, and P = 0.02, respectively).
Conclusion: We found that the performance of the medical knowledge base of resident
physicians was significantly associated with the educational environment of their hospitals.
­Improvement of the educational environment in teaching hospitals might be crucial for ­enhancing
the performance of resident physicians in Japan.
Keywords: outcome-based education, postgraduate medical education, educational­environment, general medicine, provincial hospital

Introduction
Correspondence: Taro Shimizu
Hospitalist Division, Department ofMedicine, Nerima Hikarigaoka Hospital,
2-11-1, Hikarigaoka, Nerima-ku,
Tokyo 179-0072, Japan
Tel +81 3 3979 3611, ext 8015
Email shimizutaro7@gmail.com

submit your manuscript | www.dovepress.com

Dovepress
http://dx.doi.org/10.2147/IJGM.S45336

In Japan, after completing a 6-year medical undergraduate education, students enroll
in 2-year postgraduate residency training programs.1 Resident physicians are required
to undertake supervised training in general medicine, including internal medicine,emergency medicine, and primary care. The Ministry of Health, Welfare and Laborof the Japanese Government, regulates the training programs and states the set ofobjectives for resident physicians to achieve basic knowledge of, and skill in, general
International Journal of General Medicine 2013:6 637–640
637© 2013 Shimizu et al, publisher and licensee Dove Medical Press Ltd. This is an Open Access article
which permits unrestricted noncommercial use, provided the original work is properly cited.

Dovepress

Shimizu et al

medicine. When this national policy of 2-year postgraduate
residency training programs was introduced in 2004, thecomputer-based national matching system was introduced
and students could apply to their preferred hospitals
throughout the entire country.2 In our previous study, thegreater satisfaction of non-university residents was showncompared with university residents in Japan.3 In that study,
to compare the educational environments of university andnon-university hospitals, we used the Postgraduate Hospital
Educational Environment Measure (PHEEM), which isa 40-item inventory, and an evaluation tool consisting ofa validated questionnaire with three subscales. PHEEM
is considered to be a reliable and validated instrument, toevaluate the quality assurance process for the educational
environment of a teaching hospital.4–9
It is not clear whether teaching hospitals with a better
educational environment could provide an educational program for their resident physicians, to achieve greater basic
knowledge, engagement, and skill in general medicine. In thiscontext, we conducted a multicenter study with the aim ofinvestigating the association between a hospital educational
environment and the knowledge base of medicine in Japan.
To assess the knowledge base, we developed and used theGeneral Medicine In-Training Examination (GM-ITE) basedon a similar methodology used to develop the In-Training
Examination in Internal Medicine (IM-ITE) which has beenoffered annually to all trainees in US medical residency
programs since 1988.10Methods
Measurements
For the outcome assessment, we developed and used the GMITE by employing the similar methodology used to develop
the IM-ITE of the USA. The purpose of the US-ITE is toprovide residents and program directors with an objective,
reliable, and valid assessment of each resident’s personal
performance in a written, multiple-choice examination, andthe performance of each residency program compared withthat of its peers.11,12
The GM-ITE included 100 questions testing a widerange, from clinical skills and pragmatic medical knowledge
to psychosocial care of the patient, designed and written by
a committee of experienced attending physicians organizedby the Japan Organization of Advancing Medical Education
Program (JAMEP, a non-profit organization). The examination gives a maximum score of 100 and a minimum score ofzero; higher scores indicate a better performance of knowledge base of internal medicine. The examination question

638

submit your manuscript | www.dovepress.com

Dovepress

review was conducted and the content validity was confirmed
by peer review of each member of the committee. A 2-hour
fixed time-limit was set for taking the exam.

Participants and data collection
We conducted a multicenter study involving Postgraduate
Year 2 (PGY-2) resident physicians at 21 teaching hospitals in Japan. The invitation to participate in the study wasannounced on the JAMEP website. Teaching hospitals which
voluntarily participated in our study provided the mean scoreof the PHEEM, which was collected in a survey conducted
2 years prior to the exam.3
The program director at each hospital was required toassemble residents in a room at each hospital at a scheduled
time, and provide the GM-ITE for their residents. Each
program director then collected the completed examination
answer sheets, and sent these back to us, in an envelope we
had provided. As the academic calendar in Japan starts onApril 1 and ends on March 31 of the following year, the GMITE was conducted in March of the 2011 academic year. This
study was approved by the institutional review board of theMito Kyodo General Hospital, Mito City, Ibaraki, Japan.

Statistical analyses
We tested the hypothesis of significant association between
the PHEEM score and GM-ITE score by constructing amixed-effects linear regression model to account for theclustering of the individuals within each residency program.13
Covariates as potential confounders included having a department of general internal medicine (GIM) or general medicine(GM) and hospital location (provincial or urban). Hospitals
with a department of GIM or GM are considered to provide
better educational programs for general medicine. Hospitals
located in provincial communities are considered to havea better chance of seeing primary care patients than thoselocated in urban areas. Statistical analyses were conductedusing STATA version 11 (StataCorp LP, College Station,TX, USA). A 2-tailed P , 0.05 was considered statisticallysignificant.

Results
A total of 206 PGY-2 resident physicians-in-training(67 women, 33%) participated and completed the examination.
There were no residents who declined to participate in theexam. Of the PGY-2 physicians, 34.5% were women and 65.5%
men.14 Among all participants, the mean score of the exam was57.7 (standard deviation [SD], 8.4). The mean PHEEM score
was 57.6 (SD, 5.4) among 21 hospitals. There were 14 teaching

International Journal of General Medicine 2013:6

Dovepress

General Medicine In-Training Examination in Japanhospitals with a department of GIM or GM. Thirteen hospitals(62%) were located in a provincial community, while eighthospitals (38%) were located in an urban area. The hospitalcharacteristics of the top 10 residents are shown in Table 1.
The results of the mixed-effects linear regression analysis(Table 2) showed that: a one-point increase in PHEEM score
was associated with a 0.16 (95% Confidence Interval; CI,0.023–0.29, P = 0.02) point higher GM-ITE score with betaof 0.16; having a department of GIM or GM was associatedwith a 4.07 (95% CI, 1.45–7.69, P = 0.03) point higher
GM-ITE score; and a hospital located in a city (versus aprovincial community) was associated with a −4.59 (95%CI, −8.46 to −0.72, P = 0.02) point lower GM-ITE score. Thissuggests that adjusted mean PHEEM scores were about 4.6
points lower among residents in urban hospitals than among
those in provincial hospitals.

Discussion
Residents in teaching hospitals with higher PHEEM scores,
with a GIM or GM department, or in a provincial program,demonstrated a better knowledge base of medicine. To thebest of our knowledge, this is the first study to provide theevidence that the educational environment is associated withenhanced performance of knowledge base in the area ofgeneral medicine. The educational environment in a teaching
hospital is likely to determine the engagement and motivation of residents and lead to greater achievement of educational ­outcomes.9 Because the program satisfaction level ofresidents also depends on their engagement and motivation,teaching hospitals with a better educational environment may
also lead to greater satisfaction among residents.2
The lack of a GIM (or GM) department in teaching hospitals is not rare among Japanese teaching hospitals. Historically,
Table 1 The hospital characteristics of the top ten residents for
the GM-ITE score
Ranking #of residents

Provincial/urban

GIM/GMThe PHEEMscore

12
3
4
5
6
7
8
9
10

Provincial
Urban
Provincial
Provincial
Provincial
Provincial
Provincial
Provincial
Provincial
Provincial

Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes

149
99
149
149
102
102
121
106
101
149

Abbreviations: GM-ITE, general medicine in-training examination; PHEEM,
postgraduate hospital educational environment measure; GIM, general internal
medicine; GM, general medicine.

International Journal of General Medicine 2013:6

Table 2 The results of the mixed effects linear regression modelfor the GM-ITE score
Variable

Beta

95% CIPHEEM score
GIM/GM (vs no GIM/GM)
Located in urban area (vs provincial)

0.16
4.07
-4.59

0.023
1.45
-8.46

P-value
0.29
7.69
-0.72

0.02
0.03
0.02

Abbreviations: CI, confidence interval; GM-ITE, general medicine in-training
examination; PHEEM, postgraduate hospital educational environment measure; GIM,general internal medicine; GM, general medicine; vs, versus.

Japanese hospitals did not have subspecialty medical divisions
such as cardiology, gastroenterology, nephrology; they had
only a department of medicine before the 1980s. During the1980s, there was a boom in subspecialty medical divisions.
Reflecting that change, no general medical department existed
in Japanese hospitals by about 1990. However, after the 1990s,
with an aim to educate every physician to retain the generalist’s mind even if the person is a specialist, a department ofGIM or GM was established in multiple teaching hospitals tocontribute significantly to clinical care, research, and clinicaltraining. Since the GIM or GM departments cover all subjectsof medicine and has a broad view of patients’ symptoms
and concerns, ie, not focusing on a single organ system, it
is understandable that programs with GIM or GM positively
affect the knowledge base of general medicine.
Medical residents in provincial hospitals performed better in the GM-ITE scores in our study. Training programs inprovincial areas may offer better learning opportunities, andthus improve learning outcomes in general medical knowledge
and skills. Physicians, including primary care physicians, arescarce in provincial areas, hence residents in provincial teaching hospitals have to see and solve common medical problems
of primary care patients. Due to the scarcity of physicians insuch areas, patient exposure may be richer and more diverse
in provincial hospitals than in city area hospitals.
Limitations of our study are: first, our results might havebeen influenced by sampling bias. Although the participants
were conveniently selected and assigned to the examination,
they did not necessarily represent all Japanese residents.
Other confounding factors may be the basic academic
characteristics of the residents before the postgraduate­training. Whether the higher GM-ITE score was due toa ­better educational environment or due to hard-working­students in those hospitals remains uncertain; perhaps­students who are good at taking examinations have the choiceof selecting good hospitals, and the remaining students go
to the other hospitals.
Further study will involve more participants across Japan,
which would lessen the bias. Second, with respect to thesubmit your manuscript | www.dovepress.com

Dovepress639

Dovepress

Shimizu et al

cross-sectional study design, causality cannot be determined
and thus the results require careful interpretation.

Conclusion
We conclude that the educational environment in a teaching
hospital has been shown to affect the engagement andmotivation for clinical training among residents. Training inprovincial community hospitals with a GIM division offers
residents greater exposure to patients with more common
medical problems than highly specialized hospitals, thereby
giving these residents the chance to cultivate their clinical skills
and pragmatic knowledge. The result of this exam may be usefulfor designing an improved model of postgraduate medical
education/residency training in Japan, also by collaboratingwith the Ministry of Health, Welfare and Labor of the Japanese
Government. Even among teaching hospitals in non-provincial
settings, supportive establishment of GIM division would
improve the teaching outcome of the residents.

Acknowledgment
This research was funded by the JAMEP.

Disclosure
The authors have no conflicts of interest to declare.

References

1. Kozu T. Medical education in Japan. Acad Med. 2006;81(12):1069–1075.
2. Nomura K, Yano E, Mizushima S, et al. The shift of residents fromuniversity to non-university hospitals in Japan: a survey study. J GenIntern Med. 2008;23(7):1105–1109.

3. Tokuda Y, Goto E, Otaki J, et al. Educational environment of university
and non-university hospitals in Japan. Int J Med Educ. 2010;1:10–14.
4. Roff S, McAleer S, Skinner A. Development and validation of an
instrument to measure the postgraduate clinical learning and teachingeducational environment for hospital-based junior doctors in the UK.
Med Teach. 2005;27(4):326–331.
5. Aspegren K, Bastholt L, Bested KM, et al. Validation of the PHEEM
instrument in a Danish hospital setting. Med Teach. 2007;29(5):498–500.
6. Boor K, Scheele F, van der Vleuten CP, Scherpbier AJ, Teunissen PW,Sijtsma K. Psychometric properties of an instrument to measure theclinical learning environment. Med Educ. 2007;41(1):92–99.
7. Gooneratne IK, Munasinghe SR, Siriwardena C, Olupeliyawa AM,
Karunathilake I. Assessment of psychometric properties of a modified PHEEM questionnaire. Ann Acad Med Singapore. 2008;37(12):993–997.
8. Riquelme A, Herrera C, Aranis C, Oporto J, Padilla O. Psychometricanalyses and internal consistency of the PHEEM questionnaire tomeasure the clinical learning environment in the clerkship of a Medical
School in Chile. Med Teach. 2009;31(6):e221–e225.
9. Vieira JE. The postgraduate hospital educational environment measure(PHEEM) questionnaire identifies quality of instruction as a key factor
predicting academic achievement. Clinics (Sao Paulo). 2008;63(6):741–746.
10. Garibaldi RA, Subhiyah R, Moore ME, Waxman H. The In-Training
Examination in Internal Medicine: an analysis of resident performanceover time. Ann Intern Med. 2002;137(6):505–510.
11. Kanna B, Gu Y, Akhuetie J, Dimitrov V. Predicting performance using
background characteristics of international medical graduates in an
inner-city university-affiliated Internal Medicine residency training
program. BMC Med Educ. 2009;9:42.
12. Perez JA Jr, Greer S. Correlation of United States Medical ­Licensing Examination and Internal Medicine In-Training Examination ­performance. AdvHealth Sci Educ Theory Pract. 2009;14(5):753–758.
13. Blatt B, Greenberg L. A multi-level assessment of a program to teach medicalstudents to teach. Adv Health Sci Educ Theory Pract. 2007;12(1):7–18.
14. Japanese Ministry of Health, Labor and Welfare [Questionnaire survey for persons who have completed postgraduate medical training inJapan.] 2011. [Japanese] Available from: http://www.mhlw.go.jp/stf/
shingi/2r9852000001hia9-att/2r9852000001hqdl.pdf. Accessed July 1,2013.

Dovepress

International Journal of General Medicine

Publish your work in this journal
The International Journal of General Medicine is an international,
peer-reviewed open-access journal that focuses on general and internal
medicine, pathogenesis, epidemiology, diagnosis, monitoring and treatment protocols. The journal is characterized by the rapid reporting ofreviews, original research and clinical studies across all disease areas.

A key focus is the elucidation of disease processes and management
protocols resulting in improved outcomes for the patient.The manuscript management system is completely online and includes a very
quick and fair peer-review system. Visit http://www.dovepress.com/
testimonials.php to read real quotes from published authors.

Submit your manuscript here: http://www.dovepress.com/international-journal-of-general-medicine-journal

640

submit your manuscript | www.dovepress.com

DovepressInternational Journal of General Medicine 2013:6