International Journal of General Medicine

Dovepress
open access to scientific and medical researchO r igi n a l R e s e a r c hInternational Journal of General Medicine downloaded from https://www.dovepress.com/ by 54.191.40.80 on 01-Apr-2017
For personal use only.

Open Access Full Text Article

Impact of inpatient caseload, emergency
department duties, and online learning resource
on General Medicine In-Training Examination
scores in Japan
This article was published in the following Dove Press journal:
International Journal of General Medicine
30 October 2015
Number of times this article has been viewed

Kensuke Kinoshita 1
Yusuke Tsugawa 2
Taro Shimizu 3
Yusuke Tanoue 4
Ryota Konishi 5
Yuji Nishizaki 6
Toshiaki Shiojiri 7
Yasuharu Tokuda 8
Department of Medicine, Mito
Kyodo General Hospital, Universityof Tsukuba, Mito City, Ibaraki, Japan;
2
Department of Health Policy andManagement, Harvard T.H. Chan
School of Public Health, Boston, MA,
USA; 3Tokyo Joto Hospital, Koto-ku,
Tokyo, 4Good Medicine Japan, Miyagi,
5Department of General Internal
Medicine, Kanto Rosai Hospital,
Kawasaki, Kanagawa, 6Departmentof Cardiology, Juntendo University
Graduate School of Medicine, Tokyo,
7
Department of General Internal
Medicine, Asahi General Hospital,
Asahi, Chiba, 8Japan Community
Healthcare Organization, Minato-ku,
Tokyo, Japan
1

Correspondence: Kensuke Kinoshita
Department of Medicine,
Mito Kyodo General Hospital,
University of Tsukuba, 3-2-7 Miya-Machi,
Mito, Ibaraki 310-0015, Japan
Tel +81 29 231 2371
Fax +81 29 221 5137
Email kensuke.kinoshita@nifty.ne.jp

Introduction
Teaching hospitals are expected to provide high-quality educational environments toresidents in order to enable them to attain a greater clinical knowledge and experiences.
However, educational environments comprise many factors, including workloads, learning resources, clinical teachers, and hospital educational cultures. Among these factors,
the programs of teaching hospitals can modify workloads and learning resources,
whereas the behaviors of attending physicians and hospital educational cultures canbe difficult to modify. Therefore, it is in our interest to evaluate how workloads andlearning resources would affect the learning experiences of residents.
Our previous study showed that better, overall educational environment at teaching
hospitals was associated with greater medical knowledge, but the study did not provide
effects of each component of the educational environments on clinical performanceof residents.1 Among components of educational environments, inpatient workload
may be a factor for better clinical performance, since clinical experience is considered

355

submit your manuscript | www.dovepress.com

International Journal of General Medicine 2015:8 355–360

Dovepress© 2015 Kinoshita et al. This work is published by Dove Medical Press Limited, and licensed under Creative Commons Attribution – Non Commercial (unported, v3.0)
License. The full terms of the License are available at http://creativecommons.org/licenses/by-nc/3.0/. Non-commercial uses of the work are permitted without any further
permission from Dove Medical Press Limited, provided the work is properly attributed. Permissions beyond the scope of the License are administered by Dove Medical Press Limited. Information on
how to request permission may be found at: http://www.dovepress.com/permissions.php

http://dx.doi.org/10.2147/IJGM.S81920
Powered by TCPDF (www.tcpdf.org)

Background: Both clinical workload and access to learning resource are important components
of educational environment and may have effects on clinical knowledge of residents.
Methods: We conducted a survey with a clinical knowledge evaluation involving postgraduate year (PGY)-1 and -2 resident physicians at teaching hospitals offering 2-year postgraduate
training programs required for residents in Japan, using the General Medicine In-Training
Examination (GM-ITE). An individual-level analysis was conducted to examine the impact of
the number of assigned patients and emergency department (ED) duty on the residents’ GM-ITE
scores by fitting a multivariable generalized estimating equations. In hospital-level analysis, weevaluated the relationship between for the number of UpToDate reviews for each hospital andfor the hospitals’ mean GM-ITE score.
Results: A total of 431 PGY-1 and 618 PGY-2 residents participated. Residents with four orfive times per month of the ED duties exhibited the highest mean scores compared to those withgreater or fewer ED duties. Those with largest number of inpatients in charge exhibited thehighest mean scores compared to the residents with fewer inpatients in charge. Hospitals withthe greater UpToDate topic viewing showed significantly greater mean score.
Conclusion: Appropriate ED workload and inpatient caseload, as well as use of evidence-based
electronic resources, were associated with greater clinical knowledge of residents.
Keywords: workload, online resource, knowledge, resident, postgraduate, Japan

Dovepress

International Journal of General Medicine downloaded from https://www.dovepress.com/ by 54.191.40.80 on 01-Apr-2017
For personal use only.

Kinoshita et al

to enhance learning in clinical medicine. Previous studies­suggested that the cumulative number of inpatients in charge
was associated with greater score of In-Training Examination(ITE) in internal medicine residents,2 and the cumulative
number of general pediatric admissions was correlated withan increase in ITE score in pediatric residents.3 However,these studies evaluated the cumulative number of inpatientsas a workload for residents. A few studies have investigated
the association between the average number of inpatientsin charge at a time as a real-time caseload for residents andclinical knowledge in residents. This number may better
capture the actual workload on daily basis. In addition, a
few studies have investigated the association between thefrequency of emergency department (ED) duties and clinicalknowledge in residents. This number could reflect the actual
exposure to emergency patients and could also be used asdaily workload.
In this context, we aimed to investigate the associationof clinical performance and objective workload using the
average number of inpatients caseload in charge at a timeand ED duties frequency.
Regarding learning resources, previous studies indicatedhigher scores of residents’ clinical performance for theamount of self-learning time in anesthesiologist residents
ITE4 and in internal medicine residents5,6 and self-learningby online resources7 in the Internal Medicine ITE (IM-ITE)
and the amount of self-learning time. We also evaluatedthe online learning resources as a factor for greater clinicalperformance in residents since these resources are popularin modern clinical environments, in conjunction with theworkloads.

Methods
Participants and data collection
We conducted a multicenter study involving postgraduateyear (PGY)-1 and -2 resident physicians at 114 teaching
hospitals in Japan. The participants were trainees of 2-year
postgraduate rotation training programs, including internal
medicine, emergency medicine, community medicine, andothers (PGY-1 and -2), required for all residents regardlessof their specialty, before getting into the specialty training
programs (PGY-3 or later). The invitation to participate
in the study was announced on the Japan Organization ofAdvancing Medical Education Program (JAMEP: Nonprofit
Organization, Tokyo, Japan) Web site, monthly medical
journal advertisement (Resident Note; Yodosha Co, Tokyo,
Japan), and was sent to program directors of all 1,015 teaching hospitals in Japan certified by Ministry of Health, Labor,

356

Powered by TCPDF (www.tcpdf.org)

submit your manuscript | www.dovepress.com

Dovepress

and Welfare, including 114 university hospitals and 901
community teaching hospitals.
The residency program directors of each hospital wererequired to assemble residents in a room at each hospital
at a scheduled time and provide the General Medicine ITE(GM-ITE) for their residents. Each program director thencollected the completed examination answer sheets and sent
these back to us in an envelope we had provided. As theacademic calendar in Japan starts on April 1 and ends on
March 31 of the following year, the GM-ITE was conductedin March of the 2012 academic year. Immediately after thetime of the GM-ITE, we provided a self-reporting questionnaire sheet about residents training loads (ED duties permonth and average number of inpatients in charge at a time)and use of evidence-based electronic resources (DynaMed;
EBSCO, MA, USA, and UpToDate; UpToDate Inc., MA,
USA), obtaining informed consents from each participant.
We received data about UpToDate usage from physicians in
the hospitals who participated in our study. The UpToDate
usage data were directly provided by UpToDate Inc andinclude the number of topic viewing. This study was approvedby the institutional review board of the Mito Kyodo General
Hospital, Mito City, Ibaraki, Japan.

Exposure variables
The primary exposure variables of our study were ED
duties per month, average number of inpatients in charge
at a time, and use of evidence-based electronic resources.
Japanese PGY-1 and -2 residents are usually required towork in inpatient ward and in ED. Thus, we evaluated theassociation between the test score and workload in these
two settings. The average count of ED duties per month
was deemed as overall monthly ED workload. The average
number of inpatients in charge at a time was used as overall
daily inpatient caseload. The use of evidence-based electronic
resources was indicated by regular online use of one of thefollowing two most popular resources (that indeed accountfor vast majority of the electronic resources the participants
used in their hospitals): DynaMed (EBSCO) or UpToDate(UpToDate Inc.) since these two resources were consideredas the most popular online systems available in Japan. Other
variables we collected and adjusted in the regression models
include PGY.

Outcome variables
We developed and employed the GM-ITE using the similar
methodology of developing the IM-ITE in the USA.8 The
purpose of the IM-ITE is to provide residents and program

International Journal of General Medicine 2015:8

International Journal of General Medicine downloaded from https://www.dovepress.com/ by 54.191.40.80 on 01-Apr-2017
For personal use only.

Dovepress

directors with an objective, reliable, and valid assessmentof each resident’s clinical knowledge in multiple-choice
examination, and the mean scores of each program comparedwith that of its peers.9,10 The aim of the GM-ITE is to assess
clinical knowledge of PGY-1 and -2 residents. The test scoresand the grade relative to the overall distributions of the testscores were provided for program directors of participatingteaching hospitals and for each resident.
The GM-ITE examination included 100 questions testing a wide range of clinical knowledge, from clinical skills
and practical medical knowledge to psychosocial care ofpatients, and was designed and written by a committee ofexperienced attending physicians organized by JAMEP. The
examination gave a maximum score of 100 and a minimumscore of 0; higher scores indicate a better performance ofknowledge base of general medicine. Before conducting
it, the examination question review was organized, and thecontent validity was confirmed by peer review of each member of the committee. A 3-hour fixed time limit was set for
taking the examination. The GM-ITE was first implemented
in 2011. Our study used data of the test that was performed
in 2012, and an additional questionnaire was collected forthe purpose of our study.

Statistical analyses
We investigated the association between the residents
training loads, personal use of evidence-based electronic
resources, and GM-ITE score by constructing a multivariable generalized estimating equations in order to accountfor the clustering of residents within each hospital. An
exchangeable working correlation with identity link function
and normal distribution was used for the analyses. Residents
training loads in our study included ED duties per month andaverage number of inpatients in charge at a time. Evidence
resource use was defined as personal use of online evidence
summary (DynaMed and UpToDate). Continuous variables,such as the number of ED duties per month and average
number of inpatients in charge at a time, were categorized
into quintiles in order to allow potential nonlinear relationship with the outcome variable. The number of ED duties
and inpatients in charge at a time were compared between
residents who used evidence-based electronic resources andthose who did not.
We also conducted a hospital-level analysis examining
the association between the uses of evidence-based electronic
resources and GM-ITE score aggregated at hospital level. We
analyzed the association between mean hospital score of the
GM-ITE and mean number of topic viewing per physician

International Journal of General Medicine 2015:8

Workload, online resource, and knowledge

for UpToDate in each hospital for the preceding academic
year with multivariable linear regression model. All statisticalanalysis was performed using IBM SPSS Statistics 21.0 (IBM
Corporation, Armonk, NY, USA), and two-tailed P-value of,0.05 was considered as statistically significant.

Results
A total of 431 PGY-1 and 618 PGY-2 resident physicians
in-training (290 women, 28%) underwent the GM-ITE andcompleted the survey (Table 1). As for the PGY-2 physicians,
159 (26%) were women and 459 (74%) were men. As for
PGY-1 physicians, 131 (30%) were women and 300 (70%)
were men. Among all participated residents, the mean scoreof the examination was 58.1 (standard ­deviation, 7.5). There
were 114 teaching hospitals. Nine hospitals (7.9%) were
university hospitals, while 105 hospitals (95.6%) were community hospitals. Evidence-based electronic resources wereused by 790 residents (75.3%). There were 581 residents(55.4%) who had four to five times or more ED duties in an
average month. There were 140 residents (13.3%) who cared
12 or more inpatients at any time in average ward rotation.
After adjusted for PGY, multivariable generalized estimating equation analysis of total scores for all residents isshown in Table 2. Significant variables associated with thegreater score included ED duties four to five times per month
Table 1 Characteristics of residents
Characteristics (N=1,049)
Sex
Men
Women
PGY
1
2
Hospital type
University Community
ED duty per month (n)
0–1
2–3
4–5
6 or more
Number of inpatients in charge (at any time average)
0–5
6–8
9–11
12 or more
Use of evidence-based electronic resources
 No
Yes

n

%

759
290

72.4
27.6

431
618

41.1
58.9

186
857

17.7
81.7

72
166
581
206

6.9
15.8
55.4
19.6

246
395
224
140

23.5
37.7
21.4
13.3

254
790

24.2
75.3

Note: Use of evidence-based electronic resources is defined as regular use of
online evidence summary.
Abbreviations: PGY, postgraduate year; ED, emergency department.

submit your manuscript | www.dovepress.com

Dovepress

Powered by TCPDF (www.tcpdf.org)

357

Dovepress

International Journal of General Medicine downloaded from https://www.dovepress.com/ by 54.191.40.80 on 01-Apr-2017
For personal use only.

Kinoshita et alwith mean score difference of 4.5 (95% CI, 2.7–6.3), numberof inpatients in charge at 12 or greater times with mean score
difference of 2.5 (95% confidence interval [CI], 0.96–4.0).
Thus, residents with ED duties four to five times per month
showed the greatest mean scores better than those with 0–3
or those with 6 or greater times of ED duties. Residents whoroutinely used evidence-based electronic resources showed
2 higher score (95% CI, 0.9–3.0) compared to those who
did not use.
The number of ED duties per month was not different
between residents who used evidence-based electronic
resources and those who did not use (4.5 vs 4.3, P=0.3).
The mean number of inpatients in charge at a time wassignificantly greater among residents who used evidencebased electronic resources compared to those who did notuse (8.8 vs 8.1, P=0.04). A multivariable linear regression
model evaluating the association between mean scores and
UpToDate review counts analyzed at hospital level revealed
that UpToDate greater usage per physician was significantly
associated with the greater score with mean score differenceof 0.061 (95% CI, 0.007–0.114).

Discussion
We found that residents with ED duties four or five times
per month showed significantly greater scores for the GMITE compared to those with 0 or once per month of the ED
duties. Our results also showed that residents in charge of
12 or greater number (the group with the greatest numberof caseloads) of inpatients at a time showed significantly
Table 2 Generalized estimating equation analysis on total scoresfor all residents
Characteristic
PGY-2 vs PGY-1
Use of evidence-based electronic
resources vs no use
ED duty per month (n)
0–1
2–3
4–5
6 or greater
Number of inpatients in charge
0–5
6–8
9–11
12 or greater

Mean score difference

P-valueBeta

95% CI

+1.62
+1.99

0.72
0.93

2.53
3.04

,0.001,0.001

Reference
+3.97
+4.51
+2.90

1.96
2.73
0.95

5.98
6.28
4.86

,0.001,0.001
0.004

Reference
+0.74
+0.92
+2.50

-0.41
-0.39
0.96

1.88
2.23
4.04

0.21
0.17
0.001

Note: Use of evidence-based electronic resources is defined as regular use of
online evidence summary.
Abbreviations: CI, confidence interval; PGY, postgraduate year; ED, emergency
department.

358

Powered by TCPDF (www.tcpdf.org)

submit your manuscript | www.dovepress.com

Dovepress

greater GM-ITE scores compared to those with 0–5 inpatientsin charge at a time.
A greater usage of evidence-based electronic resources
at hospital level was significantly associated with the mean
higher scores at a given hospital. Our results imply that
greater workloads are needed for better performance ofresidents and that workload balance should be adjusted forachieving better performance as well as preventing residents’burnout and adverse effects to patient safety.
Recently there has been a trend for limiting workloads forresidents since there might have been association between
greater workload and higher risk at burnout of residents.
Previous studies on residents’ workloads showed that workload was related to burnout and medical error.11,12 A study
indicated that work hour limitation improved outcomes ofpatients in the teaching service.13 However, some studies,
especially on surgical residents, suggested the opposite consequence, indicating that work hour limitation might lead tonegative effects,14 null effects15 to patients’ outcome, and negative effects to the achievement of educational outcomes.16
Furthermore, programs with work hour limitation might not
be attractive for residents with internal medicine.17
Many graduating students in Japan choose busy residency
programs since they consider these are better educational
environments for them. Our results also showed that areversed U-shaped relationship between ED duties and performance of the GM-ITE scores. Thus, excessive workload
in ED duties may have negative effects to residents’ performance. The appropriate workload balance should be aimed
regarding designs for residency program for ED duties.
Residents who used evidence-based electronic resources
exhibited greater scores of the GM-ITE in our study. Previous
studies indicated higher scores of residents’ clinical performance for the amount of self-learning time.4–7 Ecologicalstudies on teaching hospitals also suggested that hospitals
with evidence-based electronic resources had shorter meandays of their inpatients’ stay, lower mortality, and better
quality of care.18 By the use of UpToDate immediately after
inpatient rounds, 75% of clinical questions elicited by therounds received satisfactory solutions and 37% of online
reviewing led to change in clinical management.19
Although little time would be considered to be available
for self-studying during busy residency programs, our results
indicated that residents who used evidence-based electronic
resources had improved clinical knowledge and were responsible for the significantly greater number of ­inpatients. ­Availabilityof evidence-based electronic resources at the point of care may
be useful for supporting busy residents as a modern version ofInternational Journal of General Medicine 2015:8

International Journal of General Medicine downloaded from https://www.dovepress.com/ by 54.191.40.80 on 01-Apr-2017
For personal use only.

Dovepress

medical textbook and it could have better implication for thequality of patient care. Since our study provided association
between the greater test score and appropriate workloads anduse of evidence-based electronic resources, program directors
and residents may be advised to consider setting appropriate
workloads and using these resources.
Our study has several limitations. First, given that, only
1,049 of the 15,800 residents participated throughout Japan inthis study, and they were not a random sample of the underlying
population of all residents in Japan, our findings may not be
generalizable to all residents in Japan. Program directors were
responsible for the decision to participate in this examinationand thus there might have been a bias for higher proportions forthe participation among teaching hospitals with program directors with higher motivation and subsequently better programs.
However, no residents who received the GM-ITE denied theparticipation for this survey and there was little space forbias toward the participation of better-performing residents.
Second, with regard to the use of evidence-based electronic
resources, better performers who used them efficiently might
have had greater chance to provide care to the higher numberof inpatients. Third, the emergency room duty per month andthe number of inpatients in charge could be influenced by recall
bias or cognitive bias since it is a questionnaire study. We use
the term “on average” in the questionnaire about these items
over the 1-year period before the test date. Fourth, we did not
adjust potential confounders such as age and sex of individual
residents in our model. However, these variables are usually notconsidered as significant variables associated with educational
environments of teaching hospitals.

Conclusion
We found that appropriate level of workload, including ED
duties and the inpatient number in charge, might have beneficial impact on the achievement of better performance ofresidents. In addition, the evidence-based electronic resources
may be useful supporting tools for busy residents to get solutions for clinical questions raised by rounds. Our study may
provide a benchmark for setting well-balanced workload forresidents. Further study is needed for the interrelationships
between workload, quality of life, and quality of patient careamong residents.

Acknowledgments
The authors thank all executive people for their excellent
assistance. The JAMEP funded the study grant and conducted
the examination for resident physicians as explained in the text
and data collection but had no role in the study design, data

International Journal of General Medicine 2015:8

Workload, online resource, and knowledge

interpretation, or preparation of the article. Dr Tsugawa was
supported by Honjo International Scholarship Foundation.

Author contributions
K Kinoshita is the guarantor for the integrity of the data andthe accuracy of the data analysis. K Kinoshita and Y Tokuda
conceived the research idea. K Kinoshita and Y Tsugawa
designed the study, conducted statistical analysis, and drafted
the manuscript. All authors contributed to the interpretationof data, critically revised the paper for important intellectual
content, and agreed to be accountable for all aspects of thework.

Disclosure
Y Tsugawa and Y Tokuda received honorarium from the
JAMEP for preparing study questions. The other authors
report no other conflicts of interest in this work.

References

1. Shimizu T, Tsugawa Y, Tanoue Y, et al. The hospital educational
environment and performance of residents in the general medicine
in-training examination: a multicenter study in Japan. Int J Gen Med.
2013;6:637–640.
2. McCoy CP, Stenerson MB, Halvorsen AJ, Homme JH, McDonald FS.
Association of volume of patient encounters with residents’ in-training
examination performance. J Gen Intern Med. 2013;28(8):1035–1041.
3. Chase LH, Highbaugh-Battle AP, Buchter S. Residency factors that
influence pediatric in-training examination score improvement. Hosp
Pediatr. 2012;2(4):210–214.
4. Philip J, Whitten CW, Johnston WE. Independent study and performance on the anesthesiology in-training examination. J Clin Anesth.
2006;18(6):471–473.
5. Godellas CV, Hauge LS, Huang R. Factors affecting improvement on
the American Board of Surgery In-Training Exam (ABSITE). J Surg
Res. 2000;91(1):1–4.
6. Godellas CV, Huang R. Factors affecting performance on the American
Board of Surgery in-training examination. Am J Surg. 2001;181(4):294–296.
7. McDonald FS, Zeger SL, Kolars JC. Factors associated with medical
knowledge acquisition during internal medicine residency. J Gen Intern
Med. 2007;22(7):962–968.
8. Garibaldi RA, Subhiyah R, Moore ME, Waxman H. The in-training
examination in internal medicine: an analysis of resident performanceover time. Ann Intern Med. 2002;137(6):505–510.
9. Kanna B, Gu Y, Akhuetie J, Dimitrov V. Predicting performanceusing background characteristics of international medical graduates in
an inner-city university-affiliated internal medicine residency training
program. BMC Med Educ. 2009;9:42.
10. Perez JA, Greer S. Correlation of United States medical licensing examination and internal medicine in-training examination performance.
Adv Health Sci Educ Theory Pract. 2009;14(5):753–758.
11. Block L, Wu AW, Feldman L, Yeh H-C, Desai SV. Residency schedule,
burnout and patient care among first-year residents. Postgrad Med J.
2013;89(1055):495–500.
12. Blanchard P, Truchot D, Albiges-Sauvin L, et al. Prevalence and causesof burnout amongst oncology residents: a comprehensive nationwide
cross-sectional study. Eur J Cancer. 2010;46(15):2708–2715.
13. Horwitz LI, Kosiborod M, Lin Z, Krumholz HM. Changes in outcomesfor internal medicine inpatients after work-hour regulations. Ann Intern
Med. 2007;147(2):97–103.

submit your manuscript | www.dovepress.com

Dovepress

Powered by TCPDF (www.tcpdf.org)

359

Dovepress

Kinoshita et al

17. Gopal RK, Carreira F, Baker WA, et al. Internal medicine residents
reject “longer and gentler” training. J Gen Intern Med. 2007;22(1):102–106.
18. Isaac T, Zheng J, Jha A. Use of UpToDate and outcomes in US hospitals.
J Hosp Med. 2012;7(2):85–90.
19. Phua J, See KC, Khalizah HJ, Low SP, Lim TK. Utility of the electronic
information resource UpToDate for clinical decision-making at bedsiderounds. Singapore Med J. 2012;53(2):116–120.

International Journal of General Medicine downloaded from https://www.dovepress.com/ by 54.191.40.80 on 01-Apr-2017
For personal use only.

14. Lee DY, Myers EA, Rehmani SS, et al. Surgical residents’ perception of
the 16-hour work day restriction: concern for negative impact on resident
education and patient care. J Am Coll Surg. 2012;215(6):868–877.
15. Curet MJ. Resident work hour restrictions: where are we now? J Am
Coll Surg. 2008;207(5):767–776.
16. Businger A, Guller U, Oertli D. Effect of the 50-hour workweek
limitation on training of surgical residents in Switzerland. Arch Surg.
2010;145(6):558–563.

Dovepress

International Journal of General Medicine

Publish your work in this journal
The International Journal of General Medicine is an international,
peer-reviewed open-access journal that focuses on general and internal
medicine, pathogenesis, epidemiology, diagnosis, monitoring and treatment protocols. The journal is characterized by the rapid reporting ofreviews, original research and clinical studies across all disease areas.

A key focus is the elucidation of disease processes and management
protocols resulting in improved outcomes for the patient. The manuscript management system is completely online and includes a very
quick and fair peer-review system. Visit http://www.dovepress.com/
testimonials.php to read real quotes from published authors.

Submit your manuscript here: http://www.dovepress.com/international-journal-of-general-medicine-journal

360

Powered by TCPDF (www.tcpdf.org)

submit your manuscript | www.dovepress.com

Dovepress

International Journal of General Medicine 2015:8