medical residents receive most of their training within hospital programs . in brazil , after completing a medical undergraduate education , students enroll in residency programs that involve three ( basic ) to seven years of training .
resident physicians undertake supervised training in areas of the hospital that include the ambulatory care facilities , surgical centers , laboratories , radiology departments , and in - patient wards .
previous efforts to study effective learning environments resulted in the development of a questionnaire for undergraduates .
the 50-item dundee ready education environment measure ( dreem ) used a standard methodology grounded in education theory together with a delphi panel of nearly 100 professional health educators from around the world .
it was translated into a variety of languages and has been used worldwide.1,2 the dreem was translated and adapted for portuguese and has been used with brazilian undergraduate medical students as well as residents .
this is typically done in brazil to introduce students to the concept of patient interaction at an early stage in their training.3 the version of dreem used in this study also featured a psychometric performance evaluation component .
this questionnaire was given to medical residents from different programs at six institutions in three brazilian cities.4 a similar methodology to dreem was used to develop the pheem 
postgraduate hospital educational environment measure questionnare.5 this elegant 40-question survey assesses metrics of the level of autonomy , quality of teaching and social support during the hospital - based training period undertaken by all new physicians .
pheem can identify specific strengths and weaknesses within a certain educational environment . because medical residency programs in brazil have rarely been comprehensively evaluated to date , we aimed to assess pheem as a possible tool for performing future evaluations.5 the objective of this investigation was to validate the pheem questionnaire translated into portuguese and to study its reliability .
we also aimed to compare pheem results among residents from the internal medicine , surgery and anesthesiology departments at a university teaching hospital in brazil .
the method used to translate the pheem questionnaire was based on the modified brislin back - translation technique.6 briefly , two brazilian teachers of english independently translated the original pheem into portuguese .
a second translation was independently undertaken to generate a second portuguese draft copy from the en1 version , and the resulting document was again back - translated by a native english speaker ( en2 ) .
a final portuguese version was generated by comparing the two back - translated english versions ( en1 and en2 ) with the original and the two portuguese drafts .
our study was approved by the appropriate institutional review board and all participants were required to sign an informed consent document .
each questionnaire featured an anonymity barcode which was not linked to the identity of the participant .
the 40-question portuguese pheem survey was given to a randomized sample of medical residents from the hospital das clnicas , university of so paulo medical school .
our test group included physicians finishing their first year ( n=174 ) or at the end of their second year of residency ( n=89 ) .
in addition , we gave the survey to a number of medical residents who had entered a local competition in two main areas : internal medicine ( n=459 ) and surgery ( n=298 ) .
respondents were asked to indicate their agreement with each statement using a five - level likert - type scale , which went from  strongly disagree  0 to  strongly agree  4 .
higher levels of agreement were correlated with more beneficial educational environments . in order to determine the needed sample size ,
a pilot study was conducted with the dreem questionnaire to identify the impacts of differences in means .
it suggested that a sample of 27 questionnaires across three groups would yield a statistical power of 80% and a significance level of 5%.4,7 cronbach s alpha coefficients were used to assess reliability and internal consistency .
the hotelling s t - squared test was used as a multivariate analysis tool to evaluate the null hypothesis that all of the items on the scale would have the same mean .
anova was used to compare pheem - derived data from residents in the surgery , anesthesiology and internal medicine departments .
a subsample of medical residents ( n=50 ) from the clinics hospital at the university of so paulo medical school answered the pheem twice in a period of 30 days .
the kappa coefficient was used as a measure of agreement between these two samples . in general ,
excellent agreement was indicated by  > 0.74 ; good agreement ,  = 0.60 to 0.73 ; fair agreement ,  = 0.40 to 0.59 and poor agreement ,  < 0.40.8 the spearman correlation test was used to assess equivalence between the two samples .
these medical residents completed an additional survey to assess their experience in completing the pheem questionnaire .
this additional survey is known as the whoqolbref.9 we evaluated correlations across data from the two questionnaires .
principal component factor analysis was applied across the three sections in the original pheem questionnaire .
this analysis aimed to explain a considerable amount of the variance that was present in the 40-item survey .
our study was approved by the appropriate institutional review board and all participants were required to sign an informed consent document .
each questionnaire featured an anonymity barcode which was not linked to the identity of the participant .
the 40-question portuguese pheem survey was given to a randomized sample of medical residents from the hospital das clnicas , university of so paulo medical school .
our test group included physicians finishing their first year ( n=174 ) or at the end of their second year of residency ( n=89 ) .
in addition , we gave the survey to a number of medical residents who had entered a local competition in two main areas : internal medicine ( n=459 ) and surgery ( n=298 ) .
respondents were asked to indicate their agreement with each statement using a five - level likert - type scale , which went from  strongly disagree  0 to 
higher levels of agreement were correlated with more beneficial educational environments . in order to determine the needed sample size ,
a pilot study was conducted with the dreem questionnaire to identify the impacts of differences in means .
it suggested that a sample of 27 questionnaires across three groups would yield a statistical power of 80% and a significance level of 5%.4,7 cronbach s alpha coefficients were used to assess reliability and internal consistency .
the hotelling s t - squared test was used as a multivariate analysis tool to evaluate the null hypothesis that all of the items on the scale would have the same mean .
anova was used to compare pheem - derived data from residents in the surgery , anesthesiology and internal medicine departments .
a subsample of medical residents ( n=50 ) from the clinics hospital at the university of so paulo medical school answered the pheem twice in a period of 30 days .
the kappa coefficient was used as a measure of agreement between these two samples . in general ,
excellent agreement was indicated by  > 0.74 ; good agreement ,  = 0.60 to 0.73 ; fair agreement ,  = 0.40 to 0.59 and poor agreement ,  < 0.40.8 the spearman correlation test was used to assess equivalence between the two samples .
these medical residents completed an additional survey to assess their experience in completing the pheem questionnaire .
this additional survey is known as the whoqolbref.9 we evaluated correlations across data from the two questionnaires .
principal component factor analysis was applied across the three sections in the original pheem questionnaire . this analysis aimed to explain a considerable amount of the variance that was present in the 40-item survey .
principal component factor analysis was applied across the three sections in the original pheem questionnaire .
this analysis aimed to explain a considerable amount of the variance that was present in the 40-item survey .
the pheem portuguese questionnaire was answered by 30.7% of the internal medicine residents ( n=141 ) and by 20.5% of the surgery residents ( n=61 ) who had entered a local competition .
in addition , the survey was competed by 19.5% ( n=34 ) of the first year residents at clinics hospital and by 17.9% ( n=16 ) of the second year residents .
residents from governor celso ramos hospital achieved a higher response rate of 69.2% ( n=54 ) . the pheem survey in portuguese ( n=306 ) showed a cronbach s alpha of 0.899 .
the alpha value did not vary by more than 5% across groups and genders . as expected , the null hypothesis was deemed invalid according to hotelling s t - squared test .
contract of employment ;  item seven ,  racism ;  item thirteen ,  sexual discrimination ;  and item 25 ,  no - blame culture ,  as essentially uncorrelated with the total score .
the weighted kappa coefficient was used as a measure of the agreement in responses over an interval of 30 days .
the majority of questions were in excellent agreement , while items one ,  contract of employment ;  two ,  clear expectations ;  eight ,  inappropriate tasks ;  ten ,  good communication skills ;  and 26 ,  catering facilities ,  had coefficients that suggested good agreement .
the spearman correlation index r was greater than 0.8 in all cases except for question sixteen , 
the lowest recorded score was 1.2 ( for item 26 :  there are adequate catering facilities when i am on call  ) while the highest was 3.6 ( the opposite of item 7 :  i can sense the existence of racism in my position  ) .
question numbers 1 , 14 , 17 and 32 within the  autonomy  section were found to have relatively low ratings .
social support showed low ratings for question numbers 20 , 25 , 26 , and 38 ( table 1 ) .
the questionnaire presented four items that featured negative statements ( items 7 , 8 , 11 and 13 ) ; the scores for these items were inverted in order to calculate total score and percentile results from the questionnaire .
the autonomy section total score was 33.9  8.6 ( 60.5%  15.3% ) , the teaching quality score was 35.0  10.0 ( 58.3%  16.6% ) and
results from the three sections of the pheem survey were compared among residents from internal medicine , surgery and anesthesiology programs by anova followed by the holm - sidak test .
the level of autonomy was generally perceived to be higher by residents in internal medicine than by those in surgery ( p=0.000001 ) . however , there was no difference in perception between internal medicine and anesthesiology residents ( p=0.0573 )
. perceived levels of teaching quality were higher for residents in anesthesiology as compared with internal medicine ( p=0.017 ) or surgery ( p=0.00048 ) .
internal medicine residents also rated quality of teaching significantly higher than surgery residents ( p=0.0308 ) .
there were no differences in the perception of social support among any of the groups ( table 2 ) .
exploratory factor analysis followed by varimax rotation identified nine factors with eigenvalues higher than 1 ( table 3 ) .
the first factor had an eigenvalue of 12.0 , which accounted for 30.0% of the variance .
the next eight factors had eigenvalues lower than 2.3 ; these factors accounted for 58.9% of the variance in the data .
the first factor included questions 10 , 28 , 23 , 6 , 2 , and 22 from the teaching quality section of the pheem questionnaire , and items 26 and 35 from the social support section .
the whoqol s domains were considered adequate , reaching 60% for the environmental and greater than 70% for the psychological , social and physical domains .
this study evaluated the applicability of the pheem survey translated into portuguese with a sample of resident physicians in brazil .
our pheem results suggest that the teaching quality section was the most important part of the questionnaire .
five items ( 1 , 7 , 13 and 25 ) did not correlate well with the total score and , from a statistical point of view , they were found to not influence the overall results . despite the observed lack of influence , we suggest that these statements be retained to allow for comparisons between programs or institutions .
indeed , items 7 and 13 deal with racial and gender discrimination , respectively , and this sample of residents indicated that these issues were not problematic during their training programs . on the other hand , items 1 (  information about hours of work  ) and 25 (  no - blame culture  ) reached a mean of just below 2.0 , which suggests that these issues constituted an educational obstacle .
the kappa coefficient suggests excellent inter - sample agreement . the good agreement observed for five items
may be attributed to the fact that they queried different areas of the residency workload .
more specifically , during the 30-day test interval , many residents switched between programs and this may have impacted their answers to these questions.10 pheem can be used to identify strengths and weaknesses of a medical residency program . when the published guide suggested for this questionnaire
is applied to interpret mean scores , all residents had  a more positive perception 
( 33.9/56 ) with regard to the level of autonomy ; teaching quality seemed to be  moving in the right direction  ( 35.0/60 ) and there were  more pros than cons  ( 26.6/44 ) in terms of the provision of social support.5 such results should be taken into account by curriculum planners as they consider improvements to educational programs .
nevertheless , we note very positive responses in terms of the statements that confirmed minimal discrimination ( items 7 and 13 ) .
relevant items showed that residents were given appropriate levels of responsibility ( item 5 ) and opportunities ( item 30 ) .
items with a mean of between 2 and 3 identify elements of the educational program that could be enhanced.5 it is interesting , albeit disappointing , to note that 28 out of 40 questions for this sample fit in this classification . specifically , the net perception of teaching quality was within this range or , in some cases , below 2 ( question 3   educational time is safeguarded  and
significant attention should be directed to those questions with means below 2 as they may indicate serious problem areas .
in addition to the two items that pertain to perception of teaching quality , those in the most urgent need of improvement were related to social support .
certain specific responses pointed to a lack of personal and sometimes professional support ( question 25   no - blame culture  , 26   catering facilities  and 38   available counseling for junior doctors who fail  ) .
similar shortcomings were seen with regard to the level of autonomy ( question 1  
i receive adequate information about my work hours  ; 14   i am always clear on which clinical protocols are acceptable  and 17   my working hours are in accordance with the limits specified by the national board  ) .
residency training is often correlated with stress , depression and burnout due mainly to excessive work hours , sleep deprivation , challenging patients and an aggressive and challenging work environment.1115 the pheem did not ask questions that directly addressed these aspects , but respondents suggested that their social life during residency could be uninspiring , and that their level of social interaction was unsatisfactory . other researchers have evaluated how medical residents experience personal growth and have highlighted the need for an environment that fosters supportive relationships and encourages reflection.16 on the basis of our study , it would be interesting to further investigate the positive associations of teaching achievement and social interactions .
the pheem teaching quality section identified two problem areas : safeguarded educational time and feedback from instructors .
certainly , these two elements are important to obtaining a meaningful learning experience during medical residency .
indeed , in order to acquire learning skills , an apprentice needs safeguarded time away from the institutional schedule . in the same way ,
feedback from instructors is critical to the learning process.17 , 18 the item related to work hours being consistent with limits specified by the national board requires discussion , at least in the case of brazil s system of medical residency .
it has been observed that violations of residency program requirements correlate with very low perceptions of quality of life and a poor educational environment.19 .
however , this study did not identify a low quality of life , considering the results of whoqolbref , but did identify a weak educational environment .
it has been suggested that reliance on such data is not appropriate when studying associations of working hours with quality of life and patient care.20 educational as well as management theories support the understanding that the development and maintenance of a learning - oriented culture should be a high priority for residency programs and their institutions.21 it has been suggested that a lack of supervision can contribute to medical errors , but despite this fact , best practices in medical communication and supervision skills have received little evaluation.22 it would also be interesting to consider how network analysis may facilitate the mapping of ties between residents peers and supervisors and the nature and rules of their relationship in order to best understand how to create and maintain an appropriate academic environment.23 finally , the factor analysis of this sample supports boor et al . who suggest use of pheem as a one - dimensional scale instead of the three original sub - scales.5,24 the main factor that explains 30% of the variance includes seven items that are related to the perception of teaching quality and one that relates to social support and instructor mentoring skills .
even more interesting , these eight items are strongly linked to the role of the instructor .
our factor analysis is consistent with other published data in which four of seven items composed a one dimensional scale that also treated the importance of the instructor during residency.25 considering various other studies that have also used this questionnaire , we would reemphasize that pheem should be used in its original format with all 40 basic questions in order to allow for comparisons between programs and to permit evaluations of the three sections during different phases of a physician s medical residency . the results from this study support the use of pheem as a reliable instrument to identify issues related to the clinical educational environment .
our data suggest that the results from the section on teaching quality are very important .
 objectivethis study analyzes the reliability of the pheem questionnaire translated into portuguese . 
 we present the results of pheem following distribution to doctors in three different medical residency programs at a university hospital in brazil.introductionefforts to understand environmental factors that foster effective learning resulted in the development of a questionnaire to measure medical residents perceptions of the level of autonomy , teaching quality and social support in their programs.methodsthe questionnaire was translated using the modified brislin back - translation technique . 
 cronbach s alpha test was used to ensure good reliability and anova was used to compare pheem results among residents from the surgery , anesthesiology and internal medicine departments . 
 the kappa coefficient was used as a measure of agreement , and factor analysis was employed to evaluate the construct strength of the three domains suggested by the original pheem questionnaire.resultsthe pheem survey was completed by 306 medical residents and the resulting cronbach s alpha was 0.899 . 
 the weighted kappa was showed excellent reliability . 
 autonomy was rated most highly by internal medicine residents ( 63.7%  13.6% ) . 
 teaching was rated highest in anesthesiology ( 66.7%  15.4% ) . 
 residents across the three areas had similar perceptions of social support ( 59.0%  13.3% for surgery ; 60.5%  13.6% for internal medicine ; 61.4%  14.4% for anesthesiology ) . 
 factor analysis suggested that nine factors explained 58.9% of the variance.conclusionsthis study indicates that pheem is a reliable instrument for measuring the quality of medical residency programs at a brazilian teaching hospital . 
 the results suggest that quality of teaching was the best indicator of overall response to the questionnaire . 