Portfolios and assessment: meeting the challenge

Two articles in the July 2001 edition of Medical Teacher (Davis et al., 2001; Pitts et al., 2001) present differing aspects of portfolio development and assessment. The fact that these articles appeared simultaneously may be indicative of the growth of interest in the use of portfolios across medical education, and the serious attempts being made to evaluate their usefulness and feasibility as an assessment tool. The scale of the use of portfolios in medical education has recently been indicated by a report from the Preparedness to Practice Project at Imperial College School of Medicine (McKimm, 2001). Between October 2000 and January 2001, the researchers first carried out a web-based search using the key word ‘portfolio’ refined by education, healthcare, training and medicine. Examples thrown up by this search revealed that recorded use of portfolios in medical education was limited and mainly restricted to postgraduate training. A subsequent survey of medical schools in the UK showed that of 24 responding institutions, 10 used portfolios in at least part of their undergraduate programme. These were usually those medical schools whose curriculum has undergone substantial change since the publication of Tomorrow’s Doctors (GMC, 1993). Nine schools use some form of logbook or record of achievement for either formative or summative assessment. At the same time as these developments have been taking place in training the doctors of the future, the General Medical Council (GMC) has also been considering how it might ensure that practising doctors in the UK are keeping up to date in their education. The GMC’s proposals for revalidation of doctors rest on the five-yearly production of a ‘folder’ of information by each clinician seeking revalidation. The suggested ‘folder’ may be seen as a ‘portfolio’ of evidence of activity under the GMC’s stipulated categories. Although still open to change, the original consultation document (GMC, 2000) indicates seven discrete areas of professional practice: good clinical care; maintaining good medical practice (keeping up to date); relationships with patients; working with colleagues; teaching and training; probity; health. To date, the precise criteria against which information included in the folder will be judged have not been agreed. Initial feedback from the pilot phase of the GMC’s revalidation project indicates that the use of the folder is generally feasible, although ‘some of the data required for the folder are more difficult for doctors to collect than others’ (GMC, 2001). However, the work done by Pitts et al. and Davies et al. shows that more work still needs to be done before the medical education community will accept portfolios as a valid and reliable form of assessment—especially if this is to be for summative purposes. One issue militating against more widespread use of portfolios in the short term is simply that of novelty. Although any well-grounded innovation can quickly meet with approval from the ‘early adopters’, it will take time before the ‘laggards’ come on board. The difficulty is that in times of doubt, the tendency is to fall back upon the familiar—however unacceptable it has been considered over the years—and imbue this with a sanctity that it does not necessarily deserve. There was, for instance, early resistance to the OSCE on the basis that it was considered expensive and organizationally difficult to set up. However, as this method of assessment has been seen to have relatively high validity and reliability, and can be used to assess dimensions of learning that more traditional paper-based regimes could not easily assess, it is now widely accepted as one of a range of core assessment methods. As Davis et al. indicate in their article, resistance can come from both assessors and assessed, with students perhaps being the more cautious in accepting new modes of assessment as they may well perceive that they have a lot to lose if the system proves unsatisfactory, or if they do not consider themselves to have been adequately prepared. However, Van der Vleuten (1999) shows how any assessment system can be adapted for use in a very wide range of circumstances. The key to enabling this is to ensure that the right sort of thinking happens at the design stage of the assessment process to ensure that the input is appropriate for the output. For example, it would be easy to gain agreement that an essay would be a poor means of assessing practical skills; but an OSCE may still fail to produce reliable assessment of such skills if it is badly designed and managed. The same must be true for portfolios. They may be the wrong tool for the job, or may be badly conceived and make inappropriate demands on learners. Teachers are generally in a position to make a clear link between the curriculum, the teaching and learning processes, and the assessment regime, and they tend to assume that the interrelationship between the three processes is