Gareth Pierce: It's interesting—they link three quite different things there, don't they? We've discussed the textbooks in some detail. When they refer to sample assessments, we have to provide those within the regulatory time frame, and they have to be approved. So, in every subject there are sample assessments and marking schemes, and they are statutorily available, in a way. When they talk about exemplars, there are different interpretations of that. One request that we receive often from schools is for exemplars in terms of the work of pupils that has been assessed and marked by us. Of course, they're not available until the young people have taken those exams for the very first time. We can make them available digitally, and through various other methods. But in some subjects the demand has been so strong that we have worked with schools in order to ensure that work is produced by pupils before that first examination, is marked by us, and is then made available. So, we can respond to that demand when it arises. I would like to have a conversation with Estyn, indeed, to interpret on a more detailed level some of the things that they refer to.
Emyr George: May I offer a comment on that quote from Estyn? They talk about a reduction in attainment and that, to some extent, brings us back to the point in terms of the comparable outcomes method that we're talking about. That method is based on research and evidence that says that in a period of change to an exam or qualification, we know that performance in an exam can fall, and that is due to a number of different factors. Teachers aren't as familiar with the specification and the requirements of the exam. There are fewer past assessment resources available; there aren't any past papers, for example. Certainly, knowing that you're the first to sit that exam perhaps raises your concerns about that examination, and examinations are a matter of concern already. It's very difficult, if not impossible, to differentiate between those factors, but what we know on the whole is that they come together to have an impact on pupils' performance in an exam, and that's why, when it comes to awarding qualifications for the first time and looking at the grades, we look very carefully at what that the mark means in terms of the grade in order to try as best as we can, on the level of the whole national cohort, to eliminate as much unfairness as possible because they are the first cohort to sit the exam.
Gareth Pierce: Could I just make a brief comment on the sample assessments? Now, interestingly, when the English, Welsh, maths, and maths and numeracy GCSEs were new, the Welsh Government were very eager that we invest in more sample assessment materials. We had to produce one for the regulator, but they wanted to see us produce far more, and we did that. But there are risks inherent in that, too. If we create more and more sample assessments, then does that create more risk of teaching to the test? And every time we produce one of these, of course, we are using robust assessment ideas, and they're out there to be used for test purposes. So, we need to be careful that we don't weaken the real assessment by providing too many of the examiners' good ideas in the sample material. So, it's an interesting theme, but it would be interesting to have a further conversation with Estyn on that.
Philip Blaker: I just wanted to say, looking towards—. Reforms always throw up issues like this. It's almost an inevitability of any change that there will be impact. I think what we need to do is we need to think about the future reforms, particularly knowing that we've got some on the radar already, and to think about what this paradigm is and how we can isolate some of these factors in the future. So, if we look at resources and if we look particularly at textbooks as being a curriculum resource, rather than a qualification resource, I think we can change the timeline to which some of those materials are produced. So, going back to when I studied my A-levels in the 1980s, there was this common body of knowledge that was an A-level textbook in biology that wasn't focused around an individual awarding body. When we talked to unions about this on Monday, they said, 'Yes, when we used to teach many years ago, there were curriculum resources like that, and then when a particular topic or a particular area of content dropped out of the qualification, we either did or didn't teach it according to whether it was in the curriculum'. That's a little bit worrying by virtue of the fact there might have been a component of that subject that didn't get taught because it was no longer in the text, despite the fact that it was in the common body of knowledge. But I think if we can shift this paradigm—and I would really like to a shift in the paradigm for resources particularly, away from the focus on the qualification and into more of a focus on the curriculum—then that provides an opportunity for, I would suggest, a much healthier model in the future. There are various ways of doing that. In the United States, for example, there are some states within the United States that produce a state textbook. Now, that can have issues, particularly if it's politically orientated—so, if politicians are seen to have a strong influence in the content that's taught. But if there is independence in that state model, then I think that can be quite strong model. Equally, if you look at somewhere like Hong Kong, they've actually tried to de-emphasise textbooks. So, Hong Kong, which is a very high-performing jurisdiction, is trying to move away from that model in the future.
Gareth Pierce: Could I just add very, very briefly—
Lynne Neagle AM: Very briefly, because I want to bring Hefin in.
Gareth Pierce: I know that, in our letter to you, we emphasised that this is not just to do with the body of knowledge. The assessment objectives have shifted in many subjects towards analysis and evaluation. So, therefore, we really do need resources that support that way of working, and I'm sure that lots of the young people we've heard from—what they're really doing in their revision is actually not just studying the content, but thinking about how they respond analytically and evaluatively. In fact, an interesting question for us as an awarding body to debate with the qualifications regulator, maybe, is whether some of the content should actually be able to be taken into the assessment hall. Do young people need to remember all these terms and their precise definitions, sometimes in both languages if they've learnt it in that way? Shouldn't some of the reference material be more and more available in the assessment arena? And then you are really getting towards analysis and interpretation and evaluation.
Lynne Neagle AM: Thank you. Hefin.
Hefin David AM: I want to welcome these particular comments, because having come from a higher education background, what I've tried to deliver in my courses are textbook-free modules, where you're relying on academic journal articles and you're piecing together the story yourself, based on the curriculum you develop yourself. So, curriculum-focused resources are very welcome. I fear that students who are nurtured on a textbook diet, when they go to university, they are not expected to repeat what's in the textbook; they're expected to do exactly as Gareth says and analyse and combine information into a discursive argument. So, therefore, I'm interested to know how you've worked with higher education to develop that approach. I also welcome what you've just said too.
Lynne Neagle AM: Can I ask for brief answers, please?
Gareth Pierce: Yes, well some of the people who provide ideas into our digital resources certainly are either teachers who are very well aware of that progression to HE and therefore what their learners want or, in some cases, they are HE people themselves. We do draw on a body of expertise, and we are more than willing to emphasise the importance of that approach to the support resources we provide.
Hefin David AM: I think the difficulty and the difference is that, at HE level, the kind of journal articles that are available are probably not consumable at GCSE level. That's part of the problem you have with that approach.
Gareth Pierce: Yes. And I think, interestingly—I know we've pointed towards some of our religious studies resources as examples—I think some of the websites we point to are quite ambitious. They would be described as scholarly, and I think some of the young people on the video made that point, didn't they—that they need to engage with that material to understand some themes? But we need to assist them in doing so. We need to almost make those scholarly items that are worth including in their curriculum accessible and user friendly for them at their point of learning.
Lynne Neagle AM: Philip.
Philip Blaker: It was just one point. We recently published a report on the Welsh baccalaureate and skills challenge certificate, and I think it's relevant to pull in at this point that the independent study that pupils would be expected to do through the skills challenge certificate in doing their project work is really designed to try and draw out some of those skills that would be relevant in higher education. Through the reforms that we are starting to think about for the skills challenge certificate, there's a real opportunity to start honing some of those skills, particularly at the advanced level bacc, to make people really HE-ready so that they're better able to engage with that different style of learning. So, I think there's a huge opportunity that shouldn't be diminished within the advanced Welsh bacc.
Lynne Neagle AM: Thank you. Last questions from Mark then.
Mark Reckless AM: Gareth, can I just put to you—? On the Estyn criticism, I think you've answered one aspect of their criticism that WJEC did not make as many sample assessment materials available as schools would have liked early on in the reform process, and I understand your position there, but they went on and said, 'At times the sample assessment materials were provided with incorrect mark schemes' and also, 'It would be helpful if the WJEC ensure that all necessary resources were made available before the start date of each qualification.' Do you have a response to those points?
Gareth Pierce: Again, probably, I'd like a conversation with them about any problems with the specimen assessments and their mark schemes, because if they are the statutory regulatory ones they would have gone through Philip's team as well or his Welsh Government predecessors as regulators, depending on which qualifications they were. Obviously, we need to be spot-on with those. Very occasionally, somebody will identify an error, and obviously we then correct them, because these are available digitally online. So, if there is a problem, we want to hear about it and then we can correct it. But, yes, I agree with their fundamental premise, that ideally the whole package of resources should be available before teaching starts, including, as we mentioned earlier, so we can draw on that set of resources in the events we run as WJEC free of charge for teachers leading up to the first teaching. We need that information ourselves as well so we can talk about the use of these valuable resources in an appropriate pedagogical context in that preparatory year. So, the ideal is that everything is there 12 months ahead.
Mark Reckless AM: So, what mechanism do you have for picking up on this type of criticisms from Estyn and having that conversation with them? Has that not happened?
Gareth Pierce: We have met with Estyn a couple of times recently, actually—once when they were beginning their review of the key stage 4 qualifications that had been in place for two years and a bit, and also when they were planning a review of some of the A-level work. Because we do have regular meetings with them, we will have an opportunity to explore their comments here I'm sure.
Mark Reckless AM: Looking at the structure and the relationship between Qualifications Wales and the WJEC—. I understand in the model in England—you've got Ofqual, the regulator, and you have various competing exam boards, some I think with for-profit motivation. We don't have that in Wales; there is a sole regulator and a single, dominant examination board. Does that mean that there's significant overlap and potentially cost duplication between the work of the two bodies, particularly given WJEC is a charity and seems to have many of the same objectives as you in Qualifications Wales have?
Philip Blaker: The roles are very distinct, so I'd tell you there's no overlap. What we try and do is try and make sure that there's a good connection between the work that we do. WJEC has very clear responsibilities around the delivery of qualifications. We have a role in monitoring to ensure that those qualifications are delivered securely and fairly. I would say a strength of the Welsh model is that there is an independent regulator and a single awarding body largely, because much of the work that Ofqual would need to do in terms of its maintenance of standards is about ensuring that the competition between awarding bodies doesn't lead to what's been termed in the past as a race to the bottom in terms of standards. Now, the opportunity that we have in Wales is, where we have a single awarding body, we can have a close and good working relationship with WJEC, but guarding very strongly to make sure that we keep that regulatory distance to make sure that we preserve our role as the regulator and look at WJEC as one of our regulated bodies.
Gareth Pierce: Can I make a general point? Perhaps the important opportunity in Wales is to look anew at how national and regional organisations, such as the consortia, can collaborate to make a success of the broad range of resources and CPD that will be required to support the new curriculum, and also the next set of qualifications that will be reformed. We've referred to many organisations today, but there are others. The Welsh Books Council, for example, is part of this discussion. The Learned Society of Wales is a player that wants to introduce ideas into the Welsh dimension of the curriculum. There was mention of the HE sector earlier. Teachers' associations have very specific comments to make on some of these areas. There is a great opportunity for us to come together and to have a national approach. There will be a need for some financial resource from the Welsh Government—yes, certainly—and the WJEC can provide some resources, but I think we all want to collaborate in order to ensure the success of the next round of resources.
Mark Reckless AM: But doesn't that national approach and collaboration preclude the model that Philip was talking about earlier, where we had textbooks based on the curriculum, and there's that core, but that was separate from what the exam body decided to do and the textbook wasn't there for the particular exam approach? If you have a single, dominant exam board, and you have a Welsh curriculum that's developed with that board, how do you then have this separation between textbook and exam?
Philip Blaker: So, if we look at qualification, a qualification should be based on the curriculum that's being offered, because learning should be based around the curriculum, and the role of the qualification is to measure attainment against knowledge of that curriculum—knowledge, skills and understanding, the dispositions that that curriculum is trying to develop. So, I think there are two very distinct things. Now, of course, what will inevitably happen is once the curriculum is known and developed, there will be a level of detail that comes across from what the qualification is looking for. I think what we're trying to say is, from an educational perspective, what we'd like to see is a broader focus on the curriculum and less of a narrow focus on the qualification and some of the problems that that can drive, so that, if a learner is well-versed in the curriculum and has been taught well and has acquired the knowledge, skills and dispositions that the curriculum is trying to create, they should succeed in the qualification.