Mike Ebbsworth: Well, ultimately, I may have used the wrong words in saying that we're holding the English back. The English version would still be in the process, just as the Welsh version would be, and then they would catch up as they approach the date for publication and printing. There is an element of editing the English as well as the Welsh and that would happen in that eight-week period. So, a draft form might be available, and we do ensure that our draft versions are available on our secure website as soon as possible, but then that has to be held up before publication.
Lynne Neagle AM: Thank you. John.
John Griffiths AM: Just on that, Chair, it's nonetheless the case, as we've heard, I think, and as the letter to the committee from Kirsty Williams has made clear, that there might be a situation where the English-language version of a textbook could be made available, but it isn't made available because there isn't a Welsh-language version. And, as the Chair said, is that about putting those students in Wales on the same footing, and if so, how does that relate to the fact that these students, including those who could have had earlier access to the English-language version, are competing with students over the border, as we heard on the video earlier, for those precious university places? Doesn't it place the English-language student using the English-language version at a potential disadvantage, compared to students over the border?
Gareth Pierce: I think this is a point for Qualifications Wales to a certain extent, but applicants for qualifications in Wales are assessed as a national cohort, if truth be told, so, each piece of information we have on the standards is evaluated for that national cohort. The standards stem from the three-nation system. So, in discussion with Qualifications Wales, we ensure that there is no disadvantage to any cohort in Wales in any subject, be that because of a textbook being unavailable or for any other reason.
Philip Blaker: Certainly, from a comparable outcomes perspective, the fewer differences that there are that are differential—. So, the idea of textbooks being available in English and Welsh at different times could potentially create another dimension in terms of disparity. So, if we're looking at it strictly speaking from a comparable outcomes perspective, it makes the awarding easier if everybody's on a common playing field. It's worth remembering—we provided some evidence in our original letter particularly looking at religious studies on the basis that religious studies was a subject that had been mentioned. This subject has already been awarded at AS-level last year, so it has gone through an award and will go through the full A-level award this year. The comparable outcomes approach did protect those learners last year, in the absence of a textbook, and we saw stability in the results that came out of the cohort level. Similarly, with other subjects that have gone through reforms, we've seen it with the GCSEs that were awarded last year, ASs that were awarded last year and A-levels that were awarded last year: we saw good comparability year on year.
Lynne Neagle AM: Go on then, Darren. Is that okay, John?
Darren Millar AM: Just a very brief follow-up on that, at the cohort level, yes, you suggest that there is some evidence that students were protected, but what you don't know and can't tell, because you're not able to drill down to the individual student level, is whether children, young people, were disadvantaged as a result of their learning style being more textbook orientated than that of their peers.
Philip Blaker: I guess we're in a situation where there's no evidence either way that that—
Darren Millar AM: The absence of evidence doesn't mean that it didn't happen.
Philip Blaker: No, but there is no evidence to suggest that. We have no positive evidence to suggest that.
Darren Millar AM: Well, I think you've seen plenty of evidence about the stress that it's causing, and the concern that learners have about their outcomes. I think it would be very interesting, actually, to take some teacher assessments and expected grades and compare those to the actual grade outcomes. Is that something that Qualifications Wales could do on a sample basis?
Philip Blaker: Historically, that sort of evidence has been very unreliable. Gareth would probably have a stronger view on that than me. Predicted grades used to be provided. Certainly, back in my day at an awarding body they were provided, and there tended to be a very low correlation between the predicted grade and the actual achieved grade. I don't know if you've got anything on that, Gareth.
Gareth Pierce: No, we agree with you that the comparisons are not that reliable, because there are so many factors. But I think, at the end of the day, what we see is, yes, I agree there is evidence of stress, but I think there's also evidence of young people and their teachers really using the available resources and the available skills in approaching revision and preparation, and tending to do that very well. That's the evidence that we've got, for example, in the AS religious studies last year, that the cohort did extremely well in fact. We can't disagree that there's evidence of stress, but I think we can also point towards resilience and excellent approaches to drawing together the various sources that they've got: classroom notes, revision notes, as they mentioned, and the various other materials that they referred to.
Lynne Neagle AM: Okay. John.
John Griffiths AM: Coming back to the effect of the lack of resources, we heard on the video, as you've just mentioned, Gareth, that pupils were experiencing a lot of stress and anxiety, and it affected their well-being in terms of their perception that they might be disadvantaged by the lack of resources, as we discussed. We know exam time is very, very stressful—I'm sure all of us would agree with that—in general, without any additional problems or perceptions of problems. So, with that sort of background, would you accept then that some students have been negatively affected by the lack of suitable support materials for general qualifications? I know we've discussed this in terms of it being difficult, perhaps, to show cause and effect, but I think most people watching that video would clearly come to the view that some students have been negatively impacted. Would you accept that?
Gareth Pierce: I'd have to agree. There is evidence of their concern, isn't there? But as Philip said, what we don't have is evidence that that anxiety then has an impact on their attainment, because there are so many factors, there are so many different kinds of resources that they've referred to—even those who want to see textbooks, they're also talking about so many other materials that are available to them. Of course, it's also clear that they are conscientious young people who have worked hard, so you have a picture there of young people who are concerned, yes, and as you've said, examinations are a stressful time in general, but they are also young people who are doing their level best, and usually do get the results that they have worked for. That's the evidence that we have: the results tend to be achieved.
John Griffiths AM: Okay. Coming back again to matters that we've already discussed, but just to put to you in a fairly focused way, would you accept that it isn't fair that some pupils are not having the support material that others are having, depending on whether their particular subject is involved and the language that they're studying through? Would you accept that that is a basic unfairness in the system as it currently exists?
Gareth Pierce: Ideally, resources, digital accessibility and digital literacy should be available for every pupil on an equal basis. But I think perhaps if we analysed this in much detail, different individuals would be able to say, 'There are different types of unfairness.' You referred earlier to the digital inequalities. Is there an economic inequality? There is some evidence from the teaching unions that suggests that not every school is in the same situation as each other in terms of buying the resources that are available. So, I think your question drives at a very important point: what sort of inequality could be having an impact on young people in terms of their achievement during their school time? Ideally, each one of those elements of potential inequality should be levelled. So, I don't think I can go any further than acknowledging that any inequality is unfair, but there are so many different kinds and the textbook is not the only one, and therefore we're not going to be able to analyse the impact of that on its own.
John Griffiths AM: Could I put to you, finally then, what Estyn have said about reported delays in the production and distribution of educational resources such as textbooks, marking criteria and specimen papers, that that delay has impeded the ability of teachers to plan adequately, and that this effect on their ability to plan adequately is very likely to have had an impact on pupils' achievement? Would you accept that?