The Cambridge debacle

The writer is an independent education researcher and consultant. She has a PhD in Education from Michigan State University.

On August 11, Cambridge Assessment International Education (CAIE) – formerly CIE – released the much-awaited results for IGCSE and A-level students.

This year, I was again among the millions of parents anxiously waiting for its release. My child had an all A*/A-grade track record from her IGCSE in previous years. At the beginning of the school year, her school conducted a test (ALIS) of incoming A-levels students that predicts probabilities of achieving different grades.

Aside from that, the school also generates a second prediction based on academic performance in the previous year. Year round, she also worked as hard for school as any parent could ask of their child.

In her mock exams (the last major assessment of the school year before the final exams) she was on track for a perfect result. Even after the pandemic struck, and she felt she needed more help with a subject than the school could provide with its online classes, she had the benefit of a qualified online tutor in the UK. School teachers who have known her for years were so sure she would do well that the school chose her for head girl for the coming school year. All indicators gave reason to hope for an excellent result this year.

As we now know, the CIE did not conduct any IGCSE or A-levels exams this year. Instead, it announced it would use collected evidence of academic work (mock exam performances, homeworks, etc) and grades predicted by teachers as input to a blackbox algorithm to compute statistically normalized grades that would conform to ‘typical performance’ of a student’s school in past years.

With this background, the grades received last week came as a devastating shock, causing us and many others to reconsider future plans. To be fair, even in previous years, the grades students receive in Cambridge International exams are always subject to a certain degree of unpredictability and arbitrariness, as a recent article in The Economist put it (‘Arbitrary A-level results are normal’, August 15, 2020). However, this year that randomness was off the scale.

To explain what was wrong with the CIE’s approach to determining grades, let me offer a grossly simplified explanation that even someone unfamiliar with statistics can appreciate. If you conduct an experiment in which you flip a coin 1,000 times, you can predict with a good degree of certainty that you will see almost 500 heads and almost 500 tails. The actual number may differ a little every time you conduct this experiment, but you will not be very far off. This is a (successful) demonstration of predicting the behavior of a population.

However, if I ask you to predict the outcome of a single flip of the coin you would be a lot more wrong a lot more times. This is an (unsuccessful) attempt at predicting the behaviour of an individual member of a population. In a sense, this is what the CIE attempted with its grade prediction. Theoretically, predicting individual student performance could be done with more certainty, but it would require many more data points of past and present students, which was not possible in this situation.

Reaction to this methodology from students and parents, both internationally and in the UK, was loud and swift. Parents in Pakistan loudly voiced their criticism of schools and teachers who, according to them, failed to provide the evidence required for the CIE to justify better grades. Parents mulled pulling their children out of their schools for failure to secure grades for their children; this threatened to ruin schools’ reputations and business. Overnight, multiple online petitions popped up demanding the CIE review its grading policy for this year. Even Minister for Federal Education Shafqat Mahmood, lent his voice of support to affected parents.

In the end, Ofqual and the CIE had to retract predicted grades and replace them with mock exams or teacher predicted grades, first in Scotland, then in England, and later internationally. The point is that the CIE bungled the result by devising what turned out to be a poor method for awarding grades this year. In the face of the uproar, its revised decision will cause some grade inflation, but will lead to fewer complaints of unfair grades.

Given that schools follow very different grading standards for their mock exams and often do not maintain portfolios of students’ work during the year, submitted evidence was expected to vary widely. In a world where all schools and teachers could be trusted to remain fair, professional and resist pressure from parents while submitting predicted grades, the CIE’s method could have worked.

Unfortunately, that is not the world we live in. Teachers at some elite private schools in Pakistan have reported that when the CIE revealed its grading plans in spring, schools came up with a variety of schemes to trade high predicted grades for ‘tuition fees’.

What does this episode tell us about the private school sector in Pakistan or in other school systems similar to the one at home? It identifies glaring deficiencies in Pakistan’s private education system. The competition to attract students willing to pay exorbitant tuition fees is so intense, it leads them to advertise their students’ grades (along with their pictures) on billboards, something I have not seen in other parts of the world.

In such circumstances, relying on all schools and teachers to act with professional integrity was always unrealistic. Every so often, we hear about cases of bribery in the public education system, where someone bribed someone to get their child through Matric or FSc exams. Those parents share the same cultural DNA as private school parents, so why should we expect either to act too differently?

We can expect this year’s results to be significantly inflated. For students that did not graduate this year, differentiating themselves from their cohort competitors is now all up to next year’s exam.

Looking forward, the CIE should plan for a better way of dealing with such a situation, should the pandemic persist and this situation repeats itself next year. Students applying to Pakistani universities will have a harder time, because local conditions demand that admissions processes rely on transparent scoring formulas that leave no room for subjective evaluations. Universities with admissions processes that take a more holistic view of applicants like in the US, Canada, etc will be better able to adjust to this year’s complications.

Fortunately, it seems that the CIE did right by students that were kneecapped by its blackbox algorithm, even if it meant others got better grades than they deserved. The people who protested the initially announced results, the students who launched petitions and even our education minister can partake in the credit of this outcome. In the end, it was the numerous voices of informed and involved parents that compelled the CIE to bring this episode to a better conclusion.

Today there are many fewer voices of parents to be found who are involved and invested in the success of our public schools. Masquerading a routine review of minimum learning standards as major curriculum reform and beefing up religious learning contents will not be enough to lure engaged parents back into public schools.

These events are also an indictment of our own education system. It has once again exposed the flight from public schools to private schools by anyone with the means for it.

Email: arazzaque@gmail.com