Examinations work like magnets with which the instruction in the classrooms must align itself in order to be judged as ‘effective’. All of us have been through the end-of-the-school-term examinations of one type or the other and would readily recall the powerful effect they had on what was taught to us and how it was taught. Our teachers, if they were worth their while at getting their students succeed, told us exactly what to study and what to leave from the syllabus in order to get maximum marks. Not much has changed between then and now.

Since the stakes of the examinations remain high for the students and teachers, they continue to influence what happens in the classrooms. So profound is the effect of examinations on the classroom practices that substantive improvements in examinations ought to be a central plank of the overall education reforms. In this and some subsequent articles, I intend to highlight and develop further the case for reforming our examination system.

Before going any further, I would like the readers to keep in mind a distinction between examinations and other forms of educational assessments such as the sample-based studies of learning outcomes. I mention this distinction only to emphasise that my concern in these articles centres exclusively on large-scale terminal examinations traditionally administered to students in the secondary/higher secondary schools and more recently also at the end of class 5 and 8.

In my last article, I had discussed the implications of a flawed process of paper setting and scoring. I noted that the test papers were usually set by individuals who were seldom, if ever, trained in the art and craft of designing high quality assessments. Inadequacies in their training were further compounded by sheer absence of any credible external quality controls. The test papers were landing on the desks of our students on the prescribed days of judgement, year after year, without any system of quality assurance. I also suggested that examining bodies should create a demand for professionals trained in testing and that the higher education institutions in both public and private sectors should play their role in meeting this demand by offering accredited course of study.

But as we begin to push for improving the quality of examinations, we will do well to keep in view the purposes served by them. I have already written about the possible use [and misuse] of examinations elsewhere. However, I think it is worthwhile to revisit that conversation for this series of articles. Of course, the sole purpose of large-scale examinations cannot be to provide each student with merely a report card. Well-designed and well-administered tests can generate a treasure trove of information that can be used for a variety of purposes, some of which I mention below.

First, the results of examinations are used to sort the students in various professional and vocational streams. This is one purpose that is already being served by examinations in Pakistan. This is accomplished by using students’ total scores in the secondary school examinations to allocate them to various streams at the higher secondary school [or intermediate college] level. Students’ marks in the secondary school examinations are the proverbial keys that open, as well as close, various professional streams to them.

The examinations are used as a sorting machine the world over and this function has elicited mixed responses from the educators. Some see this function as necessary for social efficiency of a society; others critique sorting as contradictory to the principles of equity and social equality. Be that as it may, the use of examinations as sorting machines is very real -- we have all been sorted one way or the other by the examinations.

Second, the results can be used to assess and respond to the professional development needs of the teachers. Several years ago, I worked with a small system of schools, which were using the information generated by regular tests as a basis for meaningful conversations among the teachers. The process involved displaying the comparative results of assessments from various clusters of schools to all teachers in plenary meetings. The teachers whose students did relatively better on particular items were asked to relate their teaching strategies to their peers from other schools. It was a good example of teachers teaching teachers.

More importantly, these conversations also illustrated to me one of the ways in which information about learning outcomes could be used in the professional development of teachers. In general, by using curricular objectives as the basis of an analysis of student outcomes, we can identify curricular objectives that do not appear to have been adequately met by the students. This information can inform the content of regular teacher professional development activities.

Third, the results of regular examinations can be used to monitor and review effectiveness of new interventions/projects to improve schools. All interventions that aim at improving the quality of education on public expense should be judged on the basis of their impact on learning outcomes. This is particularly important given a deluge of ideas about fixing our education system. Here is a hypothetical example.

Recently, the government of Punjab has been somehow persuaded by some policy entrepreneurs to provide android tablets to all children enrolled in the public schools. The assumption is, as it should be, that the children will learn more and better once they have laid their hands on android tablets. Wouldn’t it be better to test this assumption by providing tablets to a small number of 5th grade classrooms for a year? At the end of the year, the government could use the results of Punjab Examination Commission (PEC) examinations to determine if the children were indeed learning more and better as expected. The government could then upscale the policy if the results showed that children with tablets learned more and better or it could scrap the proposal if they did not. The governments can, and should, commission both ex ante and ex post policy analysis based on results of examinations.

Fourth, the data generated by the examinations can also be used to monitor the progress and gaps in the learning outcomes between and within different geographical regions, i.e. at the province, district, sub-district, and school levels. In other words, these data can help determine the equity, or lack of it, in student outcomes.

A study conducted jointly by Campaign for Quality Education and academics at LUMS offers a good demonstration of this use of data generated by examinations (see this report at http://tinyurl.com/bpawpfb). The researchers used the data from examinations conducted by Punjab Examination Commission at the end of class 5 and 8. They found that the inter-school variation in the performance of schools within the same districts in Punjab was a lot more pronounced than the variation across the geographical units such as tehsils and districts. These findings suggested that the school level effects on student learning were more important than the effects of regional differences in inputs to schools. Such results can potentially spawn a series of useful policy relevant studies to study school-level effects on learning in more depth.

Finally, I will briefly touch upon test-based accountability of teachers. Although this is becoming increasingly popular, the idea of test-based accountability of teachers is deeply flawed. It is flawed inasmuch as it erroneously positions teaching as the sole determinant of student learning and performance on tests. But this assumption ignores the effects of the families’ socio-economic backgrounds, parental interest, levels of nutrition, and a host of other factors including teaching on children’s cognitive gains. The evidence does not support the use of test results to hold teachers accountable.

Since the test-based accountability raises the stakes for the teachers, there are reports that it has resulted in the use of unfair means as well as watering down of educational standards wherever it has been put to use. This kind of accountability is not likely to make teaching and learning any better.

Examinations can serve multiple purposes. Besides driving the teaching and learning practices in schools, the results of large-scale examinations at all levels can be used for policy relevant research. All new interventions in education sector, especially the ideas and projects brought into Pakistan by the aid agencies, should be scrutinised and reviewed in terms of their impact on student learning.

The present system of examinations is merely working as an inefficient and low-quality sorting machine while the remaining purposes of examinations are entirely neglected. The importance of improving the examinations and using their results appropriately cannot be ignored. It is a mistake to assume that we can reform education by training teachers or by giving tablets to the students or through any other innovative strategy without reference to data generated by large-scale examinations. When a doctor treats a sick man, she must depend on the tests of his blood or other diagnostic tests to determine whether her prescribed treatment is working or not. So it must be with an ailing system of education.