In the last two articles on the examinations, I have argued that Pakistan’s examination system at the secondary and intermediate levels is inherently unfair to the students and unable to accomplish the purposes, which must be served by any examination system. Given the attention paid to it by the media, the rampant practice of cheating may appear to be the biggest problem afflicting our examination system in the eyes of the public. But it is only part of the problem. Even if we were somehow able to destroy the booty mafia and to reduce the use of unfair means to near zero, the tests would still remain invalid, unreliable, and unable to discriminate between candidates at different ability levels. It is the archaic practice of test construction and scoring by the boards of intermediate and secondary education (BISE) that needs to change.

The world in which our children are growing is becoming increasingly complex and harder to live in. More than ever, it requires them to understand and work with complex piece of information. They must also develop skills to theorise, analyse information and solve concrete problems. The curriculum and instruction must prepare them to develop these skills. Examinations should be so constructed as to measure these skills. The data collected through examinations should be used to certify students, but also let those concerned with curriculum and instruction know whether the curricular objectives were met [or not] so they can take appropriate steps to improve their own practice.

Most commentators, including those directly concerned with the secondary and intermediate examinations, will agree that the examinations are achieving none of the above. As I have repeatedly emphasised in the previous articles, the examinations, whether we like it or not, have a profound influence on what teachers teach and students learn. We have plenty of well-qualified teachers in our secondary schools. But to be recognised as ‘good’ teachers, they must teach to the test. Wouldn’t it be better if they were teaching to the tests designed to help students use a variety of concepts and skills to solve problems. Changing the nature of examinations can encourage teachers to teach for understanding.

Improving secondary and intermediate education is also important for improvements in education at other levels as well. But governments and civil society in countries such as ours are sold to the idea that the returns on investments in primary education are supposed to be higher than returns on investments in secondary education.

In the mid 1990s, an emphasis on primary education kick-started the ‘primary school bandwagon’ at a global scale. Our government and civil society has been on this bandwagon for too long now to shift their focus toward secondary and intermediate education.

But little do they realise that primary schools cannot improve without solid improvements in secondary education. Reports after reports complain of subject knowledge deficits in the teachers. To remedy these deficits, the government, with support from donor agencies, has embarked on initiatives to reform teacher education. New pre-service programmes as the four-year Bachelors of Education (honours) and Associate Degree in Education (ADE) are outcomes of these efforts. But these initiatives, though well-intentioned, are inherently flawed since the subject knowledge deficits are produced at the secondary and intermediate levels, which is where they need to be fixed.

Higher and professional education is meant to build on the secondary education and not to remediate the deficits caused by it. So the supply of competent teachers is not likely to follow from improvements in teacher education alone, but also depends on how well we are able to fix our secondary schools.

Reforming the examination is the key to reforms in secondary and intermediate education. Although all steps in the testing cycle need improvements, we can begin this conversation with a focus on test construction. Improving the quality of test construction would have been much easier if all provinces had only one agency responsible for it. The provincial governments could allocate a dedicated team of subject and psychometric experts needed to develop high quality examinations to one such agency.

Read also: Pros and cons of standards

But currently we have 25 Boards of Intermediate and Secondary Education (BISE) in the country (eight in Punjab, eight in Khyber Pakhtunkhwa, six in Sindh, and one in Balochistan). These boards are solely responsible for the development and conduct of secondary and intermediate examinations in their jurisdictions. Children of the elite can always opt out of the examinations conducted by BISE by opting for testing services offered by foreign-based providers such as the Cambridge Assessment.

Why do we have as many as 25 boards when having multiple boards within a province can not only compromise the quality of test construction but also make it hard to improve it? The test construction is usually done centrally in most countries where testing services are part of the public sector. Singapore, Malaysia, Portugal, Australia, India, and New Zealand, to just name a few countries all have central testing services at the state or national level. Where the boards have multiplied within the jurisdiction of a state, as in the case of UK, it has happened in the marketplace of testing services and not in the public sector.

So how did we develop so many BISEs in the public sector? Let us briefly trace the development of BISEs in our country. Until the late fifties, the universities organised the intermediate education in Pakistan. In 1959, the Sharif Commission on education recommended its separation from the universities. The commission recommended transferring classes 11 and 12 to the control of Boards of Secondary Education and regarding them as an integral part of secondary schooling.

BISEs were not meant to be merely a testing service. As the Sharif Commission had recommended, they were responsible for the overall organisation of secondary and intermediate education. Thus, it is that these boards assumed a relationship to schools, both public and private, which was similar in essence to the relationship that universities had with affiliated colleges. The schools and intermediate colleges were required to be affiliated with a BISE in the same way as the colleges are affiliated with the schools. BISEs were empowered to prescribe curriculum, register and regulate private schools, hold sports events, and so on.

Read also: Valid and fair assessments 

But it is unclear if any of these functions necessarily required the development of more than one board of education in each province. In fact no province has multiple curricula or several sets of different regulations for the public and private schools in its jurisdiction. So there was no rationale for multiplying such autonomous boards with powers to prescribe curriculum and other regulations. Nevertheless, additions in boards were justified on the basis of an increase in the number of schools and students within the jurisdiction the existing BISE.

But as we have seen above, an increase in the number of candidates have not been advanced as a reason to multiply testing services in the public sector in other countries. Here is an extreme example from the neighbouring India. The board of secondary education in the state of Uttar Pradesh (UP) in India boasts to be the largest board in the world. The population of province of UP is more than the entire population of Pakistan. Yet, there is only one board in the UP.

Improving the quality of paper setting will require that we move away from nominating individuals for paper settings. The BISEs will also need to move away from a reliance on textbooks towards a curriculum-based approach to test development. They will need to focus on development of robust items by joint teams of subject experts and psychometricians instead of assigning the development of the entire papers to individual teachers.

The BISEs will have to develop a culture of empirical testing of the items to establish their validity, reliability, and difficulty levels. But it will be very difficult to build the capacity of each BISE within each province. Attempts to build capacity at multiple sites are likely to be expensive.

The provinces should consider developing central examination authorities, at least for test construction, for analysis of data, and development of results and other reports. Even now, the boards are forced to coordinate with each other when setting the papers. Isn’t there merit in taking the responsibility of test construction away from the boards and centralising it at the provincial level? Wouldn’t it be much more efficient and effective for a central authority to assemble the teams of experts needed for the development of high quality tests?

The boards cannot be too happy with their current performance as far as paper setting, scoring, analysis of data, and development of results is concerned.  There is no harm in having an informed debate on various ways of improving secondary examinations including the idea to develop a testing service at the provincial level to take over some of the functions currently, and badly, performed by the BISEs.