I once heard the ex-secretary education (KP) Afzal Latif (a long time veteran of the department) say that any policy decision should be made keeping the child in the fore of our minds. It is, therefore, no surprise that the new Education Sector Plan (ESP), KP for 2015-2020 which was designed under his tenure keeps the question "what’s in it for the child?" as the yardstick by which to measure the worth of any policy decision.

In my dealings with the department and affiliated institutes in KP, I have seen that the hearts of senior bureaucracy are in the right place. Whether, policies introduced are bringing desired changes is another question and requires extensive review of policy conceptualisation, design, implementation, and robust evaluation of its effects.

Today, I briefly review the major policies in KPK’s ESP 2015-2020, which include an equity policy, a merit-based teacher recruitment policy, revised textbooks and performance based incentives for schools and teachers. An objective assessment of the success or failure of any policy requires identifying clear metrics that can be tracked before, during and long after a policy takes effect. When the collection mechanism for data to feed those metrics does not exist, that too has to be implemented. Without robust data collection mechanisms for the long-term, even the most well-intentioned policies remain shots in the dark.

Short of such data, the review below is not a value judgment about policies discussed but raises the need for evidence based policy design and implementation as well as robust evaluations of the process so that eventual success or failure of a policy is not because of design and delivery failure.

Equity policy

A major accomplishment of the KP government is the formulation and roll out of a comprehensive equity policy. The policy acknowledges that access is not just a gender issue but is equally dependent on factors like geography and socio-economics. Similarly, equity does not just mean access to education but implies access to the same quality of education for all.

Schemes introduced by the KP government, therefore, target various causes of inequitable access to education. One such scheme is gender based stipends to increase girls’ enrolment and retention in the province at the middle and secondary school levels. According to the government figures, "In 2015-16, 443,320 female students benefitted from the girls’ stipend scheme which had an allocation of Rs1.36 billion. In 2016-17, a budget of Rs1.34 billion has been earmarked and 455,364 girls will benefit from the scheme."

Schemes like these are usually easier to evaluate to see if the policy is being effective or not since the connection between inputs and end goal (enrolment of girls for instance) is easier to establish and can be evaluated objectively. However, currently there is no evidence from a comprehensive third-party review that shows if stipends are indeed helping boost enrolments and keeping girls in schools. Other questions to address include establishing whether the stipend amount is sufficient to incentivise girls over the longer duration and if stipends are being disbursed on time. The KP government should commission studies that gather such data to make early judgements about the effectiveness of the policy.

Similarly, another initiative introduced in KP is a voucher scheme that targets out-of-school children who are 5-16 years of age. There are various models of voucher-based schemes around the world all aimed at (1) expanding parental choice; and (2) creating competition among school to improve performance. However, there is a scarcity of robust impact evaluations, based on the characteristics of receivers as well as the receiving region, done to understand the effectiveness of voucher-based schemes in developing countries.

For instance, in some countries vouchers resulted in social segregation and concentration of students of low ability in one school. Similar to the girls stipend scheme, it is also critical to study, for example, if voucher sizes are adequate, if parents have the necessary knowledge to make decisions about which school is best for the child, and ensuring that schools are not using the scheme only to increase profits without doing much for quality provision. The department should commission studies that answer these questions before commenting on effectiveness. 

Teacher recruitment policy

In 2012, the KP government revised its recruitment and service rules to introduce a merit-based hiring policy for its teachers. Its objective was to minimise political interference, making the process of hiring teachers more transparent and fair. Policy prioritises better qualified candidates, who score higher on third-party tests and are willing to be placed in a school in their own union council, thus also addressing the issue of inequitable teacher placement.

Since the policy has been implemented, the E&SED has conducted several rounds of province-wide, recruitment tests through the National Testing Service (NTS) to maintain transparency. The NTS is responsible for the entire process from test development through to development of provisional merit lists.

The introduction of this policy is indeed an important achievement by the department. However, now that the tests are part of the hiring process for some years it will be critical to see if the new process is indeed helping recruit better teachers. This can be achieved by reviewing test quality, analysing testing data, and subsequently assessing whether merit-based selection is indeed bringing in teachers that perform better in classrooms.

It may be that the teachers who come in through these tests are still underperforming but are being hired to ensure teacher presence in every classroom, which is fine given it is only gradually and by ensuring that other pieces in the policy environment are correctly placed that general quality of candidates interested in becoming teachers will improve. However, the type of analysis suggested above will still provide useful information about the quality of teachers entering the teaching profession and about their capabilities in relation to the curriculum being taught in schools in the province. This will also help allocate resources (such as targeted teacher trainings) more efficiently. To my knowledge, the department is conducting such reviews. A third party review can determine if learning based on such reviews are being used for improvement. 

Textbook revision

Like most education policies before it, KPK’s new education policy for 2015-2020 also included revised textbooks. My personal observations about KPK’s textbooks for 2015-2016 for grades 6 to 12 were the following: They were poorly edited and composed and sometimes gave the impression that they were not even proofread. Topic coverage in science subjects was superficial, sparse and poorly structured. Topics that deserved at least a chapter of their own were covered in single short paragraphs to enable the writers to check off boxes on the curriculum.

Since then textbooks are being rewritten. Books for certain grade levels are ready, while others are under development. Only a comprehensive review of textbooks can tell the nature of revisions and whether all gaps have been addressed or not. Along with such a review, it will be instructive to look at the process through which textbooks are being rewritten to ensure the government has the right processes and human resource in place to ensure the quality of the process and product, even if textbook writers are hired from the market. 

Performance-based incentives for teachers and schools

Although not part of the existing Education Sector Plan 2015-2020, the government recently announced incentive schemes to improve performance of teachers. These included performance-based promotions for teachers and performance-based awards for teachers and schools.

Predictably, the former initiative has received a lot of pushback from teachers, court cases were filed and as a result of the upcoming general elections, its introduction has been shelved. Similar conditions and benchmarks are also applied to performance-based incentives in other countries.

Two countries that come to mind, where this policy has had very different effects, are the US and South Korea. In the US, school teachers are the most underpaid professionals and have one of the highest burnout/attrition rates. On the other hand, in South Korea teaching is a top career choice with high job security where only 5 per cent of applicants are accepted, and the annual attrition rate is only 1 per cent.

Clearly, performance-based remuneration is being welcomed in one country and shunned in another. There are several cultural and demographic differences at play. One of them is the fact that in the US, like in Pakistan, there is greater disparity between schools across the country than in South Korea. For this reason, ‘performance’ has to be defined with greater nuance, relative to each school. An overly simplistic approach of assessing teacher performance is by the performance of students on a standardised test at the end of the school year. A better approach would be to look at the improvement in student test performance from last year, or from the beginning of the school year. Performance-based promotions of teachers can be done right, or can be done wrong.

The latter, less controversial scheme, awards schools and teachers based on data collected by an Independent Monitoring Unit against 5 indicators: Increase in school enrollment rate, increase in student transition, decrease in student absenteeism, attendance and absence rates of teachers, 100 per cent participation rate of students in the Secondary School Certificate exams and the overall results of schools in the exam.

Anecdotal evidence, in the absence of rigorous evaluation data, suggests that while the policy is well-intentioned, in some cases it may have perverse incentives for certain schools, where schools are less likely to admit to children who are at the risk of dropping out, to ensure they perform against the five indicators used to benchmark performance. Again, rigorous research done to address such design flaws is critical for ensuring effectiveness. 

Conclusion

As an educationist, I am often asked by friends if all this policy making is changing the state of education. Declaring policies effective by only looking at policy statements is overly simplistic. The design and implementation of new policies must be informed and guided by a continuous stream of data, that confirm or reject assumptions as well as is sufficiently detailed to answer all sorts of questions, i.e. What types of teachers have the most success improving children’s learning? What schools/districts suffer from greater teacher absenteeism and why? Where did enrollment rise/drop the fastest? Etc.

All this requires a culture of massive near-real-time electronic data collection, not paper-based annual reports. While such quick and detailed availability of data was a pipedream for policy makers 20 years ago, educational technology like learning management systems and enterprise resource planning systems have brought all this into the realm of possibility years ago. It is no longer a question about whether the success or failure of education policies can be assessed. It is a question of adopting a culture of continuous data collection and data driven decision making. Short of that, we will continue to experiment with policies, and wait for 5 years to get a vague sense of whether or not they had their desired effect.