Black hole in the research cosmos

To make higher education relevant to society, HEC needs to revise each and every metric that is used in ranking, promotion, awards and rewards in such a fashion that quality becomes the qualifier rather than quantity

Black hole in the research cosmos

The phrase "Higher Education" hardly resonated in the decision-making echelons of Pakistan until 2001, and it was not a popular career choice among the top students. Then a revolution in higher education took place, and Prof Dr. Atta-ur-Rehman aimed to resuscitate the sector through quick and elaborate measures.

Initially, the idea of university professors publishing research articles was met with resistance. He reiterated the importance of scholarly publications and always talked about the abysmal state of research reflected by the 600 or so impact factor articles in the year 2000-2001.

Because of his Chemistry background, he propagated the idea to "increase the number of impact factor publications and become a hero for the next generations". This maxim laid the foundation of revolution in the higher education institutions (HEIs). Consequently, an exponential growth happened in the number of impact factor publications and the current number stands at 12,000 for the year 2015. To reach this magical number in the short span of only 14 odd years, the tax payers had to inject more than 400 billion rupees into the sector.

The extraordinary emphasis on the number of impact factor publications has changed irreversibly the alternative reality of higher education in Pakistan -- making higher education and research an engine for inclusive social progress for the society. However, contrary to expectations, the 12,000 odd papers every year have not been able to make any significant impact on the society or even in the respective knowledge domains of their authors. The nation is now rightfully demanding to analyse the reasons why these papers did not make any impact.

It is a well-known fact that a mushroom growth of journals has been witnessed in different parts of the world and they have been successfully indexed in the Web of Science (WoS). Most of these have also been assigned impact factors. However, most of these journals do not have a credible peer review process; thus, they publish all types of "junk articles" by minting money from the authors in the garb of a publication fee.

In order to make higher education relevant to the society, HEC needs to revise each and every metric that is used in ranking, promotion, awards, rewards in such a fashion that "quality" becomes the qualifier rather than "quantity".

During the initial stages of HEC, most of the faculty members already present at Pakistani HEIs had never been published. Since the mantra of "impact factor" was the symbol of prestige, these professors skillfully applied their knowledge of optimisation to identify low-quality "impact factor" journals where they could virtually publish any piece of paper.

This created a snowball effect and the number of publications in these low-quality impact factor journals exponentially increased and below-par researchers who tricked the single metric (impact factor) system basked in the light of national awards and became role models for the next generation of researchers. Meanwhile, those who decided to aim for "high quality publications" were left behind in the rat race of number of publications. To publish in a prestigious journal, the average time from submission till acceptance (if accepted) is in between 1 and 2.5 years. In comparison, the dubious low-quality impact factor journals take less than 6 months to accept every submitted article.

The mantra of "many impact factor publications" created a quality black hole in the Pakistani research cosmos where it sucked scarce national resources but nothing valuable came out of it for the social or economic progress of the country. The professors never engaged in doing research on local problems and then publishing papers on solving them; rather, they hibernated in their offices (or labs) and mainly focused on doing incremental tweaking in the existing papers, to produce a series of nearly invaluable papers in low quality impact factor journals.

Recently, the shortcoming of an emphasis on impact factor has been felt internationally; therefore, now more than 46 metrics are defined to measure the quality (or prestige) of a journal. The most famous come from two well-known international projects: Eigen Factor and Scimago. The basic hypothesis of the new metrics is: instead of using raw citations (the impact factor definition) they take into account the prestige of the author who is citing an article and the prestige of the journal from where the citation is originating. As a result, citation by a Nobel laureate could not be equated with that of an unknown PhD student from Ghana. Similarly, a citation in the journal ‘Nature’ is not equal to a citation in an unknown (low impact factor) Chinese journal.

Now the challenge is how to use these multiple quality factors and integrate them in a systematic fashion to determine the overall quality of a journal. Journal Quality Ranking System (JQRS) [http://jqrs.ist.edu.pk] is one such effort that categorises journals into five categories -- Gold, Silver, Bronze, Honorable Mention and Unranked -- on the basis of five well known internationally published metrics: Eigen Factor, Article Influence, Scimago Journal Rank (SJR), H-index, and CD2 (impact factor computed from Scopus). Each one of them is equally weighted and mapped to 100 points; as a result, the cumulative score of a journal is calculated out of 500 points.

A journal is classified as Gold, Silver, Bronze, and Honorable Mention if its cumulative JQRS score is greater than 80 per cent, 70 per cent, 60 per cent, and 50 per cent respectively. If the score is less than 50 per cent, the journal is not ranked. The fundamental postulate is: if a journal is optimising quality, all of its quality factors will be the high; meanwhile, if a journal is artificially optimising any one factor this unethical practice will be detected by correlating with other factors.

Using the system, management of HEIs and HEC now can distinguish between researchers who publish in an impact factor journal that has 20 per cent JQRS score from those who publish in an impact factor journal that has 98 per cent JQRS score belonging to the same area of research. The current HEC policy benefits researchers who publish 10 junk articles in 20 per cent JQRS score journals to those who publish just 3 articles but in the most elitist and prestigious 98 per cent JQRS score journals.

Another intriguing insight into the impact factor papers downloaded from WoS is that in 2015, Pakistani HEIs published more than 12,000 impact factor journal articles with a national average JQRS score of 225/500 (45 per cent) and none of the five-metrics average is above 60 per cent. This shows that most Pakistani scholars prefer to publish their research in unranked journals. If we compare this with only one prestigious university of the world, Harvard University, its faculty alone has published more than 15,000 articles with an average JQRS score of 402/500 (80 per cent) in the year 2015. It is mind boggling to even comprehend for us that how a single university could consistently target Gold journals as a routine matter. The statistics for MIT are even more interesting: it has published more than 1700 articles in 2015 but with a JQRS average score of 425/500 (85 per cent). No wonder, these universities are making a great impact in their concerned knowledge domain.

If this rat race of number of papers in Pakistani universities is not arrested through a policy intervention, our HEIs will be left behind in their contribution to society. In this gloomy scenario, the one slight ray of hope is that Pakistani university faculty members have published more than 1600 (15 per cent) Gold category journal articles. However, with the emphasis on quantity over quality, the authors of these highly prestigious research papers remain unsung heroes who are neither recognised nor rewarded. The time has come to put efforts to track them, encourage them and present them as role models for a new "Pakistani Dream" to the next generation.

It is recommended that the government, in collaboration with HEC, should build a Pakistan Education Service, which should be staffed by only the most competent people, hired through very competitive modern hiring frameworks, to become elite teaching, research, and administrative professors with an innovative, out-of-box mindset. HEC has to intervene to ensure that "quality" and not "quantity" becomes the new educational anthem of the country: "aim for top quality and make an impact on society".

In order to make higher education relevant to the society, HEC needs to revise each and every metric that is used in ranking, promotion, awards, rewards in such a fashion that "quality" becomes the qualifier rather than "quantity". In this regard, a Book Quality Ranking System (BQRS), is also an intervention urgently needed to recognise quality peer reviewed research book manuscripts -- a just demand of Social Science faculty members because their prestige (as a research scholar) is only established if they publish  impactful scholarly book manuscripts.

To conclude, quality teaching, quality research, quality governance and quality entrepreneurship and community service must define the agenda for HEIs for the next decade.

Black hole in the research cosmos