Bogus University Rankings
By Dr Pervez Hoodbhoy
Islamabad
Parents everywhere, and increasingly in Pakistan, want their children to go to a good university. This is an important decision about a student’s future — and an expensive one. Looking at university websites is no help because many misrepresent or lie. So they turn to international organizations thinking them to be neutral. But they don’t know what lies underneath.
Today over one dozen commercial organizations based in the US, Europe, and China purport to rank universities by academic quality. From some distant office, a handful of employees pass judgement on hundreds — perhaps thousands — of universities they have never visited. Instead, they fire off forms hoping that university officials will fill them truthfully.
The websites of these businesses say nothing of how their own expenses and salaries are met. Unless confronted they won’t tell you they have service contracts with some of the universities they evaluate. Yet their published rankings are still taken with seriousness by prospective students and their parents, as well as by private and government agencies in deciding policies and grant distributions.
The very idea of ranking universities is questionable. Unesco’s 2010 report states: “Global university rankings fail to capture either the meaning or diverse qualities of a university or the characteristics of universities in a way that values and respects their educational and social purposes, missions and goals. At present, these rankings are of dubious value, are underpinned by questionable social science, arbitrarily privilege particular indicators, and use shallow proxies as correlates of quality.”
I agree. Particularly when it comes to Pakistan, India, China or other countries where academic corruption is rarely punished, rankings lose all meaning. Individuals and institutions often turn reality on its head by manipulating numbers. Research evaluations, through multiple pathways, count for 50-70 per cent of a university’s ranking (if not more) but are easily doctored. Here’s an example.
A recently released report by Thomson-Reuters, a Canada-based multinational media firm, says, “In the last decade, Pakistan’s scientific research productivity has increased by more than four times, from approximately 2,000 articles per year in 2006 to more than 9,000 articles in 2015. During this time, the number of Highly Cited Papers (HCPs) featuring Pakistan-based authors increased tenfold from nine articles in 2006 to 98 in 2015.”
This puts Pakistan well ahead of Brazil, Russia, India, and China in terms of HCPs. As the reader surely knows, every citation is an acknowledgement by other researchers of important research or useful new findings. The more citations a researcher earns, the more impact he/she is supposed to have had upon that field.
If Thomson-Reuters has it right then Pakistanis should be out tonight dancing in the streets. India, our not-so-friendly neighbor, has been beaten hollow. Better still, two of the world’s supposedly most advanced countries — Russia and China — are way behind us. This steroid-propelled growth means Pakistan will overtake America in just a decade or two.
But just a little thought shows something is amiss. Where there’s smoke there’s got to be fire! So a four-fold increase in scientific productivity must have some obvious signs. Are science laboratories in Pakistani universities four times busier? Does one see four times as many seminars presenting new results? Are animated discussions on scientific topics heard four times more frequently?
Nothing’s visible. Academic activity on Pakistani campuses might be unchanged or lower today, but is certainly not higher than 10 years ago. So where — and why — are the authors of the HCP’s hiding? One possibility is that these researchers are too bashful to present their results in departmental seminars or public lectures. Like the self-effacing scientific geniuses one reads about, perhaps they are content to be judged by posterity. But something smells of fish.
We now live in the age of the Paper King. A new breed of internet-age university teacher has emerged on campuses in several countries — particularly in corruption-prone ones. He is surrounded by junior teachers, PhD students, and assorted flunkeys. University administrations woo him because they know he knows how to get Thomson-Reuters’ attention. Most importantly, the Higher Education Commission (HEC) also uses Thomson-Reuters’ data to rate and rank.
The Paper King rose to the throne for good reason. He can generate countless research papers without doing real, hard research. Instead, he has mastered several steps: selective cut and paste, choosing research topics of low relevance and noticeability, trivially changing parameters, inventing data, or plagiarizing ideas. Suitably selecting or manipulating a journal means publication is a cinch. Refereeing exists only in name. Some kings become editor, others start their own bogus online journals.
From Paper King to HCA (Highly Cited Author) is then a short journey. The king in Islamabad reaches out to friendly kingdoms everywhere from Jakarta to Shanghai, and Tehran to Toronto. He cites their papers and they duly return his favor, a win-win situation. The king’s citation count rockets up, Thomson-Reuters announces a “path-breaking” gain, the king collects his tamgha and cash prize, and his university rises in international rankings.
A concrete example: for years, scientists in Pakistan and overseas have been informing the COMSATS Institute of Information Technology (CIIT) in Islamabad of a King in CIIT’s midst. His research publications are useless to mathematics research and, in several instances, demonstrably wrong. Expectedly, this protest made not the slightest difference. This year CIIT was ranked number one for research among Pakistani universities. The rector thanked Allah, but I think He was unfairly blamed.
Of course, a university is about more than research — there’s also teaching. So why not judge by teaching quality? Unfortunately, even with intimate knowledge, judging teaching within a single department of a single university is notoriously difficult. It is plagued by subjectivity, and by the very limited direct personal experiences of those called upon to assess teaching performance.
To worried Pakistani parents and students: simply ignore what you are told by international-ranking organizations or the HEC. Rankings are useless. Instead, keep your ears to the ground and listen carefully to intelligent students or teachers from within that university. They can tell you much more than so-called professionals.
(The writer teaches physics and mathematics in Lahore and Islamabad. Dawn)
-----------------------------------------------------------------------------
