Watch out for cheats in citation game

WORLD VIEW A personal take on events UC DAVIS SCHOOL OF LAW Watch out for cheats in citation game The focus on impact of published research has created new opportunities for misconduct and fraudsters, says Mario Biagioli. W hen scientists misbehave, the culture of ‘publish or perish’ is often blamed. Some researchers cut corners, massage data and images or invent results to secure academic papers and the rewards that come with them. This is rightly regarded as misconduct. But there is a new class of bad behaviour — one that is driven by a related but different pressure: ‘impact or perish’. It is no longer enough for scientists to publish their work. The work must be seen to have an influential shelf life. This drive for impact places the academic paper at the centre of a web of metrics — typically, where it is published and how many times it is cited — and a good score on these metrics becomes a goal that scientists and publishers are willing to cheat for. Collectively, these new practices don’t seek to produce articles that are based on fraudulent evidence or claims. Rather, they use fraudulent means to secure their publication, enhance their impact and inflate the importance of those who write them. They are on the march — and scientists no longer have to look far to find them. News about research now includes regular reports of authors who supply fake e-mail addresses of suggested peer reviewers. They then use those addresses to offer reports that are supportive enough to ensure that the paper is published. ‘Review and citation’ rings go a step further, trading favour- able fake reviews for citations to the reviewer’s work. Others hack publisher databases to seek more invitations to review papers, and so pos- sibly insert more citations to their own work. All metrics of scientific evaluation are bound to be abused. Goodhart’s law (named after the British economist who may have been the first to announce it) states that when a feature of the econ- omy is picked as an indicator of the economy, then it inexorably ceases to function as that indicator because people start to game it. What we see today, however, is not just the gaming of science metrics indicators, but the emergence of a new kind of metrics- enabled fraud, which we can call post-production misconduct. It seems to be as widespread as other forms, with at least 300 papers already retracted because the peer review had been tampered with. A curious feature of this kind of misconduct is that the work itself — the science reported in the paper — is usually not in ques- tion. Those responsible for this kind of post-production misconduct seek to extract value not from the article itself, but from its citations. From their point of view, it does not matter whether the article is ever read by a scientist, only that its citations will be harvested by bots. This means that unlike data fraud and other forms of conventional misconduct, post-production misconduct does not necessarily pol- lute the scientific record with false results. But it does erode the credibility of the publication system. And it is more common in ALL emerging countries, perhaps because universities there place the most emphasis on metrics to quickly become globally visible. How can it be tackled? Post-production misconduct is less likely to be the work of individuals — a hyper-productive protege operating under the protection of an established mentor who is unwilling to ask too many questions — but increasingly emerges from collaborations. As such, its traces are usually beyond the reach of peer review, which itself is often targeted by these fraudulent schemes. The exposure of citation and peer-review rings has generally been down to data analysis — of the wording of reviews, review turnaround times, citation patterns and the mutual relationships between authors and reviewers across different publications. Much of this can be mined only by teams of investigators who are carefully poring over journal databases. But publishers consider this type of information proprietary, so when irregu- larities are found and journals retract articles, they typically offer little detail. After all, these investigations expose weaknesses in their sys- tems and services. (That’s why the new breed of grass-roots watchdogs such as Retraction Watch and PubPeer are so important.) Given the increasing awareness of post- production misconduct— and how it under- mines the assessment of publicly funded research — funders, policymakers and the science com- munity should ask publishers to make available more of the information needed to investigate it. The community must realize that, unlike pre- vious fraudsters, from the unknown hoaxer who planted a mixture of bones in a British gravel pit at Piltdown to that of Paul Kammerer, who is blamed for inking features onto the feet of mid- wife toads to support Lamarckianism evolution, academic misconduct is no longer just about seeking attention. Many academic fraudsters aren’t aiming for a string of high-profile publications. That’s too risky. They want to produce — by plagiarism and rigging the peer-review system — publications that are near invisible, but can give them the kind of curriculum vitae that matches the performance metrics used by their academic institutions. They aim high, but not too high. And so do their institutions — typically not the world’s leading uni- versities, but those that are trying to break into the top rank. These are the institutions that use academic metrics most enthusiastically, and so end up encouraging post-production misconduct. The audit culture of universities — their love affair with metrics, impact factors, citation statistics and rankings — does not just incentivize this new form of bad behaviour. It enables it. ■ METRICS OF SCIENTIFIC EVALUATION ARE BOUND TO BE ABUSED. Mario Biagioli is a professor of law and of science and technology studies at the University of California, Davis. e-mail: mbiagioli@ucdavis.edu 1 4 J U LY 2 0 1 6 | V O L 5 3 5 | N AT U R E | 2 0 1