# Misreferencing practice of Scientists: Inside Researchers' Sociological and Bibliometric Profiles

Romy Sauvayre

<figure><img src="image_1.png" /></figure>

► To cite this version:

Romy Sauvayre. Misreferencing practice of Scientists: Inside Researchers' Sociological and Bibliometric Profiles. Social Epistemology, 2022, 10.1080/02691728.2021.2022807. hal-03480917

HAL Id: hal-03480917

https://hal.science/hal-03480917

Submitted on 6 May 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Misreferencing Practice of Scientists: Inside Researchers’ Sociological and Bibliometric Profiles

Romy Sauvayre

Université Clermont Auvergne, France.

Contact: Romy Sauvayre [romy.sauvayre@uca.fr](mailto:romy.sauvayre@uca.fr) Université Clermont Auvergne, CNRS, LAPSCO, F-63000 Clermont-Ferrand, France.

## Abstract

Scientists, as human beings, sometimes make mistakes. The aim of this case study was to examine the misreferencing practice of scientists. Citations of two documents about the neuroimaging of a dead salmon were collected. A total of 173 references were extracted from Google Scholar and analyzed by both qualitative and quantitative methods. The findings reveal a reference error rate of 93.1%, involving 419 authors. Sociological and bibliometric data about the authors were collected and revealed that referencing errors can be committed by all types of researchers. Further, a major referencing error was discovered in 22.5% of the citing documents, involving 121 authors, that is, the citation of a poster as if it were an article published in a prestigious journal. This major error was committed by the more prolific and recognized researchers. The higher the first author's number of citations, the more publications or h-index, the more major reference errors (citing a "ghost" paper) were committed. Nevertheless, the ScienceDirect database bears some responsibility, as it indexed abstracts of the *Organization for Human Brain Mapping* meeting as journal articles that had never been published in the prestigious *NeuroImage* journal. Finally, this case study raises epistemological concerns about research and its dissemination.

## Keywords

Questionable research practice; reference error; citation analysis; bibliographical databases; research ethics; peer-review system; publication pressure.

## Introduction

Reference accuracy reflects, on the one hand, the precision, ethics, and credibility of scientists (Wilks et al. 2017; Davies 2012) and, on the other hand, their knowledge of the literature (Davies 2012; Gilbert 1977). According to the literature (Wilks et al. 2017; Fenton et al. 2000; George and Robbins 1994; Evans, Nadjari, et Burchell 1990), reference errors should also involve the responsibility and credibility (Fenton et al. 2000) of both editors and reviewers. Even though reference accuracy is an integral part of scientific activity, this research practice, namely, committing reference errors, occurs regularly and may lead to misinformation (Granter, Laga, and Larson 2015). Reference accuracy entails a lack of citation errors ("inaccuracies in
the components of the cited reference" (Montenegro et al. 2021, 14) or the quotation errors
("inconsistency between the author's assertion and the content of its supporting source" (Montenegro et al. 2021, 14).

In studies on reference accuracy, the purpose is often to identify and count reference errors (i.e. citation errors). Such errors have been investigated in numerous research areas (Spivey and Wilks 2004; Davies 2012): medical sciences (biomedicine, biomedical informatics, orthopedics, general surgery, otolaryngology, emergency medicine, nursing, burns, anatomy, public health, and veterinary medicine); entomology; psychology (counseling psychology, and experimental psychology); social work; library and information science; business and economics; and clinical chemistry.<fnref n="1" /> Thus, the bibliographical material used is most often published in peer-reviewed journals (Speck and St. Pierre Schneider 2013) specializing in specific research areas (Davies 2012; Fenton et al. 2000; George and Robbins 1994; Montenegro et al. 2021). Rarely, the reference accuracy of a single highly cited journal article is examined (Liang, Zhong, and Rousseau 2014). By contrast, books and book chapters are often excluded from reference accuracy analyses. Several errors in the following parts of the citing reference are usually counted: the author (omitted name, incorrect name order, and misspelled name and initials), title, publication year, journal, volume, issue, and page numbers (e.g., (Wilks et al. 2017; Davies 2012; Spivey and Wilks 2004)). To summarize, the error rates reported in journal articles published between 1975 and 2017 (Wilks et al. 2017; Davies 2012; Spivey and Wilks 2004) are between 11% and 54%. The error rate may vary depending on the considered indicators and area of research. Therefore, data cannot be compared, all things being equal, and caution is when interpreting them. Nevertheless, these studies show that authors often incorrectly reference cited authors (45%), titles (25%), page numbers (16%), journals (6%), volumes and issues (6%), and publication years (4%). In addition, these studies suggest that inaccuracies may be introduced by manual transcription (Davies 2012) and can be reduced by using bibliographical software and databases (Davies 2012). However, these databases are not fully accurate. Indeed, the error rates reported in the literature are between 0.05% (Haddaway et al. 2015) and 6.6% (Franceschini, Maisano, and Mastrogiacomo 2016) for Web of Science (WoS); between 2.0% (Moed, Bar-Ilan, and Halevi 2016) and 12.4% (Valderrama-Zurián et al. 2015) for Scopus; between 2.9% (Haddaway et al. 2015) and 53.5% (Granter, Laga, and Larson 2015) for Google Scholar (GS); 1.25% for PubMed (Valderrama-Zurián et al. 2015); and 1.1% for PsycINFO (García-Pérez 2010).

# 1. Context and methods

The aim of this case study was to examine the misreferencing practice of scientists who cited a notorious, unusual work about the neuroimaging of a dead salmon (Bennett et al. 2009; 2010). Originally, Craig Bennett and colleagues wanted to calibrate a scanner in preparation for psychological experiments using functional magnetic resonance imaging (fMRI) at Dartmouth College (Bennett 2009). They used various objects to do this: a pumpkin, a Cornish game hen, and an Atlantic salmon. The data were recorded and stored at that time. Three years later, data analysis was conducted, revealing what appeared to be brain activity in this dead salmon. In June 2009, the data from the dead salmon experiment were presented as a poster at the 15th Congress of the Organization for Human Brain Mapping (OHBM poster) (Bennett et al. 2009). Thereafter, in 2010, the data were published in a peer-reviewed journal article, namely, in the

<footnote n="1">¹ Note that studies report results that are not strictly comparable because of the use of different criteria for identifying reference errors and calculating error rates.</footnote>
*Journal of Serendipitous and Unexpected Results*<fnref n="2" /> (JSUR article) (Bennett et al. 2010). Last, Bennett and colleagues received an Ig® Nobel Prize<fnref n="3" /> in 2012 for their dead salmon case study (OHBM poster and JSUR article). This neuroscience prize was awarded “for demonstrating that brain researchers, by using complicated instruments and simple statistics, can see meaningful brain activity anywhere — even in a dead salmon” (“Winners of the Ig® Nobel Prize” s. d.). Since 2009, this result has been used to illustrate the false-positive issue (type I error) that may occur in fMRI analysis and thus to alert the scientific community (Margulies 2012). Furthermore, this study became one of the few studies of neuroimaging methods to be cited more than a hundred times<fnref n="4" /> and to receive worldwide media coverage.

Usually, a neuroimaging study has a higher chance of being cited if it is published in a journal indexed by Journal Citation Reports (JCR), which provides impact factors. However, the dead salmon study was disseminated via a poster (OHBM poster) and a journal that existed only to publish a single article (JSUR article). Moreover, an indexing anomaly appears in the ScienceDirect and GS databases: the OHBM poster is referenced as an article published in a prestigious neuroimaging journal (*NeuroImage*). Without careful checking, this error is not easy to identify. The aim is, in the present study, to understand how citing authors deal with major difficulties in finding information to reference the dead salmon study accurately and thus highlight their reference practices, their rigor and their ethical values. First, a comprehensive analysis was performed to define the types of errors committed by researchers<fnref n="5" />; a broad category of document types was considered (journal articles, books, books chapter, and conference proceedings). Second, a specific analysis was conducted among publications in which an article that does not exist has been cited; that is, those in which a “ghost paper citation error” has been committed. A sociological profile of the citing authors was constructed, and the rationale for their particular misreferencing practice was analyzed. Third, four hypotheses were proposed to understand and explain this particular misreferencing practice involving a well-known bibliographical database (ScienceDirect) and how these authors’ referencing errors and this database anomaly were possible.

Consequently, the present study aimed to examine publications that cited the OHBM poster or JSUR article (based on the OHBM poster), namely, the “Dead Salmon fMRI References” (DSfR).

These citation data were extracted from GS via the Publish or Perish (PoP) software on October 6, 2017 (for more methodological details, please see the supplemental material document). A total of 281 references were examined. After a manual data cleaning process, 173 unique citing publications were included in the analysis. Thereafter, the following supplementary data were manually collected and analyzed to explore the sociological and bibliometric profile of citing

<footnote n="2">² Note that this journal had a short editing life: it published only this article. This journal probably existed, but it is currently not possible to determine this with certainty. The founders of the journal did not respond to our requests for comment, but the authors (Bennett and colleagues) list this publication in their CVs.</footnote>

<footnote n="3">³ The Ig Nobel Prizes are “the wayward son of the more righteous Nobels” (Pilcher 2004). The ceremony has been organized by the *Annals of Improbable Research* magazine at Harvard University since 1991: “The Ig Nobel Prizes honor achievements that make people LAUGH, then THINK. The prizes are intended to celebrate the unusual, honor the imaginative — and spur people’s interest in science, medicine, and technology” (“About The Ig® Nobel Prizes” s. d.). Each year, 10 laureates are selected from among 9,000 candidates (“Ig Nobel Nominations” s. d.). This unusual prize is “Increasingly covered by the media” (Gingras and Vécrin 2010, 67), thus greatly popularizing the research of the laureates.</footnote>

<footnote n="4">⁴ Only 4.12% (53/1287) of journal articles published on reliability and fMRI have at least 100 citations (source: WoS All databases; timespan: 2009-2019).</footnote>

<footnote n="5">⁵ For more detail about the reference errors considered, see the supplemental material, section “1.3 Identification of reference accuracy inconsistencies”.</footnote>
authors, mainly with nonparametric statistical tests: 1) Concerning the citing documents: type of DSfR cited, number of authors, publication year, publication type, journal impact factor from Journal Citation Reports (JCR); 2) Concerning the first authors of these citing documents: gender, academic position, research area, field of study or department, institutional affiliation, institutional Shanghai ranking, country, number of citations in WoS, number of publications in WoS, and h-index in WoS. Lastly, the accuracy of DSfR cited in each document reference list was examined, and citation errors were categorized and counted.

## 2. Results

Reference errors were categorized into two types: 1) errors usually found in the reference accuracy literature (see above) and 2) a new major error never presented before.

### 2.1. Usual reference errors

The reference errors found in the present study were categorized into 10 elements: title, year, authors, journal or meeting name, volume or issue number, page numbers, “poster” mention, state, city, and month.

A total of 631 reference errors were counted (Table 1), with a median of 3 errors per citing document.<fnref n="6" /> Furthermore, 161 citing documents (93.1%) involving 419 authors contained at least one error. Only 12 publications (6.9%) were accurate. This error rate is so unusually high that an additional data check was conducted to ensure the precision of the results. The most common error is incorrect volume or issue numbers (62.1% of citing documents)<fnref n="7" />.

1. **Author error.** The OHBM poster reference was mainly found to be affected by this type of error (70.0%), and the second author, Abigail Baird, was particularly affected. Although Baird is the coauthor of both DSfR, she was often omitted. In fact, neither the poster session program, which recorded this OHBM poster reference, nor the ScienceDirect database, which still indexes this poster program, makes mention of this coauthor. Thus, the omission of this author may have contributed to the improper referencing. Last, two other types of author errors rarely occurred: inversion of the first and last names of the fourth author, and replacement of the second, third, and fourth authors by “et al.”

2. **Year error.** The JSUR article reference was mainly found to be affected by this type of error (89.7%). Note that it is not easy to find this JSUR publication year (i.e., 2010) because it did not appear in the full text of the article downloadable from the first author's blog (http://prefrontal.org/files/papers/Bennett-Salmon-2010.pdf). Moreover, bibliographical databases have not indexed this article, and the journal and its website no longer exist. Consequently, authors made incorrect choices: from 2005 (-5 years) to 2013 (+3 years). Furthermore, among the miscited years, 2011 was the most frequent.

3. **Title error.** Authors mainly omitted part of the DSfR title (74.2%) or added an irrelevant part (19.4%). This error seems to have been caused by a strong similarity between the two DSfR titles; indeed, only a single word (“proper”) in the second part of the title differs. In addition, the JSUR article title was misreferenced more often (75.0%) than the OHBM title.

<footnote n="6">⁶ The median was used because this measure of central tendency is more appropriate than the mean in skewed distributions affected by extreme values.</footnote>

<footnote n="7">⁷ See complete results in the supplemental material, table S2.</footnote>
4. *Journal or meeting name error.* The OHBM poster reference was mainly found to be affected by this type of error (67.8%). These names were incorrect, incomplete or missing, and they also contained some capitalization and abbreviation errors.

5. *Volume or issue error.* The JSUR article reference was mainly found to be affected by this type of error (62.0%). The most common error was the omission of either the volume or the issue number. Note that this number is the same for both (i.e., “1”) because the journal published a unique article in a unique volume. In other cases, the journal issue was replaced by an irrelevant digital object identifier (DOI) or by an irrelevant page number (i.e., “125”).

6. *Page error.* The OHBM poster reference was found to be almost universally affected by this type of error (90.0%). The accurate page number was replaced by that of the OHBM program (i.e., “S125”), which contained the schedules of the poster presentations. In other cases, a double page was referenced (i.e., “S125-125”). In addition, in a single case, another part of the program was referenced (i.e., “S39-S41”).

7. *Poster error.* The OHBM poster reference was found to be almost universally affected by this type of error (97.1%). The word “poster” was often missing and was, in rare instances, replaced by the word “paper.”

8. *State error*; 9. *City error*; 10. *Month error*. These parts of the reference were missing mainly in the OHBM-citing documents (96.9%, 98.3%, and 97.2%, respectively). Rarely, the following replacements were detected: July instead of June and Chicago or Santa Barbara instead of San Francisco.

In addition, there was a significant difference in the number of errors between the two types of DSfR cited. The chances of reference errors were 9 times higher in documents citing the OHBM poster than in those citing the JSUR article (odds ratio = 8.899, 95% confidence interval = 1.12-70.56, N = 173). Furthermore, considering both the number of errors and the number of publications that contain at least one error, a significant difference was found between documents citing the OHBM poster reference and those citing the JSUR article reference (U = 537.5, p < 0.001, N = 173; U = 3201, p = 0.006, N = 173, respectively).

After the presentation of these 10 reference error types, it may be interesting to explore the sociological and bibliometric profiles of the citing authors. Therefore, it is reasonable to suppose—or expect—that the more experienced or cited the researcher is, the more accurate his or her referencing will be. However, the data revealed no significant difference or correlation between the number of errors and several sociological and bibliometric indicators, such as academic position, university ranking, journal impact factor, first author's h-index, and first author's number of citations<fnref n="8" />. In other words, all types of researchers are likely to commit at least one reference error.

## 2.2 A major reference error: the “ghost paper citation error”

The data on these 10 error types demonstrate that researchers committed numerous reference errors, but these results are very typical, as previous studies demonstrate. However, one of these error types reveals more about research practice and may explain the significant difference in error rate between the two DSfR cited. In fact, the data contain an error never reported before in the reference accuracy literature, that is, the replacement of the OHBM poster reference with a reference to an irrelevant article reportedly published in a prestigious neuroscience journal. Two journals are affected by this error: *Human Brain Mapping* and *NeuroImage*. In other

<footnote n="8">⁸ See the complete results in the supplemental material, table S10.</footnote>
words, neither *Human Brain Mapping* nor *NeuroImage* published any journal article written by Bennett and colleagues and entitled “Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon”. The coauthors of the OHBM poster do not include any supposed *Human Brain Mapping* or *NeuroImage* journal article about the dead salmon neuroimaging work in their CV publication lists. *Those two references refer to journal articles that do not exist.* In other words, citing authors committed a “ghost paper citation error.”

This type of major and questionable error was found in 39 publications (22.5%) involving 121 authors. How is that possible? The first—the Human Brain Mapping ghost paper (N = 1)—was a journal name error probably caused by its strong similarity to the meeting name, that is, “Organization for Human Brain Mapping.” The second—the NeuroImage ghost paper (N = 38)—is indexed by the ScienceDirect database (produced by Elsevier since 1997). This database has also provided a full webpage for the NeuroImage ghost paper since August 28, 2009, which has been visited more than one thousand times (https://www.sciencedirect.com/science/article/pii/S1053811909712029?via%3Dihub). This ScienceDirect webpage contains the OHBM poster title, three of the four authors, a detailed summary, a DOI (10.1016/S1053-8119(09)71202-9), a copyright, an “Article Metrics” section, and two downloadable documents available in PDF format to authorized users only. In fact, these documents are an extract of the OHBM meeting program (pages S39-S41 and page S125) that were published in a supplemental issue of *NeuroImage*. In place of articles, these PDFs provide the dates, times, places, titles, and author names of the poster sessions, including the poster of Bennett and colleagues. However, the following reference, which resembles a journal article, can then be recorded from ScienceDirect:

"Bennett C. M., M. B. Miller, G. L. Wolford, "Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: an argument for multiple comparisons correction." *NeuroImage* 47: S125 (2009). doi:10.1016/S1053-8119(09)71202-9."

However, a simple action would have avoided this major citation error. Thus, opening and reading one of the two downloadable documents would prevent this ghost paper citation error because of its content.

In addition, because ScienceDirect indexes the NeuroImage ghost paper using the same structure applied for journal articles<fnref n="9" />, GS includes this false reference in its database. More importantly, the results obtained from GS through a title search (with or without a Bennett mention) show that only the reference to the NeuroImage ghost paper is available, unlike the OHBM poster<fnref n="10" />. Therefore, the only visible and available reference in both ScienceDirect and GS is the false reference. Nevertheless, GS provides for free the full text of the OHBM poster (https://www.psychology.mcmaster.ca/bennett/psy710/readings/BennettDeadSalmon.pdf). However, again, the opening and reading of this PDF would reveal to authors that this document cannot be a *NeuroImage* journal article, but is instead a meeting program. Therefore, it is unlikely that the authors had read the OHBM poster before citing the NeuroImage ghost paper.

This surprising fact raises questions about the types of researchers who committed this ghost paper citation error. For this purpose, the collected variables described below were analyzed to

<footnote n="9">⁹ See the supplemental material, figure S2.</footnote>

<footnote n="10">¹⁰ See the supplemental material, figure S3.</footnote>
define the sociological characteristics of those scientists (highest frequencies per variable are
enclosed in brackets below)<fnref n="11" />.

Thus, the citing document was found to be an article (59.0%) published in a journal with a median impact factor (IF) (73.9%) of 3.8. The 20 journals that published this citing document included the prestigious *Nature Reviews Neuroscience* (1st neuroscience journal, 2017 JCR, IF = 32.6) and, unexpectedly, *NeuroImage* itself. For instance, *NeuroImage* published, in 2015 and 2016, two articles that referenced the NeuroImage ghost paper. Furthermore, the citing publication was written by a median of two coauthors who committed a median of 8 reference errors on the DSfR cited. The first author was most often a man (73.7%) affiliated (100.0%) with a top 100-ranked university (41.0%). This university was most frequently American (48.7%), such as the Massachusetts Institute of Technology, Stanford University, or Northwestern University. The first author was a junior researcher (e.g., Assistant Professor) (58.3%) in social sciences (56.3%), mainly in psychology (34.2%). He or she had accumulated a median of 280 citations (WoS), 27 publications (WoS), and an h-index (WoS) of 8.0¹². Finally, correlations between these indicators of scientific productivity and recognition and the type of citation error were statistically significant (Fig. 1). In other words, the higher the first author's number of citations (r<sub>s</sub> = 0.199, p = 0.011), publications (r<sub>s</sub> = 0.226, p = 0.004) or h-index (r<sub>s</sub> = 0.238, p = 0.002) were, the more major reference errors (a ghost paper citation error) were committed.

<footnote n="11">¹¹ Note that the complete aggregated data are available in the supplemental material, Table S3 (cf. the column entitled “Documents with ghost paper citation errors”).</footnote>

<footnote n="12">¹² See the comparative descriptive statistics in the supplemental material in table S3 and the complete list of publications publishing documents that cite at least one DSfR in table S9.</footnote>
<figure><img src="image_3.png" /><figcaption>**Figure 1. Reference errors as a function of the first author's bibliometric indicators.** (A) First author's number of citations (WoS) by the type of citation error (r<sub>s</sub> = 0.199, p = 0.011). (B) First author's number of publications (WoS) by the type of citation error (r<sub>s</sub> = 0.226, p = 0.004). (C) First author's h-index (WoS) by the type of citation error (r<sub>s</sub> = 0.238, p = 0.002). The following type of citation was analyzed as an ordinal classification: 1) no citation error; 2) usual citation error, that is, the main types of reference inaccuracies described in the literature (see above the 10 types of errors categorized in the present study); and 3) ghost paper citation error, that is, as defined above, referencing an article that does not exist. \* p < 0.05, \*\* p < 0.01.</figcaption></figure>

The results showed that a prolific or recognized researcher was more likely to misreference a DSfR and especially to commit a ghost paper citation error. Therefore, all affiliated researchers
may be prone to committing a ghost paper citation error, but prolific and recognized researchers
are more likely to commit such an error than others.

**3. Discussion**

One may think that referencing errors are a neglected aspect of questionable scientific activity, but they constitute a unique objective and measurable indicator of the rigor in practice and ethical values of scientists. In sum, citation practice provides valuable information about researchers. However, some reference errors have different consequences: some may seem inconsequential, such as using the wrong month for a conference meeting or making capitalization errors in journal names. Other reference errors cause citing authors to have difficulty accessing the documents, such as citing an article instead of a poster, citing the wrong journal name or the wrong date. This present case study, because of its unusual nature, questions an epistemological problem involving, in particular, the dissemination of knowledge.

The results presented reveal that 93.1% of citing documents contained at least one reference error. This error rate is substantially higher than in previous findings, where the error rate lies between 11% and 54% (Davies 2012; Spivey and Wilks 2004; Wilks et al. 2017). This difference can be explained by 1) the similarity between the titles of the OHBM poster and JSUR article (only one word was added in the latter title); 2) the fact that the JSUR article was published in a journal with a very short life because it published only one article before disappearing; 3) the indexing anomaly of the OHBM poster in both ScienceDirect and GS; 4) the ghost paper citation error detected in 22.5% of citing documents, in which the OHBM poster reference was replaced with a paper that does not exist.

The last of these four causes—the ghost paper citation error—is the most striking and a new reference error detected in this study involving 39 publications, 20 academic journals, 12 books, 34 institutions or universities and 123 coauthors. Last, although numerous thorough studies have been conducted in this field, none of them have reported the citation of references that do not exist (Wilks et al. 2017; Davies 2012; Spivey and Wilks 2004). How can one explain why this major error appeared, was published and was disseminated in the peer-review system? Four hypothetical explanations are discussed: 1) confidence in bibliographical databases; 2) ethical failures on the part of researchers who do not read or verify their cited references; 3) the absence of a control policy; and 4) publication pressure. Note that these four “explanatory hypotheses” produced by “abduction” (Peirce 1998, CP 5.171)<fnref n="13" /> are intended to bring a reflexive point of view to the epistemological problems raised by the present study about science practice and the dissemination of knowledge, that must be confronted with data beyond that in the present case study.

1) *Confidence in bibliographical databases.* Two trusted sources—ScienceDirect and GS—index the NeuroImage ghost paper. The first, ScienceDirect, provides “peer-reviewed scholarly literature” (“What is ScienceDirect” s. d.), including “articles from over 3,800 journals and more than 37,000 book titles” (“What is ScienceDirect” s. d.). Note that ScienceDirect makes no mention that it is providing poster abstracts, in addition to journal articles and books, in its database. As mentioned above, the complete ScienceDirect webpage describes the NeuroImage ghost paper with standard elements that also contributed to strengthening the reliability of those data. In fact, OHBM poster abstracts are presented with the same webpage structure as are

<footnote n="13">¹³ “Abduction is the process of forming an explanatory hypothesis. It is the only logical operation which introduces any new idea [...]. Its only justification is that from its suggestion deduction can draw a prediction which can be tested by induction, and that, if we are ever to learn anything or to understand phenomena at all, it must be by abduction that this is to be brought about” (Peirce 1998, CP 5.171).</footnote>
*NeuroImage* journal articles. Therefore, researchers had no reason to doubt the ScienceDirect bibliographical information, which is expected—and claimed—to provide the full text of journal articles. The second, GS, indexes the ScienceDirect webpage and, as a result, the NeuroImage ghost paper. However, GS has relegated the OHBM poster reference to the “version” subcategory. In the GS search results, the accurate reference (OHBM poster) was replaced by that for the ghost paper. Therefore, only the two following references could be obtained from GS: the JSUR article (163 citations) and the NeuroImage ghost paper (132 citations). Nevertheless, again, researchers had no reason to doubt the GS bibliographical information. Consequently, ScienceDirect should remove the NeuroImage ghost paper indexation and the 2,000 other OHBM posters indexed each year from its website or should at least change the presentation of these communication poster abstracts to prevent them from being confused with journal articles.

2) *The ethical failure of researchers.* Part of ethical research practice requires that researchers read their sources before citing them and subsequently verify their references. However, several studies have suggested that researchers do not do so (Evans, Nadjari, and Burchell 1990; Liang, Zhong, and Rousseau 2014; Simkin and Roychowdhury 2003; Mogull 2017). Though the present case study cannot illustrate the reading practice of citing authors, the probability of unethical conduct is high enough to hypothesize that several authors did not read the OHBM poster before citing the NeuroImage ghost paper or did not complete the necessary checks to avoid this type of major error. Furthermore, according to Mogull (2017), the major errors counted “might suggest that authors are not reading” or “suggest a deliberate attempt by authors to mislead readers”. If authors had read even one of the documents attached to the NeuroImage ghost paper reference, then they would have discovered either an OHBM meeting program (downloadable from ScienceDirect) or the OHBM poster (downloadable from several coauthors’ websites, including GS). Consequently, authors who cited the NeuroImage ghost paper probably did not read it. In addition, the considerable reference error rate (93.1%) found in the present study also suggests that researchers did not sufficiently verify their reference list.

3) *The absence of a control policy.* Concerning reference inaccuracy, the literature (Wilks et al. 2017; Fenton et al. 2000; George and Robbins 1994) has suggested for the last 25 years (Evans, Nadjari, and Burchell 1990) that responsibility should be shared by authors, reviewers, and editors. According to Franceschini and colleagues (2015), errors persist despite improvements. Indeed, the present study showed that 39 documents (22.5% of the citing documents) were published despite citing a ghost paper. Among these documents, 23 manuscript drafts were examined by reviewers and editors, who failed to detect the ghost paper reference. Furthermore, *NeuroImage* itself published two articles that cited the NeuroImage ghost paper. Nevertheless, it is important to mention that editors rely on the ethicality of their authors, and they do not require reviewers to examine the accuracy of manuscript reference lists. Neither editors nor reviewers could be reasonably expected to have identified this ghost paper citation error. However, the current academic publication system, which is based in part on the ethical practices of researchers, is imperfect, as indicated by the publication and diffusion of citations of retracted articles (Madlock-Brown and Eichmann 2015; Bar-Ilan and Halevi 2017; Wray and Andersen 2018) and citations of the present ghost paper. Furthermore, in the absence of any control policy for references, the peer review process tacitly validated the ghost paper reference and then increased readers’ confidence in this erroneous citation. Subsequently, the ghost paper may have been cited by these readers as well causing them to unintentionally, contribute to the diffusion of the ghost paper citation error and hence to the vicious cycle of misreferencing. Indeed, as Figure 2 shows, the ghost paper citation error has been committed each year since 2010 and may be perpetuated in the next few years as well.
<figure><img src="image_4.png" /><figcaption>Figure 2. Number of citing documents by publication year (between 2009 and 2017) and by type of citation error (including a cumulative frequency curve).</figcaption></figure>

4) *Publication pressure*. The literature (Fenton et al. 2000; Begley and Ellis 2012; Hicks et al. 2015) suggests that the professional survival of researchers—in terms of tenure, academic advancement, and funding—may depend on citation analysis. Consequently, it is tempting for researchers to publish their findings as quickly as possible but at the expense of quality or rigor. Some researchers may therefore avoid the considerably time-consuming task of reading their citing sources and verifying their reference list. The present study showed that nearly all the researchers who cited the dead salmon study seemed to make this choice. In contrast, few of the researchers (6.9%) properly referenced DSfR, and they were less prolific and less cited than those who did not. However, this case study does not permit us to generalize this hypothesis. Nevertheless, the data show a significant correlative link between the number of reference errors and the academic productivity of the citing authors (number of publications).

Thus, the present study pursues further questions: 1) in a “publish or perish” context, do researchers adopt questionable reference practices rather than rigorous ones because prolific researchers are more highly valued? 2) As a consequence, is publication pressure—caused by the current research evaluation process—partially responsible for the increase in researchers’ misreferencing practice? Finally, an epistemological consequence may be highlighted with reference to the ghost paper citation error, which is leading readers to assign higher epistemological value to the dead salmon study and increasing its dissemination. The more prestigious the journal (the higher the impact factor) in which an article is published, the more the article will tend to be cited (Callaham, Wears, and Weber 2002). The dead salmon case study appears to have been published in a prestigious journal after having been evaluated by experts in the neuroimaging field. Considering that, as Gilbert (1977) reports, a citation gives scientific support to and implicitly presents the results as “valid”, misreferencing may lead to the dissemination and validation of a case study that had been rejected by prestigious neuroimaging journals. The present study may contribute to research in the citation analysis field (Nicolaisen 2007).
**Conclusion**

In conclusion, the present case study showed that nearly all the researchers examined misreferenced work based on fMRI data obtained from a dead salmon. These researchers included junior and senior researchers who were highly and less highly cited, affiliated with an average-ranked or well-ranked university, and publishing papers in unranked and prestigious academic journals. However, a major reference error, namely, referencing a poster as if it were an article published in a prestigious journal, was committed by the more prolific and cited researchers. Therefore, scientific excellence does not seem to prevent reference errors. Finally, this case invites us to epistemologically question the practice of research and its dissemination based on an activity that seems innocuous (referencing) and that nevertheless speaks volumes.

**Conflict of Interest**

The author declares no competing interests.

**Funding**

This study was conducted without funding.

**Data Availability**

All relevant data are present in the manuscript and supplemental material. However, restrictions apply to the availability of the bibliometric data, which is used under license from Thomson Reuters. Readers can contact Thomson Reuters at the following URL: http://thomsonreuters.com/en/products-services/scholarly-scientific-research/scholarly-search-and-discovery/web-of-science.html.

**Notes on contributor**

**Romy Sauvayre** (PhD, HDR) is Maîtresse de conférences (Associate Professor) in sociology at the engineering school Polytech Clermont (Clermont Auvergne INP, Clermont Auvergne University, France). Her current work focuses on various fields such as epistemology, contemporary history of science, ethics of belief, belief change, sociology of diffusion, sociological methods.

**ORCID**

Romy Sauvayre [https://orcid.org/0000-0003-0806-6234](https://orcid.org/0000-0003-0806-6234)

**LinkedIn**

Romy Sauvayre [https://www.linkedin.com/in/romysauvayre/](https://www.linkedin.com/in/romysauvayre/)
## References

“About The Ig® Nobel Prizes.” 2021. *Improbable Research*. Accessed February 16. https://www.improbable.com/ig/.

Bar-Ilan, J., and G. Halevi. 2017. "Post Retraction Citations in Context: A Case Study." *Scientometrics* 113 (1): 547–565. doi:10.1007/s11192-017-2242-0.

Begley, C. G., and L. M. Ellis. 2012. "Raise standards for preclinical cancer research." *Nature* 483 (7391): 531–533. doi:10.1038/483531a.

Bennett, C. M. 2009. "The Story Behind the Atlantic Salmon." *Prefrontal.Org*. A Personal Weblog of *Developmental Cognitive Neuroscience*. September 18. http://prefrontal.org/blog/2009/09/the-story-behind-the-atlantic-salmon/.

Bennett, C. M., A. A. Baird, M. B. Miller, and G. L. Wolford. 2009. "Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon: An Argument for Multiple Comparisons Correction." Poster presented at the 15th Annual Meeting of the Organization for Human Brain Mapping, San Francisco, June 20. http://prefrontal.org/blog/2009/06/human-brain-mapping-2009-presentations/.

Bennett, C. M., A. A. Baird, M. B. Miller, and G. L. Wolford. 2010. "Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon: An Argument For Proper Multiple Comparisons Correction." *Journal of Serendipitous and Unexpected Results* 1 (1): 1–5. http://prefrontal.org/files/papers/Bennett-Salmon-2010.pdf.

Callaham, M., R. L. Wears, and E. Weber. 2002. "Journal Prestige, Publication Bias, and Other Characteristics Associated With Citation of Published Studies in Peer-Reviewed Journals." *Journal of the American Medical Association* 287 (21): 2847–2850. doi:10.1001/jama.287.21.2847.

Davies, K. 2012. "Reference Accuracy in Library and Information Science Journals." *Aslib Proceedings* 64 (4): 373-87. doi:10.1108/00012531211244734.

Evans, J. T, H. I. Nadjari, and S. A. Burchell. 1990. "Quotational and Reference Accuracy in Surgical Journals: A Continuing Peer-Review Problem." *Journal of the American Medical Association* 263 (10): 1353–1354. doi:10.1001/jama.263.10.1353.

Fenton, J. E., H. Brazier, A. de Souza, J. P. Hughes, and D. P. McShane. 2000. "The Accuracy of Citation and Quotation in Otolaryngology/Head and Neck Surgery Journals." *Clinical Otolaryngology & Allied Sciences* 25 (1): 40–44. doi:10.1046/j.1365-2273.2000.00322.x.

Franceschini, F., D. Maisano, and L. Mastrogiacomo. 2015. "Research Quality Evaluation: Comparing Citation Counts Considering Bibliometric Database Errors." *Quality & Quantity* 49 (1): 155–165. doi:10.1007/s11135-013-9979-1.

Franceschini, F., D. Maisano, and L. Mastrogiacomo. 2016. "Empirical analysis and classification of database errors in Scopus and Web of Science." *Journal of Informetrics* 10 (4): 933–953. doi:10.1016/j.joi.2016.07.003.

García-Pérez, M. A. 2010. "Accuracy and Completeness of Publication and Citation Records in the Web of Science, PsycINFO, and Google Scholar: A Case Study for the Computation of h Indices in Psychology." *Journal of the American Society for Information Science and Technology* 61 (10): 2070–2085. doi:10.1002/asi.21372.
George, P. M., and K. Robbins. 1994. "Reference Accuracy in the Dermatological Literature." *Journal of the American Academy of Dermatology* 31 (1): 61–64. doi:10.1016/S0190-9622(94)70136-9.

Gilbert, N. G. 1977. "Referencing as Persuasion." *Social Studies of Science* 7 (1): 113–122. doi:10.1177/030631277700700112.

Gingras, Y., and L. Vécrin. 2010. "Les prix Ig-Nobel: Le double tranchant de l'humour scientifique." *Actes de la recherche en sciences sociales* 141–142: 66–71.

Granter, S. R., A. C. Laga, and A. R. Larson. 2015. "Calciphylaxis and the Persistence of Medical Misinformation in the Era of Google." *American Journal of Clinical Pathology* 144 (3): 427–431. doi:10.1309/AJCPDMWVGKW9N1CU.

Haddaway, N. R., A. M. Collins, D. Coughlin, and S. Kirk. 2015. "The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching." *PLoS ONE* 10 (9): e0138237. doi:10.1371/journal.pone.0138237.

Hicks, D., P. Wouters, L. Waltman, S. De Rijcke, and I. Rafols. 2015. "Bibliometrics: The Leiden Manifesto for research metrics." *Nature* 520 (7548): 429–431. doi:10.1038/520429a.

"Ig Nobel Nominations." 2021. *Improbable Research*. Accessed February 16. https://www.improbable.com/ig/miscellaneous/nominate.html.

Liang, L., Z. Zhong, and R. Rousseau. 2014. "Scientists’ Referencing (Mis)Behavior Revealed by the Dissemination Network of Referencing Errors." *Scientometrics* 101 (3): 1973–1986. doi:10.1007/s11192-014-1275-x.

Madlock-Brown, C. R., and D. Eichmann. 2015. "The (Lack of) Impact of Retraction on Citation Networks." *Science and Engineering Ethics* 21 (1): 127–137. doi:10.1007/s11948-014-9532-1.

Margulies, D. S. 2012. "The Salmon of Doubt. Six Months of Methodological Controversy within Social Neuroscience." In Critical Neuroscience. A Handbook of the Social and Cultural Contexts of Neuroscience, edited by S. Choudhury and J. Slaby, 273–285. Oxford: Wiley-Blackwell.

Moed, H. F., J. Bar-Ilan, and G. Halevi. 2016. "A new methodology for comparing Google Scholar and Scopus." *Journal of Informetrics* 10 (2): 533–551. doi:10.1016/j.joi.2016.04.017.

Mogull, S. A. 2017. "Accuracy of Cited "Facts" in Medical Research Articles: A Review of Study Methodology and Recalculation of Quotation Error Rate." *PLoS ONE* 12 (9): e0184727. doi:10.1371/journal.pone.0184727.

Montenegro, T. S., K. Hines, G. A. Gonzalez, U. Fatema, P. P. Partyka, S. Thalheimer, and J. Harrop. 2021. "How Accurate Is the Neurosurgery Literature? A Review of References." *Acta Neurochirurgica* 163: 13–18. doi:10.1007/s00701-020-04576-3.

Nicolaisen, Jeppe. 2007. "Citation Analysis." *Annual Review of Information Science and Technology* 41 (1): 609–641. doi:10.1002/aris.2007.1440410120.

Peirce, C. S. 1998. *Collected papers of Charles Sanders Peirce. Vol. 5. Pragmatism and Pragmaticism*. Bristol: Thoemmes Press.

Pilcher, H. 2004. "Laughter in the Lab." *Nature*. doi:10.1038/news040927-20.
Simkin, M. V., and V. P. Roychowdhury. 2003. "Read Before You Cite!" *Complex Systems* 14 (3): 269–274.

Speck, K. E., and B. St. Pierre Schneider. 2013. "Effectiveness of a Reference Accuracy Strategy for Peer-Reviewed Journal Articles." *Nurse Educator* 38 (6): 265–268. doi:10.1097/01.NNE.0000435272.47774.51.

Spivey, C. A., and S. E. Wilks. 2004. "Reference List Accuracy in Social Work Journals." *Research on Social Work Practice* 14 (4): 281–286. doi:10.1177/1049731503262131.

Valderrama-Zurián, J.-C., R. Aguilar-Moya, D. Melero-Fuentes, and R. Aleixandre-Benavent. 2015. "A systematic analysis of duplicate records in Scopus." *Journal of Informetrics* 9 (3): 570–576. doi:10.1016/j.joi.2015.05.002.

"What is ScienceDirect." s. d. Elsevier. Accessed January 6 2018. https://www.elsevier.com/solutions/sciencedirect.

Wilks, S. E., J. R. Geiger, S. M. Bates, and A. L. Wright. 2017. "Reference Accuracy Among Research Articles Published in Research on Social Work Practice." *Research on Social Work Practice* 27 (7): 813–817. doi:10.1177/1049731515626802.

"Winners of the Ig® Nobel Prize." 2021. *Improbable Research*. Accessed February 16. https://www.improbable.com/ig/winners/?#ig2012.