DOI: 10.5281/zenodo.13745904
title: Generative AI aids the publication of fake articles: Methods and replication package

description: This package contains Python, shell, awk scripts, and data used to obtain the curated table associated with the above named letter. It also contains (in this file) a description of the methods employed to obtain the curated table with details regarding the published articles.

Contents

The following items are included.



README.md: This file

Makefile: Orchestrate the articles’ analysis

article-details.xlsx: Curated table with details of published articles.

get-metadata.sh: Obtain article metadata pages from the journal’s web site

apply-to-pdfs.sh: Apply the specified Python script to all article PDFs

extract-citations-emails.py: Extract number of probable in-text citations and corresponding author email from article PDF

extract-doi-affiliations.py: Extract article DOI and affiliations from an article’s metadata

extract-all-doi-affiliations.sh: Extract article DOI and affiliations from all articles’ metadata

emails-to-csv.awk: Convert emails and article numbers to CSV with URL for sending emails

global-us/metadata/: Article metadata as HTML files

global-us/global-us.mellbaou.com/index.php/global/article/download/: Article PDFs as files with no suffix
description: Methods

Data were gathered on 10 and 11 September 2024 on a host running an Anaconda Python environment version 1.12.3 and Cygwin bash version 5.2.15(3). The journal site global-us.mellbaou.com was completely crawled with the wget command in order to obtain the article PDFs. Article metadata were retrieved separately using the get-metadata.sh shell script. Citations and contact emails were extracted from the article PDF files with the extract-citations-emails.py and apply-to-pdfs.sh scripts. Article DOIs and author affiliations were extracted from the article metadata HTML files with the extract-doi-affiliations.py and extract-all-doi-affiliations.sh scripts. The two results were joined based on the article number key and used to create the initial version of the article-details.xlsx Excel file. A list of contact author emails and URLs was created with emails-to-csv.awk, and then used to inform article authors regarding the findings. Articles with a low citation count heuristic (measured through the number brackets and braces appearing before the article’s Reference section) were manually inspected for signs of entirely AI authorship (mainly formulaic content, lack of citations, tables, and figures).

The Excel document was hand-curated as follows.



Contact emails were obfuscated to comply with personal data protection regulations.

Fifty documents with a low citation count and ten with a high one were hand-verified regarding their AI content.

Email domains were extracted from emails and listed in a separate column

A column with undeliverable emails was added and hand-filled based on failed delivery reports regarding the sent notification emails.

Affiliations of authors of publications that were unlikely to have been submitted by them (evidenced by wrong contact emails or a personal communication) were marked in bold.

Notes were added with email communications and other provenance details (e.g. a publication by deceased Texas A&M professor) were added to substantiate the preceding actions.