TITLE: Coverage of DOAJ journals&#39; citations through OpenCitations - Protocol
AUTHORS: Constance  Dami, Alessandro  Bertozzi, Chiara   Manca, Umut  Kucuk

[DESCRIPTION]
This is the protocol for the research of the coverage of DOAJ journals' citations through OpenCitations. 

Our goal is to find out:
about the coverage of articles from open access journals in DOAJ journals as citing and cited articles,
how many citations do DOAJ journals receive and do, and how many of these citations involve open access articles as both citing and cited entities,
as well as the presence of trends over time of the availability of citations involving articles published in open access journals in DOAJ journals.

Our research focuses on DOAJ journals exclusively, using OpenCitations as a tool. Previous research has been made on open citations using COCI (Heibi, Peroni & Shotton 2019), and on DOAJ journals' citations (Saadat and Shabani 2012), paving the grounds for our present analysis. 

After careful considerations on the best way to retrieve data from DOAJ and OpenCitations, we opted for downloading the public data dumps. Using the API resulted in a way too long running time, and the same problem arose for using the SPARQL endpoint of OpenCitations. 


Minimal Bibliography

Björk, B.-C.; Kanto-Karvonen, S.; Harviainen, J.T. "How Frequently Are Articles in Predatory Open Access Journals Cited." Publications, 8, 17. (2020) https://doi.org/10.3390/publications8020017

Heibi, I.; Peroni, S.; Shotton, D. "Crowdsourcing open citations with CROCI -- An analysis of the current status of open citations, and a proposal" arXiv:1902.02534 (2019) https://doi.org/10.48550/arXiv.1902.02534

Pandita, R., & Singh, S. "A Study of Distribution and Growth of Open Access         Research Journals Across the World. Publishing Research Quarterly" (2022), 38(1), 131–149. https://doi.org/10.1007/s12109-022-09860-x

Saadat, R., A. Shabani. "Investigating the citations received by journals of Directory of Open Access Journals from ISI Web of Science’s articles." International Journal of Information Science and Management (IJISM) 9.1 (2012): 57-74.

Solomon, D. J., Laakso, M., Björk, B.-C. "A longitudinal comparison of citation rates and growth among open access journals", Journal of Informetrics, 7, 3 (2013): 642-650. https://doi.org/10.1016/j.joi.2013.03.008.





[BEFORE_START]
Make sure to have Python 3.9 installed on your device.
All the dependecies of the script can be installed using the requirments.txt file stored into the github repository.

Computer technical specifications:
CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz   2.59 GHz
RAM: 20,0 GB (19,9 GB usable) 2666 mhz


 


[STEPS]
SECTION: Data Gathering: DOAJ
1. Collecting data from DOAJ: we download data about journals and articles from the DOAJ website, and then refine it excluding all information that we are not interested in.

The following outputs are expected from this step:

 

SECTION: Data Gathering: DOAJ
1.2. From the DOAJ dump we create doi.json by running create_json_dois.py. Inside this JSON file we create a unique key for each journal by concatenating the issn and the eissn, having as values: the issn (if it is present), eissn (if it is present), the title of the journal, the list of all the articles' DOIs.

Firstly, we open the tarfile containing the data:

 

For every journal, we extract only the information about issn and eissn, verifying first that there is always at least one of the two for each record in the dump:

 

We then create add to the set of journals our unique identifier "issn+eissn"

 

SECTION: Data Visualization
5. We visualize our results in bar and scatter graphs with the use of the plotly Python library. 
We compute graphs for the top 10 most citing and cited articles in number of citations, normal or involving open access journals as both citing and cited entities. 

We also plot the number of these open citations by year, thanks to the data collected in open_cit_in_years.json at step 4.
SECTION: Publishing data
6. We publish the following JSON and CSV files in Zenodo and also in our Github repository (data folder).
SECTION: Data Gathering: DOAJ
1.1. Firstly we download the data dumps from DOAJ in .tar.gz. format.
 

 
Both datasets contain metadata that is not useful for our research, so we need to filter only the necessary data.
SECTION: Data Gathering: OpenCitations
2. Collecting and filtering data from OpenCitations: we take the data from the download section, on the OpenCitations website, and then refine them using the files obtained from the previous step.

These are our expected results from this section:
 

SECTION: Data Gathering: grouping by year
4. We grouped and counted the collection of citations coming and going to DOAJ journals (for simplicity we called them open citations) by years.

 


The result was open_cit_in_years.json of all of these citations for every year that was found.
 


We also encountered some citations with either a NaN creation date or a wrong creation date (ex. ''2109"). We collected them in a open_cit_w_date_err.json with this code:

 

 

SECTION: Data Gathering: merge
3. Merging the data from DOAJ and from OC: we add to the doi.json file, obtained from the first step, the information obtained from step 2 stored in the general_count_dict. In this dictionary the information is divided in the same way as the doi.json file: by journals.


 


 

We save the output as a JSON file (doi_with_count.json)

 

 

SECTION: Data Gathering: DOAJ
1.3. The extraction of data for the articles is the same as above, opening the file with tarfile, then for each article, we collect the information about issn, eissn of the journal publishing it, as well as the DOI of the article:

 

If the article doesn't have any DOI registered, we add it to a list that we will store separately.
Otherwise we handle cases where the issn and eissn have been wrongly registered in the articles dump by aligning data with the journals set previously created. 

 

Once all of the information are collected, we add them to our final json, adding a new key if it doesn't exist or adding it to the list of dois for the journal

 

We save the dictionary completed in doi.json and the list of articles without dois in a separate file articles_without_dois.json

An example of an element in the doi.json:
 

SECTION: Data Gathering: DOAJ
1.4. We also create another file dois_articles_journals.pickle containing a dictionary with all DOAJ articles' DOIs from DOAJ as keys and the "issn+eissn" identifier of the journal who published it as value, which has just the aim to simplify the next steps.


 

SECTION: Data Gathering: OpenCitations
2.1. We unzip the first level of files and obtain a series of zip repositories. In this way, we avoid extending the required memory for storing files.
 

Then we iterate over all the files inside the zip directory, following these nested steps:

Collect all zip files in a list
Iterate all over zip files
iterate all over CSVs into zip file

Moreover we initialize an empty dictionary (general_count_dict), which will be populated in the next steps.

 

SECTION: Data Gathering: OpenCitations
2.2. We decode the file obtaining a file in CSV format.

 

Using the Python library Pandas, we transform the CSV file into a dataframe.

 

SECTION: Data Gathering: OpenCitations
2.3. We create a dictionary with several filtered versions of the same CSV file, with a dictionary comprehension and the pickle file obtained from the previous step about the DOAJ data (dois_articles_journals.pickle):

A cited version with only records which have as value in the column cited a DOI from DOAJ.
A citing version with only records which have as value in the column citing a DOI from DOAJ.
An open_cit version with only records which have as value in both columns, citing and cited, a DOI from DOAJ.

 

 

SECTION: Data Gathering: OpenCitations
2.4. Then we take the previous initialized general_count_dict and we iterate all over these selected dataframes.

 

For each dataframe we follow these steps:

1.    We create another column with the id of the journal defined inside the file pickle.
2.    We group_by the column previous selected for filtering operation:

 

3.    finally with another function (counter_function) we add all the records to the general_count_dict, defined before:

 

SECTION: Data Gathering: OpenCitations
2.5. At each iteration on files from the zipped repository, after the open_cit dataframe, we save the open_cit version of the CSV as a JSON file inside the data/queried/OC/open_cit repository. These will be used for the plotting part.

 

 

Moreover, at each iteration, we save the updated version of the general_count_dict inside the data/queried repository into a JSON file format.