TITLE: Protocol: "Investigating DOIs classes of errors" V.2
AUTHORS: Ricarda Boente, Deniz Tural, Cristian Santini, Arcangelo Massari

[DESCRIPTION]
<div class = "text-blocks"><div class = "text-block">The purpose of this protocol is to provide an automated process to repair invalid DOIs that have been collected by the OpenCitations Index Of Crossref Open DOI-To-DOI References (COCI) while processing data provided by Crossref.</div><div class = "text-block"><span>The data needed for this work is </span><a href="http://doi.org/10.5281/zenodo.4625300" style = "text-decoration:underline;color:blue;cursor:pointer;"><span style = ":;">provided by COCI as a CSV</span></a><span> containing pairs of valid citing DOIs and invalid cited DOIs. With the goal to determine an automated process, we first classified the errors that characterize the wrong DOIs in the list. Our starting hypothesis is that there are two main classes of errors: factual errors, such as wrong characters, and DOIs that are not yet valid at the time of processing. The first class can be furtherly divided into three classes: errors due to irrelevant strings added to the beginning (prefix-type errors) or at the end (suffix-type errors) of the correct DOI, and errors due to unwanted characters in the middle (other-type errors). Once the classes of errors are addressed, we propose automatic processes to obtain correct DOIs from wrong ones. These processes involve the use of the information returned from </span><a href="https://www.doi.org/factsheets/DOIProxy.html" style = "text-decoration:underline;color:blue;cursor:pointer;"><span style = ":;">DOI API</span></a><span>, as well as rule-based methods, including regular expressions. to correct invalid DOIs.</span></div><div class = "text-block">The application of this methodology produces a CSV dataset containing all the pairs of citing and cited DOIs in the original dataset, each one enriched by 5 fields: "Already_Valid", which tells if the cited DOI was already valid before cleaning, "New_DOI", which contain a clean, valid DOI (if our procedure was able to produce one), and "prefix_error", "suffix_error" and "other-type_error" fields, which contain, for each cleaned DOI the number of errors that were cleaned.</div></div>

[STEPS]
?. [DATA IMPORT]
We import a CSV containing invalid DOI-to-DOI citations. The CSV is provided on public license by Peroni, Silvio. (2021). Citations to invalid DOI-identified entities obtained from processing DOI-to-DOI citations to add in COCI (Version 1.0) [Data set]. Zenodo. http://doi.org/10.5281/zenodo.4625300. The CSV is organized as follows: the first column contains valid citing DOIs and the second contains invalid cited DOIs.
10.14778/1920841.192095410.5555/646836.708343	10.5406/ethnomusicology.59.2.0202	10.2307/20184517	10.1161/01.cir.63.6.1391	10.1161/circ.37.4.509
?. [ERROR ANALYSIS]
Taking as a reference the DOI error's taxonomy proposed in "Errors in DOI indexing by bibliometric databases" (Franceschini et al. 2015), there are two main classes of errors: author errors, made by authors when creating the list of cited articles for their publication, and database mapping errors, related to a data-entry error. This protocol deals only with the second kind of error, which can be further divided between prefix errors, suffix errors, and other-type errors, as proposed in "DOI errors and possible solutions for Web of Science" (Xu et al., 2019). In order to solve our problem, we manually isolated from a subset of 100 DOIs recurrent strings at the beginning and at the end with corrupted DOI prefix and suffix respectively. In addition, we found other types of errors, like double underscores, double periods, XML tags, spaces, and forward slashes that could be removed at the end of the data cleaning process.
?. [DATA CLEANING PROCEDURE]
Here we describe the steps through which we carried out the cleaning procedure of factually invalid DOIs. The workflow of our procedure is organized as follows:We created a Python software for handling the whole procedure. The source code is publicly available on ISC license at http://doi.org/10.5281/zenodo.4723984. For our data cleaning system, we devised two class objects: one called class Clean_DOIs(), with all the methods required to carry our cleaning/validation procedure, and class Support(), with all the methods required to process and dump CSV as well as carry HTTP requests. In this protocol, we describe the use of the different methods that can be imported from our classes. If you want to reproduce the procedure described in the next step, once you have verified to have Python 3.x installed and the required libraries, you can clone our Github repository and launch the tutorial.py file in a terminal inside the folder or you can execute the commands listed in the following sub-sections.
?. [DATA CLEANING PROCEDURE]
Our procedure first checks for each cited DOI if it is factually invalid or it has become valid in the meanwhile. For this purpose, we use the function check_dois_validity(), from our class Clean_DOIs(), which takes in input a list of dictionaries like the one described above and does a GET request to the DOI Proxy with the cited DOI (https://www.doi.org/factsheets/DOIProxy.html) of each row of the CSV: if the status code corresponding to that specific DOI is different from 1, it means that it is not valid; otherwise, that DOI has become valid in the meanwhile. The function outputs an enriched version of the list of dictionaries in input, with two additional fields: "Valid_DOI", which stores a validated DOI, and "Already_valid", which stores 1 if it has become valid in the meanwhile, 0 otherwise.def check_dois_validity(self, data:list) -> list:        checked_dois = list()        pbar = tqdm(total=len(data))        for row in data:            invalid_dictionary = {                "Valid_citing_DOI": row["Valid_citing_DOI"],                "Invalid_cited_DOI": row["Invalid_cited_DOI"],                 "Valid_DOI": "",                "Already_valid": 0            }            handle = Support().handle_request(url=f"https://doi.org/api/handles/{row['Invalid_cited_DOI']}", cache_path=self.cache_path, error_log_dict=self.logs)            if handle is not None:                if handle["responseCode"] == 1:                    checked_dois.append(                        {"Valid_citing_DOI": row["Valid_citing_DOI"],                        "Invalid_cited_DOI": row["Invalid_cited_DOI"],                         "Valid_DOI": row['Invalid_cited_DOI'],                        "Already_valid": 1                        })                else:                    checked_dois.append(invalid_dictionary)                   else:                checked_dois.append(invalid_dictionary)                         pbar.update(1)        pbar.close()        return checked_doisFor performing the HTTP request, the function uses the handle_request() method of the class Support().def _requests_retry_session(        self,        tries=1,        status_forcelist=(500, 502, 504, 520, 521),        session=None    ) -> Session:        session = session or requests.Session()        retry = Retry(            total=tries,            read=tries,            connect=tries,            status_forcelist=status_forcelist,        )        adapter = HTTPAdapter(max_retries=retry)        session.mount('http://', adapter)        session.mount('https://', adapter)        return sessiondef handle_request(self, url:str, cache_path:str="", error_log_dict:dict=dict()):        if cache_path != "":            requests_cache.install_cache(cache_path)        try:            data = self._requests_retry_session().get(url, timeout=10)            if data.status_code == 200:                return data.json()            else:                error_log_dict[url] = data.status_code        except Exception as e:            error_log_dict[url] = str(e)In our procedure we applied our check_dois_validity() function to our  and we store the results in a  variable:This is how the  variable appears:
[{"Valid_citing_DOI": "10.1007/s11771-020-4410-2", "Invalid_cited_DOI": "10.13745/j.esf.2016.02.011", "Valid_DOI": "10.13745/j.esf.2016.02.011", Already_valid": 1},...]
?. [DATA CLEANING PROCEDURE]
We then pass the  executed by the previous function to another function called procedure() aimed to clean the DOIs with the regular expressions described above and to verify the validity of the cleaned DOI.def clean_doi(self, doi:str) -> str:        tmp_doi = doi.upper().replace(" ", "")        prefix_match = re.search(self.prefix_regex, tmp_doi)        classes_of_errors = {            "prefix": 0,            "suffix": 0,            "other-type": 0        }        if prefix_match:            tmp_doi = prefix_match.group(1)            classes_of_errors["prefix"] += 1        suffix_match = re.search(self.suffix_regex, tmp_doi)        if suffix_match:            tmp_doi = suffix_match.group(1)            classes_of_errors["suffix"] += 1        new_doi = re.sub("\\\\", "", tmp_doi)        new_doi = re.sub("__", "_", tmp_doi)        new_doi = re.sub("\\.\\.", ".", tmp_doi)        new_doi = re.sub(".*?", "", tmp_doi)        new_doi = re.sub("", "", tmp_doi)        if new_doi != tmp_doi:            classes_of_errors["other-type"] += 1        return new_doi, classes_of_errorsdef procedure(self, data:list) -> list:        output = list()        pbar = tqdm(total=len(data))        for row in data:            invalid_doi = row["Invalid_cited_DOI"]            invalid_dictionary = {                "Valid_citing_DOI": row["Valid_citing_DOI"],                "Invalid_cited_DOI": row["Invalid_cited_DOI"],                "Valid_DOI": row["Valid_DOI"],                "Already_valid": row["Already_valid"],                "Prefix_error": 0,                "Suffix_error": 0,                "Other-type_error": 0            }            if row["Already_valid"] != 1:                new_doi, classes_of_errors = self.clean_doi(row["Invalid_cited_DOI"])                valid_dictionary = {                    "Valid_citing_DOI": row["Valid_citing_DOI"],                    "Invalid_cited_DOI": row["Invalid_cited_DOI"],                    "Valid_DOI": new_doi,                    "Already_valid": row["Already_valid"],                    "Prefix_error": classes_of_errors["prefix"],                    "Suffix_error": classes_of_errors["suffix"],                    "Other-type_error": classes_of_errors["other-type"]                }                if new_doi != row["Invalid_cited_DOI"]:                    handle = Support().handle_request(url=f"https://doi.org/api/handles/{new_doi}", cache_path=self.cache_path, error_log_dict=self.logs)                    if handle is not None:                        output.append(valid_dictionary)                    else:                        output.append(invalid_dictionary)                else:                    output.append(invalid_dictionary)            else:                output.append(invalid_dictionary)            pbar.update(1)        pbar.close()        return outputIn our data cleaning procedure we store the output of the procedure() function in the  variable:For each cited DOI, the function procedure() performs five actions:1) it removes white spaces in the string2) it applies the prefix regular expression to catch eventual unwanted characters that precede a DOI. If there's a match, the regular expression removes the unwanted characters and the DOI is replaced with the cleaned one.3) it applies the suffix regular expressions as a list of alternative unwanted patterns that may occur at the end of the string. If there's a match, the regular expression removes the unwanted characters and the DOI is replaced with the cleaned one.4) finally, it removes eventual unwanted characters like double slashes, double points, xml tags and backward slashes.5) Finally the algorithm checks if the cited DOI after the cleaning procedure is different from the one in  and it also checks if it is returned as valid by the DOI Handle API. If it is different and is validated by the DOI API, the algorithm stores the cleaned DOI in the "Valid_DOI"  field of the dictionary; if it is not different, the dictionary will store no new DOI.By applying procedure() to our  we obtain a list of dictionaries containing 7 fields: the four already present in the input list ("Valid_citing_DOI", "Invalid_cited_DOI", "Valid_DOI", "Already_Valid") and, in addition, "prefix_error", "suffix_error", "other-type" fields, which contain, for each cleaned DOI the number of errors that were cleaned. This is how the  variable looks like:
[{"Valid_citing_DOI": "10.1007/s40617-018-00299-1", "Invalid_cited_DOI": "10.1901/jaba.2012.45-657", "Valid_DOI": "10.1901/JABA.2012.45-657", Already_valid": 1, "prefix_error": 0, "suffix_error" 0:, "other-type": 0 }, {"Valid_citing_DOI": "10.1074/jbc.m508416200", "Invalid_cited_DOI": "10.1059/0003-4819-100-4-483", "Valid_DOI": "", Already_valid": 0, "prefix_error": 0, "suffix_error" 0:, "other-type": 0 }, {"Valid_citing_DOI": "10.1177/2054358119836124", "Invalid_cited_DOI": "10.1016/j.amepre.2015.07.017.", "Valid_DOI": "10.1016/J.AMEPRE.2015.07.017", Already_valid": 0, "prefix_error": 0, "suffix_error" 1:, "other-type": 0 },...]
?. [STATISTICS AND VISUALIZATION]
Finally, a visualization was produced starting from the output file. For this purpose, d3.js v6.7.0, an open-source Javascript library for manipulating data-driven documents, was used. In particular, a barchart was created that compares the total number of DOIs with the number of valid DOIs after cleaning, the number of those that were already valid and the number of those that presented the various types of errors, i.e. errors of prefix, suffix and other-type. The barchart can be sorted in ascending or descending order and you can hover the mouse over the bars to view the exact represented value. It is possible to interact with the visualization at the following address: https://open-sci.github.io/2020-2021-grasshoppers-code/.
?. [ERROR ANALYSIS]
For cleaning prefix-type errors, we took a modified version of the regular expression from (Xu et al., 2019) in order to remove unwanted characters at the beginning of a DOI. The regular expression for matching prefix-type errors is the following:
"(.*?)(?:\.)?(?:HTTP:\/\/DX\.D[0|O]I\.[0|O]RG\/|HTTPS:\/\/D[0|O]I\.[0|O]RG\/)(.*)"
?. [ERROR ANALYSIS]
For capturing suffix-type errors, we enriched the approach provided by (Xu et al., 2019) and we devised a set of 17 alternative suffix regular expressions. The regular expression are the following:All the suffix regular expressions can be combined in a single regular expression by using alternation within a non-capturing group. This is how the regular expression for suffix-type errors looks like:
1) "(.*?)(?:\/-\/DCSUPPLEMENTAL)$"2) "(.*?)(?:SUPPINF[0|O](\.)?)$"3) "(.*?)(?:[\.|\(|,|;]?PMID:\d+.*?)$"4) "(.*?)(?:[\.|\(|,|;]?PMCID:PMC\d+.*?)$"5) "(.*?)(?:[\(|\[]EPUBAHEADOFPRINT[\)\]])$"6) "(.*?)(?:[\.|\(|,|;]?ARTICLEPUBLISHEDONLINE.*?\d{4})$"7) "(.*?)(?:[\.|\(|,|;]*HTTP:\/\/.*?)$"8) "(.*?)(?:\/(META|ABSTRACT|FULL|EPDF|PDF|SUMMARY)([>|\)](LAST)?ACCESSED\d+)?)$"9) "(.*?)(?:[>|)](LAST)?ACCESSED\d+)$"10) "(.*?)(?:[\.|(|,|;]?[A-Z]*\.?SAGEPUB.*)$?"11) "(.*?)(?:\.{5}.*?)$"12) "(.*?)(?:[\.|,|13) "(.*?)(?:[\.|(|,|;]10.\d{4}\/.*?)$"14) "(.*?)(?:\[DOI\].*?)$"15) "(.*?)(?:\(\d{4}\)?)$"16) "(.*?)(?:\?.*?=.*?)$"17) "(.*?)(?:#.*?)$"
"(.*?)(?:\/-\/DCSUPPLEMENTAL|SUPPINF[0|O](\.)?|[\.|\(|,|;]?PMID:\d+.*?|[\.|\(|,|;]?PMCID:PMC\d+.*?|[\(|\[]EPUBAHEADOFPRINT[\)\]]|[\.|\(|,|;]?ARTICLEPUBLISHEDONLINE.*?\d{4}|[\.|\(|,|;]*HTTP:\/\/.*?|\/(META|ABSTRACT|FULL|EPDF|PDF|SUMMARY)([>|\)](LAST)?ACCESSED\d+)?|[>|\)](LAST)?ACCESSED\d+|[\.|\(|,|;]?[A-Z]*\.?SAGEPUB.*?|\.{5}.*?|[\.|,|
?. [DATA CLEANING PROCEDURE]
At the end of our procedure we dump our  in a csv file through the dump_csv() method of the class Support():def dump_csv(data:list, path:str):        print(f"[Support:INFO Writing csv at path {path}]")        with open(path, 'w', newline='', encoding='utf8')  as output_file:            keys = data[0].keys()            dict_writer = csv.DictWriter(output_file, keys)            dict_writer.writeheader()            dict_writer.writerows(data)This is how the method is used in our data cleaning procedure:The output of the function is then stored in a CSV file, organized in 7 fields as the dictionary mentioned above:10.1007/s40617-018-00299-110.1901/jaba.2012.45-65710.1901/JABA.2012.45-657100010.1074/jbc.m50841620010.1059/0003-4819-100-4-483000010.1177/205435811983612410.1016/j.amepre.2015.07.017.10.1016/J.AMEPRE.2015.07.0170010A sample of the output data is available in our code release (http://doi.org/10.5281/zenodo.4723984). Further information about the possible uses of research data generated by this project, and the support provided by the authors for reuse, are available in the DMP provided by the authors at https://doi.org/10.5281/zenodo.4665853.
10.1007/s40617-018-00299-110.1901/jaba.2012.45-65710.1901/JABA.2012.45-657100010.1074/jbc.m50841620010.1059/0003-4819-100-4-483000010.1177/205435811983612410.1016/j.amepre.2015.07.017.10.1016/J.AMEPRE.2015.07.0170010
?. [DATA CLEANING PROCEDURE]
First, with the function process_csv_input() from our class Support()  we import our data in CSV format and we store it in a DictReader, an object that maps the information in each row to a dictionary whose keys are given by the optional fieldnames parameter.def process_csv_input(path:str) -> list:        print(f"[Support:INFO Proccessing csv at path {path}]")        with open(path, 'r', encoding='utf-8') as csvfile:            reader = csv.DictReader(csvfile)            return list(reader)On our procedure, we perform this method on our input csv and we store it as a  variable. You can reproduce the experiment by launching:After this procedure, our  variable will consist of a list of dictionaries as shown below:
[{"Valid_citing_DOI": "10.14778/1920841.1920954", "Invalid_cited_DOI": "10.5555/646836.708343"}, {"Valid_citing_DOI": "10.5406/ethnomusicology.59.2.0202", "Invalid_cited_DOI": "10.2307/20184517"}, ...]