# Chemical bibliographic databases: the influence of term indexing policies on topic searches

Gilles Niel, Fabrice Boyrie, David Virieux

► To cite this version:

Gilles Niel, Fabrice Boyrie, David Virieux. Chemical bibliographic databases: the influence of term indexing policies on topic searches. New Journal of Chemistry, 2015, 39, pp.8807-8817. 10.1039/C5NJ01077B. hal-01203527

HAL Id: hal-01203527

https://hal.science/hal-01203527

Submitted on 27 Sep 2015

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

CrossMark

← click for updates

Cite this: DOI: 10.1039/c5nj01077b

Received (in Montpellier, France)
30th April 2015,

Accepted 28th August 2015

DOI: 10.1039/c5nj01077b

www.rsc.org/njc

# Chemical bibliographic databases: the influence of term indexing policies on topic searches†

Gilles Niel,*a Fabrice Boyrieb and David Virieuxa

A comparative study of the three main chemical information systems (Scifinder, Web of Science and Scopus) was performed by studying the indexing policies of titles, abstracts and keywords within selected literature articles. Various chemical expressions were introduced as topic searches to illustrate the different search tools related to term indexing. The resulting article lists were compared two-by-two by means of a script designed to identify common reference lists and specific ones to each editor. Analyzing these specific reference lists reveals that only partial coverage areas of references should be expected when querying a single platform. The discussion covers the term and keyword indexing policies, their influence on the retrievability of references and on the retrievability of the highly cited papers.

## 1. Introduction

If many previous studies compare bibliographic databases¹⁻³ in terms of citation analysis very few ones deal with the herein concerned topic. Falagas compared the strengths and weaknesses of PubMed, Scopus, WoS and Google Scholar providing an interesting overview about their main available search tools.⁴ This author introduced a single keyword as a topic search but did not provide any hit counts resulting from this particular search. An other in-depth analysis on chemical databases was proposed by Zass and shows some inconsistencies in indexing policies of the Chemical Abstracts Service (CAS) but his results were not compared with other major bibliographic platforms.⁵ These preliminary studies prompted us to analyze the consequences of term indexing policies on the number and on the consistency of retrieved answers by comparing the three above-mentioned platforms.

Term indexing has received much attention for many years from the herein compared information systems. The CAS indexes journal articles, among other document types, since the beginning of the twentieth century in a highly hierarchical way. The bibliographic CAplus database contains currently more than

40 million records covering a wide range of chemical domains including biochemistry, organic, macromolecular and applied chemistry as well as inorganic, analytical and physical chemistry.⁶ The CAS's title coverage comes close to 10 000 titles among them 1700 key journals are gathered to form a core journal list.⁷ From the outset CAS's indexing policy is document-oriented by the CAS that provides indexed terms from titles, abstracts and author keywords to a large extent in the CAplus database. In its current version, supplementary information using a hierarchical set of controlled terms is also provided.⁸ At the top level of the hierarchy a reference is first associated with one of the 80 CAS's sections and then indexing is divided into three main categories: concepts, substance related information and supplementary indexing terms.⁹ The concept category contains one or several subject headings at the first level and then terms or text-modifying phrases at the second level, both levels constituting the controlled vocabulary. Supplementary terms are keywords added by the editor that may be either different from controlled terms or may be excerpted from author keywords. The substance related information is categorized in a similar way *i.e.* the first level displays substance identifiers such as the Registry Number, the common chemical names linked with the official chemical name. The second level consists of index terms excerpted from the controlled vocabulary and also from CAS's specific terms such as substance roles.¹⁰ Thus this powerful indexing relies on both CAplus and Registry¹¹ databases enabling the user to retrieve a large reference set while using a text-only querying language.¹² Moreover Scifinder, the CAS's web interface, enables reference searching from both CAplus and MEDLINE¹³ databases, the indexing of the latter relying on the National Library of Medicine's controlled vocabulary thesaurus, named Medical Subject Headings (MeSH).¹⁴

<footnote>a Institut Charles Gerhardt Montpellier (ICGM), UMR 5253 CNRS-UM-ENSCM, Ecole Nationale Supérieure de Chimie, 8, rue de l'Ecole Normale, 34296 Montpellier, France. E-mail: gilles.niel@enscm.fr</footnote>

<footnote>b Institut Charles Gerhardt Montpellier (ICGM), UMR 5253 CNRS-UM-ENSCM, Université de Montpellier, Place E. Bataillon, 34095 Montpellier, France</footnote>

<footnote>† Electronic supplementary information (ESI) available: Additional information on the diversity of studied domains (Table S1), on the expanded timespan from 1990–2005 (Table S2), the studied references in Table 8 (Doc 1), the studied references in Table 10 (Doc 2) and the source code of the script (Doc 3). See DOI: 10.1039/c5nj01077b</footnote>
Among the whole WoS's databases, the Science Citation Index Expanded™ (SCIE) gives access to more than 40 million records from a large range of scientific domains.¹⁵ The 8500 indexed journals cover a larger set of scientific domains divided into 182 categories related to mathematics, physics, chemistry, biology, medicine, engineering, *etc*.¹⁶ Besides the title, abstract and author keyword fields, WoS provides ESI,† gathered in the Keywords Plus® field.¹⁷ This information results from an algorithmic process that excerpts terms appearing at least two times in the titles of the cited references of a processed article.¹⁸

SciVerse Scopus¹⁹ indexes more than 21 000 titles in all scientific topics classified into four domains: social sciences, physics, life and health sciences.²⁰ These two latter domains are especially well represented and the total record number comes close to 50 millions today. The term indexing policy includes titles, abstracts, author keywords as well as matched terms. These matched terms include chemical names, CAS Registry Numbers, trade names, manufacturer names and index keywords. These index keywords form the hierarchically controlled vocabulary gathered in several thesauri such as the Compendex index,²¹ EMTREE index,²² MeSH, Species index, and GeoBase subject index.²³ This list is non-exhaustive but refers to the main indexes concerned by this comparative study.

A second important factor concerns the query language and the related query tools. The CAS introduced gradually the use of a natural language to process queries by developing a computed generation of index entries from natural language phrases.²⁴⁻²⁶ In recent years this led to the natural language query (NLQ) system, an algorithmic process that breaks down phrases into concepts.²⁷ Different instructions of the process were first described by J. Williams²⁸ then thoroughly analyzed by A. Ben Wagner.²⁹ The last step of the algorithmic process consists in truncating any remaining term that is not parsed in a prior instruction, thus the term 'organocatalysis' will furnish references containing the terms: organocatalysis, organocatalyst(s), organocatalytic, and organocatalys(z)ed. The main characteristics of the NLQ system lie in avoiding: (i) the use of Boolean operators that are interpreted like prepositions,³⁰ (ii) the use of proximity operators, and (iii) any knowledge about specific field searches. Prepositions are only used to break down phrases into simpler concepts. The NLQ process enables the end-user to focus on the scientific content owing to an easy-to-use topic search interface that may appear simpler at the outset by comparison to those of WoS or of Scopus. Both latter editors provide either basic or advanced search modes that enable searches on specific fields. WoS and Scopus provide a more classic use of Boolean operators including proximity operators thus giving the searcher a higher precision on the queried expressions. In advanced query mode, many different search fields of WoS and Scopus are searchable using a quite simple syntax based on field codes.

To assess the influence of some factors such as term indexing and journal coverage, we selected some single terms or short expressions that attempt to be representatives of different chemical domains such as organic and inorganic chemistry, analytical and physical chemistry, chemistry related to energy and fuels or materials science, biochemistry and molecular

biology, and biotechnology and biochemical research methods (see ESI,† Table S1). All selected terms and expressions were submitted to the query interfaces of Scifinder, WoS and Scopus and the resulting hit sets were thoroughly analyzed.

## 2. Results

### 2.1. Querying methods

This study was limited to some document types such as journal articles, book chapters, conference papers, notes, letters and reviews because all these citation types cover the most informative part of the chemical literature. As a second argument, chemists frequently need to refer to experimental procedures that are more often embedded in journal articles than in other document types. Thus meeting abstracts, errata and corrections were discarded from the initial queries. Patents were also discarded herein because they would require a parallel study owing to their intrinsic indexing that is distinct from the ones of academic papers. The CPlus and Medline databases were queried through the Explore References by Research Topic of Scifinder. The Science Citation Index Expanded and Conference Proceedings Citation Index of WoS were selected while querying these databases in advanced search mode. The three subject areas – life sciences, health sciences and physical sciences – were queried from Scopus's databases such as Embase and Medline.³¹ Most queries were performed in 2010 but some queries were performed in previous years to check the reproducibility of the initial results over a larger timespan. Only lists of English-written papers were saved and then exported into a standard bibliographic format for comparison.

Table 1 displays the Scifinder's specific queries corresponding to some selected terms and expressions and then the whole filtering process towards the selection of unique articles. Thus column 2 displays the queried terms as they were typed in the Research Topic form of Scifinder's interface and column 3 specifies which candidate list was chosen at the next step unless otherwise noted. Filtering by year and language leads to crude hit counts (column 4). Column 5 displays the hit counts after combining answer sets when required. The citation column 6 refers to all citations after automatic removal of duplicates from the CPlus and Medline databases while column 7 corresponds to article counts after the selection of document types such as journal articles, book chapters, conference papers, notes, letters and reviews. In some entries, discarding patents from this study involves a dramatic decrease between the citation column and the article one. Other document types, i.e. meeting abstracts, errata and corrections, were discarded from citation lists by means of a script, named Iddup, that will be described below. Unique articles in column 8 result from parsing each reference list by this script so that each list does not contain any duplicate reference. The differences between the reference counts of columns 7 and 8 result from incomplete duplicate removal between the CPlus and Medline and from some errata that could not be filtered during the document type selection.
<table><caption>**Table 1** Scifinder's queried expressions and the filtering process</caption><thead><tr><th>Entry</th><th>Queried expression</th><th>Candidate</th><th>Hit counts<sup>a</sup></th><th>Combine answer sets<sup>b</sup></th><th>Citations<sup>c</sup></th><th>Articles<sup>d</sup></th><th>Unique articles<sup>e</sup></th></tr></thead><tbody><tr><td>1</td><td>Allene</td><td>The concept "allene"</td><td>366</td><td>917</td><td>771</td><td>630</td><td>585</td></tr><tr><td>2</td><td>Allenes</td><td>The concept "allenes"</td><td>879</td><td></td><td></td><td></td><td></td></tr><tr><td>3</td><td>Organocatalysis</td><td>The concept "organocatalysis"</td><td>1034</td><td>—</td><td>827</td><td>667</td><td>603</td></tr><tr><td>4</td><td>Peptidomimetics</td><td>The concept "peptidomimetics"</td><td>726</td><td>—</td><td>610</td><td>314</td><td>305</td></tr><tr><td>5</td><td>Agostic interactions</td><td>The concept "agostic interactions"</td><td>75</td><td>—</td><td>59</td><td>53</td><td>51</td></tr><tr><td>6</td><td>Battery electrodes</td><td>The concept "battery electrodes"</td><td>1554</td><td>—</td><td>1493</td><td>807</td><td>806</td></tr><tr><td>7</td><td>Graphene biosensors</td><td>The concept "graphene biosensors"</td><td>119</td><td>—</td><td>95</td><td>89</td><td>87</td></tr><tr><td>8</td><td>N-Heterocyclic carbene</td><td>"N-Heterocyclic carbene" as entered</td><td>668</td><td>793</td><td>671</td><td>508</td><td>450</td></tr><tr><td>9</td><td>N-Heterocyclic carbenes</td><td>"N-Heterocyclic carbenes" as entered</td><td>231</td><td></td><td></td><td></td><td></td></tr><tr><td>10</td><td>Modified nucleoside</td><td>The concept "modified nucleoside"</td><td>199</td><td>213</td><td>153</td><td>106</td><td>98</td></tr><tr><td>11</td><td>Modified nucleosides</td><td>The concept "modified nucleosides"</td><td>213</td><td></td><td></td><td></td><td></td></tr><tr><td>12</td><td>Phosphine ligand</td><td>"Phosphine ligand" as entered</td><td>190</td><td>378</td><td>330</td><td>274</td><td>253</td></tr><tr><td>13</td><td>Phosphine ligands</td><td>"Phosphine ligands" as entered</td><td>225</td><td></td><td></td><td></td><td></td></tr><tr><td>14</td><td>Renewable feedstock</td><td>The concept "renewable feedstock"</td><td>211</td><td>—</td><td>187</td><td>113</td><td>113</td></tr><tr><td>15</td><td>Copper (Cu) catalyzed arylation</td><td>See text</td><td>76</td><td>—</td><td>65</td><td>53</td><td>51</td></tr><tr><td>16</td><td>Hybrid materials and nanoparticles</td><td>See text</td><td>266</td><td>—</td><td>217</td><td>179</td><td>177</td></tr><tr><td>17</td><td>Viscosity of ionic liquids</td><td>The two concepts "viscosity" and "ionic liquids" were present anywhere in the reference</td><td>509</td><td>—</td><td>429</td><td>343</td><td>335</td></tr><tr><td>18</td><td>Band gap in solar cells</td><td>The two concepts "band gap" and "solar cells" closely associated with one another</td><td>554</td><td>—</td><td>526</td><td>431</td><td>430</td></tr><tr><td>19</td><td>Statistical analyses of DNA microarrays</td><td>References were found where the two concepts "statistical analyses" and "DNA microarrays" were present anywhere in the reference</td><td>154</td><td>—</td><td>150</td><td>98</td><td>93</td></tr><tr><td>20</td><td>Surface area in mesoporous materials</td><td>The two concepts "surface area" and "mesoporous materials" closely associated with one another</td><td>372</td><td>—</td><td>346</td><td>296</td><td>294</td></tr></tbody></table>

As pointed out by Ben Wagner, singular form vs. plural form queries in Scifinder may lead to somewhat different results. Therefore we tested each term or expression under both forms.²⁹ In most cases the hit counts are equal except for entries 1, 2, 8, 9, 10, 11, 12 and 13. For example the answer lists corresponding to 'allene' (entry 1) and 'allenes' (entry 2) contain references where the queried term was found as a concept. Combining the two answer lists (917 hits) and then the removal of duplicates (771 citations) furnishes 630 journal articles. The expression 'N-heterocyclic carbene' (entry 8) leads to a greater hit count (668 hits) than the corresponding plural form (231 hits). A processing similar to entries 1 and 2 led to 671 citations and 508 journal articles. This emphasizes in such cases that both singular and plural forms need to be searched. With respect to the expressions 'modified nucleoside' and 'modified nucleosides' (entries 10 and 11) the largest list contains the smallest one after combining them. Because the references corresponding to the terms and expressions of entries 1, 2, 10 and 11 were selected through a concept search, the process of combining answer lists may be simplified by typing the singular and the plural forms within the same search and by using one of these forms within brackets. However this trick is not valid if the references corresponding to an expression are found containing this expression 'as entered' as in the cases of entries 8, 9, 12 and 13. Finally the term 'material' (entries 16 and 20) was searched as a concept on both databases under singular vs. plural forms, the resulting hit counts were found different from less than 0.05%.

The results of the expression in entry 15 are worthy of some specific explanations because we initially performed this search by selecting the expression 'copper (Cu) catalyzed arylation' found as a concept thus leading to 375 hit counts. This high value is mostly due to a high occurrence number of the term 'aryl' resulting from the truncating step of the NLQ process. In order to retrieve only chemically answers relevant to the arylation concept, we ruled out the term 'aryl' by building this query as follows: (i) references were found containing 'copper catalyzed' as entered (869 hits), (ii) references were found containing 'Cu catalyzed' as entered (254 hits), and (iii) the two answer sets were combined (104 hits). In parallel a reference list was found containing 'arylation' as entered (924 hits) and this latter hit set was intersected with the previously obtained 1044 hit set thus furnishing a final list of 76 hits. For entry 16 a similar process was set in order to get the terms 'hybrid' and 'materials' closer to each other. This query was built following the sequence: (i) references were found containing 'hybrid material' as entered (470 hits), (ii) references were found containing 'hybrid materials' as entered (780 hits), and (iii) the two answer sets were combined (1105 hits). In parallel a third reference set was found containing the concept 'nanoparticles' (39914 hits) and this latter set was intersected with the 1105 hit count set providing 266 hits as a final result. Because all queries were performed in March, April and May 2013, the hit counts may vary slightly if performed now.

The first point we attempted to address is related to the non-negligible proportion of duplicate answers observed within the

<footnote><sup>a</sup> Crude hit counts after filtering by year and language. <sup>b</sup> After combining answer sets when required. <sup>c</sup> After automatic removal of duplicates from the CAplus and Medline databases. <sup>d</sup> Selection of some document types. <sup>e</sup> After parsing by the Iddup script.</footnote>
Scifinder's answers whose total count is equal to 204 when summing all duplicates corresponding to each query. These internal duplicates were found among many Medline's articles that miss a DOI whereas the corresponding articles are assigned a DOI if the PubMed interface is queried. Among these 204 references, we observed too that some journal names are distinctly indexed between Medline and CAplus databases. Representative examples are given in Table 2. With respect to the Scopus's and WoS's databases only one and two duplicates were found respectively.

Tables 3 and 4 display the queries specific to WoS and Scopus, respectively, and the resulting hit counts related to the selected terms and expressions used within Scifinder's topic searches. Keeping in mind that Scifinder's topic searches include by default all indexing terms from titles, abstracts, index terms and supplementary terms we selected the corresponding WoS's search field TS (column 3) that covers the fields: title, abstract, author keywords and keywords Plus®. Queries to Scopus (column 3) were performed through the document search tab in basic mode together with the option gathering together title, abstract and keywords. By this way the retrieved answer lists are equivalent to the ones retrieved by using the field sum 'TITLE-ABS-KEY-AUTH' available in advanced search mode. In order to perform topic searches comparable to Scifinder's topic searches, the use of the right-hand truncation was systematically preferred because this enables a better control on WoS's and Scopus's queries. Boolean operators were also employed to target precisely all queries, especially the proximity operators, available in WoS and Scopus that retrieve the searched terms within the same bibliographic field.

The WoS's operator NEAR searches terms that are distant by default at a maximum of 15 terms but this distance may be shortened. Terms within double quotes were alternatively searched as an exact expression (entries 7, 11, 12 and 17, Table 3). The logic for the proximity operator W/n is similar in Scopus. This operator requires defining a number n equivalent to the distance between the searched terms. The automatic truncation in Scifinder was offset within WoS's and Scopus's searches by extensive use of wildcards as exemplified in entry 1 (Tables 3 and 4) thus enabling the terms 'allene(s)' or 'allenyl' or 'allenic' to be retrieved.

## 2.2. Result analysis automation

All article lists (Table 1, column 7 and Tables 3 and 4, column 4) were exported as text files in a tagged format in order to analyze them and to find both common and specific references to each editor. The RIS file format was chosen as an export file format from Scifinder and Scopus while WoS's data were exported into the CIW file format. In order to quickly identify duplicates among two or three reference lists we used the Iddup script whose main instructions are described as follows. For each single input file, Iddup furnishes two text files in the RIS format, the first file contains unique articles (Table 1, column 8; Tables 3 and 4, column 5) while the second file contains duplicate references. When analyzing two different input files, Iddup identifies first internal duplicates in each list, discards them and then compares pair to pair the remaining references of the two lists. As output files, Iddup provides a file containing common references and two files containing specific references from each input file.

<table><caption>**Table 2** Different indexed journal titles between CAPlus and Medline</caption><thead><tr><th>Article count</th><th>CAplus</th><th>Medline</th></tr></thead><tbody><tr><td>18</td><td>Acta Crystallographica, Section E. Structure Reports Online</td><td>Acta crystallographica.chrom Section E, Structure reports online</td></tr><tr><td>39</td><td>Angewandte Chemie, International Edition</td><td>Angewandte Chemie (International ed. in English)</td></tr><tr><td>86</td><td>Chemistry -- A European Journal</td><td>Chemistry (Weinheim an der Bergstrasse, Germany)</td></tr><tr><td></td><td>Chemistry - A European Journal</td><td></td></tr></tbody></table>

<table><caption>**Table 3** Queries and results from WoS</caption><thead><tr><th>Entry</th><th>Queried expression in Scifinder</th><th>Queried expression in WoS</th><th>Articles<sup>a</sup></th><th>Unique articles<sup>b</sup></th></tr></thead><tbody><tr><td>1</td><td>Allene(s)</td><td>TS = allene* OR TS = alleny* OR TS = alleni*</td><td>503</td><td>466</td></tr><tr><td>2</td><td>Organocatalysis</td><td>TS = organocataly*</td><td>997</td><td>963</td></tr><tr><td>3</td><td>Peptidomimetics</td><td>TS = peptidomimetic*</td><td>282</td><td>257</td></tr><tr><td>4</td><td>Agostic interactions</td><td>TS = (agostic NEAR interaction*)</td><td>65</td><td>63</td></tr><tr><td>5</td><td>Battery electrodes</td><td>TS = ((battery OR batteries) NEAR electrode*)</td><td>1068</td><td>1008</td></tr><tr><td>6</td><td>Graphene biosensors</td><td>TS = (graphene NEAR biosensor*)</td><td>47</td><td>47</td></tr><tr><td>7</td><td>N-Heterocyclic carbene(s)</td><td>TS = "N-heterocyclic carbene*"</td><td>629</td><td>629</td></tr><tr><td>8</td><td>Modified nucleoside(s)</td><td>TS = (modif* NEAR nucleoside*)</td><td>117</td><td>114</td></tr><tr><td>9</td><td>Phosphine ligand(s)</td><td>TS = (phosphine NEAR1 ligand*)</td><td>322</td><td>319</td></tr><tr><td>10</td><td>Renewable feedstock</td><td>TS = (renewable NEAR feedstock*)</td><td>110</td><td>92</td></tr><tr><td>11</td><td>Copper (Cu) catalyzed arylation</td><td>(TS = ("copper catalyzed") OR TS = ("Cu catalyzed")) AND TS = arylation</td><td>105</td><td>103</td></tr><tr><td>12</td><td>Hybrid materials and nanoparticles</td><td>TS = ("hybrid material*") AND TS = nanoparticle*</td><td>256</td><td>234</td></tr><tr><td>13</td><td>Viscosity of ionic liquids</td><td>TS = viscosity AND TS = ionic liquid*</td><td>360</td><td>350</td></tr><tr><td>14</td><td>Band gap in solar cells</td><td>TS = ((band NEAR gap) AND (solar NEAR cell*))</td><td>677</td><td>599</td></tr><tr><td>15</td><td>Statistical analyses of DNA microarrays</td><td>TS = statistical analyses of dna microarrays</td><td>93</td><td>93</td></tr><tr><td>16</td><td>Surface area in mesoporous materials</td><td>TS = ("surface area") AND TS = (mesopor* material*)</td><td>207</td><td>193</td></tr></tbody></table>

<figure><img src="image_3.png" /></figure>

<footnote><sup>a</sup> Article counts after filtering by year, language and document type. <sup>b</sup> After parsing by the Iddup script.</footnote>
<figure><img src="image_4.png" /></figure>

<table><caption>**Table 4** Queries and results from Scopus</caption><thead><tr><th>Entry</th><th>Queried expression in Scifinder</th><th>Queried expression in Scopus</th><th>Articles<sup>a</sup></th><th>Unique articles<sup>b</sup></th></tr></thead><tbody><tr><td>1</td><td>Allene(s)</td><td>Allene* OR alleny* OR alleni*</td><td>359</td><td>356</td></tr><tr><td>2</td><td>Organocatalysis</td><td>Organocataly*</td><td>770</td><td>758</td></tr><tr><td>3</td><td>Peptidomimetics</td><td>Peptidomimetic*</td><td>299</td><td>294</td></tr><tr><td>4</td><td>Agostic interactions</td><td>Agostic W/15 interaction*</td><td>49</td><td>48</td></tr><tr><td>5</td><td>Battery electrodes</td><td>Batter* W/15 electrode*</td><td>1020</td><td>816</td></tr><tr><td>6</td><td>Graphene biosensors</td><td>graphene W/15 biosensor*</td><td>66</td><td>59</td></tr><tr><td>7</td><td>N-Heterocyclic carbene(s)</td><td>N-Heterocyclic W/1 carbene*</td><td>462</td><td>458</td></tr><tr><td>8</td><td>Modified nucleoside(s)</td><td>(modif* W/15 nucleoside*)</td><td>114</td><td>113</td></tr><tr><td>9</td><td>Phosphine ligand(s)</td><td>Phosphine W/1 ligand*</td><td>286</td><td>284</td></tr><tr><td>10</td><td>Renewable feedstock</td><td>Renewable W/15 feedstock*</td><td>242</td><td>154</td></tr><tr><td>11</td><td>Copper (Cu) catalyzed arylation</td><td>((copper W/1 catalyzed) OR (Cu W/1 catalyzed)) AND arylation</td><td>52</td><td>52</td></tr><tr><td>12</td><td>Hybrid materials and nanoparticles</td><td>(Hybrid W/1 material* and nanoparticle*)</td><td>271</td><td>235</td></tr><tr><td>13</td><td>Viscosity of ionic liquids</td><td>Viscosity of ionic W/1 liquid*</td><td>341</td><td>327</td></tr><tr><td>14</td><td>Band gap in solar cells</td><td>Band gap in solar cell*</td><td>606</td><td>424</td></tr><tr><td>15</td><td>Statistical analyses of DNA microarrays</td><td>Statistical analys* of dna microarray*</td><td>310</td><td>298</td></tr><tr><td>16</td><td>Surface area in mesoporous materials</td><td>Surface area in mesopor* W/1 material*</td><td>383</td><td>362</td></tr></tbody></table>

When comparing two references from input lists without internal duplicates, Iddup assigns each pair a score that is computed based on the following filters:

* initial score = 0

* if same DOI then score = 10 (and references are identical)

* if similar title then increment score +3

* if same journal then increment score +1

* if same author count then increment score +0.5

* if similar author and same position then increment score +0.5

* if same starting page then increment score +1.5

* if same volume then increment score +0.5

* if same issue then increment score +0.5

* if scores > 5, then the two references are considered as identical.

The second instruction enables the script to overlook the next instruction in case of same DOIs are found. A similarity computing was introduced at the third instruction that compares the titles because many titles contain abbreviations or Greek characters that are not always indexed in the same way by the different editors. These statements prompted us to introduce a 12% similarity score – 12% of the length of the longest title – that was computed using the Levenshtein distance.³² The influence of this parameter is discussed in Section 3.3. Likewise the author names present many discrepancies due to different spelling languages, typing errors or due to a different ranking in indexing their names. Our script was completed by correspondence arrays for some journal titles and for the Latin transcription of Greek characters. Finally Iddup discards citations corresponding to errata or corrections.

## 2.3. Comparison of reference lists

Unique articles of Tables 1, 3 and 4 are reported in Fig. 1. Overall the magnitude orders range similarly except for entries 2, 5, 7 and 14 that display higher article counts found by WoS and except for entry 15 where Scopus retrieves more articles than the other two systems. Scopus and Scifinder retrieve more articles in entries 9, 10, 15 and entries 1, 6 respectively.

These results were refined through Iddup computing by identifying the common articles (column 4 in Tables 5–7) to each

pair of editors and the specific articles to each editor (columns 3 and 6 in Tables 5–7). The union of the total article counts (column 7, Tables 5–7) is given by the sum of columns 3, 4 and 6 while column 8 represents the proportion of common articles to two editors. Preliminary observations show that these proportions vary dramatically from a maximum of 80.0 to a minimum of 11.4 percent (entries 4 and 15, Table 6). Higher proportions of common articles were generally observed for single-, double- or triple-term queries than for the queries including four terms.

Though the main results were recorded in 2010, we have extended the query timespan to the years 1990, 1995, 2000, and 2005 for the four expressions: ‘allenes’, ‘peptidomimetics’, ‘battery electrodes’ and ‘band gap in solar cells’. These expressions were selected because their corresponding queries furnished sufficient hit counts to be representative as soon as 1990. For example expressions such as ‘organocatalysis’ or ‘N-heterocyclic carbenes’ returned no answer in 1990 and 1995 and were thus discarded. A second selection criterion was based on variable lengths of these four expressions.

Full resulting data are included in the ESI† (Table S2). As general conclusions of this supplementary study, we noticed that: (i) the three databases lead to different result sets as in 2010, (ii) large non-overlapping result sets were found during the years 1990, 1995, 2000, and 2005, and (iii) the proportion of overlapping papers increases over the years except for ‘peptidomimetics’.

In order to close this section, we may mention that the overall averages of shared references by Scifinder/WoS, Scifinder/Scopus and Scopus/WoS are 40.8, 46.8 and 52.2% respectively.

# 3. Discussion

These quite low overlaps between the three information systems may appear surprising but at least one precedent was observed in the computer sciences.³³

## 3.1. Influence of term indexing

Which are the reasons why these differences are often so high? To answer this question, some reference lists corresponding to

<footnote><sup>a</sup> Article counts after filtering by year, language and document type. <sup>b</sup> After parsing by the Iddup script.</footnote>
<figure><img src="image_5.png" /><figcaption>Fig. 1 Unique articles retrieved by Scifinder, WoS and Scopus.</figcaption></figure>

<table><caption>Table 5 Iddup parsing of reference lists from Scifinder and WoS</caption><thead><tr><th rowspan="2"></th><th colspan="3">Scifinder</th><th colspan="3">WoS</th><th rowspan="2">Comm./<br>union<sup>f</sup> (%)</th></tr><tr><th>Entry<sup>a</sup></th><th>Uniq.<br>articles</th><th>Spec.<sup>b</sup></th><th>Common<sup>c</sup></th><th>Uniq.<br>articles</th><th>Spec.<sup>d</sup></th><th>Union<sup>e</sup></th></tr></thead><tbody><tr><td>1</td><td>585</td><td>278</td><td>307</td><td>466</td><td>159</td><td>744</td><td>41.3</td></tr><tr><td>2</td><td>603</td><td>50</td><td>553</td><td>963</td><td>410</td><td>1013</td><td>54.6</td></tr><tr><td>3</td><td>305</td><td>123</td><td>182</td><td>257</td><td>75</td><td>380</td><td>47.9</td></tr><tr><td>4</td><td>51</td><td>6</td><td>45</td><td>63</td><td>18</td><td>69</td><td>65.2</td></tr><tr><td>5</td><td>806</td><td>328</td><td>478</td><td>1008</td><td>530</td><td>1336</td><td>35.8</td></tr><tr><td>6</td><td>87</td><td>46</td><td>41</td><td>47</td><td>6</td><td>93</td><td>44.1</td></tr><tr><td>7</td><td>450</td><td>46</td><td>404</td><td>629</td><td>225</td><td>675</td><td>59.9</td></tr><tr><td>8</td><td>98</td><td>30</td><td>68</td><td>114</td><td>46</td><td>144</td><td>47.2</td></tr><tr><td>9</td><td>253</td><td>90</td><td>163</td><td>319</td><td>156</td><td>409</td><td>39.9</td></tr><tr><td>10</td><td>113</td><td>47</td><td>66</td><td>92</td><td>26</td><td>139</td><td>47.5</td></tr><tr><td>11</td><td>51</td><td>16</td><td>35</td><td>103</td><td>68</td><td>119</td><td>29.4</td></tr><tr><td>12</td><td>177</td><td>50</td><td>127</td><td>234</td><td>107</td><td>284</td><td>44.7</td></tr><tr><td>13</td><td>335</td><td>105</td><td>230</td><td>350</td><td>120</td><td>455</td><td>50.5</td></tr><tr><td>14</td><td>430</td><td>248</td><td>182</td><td>599</td><td>417</td><td>847</td><td>21.5</td></tr><tr><td>15</td><td>93</td><td>72</td><td>21</td><td>93</td><td>72</td><td>165</td><td>12.7</td></tr><tr><td>16</td><td>294</td><td>232</td><td>62</td><td>193</td><td>131</td><td>425</td><td>14.6</td></tr></tbody></table>

specific references (columns 3 and 6) were selected and each reference of these lists was thoroughly examined in order to determine for which reason this reference was found by one editor or omitted by another one. Such reasons may be related *a priori* to journal indexing or keyword indexing but we finally found some other reasons that enabled us to assign each reference to one of the following categories:

- Journal: journal indexing may be absent or is stopped before 2010 or issue indexing is incomplete.

<table><caption>Table 6 Iddup parsing of reference lists from Scifinder and Scopus</caption><thead><tr><th rowspan="2"></th><th colspan="3">Scifinder</th><th colspan="3">Scopus</th><th rowspan="2">Comm./<br>union<sup>f</sup> (%)</th></tr><tr><th>Entry<sup>a</sup></th><th>Uniq.<br>articles</th><th>Spec.<sup>b</sup></th><th>Common<sup>c</sup></th><th>Uniq.<br>articles</th><th>Spec.<sup>d</sup></th><th>Union<sup>e</sup></th></tr></thead><tbody><tr><td>1</td><td>585</td><td>275</td><td>310</td><td>356</td><td>46</td><td>631</td><td>49.1</td></tr><tr><td>2</td><td>603</td><td>33</td><td>570</td><td>758</td><td>188</td><td>791</td><td>72.1</td></tr><tr><td>3</td><td>305</td><td>103</td><td>202</td><td>294</td><td>92</td><td>397</td><td>50.9</td></tr><tr><td>4</td><td>51</td><td>7</td><td>44</td><td>48</td><td>4</td><td>55</td><td>80.0</td></tr><tr><td>5</td><td>806</td><td>319</td><td>487</td><td>816</td><td>329</td><td>1135</td><td>42.9</td></tr><tr><td>6</td><td>87</td><td>39</td><td>48</td><td>59</td><td>11</td><td>98</td><td>49.0</td></tr><tr><td>7</td><td>450</td><td>49</td><td>401</td><td>458</td><td>57</td><td>507</td><td>79.1</td></tr><tr><td>8</td><td>98</td><td>29</td><td>69</td><td>113</td><td>44</td><td>142</td><td>48.6</td></tr><tr><td>9</td><td>253</td><td>75</td><td>178</td><td>284</td><td>106</td><td>359</td><td>49.6</td></tr><tr><td>10</td><td>113</td><td>34</td><td>79</td><td>154</td><td>75</td><td>188</td><td>42.0</td></tr><tr><td>11</td><td>51</td><td>14</td><td>37</td><td>52</td><td>15</td><td>66</td><td>56.1</td></tr><tr><td>12</td><td>177</td><td>41</td><td>136</td><td>235</td><td>99</td><td>276</td><td>49.3</td></tr><tr><td>13</td><td>335</td><td>100</td><td>235</td><td>327</td><td>92</td><td>427</td><td>55.0</td></tr><tr><td>14</td><td>430</td><td>236</td><td>194</td><td>424</td><td>230</td><td>660</td><td>29.4</td></tr><tr><td>15</td><td>93</td><td>53</td><td>40</td><td>298</td><td>258</td><td>351</td><td>11.4</td></tr><tr><td>16</td><td>294</td><td>208</td><td>86</td><td>362</td><td>276</td><td>570</td><td>15.1</td></tr></tbody></table>

- Document types: Conference Proceedings, Book Reviews, and International Symposia that are not homogeneously indexed by the editors.

- Index terms: Indexing terms, Keywords and Keywords Plus®. In case of Scifinder, supplementary terms are included in index terms.

- Modified terms: (a) some journals do not provide any abstract; in those cases Scifinder designs an abstract that seems to be excerpted from the article conclusion, (b) some queried terms are

<figure><img src="image_6.png" /></figure>

<footnote>a Entries 1–16 correspond to the queried expressions of previous Tables 3 and 4. b Specific articles to Scifinder. c Shared articles by both editors. d Specific articles to WoS. e Sum of columns 3, 4 and 6. f Proportion of common articles to two editors.</footnote>

<footnote>a Entries 1–16 correspond to the queried expressions of previous Tables 3 and 4. b Specific articles to Scifinder. c Shared articles by both editors. d Specific articles to Scopus. e Sum of columns 3, 4 and 6. f Proportion of common articles to two editors.</footnote>
<table><caption>**Table 7** Iddup parsing of reference lists from Scopus and WoS</caption><thead><tr><th rowspan="2">Entry<sup>a</sup></th><th colspan="3">Scopus</th><th colspan="3">WoS</th></tr><tr><th>Uniq. articles</th><th>Spec.<sup>b</sup></th><th>Common<sup>c</sup></th><th>Uniq. articles</th><th>Spec.<sup>d</sup></th><th>Union<sup>e</sup></th></tr></thead><tbody><tr><td>1</td><td>356</td><td>51</td><td>305</td><td>466</td><td>161</td><td>517</td><td>59.0</td></tr><tr><td>2</td><td>758</td><td>44</td><td>714</td><td>963</td><td>249</td><td>1007</td><td>70.9</td></tr><tr><td>3</td><td>294</td><td>92</td><td>202</td><td>257</td><td>55</td><td>349</td><td>57.9</td></tr><tr><td>4</td><td>48</td><td>3</td><td>45</td><td>63</td><td>18</td><td>66</td><td>68.2</td></tr><tr><td>5</td><td>816</td><td>268</td><td>548</td><td>1008</td><td>460</td><td>1276</td><td>42.9</td></tr><tr><td>6</td><td>59</td><td>16</td><td>43</td><td>47</td><td>4</td><td>63</td><td>68.3</td></tr><tr><td>7</td><td>458</td><td>38</td><td>420</td><td>629</td><td>209</td><td>667</td><td>63.0</td></tr><tr><td>8</td><td>113</td><td>28</td><td>85</td><td>114</td><td>29</td><td>142</td><td>59.9</td></tr><tr><td>9</td><td>284</td><td>63</td><td>221</td><td>319</td><td>98</td><td>382</td><td>57.9</td></tr><tr><td>10</td><td>154</td><td>66</td><td>88</td><td>92</td><td>4</td><td>158</td><td>55.7</td></tr><tr><td>11</td><td>52</td><td>14</td><td>38</td><td>103</td><td>65</td><td>117</td><td>32.5</td></tr><tr><td>12</td><td>235</td><td>87</td><td>148</td><td>234</td><td>86</td><td>321</td><td>46.1</td></tr><tr><td>13</td><td>327</td><td>83</td><td>244</td><td>350</td><td>106</td><td>433</td><td>56.4</td></tr><tr><td>14</td><td>424</td><td>132</td><td>292</td><td>599</td><td>307</td><td>731</td><td>39.9</td></tr><tr><td>15</td><td>298</td><td>226</td><td>72</td><td>93</td><td>21</td><td>319</td><td>22.6</td></tr><tr><td>16</td><td>362</td><td>204</td><td>157</td><td>193</td><td>36</td><td>397</td><td>39.5</td></tr></tbody></table>

<sup>a</sup> Entries 1–16 correspond to the queried expressions of previous Tables 3 and 4. <sup>b</sup> Specific articles to Scopus. <sup>c</sup> Shared articles by both editors. <sup>d</sup> Specific articles to WoS. <sup>e</sup> Sum of columns 3, 4 and 6. <sup>f</sup> Proportion of common articles to two editors.

indexed using a hyphen included in the retrieved term i.e. organocatalytic, (c) the journal title is indexed in two different spellings, and (d) author keywords or titles or abstracts are modified.

– Abstracts: though provided by the journal, some abstracts are not indexed.

– Author keywords: though provided by the publisher, some author keywords are excluded from indexing.

– Different year: some issues are assigned a different year because the dates of the online publication and of the printed version are different.

– Wrong DOI: typographic errors were found in agreement with recent similar observations.<sup>34</sup> We noticed that a non-negligible amount of articles were missing an assigned DOI. Indeed concatenation of all articles from a particular editor followed by the removal of internal duplicates revealed that 8.7, 6.5 and 4.7% of articles from Scifinder, Scopus and WoS, respectively, were missing a DOI.

– Miscellaneous.

ESI† (Doc 1) details the whole results corresponding to the ‘organocatalysis’ queried term, the ‘N-heterocyclic carbenes’, the ‘phosphine ligands’ and the ‘viscosity of ionic liquids’ expressions. Table 8 displays the results obtained for the ‘organocatalysis’ queried term. The main observed differences arise from the Index Terms row. The Keywords Plus<sup>R</sup> indexing of WoS provided more articles than those retrieved by Scopus’s or Scifinder’s term indexing, this latter editor showing the weakest efficiency of its term indexing policy within this example. We also checked the relevance of 50 randomly selected references from the 234 references only retrieved by the Keywords Plus<sup>R</sup>. At least 45 over these 50 references were strongly related to organocatalysis. With respect to the Modified terms row, Scifinder designed an abstract excerpted from the article conclusion in one case and in the other one a hyphen was introduced in the term ‘organocatalytic by WoS’ (Table 8, column 3). On the same row (Table 8, entry 4, column 4), a hyphen was introduced in the term ‘organocatalytic’ eight times by Scifinder and in one case the term ‘organocatalyst*’ was shortened to ‘catalyst*’ within the title. Within the Abstracts row the reference found by Scifinder (Table 8, column 3) presents an abstract that was not indexed by WoS. In the case of the journal ‘Angewandte Chemie, International Edition in English’, we checked 500 articles of this journal and we found that they were missing an indexed abstract by WoS. This statement is valid up to 2010 but many abstracts are indexed in more recent years. In column 4 (entry 5) the 10 references found specifically by WoS result from a left truncation of the term ‘organocatalyst’ to ‘catalyst’ in Scifinder. More surprising are the 149 references (entry 6, column 4) where Scifinder modified the original author keywords by shortening or suppressing the queried term.

Five articles were indexed by WoS with one misspelled character on their DOI compared to the original DOI (Table 8, entry 8). Finally the miscellaneous category contains articles where: (i) the filters applied to the document types during the querying step differ from one editor to another one thus during the analysis step Iddup discards citations corresponding to some unwanted document types i.e. book chapters and corrections, and (ii) the 0.8 similarity score on the titles and on the author names was in one case the reason why two references were wrongly differentiated.

<table><caption>**Table 8** Study of reference lists corresponding to the ‘organocatalysis’ term</caption><thead><tr><th rowspan="2">Entry</th><th rowspan="2">Category</th><th colspan="2">Scifinder/WoS</th><th colspan="2">Scifinder/Scopus</th><th colspan="2">Scopus/WoS</th></tr><tr><th>Scifinder (50)<sup>a</sup></th><th>WoS (410)<sup>a</sup></th><th>Scifinder (33)<sup>a</sup></th><th>Scopus (188)<sup>a</sup></th><th>Scopus (44)<sup>a</sup></th><th>WoS (249)<sup>a</sup></th></tr></thead><tbody><tr><td>1</td><td>Journals</td><td>22</td><td>0</td><td>16</td><td>4</td><td>10</td><td>5</td></tr><tr><td>2</td><td>Document types</td><td>5</td><td>1</td><td>5</td><td>0</td><td>4</td><td>1</td></tr><tr><td>3</td><td>Index terms</td><td>4</td><td>234</td><td>7</td><td>148</td><td>0</td><td>232</td></tr><tr><td>4</td><td>Modified terms</td><td>2</td><td>9</td><td>0</td><td>5</td><td>1</td><td>0</td></tr><tr><td>5</td><td>Abstracts</td><td>1</td><td>10</td><td>2</td><td>29</td><td>11</td><td>1</td></tr><tr><td>6</td><td>Author keywords</td><td>0</td><td>149</td><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><td>7</td><td>Different year</td><td>6</td><td>2</td><td>1</td><td>1</td><td>11</td><td>3</td></tr><tr><td>8</td><td>Wrong DOI</td><td>5</td><td>5</td><td>1</td><td>1</td><td>7</td><td>6</td></tr><tr><td>9</td><td>Miscellaneous</td><td>5</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td></tr><tr><td>10</td><td>Checked index terms<sup>b</sup></td><td>6 (7)</td><td>387 (402)</td><td>9 (9)</td><td>182 (182)</td><td>10 (12)</td><td>231 (232)</td></tr></tbody></table>

<sup>a</sup> Numbers within brackets correspond to specific articles reported in entries 2 of Tables 5–7. <sup>b</sup> Numbers within brackets correspond to the sum of articles from the indexing categories 3 to 6.

<figure><img src="image_7.png" /></figure>
<figure><img src="image_8.png" /></figure>

If we consider all articles of a particular editor that are classified in the index terms or modified terms or abstracts or author keywords categories, the next question remains to verify whether the concurrent editor's database is really missing this specific information or not? To check this hypothesis we injected the DOIs or the bibliographic data of a given editor's articles corresponding to the above-mentioned indexing categories into the query interface of the concurrent editor. The results are displayed in the last row (Table 8, entry 10). For example 6 over 7 specific articles retrieved by Scifinder (Table 8, column 3) are also present in the WoS thus emphasizing the importance of the Scifinder's indexing policy in this case. Once this statement has been established we noted that only 22 articles (Table 8, entry 1) from specific journals and 5 articles (Table 8, entry 2) from the document type category belong specifically to Scifinder. The vast majority of articles retrieved by WoS (Table 8, entry 10, column 4) would have been retrieved likewise by Scifinder if different indexing rules have been applied.

Comparing Scifinder and Scopus (Table 8, columns 5 and 6) on their specific references led to similar observations. The coverage of journals is in favour of Scifinder whereas Scopus retrieves a higher article count owing to its term indexing. Moreover Scopus indexes in the case of 3 reviews not only the abstracts but also the tables of contents where the queried term is present. We noticed too that author keywords were neither suppressed nor modified.

By comparing Scopus and WoS (Table 8, columns 7 and 8), we observed that WoS shows a high count of articles retrieved by the Keywords Plus<sup>a</sup> indexing. Among the 11 articles included in the abstracts category (Table 8, column 7) 3 reviews are present indexed by Scopus within their tables of contents. The 8 remaining articles of the abstracts category correspond to references for which WoS did not index the abstract. We observed that the different year category displays a rather important amount of articles: 11 articles are indexed by WoS in 2009 or 2011 and 3 articles are indexed by Scopus in 2009. Obviously these articles would have been retrieved by a multiple-year query. In the wrong DOI category were found the same articles as previously noticed.

In order to confirm the results displayed in Table 8, we analyzed some data from two-term queries and a three-term query (Table 9). The first studied expression was 'N-heterocyclic carbenes' (Table 9, columns 3 and 4) and the articles retrieved by Scifinder and Scopus respectively. Here again the influence of term indexing is predominant but to a smaller extent than previously. Within the modified terms category we observed that in some cases Scifinder developed the NHC acronym to 'N-heterocyclic carbenes' thus enabling the corresponding article to be retrieved. Finally 6 over 7 articles present in the miscellaneous category (Table 9, column 3) correspond to misspellings or typographic errors from Scopus.

The next results concerned the two-term expression 'phosphine ligands' and the retrieved articles by Scopus and WoS (Table 9, columns 5 and 6). Apart from the predominant influence of term indexing by both editors, Scopus offers in this case a slightly better journal coverage and a better abstract coverage. In the miscellaneous category Scopus retrieved some articles containing the expanded forms of the 'phosphine' term such as 'bisphosphine' or 'triphenylphosphine'. Finally we looked at the three-term query 'viscosity of ionic liquids' (Table 9, columns 7 and 8) and examined the specific articles retrieved by Scopus and WoS. The observed proportions within the different categories are similar to those obtained in previous cases, the index term category remaining the main differentiating one.

These last results (Tables 8 and 9) were not computed by any algorithmic process and only affect a part of the study presented in Tables 5–7. Nevertheless they reveal some interesting trends about the scope and the limits of term and keyword indexing policies of Scifinder, Scopus and WoS. If we focus now on the values displayed in different columns of entry 10 (Tables 8 and 9), we observe that a high proportion of articles retrieved in the indexing categories by a particular editor are present in both other editor's databases. Ultimately this emphasizes the influence of term and keyword indexing policies of these editors because most informative articles are shared by the three editors. In other words the proportion of information specific to a given editor is not as high as it could be expected from preliminary results displayed in Tables 5–7. Moreover the term and keyword indexing policies clearly

<table><caption>**Table 9** Study of reference lists corresponding to the expressions 'N-heterocyclic carbenes', 'phosphine ligands' and 'viscosity of ionic liquids'</caption><thead><tr><th rowspan="2">Entry</th><th rowspan="2">Category</th><th colspan="2">N-Heterocyclic carbenes</th><th colspan="2">Phosphine ligands</th><th colspan="2">Viscosity of ionic liquids</th></tr><tr><th>Scifinder (49)<sup>a</sup></th><th>Scopus (57)<sup>a</sup></th><th>Scopus (63)<sup>a</sup></th><th>WoS (98)<sup>a</sup></th><th>Scopus (83)<sup>a</sup></th><th>WoS (106)<sup>a</sup></th></tr></thead><tbody><tr><td>1</td><td>Journals</td><td>7</td><td>13</td><td>8</td><td>2</td><td>3</td><td>4</td></tr><tr><td>2</td><td>Document types</td><td>2</td><td>0</td><td>2</td><td>1</td><td>2</td><td>0</td></tr><tr><td>3</td><td>Index terms</td><td>15</td><td>25</td><td>31</td><td>94</td><td>72</td><td>93</td></tr><tr><td>4</td><td>Modified terms</td><td>9</td><td>14</td><td>1</td><td>0</td><td>0</td><td>0</td></tr><tr><td>5</td><td>Abstracts</td><td>3</td><td>0</td><td>13</td><td>0</td><td>2</td><td>0</td></tr><tr><td>6</td><td>Author keywords</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><td>7</td><td>Different year</td><td>6</td><td>3</td><td>0</td><td>1</td><td>0</td><td>1</td></tr><tr><td>8</td><td>Wrong DOI</td><td>0</td><td>0</td><td>0</td><td>0</td><td>2</td><td>1</td></tr><tr><td>9</td><td>Miscellaneous</td><td>7</td><td>2</td><td>8</td><td>0</td><td>2</td><td>6</td></tr><tr><td>10</td><td>Checked index terms<sup>b</sup></td><td>27 (27)</td><td>39 (39)</td><td>45 (45)</td><td>91 (94)</td><td>71 (74)</td><td>88 (93)</td></tr></tbody></table>

<footnote><sup>a</sup> Numbers within brackets correspond to specific articles reported in entries 2 of Tables 5–7. <sup>b</sup> Numbers within brackets correspond to the sum of articles from the indexing categories 3 to 6.</footnote>
<table><caption>**Table 10** Influence of the citation impact on the retrievability of references</caption><thead><tr><th rowspan="2">Expression</th><th rowspan="2"></th><th colspan="3">Scifinder/WoS</th><th colspan="3">Scifinder/Scopus</th><th colspan="3">Scopus/WoS</th></tr><tr><th>Specific (50)<sup>a</sup></th><th>Common (553)</th><th>Specific (410)</th><th>Specific (33)</th><th>Common (570)</th><th>Specific (188)</th><th>Specific (44)</th><th>Common (714)</th><th>Specific (249)</th></tr></thead><tbody><tr><td rowspan="2">'Organocatalysis'</td><td rowspan="2">Two best citation counts</td><td>442</td><td>707</td><td>496</td><td>75</td><td>707</td><td>243</td><td>442</td><td>707</td><td>496</td></tr><tr><td>298</td><td>390</td><td>280</td><td>51</td><td>442</td><td>197</td><td>243</td><td>390</td><td>280</td></tr><tr><td rowspan="2">'N-Heterocyclic carbenes'</td><td rowspan="2">Two best citation counts</td><td>Specific (46)</td><td>Common (404)</td><td>Specific (225)</td><td>Specific (49)</td><td>Common (401)</td><td>Specific (57)</td><td>Specific (38)</td><td>Common (420)</td><td>Specific (209)</td></tr><tr><td>200</td><td>445</td><td>1320</td><td>89</td><td>445</td><td>755</td><td>755</td><td>445</td><td>1320</td></tr><tr><td rowspan="2">'Phosphine ligands'</td><td rowspan="2">Two best citation counts</td><td>82</td><td>233</td><td>649</td><td>80</td><td>233</td><td>395</td><td>395</td><td>388</td><td>649</td></tr><tr><td>Specific (90)</td><td>Common (163)</td><td>Specific (156)</td><td>Specific (75)</td><td>Common (178)</td><td>Specific (106)</td><td>Specific (63)</td><td>Common (221)</td><td>Specific (98)</td></tr><tr><td rowspan="2">'Band gap in solar cells'</td><td rowspan="2">Two best citation counts</td><td>182</td><td>200</td><td>259</td><td>182</td><td>200</td><td>755</td><td>755</td><td>200</td><td>259</td></tr><tr><td>119</td><td>101</td><td>249</td><td>119</td><td>101</td><td>92</td><td>92</td><td>101</td><td>249</td></tr><tr><td rowspan="2"></td><td rowspan="2"></td><td>Specific (248)</td><td>Common (182)</td><td>Specific (417)</td><td>Specific (236)</td><td>Common (194)</td><td>Specific (230)</td><td>Specific (132)</td><td>Common (292)</td><td>Specific (307)</td></tr><tr><td>1358</td><td>394</td><td>922</td><td>538</td><td>1358</td><td>628</td><td>1358</td><td>477</td><td>922</td></tr><tr><td></td><td></td><td>538</td><td>245</td><td>590</td><td>394</td><td>245</td><td>477</td><td>628</td><td>245</td><td>590</td></tr></tbody></table>

differentiate the three studied editors in a higher proportion than their respective journal coverages do.

## 3.2. Influence of the citation counts

Apart from the influence of term and keyword indexing policies, we checked the influence of the citation counts on the retrievability of references. In other words how are the most cited references distributed between specific references to an editor vs. the common ones? To answer this question, four expressions were thoroughly studied (Table 10). Our previous results mentioned in Tables 8 and 9, especially entries 10 including the checked indexed terms, revealed that most of the references could have been retrieved if homogeneous indexing policies would have been applied between the three editors. From this postulate we selected the WoS to extract the best cited papers by querying the DOIs of each reference list. The whole references corresponding to Table 10 are given in the ESI,† Doc 2.

If we look only at the lines corresponding to the two best citation counts, we notice that high values are found either for specific or for common references. For the 'organocatalysis' expression, the common reference lists collect the best cited papers but for the three other expressions the best cited paper is found in one or two specific reference lists. We are interested in these references that are assigned such high citation counts. First the reference (Table 10, column 5 and 11, line 5) displaying a 1320 citation count corresponds to the journal 'Chemical Reviews' from the ACS editor. Any usual abstract is given by the ACS and the WoS enables to retrieve this article owing to its Keywords Plus®. We came to the same conclusion for the article that was assigned a 259 citation count (Table 10, column 5 and 11, line 11).

For the paper that was assigned 755 citations (Table 10, columns 5 and 6, lines 5 and 11), Scopus designed its own abstract from the conclusion of the original paper – still an article from 'Chemical Reviews'. Finally the paper, that was

assigned 1358 citations (Table 10, columns 3, 7 and 9, line 15), was retrieved as a common reference by Scifinder and Scopus that both designed their own abstract containing the queried expression. These abstracts were however different. The WoS did not retrieve this paper because it seems to not design abstracts from scratch.

General trends may not be concluded from these few results, but highly cited papers are retrieved by all three editors. Moreover abstract and keyword indexing play a non-negligible role within these examples.

## 3.3. Influence of the similarity computing

We introduced a similarity parameter in the Iddup script – the Levenshtein distance – that may affect the duplicate counts when running either on a single reference list or a double one. This influence was studied by introducing increasing discrete values from 3 to 6, 9, 12, 18, 24 and 30% successively. In view of the great proportion of these duplicates obtained from Scifinder we concatenated the whole reference lists from this editor, the resulting file containing 5193 references. Submitting this file to Iddup computing furnished 243, 263, 269, 269, 270, 271 and 271 duplicates respectively. A maximum is reached approximately for a 24% threshold. Using a 12% threshold for the Levenshtein distance – the selected value all along this study – returned 270 duplicates and thus was different from just one unit of the maximum value (271).

Moreover a test was performed on the 'organocatalysis' expression and the results are summarized in Table 11. They correlate with the results of the previous paragraph but with an inverse trend. Low values of the threshold furnish a higher count of duplicates between Scifinder and another database because the count of unique articles from Scifinder is higher for low values of the Levenshtein distance. No difference was observed when comparing the Scopus's and the WoS's reference lists. It is worthwhile mentioning that the observed values for the common references (columns 3 and 6) as well as for

<figure><img src="image_9.png" /></figure>

<footnote>a Numbers within brackets correspond to specific articles reported in entries 2 of Tables 5–7.</footnote>
<table><caption>**Table 11** Influence of increasing Levenshtein distances on some duplicate reference lists</caption><thead><tr><th>Levenshtein<br>distance (%)</th><th>Scifinder</th><th>Common</th><th>WoS</th><th>Scifinder</th><th>Common</th><th>Scopus</th></tr></thead><tbody><tr><td>3</td><td>52</td><td>553</td><td>410</td><td>35</td><td>570</td><td>188</td></tr><tr><td>6</td><td>51</td><td>553</td><td>410</td><td>34</td><td>570</td><td>188</td></tr><tr><td>9</td><td>50</td><td>553</td><td>410</td><td>33</td><td>570</td><td>188</td></tr><tr><td>12</td><td>50</td><td>553</td><td>410</td><td>33</td><td>570</td><td>188</td></tr><tr><td>15</td><td>50</td><td>553</td><td>410</td><td>33</td><td>570</td><td>188</td></tr><tr><td>18</td><td>50</td><td>553</td><td>410</td><td>33</td><td>570</td><td>188</td></tr></tbody></table>

the WoS's and Scopus's specific references are stable. In a second time the variation of the Levenshtein distance only affects Scifinder's specific references for values less than or equal to 9.

## 4. Conclusions

Topic searches in chemical information systems are expected to return precise answers and we attempted to show in the first section of this paper how it can be challenging to query the web interfaces of Scifinder, Scopus and WoS using the most suitable syntax. If the personal learning involvement is shorter when starting a topic search with Scifinder, the higher precision of WoS's and Scopus's query languages may justify a slightly higher learning period. Crude results of these topic searches using simple terms or expressions up to four-term queries show rather uniform trends of the three information systems in retrieving large reference lists but with a noticeably greatest hit count retrieved by WoS over the whole answer sets. This feature results from a combined citation and semantic indexing affording new indexed terms that really expand capacities of topic searches. Though the coverage of common references retrieved by Scifinder, Scopus and WoS was shown to be incomplete owing mainly to keyword indexing (and to journal indexing though to a lesser extent), most of the references are shared by the three information systems including highly cited papers. Ideally they should be queried to get exhaustive answer lists or they should combine the powerful capacities of reliable thesauri and citation computing.³⁵

## Acknowledgements

Elsevier's French managers are gratefully acknowledged for having provided full access to Scopus's content during the period of March to April 2013.

## References

1. D. D. Ridley, *Trend. Anal. Chem.*, 2001, **20**, 1-10.

2. K. M. Whitley, *J. Am. Soc. Inf. Sci. Technol.*, 2002, **53**, 1210-1215.

3. J. Li, F. Burnham, T. Lemley and R. M. Britton, *J. Electron. Resour. Med. Libr.*, 2010, **7**, 196-217.

4. M. E. Falagas, E. I. Pitsouni, G. A. Malietzis and G. Pappas, *FASEB J.*, 2008, **22**, 338-342.

5. E. Zass, *Heterocycles*, 2010, **82**, 63-86.

6. References – CPlus – Worldwide coverage of many scientific disciplines all in one source: http://www.cas.org/content/references.

7. References – CPlus – Core Journals: http://www.cas.org/expertise/cascontent/caplus/corejournals.html.

8. In the non-academic STN version, indexing terms are reachable by querying the CA Lexicon®, a controlled term thesaurus. Thus expanding a searched term opens the query to further related terms. For a historical point of view the interested reader could refer to: D. Zaye, W. Metanomski and A. Beach, *J. Chem. Inf. Comput. Sci.*, 1985, **25**, 392-399.

9. D. D. Ridley, *Information Retrieval: SciFinder*, John Wiley & Sons Ltd, Chichester, 2nd edn, 2009, p. 39.

10. CAS Roles in CA<sup>SM</sup>/Caplus<sup>SM</sup>: http://www.cas.org/ASSETS/EB85B919049C4E448DCF8D391788F0DD/casroles.pdf.

11. CAS REGISTRY – The gold standard for chemical substance information: http://www.cas.org/content/chemical-substances.

12. D. D. Ridley, *Chem. Aust.*, 1996, **63**, 367-369.

13. Pubmed: http://www.ncbi.nlm.nih.gov/pubmed.

14. Pubmed – Medical Subject Headings: http://www.nlm.nih.gov/mesh/filelist.html.

15. The Thomson Reuters journal selection process: http://wokinfo.com/essays/journal-selection-process/.

16. Science Citation Index Expanded: http://ip-science.thomsonreuters.com/mjl/scope/scope_scie/.

17. The KeyWords Plus® are index terms introduced by Thomson Reuters in which the terms are algorithmically derived from the titles of articles cited by the author of the article being indexed. E. Garfield and I. H. Sher, *J. Am. Soc. Inf. Sci.*, 1993, **44**, 298-299.

18. Personal communication of G. Rivalle, Strategic Business Manager at Thomson Reuters.

19. Elsevier – Scopus: http://www.elsevier.com/online-tools/scopus.

20. Elsevier – Scopus's content overview: http://www.info.sci verse.com/scopus/scopus-in-detail/facts.

21. Elsevier – Content/Database overview: http://www.elsevier. com/online-tools/engineering-village/contentdatabase- overview.

22. Elsevier – Embase: http://www.elsevier.com/online-tools/ embase/about/emtree.

23. Elsevier – Geobase: http://www.elsevier.com/elsevier-products/ geobase.

24. R. Nelson, W. Hensel, D. Baron and A. Beach, *J. Chem. Inf. Comput. Sci.*, 1975, **15**, 85-94.

25. S. M. Cohen, D. L. Dayton and R. Salvador, *J. Chem. Inf. Comput. Sci.*, 1976, **16**, 93-99.

26. K. Baser, S. Cohen, D. Dayton and P. Watkins, *J. Chem. Inf. Comput. Sci.*, 1978, **18**, 18-25.

27. If the notion of concept in Scifinder is not clearly defined by the CAS, the definition of a concept heading may be found within the Help index as follows: a concept heading is one of the index terms that make up an index entry in CPlus<sup>SM</sup>. Scifinder searches the controlled term (CT) field in CPlus for concept heading information, which characterizes the general subject matter of the reference.

<figure><img src="image_10.png" /></figure>
28 J. Williams, *Online*, 1995, **19**, 60–66.

29 A. Ben Wagner, *J. Chem. Inf. Model.*, 2006, **46**, 767–774.

30 Except for the NOT Boolean operator that really excludes a specific term from queried terms.

31 A more comprehensive list of Scopus's databases is available within the Scopus Content Coverage Guide.

32 V. I. Levenshtein, *Sov. Phys. Dokl. (Engl. Transl.)*, 1966, **10**, 707–710.

33 A. Cavacini, *Scientometrics*, 2015, **102**, 2059–2071.

34 F. Franceschini, D. Maisano and L. Mastrogiacomo, *Scientometrics*, 2015, **102**, 2181–2186.

35 J. Qin, *J. Am. Soc. Inf. Sci.*, 2000, **51**, 166–180.