# Automatic extraction of materials and properties from superconductors scientific literature

Luca Foppiano, Pedro Baptista de Castro, Pedro Ortiz Suarez, Kensei

Terashima, Yoshihiko Takano, Masashi Ishii

► To cite this version:

Luca Foppiano, Pedro Baptista de Castro, Pedro Ortiz Suarez, Kensei Terashima, Yoshihiko Takano, et al.. Automatic extraction of materials and properties from superconductors scientific literature. Science and Technology of Advanced Materials: Methods, 2022, 10.1080/27660400.2022.2153633. hal-03776658v2

HAL Id: hal-03776658

https://hal.science/hal-03776658v2

Submitted on 20 Nov 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# RESEARCH PAPER

## Automatic extraction of materials and properties from superconductors scientific literature

Luca Foppiano<sup>a</sup>, Pedro Baptista de Castro<sup>b</sup>, Pedro Ortiz Suarez<sup>c</sup>, Kensei Terashima<sup>b</sup>, Yoshihiko Takano<sup>b</sup>, Masashi Ishii<sup>a</sup>

<sup>a</sup>Material Database Group, MaDIS, NIMS, Tsukuba, JP; <sup>b</sup>Nano Frontier Superconducting Materials Group, MANA, NIMS, Tsukuba, JP; <sup>c</sup>Data and Web Science Group, University of Mannheim, Mannheim, DE

### ARTICLE HISTORY

Compiled November 18, 2022

### ABSTRACT

The automatic extraction of materials and related properties from the scientific literature is gaining attention in data-driven materials science (Materials Informatics). In this paper, we discuss Grobid-superconductors, our solution for automatically extracting superconductor material names and respective properties from text. Built as a Grobid module, it combines machine learning and heuristic approaches in a multi-step architecture that supports input data as raw text or PDF documents. Using Grobid-superconductors, we built SuperCon<sup>2</sup>, a database of 40324 materials and properties records from 37700 papers. The material (or sample) information is represented by name, chemical formula, and material class, and is characterized by shape, doping, substitution variables for components, and substrate as adjoined information. The properties include the T<sub>c</sub> superconducting critical temperature and, when available, applied pressure with the T<sub>c</sub> measurement method.

### KEYWORDS

materials informatics, superconductors, machine learning, nlp, tdm

## 1. Introduction

In recent years, with the creation of computational databases, such as the Materials Project (MP) [1] and the Open Quantum Materials Database (OQMD) [2], and then experimental data repositories such as NIMS MDR (http://mdr.nims.go.jp) [3], focus has been steadily shifting towards a data-driven design of materials, which is often called Materials Informatics (MI). Such an approach is expected to accelerate the exploration of functional materials because it is not limited to the intuition or experience of very little genius researchers. In this new paradigm, the efficient use of data to guide experiments and material property prediction through the use of machine learning methods takes center stage. For example, data-driven methods have been used to search/design magneto-caloric materials [4–6], photo-catalysts for hydrogen splitting [7], thermoelectrics [8], and superconductors [9]. In such a data-driven search, one of the most important keys lies in the availability of the data, which should at least should consist of compositions of materials and their physical properties. In the
specific case of superconductivity, most of the data-driven works [9–11] rely on a single database: SuperCon (http://supercon.nims.go.jp).

SuperCon is a structured database of superconductor materials and properties; it was developed at the National Institute for Materials Science (NIMS) in Japan. At the time of writing this paper, SuperCon contained about 33000 inorganic and 600 organic materials and is the “de-facto” standard in data-driven research for superconductors materials (about 4400 articles contain the mention “*SuperCon database*” in Google Scholar). However, the SuperCon harvesting process is currently fully manual “from scratch”: humans have to read the human-readable printed matter such as PDF documents and enter the information into the system. The efficiency is directly proportional to the number of available human curators. Considering the cost of database construction, it is necessary to consider an assisted or alternative system that improves throughput while ensuring data quality equivalent to that of manual extraction.

As a solution, we are developing a hybrid data extraction method from scientific literature that combines automation using text data mining and manual curation. The automated system extracts and formats potential data and proposes them to the curator as “pre-cooked” structured data by (1) highlighting the relevant entities on the original document and (2) pre-filling the extracted information in a tabular format. In building the automated part of this hybrid system (training, evaluation) we used SuperMat, an annotated linked dataset from scholarly documents in superconductor research, which we recently constructed [12].

In this work, we present Grobid-superconductors: a system that automatically extracts structured information related to superconductor materials and properties from scientific literature. The tool is a specialised module of Grobid (Generation of Bibliographic Data) [13], a machine learning library designed to parse and structure scientific documents. Grobid provides an open-source platform for building specialised modules including astronomical entities recognition[14], dictionaries [15], software mentions [16], and physical measurements extraction [17]. Grobid provides several built-in features including access to PDF document layout information, citation resolution, bibliographic information consolidation through biblio-glutton [18] (a fast open-source reference matching service for CrossRef data), and a diverse set of machine learning (ML) architectures from a fast linear Conditional Random Field (CRF) to the latest state-of-the-art deep learning implementations.

Using Grobid-superconductors and other sub-tools, we established a pipeline to process a large number of documents and obtain an automated database of superconductor materials and properties. We processed 37770 papers from ArXiv (https://arxiv.org) and obtained a database of 40324 records. This new database, named SuperCon², can become an automated staging area for SuperCon, when bridged by a curation interface. During the project, there was also an opportunity to focus on properties that that have recently become of greater interest and that are underrepresented in SuperCon. For example, the “pressure” applied to obtain superconductivity (about 20 records in SuperCon), has gained attention because it can radically change the physical structure of a material. In addition, the “method” used to measure the superconducting transition temperature T_c (about 600 records in SuperCon) can be used to semantically recognise multiple T_c’s obtained from the same material or sample (e.g., distinguish calculated and experimental values of T_c).
## 2. Grobid-superconductors

We developed Grobid-superconductors as a Grobid module following principles (multi-step, sentence-based, full-text-based) discussed in a previous preliminary study [19]. Grobid has several advantages: 1) it can be integrated with pdfalto (https://github.com/kermitt2/pdfalto), a specialised tool for converting PDF to XML, which mitigates extraction issues such as the resolution of embedded fonts, invalid character encoding, and the reconstruction of the correct reading order, 2) it allows access to PDF document layout information for both machine learning and document decora-tion (e.g., coordinates in the PDF document); and, 3) it provides access to a set of high-quality, pre-trained machine learning models for structuring documents. Grobid-superconductors is structured as a three-steps process illustrated in Figure 1 and described in the Sections 2.1, 2.2, and 2.3.

**Abstract versus full-text** At the time of writing this paper, we are aware of related works that utilise text from abstracts as training data for machine learning. The main reason for using abstracts is that they are usually freely available as text [20], and contain condensed information [21,22]. Accurately parsing the full-text presents more challenges, however, but they are mitigated by Grobid and, the full-text contain a broader range of information, including the sample preparation process, negative results (e.g., absence of superconductivity for certain samples), and background information (e.g., reports on other materials from referenced works). Thus, grobid-superconductors is built to support full-text documents.

**Paragraphs versus sentences** Another question related to natural language processing (NLP) is whether to use sentence-based or paragraph-based text. While paragraphs can be extracted as part of the layout of PDF documents, obtaining sentences adds an additional step in which text is processed with a sentence segmenter. However, sentences are almost always shorter by definition, and in deep learning, this has advantages. In training and prediction, sentences will likely be shorter than the “max sequence length” limitation (e.g., 512 tokens for transformers). During training, sentences also use less memory and allow us to train models with a larger “batch size”, which has been shown to improve efficiency and obtain better results [23].

We chose to use sentence-based text in Grobid-superconductors after performing preliminary experiments on our tasks typologies, but on a smaller scale. For the Named Entities Recognition (NER) task we trained and evaluated a sequence labelling model for each version (paragraph-based and sentence-based) on four annotated documents (3/1 document partition for training/evaluation) from SuperMat [12]. As indicated in Table 1, the F1-score increased by 17.94 percentage points when the sentence-based text was used.

For the Entity Linking (EL) task, we want to maximise precision. In our previous work [24] we noticed that limiting linking entities within the same sentence (versus paragraph) would obtain higher precision (68.7% versus 57%) at the expense of lower recall (6.5% versus 10.7%), and F1-score (11.87% versus 18.01%). Therefore, in both our tasks we found evidence that a sentence-based dataset is more beneficial than paragraph-based dataset.
## 2.1. Document structuring and pre-processing

In the first step of our process, the PDF document is converted into an internal model based on a list of text statements, tokens, and features. The input document is processed using the Grobid original models, where we apply customised processes for document header and content. We select a subset of bibliographic information from the header: title, authors, DOI, publisher, journal, and year of publication, and we consolidate them via Grobid to match the publisher's quality (even by processing the "preprint version" of the publication). The superconductors entities extraction is applied to the content, only on relevant text items: title, abstract, text content from body or annexes, text content from figure and table captions (Figure 2).

We use the collected reference markers (also called reference callouts) from the text as features for improving the paragraph segmentation in sentences: the segmentation is cancelled if the end of sentence falls within the boundaries of a reference marker. For example, a sentence containing a reference in the form "Foppiano et. al." may be mistakenly segmented in the middle at the token "et.".

## 2.2. Named Entity Recognition

In the second step, the Named Entity Recognition (NER) task is performed on the previously extracted text.

### 2.2.1. Overview

As illustrated in Figure 3 the "superconductor parser" extracts the main superconductor-related information by aggregating the resulting entities from two ML models. The Superconductors ML model was developed based on the SuperMat schema [12], and the Quantity ML model was developed in a separated Grobid module for measurement extraction [17] and the output is limited to only temperatures and pressures. Overlapping entities are merged, exacted duplicates are removed, and the largest entities (in terms of string length) are preserved. The resulting entities are summarised in Table 2.

Entities of type `<material>`, which may contain mixed heterogeneous information, are passed in the cascade to the "Material parser" which aggregates ML and other tools. First, the entity is passed through a Material ML model to segment and identify its content (Table 3). Then, different processes are applied, depending on which information is available. These processes include the following:

* Formulas are decomposed into a structured composition. We identify each element-stoichiometry pair (e.g., "O": 7.0) using `mat2chem` [20] and `Pymatgen` [25]; if only the material name is available, we lookup its formula (e.g., hydrogen to `H`),

* Using heuristics, we classify the formula by assigning multiple classes as they are understood from superconductor researchers, for example cuprate, oxides, alloys, etc.

* Using the variables and values extracted, we substitute them into partial formulas. For example, in La 4 Fe 2 A 1-x O 7 (A=Mg,Co; x=0.1,0.2), we substitute A and x using their parsed values, and applying permutations, we obtain four resolved formulas: La 4 Fe 2 Mg 0.9 O 7, La 4 Fe 2 Mg 0.8 O 7, La 4 Fe 2 Co 0.9 O 7, and La 4 Fe 2 Co 0.8 O 7.
Finally, after all entities are extracted, the post-processing aggregates different men-
tions of the same materials using the parsed formulas at the document-level. For ex-
ample, formula with partial substitutions such as La 2 Fe 1-x 0 7 (x = 0.1, 0.2)
will be aggregated with materials like La 2 Fe 0.9 0 7 appearing in other sections
of the same document.

2.2.2. *Machine Learning study*

In this section we discuss the novel ML models we have trained for extracting spe-
cialised entities: the Superconductor ML model and the Material ML model (Figure 3).
SuperMat [12], our training dataset, contains 164 papers as of the time of writing and
is composed of annotated full-text and layout features from PDF documents.

For both ML models we trained and evaluated the following four architec-
ture/implementations: linear CRF (CRF), bidirectional LSTM with CRF [26]
(BidLSTM_CRF), bidirectional LSTM with CRF with Features [26] (the same
as (BidLSTM_CRF) with an additional input channel for features; BidL-
STM_CRF_FEATURES), and SciBERT [27] using a CRF as the activation layer (Scib-
ert).

The ML models are interfaced by Grobid, which uses the Wapiti[28] implementation for linear CRF, and DeLFT (Deep Learning For Text) [29] for deep learning models. The architectures CRF and BidLSTM_CRF_FEATURES make use of the orthogonal features we have summarised in Table A5.

Superconductor ML model

Holdout set The holdout set evaluation consists in using a fixed part of a dataset for validation. The selection must be performed to reproduce the same distribution of entities of the original dataset. We assembled the holdout set by manually selecting 32 documents (24%) from SuperMat, making sure they had a similar ratio of examples, entities and unique entities with the remaining 76% (132 documents) which was used as training set (Figure 4a). Maintaining the same rate for entity type distribution between the two sets was more challenging: on average, we obtained about 15-18% of labels of each type in the holdout set (Figure 4b), except for the `<material>` label (23%).

We defined the “out-of-domain” ratio as the number of unique entities from the holdout set that were not in the training set. The holdout set “out-of-domain” ratio was on average around 72%, which challenge the model generalisation (every 100 entities in the holdout set, 72 were never seen before during training). Most of the labels had an “out-of-domain” ratio above 50% (Figure 5); `<material>`, the most important label, had the highest ratio (82%) while `<me_method>` and `<pressure>` have the lowest (25% and 33%). The low ratio of `<me_method>` can be explained by their low entity variability (11.44%).

Positive sampling We trained the model with positive sampling by removing the examples without entities (negative examples, Figure 4a). This approach provided an improvement of 2% in both precision and recall as compared to the result without sampling when testing against the holdout set. Additional experiments with active and random sampling [16] with ratios of negative examples of 0.1, 0.25, 0.5 and 1.0 did not provide stable evidence suggesting scoring improvements when testing against
the holdout set.

**Evaluation** The best results were obtained by Scibert with an F1 of 77.03% and a recall of around 80.69% (Table 4). The features did not provide any improvements with RNN models: BidLSTM_CRF and BidLSTM_CRF FEATURES resulted in the same F1 score. This result comes as a surprise because features such as superscript/subscript were expected to be determinants for recognising material sequences.

The `<pressure>` label had the lowest performance scores in all architectures. We believe that 274 training examples are not a sufficient large number considering that pressure expressions can be dependent on the context because they can refer to different types of pressures (e.g., annealing pressure). The label with the highest score was `<material>`, with F1 values of 80.77% and 78.06% for Scibert and BidLSTM_CRF, respectively. In addition, `<material>` had the highest “out-of-domain” ratio in the holdout set (greater than 75%, Figure 5) and the highest “label variability” (the ratio between unique entities and total entities, about 42%), which suggests that the model recognises correctly materials that has not been “seen” during the training. On the other hand, the `<me_method>` label, which has lower “label variability” (around 11%) and a low “out-of-domain” ratio, had an F1 score of 66.56% with Scibert and 65.92% with BidLSTM_CRF. For `<tc>`, the CRF outperformed the other architectures (F1 score of 83.96%), especially Scibert (78.35%). This outcome can be explained by the extremely low variability (12.69%) of entities labelled as `<tc>`.

Scibert shows good generalisation capacity for unseen examples or examples appearing in different contexts. For example, in Figure 6a, only Scibert correctly extracts “above 100K”, while CRF misses it completely and BidLSTM_CRF misses “above”. In the training data, “above 100K” is not present, but “below 100K” and “100K” are present, and several other entities contain the token “above” and Scibert can understand that the token “above” is relevant to the temperature. In a second example (Figure 6b), only Scibert can correctly extract “W-C nanowire” which is not present in the SuperMat training data. Unfortunately, we cannot check whether “above 100K” or “W-C nanowire” are also present in the dataset used in the pre-train of SciBERT by their authors [27] because the data are not available.

**Material ML model** To train the Material ML model we created a special dataset with an additional layer of labels (Table 3), which included the material information represented by entities annotated as `<material>` in the SuperMat documents.

**Holdout set** In this model we created an independent holdout set because the manual annotation work is performed on smaller chunks of text and requires less effort than annotating sentences as when we developed SuperMat. We used material data extracted from a dataset of 500 documents (500-papers) from three publishers: *American Institute of Physics* (AIP), *American Physical Society* (APS) and *Institute of Physics* (IOP) [24]. The resulting holdout set has an average coverage greater than 25% (Figure 7) and an average “out-of-domain” ratio of 83.93% (Figure 8).

**Evaluation** Scibert obtained the best results, with F1 at 84.15% (Table 5). The inclusion of features in the BidLSTM_CRF architecture only improved results by less than 1% (from 83.13 to 83.76%). The label `<fabrication>` did not perform well with any architecture, most likely because it is too generic (Table 3), and the content is too heterogeneous. Another label, `<substrate>`, has only one-third of the training exam-
ples of <fabrication> but obtained results that were three times higher with Scibert, suggesting that <fabrication> should be split into separate and more homogeneous labels.

## 2.3. Entity Linking

Entity linking (EL) links materials and their corresponding properties.

We use a rule-based algorithm, but there are other approaches such as the use of dependency parsing [30–33]. We did not use these because it was difficult to find a suitable dependency parser for scientific texts, and complementary methods based on complex rule sets were needed to compensate for the poor performance of the parser.

In our algorithm, pairs of entities are linked focusing on three types of link:

* **material-tcValue**: The link between a material and its corresponding $T_c$.

* **tcValue-pressure**: The link between $T_c$ and its related critical pressure.

* **me_method-tcValue**: The link between $T_c$ and its corresponding measurement method.

Entities of type `<tcValue>` are pre-processed through a classifier that establishes whether or not they temperatures related to the superconductivity. This is used to exclude other temperatures (e.g., annealing, transition, Curie) which might be incorrectly extracted by the previous step. This rule-based classifier combines the extracted entities of $T_c$ expressions (label `<tc>`) with a set of predefined standard terms. If a temperature is not considered a $T_c$, it is excluded from the list of possible linking candidates.

Two scenarios are considered. First, if entities to be linked in the sentence are only two they are linked automatically, else further rules are applied. If the word “respectively” appears in the sentence, we apply “order-linking”. For example, consider the following sentence:

> P-or Ba-122 and Co-doped Ba-122 have lower $T_c$'s of about 30 K and 24 K, respectively, which makes helium free operation questionable.

It contains the word “respectively”, and by applying “order-linking”, P-or Ba122 is assigned to 30 K and Co-doped Ba-122 to 24 K.

If the word “respectively” does not appear in the sentence, we apply “distance-linking” which works by defining the distance measurement *d* as a value calculated as the numbers of characters between the centroid of each entity. Entities surrounded by parenthesis are expanded to the whole parenthesis, and its centroid is updated. As an example, in the sentence

> We tested two materials MgB2 ($T_c$ = 39 K) and FeSe ($T_c$ = 16 K).

39 K is closer to FeSe ($d$=10) than to MgB2 ($d$=11). In this example, however, both temperatures entities would be expanded to their containing parenthesis (e.g. “**39 K**” to “**(T_c = 39 K)**”. In this case the centre of the entity “**39 K**” is shifted toward the left, from the initial value of 38 to 35 and the distance from MgB2 is reduced from $d$=11 to $d$=8. As a result, the MgB2 entity is correctly linked to “**39 K**”.

The distance calculation is also adjusted with the addition of “penalties” by doubling the calculated distance when certain keywords or punctuations (“,”, “.”, “;”, “and”, “but”, “while”, “whereas”, “which”, “although”) appear between two entities because they represent a logical separation of predicates [34]. In the above example, the distance between 39 K and FeSe would be doubled ($d$=20) and the link would not
be made.

This rule-based linking was evaluated using the linked entities from Super-Mat [12] (Table 6) and is divided considering each link type. The F1 score for the **material-tcValue** was about 80% with a precision of 88.40%. **tcValue-pressure** F1 score was 3% lower than **material-tcValue** considering much less data available (support was 118 compared with 726).

## 2.4. End to end evaluation

End-to-end evaluation (E2EE) measures the capacity of the system from the PDF documents until the final linked results. We limited the scope of the E2EE to the triplet ‘material-T_c-pressure’ which, at the moment, is the backbone upon which the database is built. We performed the E2EE on the “500-papers” dataset where we manually examined the resulting database as follows: 1) we marked invalid records and 2) we identified the cause of failure from a predefined set of five error types (Figure 9):

* **From table:** the extracted text is wrongly extracted from a table. Although table content is ignored, the error rate from the Grobid library is still relevant due to the lack of training data.

* **Extraction:** entities are not recognised, wrongly recognised, or partially recognised.

* **Quantity extraction:** quantity entities (pressure, temperature) are not correctly extracted. We measured this error separately to identify the failure that could be shared with the Quantity ML model.

* **T_c classification:** the temperature is wrongly classified as superconducting T_c.

* **Linking:** given the initial steps were performed correctly, the resulting entities are not linked correctly.

The E2EE scores are summarised in Table 7. Recall is omitted because it is less relevant and difficult to calculate manually. The precision score (micro average) was 72.60% for all the subsections, although the error rates of figure captions (59.28%) and unknown subsections (57.14%) were clearly lower than those of the other subsections (> 70%). The ‘unknown’ subsections indicate that the extracted text’s structure was not well identified by Grobid but it was nevertheless aggregated. The overall score increases to 73% when excluding unknown subsections, 75.24% when excluding figure captions, and 79.14% when excluding both. Excluding these two subsections will not impact the amount of text, because both account for less than 20% of the total number of subsections.

The error types are summarised in Figure 10. The most common failures originate from T_c classification (40%), Linking (32%), and Extraction (20%). The most common T_c classification failures are as incorrect recognition of 1) relative values of T_c (e.g., 1 K higher than material X); 2) values indicating the transition temperature width (ΔT_c); 3) temperature values that are not T_c, for example, material synthesis temperatures (T), other critical transition temperatures that are not superconducting (e.g., T_Curie); and 4) values of temperature at which there is no superconductivity (e.g., “at 70 K there is no superconductivity”). “Linking errors” mainly occur when the text compare relative values of T_c using materials as the basis for comparison (e.g., “The T_c = 38 K is similar to the one of MgB_2”). Finally, “Extraction” issues mainly originate from: 1) implicit mention of the main material when experimented using different “sub- strates” combination, and 2) mismatches between <material> and <class> which,
by definition, overlap.

## 3. Supercon²

We created SuperCon² by processing 37770 research papers belonging to the category *cond-mat.supr-cond* in ArXiv. Currently SuperCon² contains 40324 records including 2052 triplets with applied pressure (*material-Tc-pressure*), and 3602 records with explicit measurement method (*material-Tc-measurement method*). The schema of SuperCon² is summarised with examples in Table 8.

The data is processed and ingested through the asynchronous Map-Reduce approach [35]. The “extraction task” (Map) processes the PDF documents by accessing Grobid-superconductors via REST API and stores their processed representation together with the original PDF document. Furthermore, the “aggregation task” (Reduce) reduces the document information into a synthesised tabular format. We store the processed document representation in JSON format. The processed documents are kept separately and used for displaying the enhanced PDF document (Figure 11). The pipeline uses a persistence layer for storage and reporting (logger).

We built a visualisation interface to exploit the extracted information. Users can search in the synthesised tabular data, access the PDF document enriched with the extracted information (Figure 11), and export locally in CSV, TSV and Microsoft Excel formats.

## 4. Conclusion

In this work, we present our solution for automatically building a database of materials and properties from scientific literature. Our contribution is composed of: 1) Grobid-superconductors, a specialised open-source system that processes PDF documents combining ML and rule-based methods to extract and link relevant information in superconductors research; 2) a pipeline allowing large-scale document processing; and 3) a visualisation interface for rapid data exploration, which includes PDF document information enrichment.

We made SuperCon², a database with 40324 records of superconductors materials and properties, including the applied pressure and the Tc measurement method. SuperCon² is available in text format at https://github.com/lfoppiano/supercon.

In the future, we plan to improve our tools by 1) extracting more properties, such as crystal structure type, space groups type, and lattice structure; 2) training supervised models for the “Linking step”; and 3) extending the interface to support data correction toward efficient curation. We confirmed the good generalisation ability of the Scibert architecture for the entity extraction task. Although we hope to obtain better results using materials science pre-trained BERT, such as MatSciBERT [36], the gain might be just minimal for relatively larger models [37].

## Acknowledgement

Our warmest thanks to Sae Dieb, who contributed with fruitful discussion on this project, and to Patrice Lopez, the author of Grobid [13], DeLFT [29], and other several interesting TDM projects for his continuous support and help with ideas, suggestions,
and discussions.

## Data and code availability

Grobid-superconductors is available on Github at https://github.com/lfoppiano/grobid-superconductors and the code is released under license Apache 2.0. SuperCon² is available in text format at https://github.com/lfoppiano/supercon.

## Competing interests

The authors declare no competing interests.

## References

[1] Jain A, Ong SP, Hautier G, et al. Commentary: The materials project: A materials genome approach to accelerating materials innovation. APL Materials. 2013;1(1):011002. Available from: https://doi.org/10.1063/1.4812323.

[2] Kirklin S, Saal JE, Meredig B, et al. The open quantum materials database (oqmd): assessing the accuracy of dft formation energies. npj Computational Materials. 2015; 1(1):1–15.

[3] Ranganathan A, Matsuda A, Tanifuji M, et al. The Development of an Integrated Next Generation Data Repository for Materials Science; 2019. Available from: https://doi.org/10.5281/zenodo.3553963.

[4] Bocarsly JD, Levin EE, Garcia CA, et al. A simple computational proxy for screening magnetocaloric compounds. Chemistry of Materials. 2017 2:29:1613–1622.

[5] de Castro PB, Terashima K, Yamamoto TD, et al. Machine-learning-guided discovery of the gigantic magnetocaloric effect in hob₂ near the hydrogen liquefaction temperature. NPG Asia Materials. 2020 12:12:35.

[6] Court CJ, Jain A, Cole JM. Inverse design of materials that exhibit the magnetocaloric effect by text-mining of the scientific literature and generative deep learning. Chemistry of Materials. 2021;33(18):7217–7231.

[7] Xiong Y, Campbell QT, Fanghanel J, et al. Optimizing accuracy and efficacy in data-driven materials discovery for the solar production of hydrogen. Energy & Environmental Science. 2021;14(4):2335–2348.

[8] Iwasaki Y, Takeuchi I, Stanev V, et al. Machine-learning guided discovery of a new thermoelectric material. Scientific reports. 2019;9(1):1–7.

[9] Stanev V, Oses C, Kusne A, et al. Machine learning modeling of superconducting critical temperature. npj Computational Materials. 2017;4.

[10] Le TD, Noumeir R, Quach HL, et al. Critical temperature prediction for a superconductor: A variational bayesian neural network approach. IEEE Transactions on Applied Superconductivity. 2020;30(4):1–5.

[11] Hamlin JJ. Superconductivity near room temperature. Nature. 2019;569:491–492.

[12] Foppiano L, Dieb S, Suzuki A, et al. Supermat: construction of a linked annotated dataset from superconductors-related publications. Science and Technology of Advanced Materials: Methods. 2021;1(1):34–44. Available from: https://doi.org/10.1080/27660400.2021.1918396.

[13] GROBID contributors. Grobid [https://github.com/kermitt2/grobid]; 2008–2022.

[14] Grobid Astro contributors. Grobid-astro: A machine learning software for extracting astronomical entities from scholarly documents [https://github.com/kermitt2/grobid-astro]; 2017–2022.
[15] Khemakhem M, Foppiano L, Romary L. Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields. In: electronic lexi-cography, eLex 2017; Sep.; Leiden, Netherlands; 2017. Available from: https://hal.archives-ouvertes.fr/hal-01508868.

[16] Lopez P, Du C, Cohoon J, et al. Mining software entities in scientific literature: Document-level ner for an extremely imbalance and large-scale task. New York, NY, USA: Association for Computing Machinery; 2021. p. 3986-3995. Available from: https://doi.org/10.1145/3459637.3481936.

[17] Foppiano L, Romary L, Ishii M, et al. Automatic identification and normalisation of physical measurements in scientific literature. In: Proceedings of the ACM Symposium on Document Engineering 2019; New York, NY, USA. Association for Computing Machinery; 2019. DocEng '19. Available from: https://doi.org/10.1145/3342558.3345411.

[18] Biblio Glutton Lookup contributors. Biblio glutton lookup [https://github.com/kermitt2/biblio-glutton]; 2018-2022.

[19] Foppiano L, Dieb TM, Suzuki A, et al. Proposal for Automatic Extraction Framework of Superconductors Related Information from Scientific Literature. In: The Institute of Electronics, Information and Communication (IEICE) - Services Computing Research Group - 2019, First Research Meeting, The 43rd Joint Meeting of MaDIS Research Exchange Group; May; Tsukuba, Japan; 2019. Available from: https://hal.archives-ouvertes.fr/hal-02870896.

[20] Kononova O, Huo H, He T, et al. Text-mined dataset of inorganic materials synthesis recipes. Scientific Data. 2019 Oct;6(1):203. Available from: https://doi.org/10.1038/s41597-019-0224-1.

[21] Yamaguchi K, Asahi R, Sasaki Y. SC-CoMIcs: A superconductivity corpus for materials informatics. In: Proceedings of the 12th Language Resources and Evaluation Conference; May; Marseille, France. European Language Resources Association; 2020. p. 6753-6760. Available from: https://www.aclweb.org/anthology/2020.lrec-1.834.

[22] Court CJ, Cole JM. Magnetic and superconducting phase diagrams and transition temperatures predicted using text mining and machine learning. npj Computational Materials. 2020 Mar;6(1):18. Available from: https://doi.org/10.1038/s41524-020-0287-8.

[23] Liu Y, Ott M, Goyal N, et al. Roberta: A robustly optimized bert pretraining approach; 2019. Available from: https://arxiv.org/abs/1907.11692.

[24] Foppiano L, Dieb TM, Suzuki A, et al. Proposal for automatic extraction framework of superconductors related information from scientific literature. THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS. 2019.

[25] Ong SP, Richards WD, Jain A, et al. Python materials genomics pymatgen: A robust open-source python library for materials analysis. Computational Materials Science. 2013 2:68-314-319. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0927025612006295.

[26] Lample G, Ballesteros M, Subramanian S, et al. Neural architectures for named entity recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; Jun.; San Diego, California. Association for Computational Linguistics; 2016. p. 260-270. Available from: https://aclanthology.org/N16-1030.

[27] Beltagy I, Lo K, Cohan A. SciBERT: A pretrained language model for scientific text. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP); Nov.; Hong Kong, China. Association for Computational Linguistics; 2019. p. 3615-3620. Available from: https://aclanthology.org/D19-1371.

[28] Lavergne T, Cappé O, Yvon F. Practical very large scale CRFs. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL); July. Association for Computational Linguistics; 2010. p. 504-513. Available from: http://www.aclweb.org/anthology/P10-1052.

[29] DeLFT contributors. Delft [https://github.com/kermitt2/delft]; 2018-2022.
[30] Yoshikawa M, Noji H, Matsumoto Y. A* ccg parsing with a supertag and dependency factored model. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics; 2017. p. 277–287. Available from: http://aclweb.org/anthology/P17-1026.

[31] Tiktinsky A, Goldberg Y, Tsarfaty R. pybart: Evidence-based syntactic transformations for ie. In: ACL; 2020.

[32] Swayamdipta S, Thomson S, Dyer C, et al. Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold. arXiv preprint arXiv:170609528. 2017;.

[33] Zhou J, Zhao H. Head-driven phrase structure grammar parsing on Penn treebank. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics; Jul.; Florence, Italy. Association for Computational Linguistics; 2019.

[34] Oka H, Yoshizawa A, Shindo H, et al. Automatic extraction of solvent-names on polymer's solubility from academic articles. In: The 66th JSAP Spring Meeting 2019; Mar.; Tokyo, Japan. Japan Society of Applied Physics; 2019.

[35] Dean J, Ghemawat S. Mapreduce: Simplified data processing on large clusters. Commun ACM. 2008 jan;51(1):107–113. Available from: https://doi.org/10.1145/1327452.1327492.

[36] Gupta T, Zaki M, Krishnan NMA, et al. MatSciBERT: A materials domain language model for text mining and information extraction. npj Computational Materials. 2022 May;8(1):102. Available from: https://doi.org/10.1038/s41524-022-00784-w.

[37] Hong Z, Ajith A, Pauloski G, et al. Scholarbert: Bigger is not always better ; 2022. Available from: https://arxiv.org/abs/2205.11342.

[38] Gajda G, Morawski A, Rogacki K, et al. Ag-doped FeSe$_{0.94}$ polycrystalline samples obtained through hot isostatic pressing with improved grain connectivity. Superconductor Science and Technology. 2016 jul;29(9):095002. Available from: https://doi.org/10.1088/0953-2048/29/9/095002.

[39] Shibata Y, Nomura S, Ishiguro R, et al. Magnetic field imaging of a tungsten carbide film by scanning nano-SQUID microscope. Superconductor Science and Technology. 2016 aug; 29(10):104004. Available from: https://doi.org/10.1088/0953-2048/29/10/104004.

[40] Durajski AP, Szczśniak R. Structural, electronic, vibrational, and superconducting properties of hydrogenated chlorine. The Journal of Chemical Physics. 2018;149(7):074101. Available from: https://doi.org/10.1063/1.5031202.

[41] Swain MC, Cole JM. Chemdataextractor: A toolkit for automated extraction of chemical information from the scientific literature. Journal of Chemical Information and Modeling. 2016;56(10):1894–1904. PMID: 27669338; Available from: https://doi.org/10.1021/acs.jcim.6b00207.

Figures & Tables

<figure><img src="image_2.png" /><figcaption>**Figure 1.** Processing pipeline for extracting superconductors materials and properties.</figcaption></figure>
<table><caption>**Table 1.** Results from cross-validation for sentence-based and paragraphs-based text.</caption><thead><tr><td>Label</td><td>Precision</td><td>Recall</td><td>F1</td></tr></thead><tbody><tr><td>Paragraph-based micro avg.</td><td>44.44</td><td>27.21</td><td>33.76</td></tr><tr><td>Sentence-based micro avg.</td><td>48.41</td><td>50.00</td><td>51.70</td></tr></tbody></table>

<figure><img src="image_3.png" /><figcaption>**Figure 2.** Grobid-superconductors extraction processes (bibliographic information, superconductor entity extraction and sentence segmentation) within the Grobid cascade data flow.</figcaption></figure>

<table><caption>**Table 2.** Entities extracted by the superconductors parser.</caption><thead><tr><td>Entity (tag)</td><td>Description</td></tr></thead><tbody><tr><td>Material (&lt;material&gt;)</td><td>Materials and samples names, formulas (including stoichiometric formulas), substitution variables of values and elements, shape, doping, and substrate</td></tr><tr><td>Class (&lt;class&gt;)</td><td>Groups of materials having similar characteristics or common strategic compounds that define their nature</td></tr><tr><td>T<sub>c</sub> value (&lt;tcValue&gt;)</td><td>The value of the superconductor critical temperature</td></tr><tr><td>T<sub>c</sub> expressions (&lt;tc&gt;)</td><td>Expressions in the text that provide information about the phenomenon of superconductivity related to a value, interval or variation of the T<sub>c</sub></td></tr><tr><td>Measurement method (&lt;me_method&gt;)</td><td>Technique used to measure or calculate the presence of superconductivity</td></tr><tr><td>Applied pressure (&lt;pressure&gt;)</td><td>Applied pressure when superconductivity is recorded</td></tr></tbody></table>

<table><caption>**Table 3.** Entities extracted by the material parser.</caption><thead><tr><td>Entity (tag)</td><td>Description</td></tr></thead><tbody><tr><td>Name (&lt;name&gt;)</td><td>The canonical name of a material (e.g., hydrogen, PCCO, carbon)</td></tr><tr><td>Formula (&lt;formula&gt;)</td><td>Chemical formula of the material (e.g., Pr1.869Ce0.131Cu0 4-, MgB2, La 2-x Sr x CuO 4)</td></tr><tr><td>Doping (&lt;doping&gt;)</td><td>Doping ratio and doping materials that are adjoined to the material name (e.g., Zn-doped, 2% Zn-doped)</td></tr><tr><td>Shape (&lt;shape&gt;)</td><td>shape of the material (e.g. single crystal, polycrystalline, thin film, powder, film)</td></tr><tr><td>Substitution variables (&lt;variable&gt;)</td><td>Variables that can be substituted in the formula.</td></tr><tr><td>Substitution values (&lt;value&gt;)</td><td>Values expressed in the doping.</td></tr><tr><td>Substrate (&lt;substrate&gt;)</td><td>Substrates as defined in the material name</td></tr><tr><td>Fabrication (&lt;fabrication&gt;)</td><td>Additional information that does not belong to any of the previous tags (e.g., intercalated, electron-doped)</td></tr></tbody></table>
<figure><img src="image_4.png" /><figcaption>**Figure 3.** Cascade architecture in the NER step. The white rectangles indicate the extracted information (described in Tables 2 and 3).</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Figure 4.** Holdout/training set distribution for (a) general metrics and (b) entity labels; entities and unique entities indicate the number of labelled entities with and without value duplicates, respectively, and positive examples (+) and negative examples (-) indicate the number of sentences with at least one entity and with no entities, respectively.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Figure 5.** Holdout “out-of-domain” rates. The entities from the holdout set that are also in the training set are “in-domain” and the entities that are not in the training set are “out-of-domain”.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**Figure 6.** Examples taken from two sources [38,39] of results from three different architectures: CRF, BidLSTM-CRF and, Scibert. The boxes annotating the text represent the extracted entities (material are indicated in light blue, T<sub>c</sub> in green, and T<sub>c</sub> expressions in yellow).</figcaption></figure>

<table><caption>**Table 4.** Evaluation scores (%) for the Superconductor ML model in the four architectures. For the DL architecture the results are averaged over 5 runs. Support (Supp) indicates the number of labels in the training data. Values in bold indicate the highest score. P: precision, R: recall.</caption><thead><tr><th rowspan="2">Label</th><th colspan="3">CRF</th><th colspan="3">BidLSTM_CRF</th><th colspan="3">BidLSTM_CRF FEATURES</th><th colspan="3">Scibert</th><th rowspan="2">Supp</th></tr><tr><th>P</th><th>R</th><th>F1</th><th>P</th><th>R</th><th>F1</th><th>P</th><th>R</th><th>F1</th><th>P</th><th>R</th><th>F1</th></tr></thead><tbody><tr><td>&lt;class&gt;</td><td>79.74</td><td>66.79</td><td>72.69</td><td>79.01</td><td>72.62</td><td><strong>75.66</strong></td><td>77.84</td><td>72.40</td><td>74.97</td><td>72.95</td><td>75.28</td><td>74.09</td><td>1646</td></tr><tr><td>&lt;material&gt;</td><td>79</td><td>72.15</td><td>75.42</td><td>79.25</td><td>76.94</td><td>78.06</td><td>81.07</td><td>75.10</td><td>77.94</td><td>80.15</td><td>81.42</td><td><strong>80.77</strong></td><td>6943</td></tr><tr><td>&lt;me_method&gt;</td><td>60.25</td><td>68.73</td><td>64.21</td><td>56.41</td><td>79.49</td><td>65.92</td><td>55.86</td><td>80.45</td><td>65.90</td><td>56.26</td><td>81.52</td><td><strong>66.56</strong></td><td>1883</td></tr><tr><td>&lt;pressure&gt;</td><td>46.15</td><td>29.27</td><td>35.82</td><td>49.45</td><td>58.05</td><td>52.53</td><td>50.25</td><td>60.49</td><td><strong>54.36</strong></td><td>41.72</td><td>52.68</td><td>46.51</td><td>274</td></tr><tr><td>&lt;tc&gt;</td><td>84.36</td><td>83.57</td><td><strong>83.96</strong></td><td>78.61</td><td>82.54</td><td>80.48</td><td>79.19</td><td>82.07</td><td>80.60</td><td>74.46</td><td>82.66</td><td>78.35</td><td>3741</td></tr><tr><td>&lt;tcValue&gt;</td><td>69.8</td><td>66.24</td><td>67.97</td><td>70.36</td><td>75.16</td><td>72.67</td><td>68.95</td><td>76.56</td><td>72.52</td><td>70.90</td><td>79.74</td><td><strong>75.06</strong></td><td>1099</td></tr><tr><td>All (micro avg)</td><td>76.88</td><td>72.77</td><td>74.77</td><td>74.59</td><td>77.67</td><td>76.09</td><td><strong>75.17</strong></td><td>76.79</td><td>75.96</td><td>73.69</td><td><strong>80.69</strong></td><td><strong>77.03</strong></td><td></td></tr></tbody></table>

<figure><img src="image_8.png" /><figcaption>**Figure 7.** Holdout/training set for the Material ML model: (a) general metrics and (b) entity labels.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>**Figure 8.** Holdout “out-of-domain” rates for the Material ML model. The entities from the holdout set that are also in the training set are the in-domain, and the entities that are not in the training set are the out-of-domain.</figcaption></figure>

<table><caption>**Table 5.** Evaluation scores (%) of the Material ML model with holdout set. Support (Supp) indicates the number of labels in the training data. Values in bold indicate the highest score. P: precision, R: recall.</caption><thead><tr><th rowspan="2">Label</th><th colspan="3">CRF</th><th colspan="3">BidLSTM_CRF</th><th colspan="3">BidLSTM_CRF FEATURES</th><th colspan="3">SciBERT</th><th rowspan="2">Supp</th></tr><tr><th>P</th><th>R</th><th>F1</th><th>P</th><th>R</th><th>F1</th><th>P</th><th>R</th><th>F1</th><th>P</th><th>R</th><th>F1</th></tr></thead><tbody><tr><td>&lt;doping&gt;</td><td>60.41</td><td>55.85</td><td>58.04</td><td>67.98</td><td>62.42</td><td>64.95</td><td>69.00</td><td>62.34</td><td><b>65.43</b></td><td>63.58</td><td>62.79</td><td>63.16</td><td>792</td></tr><tr><td>&lt;fabrication&gt;</td><td>40.00</td><td>4.55</td><td>8.16</td><td>23.61</td><td>5.91</td><td>9.24</td><td>37.33</td><td>9.09</td><td>14.48</td><td>22.51</td><td>13.18</td><td><b>16.52</b></td><td>94</td></tr><tr><td>&lt;formula&gt;</td><td>80.81</td><td>82.29</td><td>81.54</td><td>82.59</td><td>84.14</td><td>83.35</td><td>83.83</td><td>85.14</td><td>84.47</td><td>84.53</td><td>86.56</td><td><b>85.53</b></td><td>6301</td></tr><tr><td>&lt;name&gt;</td><td>72.2</td><td>63.75</td><td>67.71</td><td>76.29</td><td>78.76</td><td>77.43</td><td>74.51</td><td>80.38</td><td>77.33</td><td>77.18</td><td>81.86</td><td><b>79.44</b></td><td>1930</td></tr><tr><td>&lt;shape&gt;</td><td>90.89</td><td>92.51</td><td>91.69</td><td>90.93</td><td>95.79</td><td><b>93.29</b></td><td>90.33</td><td>95.74</td><td>92.96</td><td>89.67</td><td>97.20</td><td>93.28</td><td>809</td></tr><tr><td>&lt;substrate&gt;</td><td>37.04</td><td>6.76</td><td>11.43</td><td>54.31</td><td>32.43</td><td>40.44</td><td>60.08</td><td>33.38</td><td>42.82</td><td>56.32</td><td>41.22</td><td><b>47.59</b></td><td>32</td></tr><tr><td>&lt;value&gt;</td><td>80.21</td><td>83.15</td><td>81.65</td><td>84.81</td><td>89.33</td><td>86.99</td><td>85.16</td><td>90.15</td><td><b>87.58</b></td><td>83.14</td><td>85.92</td><td>84.50</td><td>1895</td></tr><tr><td>&lt;variable&gt;</td><td>96.85</td><td>95.98</td><td>96.41</td><td>95.19</td><td>97.77</td><td>96.46</td><td>96.32</td><td>97.90</td><td><b>97.10</b></td><td>96.22</td><td>96.52</td><td>96.37</td><td>1795</td></tr><tr><td>All (micro avg)</td><td>81.15</td><td>78.09</td><td>79.59</td><td>82.76</td><td>83.50</td><td>83.13</td><td><b>83.20</b></td><td>84.33</td><td>83.76</td><td>83.11</td><td><b>85.23</b></td><td><b>84.15</b></td><td></td></tr></tbody></table>

<table><caption>**Table 6.** Evaluation scores for the Linking. Support (Supp) indicates the number of labels in the training data. Values in bold indicate the highest score. P: precision, R: recall.</caption><thead><tr><th>Relationship type</th><th>P</th><th>R</th><th>F1-</th><th>Supp</th></tr></thead><tbody><tr><td>material-tcValue</td><td>88.40</td><td>74.52</td><td>80.87</td><td>726</td></tr><tr><td>tcValue-pressure</td><td>85.71</td><td>71.52</td><td>77.98</td><td>118</td></tr><tr><td>me_method-tcValue</td><td>62.28</td><td>65.74</td><td>63.96</td><td>151</td></tr></tbody></table>

<figure><img src="image_10.png" /><figcaption>**Figure 9.** *Error types* in the context of the data flow.</figcaption></figure>
<table><caption>**Table 7.** Summary of the E2EE evaluation scores. Support indicates the number of labels in the training data.</caption><thead><tr><th>Subsection</th><th>Precision</th><th>Support</th></tr></thead><tbody><tr><td>Title</td><td>100</td><td>2</td></tr><tr><td>Abstract</td><td>80.32</td><td>61</td></tr><tr><td>Paragraph</td><td>75.2</td><td>623</td></tr><tr><td>Figure captions</td><td>59.28</td><td>140</td></tr><tr><td>Unknown</td><td>57.14</td><td>21</td></tr><tr><td>Micro avg.</td><td>72.60</td><td>847</td></tr><tr><td>Micro avg. (excl. figures)</td><td>75.24</td><td>707</td></tr><tr><td>Micro avg. (excl. unknown sections)</td><td>73.00</td><td>603</td></tr><tr><td>Micro avg. (excl. figures and unknown sections)</td><td>79.14</td><td>657</td></tr></tbody></table>

<figure><img src="image_11.png" /><figcaption>**Figure 10.** Error type distribution in the E2EE of the 500-papers dataset.</figcaption></figure>

Structural, electronic, vibrational, and superconducting properties of hydrogenated chlorine

Artur P. Durajski[1] and Radosław Szczęśniak

Institute of Physics, Częstochowa University of Technology, Ave. Armii Krajowej 19,
42-200 Częstochowa, Poland

(Received 29 March 2018; accepted 29 July 2018; published online 16 August 2018)

Recent measurements have set a new record for the superconducting transition temperature ($T_c$) at which a material loses electrical resistivity and exhibits ideal diamagnetism. Theory-oriented experiments show that the compressed hydride of Group VI (hydrogen sulfide, $H_2S$) exhibits a superconducting state at 203 K. Moreover, a Group V hydride (phosphorus hydride, $PH_3$) has also been studied and its $T_c$ reached a maximum of 103 K. The experimental realisation of the superconductivity in $H_2S$ and $PH_3$ inspired us to search for other hydride superconductors. Herein, we report theoretical studies of the electronic, vibrational, and superconducting properties of hydrogenated chlorine ($H_2Cl$), representative of the Group VII hydride. First-principles calculations performed for $H_2Cl$ in the pressure range [150-250 GPa] show that the investigated $Im\bar{3}m$ phase has a large electron-phonon coupling parameter and the resulting application of the Mjdal-Eliashberg formalism yields a remarkably high superconducting temperature of 198 K at [150 GPa]. Published by AIP Publishing, https://doi.org/10.1063/1.5031202

I. INTRODUCTION

Searching for the superconducting state at critical temperature as high as possible is currently one of the major activities in condensed matter physics.¹⁻⁶ The great breakthrough in this area occurred in 2014 when the hydrogen sulfide ($H_2S$) was predicted theoretically by Li et al. to be a novel conventional high-temperature superconductor with an estimated maximal transition temperature of about 80 K at [160 GPa].⁷

Shortly after this theoretical report, compression in a diamond anvil cell of a sulfur hydride system leads to the confirmation of these predictions and direct experimental observation of two superconducting states: (i) $T_c$ = [30-150 K] for low-temperature prepared samples and (ii) $T_c$ = [170-203 K] for room-temperature prepared samples.⁸ The superconduct-

electrical resistance measurements¹,¹³,¹⁵,¹⁶ confirming that the stable $Im\bar{3}m$ phase is responsible for high-$T_c$ superconductivity. Interestingly, Guigue *et al.* conducted experiments which employed direct synthesis of pure $H_2S$ from S and H elements.¹⁷ At high pressure, the obtained $H_2S$ samples are identified to have the $Ccc\bar{m}$ phase up to [160 GPa]. On this basis, Guigue *et al.* suggested that the body-centered (cubic) $Im\bar{3}m$ structure is rather more metastable than the thermodynamic ground state.⁴,¹⁷ Most recently, Goncharov *et al.* reported that $Ccc\bar{m}$ is admittedly stable in a wide pressure range, but unlike the previous observations of Guigue *et al.*, they found that $Im\bar{3}m$ $H_2S$ is the most favorable crystalline phase above [40 GPa].¹⁸

The measured critical temperature exhibits a pronounced isotope shift consistent with BCS theory.¹⁹,²⁰ This fact allows considered $H_2S$ to be a conventional phonon-mediated super-

material †

name: H₂ Cl

Linked: 198 K (tCValue)

[simple]

class: Alloys, Hydrides

formula: H 3 Cl

<caption>**Figure 11.** Example of a superconductors research PDF document [40] enriched with extracted annotations. Materials information (class, formula) and properties ($T_c$) are summarised in the information box when the users click on the highlighted annotated entity in the text.</caption>
<table><caption>**Table 8.** Summary and description of the SuperCon² schema. “Internal information” is technical information not accessible to the users.</caption><thead><tr><th>Field name</th><th>Description</th><th>Examples</th></tr></thead><tbody><tr><td colspan="3"><strong>Material information</strong></td></tr><tr><td>Raw material</td><td>The material or sample as it appears in the text</td><td></td></tr><tr><td>Name</td><td>Canonical name of a material</td><td>PCCO, PCO, Metal diboride, hydrogen, carbon</td></tr><tr><td>Formula</td><td>Material expressed as chemical formula. This includes also formulas with stoichiometric variables</td><td>Pr<sub>1.869</sub>Ce<sub>0.131</sub>CuO<sub>4</sub> - δ, MgB<sub>2</sub>, La<sub>2-x</sub>Sr<sub>x</sub>CuO<sub>4</sub></td></tr><tr><td>Doping</td><td>Doping ratio and doping materials that might be adjoined to the material</td><td>Overdoped, underdoped, optimally doped, bulk, pure, 1% Zn, Zn (from Zn-doped XYZ)</td></tr><tr><td>Shape</td><td>The shape of the material or the sample</td><td>Single crystal, polycrystal, wire, powder, film</td></tr><tr><td>Variables</td><td>Variables that can be substituted in the formula</td><td>x = 0, RE=Ln,St</td></tr><tr><td>Class</td><td>Material classification according to the domain-experts taxonomy</td><td>cuprates, oxides, and alloys</td></tr><tr><td>Fabrication</td><td>All the information that does not belong to any of the previous tags</td><td>Intercalated, synthesized by MBE method, electron-doped, hole-doped</td></tr><tr><td>Substrate</td><td>Substrate material described in the raw material</td><td>PCCO films onto Pr<sub>2</sub>CuO<sub>4</sub>(PCO)/SrTiO<sub>3</sub></td></tr><tr><td colspan="3"><strong>Properties</strong></td></tr><tr><td>Critical Temperature</td><td>Superconducting critical temperature</td><td></td></tr><tr><td>Applied Pressure</td><td>Pressure applied when measuring the superconducting critical temperature</td><td></td></tr><tr><td>Measurement Method</td><td>Method for measurement of the superconducting critical temperature</td><td>Magnetic susceptibility, specific heat, calculation, prediction, resistivity</td></tr><tr><td colspan="3"><strong>Document bibliographic information</strong></td></tr><tr><td>Section</td><td>The main body section of the paper</td><td>Header, body, annex</td></tr><tr><td>Subsection</td><td>The secondary segmentation area of the paper</td><td>Paragraph, table caption, figure caption, title, abstract</td></tr><tr><td>Authors, Title, DOI, Publisher, Journal, Year</td><td colspan="2">Bibliographic information of the document</td></tr><tr><td colspan="3"><strong>Internal information</strong></td></tr><tr><td>Hash, Timestamp</td><td colspan="2">Hash calculated on the binary content of the original PDF document and the timestamp when the document was processed.</td></tr></tbody></table>
# Appendix A. Additional information

<table><caption>**Table A1.** Holdout/Training set distribution (%) between training and holdout sets for the Superconductor ML model. Positive examples indicate the number of sentences with at least one entity, and negative examples the number of sentences with no entities.</caption><thead><tr><th></th><th>training</th><th>holdout</th><th>% holdout/training</th></tr></thead><tbody><tr><td>documents</td><td>132</td><td>32</td><td>24.24%</td></tr><tr><td>examples</td><td>16902</td><td>3905</td><td>23.10%</td></tr><tr><td>entities</td><td>15586</td><td>3112</td><td>19.97%</td></tr><tr><td>unique entities</td><td>6699</td><td>1372</td><td>20.48%</td></tr><tr><td>positive examples</td><td>8380</td><td>1776</td><td>21.19%</td></tr><tr><td>negative examples</td><td>8522</td><td>2129</td><td>24.98%</td></tr></tbody></table>

<table><caption>**Table A2.** Holdout/Training set distribution (%) between training and holdout sets on different labels for the Superconductors ML model.</caption><thead><tr><th>label</th><th>training</th><th>holdout</th><th>% holdout/training</th></tr></thead><tbody><tr><td>&lt;tc&gt;</td><td>3741</td><td>639</td><td>17.08%</td></tr><tr><td>&lt;class&gt;</td><td>1646</td><td>271</td><td>16.46%</td></tr><tr><td>&lt;material&gt;</td><td>6943</td><td>1649</td><td>23.75%</td></tr><tr><td>&lt;me_method&gt;</td><td>1883</td><td>355</td><td>18.85%</td></tr><tr><td>&lt;tcValue&gt;</td><td>1099</td><td>157</td><td>14.29%</td></tr><tr><td>&lt;pressure&gt;</td><td>274</td><td>41</td><td>14.96%</td></tr></tbody></table>

<table><caption>**Table A3.** Holdout/Training set distribution (%) training and holdout sets for the Material ML model.</caption><thead><tr><th></th><th>training</th><th>holdout</th><th>% holdout/training</th></tr></thead><tbody><tr><td>examples</td><td>13648</td><td>5728</td><td>41.97%</td></tr><tr><td>entities</td><td>4512</td><td>2817</td><td>62.43%</td></tr><tr><td>unique entities</td><td>9268</td><td>3292</td><td>35.52%</td></tr></tbody></table>
<table><caption>**Table A4.** Holdout/Training set distribution (%) training and hold-out sets on different labels for the Material ML model.</caption><thead><tr><th>label</th><th>training</th><th>holdout</th><th>% holdout/training</th></tr></thead><tbody><tr><td>&lt;fabrication&gt;</td><td>94</td><td>44</td><td>46.81%</td></tr><tr><td>&lt;formula&gt;</td><td>6301</td><td>2569</td><td>40.77%</td></tr><tr><td>&lt;shape&gt;</td><td>809</td><td>841</td><td>103.96%</td></tr><tr><td>&lt;substrate&gt;</td><td>32</td><td>148</td><td>462.50%</td></tr><tr><td>&lt;name&gt;</td><td>1930</td><td>949</td><td>49.17%</td></tr><tr><td>&lt;variable&gt;</td><td>1795</td><td>449</td><td>25.01%</td></tr><tr><td>&lt;value&gt;</td><td>1895</td><td>463</td><td>24.43%</td></tr><tr><td>&lt;doping&gt;</td><td>792</td><td>265</td><td>33.46%</td></tr></tbody></table>

<table><caption>**Table A5.** Summary of the features used in the *superconductors* and *material* ML models. All under Architecture indicate only BidLSTM_CRF FEATURES and CRF.</caption><thead><tr><th>#</th><th>Feature</th><th>Model</th><th>Architecture</th></tr></thead><tbody><tr><td>1</td><td>current token</td><td>all</td><td>all</td></tr><tr><td>2</td><td>current token lower cased</td><td>all</td><td>all</td></tr><tr><td>3-6</td><td>(four features) current token, prefix characters 1 to 4</td><td>all</td><td>CRF</td></tr><tr><td>7-10</td><td>(four features) current token, suffix characters 1 to 4</td><td>all</td><td>CRF</td></tr><tr><td>11</td><td>information about capitalisation: first character (INITCAP), all characters (ALLCAPS), none (NOCAPS)</td><td>all</td><td>all</td></tr><tr><td>12</td><td>digits content: all (ALLDIGIT), some digits (CONTAINDIGIT), no digits (NODIGIT)</td><td>all</td><td>all</td></tr><tr><td>13</td><td>(boolean) the token is composed by a single character</td><td>all</td><td>all</td></tr><tr><td>14</td><td>punctuation information and normalisation to placeholders: no punctuation (NOPUNCT), open or end brackets (OPENBRACKET, ENDBRACKET), various punctuation (DOT, COMMA, HYPHEN, QUOTE), open or close quotes (OPENQUOTE, ENDQUOTE), anything else (PUNCT)</td><td>all</td><td>all</td></tr><tr><td>15</td><td>Shadow the numbers</td><td>all</td><td>CRF</td></tr><tr><td>16</td><td>Shadow any characters: "x" for lowercase, "X" for uppercase, "d" for digits</td><td>all</td><td>CRF</td></tr><tr><td>17</td><td>As the previous but compressed</td><td>all</td><td>CRF</td></tr><tr><td>18</td><td>Font name</td><td>superconductors</td><td>all</td></tr><tr><td>19</td><td>Font size</td><td>superconductors</td><td>all</td></tr><tr><td>20</td><td>Font style: standard (BASELINE), superscript (SUPERSCRIPT) or subscript (SUBSCRIPT)</td><td>superconductors</td><td>all</td></tr><tr><td>21</td><td>(boolean) if the token style is bold</td><td>superconductors</td><td>all</td></tr><tr><td>22</td><td>(boolean) if the token style is italic</td><td>superconductors</td><td>all</td></tr><tr><td>23</td><td>(boolean) the token is identified as a chemical compound by ChemDataExtractor[41]</td><td>superconductors</td><td>all</td></tr></tbody></table>