Incorporating Domain Knowledge into Materials Tokenization
Authors: Yerim Oh, Jun-Hyung Park, Junho Kim, SungHo Kim, SangKeun Lee
Accepted at: acl25

Incorporating Domain Knowledge into Materials Tokenization

Yerim Oh1    Jun-Hyung Park2    Junho Kim1    SungHo Kim1    SangKeun Lee1,3
1Department of Artificial Intelligence, Korea University
2Division of Language & AI, Hankuk University of Foreign Studies
3Department of Computer Science and Engineering, Korea University
{yerim0210, monocrat, sungho3268, yalphy}@korea.ac.kr, jhp@hufs.ac.kr

Abstract
While language models are increasingly utilized in materials science, typical models rely on frequency-centric tokenization methods originally developed for natural language processing. However, these methods frequently produce excessive fragmentation and semantic loss, failing to maintain the structural and semantic integrity of material concepts. To address this issue, we propose MATTER, a novel tokenization approach that integrates material knowledge into tokenization. Based on MatDetector trained on our materials knowledge base and a re-ranking method prioritizing material concepts in token merging, MATTER maintains the structural integrity of identified material concepts and prevents fragmentation during tokenization, ensuring their semantic meaning remains intact. The experimental results demonstrate that MATTER outperforms existing tokenization methods, achieving an average performance gain of 4% and 2% in the generation and classification tasks, respectively. These results underscore the importance of domain knowledge for tokenization strategies in scientific text processing.1

1 Introduction
Recent advances in language models have expanded their applications in materials science (Pillania, 2021; Olivetti et al., 2020). However, typical language models for materials science utilize frequency-centric subword tokenization methods originally developed for general natural language processing (NLP) tasks (Trewarthia et al., 2022; Gupta et al., 2022; Huang and Cole, 2022). These methods prioritize high-frequency words in tokenization, resulting in misrepresentation of low-frequency words (Yuan et al., 2024; Lee et al., 2024; Liang et al., 2023), which is particularly problematic in material corpora.

Figure 1: (a) Frequency histograms of material concepts and general words on 150K materials-related scientific papers. (b) Tokenization results of material concepts using conventional tokenization and MATTER (ours).

Material concepts—such as material names and chemical formulas—tend to appear infrequently in materials-related scientific papers as shown in Figure 1(a). This can lead to the oversight of material concepts in frequency-centric tokenization methods, whereas high-frequency general words dominate the subword vocabulary. As a result, material concepts are indeed fragmented into semantically unrelated subwords. For example, as shown in Figure 1(b), the word germanium, which means a chemical element, is split into semantically unrelated subwords german and -ium. Such fragmentation may cause language models to misinterpret the meaning of material concepts, resulting in performance degradation in materials science tasks. Several previous studies have also shown that preserving domain-specific subwords is crucial for maintaining model effectiveness (Gutiérrez et al., 2023; Gu et al., 2021; Hofmann et al., 2021), but how to identify and preserve such words remains unexplored in the materials science domain.

1Our code is available at https://github.com/yerimoh/MATTER
To address this issue, we propose MATTER (Materials Tokenization Framework), a novel approach that integrates material knowledge into tokenization. MATTER involves carefully designed frequency computation and merging processes to effectively capture the material concepts. We present MatDetector, a material concept identifier that scores each concept by its relevance to the materials domain, trained on a corpus of material knowledge that we carefully constructed. Subsequently, jointly considering the relevance scores and statistics of words, MATTER re-ranks the score of multiple possible merged tokens, prioritizing material-related subwords to be preserved. By integrating material knowledge into frequency computation and restructuring token merging, MATTER addresses the limitations of standard frequency-centric tokenization and enhances the representation of material concepts.

To verify the efficacy of MATTER, we conduct comprehensive experiments across diverse downstream tasks in materials science, including both generation and classification. The results demonstrate that MATTER significantly enhances performance on material-specific tasks while preserving the unique characteristics of material terminology. By integrating material knowledge into tokenization training, MATTER enables more precise learning of domain-specific concepts, underscoring the effectiveness of this tailored approach. In summary, this paper presents the following key contributions:

• We introduce MATTER, a novel domain-specific tokenization framework that integrates material knowledge into the tokenization process.

• We develop a novel scheme for materials tokenization based on MatDetector trained on our materials knowledge corpus integrated into our re-ranked token merging process.

• We demonstrate that MATTER outperforms existing tokenization methods, achieving an average improvement of 4% on generation tasks and 2% on classification tasks through extensive experiments.

2 Related Work

2.1 Subword Tokenization

Tokenization plays a crucial role in the performance of language models (Rust et al., 2021; Singh and Strouse, 2024; Wang et al., 2024a). One significant advancement in this area is subword tokenization, a pivotal approach in NLP. Various subword tokenization techniques exist, among which frequency-centric methods, such as Byte Pair Encoding (BPE; Gage 1994; Sennrich et al. 2016) and WordPiece (Wu, 2016), construct subword vocabularies by merging frequently co-occurring character sequences. Recent studies have explored integrating additional linguistic and contextual signals into tokenization. SAGE (Yehezkel and Pinter, 2023) introduces contextual embeddings to guide token segmentation, while PickyBPE (Chizhov et al., 2024) refines intermediate “junk” tokens.

However, while these methods effectively preserve high-frequency words, they often fragment low-frequency words, obscuring their meaning (Schmidt et al., 2024; Wu, 2016; Sennrich, 2015; Mikolov et al., 2012). Additionally, they are designed for general-domain corpora and fail to account for specialized terminology in the materials domain, where key concepts are both semantically significant and infrequent. As a result, conventional tokenization methods frequently split material concepts into unrelated subwords, disrupting their meaning. In contrast, MATTER is designed to address these domain-specific challenges in materials science.

2.2 Language Models in Materials Science

The discovery and practical application of materials is a time-intensive process, often spanning decades (Science and , US; Jain et al., 2013). To accelerate this process, leveraging the wealth of knowledge captured in textual datasets has become essential. NLP-based approaches have potential in materials informatics, enabling advancements in extracting and utilizing domain-specific knowledge (Wang et al., 2024b; Friedrich et al., 2020; Weston et al., 2019; Mysore et al., 2019). Tshitoyan et al. (2019) introduced embedding-based unsupervised methods, effectively capturing chemical knowledge and understanding chemical properties. Building on this foundation, Trewartha et al. (2022) introduced pre-trained language models trained on a materials science corpus, utilizing BERT (Devlin et al., 2019). Further extending the capabilities of BERT-based models, SciBERT (Beltagy et al., 2019), trained on material and battery-specific corpora, was adapted into MatSciBERT (Gupta et al., 2022) and BatteryBERT (Huang and Cole, 2022), respectively.
(a) Frequency-centric Tokenization

1. Count word frequency
The germanium oxide framework is charged with a formal charge ... LiMn2O4 ...

Material-domain corpus

2. Create vocabulary based on frequency
the th the germanium ge an um ger man ium german limn2o4 li mn2 o4 lim∞ th, ge, an, um, ger, man, ium, the, german lim∞ ...

Final Vocab

(b) MATTER (ours)

1. Count word frequency
The germanium oxide framework is charged with a formal charge ... LiMn2O4 ...

Material-domain corpus

Using MatDetector incorporate material knowledge

2. Material knowledge

3. Create vocabulary with re-ranked merging order
the th the germanium ge an um ger man ium german germanium limn2o4 li mn2 o4 lim∞ mn2 limn2 ...

Final Vocab

Figure 2: Comparison of the overall methodology between the existing frequency-centric tokenization and MATTER: (a) The existing frequency-centric tokenization creates the vocabularies based on word frequency. (b) In contrast, our approach, MATTER, incorporates material knowledge from MatDetector into subword vocabularies.

However, these models rely on tokenization strategies originally designed for general NLP tasks, which can be suboptimal for material specialized terminology. In contrast, MATTER introduces a tokenization approach tailored to the unique linguistic characteristics of the materials domain.

3 MATTER

We propose a materials-aware tokenization approach that integrates material knowledge into tokenization training and re-ranks token merging order. The overall procedure is illustrated in Figure 2.

3.1 Word Frequency Calculation

MATTER incorporates the WordPiece algorithm, a frequency-centric tokenization method, with material domain knowledge. The standard WordPiece algorithm first computes the frequency of each word in the corpus. Then, it tokenizes words into sequences of characters or byte units and iteratively merges the most frequent pair of tokens. Similarly, MATTER also computes word frequencies, denoted as \( \text{freq}_{\text{origin}}(w) \) for a word \( w \):

\[
\text{freq}_{\text{origin}}(w) = \text{count}(w)
\]

where \( \text{count}(w) \) represents the number of occurrences of word \( w \) in the corpus. However, MATTER further incorporates material knowledge (\S 3.2) and re-ranks the token merging order (\S 3.3) to better preserve domain-specific terminology.

3.2 Material Knowledge Incorporation

To integrate material knowledge into MATTER, we adjust word frequencies (\S 3.1) by assigning weights to material concepts. Therefore, precise identification of material concepts is crucial. Traditionally, ChemDataExtractor (Kumar et al., 2024) has been widely used in materials science for this purpose, but since it was trained on biomedical data, its accuracy in identifying material concepts is limited (Kim et al., 2024; Kumar et al., 2024; Tran et al., 2024; Xu et al., 2023).

To address this, we introduce MatDetector, a material-agnostic tool that detects material concepts in a target corpus and assigns probability scores to each concept. Developed using the architecture of Trewartha et al. (2022), MatDetector is optimized for material concept detection. The dataset creation process is as follows:

Material Concept Extraction  The MatDetector searches the PubChem database (Kim et al., 2019) for material-related concepts, extracting 80K material concepts (chemical names, IUPAC names, synonyms, and molecular formulas).

Material Corpus Crawling  Using these concepts extracted from PubChem, we crawl Semantic Scholar, collecting around 42K scientific papers.

Crawled Data Tagging  The collected corpus is tagged with PubChem material concepts, creating a NER material dataset with labels "material name",
Algorithm 1 MATTER Tokenization Training
Input: Corpus \( C \), Vocabulary size \( V \), MatDetector \( MD \), Material importance factor \( \lambda \)
Output: Vocabulary \( \mathcal{V} \) of size \( V \) (ordered)
1: procedure MATTER(C, V, MD, \( \lambda \))
2: \( \mathcal{V} \leftarrow \{ c \mid c \in C \} \) \( \triangleright \) Unique characters
3: freq_{origin}(w) \leftarrow word frequency for all \( C \)
4: \( \hat{y}_{\text{mat}}(w) \leftarrow MD(w) \)
5: for all \( w \in C \) do \( \triangleright \) Re-ranking
6:   if \( \hat{y}_{\text{mat}}(w) \neq \emptyset \) then
7:     freq_{mat}(w) \leftarrow freq_{origin}(w) + \( \lambda \cdot \frac{\hat{y}_{\text{mat}}(w)}{1 - \hat{y}_{\text{mat}}(w)} \)
8:   else
9:     freq_{mat}(w) \leftarrow freq_{origin}(w)
10:   end if
11: end for
12: Compute Score(\( t_L, t_R \)) for all token pairs
13: while \( |\mathcal{V}| < V \) do \( \triangleright \) Merge tokens
14:   \( (t_L, t_R) \leftarrow \arg\max_{(t_L, t_R)} \text{MatScore}(t_L, t_R) \)
15:   \( t_{new} \leftarrow t_L \oplus t_R \) \( \triangleright \) Create new token
16:   \( \mathcal{V} \leftarrow \mathcal{V} \cup \{ t_{new} \} \)
17:   \( C.\text{ReplaceAll}((t_L, t_R), t_{new}) \) \( \triangleright \) Update corpus
18:   Recompute Score for updated token set
19: end while
20: return \( \mathcal{V} \)
21: end procedure

"material formula", and "other". Labels other than "other" are treated as "material concept".

Data Augmentation While Semantic Scholar offers relatively clean data, material-related datasets from journals and repositories often contain formatting inconsistencies, OCR errors, and structural variations. To address this, we standardized common noise and expanded the dataset fourfold to enhance model robustness. Details in Appendix B.

Using the MatDetector, we can detect material concepts and compute their probability. Specifically, for Given a word \( w \) that is split into \( n \) subword tokens \( \{ t_1, t_2, ..., t_n \} \), the label for the word is determined as follows:

\[
\hat{y}(w) = \arg\max_{c \in C} \frac{1}{n} \sum_{i=1}^n P(t_i, c)
\]

where \( C \) is the set of all possible labels, and \( P(t_i, c) \) denotes the probability of subword token \( t_i \) being classified as label \( c \). If the predicted label \( \hat{y}(w) \) falls under "material concept," we denote it as \( \hat{y}_{\text{mat}}(w) \). The equation is as follows:

\[
\hat{y}_{\text{mat}}(w) = \begin{cases}
\hat{y}(w), & \text{if } \hat{y}(w) \in \{\text{material}\} \\
\emptyset, & \text{otherwise}
\end{cases}
\]

Ultimately, material concepts identified within the vocabulary are assigned \( \hat{y}_{\text{mat}}(w) \), representing the likelihood of a word being relevant to the material domain. A higher probability value indicates stronger relevance to material concepts, ensuring that domain-specific concepts are effectively distinguished from general words.

3.3 Vocab Creation with Re-ranked Order
To integrate \( \hat{y}_{\text{mat}}(w) \) into tokenization, we adjust word frequency computations by weighting material concepts based on their assigned probability scores. This adjustment prevents material concepts from being underrepresented, preserving their structural and semantic integrity during tokenization. To incorporate material information, we assign weighted frequencies to material concepts as follows:

Using this \( \hat{y}_{\text{mat}}(w) \), MATTER adjusts the original frequency to prioritize material concepts. The adjusted frequency is computed as follows:

\[
\text{freq}_{\text{mat}}(w) = \text{freq}_{\text{origin}}(w) + \lambda \cdot \frac{\hat{y}_{\text{mat}}(w)}{1 - \hat{y}_{\text{mat}}(w)}
\]

With the adjusted frequency incorporating material knowledge, MATTER re-ranks the merging order based on incorporated material knowledge. Words are initially decomposed into sequences of characters or byte units, and the algorithm iteratively merges token pairs according to the re-ranked order guided by material relevance. The detailed algorithm is provided in Algorithm 1.

4 Experiments

4.1 Experimental Setups

Baselines To verify the efficacy of MATTER, we mainly compare ours with strong tokenization method: BPE (Sennrich et al., 2016), WordPiece (Wu, 2016), SAGE (Yehezkel and Pinter, 2023), PickyBPE (Chizhov et al., 2024). More hyperparameters are detailed in Appendix A.1. The detailed experimental setups are described in Appendix A.2.

Pre-training To evaluate the impact of tokenization on performance, we trained models using both baseline and MATTER specifically for the domain of materials science. Consistent with prior methodology (Gupta et al., 2022), we adopt SciBERT (Beltagy et al., 2019) as the encoder backbone for all experiments, due to its widespread use in materials-specific language modeling (Gupta et al., 2022; Huang and Cole, 2022; Kim et al., 2024). All models are trained with a fixed vocabulary size of 31,090 and a corpus of 150K materials science
<table>
  <tr>
    <th rowspan="2">Tokenization</th>
    <th rowspan="2">Metric</th>
    <th colspan="8">Generation Task</th>
  </tr>
  <tr>
    <th>NER</th>
    <th>RC</th>
    <th>EAE</th>
    <th>PC</th>
    <th>SAR</th>
    <th>SC</th>
    <th>SF</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td rowspan="2">(Sennrich et al., 2016)</td>
    <td>BPE</td>
    <td>Micro-F1: 55.7<sub>±0.4</sub></td>
    <td>49.3<sub>±0.5</sub></td>
    <td>48.3<sub>±0.8</sub></td>
    <td>67.3<sub>±0.1</sub></td>
    <td>61.1<sub>±1.8</sub></td>
    <td>90.7<sub>±2.4</sub></td>
    <td>36.3<sub>±1.4</sub></td>
    <td>63.5<sub>±0.5</sub></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>47.1<sub>±0.5</sub></td>
    <td>47.2<sub>±0.9</sub></td>
    <td>36.3<sub>±0.3</sub></td>
    <td>40.2<sub>±0.0</sub></td>
    <td>41.8<sub>±1.3</sub></td>
    <td>47.6<sub>±0.0</sub></td>
    <td>16.7<sub>±1.6</sub></td>
    <td>42.0<sub>±0.9</sub></td>
  </tr>
  <tr>
    <td rowspan="2">(Wu, 2016)</td>
    <td>WordPiece</td>
    <td>Micro-F1: 76.6<sub>±0.2</sub></td>
    <td>80.9<sub>±0.3</sub></td>
    <td>48.5<sub>±0.2</sub></td>
    <td>73.1<sub>±0.5</sub></td>
    <td>81.9<sub>±0.4</sub></td>
    <td>90.0<sub>±0.1</sub></td>
    <td>57.4<sub>±0.2</sub></td>
    <td>72.6<sub>±0.1</sub></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>56.1<sub>±0.2</sub></td>
    <td>58.5<sub>±0.6</sub></td>
    <td>29.4<sub>±0.3</sub></td>
    <td>58.9<sub>±1.0</sub></td>
    <td>74.6<sub>±0.9</sub></td>
    <td>60.3<sub>±0.8</sub></td>
    <td>32.6<sub>±0.2</sub></td>
    <td>52.9<sub>±0.2</sub></td>
  </tr>
  <tr>
    <td rowspan="2">(Yehezkel and Pinter, 2023)</td>
    <td>SAGE</td>
    <td>Micro-F1: 77.0<sub>±0.2</sub></td>
    <td>82.3<sub>±0.4</sub></td>
    <td>47.3<sub>±0.1</sub></td>
    <td>68.3<sub>±0.8</sub></td>
    <td>77.1<sub>±0.4</sub></td>
    <td>90.9<sub>±0.1</sub></td>
    <td>57.1<sub>±0.3</sub></td>
    <td>71.4<sub>±0.2</sub></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>57.0<sub>±0.3</sub></td>
    <td>61.6<sub>±0.4</sub></td>
    <td>28.3<sub>±0.3</sub></td>
    <td>59.6<sub>±1.3</sub></td>
    <td>67.4<sub>±0.9</sub></td>
    <td>61.6<sub>±0.8</sub></td>
    <td>35.0<sub>±0.3</sub></td>
    <td>52.9<sub>±0.3</sub></td>
  </tr>
  <tr>
    <td rowspan="2">(Chizhov et al., 2024)</td>
    <td>PickyBPE</td>
    <td>Micro-F1: 55.4<sub>±0.1</sub></td>
    <td>92.1<sub>±0.1</sub></td>
    <td>47.9<sub>±0.4</sub></td>
    <td>67.2<sub>±0.0</sub></td>
    <td>75.7<sub>±0.2</sub></td>
    <td>90.7<sub>±0.0</sub></td>
    <td>43.6<sub>±0.1</sub></td>
    <td>67.5<sub>±0.1</sub></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>41.7<sub>±0.1</sub></td>
    <td>65.1<sub>±0.2</sub></td>
    <td>36.5<sub>±0.6</sub></td>
    <td>40.2<sub>±0.0</sub></td>
    <td>66.1<sub>±0.7</sub></td>
    <td>47.6<sub>±0.0</sub></td>
    <td>23.1<sub>±0.1</sub></td>
    <td>45.8<sub>±0.1</sub></td>
  </tr>
  <tr>
    <td>MATTER (ours)</td>
    <td>Micro-F1</td>
    <td>80.0<sub>±0.0</sub></td>
    <td>83.8<sub>±0.1</sub></td>
    <td>53.1<sub>±0.2</sub></td>
    <td>73.7<sub>±0.2</sub></td>
    <td>85.5<sub>±0.3</sub></td>
    <td>91.2<sub>±0.1</sub></td>
    <td>61.9<sub>±0.3</sub></td>
    <td>75.6<sub>±0.1</sub></td>
  </tr>
  <tr>
    <td>MATTER (ours)</td>
    <td>Macro-F1</td>
    <td>59.3<sub>±0.2</sub></td>
    <td>59.1<sub>±0.5</sub></td>
    <td>36.9<sub>±0.3</sub></td>
    <td>67.6<sub>±0.6</sub></td>
    <td>79.3<sub>±0.7</sub></td>
    <td>64.9<sub>±0.5</sub></td>
    <td>38.0<sub>±0.3</sub></td>
    <td>57.9<sub>±0.1</sub></td>
  </tr>
</table>

Table 1: Evaluation results on MatSci-NLP (generation tasks): The tasks encompass Named Entity Recognition (NER), Relation Classification (RC), Event Argument Extraction (EAE), Paragraph Classification (PC), Synthesis Action Retrieval (SAR), Sentence Classification (SC), and Slot Filling (SF). The best-performing results are highlighted in boldface.

papers. All training conditions—including model architecture, optimizer, and learning rate—are held constant across tokenizers to ensure fair comparison. In MATTER, the weighting parameter \( \lambda \) was set to 1 based on empirical analysis (see § 4.6), and further implementation details are provided in Appendix A.2.

Downstream Tasks and Datasets To comprehensively evaluate the performance of MATTER, we compare models trained with different tokenization methods on both generation and classification tasks. For generation tasks, we assess each baseline on the MatSci-NLP dataset (Song et al., 2023a), which includes seven materials-related tasks. We follow the MatSci-NLP benchmark protocol, which evaluates domain-specific encoders using a transformer-based schema decoder tailored for generation-based tasks. For classification tasks, we adopt four distinct benchmarks from prior work (Gupta et al., 2022), including named entity recognition (Weston et al., 2019; Friedrich et al., 2020), paragraph classification (Venugopal et al., 2021), and slot filling (Friedrich et al., 2020). These classification models are evaluated under standard encoder-only settings as used in prior work (Gupta et al., 2022). Detailed descriptions of evaluation metrics are provided in Appendix A.3.

4.2 Main Results

Generation Tasks Table 1 shows that MATTER outperforms existing tokenization methods, boosting Micro-F1 and Macro-F1 by 3% and 5% on average. These gains highlight MATTER’s broad applicability across materials science tasks. Notably, SAGE and PickyBPE, which introduce non-material-specific signals, perform worse than WordPiece, emphasizing the need for domain-specific knowledge in tokenization. To further examine the generalizability of MATTER, we additionally evaluate its performance on materials-domain QA tasks using decoder-based and encoder-decoder models.2

Classification Tasks Similar to the generation tasks (Table 2), classification results confirm MATTER’s superiority, with an average Micro-F1 and Macro-F1 improvement of 1.6% and 1.8%, respectively. These consistent gains highlight its robustness and ability to generalize across diverse materials science contexts, reinforcing its impact on materials informatics. To rigorously verify that these improvements are not attributed to random variation, we conducted paired t-tests for both generation and classification tasks. The detailed statistical analysis is presented in Appendix D, confirming

2See Appendix C for full details and results. MATTER outperforms other tokenization methods on the MaScQA benchmark, showing consistent gains in both model types.
<table>
  <tr>
    <th rowspan="2">Tokenization</th>
    <th rowspan="2">Metric</th>
    <th colspan="4">NER<sub>SOFC</sub></th>
    <th colspan="4">NER<sub>Matscholar</sub></th>
    <th colspan="4">SF</th>
    <th colspan="4">RC</th>
    <th colspan="4">PC*</th>
  </tr>
  <tr>
    <th>val</th><th>test</th><th>val</th><th>test</th><th>val</th><th>test</th><th>val</th><th>test</th><th>val</th><th>test</th><th>val</th><th>test</th><th>val</th><th>test</th><th>val</th><th>test</th><th>val</th><th>test</th>
  </tr>
  <tr>
    <td rowspan="2">BPE (Sennrich et al., 2016)</td>
    <td>Micro-F1</td>
    <td>81.6<sub>0.02</sub></td><td>81.4<sub>0.1</sub></td><td>86.4<sub>0.3</sub></td><td>84.3<sub>0.5</sub></td><td>68.1<sub>0.5</sub></td><td>68.3<sub>0.6</sub></td><td>90.2<sub>0.4</sub></td><td>89.9<sub>0.0</sub></td><td>95.5<sub>0.0</sub></td><td>95.6<sub>0.0</sub></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>80.7<sub>0.2</sub></td><td>78.9<sub>0.1</sub></td><td>85.0<sub>0.6</sub></td><td>82.9<sub>0.7</sub></td><td>65.5<sub>0.4</sub></td><td>59.3<sub>0.8</sub></td><td>86.4<sub>0.1</sub></td><td>85.5<sub>0.1</sub></td><td></td><td></td>
  </tr>
  <tr>
    <td rowspan="2">WordPiece (Wu, 2016)</td>
    <td>Micro-F1</td>
    <td>82.0<sub>0.06</sub></td><td>80.9<sub>0.4</sub></td><td>88.8<sub>0.2</sub></td><td>86.1<sub>0.3</sub></td><td>67.4<sub>0.5</sub></td><td>60.4<sub>0.7</sub></td><td>90.6<sub>0.2</sub></td><td>91.0<sub>0.7</sub></td><td>95.2<sub>0.1</sub></td><td>95.2<sub>0.1</sub></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>83.0<sub>0.2</sub></td><td>83.0<sub>0.4</sub></td><td>87.6<sub>0.3</sub></td><td>85.8<sub>0.2</sub></td><td>69.2<sub>0.4</sub></td><td>69.6<sub>0.4</sub></td><td>86.3<sub>0.3</sub></td><td>87.5<sub>0.1</sub></td><td></td><td></td>
  </tr>
  <tr>
    <td rowspan="2">SAGE (Yehezkel and Pinter, 2023)</td>
    <td>Micro-F1</td>
    <td>82.0<sub>0.02</sub></td><td>79.7<sub>0.4</sub></td><td>88.4<sub>0.3</sub></td><td>86.7<sub>0.4</sub></td><td>67.9<sub>0.5</sub></td><td>60.3<sub>0.4</sub></td><td>89.8<sub>0.4</sub></td><td>90.6<sub>0.3</sub></td><td>95.3<sub>0.0</sub></td><td>95.6<sub>0.2</sub></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>82.7<sub>0.12</sub></td><td>82.5<sub>0.4</sub></td><td>87.6<sub>0.2</sub></td><td>86.1<sub>0.4</sub></td><td>69.7<sub>0.3</sub></td><td>69.5<sub>0.6</sub></td><td>86.4<sub>0.7</sub></td><td>87.1<sub>0.0</sub></td><td></td><td></td>
  </tr>
  <tr>
    <td rowspan="2">PickyBPE (Chizhov et al., 2024)</td>
    <td>Micro-F1</td>
    <td>77.3<sub>0.13</sub></td><td>78.8<sub>0.6</sub></td><td>81.4<sub>1.0</sub></td><td>83.4<sub>0.6</sub></td><td>62.0<sub>0.3</sub></td><td>60.2<sub>0.4</sub></td><td>88.6<sub>0.1</sub></td><td>85.8<sub>0.2</sub></td><td>95.7<sub>0.3</sub></td><td>95.8<sub>0.2</sub></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>78.6<sub>0.4</sub></td><td>81.0<sub>0.7</sub></td><td>86.1<sub>0.3</sub></td><td>84.7<sub>0.3</sub></td><td>67.1<sub>0.1</sub></td><td>55.4<sub>0.2</sub></td><td>88.8<sub>0.6</sub></td><td>87.0<sub>0.2</sub></td><td></td><td></td>
  </tr>
  <tr>
    <td rowspan="2">MATTER (ours)</td>
    <td>Micro-F1</td>
    <td>83.1<sub>0.12</sub></td><td>82.0<sub>0.4</sub></td><td>89.6<sub>0.1</sub></td><td>87.8<sub>0.4</sub></td><td>68.4<sub>0.1</sub></td><td>60.4<sub>0.4</sub></td><td>90.9<sub>0.2</sub></td><td>92.6<sub>0.6</sub></td><td>96.9<sub>0.1</sub></td><td>96.2<sub>0.2</sub></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>84.3<sub>0.2</sub></td><td>84.4<sub>0.3</sub></td><td>88.6<sub>0.2</sub></td><td>86.3<sub>0.3</sub></td><td>69.7<sub>0.4</sub></td><td>70.1<sub>0.3</sub></td><td>87.3<sub>0.4</sub></td><td>87.9<sub>0.9</sub></td><td></td><td></td>
  </tr>
</table>

Table 2: Evaluation results are presented across five classification tasks. Here, PC* represents accuracy, while the remaining metrics are reported as Micro-F1 and Macro-F1 scores. The best-performing results are highlighted in boldface.

<table>
  <tr>
    <th>Train</th>
    <th>Dev</th>
    <th>Test</th>
    <th>Total</th>
  </tr>
  <tr>
    <td>English Set</td>
    <td>458,692</td>
    <td>57,371</td>
    <td>57,755</td>
    <td>573,818</td>
  </tr>
  <tr>
    <td>Material Subset</td>
    <td>16,286</td>
    <td>2,010</td>
    <td>2,173</td>
    <td>20,469</td>
  </tr>
</table>

Table 3: Statistics for the SIGMORPHON 2022 morpheme segmentation dataset and the material dataset, as described in Section 4.3.

<table>
  <tr>
    <th>Tokenization for MatSciBERT</th>
    <th>Segmentation F1</th>
  </tr>
  <tr>
    <td>WordPiece (Wu, 2016)</td>
    <td>44.3</td>
  </tr>
  <tr>
    <td>SAGE (Yehezkel and Pinter, 2023)</td>
    <td>43.4</td>
  </tr>
  <tr>
    <td>PickyBPE (Chizhov et al., 2024)</td>
    <td>36.2</td>
  </tr>
  <tr>
    <td>MATTER (ours)</td>
    <td>59.9</td>
  </tr>
</table>

Table 4: Material morpheme segmentation performance for different tokenization of the MatSciBERT model. The best-performing results are highlighted in boldface.

<table>
  <tr>
    <th>Tool</th>
    <th>Recall</th>
    <th>Precision</th>
    <th>F1 Score</th>
  </tr>
  <tr>
    <td>ChemDataExtractor</td>
    <td>18%</td>
    <td>57%</td>
    <td>27%</td>
  </tr>
  <tr>
    <td>MatDetector (ours)</td>
    <td>57%</td>
    <td>69%</td>
    <td>63%</td>
  </tr>
</table>

Table 5: Average performance of two material concept extraction tools on external materials NER datasets across all evaluation metrics.

that MATTER’s performance gains are statistically significant across all major benchmarks.

4.3 Material Morpheme Segmentation

To validate MATTER’s ability to segment material concepts into meaningful subwords, we evaluated its performance on the material subset of the SIGMORPHON dataset (Batsuren et al., 2022). The SIGMORPHON 2022 Shared Task provides a reliable benchmark for assessing whether words are segmented into morphologically meaningful units. For this analysis, we identified material concepts shared between SIGMORPHON, PubChem, and MatKG (Venugopal et al., 2022). The resulting subset, as shown in Table 3, revealed that approximately 20% of annotated words are relevant material concepts.

Using this subset, we evaluated the morpheme segmentation. As shown in Table 4, MATTER achieved an average improvement of 18.6% in segmentation accuracy compared to other tokenization algorithms. These results confirm that MATTER tokenization, effectively incorporates the characteristics of material corpora, enabling it to segment material concepts into meaningful subwords.

4.4 Extracted Material Concepts

Validation on Training Corpus To validate MatDetector on the training corpus, we constructed a reference lexicon of 100K material-related entries from PubChem and MatKG, including names, formulas, and synonyms. These were decomposed into 1.6M normalized tokens for broader coverage. Entities extracted from 150K materials papers were matched to the lexicon, and considered valid if found in the lexicon. MatDetector extracted 6x
Figure 3: Comparison tokenization methods by word-initial token ratio (bar), materials token ratio (line), and average token length.

Figure 4: Comparison of Macro-F1 scores for MATTER and w/o material knowledge across during tokenization training different number of tokens.

more material concepts than ChemDataExtractor and achieved 64% higher match rate, confirming its precision and suitability for identifying material concepts in materials science corpus.

Validation on Materials NER To quantify absolute performance, we additionally evaluate both tools on two external materials NER datasets: MatScholar (Weston et al., 2019) and SOFC (Friedrich et al., 2020). As shown in Table 5, which reports the average performance across the two datasets, MatDetector consistently outperforms ChemDataExtractor across precision, recall, and F1 score. Notably, it achieves over twice the F1 score on average, highlighting its effectiveness not only in coverage but also in accurately identifying material entities. Detailed per-dataset results are provided in Appendix E. These results further validate MatDetector’s ability to accurately and comprehensively detect material concepts in domain-specific NER tasks.

4.5 Token Qualities

To assess material token quality, we extract material-related tokens using MatDetector and compare tokenization methods. More hyperparameters are detailed in Appendix A.4.

Word-Initial Token One key aspect of token quality is the proportion of word-initial tokens, which help preserve word structure and meaning (Yehezkel and Pinter, 2023; Chizhov et al., 2024). For example, in tokenizing "germanium" into "ger-man" and "-ium", the word-initial token is "ger-man". As shown in the left part of Figure 3, MATTER preserves a higher proportion of word-initial tokens (bar) compared to other methods. To evaluate this more rigorously, we further measured the materials-related word-initial token ratio (line) using a manually annotated set of approximately 9,000 material concepts, curated for downstream evaluation only (details in Appendix F). While this represents a small fraction of the full corpus, the results consistently demonstrate that MATTER achieves a significantly higher proportion of materials-related word-initial tokens, even on unseen datasets. This indicates that its vocabulary is enriched with material-specific terms, enabling better preservation of the semantic integrity of materials-related concepts.

Token Length According to Bostrom and Durrett (2020), longer mean token length reflects gold-standard morphologically-aligned tokenization, which enhances token quality. Based on this, we also measure mean token length. As shown in the right part of Figure 3, our method achieves a higher mean token length. Notably, it surpasses even SAGE and PickyBPE, which deliberately eliminate shorter intermediate tokens through compression at the cost of increased computational expense. This demonstrates that our approach not only maintains morphological alignment for material concepts but also preserves higher-quality tokenization.

Number of Tokens Figure 4 presents the experimental results comparing MATTER with a tokenizer trained without material knowledge during tokenization training. The number of tokens was varied from 0.5x to 1.5x of the original size. The results show that MATTER consistently outper-
<table>
  <tr>
    <th>Tokenization</th>
    <th>Concept</th>
    <th>Word Embedding Sim.</th>
    <th>Formula</th>
    <th>Word Embedding Sim.</th>
    <th>Abbr</th>
    <th>Word Embedding Sim.</th>
  </tr>
  <tr>
    <td rowspan="2">WordPiece</td>
    <td>germanium (german-iun)</td>
    <td>agilent fri</td>
    <td>PbI2 (pib-2) 10c</td>
    <td>nowak 81.8 (lf-p)</td>
    <td>LFP inlet chattopadhyay</td>
    <td>95.5 93.7</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="2">SAGE</td>
    <td>germanium (german-iun)</td>
    <td>lot segregation</td>
    <td>PbI2 (p-ib-2) pounds</td>
    <td>-gen 43.9 43.6</td>
    <td>LFP occupation multiphonon</td>
    <td>95.8 95.2</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="2">PickyBPE</td>
    <td>germanium (g-erma-n-iun)</td>
    <td>nomin inex</td>
    <td>PbI2 (p-bi-2) p</td>
    <td>gaussian 63.1 62.8</td>
    <td>LFP her, (l-f-p) consideration</td>
    <td>75.8 75.0</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="2">MATTER (ours)</td>
    <td>germanium (germanium)</td>
    <td>dithiocarbamate ammonium</td>
    <td>PbI2 pb5 (pbi2) pbf2</td>
    <td>89.9 89.2</td>
    <td>LFP zrF7 (lfp) acyclohex</td>
    <td>90.9 90.8</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
</table>

Table 6: Comparison of subword embedding averaging results across different tokenization methods. The table presents the five nearest neighbor words based on subword embedding averages for each method. The similarity scores (Sim.) indicate the relevance of the nearest neighbors to the target material concept. Boldface highlights words that are directly related to materials.

forms the tokenizer trained without material knowledge in all cases. This demonstrates that providing material-specific information during tokenization training is crucial, regardless of the token count.

Subword Embedding Analysis Table 6 presents the two nearest neighbors of material concepts using cosine similarity. The results show that the nearest neighbors of MATTER are more material-specific and semantically relevant compared to other methods. For instance, while WordPiece and SAGE generate less relevant neighbors (fri, segregation for germanium ), our method produces material concepts such as dithiocarbamate and ammonium for germanium . This indicates our tokenizer better preserves material-specific meanings, improving representation quality for scientific text.

Further inspection reveals that the learned subword embeddings capture a variety of chemically meaningful relationships. For example, pairs such as PbI2 and PbF2 belong to the same chemical family of lead halides, while germanium and dithiocarbamate co-occur as known compound pairs in Ge–S coordination complexes. Other relationships reflect compositional connections, such as the co-existence of germanium and ammonium in ammonium tris(oxalato)germanate, or functional similarity, as seen in LFP and ZrF7, both of which are used in energy storage and sensing applications.

These findings support the claim that the embedding space goes beyond capturing surface-level co-occurrence, instead reflecting deeper, domain-relevant semantics. A more comprehensive analysis and additional examples can be found in Appendix G.2.

4.6 Ablation Study

Comparison of Detectors To confirm whether using MatDetector to extract material concepts and assign weights is more suitable for providing accurate and domain-relevant signals in the material domain compared to the widely used ChemDataExtractor, we performed ablation studies. Specifically, we replaced MatDetector with ChemDataExtractor to assign weights. While ChemDataExtractor is capable of partially extracting material concepts, it lacks the ability to assess the importance of the extracted concepts within the material domain. Consequently, all material concepts extracted by ChemDataExtractor were assigned the highest signal weight of 0.99.

Table 7 show that using MatDetector outperforms ChemDataExtractor, achieving a 2% higher average Micro-F1 score and a 2.7% higher Macro-F1 score. This confirms that MatDetector is more effective in providing material domain-relevant signals. Additionally, when examining the performance of ChemDataExtractor, we observed that it achieved a 1.1% higher Micro-F1 score and a 2.3% higher Macro-F1 score compared to the baseline method, which did not incorporate any material signals. This underscores the importance of incorporating material signals into tokenization.

However, as evidenced by the performance gap between ChemDataExtractor and MatDetector, it is clear that the accuracy of the material signals plays a critical role. These results highlight the necessity of not only incorporating material signals but also ensuring that accurate material concepts and their respective significance are properly consid-
<table>
  <tr>
    <th rowspan="2">Tokenization</th>
    <th rowspan="2">Metric</th>
    <th colspan="8">MatSci-NLP</th>
  </tr>
  <tr>
    <th>NER</th>
    <th>RC</th>
    <th>EAE</th>
    <th>PC</th>
    <th>SAR</th>
    <th>SC</th>
    <th>SF</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>w/o material knowledge (WordPiece)</td>
    <td>Micro-F1</td>
    <td>76.6<sub>±0.2</sub></td>
    <td>80.9<sub>±0.3</sub></td>
    <td>48.5<sub>±0.2</sub></td>
    <td>73.1<sub>±0.5</sub></td>
    <td>81.9<sub>±0.4</sub></td>
    <td>90.0<sub>±0.1</sub></td>
    <td>57.4<sub>±0.2</sub></td>
    <td>72.6<sub>±0.1</sub></td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>56.1<sub>±0.2</sub></td>
    <td>58.5<sub>±0.6</sub></td>
    <td>29.4<sub>±0.3</sub></td>
    <td>58.9<sub>±1.0</sub></td>
    <td>74.6<sub>±0.9</sub></td>
    <td>60.3<sub>±0.8</sub></td>
    <td>32.6<sub>±0.2</sub></td>
    <td>52.9<sub>±0.2</sub></td>
  </tr>
  <tr>
    <td>ChemDataExtractor (Swain and Cole, 2016)</td>
    <td>Micro-F1</td>
    <td>77.1<sub>±1.1</sub></td>
    <td>81.2<sub>±0.7</sub></td>
    <td>53.1<sub>±3.5</sub></td>
    <td>73.6<sub>±0.6</sub></td>
    <td>80.6<sub>±3.3</sub></td>
    <td>91.2<sub>±1.0</sub></td>
    <td>58.8<sub>±2.5</sub></td>
    <td>73.7<sub>±1.3</sub></td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>56.4<sub>±1.3</sub></td>
    <td>58.9<sub>±0.7</sub></td>
    <td>35.0<sub>±4.2</sub></td>
    <td>67.6<sub>±1.2</sub></td>
    <td>68.0<sub>±9.5</sub></td>
    <td>64.8<sub>±0.1</sub></td>
    <td>35.6<sub>±1.6</sub></td>
    <td>55.2<sub>±2.8</sub></td>
  </tr>
  <tr>
    <td>MatDetector (ours)</td>
    <td>Micro-F1</td>
    <td>80.0<sub>±0.0</sub></td>
    <td>83.8<sub>±0.1</sub></td>
    <td>53.1<sub>±0.2</sub></td>
    <td>73.7<sub>±0.2</sub></td>
    <td>85.5<sub>±0.3</sub></td>
    <td>91.2<sub>±0.1</sub></td>
    <td>61.9<sub>±0.3</sub></td>
    <td>75.6<sub>±0.1</sub></td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>59.3<sub>±0.2</sub></td>
    <td>59.1<sub>±0.5</sub></td>
    <td>36.9<sub>±0.3</sub></td>
    <td>67.6<sub>±0.6</sub></td>
    <td>79.3<sub>±0.7</sub></td>
    <td>64.9<sub>±0.5</sub></td>
    <td>38.0<sub>±0.3</sub></td>
    <td>57.9<sub>±0.1</sub></td>
  </tr>
</table>

Table 7: Ablation results on different detectors for the MatSci-NLP dataset across multiple tasks. w/o material knowledge represents frequency-centric tokenization without any additional signal. ChemDataExtractor and MatDetector incorporate additional signals using their respective tools. Bold values indicate the highest scores for each metric-task pair, while underline represent the second-highest scores.

![Comparison of Macro-F1 scores for ChemDataExtractor and MatDetector across λ values.](page_186_1012_482_246.png)

Figure 5: Comparison of Macro-F1 scores for ChemDataExtractor and MatDetector across \( \lambda \) values.

ered. The use of MatDetector effectively addresses both aspects, demonstrating its suitability for enhancing performance in the material domain. Both detectors achieved their highest performance at a \( \lambda \) value of 1 as show in Figure 5.

Comparison of Lambda Figure 5 demonstrates that adding material signals, regardless of the weighting method used, consistently yields better performance compared to the baseline where no material signals were incorporated (\( \lambda = 0 \)). This observation aligns with previous findings and further substantiates that the inclusion of material Knowledge is beneficial. Moreover, it emphasizes the necessity of using appropriate tools to effectively assign these signals for optimal performance.

Notably, both ChemDataExtractor and MatDetector achieved their highest performance at \( \lambda = 1 \). Based on this consistent observation across models, all preceding experiments in this study were conducted using this optimal setting.

5 Conclusion

We proposed MATTER, a novel tokenization approach that incorporates material knowledge derived from material corpora into the tokenization process. MATTER has enabled the creation of vocabularies tailored to the material domain, effectively maintaining the structure and semantics of material concepts. Our extensive experiments have demonstrated that MATTER tokenization significantly improves performance across a wide range of material generation and classification tasks, outperforming conventional tokenization methods. Our work has provided a strong, adaptable foundation components for materials NLP, empowering future research on materials science.

Limitations

While we have demonstrated that MATTER effectively enhances tokenization for pretrained language models in the materials science domain. Nevertheless, our work also opens several valuable opportunities for further improvements and exploration.

Hyperparameter Selection. MATTER introduces a tunable hyperparameter (\( \lambda \)) to balance frequency statistics with material-specific signals during vocabulary construction. While we observed stable improvements across a range of \( \lambda \) values, the method still requires manual selection of this parameter. Although \( \lambda = 1 \) was found to be effective in our experiments, identifying an optimal value for different domains or corpora may require
additional tuning. This reliance on hyperparameter selection may affect general usability in practice.

Further Analysis on Corpus The current experiments were conducted following the prior methodology outlined in (Gupta et al., 2022), which emphasizes the use of material-specialized corpora. Although this ensures consistency and relevance to domain-specific evaluation, future work may benefit from expanding the diversity of training corpora to test MATTER’s generalizability across subdomains and heterogeneous sources.

NER Dependency and Scalability Our approach relies on the identification of material concepts through NER-based classification. To support this, we constructed a high-quality NER dataset using a curated materials knowledge base, ensuring accurate detection of domain-specific terminology essential for effective vocabulary construction in materials science. However, this reliance on supervised signals may introduce challenges in scalability, particularly when applied to broader or less-structured corpora. Addressing this limitation remains an important direction for future work.

Acknowledgements

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.RS-2025-00517221 and No.RS-2024-00415812) and Institute of Information communications Technology Planning Evaluation (IITP) grant funded by the Korea government (MSIT) (No.RS-2024-00439328, Karma: Towards Knowledge Augmentation for Complex Reasoning (SW Starlab), No.RS-2024-00457882, AI Research Hub Project, and No.RS-2019-II190079, Artificial Intelligence Graduate School Program (Korea University)).

References

Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, et al. 2018. Construction of the literature graph in semantic scholar. arXiv preprint arXiv:1805.02262.

Khuyagbaatar Batsuren, Gábor Bella, Aryaman Arora, Viktor Martinovic, Kyle Gorman, Zdeněk Žabokrtský, Amarsanaa Ganbold, Šárka Dohnalová, Magda Ševčíková, Kateřina Pelegrinová, et al. 2022. The signmorphon 2022 shared task on morpheme segmentation. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 103–116.

Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. Scibert: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3615–3620.

Kaj Bostrom and Greg Durrett. 2020. Byte pair encoding is suboptimal for language model pretraining. arXiv preprint arXiv:2004.03720.

Yizhou Chen, Seira Yamaguchi, Atsushi Sato, Dong Xue, and Kazuhiro Marumoto. 2025. Operando spin observation elucidating performance-improvement mechanisms during operation of ruddlesden–popper sn-based perovskite solar cells. npj Flexible Electronics, 9(1):1.

Pavel Chizhov, Catherine Arnett, Elizaveta Korotkova, and Ivan Yamshchikov. 2024. Bpe gets picky: Efficient vocabulary refinement during tokenizer training. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 16587–16604.

Qingchen Deng, Jiangen Li, Xiang Li, Xuye Du, Lanlan Wu, Junrui Wang, and Xinlong Wang. 2024. Incorporating nano-znco-zif particles in the electrospinning polyactide membranes to improve their filtration and antibacterial performances. Polymer Bulletin, 81(15):14067–14081.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.

Annemarie Friedrich, Heike Adel, Federico Tomazic, Johannes Hingerl, Renou Benteau, Anika Maruscyk, and Lukas Lange. 2020. The sofc-exp corpus and neural approaches to information extraction in the materials science domain. arXiv preprint arXiv:2006.03039.

Philip Gage. 1994. A new algorithm for data compression. The C Users Journal, 12(2):23–38.

Darren S Gray, Joe Tien, and Christopher S Chen. 2004. High-conductivity elastomeric electronics. Advanced Materials, 16(5):393–397.

Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1–23.
Tanishq Gupta, Mohd Zaki, NM Anoop Krishnan, and Mausam. 2022. Matscibert: A materials domain language model for text mining and information extraction. npj Computational Materials, 8(1):102.

Bernal Jiménez Gutiérrez, Huan Sun, and Yu Su. 2023. Biomedical language models are robust to sub-optimal tokenization. In The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pages 350–362.

Valentin Hofmann, Janet Pierrehumbert, and Hinrich Schütze. 2021. Superbizarre is not superb: Derivational morphology improves bert’s interpretation of complex words. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3594–3608.

Shu Huang and Jacqueline M Cole. 2022. Batterybert: A pretrained language model for battery database enhancement. Journal of chemical information and modeling, 62(24):6365–6377.

Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. 2013. Commentary: The materials project: A materials genome approach to accelerating materials innovation. APL materials, 1(1).

Junho Kim, Yeachan Kim, Jun-Hyung Park, Yerim Oh, Suho Kim, and SangKeun Lee. 2024. Melt: Materials-aware continued pre-training for language model adaptation to materials science. In Findings of the Association for Computational Linguistics: EMNLP. Association for Computational Linguistics.

Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Sirijian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, et al. 2019. Pubchem 2019 update: improved access to chemical data. Nucleic acids research, 47(D1):D1102–D1109.

Pankaj Kumar, Saurabh Kabra, and Jacqueline M Cole. 2024. a database of stress-strain properties auto-generated from the scientific literature using chemdataextractor. Scientific Data, 11(1):1273.

Jungseob Lee, Hyeonseok Moon, Seungjun Lee, Chan-jun Park, Sugyeong Eo, Hyunwoong Ko, Jaeyoung Seo, Seungyun Lee, and Heui-Seok Lim. 2024. Length-aware byte pair encoding for mitigating over-segmentation in korean machine translation. In Findings of the Association for Computational Linguistics ACL 2024, pages 2287–2303.

Davis Liang, Hila Gonen, Yuning Mao, Rui Hou, Naman Goyal, Marjan Ghazvininejad, Luke Zettlemoyer, and Madian Khabsa. 2023. Xlm-v: Overcoming the vocabulary bottleneck in multilingual masked language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13142–13152.

Tomáš Mikolov, Ilya Sutskever, Anoop Deoras, Hai-Son Le, Stefan Kombrink, and Jan Cernocky. 2012. Subword language modeling with neural networks. preprint (http://www. fit. vutbr. cz/imikolov/rnnlm/char. pdf), 8(67).

Sheshera Mysore, Zach Jensen, Edward Kim, Kevin Huang, Haw-Shiuan Chang, Emma Strubell, Jeffrey Flanigan, Andrew McCallum, and Elsa Olivetti. 2019. The materials science procedural text corpus: Annotating materials synthesis procedures with shallow semantic structures. arXiv preprint arXiv:1905.06939.

Elsa A Olivetti, Jacqueline M Cole, Edward Kim, Olga Kononova, Gerbrand Ceder, Thomas Yong-Jin Han, and Anna M Hiszpanski. 2020. Data-driven materials research enabled by natural language processing and information extraction. Applied Physics Reviews, 7(4).

Ghanshyam Pilania. 2021. Machine learning in materials science: From explainable predictions to autonomous design. Computational Materials Science, 193:110360.

Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, and Iryna Gurevych. 2021. How good is your tokenizer? on the monolingual performance of multilingual language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3118–3135.

Craig W Schmidt, Varshini Reddy, Haoran Zhang, Alec Alameddine, Omri Uzan, Yuval Pinter, and Chris Tanner. 2024. Tokenization is more than compression. arXiv preprint arXiv:2402.18376.

National Science and Technology Council (US). 2011. Materials genome initiative for global competitiveness. Executive Office of the President, National Science and Technology Council.

Rico Sennrich. 2015. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909.

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Linguistics.

Aaditya K Singh and DJ Strouse. 2024. Tokenization counts: the impact of tokenization on arithmetic in frontier llms. arXiv preprint arXiv:2402.14903.

Yu Song, Santiago Miret, and Bang Liu. 2023a. Matsci-nlp: Evaluating scientific language models on materials science language tasks using text-to-schema modeling. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). Association for Computational Linguistics.
Yu Song, Santiago Miret, Huan Zhang, and Bang Liu. 2023b. Honeybee: Progressive instruction finetuning of large language models for materials science. arXiv preprint arXiv:2310.08511.

Matthew C Swain and Jacqueline M Cole. 2016. Chemdataextractor: a toolkit for automated extraction of chemical information from the scientific literature. Journal of chemical information and modeling, 56(10):1894–1904.

Huan Tran, Rishi Gurnani, Chiho Kim, Ghanshyam Pilania, Ha-Kyung Kwon, Ryan P Lively, and Rampi Ramprasad. 2024. Design of functional and sustainable polymers assisted by artificial intelligence. Nature Reviews Materials, pages 1–21.

Amalie Trewartha, Nicholas Walker, Haoyan Huo, Sanghoon Lee, Kevin Cruse, John Dagdelen, Alexander Dunn, Kristin A Persson, Gerbrand Ceder, and Anubhav Jain. 2022. Quantifying the advantage of domain-specific pre-training on named entity recognition tasks in materials science. Patterns, 3(4).

Vahe Tshitoyan, John Dagdelen, Leigh Weston, Alexander Dunn, Ziqin Rong, Olga Kononova, Kristin A Persson, Gerbrand Ceder, and Anubhav Jain. 2019. Unsupervised word embeddings capture latent knowledge from materials science literature. Nature, 571(7763):95–98.

Vineeth Venugopal, Sumit Pai, and Elsa Olivetti. 2022. The largest knowledge graph in materials science—entities, relations, and link prediction through graph representation learning. In AI for Accelerated Materials Design NeurIPS 2022 Workshop.

Vineeth Venugopal, Sourav Sahoo, Mohd Zaki, Manish Agarwal, Nitya Nand Gosvami, and NM Anoop Krishnan. 2021. Looking through glass: Knowledge discovery from materials science literature using natural language processing. Patterns, 2(7).

Dixuan Wang, Yanda Li, Junyuan Jiang, Zepeng Ding, Guochao Jiang, Jiaqing Liang, and Deqing Yang. 2024a. Tokenization matters! degrading large language models through challenging their tokenization. arXiv preprint arXiv:2405.17067.

Lei Wang, Fei Wu, Xiaoqing Liu, Chong Wang, Wanxin Wang, Mingshi Cui, and Zhaoyang Qu. 2024b. A joint extraction method for fault text entity relationships in smart grid considering nested entities and complex semantics. Energy Reports, 11:6150–6159.

Leigh Weston, Vahe Tshitoyan, John Dagdelen, Olga Kononova, Amalie Trewartha, Kristin A Persson, Gerbrand Ceder, and Anubhav Jain. 2019. Named entity recognition and normalization applied to large-scale information extraction from the materials science literature. Journal of chemical information and modeling, 59(9):3692–3702.

Yonghui Wu. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144.

Pengcheng Xu, Xiaobo Ji, Minjie Li, and Wencong Lu. 2023. Small data machine learning in materials science. npj Computational Materials, 9(1):42.

Shaked Yehezkel and Yuval Pinter. 2023. Incorporating context into subword vocabularies. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 623—635.

Fei Yuan, Shuai Yuan, Zhiyong Wu, and Lei Li. 2024. How vocabulary sharing facilitates multilingualism in llama? In Findings of the Association for Computational Linguistics ACL 2024, pages 12111–12130.

Mohd Zaki, NM Anoop Krishnan, et al. 2024. Mascqa: investigating materials science knowledge of large language models. Digital Discovery, 3(2):313–327.

Lin Zhang, Zonghui Lu, Zhe Su, Ye Zhang, and Hui He. 2025. Efficiency of carbothermal reduction in treating norm waste containing ba (226ra) so4. Journal of Radioanalytical and Nuclear Chemistry, pages 1–8.

A Implementation Details and Setups

A.1 Tokenization baseline

We compared our tokenization approach against baseline methods, including the widely used frequency-centric tokenization, WordPiece, as well as more recent and strong tokenization methods, SAGE and PickyBPE. To ensure a fair comparison, all tokenization methods adhered to the vocabulary size of 31,090, as defined in the prior methodology (Gupta et al., 2022). The implementation details are as follows:

WordPiece (Wu, 2016) being one of the most widely used and fundamental frequency-centric tokenization methods, was configured with a min frequency of 2 and a limit alphabet of 6,000.

SAGE (Yehezkel and Pinter, 2023) enhances frequency-centric tokenization by incorporating contextual signals into the process. The implementation of SAGE included several key parameters: vocabulary schedule progressively reducing from 32,000 to the target size of 31,090; an embedding schedule synchronized with the vocabulary schedule; a maximum token length of 17 bytes; and the use of skip-gram embedding training with a vector size of 50, a context window size of 5, and 15 negative samples. To ensure reproducibility, the random seed was set to 692,653.

PickyBPE (Chizhov et al., 2024) was employed to construct the vocabulary, with a desired vocabulary size of 31,090 and an IoS (Importance of
Symbols) threshold set to 0.9. The initial vocabulary ensured comprehensive coverage with a relative symbol coverage of 0.9999. During training, the frequency of merges was logged at intervals of 200 merges to monitor the tokenization process effectively.

A.2 Hyper-parameters
Pre-Train. We follow the previous work (Gupta et al., 2022). The detailed configuration of the main model and training hyperparameters is summarized as follows:

<table>
  <tr>
    <th>Parameter</th>
    <th>Value</th>
  </tr>
  <tr>
    <td>Encoder Layers</td>
    <td>12</td>
  </tr>
  <tr>
    <td>Embedding Dim</td>
    <td>768</td>
  </tr>
  <tr>
    <td>Hidden Dim</td>
    <td>768</td>
  </tr>
  <tr>
    <td>Attention Heads</td>
    <td>12</td>
  </tr>
  <tr>
    <td>Max Tokens in a Batch</td>
    <td>128</td>
  </tr>
  <tr>
    <td>Optimizer</td>
    <td>Adam</td>
  </tr>
  <tr>
    <td>Weight Decay</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>Learning Rate (LR)</td>
    <td>2e-5</td>
  </tr>
  <tr>
    <td>LR Scheduler</td>
    <td>Linear with Warmup</td>
  </tr>
  <tr>
    <td>Warmup Strategy</td>
    <td>Linear</td>
  </tr>
  <tr>
    <td>Precision</td>
    <td>FP16</td>
  </tr>
</table>

A.3 Evaluation metrics
Classification task. We follow the previous work (Gupta et al., 2022). The detailed configuration of the main model and training hyperparameters is summarized in Table 8.

Generation task. We follow the previous work (Song et al., 2023a). The detailed configuration of the main model and training hyperparameters is summarized as follows:

<table>
  <tr>
    <th>Parameter</th>
    <th>Value</th>
  </tr>
  <tr>
    <td>Decoder Layers</td>
    <td>3</td>
  </tr>
  <tr>
    <td>Embedding Dim</td>
    <td>768</td>
  </tr>
  <tr>
    <td>Hidden Dim</td>
    <td>768</td>
  </tr>
  <tr>
    <td>Attention Heads</td>
    <td>8</td>
  </tr>
  <tr>
    <td>Max Tokens in a Batch</td>
    <td>4</td>
  </tr>
  <tr>
    <td>Optimizer</td>
    <td>Adam</td>
  </tr>
  <tr>
    <td>Learning Rate (LR)</td>
    <td>2e-5</td>
  </tr>
  <tr>
    <td>Precision</td>
    <td>FP32</td>
  </tr>
  <tr>
    <td>Training Epochs</td>
    <td>Up to 20<br>(early stopping)</td>
  </tr>
</table>

We evaluate using metrics from MatSciNLP (Song et al., 2023a) and MatSciBERT (Gupta et al., 2022). Generation tasks use Micro-F1 and Macro-F1, averaged over five seeds. Classification tasks report Macro-F1 (SOFC-NER, SOFC-Filling), Micro-F1 (MatScholar), and accuracy (Glass Science), with cross-validation over five folds and three seeds.

A.4 Token Qualities Details
Among 31,090 vocabulary entries, we extract material-related tokens using MatDetector and compare tokenization methods.

<table>
  <tr>
    <th>Tokenization</th>
    <th>#material token</th>
  </tr>
  <tr>
    <td>WordPiece</td>
    <td>10,420</td>
  </tr>
  <tr>
    <td>SAGE</td>
    <td>9,602</td>
  </tr>
  <tr>
    <td>PickyBPE</td>
    <td>9,203</td>
  </tr>
  <tr>
    <td>MATTER (ours)</td>
    <td><b>10,633</b></td>
  </tr>
</table>

B Details of MatDetector Construction
MatDetector is a domain-specific Named Entity Recognition (NER) tool designed to extract material concepts from scientific texts. The detailed steps for its construction are as follows:

Crawling Material Corpus To construct the training dataset for the MatDetector, we first extract chemical names, IUPAC names, synonyms, and molecular formulas from PubChem (Kim et al., 2019), obtaining 80K material concepts. The number of concepts by category is provided in Table (Kim et al., 2019). Using these extracted concepts as keywords, we collect 42K scientific papers from Semantic Scholar (Ammar et al., 2018), focusing on titles and abstracts that contain high-density material knowledge, with detailed comparative information in Table 9.

Creating Train Dataset While Semantic Scholar provides relatively clean data, most material-related data is collected from various journals and repositories, where formatting inconsistencies, OCR errors, and structural variations introduce significant noise. To address this, we construct a Noisy NER Dataset, improving model robustness and expanding the dataset to be four times larger than the original. The details of noise augmentation are as follow:

• Material Name Noise: This includes capitalization errors in element symbols, misplaced
<table>
  <tr>
    <th rowspan="2">Parameter</th>
    <th colspan="5">Value</th>
  </tr>
  <tr>
    <th>NER<sub>SOFc</sub></th>
    <th>NER<sub>Matscholar</sub></th>
    <th>SF</th>
    <th>RC</th>
    <th>PC</th>
  </tr>
  <tr>
    <td>Max Tokens in a Batch</td>
    <td>32</td>
    <td>32</td>
    <td>32</td>
    <td>64</td>
    <td>64</td>
  </tr>
  <tr>
    <td>Training Epochs</td>
    <td>20</td>
    <td>15</td>
    <td>40</td>
    <td>10</td>
    <td>10</td>
  </tr>
  <tr>
    <td>Optimizer</td>
    <td colspan="5">Adam</td>
  </tr>
  <tr>
    <td>Learning Rate (LR)</td>
    <td colspan="5">[2e-5, 3e-5, 5e-5]</td>
  </tr>
  <tr>
    <td>LR Scheduler</td>
    <td colspan="5">Linear with Warmup</td>
  </tr>
  <tr>
    <td>Warmup Strategy</td>
    <td colspan="5">Warmup ratio of 0.1</td>
  </tr>
  <tr>
    <td>Precision</td>
    <td colspan="5">FP32</td>
  </tr>
</table>

Table 8: Detailed configuration of the main model and training hyperparameters for classification task.

<table>
  <tr>
    <th rowspan="2">Tool</th>
    <th colspan="2">ChemDataExtractor<br>(Swain and Cole, 2016)</th>
    <th colspan="2">MatDetector (ours)</th>
  </tr>
  <tr>
    <th>Entity type</th>
    <th>chemical mention</th>
    <th>chemical concepts, formulas</th>
  </tr>
  <tr>
    <td>domain</td>
    <td>BioMedical</td>
    <td>Material</td>
  </tr>
  <tr>
    <td>Train data annotated method</td>
    <td>manually annotated</td>
    <td>manually & automatic annotated</td>
  </tr>
  <tr>
    <td>number of abstract</td>
    <td>10,000</td>
    <td>404,262</td>
  </tr>
</table>

Table 9: Comparison of Extractable Entity Types and Training Data in ChemDataExtractor and MatDetector.

or duplicated digits, reordering of elements, and insertion of unnecessary characters or special symbols. These modifications reflect common errors found in chemical names and mimic the inconsistencies in scientific documents.

• Material Formula Noise: Common formatting inconsistencies in formulas are simulated by adding spaces around special symbols such as (, ), [, and ], or by replacing digits with placeholders. Combined patterns are also introduced to replicate multiple error types.

Using this dataset, we generate a material NER dataset by tagging the collected corpus with material concepts extracted from PubChem, ensuring precise identification of material-related terminology. In this tagging process, Material Name, IU-PAC Name, and Synonym of Material Name are categorized as Material Concept, while Material Formula is tagged separately as Material Formula. This approach maintains a clear distinction between conceptual material entities and their chemical formulas, enabling more accurate entity recognition in materials science applications.

Training the MatDetector We train the MatDetector using the material NER dataset constructed in the previous step and the Trewartha et al. (2022) model architecture. The model achieves high accuracy in detecting material concepts, even in noisy corpora, and provides NER tagging probabilities, estimating the likelihood that a concepts belongs to materials science.

C Additional QA Experiments on MaScQA

To evaluate the generalizability of MATTER beyond classification tasks, we conducted additional experiments on the MaScQA (Zaki et al., 2024) benchmark, which focuses on materials-domain question answering.

Decoder-based setup. We fine-tuned two decoder-based models—Llama-3.2-1B-Instruct and SciBERT—on the HoneyBEE (Song et al., 2023b) instruction dataset and evaluated their performance on MaScQA. MATTER consistently achieved higher accuracy compared to other tokenizations:

Encoder-decoder setup. Following the setup in MatSciNLP (Song et al., 2023a), we used MatSciBERT as the encoder and a transformer-based decoder. We trained on 10% of the HoneyBEE QA data and evaluated on the remaining 90%, simulat-
<table>
  <tr>
    <th>Tokenization</th>
    <th>Accuracy (%)</th>
  </tr>
  <tr>
    <td>BPE</td>
    <td>7.1</td>
  </tr>
  <tr>
    <td>PickyBPE</td>
    <td>7.3</td>
  </tr>
  <tr>
    <td>MATTER (ours)</td>
    <td><b>8.9</b></td>
  </tr>
</table>
Table 10: MaScQA benchmark accuracy using decoder-based models.

ing a low-resource QA scenario. MATTER again yielded the best performance:

<table>
  <tr>
    <th>Tokenization</th>
    <th>Accuracy (%)</th>
  </tr>
  <tr>
    <td>BPE</td>
    <td>20.01</td>
  </tr>
  <tr>
    <td>WordPiece</td>
    <td>22.74</td>
  </tr>
  <tr>
    <td>SAGE</td>
    <td>22.93</td>
  </tr>
  <tr>
    <td>PickyBPE</td>
    <td>21.01</td>
  </tr>
  <tr>
    <td>MATTER (ours)</td>
    <td><b>23.96</b></td>
  </tr>
</table>
Table 11: MaScQA benchmark performance with encoder-decoder model.

These results confirm MATTER’s effectiveness in enhancing QA performance across diverse model architectures and reinforce its generalizability to downstream materials tasks.

D Statistical Significance

Generation task To quantitatively assess the statistical significance of performance improvements introduced by MATTER, we conducted paired t-tests on the <i>average F1 scores</i> across eight generation tasks (NER, RC, EAE, PC, SAR, SC, SF, Overall), comparing MATTER against four widely-used tokenization baselines: BPE, WordPiece, SAGE, and PickyBPE. The average F1 score was computed as the arithmetic mean of the Micro-F1 and Macro-F1 values for each task.

The paired t-test evaluates whether the mean difference in Avg-F1 scores between MATTER and a baseline is statistically significant. The t-statistic is given by:

\[
t = \frac{\bar{d}}{s_d / \sqrt{n}}
\]

where \( \bar{d} \) is the mean of the differences between MATTER and a baseline across tasks, \( s_d \) is the standard deviation of those differences, and \( n = 8 \) is the number of generation tasks.

<table>
  <tr>
    <th>Tokenization</th>
    <th>Avg-F1 (\( p \))</th>
    <th>Significant</th>
  </tr>
  <tr>
    <td>BPE</td>
    <td>0.0009</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>WordPiece</td>
    <td>0.0001</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>SAGE</td>
    <td>0.0066</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>PickyBPE</td>
    <td>0.0155</td>
    <td>Yes</td>
  </tr>
</table>
Table 12: Paired t-test results comparing the average F1 score between MATTER and each baseline across generation tasks.

As shown in Table 12, MATTER achieves statistically significant improvements over all four baselines in terms of average F1 score. All comparisons yield \( p < 0.05 \), confirming that MATTER’s performance gains are unlikely due to random variation. These results reinforce the effectiveness of MATTER’s domain-aware tokenization strategy in improving generation performance across diverse material-related tasks.

Classification task We conducted the same analysis for classification tasks to evaluate whether MATTER’s improvements generalize to discriminative settings. Paired t-tests were performed on the <i>average F1 scores</i> across five classification tasks (SOFC-NER, MatScholar-NER, SF, RC, PC), using the same computation.

<table>
  <tr>
    <th>Tokenization</th>
    <th>Avg-F1 (\( p \))</th>
    <th>Significant</th>
  </tr>
  <tr>
    <td>BPE</td>
    <td>0.0001</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>WordPiece</td>
    <td>0.0001</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>SAGE</td>
    <td>0.0021</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>PickyBPE</td>
    <td>0.0009</td>
    <td>Yes</td>
  </tr>
</table>
Table 13: Paired t-test results comparing the average F1 score between MATTER and each baseline across classification tasks.

As shown in Table 13, all comparisons again yield statistically significant results (\( p < 0.005 \)), confirming that MATTER consistently outperforms all baselines in overall classification performance. This aggregated F1-based analysis further demonstrates the robustness of MATTER’s tokenization advantages in both generation and classification tasks, effectively balancing frequency-weighted and class-balanced evaluation perspectives.
E  Details of validation on materials NER

<table>
  <tr>
    <th>Tool</th>
    <th>MatScholar</th>
    <th>SOFC</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>ChemDataExtractor</td>
    <td>12%</td>
    <td>24%</td>
    <td>18%</td>
  </tr>
  <tr>
    <td>MatDetector (ours)</td>
    <td><b>53%</b></td>
    <td><b>60%</b></td>
    <td><b>57%</b></td>
  </tr>
</table>

Table 14: Recall of two material concept extraction tools on external materials NER datasets—MatScholar and SOFC.

<table>
  <tr>
    <th>Tool</th>
    <th>MatScholar</th>
    <th>SOFC</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>ChemDataExtractor</td>
    <td>52%</td>
    <td>61%</td>
    <td>57%</td>
  </tr>
  <tr>
    <td>MatDetector (ours)</td>
    <td><b>63%</b></td>
    <td><b>75%</b></td>
    <td><b>69%</b></td>
  </tr>
</table>

Table 15: Precision of two material concept extraction tools on external materials NER datasets—MatScholar and SOFC.

<table>
  <tr>
    <th>Tool</th>
    <th>MatScholar</th>
    <th>SOFC</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>ChemDataExtractor</td>
    <td>20%</td>
    <td>34%</td>
    <td>27%</td>
  </tr>
  <tr>
    <td>MatDetector (ours)</td>
    <td><b>58%</b></td>
    <td><b>67%</b></td>
    <td><b>63%</b></td>
  </tr>
</table>

Table 16: F1 Score of two material concept extraction tools on external materials NER datasets—MatScholar and SOFC.

F  Details of the Word-Initial Token Analysis

To validate the effectiveness of our tokenization and avoid any potential circularity in evaluation, we perform an additional analysis using external and independent sources of material-related terms, separate from those used to construct the tokenization. Specifically, we collect named entities from two manually annotated materials NER datasets used in the paper:

<table>
  <tr>
    <th>NER Dataset</th>
    <th>#Material Entity</th>
  </tr>
  <tr>
    <td>MatScholar (<a href="https://doi.org/10.1007/s11671-019-03238-6">Weston et al., 2019</a>)</td>
    <td>8,660</td>
  </tr>
  <tr>
    <td>SOFC (<a href="https://doi.org/10.1007/s11671-020-03447-4">Friedrich et al., 2020</a>)</td>
    <td>1,201</td>
  </tr>
  <tr>
    <td>Total</td>
    <td>9,861</td>
  </tr>
</table>

G  Case Study: Tokenization Robustness

G.1  Analysis in Material Science Papers

In this section, we applied WordPiece, SAGE, PickyBPE, and the proposed method, MATTER, to tokenization results from real materials science papers. As shown in Table 18, existing tokenization methods such as WordPiece, SAGE, and PickyBPE tend to overtokenize important material concepts. For instance, the chemical formula for Lead, "Pb", is split into "p-b", while "dimethylsiloxane" is divided into "dimethyl-sil-oxane or d-imethyl-sil-oxane". Such overtokenization distorts the semantic integrity of material concepts and can degrade the performance of downstream natural language processing tasks.

In contrast, our proposed MATTER method effectively prevents the overtokenization of material concepts. When applying MATTER, essential material concepts such as "Pb", "dimethylsiloxan"e, and "barium sulfate" remain intact, preserving their contextual meaning. Notably, complex material concepts such as "perovskite" and "ethylenediaminetetraacetic acid" are properly maintained, demonstrating that MATTER provides a more suitable tokenization approach for materials science texts.

G.2  Subword Embedding Analysis

To evaluate the impact of different tokenization methods on word representations in materials science, we analyze the nearest neighbors of material concepts based on subword embedding averaging. This experiment is conducted in conjunction with the tokenization results presented in Figure 1 and Table 18, allowing us to assess how subword segmentation affects semantic consistency in word embeddings. We compare four tokenization strategies—WordPiece, SAGE, PickyBPE, and our proposed method, MATTER—by computing word embeddings as the mean of their constituent subword embeddings. The similarity between words is measured using cosine similarity, and the five nearest neighbors (5-NN) for each concept are retrieved. The retrieved neighbors allow us to assess whether the tokenization method preserves materials science semantics or introduces artifacts from suboptimal subword segmentation. The dataset used for evaluation includes materials science terminology, chemical formulas, and domain-specific abbreviations, ensuring a realistic assessment of tokenization impact.

The results, presented in Table 19, indicate that WordPiece and SAGE exhibit a strong tendency to retrieve words that share surface-level subword structures rather than those with true material relevance. For instance, 'germanium' is tokenized as german-ium in WordPiece, leading to nearest neigh-
<table>
  <tr>
    <th>Type of material concept</th>
    <th>#material concept</th>
  </tr>
  <tr>
    <td>Material name</td>
    <td>22,482</td>
  </tr>
  <tr>
    <td>IUPAC name</td>
    <td>22,482</td>
  </tr>
  <tr>
    <td>Synonym of material name</td>
    <td>719,885</td>
  </tr>
  <tr>
    <td>Material formula</td>
    <td>22,479</td>
  </tr>
</table>

Table 17: Summary of approximately 80K extracted material concepts from PubMed, categorized by concepts type.

![Line graph comparing Micro-F1 scores for ChemDataExtractor and MatDetector across different λ values.](page_362_682_482_246.png)

Figure 6: Comparison of Micro-F1 scores for ChemDataExtractor and MatDetector across different \( \lambda \) values.

bors such as 'german' and '-ium', which lack meaningful chemical associations. PickyBPE partially alleviates this issue by merging frequent subwords, but still retrieves words that reflect tokenization artifacts rather than conceptually related material concepts. In contrast, our MATTER method significantly improves semantic alignment by retrieving chemically relevant words. For example, the nearest neighbors of 'germanium' include 'dithiocarbamate', 'ammonium', and 'borohydride', demonstrating a stronger connection to materials science concepts. Similarly, 'ethylenediaminetetra-acetic' acid retrieves '-oxycarb' and '-sulfonyl', which accurately reflect its chemical properties. These results suggest that MATTER effectively mitigates tokenization-induced distortions, leading to more precise materials science word representations that enhance performance in downstream NLP tasks such as entity linking, material property prediction, and knowledge graph construction.

G.3 Comparison of Lambda Details

The Macro-F1 scores for ChemDataExtractor and MatDetector were compared across different \( \lambda \) values to evaluate their performance. The specific numerical values are detailed in Table 20 and Table 21, while Figure 6 provides a visual representation for easier interpretation.
<table>
  <tr>
    <th>Method</th>
    <th>Tokenized Output</th>
  </tr>
  <tr>
    <td>Origin</td>
    <td><b>poly ( dimethylsiloxane ) ( pdms )</b> waschosen to form the <b>elastomeric</b> circuit board, mechanically protecting and electrically insulating the wires, based on itsdurability, adjustable stiffness, <b>biocompatibility</b>, and commer-cial availability as an insulating compound. (<i>Gray et al., 2004</i>)</td>
  </tr>
  <tr>
    <td>WordPiece</td>
    <td><b>poly ( dimethyl-sil-oxane ) ( pd-ms )</b> was-cho-sen to form the <b>elast-omeric</b> circuit board , mechanically protecting and electrically ins-ultating the wires , based on its-du-rab-ility , adjustable stiffness , <b>bio-c-omp-at-i-bility</b> , and comme-r-ci-al availability as an ins-ultating compound.</td>
  </tr>
  <tr>
    <td>SAGE</td>
    <td><b>poly ( dimethyl-siloxane ) ( pd-ms )</b> was-cho-sen to form the <b>elastomer-ic</b> circuit board , mechanically protecting and electrically insulating the wires , based on its-du-rab-ility , adjustable stiffness , <b>biocompatibility</b> , and comme-r-ci-al availability as an insulating compound.</td>
  </tr>
  <tr>
    <td>PickyBPE</td>
    <td><b>p-o-ly ( d-imethyl-sil-xane ) ( p-d-ms )</b> was-ch-osen to form the <b>elast-omeric</b> circuit board , mechanically protecting and electrically insulating the wires , based on its-d-urability , adjustable stiffness , <b>biocompatibility</b> , and comm-er-c-ial availability as an insulating compound.</td>
  </tr>
  <tr>
    <td>MATTER (ours)</td>
    <td><b>poly ( dimethyl-siloxane ) ( pdms )</b> was-cho-sen to form the <b>elastomeric</b> circuit board , mechanically protecting and electrically insulating the wires , based on its-du-rab-ility , adjustable stiffness , <b>biocompatibility</b> , and comme-r-ci-al availability as an insulating compound.</td>
  </tr>
  <tr>
    <td>Origin</td>
    <td>t he waste was solubilized using <b>ethylenediaminetetraacetic acid</b>, and its constituents were determined employing x-ray diffraction and inductively coupled <b>plasma-atomic emission spectrometry</b>, identifying <b>barium</b> sulfate (<b>baso4</b>) as the predominant component at a weight percentage of 67.13%. (<i>Zhang et al., 2025</i>)</td>
  </tr>
  <tr>
    <td>WordPiece</td>
    <td>the waste was solubil-ized using <b>ethylenedi-amine-tetr-aa-ce-tic acid</b>, and its constituents wer determined employing x - ray diffraction and inductively coupled <b>plasma - atomic emission spectrometry</b>, identifying <b>barium</b> sulfate (<b>bas-o-4</b>) as the predominant component at a weight percentage of 67 . 13 %.</td>
  </tr>
  <tr>
    <td>SAGE</td>
    <td>the waste was solub-ilized using <b>ethylene-diam-inet-eta-ra-ace-tic acid</b>, and its constituents were determined employing x - ray diffraction and inductively coupled <b>plasma - atomic emission spectrometry</b>, identifying <b>barium</b> sulfate (<b>bas-o-4</b>) as the predominant component at a weight percent-ag-e of 67 . 13 %.</td>
  </tr>
  <tr>
    <td>PickyBPE</td>
    <td>The waste was solub-ilized using <b>ethyl-ened-i-amin-et-eta-acetic acid</b>, and its constituents were determined employing x - ray diffraction and inductively coupled <b>plasma - atomic emission spectrometry</b>, identifying <b>barium</b> sulfate (<b>b-as-o-4</b>) as the predominant component at a weight percentage of 67.13 %.</td>
  </tr>
  <tr>
    <td>MATTER (ours)</td>
    <td>the waste was solub-ilized using <b>ethylenediaminetetra-acetic acid</b>, and its constituents were determined employing x - ray diffraction and inductively coupled <b>plasma - atomic emission spectrometry</b>, identifying <b>barium</b> sulfate (<b>baso4</b>) as the predominant component at a weight percentage of 67 . 13 %.</td>
  </tr>
</table>
<table>
  <tr>
    <th>Origin</th>
    <td>it should be described that the <span style="color:#ff6b6b">ers</span> signals derived from <span style="color:#ff6b6b">perovskite</span> layers cannot be observed because of their <span style="color:#ff6b6b">pauli paramagnetism</span> nature, resulting in low <span style="color:#ff6b6b">ers</span> intensity, and because of the heavy-atom effects of <span style="color:#ff6b6b">pb</span> or <span style="color:#ff6b6b">sn</span>, leading to short spin-lattice relaxation time and broad <span style="color:#ff6b6b">ers</span> linewidths. (Chen et al., 2025)</td>
  </tr>
  <tr>
    <th>WordPiece</th>
    <td>it should be described that the <span style="color:#ff6b6b">ers</span> signals derived from <span style="color:#ff6b6b">perovskite</span> layers cannot be observed because of their <span style="color:#ff6b6b">paul-i param-agne-tism</span> nature , resulting in low <span style="color:#ff6b6b">ers</span> intensity , and because of the heavy - atom effects of <span style="color:#ff6b6b">p-b</span> or <span style="color:#ff6b6b">sn</span> , leading to short spin - lattice relaxation time and broad <span style="color:#ff6b6b">ers</span> line-width-s.</td>
  </tr>
  <tr>
    <th>SAGE</th>
    <td>it should be described that the <span style="color:#ff6b6b">e-sr</span> signals derived from <span style="color:#ff6b6b">per-o-v-skite</span> layers cannot be ob-served because of their <span style="color:#ff6b6b">paul-i param-agnetism</span> nature , resulting in low <span style="color:#ff6b6b">e-sr</span> intens-ity , and because of the heavy - atom effects of <span style="color:#ff6b6b">p-b</span> or <span style="color:#ff6b6b">sn</span> , leading to short spin - lattice relaxation time and broad <span style="color:#ff6b6b">e-sr</span> linewidth-s.</td>
  </tr>
  <tr>
    <th>PickyBPE</th>
    <td>it should be described that the <span style="color:#ff6b6b">es-r</span> signals derived from <span style="color:#ff6b6b">perovskite</span> layers cannot be observed because of their <span style="color:#ff6b6b">pa-uli param-agnetism</span> nature , resulting in low <span style="color:#ff6b6b">es-r</span> intensity , and because of the heavy - atom effects of <span style="color:#ff6b6b">p-b</span> or <span style="color:#ff6b6b">sn</span> , leading to short spin - lattice relaxation time and broad <span style="color:#ff6b6b">es-r</span> linewidths.</td>
  </tr>
  <tr>
    <th>MATTER (ours)</th>
    <td>it should be described that the <span style="color:#ff6b6b">esr</span> signals derived from <span style="color:#ff6b6b">perovskite</span> layers cannot be observed because of their <span style="color:#ff6b6b">pauli paramagnetism</span> nature , resulting in low <span style="color:#ff6b6b">esr</span> intensity , and because of the heavy - atom effects of <span style="color:#ff6b6b">pb</span> or <span style="color:#ff6b6b">sn</span> , leading to short spin - lattice relaxation time and broad <span style="color:#ff6b6b">esr</span> linewidths.</td>
  </tr>
</table>

<table>
  <tr>
    <th>Origin</th>
    <td>in this study, the <span style="color:#ff6b6b">nanoparticles</span> of the zinc and cobalt <span style="color:#ff6b6b">imidazolate</span> framework (<span style="color:#ff6b6b">znco-zif</span>) were synthesized and directly incorporated into <span style="color:#ff6b6b">polylactide</span> (<span style="color:#ff6b6b">pla</span>) to prepare <span style="color:#ff6b6b">pla / znco-zif fibrous</span> membranes through <span style="color:#ff6b6b">electrospinning</span> methodology. (Deng et al., 2024)</td>
  </tr>
  <tr>
    <th>WordPiece</th>
    <td>in this study , the <span style="color:#ff6b6b">nanoparticles</span> of the zinc and cobalt <span style="color:#ff6b6b">im-ida-zol-ate</span> framework (<span style="color:#ff6b6b">zn-co - zi-f</span>) were synthesized and directly incorporated into <span style="color:#ff6b6b">poly-lact-ide</span> (<span style="color:#ff6b6b">pl-a</span>) to prepare <span style="color:#ff6b6b">pla / zn-co - zi-f fibrous</span> membranes through <span style="color:#ff6b6b">electros-pin-ning</span> methodology.</td>
  </tr>
  <tr>
    <th>SAGE</th>
    <td>in this study , the <span style="color:#ff6b6b">nanoparticles</span> of the zinc and cobalt <span style="color:#ff6b6b">imid-azol-ate</span> framework (<span style="color:#ff6b6b">z-nc-o - z-if</span>) were synthesized and directly incorporated into <span style="color:#ff6b6b">poly-l-act-ide</span> (<span style="color:#ff6b6b">pl-a</span>) to prepare <span style="color:#ff6b6b">pl-a / z-nc-o - z-if fibrous</span> membranes through <span style="color:#ff6b6b">electrospinning</span> methodology.</td>
  </tr>
  <tr>
    <th>PickyBPE</th>
    <td>in this study, the <span style="color:#ff6b6b">nanoparticles</span> of the zinc and cobalt <span style="color:#ff6b6b">imid-az-olate</span> framework (<span style="color:#ff6b6b">z-n-co - z-if</span>) were synthesized and directly incorporated into <span style="color:#ff6b6b">pol-y-l-actide</span> (<span style="color:#ff6b6b">pl-a</span>) to prepare <span style="color:#ff6b6b">pl-a / z-n-co - z-if fibrous</span> membranes through <span style="color:#ff6b6b">electrospinning</span> methodology.</td>
  </tr>
  <tr>
    <th>MATTER (ours)</th>
    <td>in this study , the <span style="color:#ff6b6b">nanoparticles</span> of the zinc and cobalt <span style="color:#ff6b6b">imidazol-ate</span> framework (<span style="color:#ff6b6b">zn-co - zif</span>) were synthesized and directly incorporated into <span style="color:#ff6b6b">polylactide</span> (<span style="color:#ff6b6b">pla</span>) to prepare <span style="color:#ff6b6b">pla / zn-co - zif fibrous</span> membranes through <span style="color:#ff6b6b">electrospinning</span> methodology.</td>
  </tr>
</table>

Table 18: Boldface and pink concepts are important material concepts extracted using MatDetector. Boldface concepts are correctly tokenized in both the baseline and our method, indicating no issues. In contrast, pink concepts are highly important but are often split into unrelated subwords or overtokenized in conventional tokenization. However, as shown in this table, our method, MATTER, effectively prevents the overtokenization of important material concepts, preserving their semantic integrity.
<table>
  <tr>
    <th rowspan="2">Tokenization Method</th>
    <th rowspan="2">Concept</th>
    <th colspan="2">Word Embedding 5-NN</th>
    <th colspan="2">Word Embedding 5-NN</th>
    <th colspan="2">Word Embedding 5-NN</th>
  </tr>
  <tr>
    <th>Sim.</th>
    <th>Formula</th>
    <th>Sim.</th>
    <th>Abbr</th>
    <th>Sim.</th>
  </tr>
  <tr>
    <td rowspan="4">WordPiece</td>
    <td>agilent</td>
    <td>90.6</td>
    <td></td>
    <td>nowak</td>
    <td>81.8</td>
    <td>inlet</td>
    <td>95.5</td>
  </tr>
  <tr>
    <td>fri</td>
    <td>85.9</td>
    <td>PbI2</td>
    <td>10c</td>
    <td>81.8</td>
    <td>chattopadhyay</td>
    <td>93.7</td>
  </tr>
  <tr>
    <td>germanium (german-iun)</td>
    <td>stephan</td>
    <td>85.3</td>
    <td>(pbi-2)</td>
    <td>-agnetically</td>
    <td>81.0</td>
    <td>LFP</td>
    <td>1263.0</td>
    <td>92.7</td>
  </tr>
  <tr>
    <td></td>
    <td>valley</td>
    <td>85.2</td>
    <td></td>
    <td>colouring</td>
    <td>79.6</td>
    <td>(ll-p)</td>
    <td>foreland</td>
    <td>92.7</td>
  </tr>
  <tr>
    <td></td>
    <td>galvanic</td>
    <td>86.1</td>
    <td></td>
    <td><b>quasicrystal</b></td>
    <td>79.6</td>
    <td>-rink</td>
    <td>92.6</td>
  </tr>
  <tr>
    <td rowspan="4">SAGE</td>
    <td>lot</td>
    <td>83.0</td>
    <td></td>
    <td>-gen</td>
    <td>43.9</td>
    <td>occupation</td>
    <td>95.8</td>
  </tr>
  <tr>
    <td>germanium (german-iun)</td>
    <td>segregation</td>
    <td>82.8</td>
    <td>PbI2</td>
    <td>pounds</td>
    <td>43.6</td>
    <td>multiphonon</td>
    <td>95.2</td>
  </tr>
  <tr>
    <td></td>
    <td>100</td>
    <td>82.9</td>
    <td>(p-bi-2)</td>
    <td>-pt</td>
    <td>43.5</td>
    <td>LFP</td>
    <td>-l</td>
    <td>95.2</td>
  </tr>
  <tr>
    <td></td>
    <td>segregation</td>
    <td>82.9</td>
    <td></td>
    <td>-uck</td>
    <td>43.2</td>
    <td>(ll-p)</td>
    <td>-circ</td>
    <td>95.1</td>
  </tr>
  <tr>
    <td></td>
    <td>agi</td>
    <td>82.0</td>
    <td></td>
    <td>-s</td>
    <td>43.2</td>
    <td>multiphonon</td>
    <td>95.1</td>
  </tr>
  <tr>
    <td rowspan="4">PickyBPE</td>
    <td>lot</td>
    <td>83.0</td>
    <td></td>
    <td>gaussian</td>
    <td>63.1</td>
    <td>her,</td>
    <td>75.8</td>
  </tr>
  <tr>
    <td>germanium (g-eran-iun)</td>
    <td>segregation</td>
    <td>82.8</td>
    <td>PbI2</td>
    <td>p</td>
    <td>62.8</td>
    <td>consideration</td>
    <td>75.0</td>
  </tr>
  <tr>
    <td></td>
    <td>-inov</td>
    <td>81.3</td>
    <td>(p-bi-2)</td>
    <td>total</td>
    <td>62.3</td>
    <td>LFP</td>
    <td>-ermany</td>
    <td>75.0</td>
  </tr>
  <tr>
    <td></td>
    <td>-w.</td>
    <td>81.1</td>
    <td></td>
    <td>-s</td>
    <td>62.0</td>
    <td>(l-l-p)</td>
    <td>-sd102</td>
    <td>75.0</td>
  </tr>
  <tr>
    <td></td>
    <td>compatibility</td>
    <td>81.0</td>
    <td></td>
    <td>-ories</td>
    <td>61.0</td>
    <td>{||}40{||}</td>
    <td>75.0</td>
  </tr>
  <tr>
    <td rowspan="4">MATTER (ours)</td>
    <td>dithiocarbamate</td>
    <td>81.5</td>
    <td></td>
    <td>ph5</td>
    <td>89.9</td>
    <td>zr7f</td>
    <td>90.9</td>
  </tr>
  <tr>
    <td>germanium (germanium)</td>
    <td>monium</td>
    <td>81.4</td>
    <td>PbI2</td>
    <td>ph2</td>
    <td>89.2</td>
    <td>LFP</td>
    <td>acyclohex</td>
    <td>90.8</td>
  </tr>
  <tr>
    <td></td>
    <td>-orib</td>
    <td>81.3</td>
    <td>(ph2)</td>
    <td>-anesulfonic</td>
    <td>88.9</td>
    <td>(lip)</td>
    <td>dodecane</td>
    <td>90.0</td>
  </tr>
  <tr>
    <td></td>
    <td>borohydride</td>
    <td>81.2</td>
    <td></td>
    <td>-nh2o3</td>
    <td>88.7</td>
    <td></td>
    <td>-acyclohex</td>
    <td>90.0</td>
  </tr>
  <tr>
    <td></td>
    <td>-stannyl</td>
    <td>81.2</td>
    <td></td>
    <td>-dithiocarbamate</td>
    <td>88.5</td>
    <td></td>
    <td>-azobenzene</td>
    <td>90.0</td>
  </tr>
  <tr>
    <td rowspan="2">Tokenization Method</td>
    <td>Concept</td>
    <td colspan="2">Word Embedding 5-NN</td>
    <td colspan="2">Word Embedding 5-NN</td>
    <td colspan="2">Word Embedding 5-NN</td>
  </tr>
  <tr>
    <td></td>
    <td>Sim.</td>
    <td>Formula</td>
    <td>Sim.</td>
    <td>Abbr</td>
    <td>Sim.</td>
  </tr>
  <tr>
    <td rowspan="4">WordPiece</td>
    <td>-oreg</td>
    <td>92.4</td>
    <td></td>
    <td>bas</td>
    <td>91.3</td>
    <td>-ms</td>
    <td>87.6</td>
  </tr>
  <tr>
    <td>sulphates</td>
    <td>92.3</td>
    <td>BaSo4</td>
    <td>-o4</td>
    <td>87.3</td>
    <td>PDMS</td>
    <td>pd</td>
    <td>82.9</td>
  </tr>
  <tr>
    <td>consistence</td>
    <td>92.2</td>
    <td>(bas-o-4)</td>
    <td>adopts</td>
    <td>87.0</td>
    <td>(pd-ms)</td>
    <td>drilled</td>
    <td>75.2</td>
  </tr>
  <tr>
    <td>crop</td>
    <td>92.1</td>
    <td></td>
    <td>somehow</td>
    <td>85.3</td>
    <td>gilbert</td>
    <td>75.1</td>
  </tr>
  <tr>
    <td></td>
    <td>-ulos</td>
    <td>92.1</td>
    <td></td>
    <td>reflect</td>
    <td>85.1</td>
    <td>connect</td>
    <td>74.3</td>
  </tr>
  <tr>
    <td rowspan="4">SAGE</td>
    <td>-lient</td>
    <td>94.8</td>
    <td></td>
    <td>bas</td>
    <td>85.2</td>
    <td>p</td>
    <td>93.1</td>
  </tr>
  <tr>
    <td>ethylenediaminetetra-acetic acid (ethylenedi-amine-tetr-aa-ce-tic acid)</td>
    <td>othermal</td>
    <td>94.7</td>
    <td>BaSo4</td>
    <td>nanobelts</td>
    <td>75.3</td>
    <td>others</td>
    <td>91.5</td>
  </tr>
  <tr>
    <td></td>
    <td>-stoichi</td>
    <td>94.6</td>
    <td>(bas-o-4)</td>
    <td>interv</td>
    <td>75.2</td>
    <td>PDMS (pd-ms)</td>
    <td>heas</td>
    <td>91.4</td>
  </tr>
  <tr>
    <td></td>
    <td>thermogravimetric</td>
    <td>94.6</td>
    <td></td>
    <td>)(</td>
    <td>75.1</td>
    <td>ellips</td>
    <td>91.3</td>
  </tr>
  <tr>
    <td></td>
    <td>-true</td>
    <td>94.5</td>
    <td></td>
    <td>-rino</td>
    <td>74.8</td>
    <td>-rac</td>
    <td>91.2</td>
  </tr>
  <tr>
    <td rowspan="4">PickyBPE</td>
    <td>contrast.</td>
    <td>89.6</td>
    <td></td>
    <td>sliding</td>
    <td>71.6</td>
    <td>ppe</td>
    <td>76.5</td>
  </tr>
  <tr>
    <td>represents</td>
    <td>89.2</td>
    <td>BaSo4</td>
    <td>charged</td>
    <td>69.4</td>
    <td>;</td>
    <td>69.9</td>
  </tr>
  <tr>
    <td>ethyl-ened-i-amin-et-tra-acetic acid</td>
    <td>sophistic</td>
    <td>89.1</td>
    <td>(bas-o-4)</td>
    <td>-adi</td>
    <td>67.7</td>
    <td>PDMDS (p-d-ms)</td>
    <td>prem</td>
    <td>69.5</td>
  </tr>
  <tr>
    <td></td>
    <td>zm(II)</td>
    <td>89.1</td>
    <td>2013.0</td>
    <td>inductively</td>
    <td>60.2</td>
    <td></td>
    <td>inductively</td>
    <td>66.6</td>
  </tr>
  <tr>
    <td></td>
    <td>distribution</td>
    <td>89.0</td>
    <td></td>
    <td>mocvd</td>
    <td>59.8</td>
    <td>bat</td>
    <td>66.6</td>
  </tr>
  <tr>
    <td rowspan="4">MATTER (ours)</td>
    <td>ethylenediaminetetra-acetic acid</td>
    <td>93.7</td>
    <td></td>
    <td>bas</td>
    <td>91.4</td>
    <td>perfluoroalkyl</td>
    <td>85.7</td>
  </tr>
  <tr>
    <td></td>
    <td>-acetic</td>
    <td>93.6</td>
    <td>BaSo4</td>
    <td>-o4</td>
    <td>87.5</td>
    <td>PDMS (pdms)</td>
    <td>trimethoxysilyl</td>
    <td>85.6</td>
  </tr>
  <tr>
    <td></td>
    <td>-oxy carb</td>
    <td>91.8</td>
    <td>(baso40)</td>
    <td>bast</td>
    <td>84.4</td>
    <td>-loxy</td>
    <td>85.5</td>
  </tr>
  <tr>
    <td></td>
    <td>-sulfanyl</td>
    <td>91.7</td>
    <td></td>
    <td>cyclohexyl</td>
    <td>82.0</td>
    <td>-benzoic</td>
    <td>85.4</td>
  </tr>
  <tr>
    <td></td>
    <td>agree</td>
    <td>91.6</td>
    <td></td>
    <td>-cyclopentadienyl</td>
    <td>81.9</td>
    <td>borohyd</td>
    <td>85.4</td>
  </tr>
</table>

Table 19: Comparison of subword embedding averaging results across different tokenization methods, including WordPiece, SAGE, PickyBPE, and our proposed method, MATTER. The table presents the five nearest neighbor words based on subword embedding averages for each method, illustrating how different tokenization strategies impact semantic similarity in word embeddings. The similarity scores (Sim.) indicate the relevance of the nearest neighbors to the target material concept. Boldface highlights words that are directly related to materials.
<table>
  <tr>
    <th rowspan="2">MatDetector (ours)</th>
    <th colspan="8">MatSci-NLP</th>
  </tr>
  <tr>
    <th>NER</th>
    <th>RC</th>
    <th>EAE</th>
    <th>PC</th>
    <th>SAR</th>
    <th>SC</th>
    <th>SF</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>w/o material knowledge</td>
    <td>Micro-F1</td>
    <td>76.6</td>
    <td>80.9</td>
    <td>48.5</td>
    <td>73.1</td>
    <td>81.9</td>
    <td>90.0</td>
    <td>57.4</td>
    <td>72.6</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>56.1</td>
    <td>58.5</td>
    <td>29.4</td>
    <td>58.9</td>
    <td>74.6</td>
    <td>60.3</td>
    <td>32.6</td>
    <td>52.9</td>
  </tr>
  <tr>
    <td rowspan="2">0.1</td>
    <td>Micro-F1</td>
    <td>76.4</td>
    <td>78.6</td>
    <td>47.4</td>
    <td>74.1</td>
    <td>79.5</td>
    <td>91.0</td>
    <td>61.2</td>
    <td>72.6</td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>54.3</td>
    <td>54.4</td>
    <td>32.6</td>
    <td>69.5</td>
    <td>62.7</td>
    <td>61.1</td>
    <td>37.0</td>
    <td>53.1</td>
  </tr>
  <tr>
    <td rowspan="2">0.2</td>
    <td>Micro-F1</td>
    <td>78.3</td>
    <td>78.7</td>
    <td>49.7</td>
    <td>74.5</td>
    <td>76.2</td>
    <td><b>91.2</b></td>
    <td>60.4</td>
    <td>72.7</td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>58.5</td>
    <td>53.1</td>
    <td>30.2</td>
    <td>68.8</td>
    <td>63.3</td>
    <td>58.0</td>
    <td>36.7</td>
    <td>52.6</td>
  </tr>
  <tr>
    <td rowspan="2">0.3</td>
    <td>Micro-F1</td>
    <td>78.5</td>
    <td>80.2</td>
    <td>51.2</td>
    <td><b>76.6</b></td>
    <td>73.9</td>
    <td>91.5</td>
    <td><b>62.7</b></td>
    <td>73.5</td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>56.2</td>
    <td>55.7</td>
    <td>35.7</td>
    <td>69.2</td>
    <td>58.9</td>
    <td>62.5</td>
    <td><b>41.3</b></td>
    <td>54.2</td>
  </tr>
  <tr>
    <td rowspan="2">0.4</td>
    <td>Micro-F1</td>
    <td>75.4</td>
    <td>80.7</td>
    <td>52.9</td>
    <td>73.3</td>
    <td>77.2</td>
    <td>90.6</td>
    <td>59.1</td>
    <td>72.7</td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>54.5</td>
    <td>56.5</td>
    <td>33.4</td>
    <td>68.4</td>
    <td>59.9</td>
    <td>63.5</td>
    <td>35.0</td>
    <td>53.0</td>
  </tr>
  <tr>
    <td rowspan="2">0.5</td>
    <td>Micro-F1</td>
    <td>78.7</td>
    <td>82.4</td>
    <td>53.0</td>
    <td>74.2</td>
    <td>76.2</td>
    <td>90.8</td>
    <td>60.2</td>
    <td>73.6</td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>55.2</td>
    <td>58.4</td>
    <td>32.6</td>
    <td>66.5</td>
    <td>63.9</td>
    <td>61.5</td>
    <td>33.5</td>
    <td>53.1</td>
  </tr>
  <tr>
    <td rowspan="2">0.6</td>
    <td>Micro-F1</td>
    <td>77.8</td>
    <td><b>83.6</b></td>
    <td>49.6</td>
    <td><b>77.3</b></td>
    <td>78.3</td>
    <td><b>91.2</b></td>
    <td>61.2</td>
    <td>74.1</td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>57.2</td>
    <td><b>60.7</b></td>
    <td>30.7</td>
    <td>68.9</td>
    <td>63.1</td>
    <td>59.3</td>
    <td>38.0</td>
    <td>54.0</td>
  </tr>
  <tr>
    <td rowspan="2">0.7</td>
    <td>Micro-F1</td>
    <td>78.3</td>
    <td>81.5</td>
    <td>48.3</td>
    <td>74.5</td>
    <td><b>82.4</b></td>
    <td>90.9</td>
    <td>59.8</td>
    <td>73.7</td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>56.7</td>
    <td>58.0</td>
    <td>34.9</td>
    <td>69.7</td>
    <td>60.0</td>
    <td>59.6</td>
    <td>37.4</td>
    <td>53.8</td>
  </tr>
  <tr>
    <td rowspan="2">0.8</td>
    <td>Micro-F1</td>
    <td>76.9</td>
    <td>80.4</td>
    <td><b>54.0</b></td>
    <td>75.2</td>
    <td>80.4</td>
    <td>91.1</td>
    <td>61.5</td>
    <td><b>74.2</b></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>54.9</td>
    <td>55.7</td>
    <td>37.0</td>
    <td><b>71.4</b></td>
    <td><b>74.7</b></td>
    <td>59.5</td>
    <td>37.8</td>
    <td>55.9</td>
  </tr>
  <tr>
    <td rowspan="2">0.9</td>
    <td>Micro-F1</td>
    <td><b>79.0</b></td>
    <td>81.3</td>
    <td>53.1</td>
    <td>74.2</td>
    <td>78.6</td>
    <td>90.2</td>
    <td>60.8</td>
    <td>73.9</td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td><b>58.8</b></td>
    <td>59.0</td>
    <td><b>37.3</b></td>
    <td>69.5</td>
    <td>64.1</td>
    <td><b>66.4</b></td>
    <td>37.7</td>
    <td><b>56.1</b></td>
  </tr>
  <tr>
    <td rowspan="2">1.0</td>
    <td>Micro-F1</td>
    <td><b>80.0</b></td>
    <td><b>83.8</b></td>
    <td><b>53.1</b></td>
    <td>73.7</td>
    <td><b>85.5</b></td>
    <td><b>91.2</b></td>
    <td><b>61.9</b></td>
    <td><b>75.6</b></td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td><b>59.3</b></td>
    <td><b>59.1</b></td>
    <td><b>36.9</b></td>
    <td>67.6</td>
    <td><b>79.3</b></td>
    <td><b>64.9</b></td>
    <td>38.0</td>
    <td><b>57.9</b></td>
  </tr>
  <tr>
    <td rowspan="2">2.0</td>
    <td>Micro-F1</td>
    <td>77.3</td>
    <td>79.2</td>
    <td>52.1</td>
    <td>75.1</td>
    <td>75.7</td>
    <td>91.1</td>
    <td>61.1</td>
    <td>73.1</td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>55.7</td>
    <td>55.9</td>
    <td>36.6</td>
    <td>66.6</td>
    <td>62.6</td>
    <td>60.1</td>
    <td>38.0</td>
    <td>53.7</td>
  </tr>
  <tr>
    <td rowspan="2">3.0</td>
    <td>Micro-F1</td>
    <td>76.1</td>
    <td>79.2</td>
    <td>50.5</td>
    <td>71.6</td>
    <td>77.9</td>
    <td>90.2</td>
    <td>61.5</td>
    <td>72.4</td>
  </tr>
  <tr>
    <td>Macro-F1</td>
    <td>54.2</td>
    <td>57.6</td>
    <td>34.2</td>
    <td>65.9</td>
    <td>63.8</td>
    <td>59.7</td>
    <td><b>38.6</b></td>
    <td>53.4</td>
  </tr>
</table>

Table 20: Specific numerical results of MatDetector’s Macro-F1 and Micro-F1 scores across different \( \lambda \) values.
<table>
  <tr>
    <th rowspan="2">ChemDataExtractor (Swain and Cole, 2016)</th>
    <th colspan="8">MatSci-NLP</th>
  </tr>
  <tr>
    <th>NER</th>
    <th>RC</th>
    <th>EAE</th>
    <th>PC</th>
    <th>SAR</th>
    <th>SC</th>
    <th>SF</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>w/o material knowledge</td>
    <td>Micro-F1</td>
    <td>76.6</td>
    <td>80.9</td>
    <td>48.5</td>
    <td>73.1</td>
    <td>81.9</td>
    <td>90.0</td>
    <td>57.4</td>
    <td>72.6</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>56.1</td>
    <td>58.5</td>
    <td>29.4</td>
    <td>58.9</td>
    <td><b>74.6</b></td>
    <td>60.3</td>
    <td>32.6</td>
    <td>52.9</td>
  </tr>
  <tr>
    <td>0.1</td>
    <td>Micro-F1</td>
    <td>75.5</td>
    <td>81.0</td>
    <td>52.7</td>
    <td>72.8</td>
    <td>77.9</td>
    <td>90.4</td>
    <td>55.8</td>
    <td>72.3</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>52.4</td>
    <td>61.1</td>
    <td>34.4</td>
    <td>63.0</td>
    <td>66.6</td>
    <td>62.2</td>
    <td>31.9</td>
    <td>53.1</td>
  </tr>
  <tr>
    <td>0.2</td>
    <td>Micro-F1</td>
    <td>76.4</td>
    <td>83.2</td>
    <td>52.1</td>
    <td>70.5</td>
    <td>76.7</td>
    <td>91.0</td>
    <td>58.6</td>
    <td>72.6</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td><u>56.3</u></td>
    <td>61.0</td>
    <td>32.8</td>
    <td>64.8</td>
    <td>63.8</td>
    <td>60.3</td>
    <td>34.2</td>
    <td>53.3</td>
  </tr>
  <tr>
    <td>0.3</td>
    <td>Micro-F1</td>
    <td>75.4</td>
    <td>82.3</td>
    <td>52.3</td>
    <td>73.1</td>
    <td>76.4</td>
    <td>90.4</td>
    <td>56.8</td>
    <td>72.4</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>53.2</td>
    <td>61.0</td>
    <td>29.8</td>
    <td>65.2</td>
    <td>64.4</td>
    <td>58.9</td>
    <td>33.0</td>
    <td>52.2</td>
  </tr>
  <tr>
    <td>0.4</td>
    <td>Micro-F1</td>
    <td>73.4</td>
    <td>84.0</td>
    <td><b>55.1</b></td>
    <td>71.9</td>
    <td>79.5</td>
    <td>90.6</td>
    <td>57.1</td>
    <td>73.1</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>51.4</td>
    <td>58.4</td>
    <td><b>37.9</b></td>
    <td><b>68.8</b></td>
    <td>66.6</td>
    <td>60.0</td>
    <td>30.5</td>
    <td>53.4</td>
  </tr>
  <tr>
    <td>0.5</td>
    <td>Micro-F1</td>
    <td><b>77.0</b></td>
    <td>82.3</td>
    <td>53.8</td>
    <td>72.2</td>
    <td>79.8</td>
    <td>91.0</td>
    <td>57.7</td>
    <td>73.4</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td><u>56.4</u></td>
    <td>61.3</td>
    <td>35.8</td>
    <td><u>67.7</u></td>
    <td>62.0</td>
    <td>61.8</td>
    <td>30.8</td>
    <td>53.7</td>
  </tr>
  <tr>
    <td>0.6</td>
    <td>Micro-F1</td>
    <td>76.5</td>
    <td><u>84.1</u></td>
    <td><u>54.1</u></td>
    <td>67.3</td>
    <td>78.2</td>
    <td><b>91.2</b></td>
    <td>57.8</td>
    <td>72.7</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>55.1</td>
    <td>61.8</td>
    <td>36.6</td>
    <td>59.6</td>
    <td>66.2</td>
    <td>61.9</td>
    <td>33.6</td>
    <td>53.5</td>
  </tr>
  <tr>
    <td>0.7</td>
    <td>Micro-F1</td>
    <td>75.4</td>
    <td>82.4</td>
    <td>52.5</td>
    <td>71.8</td>
    <td><b>82.3</b></td>
    <td>90.2</td>
    <td>58.0</td>
    <td>73.2</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>54.5</td>
    <td>60.7</td>
    <td>33.3</td>
    <td>65.8</td>
    <td>68.1</td>
    <td>63.2</td>
    <td><b>37.9</b></td>
    <td>54.8</td>
  </tr>
  <tr>
    <td>0.8</td>
    <td>Micro-F1</td>
    <td>76.0</td>
    <td>84.0</td>
    <td>53.4</td>
    <td>71.1</td>
    <td>78.1</td>
    <td>91.1</td>
    <td>58.1</td>
    <td>73.1</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>55.5</td>
    <td><u>62.8</u></td>
    <td>34.7</td>
    <td>64.7</td>
    <td>65.8</td>
    <td><b>65.0</b></td>
    <td>35.5</td>
    <td><u>54.9</u></td>
  </tr>
  <tr>
    <td>0.9</td>
    <td>Micro-F1</td>
    <td>75.6</td>
    <td>81.9</td>
    <td>52.8</td>
    <td>73.3</td>
    <td>82.0</td>
    <td>91.1</td>
    <td>58.4</td>
    <td>73.6</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>54.1</td>
    <td>59.0</td>
    <td><u>37.5</u></td>
    <td>67.5</td>
    <td>65.5</td>
    <td>58.0</td>
    <td>35.7</td>
    <td>53.9</td>
  </tr>
  <tr>
    <td>1.0</td>
    <td>Micro-F1</td>
    <td><b>77.1</b></td>
    <td>81.5</td>
    <td>53.1</td>
    <td><b>73.6</b></td>
    <td>80.6</td>
    <td><b>91.2</b></td>
    <td><b>58.8</b></td>
    <td><b>73.7</b></td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td><u>56.4</u></td>
    <td>58.9</td>
    <td>35.0</td>
    <td>67.6</td>
    <td>68.0</td>
    <td><u>64.8</u></td>
    <td><u>35.6</u></td>
    <td><u>55.2</u></td>
  </tr>
  <tr>
    <td>2.0</td>
    <td>Micro-F1</td>
    <td><u>77.0</u></td>
    <td><b>84.7</b></td>
    <td>52.3</td>
    <td>69.4</td>
    <td>80.7</td>
    <td><u>91.1</u></td>
    <td>57.3</td>
    <td>73.2</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>55.3</td>
    <td><b>64.2</b></td>
    <td>34.1</td>
    <td>64.3</td>
    <td><u>68.2</u></td>
    <td>60.3</td>
    <td>35.4</td>
    <td>54.5</td>
  </tr>
  <tr>
    <td>3.0</td>
    <td>Micro-F1</td>
    <td>76.1</td>
    <td>83.2</td>
    <td>52.0</td>
    <td>67.7</td>
    <td>75.6</td>
    <td>90.2</td>
    <td><u>58.7</u></td>
    <td>71.9</td>
  </tr>
  <tr>
    <td></td>
    <td>Macro-F1</td>
    <td>55.5</td>
    <td>60.6</td>
    <td>34.0</td>
    <td>65.8</td>
    <td>67.8</td>
    <td>60.6</td>
    <td>34.5</td>
    <td>54.1</td>
  </tr>
</table>

Table 21: Specific numerical results of ChemDataExtractor’s Macro-F1 and Micro-F1 scores across different \( \lambda \) values.

Reviewer paper summaries: TLDR:
- The paper introduces MATTER, a novel tokenization framework specifically designed for materials science texts.
- This paper also introduces MatDetector, a material concept identifier trained on a curated materials knowledge base, which can be used in the MATTER framework.

Traditional tokenization methods, such as Byte Pair Encoding and WordPiece, often fragment material-specific terms, leading to semantic loss. To address this problem, the authors introduce MATTER, a novel tokenization framework that integrates domain knowledge into materials science tokenization. MATTER adjusts token frequency by assigning higher weights to material-related terms identified by MatDetector, thereby prioritizing and preserving the material concepts during tokenization. Experimental results show that MATTER achieves an average performance gain of 2.3% in generation tasks and 3.0% in classification tasks compared to existing tokenization methods.

This paper argues that existing tokenizers overindex on high-frequency words, and as a result perform poorly on important but low frequency words, which is a class that includes domain specific vocabulary in materials science. To address this issue, they propose Matter, a tokenizer that adjusts the frequency of domain-specific words that they detect with a new system called MatDetector. They evaluate the downstream performance of models trained with a variety of tokenizers on a number of tasks, and find that the Matter tokenization algorithm results in improvements across most tasks. 

Overall, I found this paper strong, and that it supports its claims with adequate proof. However, I think the framing of this paper could be adjusted in light of the potential for future work that expands the scope and impact of the contribution; in effect, this paper establishes that materials NLP can benefit from domain-specific tokenization, but does not fully explore all of the avenues that it introduces to do so. I don't think that needs to be in this paper necessarily, but framing this as a platform for future work would hopefully encourage that future work.
Summary of strengths: 1. The authors identify the key tokenization issue in materials science modeling and propose MATTER. Experimental results demonstrate that the new tokenization algorithm enhances performance in materials-related tasks (Table 1).
2. The introduction of MatDetector significantly improves material concept identification compared to the previous ChemDataExtractor. This new model serves as a valuable component in the MATTER framework (Table 6).
3. The authors provide an in-depth analysis of MatDetector, such as how it influences the word-initial tokens and how it leads to meaningful segmentation of material terms.

This paper is well-motivated, and situates itself well in related research. I think this is an important problem to be working on, and tokenization is an underaddressed source of poor performance in specialized domains like materials science. 

This paper also does a solid job of evaluating the downstream effects of this method. I'm convinced that domain-aware tokenization in materials science is ultimately beneficial because of this paper, and I think that there's a lot of interesting followup work that this paper generates.
Summary of weaknesses: 1. MATTER outperforms other tokenization methods, but the improvements in Micro-F1 and Macro-F1 scores are often modest (1-2% in classification tasks, Table 2). The authors should conduct further analysis to demonstrate the statistical significance of these methods.
2. The MatDetector is trained on a manually curated material knowledge base, which could bias tokenization toward known materials. This might limit the models' ability to handle emerging materials or novel formulations.

Overall, I think this paper is strong, but I do have a few concerns. These concerns don't really question the validity of the result here — i.e. that there's real benefit in domain-aware tokenization — but are more concerned with the magnitude of that benefit: to what extent is this paper a good estimate of that? I bring this up because the way the paper is framed argues that Matter effectively addresses the question of incorporating domain knowledge, but I think there's several unanswered questions still. I enumerate them below:

**Validation of MatDetector**: While the paper claims that they validate the performance of MatDetector in a couple of ways, they specify very few details. They argue on L343, for example, that MatDetector gets 6x the chemical entities as compared to ChemDataExtractor, and is "cross referenced [] with materials-specific databases," but how is that cross-referencing performed? Further, while that cross-referencing may be a good indication of the precision of MatDetector's results, we do not have a good estimate of recall - the 6x number provides a relative performance, but not an absolute performance. I'd appreciate some explicit analysis of the performance of MatDetector in this paper — especially if, on analysis, that performance can be improved. If so, there is potentially more benefit to this method than the paper claims, and it could be an interesting avenue for future work.


**Evaluation of tokenization** The analysis of the word-initial tokens seems somewhat tautological: the concepts whose tokenization are being evaluated are gathered using MatDetector, the same method being used to boost the frequencies and cause the combination of those tokens. While this does confirm that the frequency boosting is resulting in the desired effect, I do not find this convincing as an evaluation of the tokenization quality. To fix this, I'd suggest using an external source of materials words to evaluate the word-initial token ratio - for example, the named entities from the materials NER dataset that is already used in this paper. Further, similar to my point above, while the subword embeddings similarities do establish that these embeddings relate in more materials terms being related to materials tokens, are these tokens in fact reasonably similar? Does the occurrence of _those_ particular materials make sense for that embedding? This additionally points at the potential for there to be more benefit here than the paper establishes.

In effect, I think this paper answers the question of "does domain-specific tokenization improve models' performance in the materials science domain" affirmatively; it opens the door to a lot of future work that asks the question of how to best tokenize to realize those gains, and would benefit from framing that argues that.
Comments, suggestions and typos: 1. In Section 4.5, the authors mention the "upper part" and "bottom part" of Figure 3 (line 371 and line 383), which should be the "left part" and "right part".

On my first read through, the vague assertion that the value of $\lambda$ was set based on "empirical analysis" (L284) raised a red flag for me; a pointer to where you do discuss this at L453 "Comparison of Lambda" would be helpful. 

Re: training conditions, they are specified in the appendix and with reference to prior work; I'd appreciate a quick summary of all the things that remain the same across training runs with different tokenizers, and what, if anything changed in the main text. 

Table 5 says it shows the five nearest neighbors, it only seems to show two?

Typo: figure 4 indicates number of tokens, without scale. Am I misunderstanding that table, or should it be e.g. in thousands of tokens? Additionally, toknes -> tokens in the caption.

Typo/unclear: L452 and L464 say that the lambda value that performs best is $10^1$, while the chart axis is not labeled as powers of 10, and earlier in the paper, the best lambda value is supposed to be 1, i.e. $10^0$. Which one is it?? Limitations also says the best lambda is 1, not 10. What is the best value for lambda??
Overall assessment scores: 3

4
Metareview score: 4
Metareview: The paper proposes MATTER, a tokenization framework for materials science texts that incorporates domain knowledge in tokenization. Traditional frequency-based tokenizers usually split materials-specific terms into many small fragments and lose the semantic integrity of shared subword tokens. MATTER reranks subword tokens so that key materials concepts are preserved. The authors demonstrate that their approach yields modest but consistent performance improvements (around 2-3% gains) on tasks in the MatSci-NLP benchmark when used to train a MatSciBERT.
Summary of reasons to publish: - Demonstrates the value of incorporating domain knowledge into tokenization (the first step of the LM pretraining pipeline)
- Experiments demonstrate consistent improvements across tasks over standard subword tokenizers