# A framework to predict binary liquidus by combining machine learning and CALPHAD assessments

Guillaume Deffrennes, Kei Terayama, Taichi Abe, Etsuko Ogamino, Ryo Tamura

► To cite this version:

Guillaume Deffrennes, Kei Terayama, Taichi Abe, Etsuko Ogamino, Ryo Tamura. A framework to predict binary liquidus by combining machine learning and CALPHAD assessments. Materials & Design, 2023, 232, pp.112111. 10.1016/j.matdes.2023.112111. hal-04187514

HAL Id: hal-04187514

https://hal.science/hal-04187514v1

Submitted on 24 Aug 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

# A framework to predict binary liquidus by combining machine learning and CALPHAD assessments

<figure><img src="image_3.png" /></figure>

Check for
updates

Guillaume Deffrennes<sup>a,b,*</sup>, Kei Terayama<sup>c,d</sup>, Taichi Abe<sup>e</sup>, Etsuko Ogamino<sup>b</sup>, Ryo Tamura<sup>b,f,g,*</sup>

## HIGHLIGHTS

* A framework to predict binary liquidus from the properties of the pure elements and estimates from Miedema's model is proposed.

* The liquidus prediction is broken down into three simpler prediction tasks to improve both accuracy and interpretability.

* The equilibrium onset temperature of solidification is predicted with a mean absolute error of 102 K.

* A large dataset comprising data on 2016 binary liquidus between 64 elements is shared in open access.

* CALPHAD assessments enable the creation of large machine learning datasets for predicting phase equilibria.

## ARTICLE INFO

**Article history:**
Received 13 April 2023
Revised 11 June 2023
Accepted 20 June 2023
Available online 23 June 2023

**Keywords:**
Liquidus
Machine learning
Miscibility gaps
Phase equilibria

## GRAPHICAL ABSTRACT

<figure><img src="image_4.png" /></figure>

## ABSTRACT

Knowledge of the liquidus is important for the design and processing of many materials. For instance, deep eutectics are important for the design of metallic glasses, and recently multi-principal element alloys have been designed based on eutectic compositions or melting temperatures extrapolated from binary liquidus data. In this study, we provide a general framework for predicting binary liquidus only from the properties of the pure elements and thermodynamic properties calculated by Miedema's model. Our framework combines three machine learning models that are trained and evaluated on liquidus data collected from 466 CALPHAD assessments of binary phase diagrams. The first model predicts the formation of liquid miscibility gaps with a prediction accuracy of 95.3%, outperforming the empirical Mott model. The second and third models predict the equilibrium onset temperature of solidification and the critical temperature of liquid miscibility gaps, respectively. An important feature of our models is that

<footnote><sup>a</sup> Univ. Grenoble Alpes, CNRS, Grenoble INP, SIMaP, F-38000 Grenoble, France</footnote>

<footnote><sup>b</sup> International Center for Materials Nanoarchitectonics (WPI-MANA), National Institute for Materials Science, 1-1 Namiki, Tsukuba, Ibaraki 305-0044, Japan</footnote>

<footnote><sup>c</sup> Graduate School of Medical Life Science, Yokohama City University, 1-7-29, Suehiro-cho, Tsurumi-ku, Kanagawa 230-0045, Japan</footnote>

<footnote><sup>d</sup> MDX Research Center for Element Strategy, Tokyo Institute of Technology, 4259 Nagatsuta-cho, Midori-ku, Yokohama, Kanagawa 226-8501, Japan</footnote>

<footnote><sup>e</sup> Research Center for Structural Materials, National Institute for Materials Science, 1-2-1 Sengen, Tsukuba, Ibaraki 305-0047, Japan</footnote>

<footnote><sup>f</sup> Research and Services Division of Materials Data and Integrated System, National Institute for Materials Science, 1-1 Namiki, Tsukuba, Ibaraki 305-0044, Japan</footnote>

<footnote><sup>g</sup> Graduate School of Frontier Sciences, The University of Tokyo, 5-1-5 Kashiwa-no-ha, Kashiwa, Chiba 277-8561, Japan</footnote>

<footnote>* Corresponding authors.
E-mail addresses: Guillaume.DEFFRENNES@cnrs.fr (G. Deffrennes), TAMURA.Ryo@nims.go.jp (R. Tamura).</footnote>
# 1. Introduction

Phase diagrams assessments play an important role in the design of a multitude of materials [1], from alloys [2] to semiconductors [3] or composites [4]. Yet, phase diagram determination is time consuming, and the number of materials systems that remain to be explored is immense. In recent years, machine learning (ML) has emerged as a valuable tool to identify compositions worth investigating by predicting specific features that are desirable for a given application [5-7]. For instance, one can refer to the prediction of single- or two-phase domains to design multi-principal element alloys (MPEAs) [8], of metallic glasses [9], or of new quasicrystals [10].

Data collection is a crucial step of such data-driven studies. To obtain the necessary phase diagram data, two main sources have been used. The first is high-throughput DFT calculations. Here, the thermodynamic stability of compounds is evaluated from their enthalpy of formation calculated at 0 K, as in Ref. [11]. The great advantage of this approach is that large amounts of data can be generated in an efficient way. For instance, databases such as The Materials Project [12] contain data on more than 100,000 compounds, and it is possible to create tailor-made datasets of more than 10,000 entries [13]. A disadvantage of this approach is that the data can be of limited accuracy [14] and do not provide information on the thermal stability of the compounds. The second source is the experimental literature. High-quality data can be obtained from it, but it is very time consuming to collect and assess the data. Therefore, the datasets, often collected from non-digital compilations, are relatively small, comprising tens [15-17] to thousands [8,18] of entries on phase equilibria. This can be detrimental to the performance of ML models [19].

An interesting alternative to collecting phase diagram data is to rely on CALPHAD-based thermodynamic assessments. A major advantage of this strategy is that CALPHAD assessments integrate and combine both thermodynamic and phase equilibria data, obtained from both experimental measurements and first-principles calculations [20]. Consistency between data of different nature can be evaluated as they are all translated in terms of Gibbs energy. Moreover, compatibility between binary data and ternary data can be evaluated, as illustrated in Ref. [21]. It is therefore suggested that CALPHAD assessments can be used to create large and unique datasets for predicting phase equilibria and thermodynamic properties. However, to the best of our knowledge, there have been only few attempts to predict phase diagrams properties from datasets collected from CALPHAD assessments [22-25].

An important part of phase diagrams is the liquidus that is the lowest temperature above which a material is at equilibrium in a homogeneous liquid phase. The liquidus provides information on the thermal stability of solids. As such, knowledge of the liquidus is important for materials processing, such as casting [26], additive manufacturing [27] or biomass combustion [28], and for the design of all kinds of materials, such as thermoelectrics [29] or superalloys [30], notably as there is a correlation between the melting point of precipitates and creep resistance [31,32]. Of particular importance are local minima in the liquidus, eutectics. Eutectic casting is an essential concept in metallurgy [33], and a promising route to the design of MPEAs [34,35]. Besides, deep eutectics are important

for the design of metallic glasses [9,36]. To further illustrate the wide range of applications, eutectic alloys are promising candidates as phase change materials for thermal energy storage [37], and binary alloys are being investigated for this application [38,39]. Information on binary phase diagrams have been used to predict phase equilibria in higher-order systems using machine learning [25,40], and notably, strategies to design MPEAs based on eutectic compositions [41,42] or melting temperatures [43] extrapolated from binary liquidus data have been proposed. Therefore, a model to predict binary liquidus can be useful for numerous applications.

In oxide systems, empirical [44] and ML [45] models have been developed to reproduce the liquidus. However, these interpolation models were not designed to be predictive outside the system of interest. Several ML studies have been conducted to predict the congruent melting point of compounds corresponding to a local maximum in the liquidus. Most of these studies focus on ionic liquids, salts melting below 373 K and composed of organic and inorganic ions, as in Ref. [46]. Studies on the prediction of the melting point of equimolar binary compounds have been carried out on the basis of limited datasets containing less than 50 entries [16,17]. An ML model was developed based on a dataset of 248 melting points of pure elements and binary compounds [18]. Despite the limited size of the dataset, performance was improved after including physical properties calculated by DFT as features. An ensemble model of 30 graph neural network models was trained on a much larger dataset of 10,000 melting points of compounds [47]. A neural network was developed to predict binary liquidus from a dataset collected from CALPHAD assessments comprising 287 binaries and 57 elements [23]. However, this ML model failed to reproduce miscibility gaps. This limitation can be explained as follows. A maximum in the liquidus can correspond to two very different cases. The first is the presence of a compound that melts congruently, which suggests some affinity between the elements, while the second is the presence of a liquid miscibility gap that comes from repulsive interactions between the elements.

Despite the importance of the liquidus for materials design, the prediction of liquidus temperatures in a vast compositional space has only been attempted once in 2020 [23]. That is because it is challenging to collect data of sufficient quantity and quality to build models upon. In this study, we propose a framework to predict binary liquidus including miscibility gaps (Fig. 1). From data collected from CALPHAD assessments on 466 binary systems and 64 elements, our framework is tested, and 1563 binary liquidus not included in our dataset are predicted. Our framework consists of three ML prediction models. The first, referred to as Model 1, is a classification model to predict the formation of liquid miscibility gaps. The second, Model 2, predicts the equilibrium onset temperature of solidification, noted T<sub>S+L</sub>, disregarding the fact the liquid may not be homogeneous. This information is obtained from Model 1 and from our third model, Model 3. Model 3 predicts the critical temperature above which the elements are completely miscible in the liquid, noted T<sub>L+L</sub>, whether the miscibility gap is metastable (T<sub>L+L</sub> < T<sub>S+L</sub> = T<sub>liquidus</sub>) or stable (T<sub>S+L</sub> < T<sub>L+L</sub> = T<sub>liquidus</sub>). The liquidus is thus obtained as the maximum between Model 2 and Model 3. However, Model 3 can only be trained on data from systems where the elements show some degree of immiscibility
<figure><img src="image_5.png" /><figcaption>**Fig. 1.** Framework adopted in this study to predict binary liquidus. (a) From a compilation of CALPHAD assessments, liquidus data are collected on 466 binary systems. (b) Binary liquidus between 64 elements are then predicted from 3 ML models. "w/o" stands for without.</figcaption></figure>

(T<sub>L+L</sub> greater than 300 K). Therefore, in systems where Model 1 predicts that the formation of a miscibility gap is very unlikely, Model 3 is not expected to be valid, and the liquidus is directly obtained from Model 2.

## 2. Methods

### 2.1. Dataset

Training datasets are collected from 466 CALPHAD assessments of binary phase diagrams obtained from the Computational Phase Diagram Database (CPDDB) [48] using the Pandat software [49]. A reference list of the assessments can be found in the data repository associated with this article [50]. First, in these binaries, a stable liquid miscibility gap is found in 44 systems, and this information is used to develop Model 1. Second, for each binary, the liquidus is calculated over the entire compositional range in steps of 0.5 at.%. The data on the 13 systems in which there is a very steep miscibility gap, stable above 4000 K, are removed. The liquidus temperature in the remaining 453 binaries are used to develop Model 2 after removing the data corresponding to miscibility gaps. Last, liquid miscibility gap temperatures from 33 binaries in which the miscibility gap is eventually stable and 171 binaries in which it is metastable are used to develop Model 3. All data were collected disregarding the presence of the gas phase. Additional information on composition of each dataset is given in Supplementary Note A.

### 2.2. Descriptors

Up to 58 descriptors are obtained from the properties of the pure elements weighted by the composition. The melting properties of the pure elements are calculated from the SGTE unary database [51]. In this way, feature extraction is consistent with data collection, as this database is the backbone of CALPHAD assessments. It is pointed out that the melting point of C is set at 4765 K, which is an upper limit [52], and that for P, data on the red allotrope which melts at 852 K are adopted. The other properties are calculated using matminer [53] from the data accumulated in the magpie repository [54].

Up to 4 descriptors are obtained from Miedema's semi-empirical model [55]. The element-dependent parameters of the model are taken from the matminer repository [53]. To compute the variables of the model, we follow the procedure of Ref. [56].

### 2.3. ML methods

The following algorithms are used: k-nearest neighbor (KNN) and random forest (RF) as implemented in scikit-learn [57], lightGBM [58], and artificial neural network (ANN) with two hidden layers as implemented in tensorflow [59].

All models are optimized and tested from a nested cross-validation (CV) approach schematized in Fig. S4. Data on each binary system form a group and are never split between the training, validation, or test sets. A group 10-fold CV is performed in the inner loop to determine the best set of hyperparameters from grid search, except for lightGBM for which the Optuna package [60] is used. A group 20-fold CV is performed in the outer loop to evaluate model performance. More details are in Supplementary Note B.

## 3. Results and discussion

### 3.1. Prediction of the formation of liquid miscibility gaps

To train Model 1, five descriptors are used. Two descriptors are obtained from the absolute difference of the enthalpy of fusion of the pure elements and of their entropy of fusion. The remaining three descriptors obtained from Miedema's model [55,61] are (i) the enthalpy of mixing in the liquid phase estimated at equiatomic composition, and the square of the difference (ii) of the so-called work function, denoted $\phi$, and (iii) of the cube root of the average electron density at the boundary of the Wigner-Seitz cell, denoted $n_{WS}^{1/3}$. It has been shown that the latter two descriptors allow to distinguish between binary systems in which compounds exist from others [55,62].

In the dataset, the classes are imbalanced as liquid miscibility gaps are stable in only 44 of the 466 included binaries. This can cause the classifier to be biased toward the majority class, i.e., to underestimate the stability of miscibility gaps. This imbalance can be addressed by over-sampling the minority class during training. For this purpose, two over-sampling methods implemented in Imbalanced-learn [63] are used: SMOTE [64] and BorderlineSMOTE [65]. These algorithms generate synthetic samples in the minority class until the desired $N_{rmin}/N_{maj}$ ratio is reached, with the number of samples in the minority class after resampling, $N_{rmin}$, and the number of samples in the majority class, $N_{maj}$. An optimal value
<caption>**Table 1**</caption>

Results obtained when predicting the stability of liquid miscibility gaps using the RF algorithm and different over-sampling strategies. The confusion matrix is presented in the four rightmost columns. It shows how many observations were correctly (true positive, TP and true negative, TN) and incorrectly classified (false positive, FP and false negative, FN), with the positive class being "there is a stable miscibility gap in the system".

<table><caption>**Table 2**</caption><thead><tr><th>Over-sampling</th><th>N<sub>rmin</sub>/N<sub>maj</sub></th><th>Accuracy</th><th>Macro F1 score</th><th>TP</th><th>FN</th><th>FP</th><th>TN</th></tr></thead><tbody><tr><td>None</td><td>0.104</td><td>0.953</td><td>0.846</td><td>28</td><td>16</td><td>6</td><td>416</td></tr><tr><td>SMOTE</td><td>0.20</td><td>0.951</td><td>0.851</td><td>31</td><td>13</td><td>10</td><td>412</td></tr><tr><td>BorderlineSMOTE</td><td>0.20</td><td>0.953</td><td>0.856</td><td>31</td><td>13</td><td>9</td><td>413</td></tr></tbody></table>

Descriptors for Model 2 and Model 3.

<table><thead><tr><th>Descriptor</th><th>Characteristics</th></tr></thead><tbody><tr><td>Melting point</td><td>Mean and average absolute deviation of the melting property among elements in composition</td></tr><tr><td>Enthalpy of fusion</td><td>(20 dimensional)</td></tr><tr><td>Enthalpy of the liquid at the melting point</td><td></td></tr><tr><td>Enthalpy of the solid at the melting point</td><td></td></tr><tr><td>Entropy of fusion</td><td></td></tr><tr><td>Entropy of the liquid at the melting point</td><td></td></tr><tr><td>Entropy of the solid at the melting point</td><td></td></tr><tr><td>Heat capacity of fusion</td><td></td></tr><tr><td>Heat capacity of the liquid at the melting point</td><td></td></tr><tr><td>Heat capacity of the solid at the melting point</td><td></td></tr><tr><td>Atomic number</td><td>Mean and average absolute deviation of the property among elements in composition</td></tr><tr><td>Mendeleev number</td><td>(38 dimensional)</td></tr><tr><td>Periodic table row</td><td></td></tr><tr><td>Periodic table column</td><td></td></tr><tr><td>Atomic weight</td><td></td></tr><tr><td>Covalent radius</td><td></td></tr><tr><td>Electronegativity</td><td></td></tr><tr><td>Volume (at 0 K from DFT)</td><td></td></tr><tr><td>Number of valence electrons</td><td></td></tr><tr><td>Filled s orbitals</td><td></td></tr><tr><td>Filled p orbitals</td><td></td></tr><tr><td>Filled d orbitals</td><td></td></tr><tr><td>Filled f orbitals</td><td></td></tr><tr><td>Unfilled valence orbitals</td><td></td></tr><tr><td>Unfilled s orbitals</td><td></td></tr><tr><td>Unfilled p orbitals</td><td></td></tr><tr><td>Unfilled d orbitals</td><td></td></tr><tr><td>Unfilled f orbitals</td><td></td></tr><tr><td>Space group number</td><td></td></tr><tr><td>Enthalpy of mixing in the binary liquid phase</td><td>Estimates from Miedema's model</td></tr><tr><td>Enthalpy of formation of an ordered binary intermetallic compound</td><td>(2 dimensional)</td></tr></tbody></table>

for this ratio was obtained from a grid search. The results obtained using the RF algorithm and different over-sampling strategies are summarized in Table 1. While over-sampling does not improve the accuracy of the model, the best results are obtained when using the BorderlineSMOTE algorithm [65]. It is pointed out that similar results are obtained with the lightGBM algorithm (Supplementary Note C).

In a previous study [66], an empirical model called Mott model was proposed to predict liquid immiscibility in binaries. It was applied to the present dataset to allow for a direct comparison with our ML model. For each binary, the Mott number was calculated from the data in the magpie repository [54], and compared with "Mott criterion 1" from Ref. [66]. An accuracy of 74.2% was obtained, which is significantly lower than the accuracy of our ML model. Furthermore, an advantage of RF is that the probability of belonging to each class can be evaluated. This is more informative than simply knowing whether a miscibility gap is stable or not. For instance, our model predicts a probability of having a miscibility gap of 0.31 in the La-Zr system where a miscibility gap is almost stable [67], and of 0.73 in the Cr-Sn system where a miscibility gap is stable only over a limited range of composition and temperature [68] (Supplementary Note E).

## 3.2. Prediction of the equilibrium onset temperature of solidification

To train Model 2, the 60 descriptors listed in Table 2 are used. The target variable of Model 2 is the difference between the liquidus temperature and the weighted average of the melting points of the elements, as in Ref. [23].

Performance metrics for different ML algorithms are presented in Table 3. The best results are obtained using the lightGBM and RF algorithms, and the RF algorithm is chosen to build Model 2. The mean absolute percentage error (MAPE) is shown per element in Fig. 2. This is a measure of the error relative to the temperature.

<caption>**Table 3**</caption>

Performance metrics obtained over all the 453 binaries included in the dataset for different algorithms when predicting the equilibrium onset temperature of solidification. R² is the coefficient of determination.

<table><thead><tr><th>Algorithm</th><th>MAE (in K)</th><th>R<sup>2</sup></th></tr></thead><tbody><tr><td>lightGBM</td><td>102.4</td><td>0.945</td></tr><tr><td>RF</td><td>102.4</td><td>0.944</td></tr><tr><td>ANN</td><td>118.3</td><td>0.931</td></tr><tr><td>KNN</td><td>114.4</td><td>0.931</td></tr></tbody></table>
<figure><img src="image_6.png" /><figcaption>Fig. 2. The MAPE of Model 2 per element. For the elements that are underrepresented in the dataset, such as As, Ba, Cs, Rh and Ti for which only one binary is included (Fig. S4), the calculated MAPE may not be representative.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 3. Liquidus predicted using Model 2 for (a) the Au-Tb system and (b) the Cu-Y system. The liquidus calculated from CALPHAD assessments [73,74] that have not been used during training and experimental data [74–79] are plotted for comparison.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 4. Scatter plot of the liquidus temperatures predicted by Model 2 against the ones calculated from CALPHAD assessments across the whole dataset comprising 453 binaries. The diagonal line illustrates perfect agreement. Outliers obtained in the Al-Ga and B-Ga systems are circled and discussed in the text.</figcaption></figure>

For a given element A, it is calculated over all binaries in the dataset between A and any other element B as follows:

$$ \text{MAPE}(\text{el A}) = 100 \sum_{\text{(el B/A-B\in\text{Dataset})}} \frac{1}{199} \sum_{0.5 \le x_B \le 99.5} \times \frac{|T_{\text{CALPHAD}}(x_B) - T_{\text{ML}}(x_B)|}{T_{\text{CALPHAD}}(x_B)} \quad (1) $$

with $T_{\text{CALPHAD}}$ the liquidus temperature calculated from CALPHAD assessments, $T_{\text{ML}}$ the liquidus temperature predicted from Model 2. The predictions are especially accurate for rare earths, with an MAE of 72 K calculated over all the 158 binaries in the dataset between a lanthanide and any other element, which can be as low as 30 K for Ho and Er (Fig. S6). The predicted Au-Tb liquidus, which has an MAE of 35 K, is shown in Fig. 3(a). This good agreement is explained by the fact that there are particularly strong
trends in phase diagrams throughout the lanthanide series [69,70]. However, the predictions are less reliable for group (1) and group 15 elements. This is because these electropositive and electronegative elements tend to form complex liquids with short-range ordering [71,72], and they are underrepresented in the dataset (Fig. S2). An important feature of Model 2 is that can give indications of the presence of congruent melting phases and eutectics that correspond to maxima and minima of the liquidus. This is true even when the liquidus temperature is not closely predicted, as shown in Fig. 3(b) with the Cu-Y case.

Fig. 4 shows a scatter plot between the liquidus temperatures predicted by Model 2 and calculated from CALPHAD assessments across the whole dataset. The absolute error is less than 100 K in 63% of the cases, and less than 200 K in 85% of the cases. The model is abnormally inaccurate in the B-Ga and Al-Ga systems, for which mean absolute errors (MAE) of 724 K and 645 K are obtained, respectively. In the B-Ga system, no compounds have been synthesized and the existence of a large liquid miscibility gap has been assumed [80]. The B-Ga liquidus calculated from a CALPHAD assessment [81] based on this limited information only is therefore unreliable. In Ref. [81], the liquidus temperature was modeled to increase from 303 K for pure Ga to 2285 K at 4.5 at.%B, whereas Model 2 predict a much lower liquidus temperature of 648 K at this composition. Thus, the liquidus temperature in the B-Ga system appears to be severely overestimated in [81]. This is further supported by the fact that a critical miscibility gap temperature of 3796 K is calculated from Ref. [81], whereas a temperature of 1966 K is predicted from Model 3. This illustrates how our ML models can provide CALPHAD assessors with reasonable liquidus estimates where no experimental data exist. On the other hand, in the Al-Ga system, the liquidus calculated from Ref. [82] is reliable, as it is supported by several consistent experimental datasets. Model 2 greatly overestimates the liquidus temperature in this system. This is explained by the fact that our model was trained to reproduce the overestimated B-Ga liquidus and assumes similarities between two binaries of group 13 neighbors. If data on B-Ga are removed from the training set, the MAE for Al-Ga decreases from 724 K to 330 K, a value still high, but more consistent with the expected accuracy for group 13 elements (Fig. 2). This illustrates how unreliable data in the training set can mislead the model.

If a single RF model was trained to predict the liquidus temperature, miscibility gaps included, the MAE would increase from 102 K to 132 K. This illustrates that our framework based on three ML models (Fig. 1) improves prediction accuracy. Besides, a RF model was trained using the 7 descriptors used in Ref. [23] instead of the 60 proposed in Table 1. A higher MAE of 110 K was obtained.

So far, the prediction performance of Model 2 has been evaluated in the case where the binary liquidus to be predicted is completely unknown. If some liquidus data are available in the system of interest, they can be included in the training set, which will improve the accuracy of the model. To demonstrate this, a RF model was trained after the data points at equiatomic composition were systematically removed from the test set and added to the training set over all folds. A significantly improved MAE of 87 K was obtained. When the data points at 25 and 75 at.% are also added to the training set, the MAE is further improved to 71 K.

### 3.3. Prediction of the liquid miscibility gap temperature

To train Model 3, the 60 descriptors listed in Table 3 are used. When only the data on the 33 binaries in which a liquid miscibility gap is eventually stable are used, performance metrics obtained for different algorithms are summarized in Table 4, and a scatter plot between the CALPHAD and the predicted miscibility gap temperatures is shown in Fig. 5(a). The predictions are rather inaccurate,

with an MAE of 460 K for the RF model. A possible explanation is that the dataset is too small, since it contains 30 elements but only 33 binaries. To investigate this hypothesis, a RF model is built from a larger dataset, which now also includes the data from the 171 entirely metastable miscibility gaps. A lower overall MAE of 356 K is obtained for this new model. However, for the 33 binaries with a stable miscibility gap, a higher MAE of 580 K is now obtained. It appears from Fig. 5(b) that higher miscibility gap temperatures are systematically predicted to be lower than those calculated from CALPHAD assessments. It is concluded that the main

<caption>**Table 4**</caption>

Performance metrics obtained for different ML algorithms when predicting liquid miscibility gap temperatures from data on 33 binaries in which a miscibility gap is eventually stable.

<table><thead><tr><td>Algorithm</td><td>MAE (in K)</td><td>R<sup>2</sup></td></tr></thead><tbody><tr><td>RF</td><td>459.6</td><td>0.541</td></tr><tr><td>lightGBM</td><td>471.7</td><td>0.52</td></tr><tr><td>ANN</td><td>532</td><td>0.454</td></tr><tr><td>KNN</td><td>523.7</td><td>0.337</td></tr></tbody></table>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /><figcaption>**Fig. 5.** Scatter plot of the miscibility gap temperatures predicted using the RF algorithm against the ones calculated from CALPHAD assessments. The dataset comprises (a) data on 33 binaries in which a miscibility gap is eventually stable and (b) from data on 204 binaries in which the miscibility gap is either stable or metastable.</figcaption></figure>
limitation of miscibility gap temperature prediction is not the quantity of data, but the quality of data. Experimental data on liquid miscibility gaps are often lacking, especially at high temperatures. As a result, the calculated miscibility gap temperatures in our dataset are often unreliable. For instance, in the Mg-Mn system, a monotectic reaction has been measured at 1471 K [83], but no solubility data are available. Two CALPHAD assessments [83,84] have been proposed based on this limited information, and a difference of 1300 K is found between them for the critical temperature at which the miscibility gap closes. Although it has not been possible at this stage to predict miscibility gap temperatures accurately, the R² value of Model 3 is greater than 0.5, which suggests it can reproduce miscibility gap trends in unknown binaries.

### 3.4. Prediction of the liquidus in 1563 binary systems

By combining our three ML models, we predict the liquidus in 1563 binary systems not included in our dataset, many of which are unknown. The prediction results can be found in an open-access repository [50]. As discussed in the last paragraph of the introduction, Model 2 is not valid in systems where the elements are always miscible in the stable or metastable liquid phase. Therefore, Model 3 was only used in systems where the probability of having a stable liquid miscibility gap is predicted by Model 1 to be greater or equal to 0.25. The prediction of two unknown liquidus is shown in Fig. 6 as an example. The Cd-Ce binary is a sys-

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /><figcaption>**Fig. 6.** Prediction of two unknown liquidus. (a) In the Cd-Ce system, Model 1 predicts that the probability of having a miscibility gap is zero, so the liquidus is directly obtained from Model 2. (b) In the Ca-Sm system, Model 1 predicts that the probability of having a miscibility gap is 0.47, and the liquidus is obtained as the maximum between Model 2 and Model 3.</figcaption></figure>

tem of interest for the reprocessing of nuclear fuels [85,86], and because icosahedral quasicrystals can be formed between Cd and rare earths [87]. Our predictions suggest the existence of one or several phases in the 15 to 60 at.%Ce range that melt congruently around 1100 K, and that there is a relatively deep eutectic at about 80 at.%Ce, which suggests it may be possible to form metallic glasses in this system following Ref. [9]. The Ca-Sm system is of interest for the development of new magnesium alloys [88]. Our prediction suggests the presence of a liquid miscibility gap with a critical temperature of about 2100 K.

### 4. Conclusion

In this study, we developed a general framework to predict binary liquidus. The liquidus prediction is broken down into three prediction tasks, which enables to differentiate between miscibility gaps and congruent melting phases while also improving the prediction accuracy.

Knowledge of the liquidus is essential for the processing and design of all kinds of materials. With our framework, reasonable estimates of the liquidus and indications of the presence of eutectics and congruent melting phases can be obtained in domains where no data are available. This can be valuable in the early stages of materials development. This is also valuable for developing CALPHAD databases, which play an important role in computational materials design.

This work also demonstrates how CALPHAD assessments can be used to create unique ML datasets for predicting phase equilibria. However, to create high-quality datasets, the assessments should only be used within their range of validity and if they are based on sufficient information. This was not checked for in the present study. As a result, while our datasets contain high-quality data validated by experiments, as in the Au-Tb, Cu-Y or Al-Ga cases, it also contains unreliable data, as in the B-Ga case. This was shown to impact the accuracy of the ML models. We are working on improving our datasets, both in terms of quality and quantity, to further increase the performance of our models.

Another perspective is to find better features for our models. In this work, the enthalpy of mixing predicted by Miedema's model [55] is found to be an important feature for liquidus prediction. Miedema's model is commonly used in machine learning studies focusing on phase equilibria, such as Refs. [8,89]. However, as its accuracy can be questioned [90], we are working on the prediction of enthalpies of mixing in binary solutions.

### CRediT authorship contribution statement

**Guillaume Deffrennes:** Conceptualization, Methodology, Formal analysis, Investigation, Data curation, Writing - original draft.

**Kei Terayama:** Methodology, Writing - review & editing.

**Taichi Abe:** Resources, Writing - review & editing.

**Etsuko Ogamino:** Investigation, Data curation. **Ryo Tamura:** Conceptualization, Methodology, Writing - original draft, Supervision, Funding acquisition.

### Data availability

The datasets analyzed in this study and predictions on 1563 binary liquidus can be found in an open repository [50].

### Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
## Acknowledgements

This study was supported by a project subsidized by Core Research for Evolutional Science and Technology (CREST) (Grant No. JPMJCR17J2) from the Japan Science and Technology Agency (JST). This work was supported by MEXT Program: Data Creation and Utilization Type Material Research and Development Project (Grant Number JPMXP1122683430).

## Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.matdes.2023.112111.

## References

[1] R. Schmid-Fetzer, Phase diagrams: the beginning of wisdom, J. Phase Equilib. Diffus. 35 (2014) 735–760, https://doi.org/10.1007/s11669-014-0343-5.

[2] E. Menou, I. Toda-Caraballo, P.E.J. Rivera-Díaz-del-Castillo, C. Pineau, E. Bertrand, G. Ramstein, F. Tancre, Evolutionary design of strong and stable high entropy alloys using multi-objective optimisation based on physical models, statistics and thermodynamics, Mater. Des. 143 (2018) 185–195, https://doi.org/10.1016/j.matdes.2018.01.045.

[3] X. Li, Z. Li, C. Chen, Z. Ren, C. Wang, X. Liu, Q. Zhang, S. Chen, CALPHAD as a powerful technique for design and fabrication of thermoelectric materials, J. Mater. Chem. A. 9 (2021) 6634–6649, https://doi.org/10.1039/D9TA12602A.

[4] J. Andrieux, B. Gardiola, O. Dezellus, Synthesis of Ti matrix composites reinforced with TiC particles: in situ synchrotron X-ray diffraction and modeling, J. Mater. Sci. 53 (2018) 9533–9544, https://doi.org/10.1007/s10853-018-2258-8.

[5] J.E. Saal, A.O. Oliynyk, B. Meredig, Machine learning in materials discovery: confirmed predictions and their underlying approaches, Annu. Rev. Mater. Res. 50 (2020) 49–69, https://doi.org/10.1146/annurev-matsci-090319-010954.

[6] G.L.W. Hart, T. Mueller, C. Toher, S. Curtarolo, Machine learning for alloys, Nat. Rev. Mater. 6 (2021) 730–755, https://doi.org/10.1038/s41578-021-00340-w.

[7] R. Arróyave, Phase stability through machine learning, J. Phase Equilib. Diffus. 43 (2022) 606–628, https://doi.org/10.1007/s11669-022-01009-9.

[8] K. Lee, M.V. Ayyasamy, P. Delsa, T.Q. Hartnett, P.V. Balachandran, Phase classification of multi-principal element alloys via interpretable machine learning, Npj Comput. Mater. 8 (2022) 25, https://doi.org/10.1038/s41524-022-00704-y.

[9] A. Dasgupta, S.R. Broderick, C. Mack, B.U. Kota, R. Subramanian, S. Setlur, V. Govindaraju, K. Rajan, Probabilistic assessment of glass forming ability rules for metallic glasses aided by automated analysis of phase diagrams, Sci. Rep. 9 (2019) 357, https://doi.org/10.1038/s41598-018-36224-3.

[10] C. Liu, E. Fujita, Y. Katsura, Y. Inada, A. Ishikawa, R. Tamura, K. Kimura, R. Yoshida, Machine learning to predict quasicrystals from chemical compositions, Adv. Mater. 33 (2021) 2102507, https://doi.org/10.1002/adma.202102507.

[11] G.G.C. Peterson, J. Brgoch, Materials discovery through machine learning formation energy, J. Phys. Energy. 3 (2021) 022002. https://doi.org/10.1088/2515-7655/abe425.

[12] A. Jain, S.P. Ong, G. Hautier, W. Chen, W.D. Richards, S. Dacek, S. Cholia, D. Gunter, D. Skinner, G. Ceder, K.A. Persson, Commentary: The Materials Project: A materials genome approach to accelerating materials innovation, APL Materials. 1 (2013) 011002. https://doi.org/10.1063/1.4812323.

[13] J.-C. Crivello, J.-M. Joubert, N. Sokolovska, Supervised deep learning prediction of the formation enthalpy of complex phases using a DFT database: the σ – phase as an example, Comput. Mater. Sci 201 (2022), https://doi.org/10.1016/j.commatsci.2021.110864 110864.

[14] M.K. Horton, S. Dwaraknath, K.A. Persson, Promises and perils of computational materials databases, Nat. Comput. Sci. 1 (2021) 3–5, https://doi.org/10.1038/s43588-020-00016-5.

[15] P. Liu, H. Huang, S. Antonov, C. Wen, D. Xue, H. Chen, L. Li, Q. Feng, T. Omori, Y. Su, Machine learning assisted design of γ'-strengthened Co-base superalloys with multi-performance optimization, Npj. Comput. Mater. 6 (2020) 62, https://doi.org/10.1038/s41524-020-0334-5.

[16] Y. Saad, D. Gao, T. Ngo, S. Bobbitt, J.R. Chelikowsky, W. Andreoni, Data mining for materials: computational experiments with A B compounds, Phys. Rev. B. 85 (2012), https://doi.org/10.1103/PhysRevB.85.104104 104104.

[17] G. Pilania, J.E. Gubernatis, T. Lookman, Structure classification and melting temperature prediction in octet AB solids via machine learning, Phys. Rev. B. 91 (2015), https://doi.org/10.1103/PhysRevB.91.214302 214302.

[18] A. Seko, T. Maekawa, K. Tsuda, I. Tanaka, Machine learning with systematic density-functional theory calculations: Application to melting temperatures of single- and binary-component solids, Phys. Rev. B. 89 (2014), https://doi.org/10.1103/PhysRevB.89.054303 054303.

[19] Y. Zhang, C. Ling, A strategy to apply machine learning to small datasets in materials science, Npj Comput. Mater. 4 (2018) 25, https://doi.org/10.1038/s41524-018-0081-z.

[20] Z.-K. Liu, First-principles calculations and CALPHAD modeling of thermodynamics, J. Phase Equilib. Diffus. 30 (2009) 517–534, https://doi.org/10.1007/s11669-009-9570-6.

[21] B. Lindahl, X.L. Liu, Z.-K. Liu, M. Selleby, A thermodynamic re-assessment of Al-V toward an assessment of the ternary Al–Ti–V system, Calphad 51 (2015) 75–88, https://doi.org/10.1016/j.calphad.2015.07.002.

[22] Q. Wu, Z. Wang, X. Hu, T. Zheng, Z. Yang, F. He, J. Li, J. Wang, Uncovering the eutectics design by machine learning in the Al–Co–Cr–Fe–Ni high entropy system, Acta Mater. 182 (2020) 278–286, https://doi.org/10.1016/j.actamat.2019.10.043.

[23] P.-W. Guan, V. Viswanathan, MeltNet: Predicting alloy melting temperature by machine learning, (2020), https://doi.org/10.48550/ARXIV.2010.14048.

[24] Y. Zeng, M. Man, K. Bai, Y.-W. Zhang, Revealing high-fidelity phase selection rules for high entropy alloys: a combined CALPHAD and machine learning study, Mater. Des. 202 (2021), https://doi.org/10.1016/j.matdes.2021.109532 109532.

[25] G. Deffrennes, K. Terayama, T. Abe, R. Tamura, A machine learning-based classification approach for phase diagram prediction, Mater. Des. 215 (2022), https://doi.org/10.1016/j.matdes.2022.110497 110497.

[26] M. Bernhard, P. Presoly, C. Bernhard, S. Hahn, S. Ilie, An assessment of analytical liquidus equations for Fe–C–Si–Mn–Al–P–alloyed steels using DSC/DTA techniques, Metall. Mater. Trans. B. 52 (2021) 2821–2830, https://doi.org/10.1007/s11663-021-02251-1.

[27] P.A. Rometsch, Y. Zhu, X. Wu, A. Huang, Review of high-strength aluminum alloys for additive manufacturing by laser powder bed fusion, Mater. Des. 219 (2022), https://doi.org/10.1016/j.matdes.2022.110779 110779.

[28] E. Atallah, F. Defoort, A. Pisch, C. Dupont, Thermodynamic equilibrium approach to predict the inorganic interactions of ash from biomass and their mixtures: a critical assessment, Fuel Process. Technol. 235 (2022), https://doi.org/10.1016/j.fuproc.2022.107369 107369.

[29] D. Huang, D. Xia, T. Ye, T. Fujita, New experimental studies on the phase relationship of the Bi–Pb–Te system, Mater. Des. 224 (2022), https://doi.org/10.1016/j.matdes.2022.111384 111384.

[30] N. Ren, J. Li, B. Wang, L. Zeng, M. Xia, J. Li, Design of variable withdrawal rate for superalloy single-crystal blade fabrication, Mater. Des. 198 (2021), https://doi.org/10.1016/j.matdes.2020.109347 109347.

[31] Y. Chen, C. Wang, J. Ruan, T. Omori, R. Kainuma, K. Ishida, X. Liu, High-strength Co–Al–V–base superalloys strengthened by γ'-Co3(Al, V) with high solvus temperature, Acta Mater. 170 (2019) 62–74, https://doi.org/10.1016/j.actamat.2019.03.013.

[32] N. Mo, Q. Tan, M. Bermingham, Y. Huang, H. Dieringa, N. Hort, M.-X. Zhang, Current development of creep-resistant magnesium cast alloys: A review, Mater. Des. 155 (2018) 422–442, https://doi.org/10.1016/j.matdes.2018.06.032.

[33] M.E. Glicksman, Principles of solidification, Springer, New York, New York, NY (2011), https://doi.org/10.1007/978-1-4419-7344-3.

[34] Y. Lu, Y. Dong, H. Jiang, Z. Wang, Z. Cao, S. Guo, T. Wang, T. Li, P.K. Liaw, Promising properties and future trend of eutectic high entropy alloys, Scr. Mater. 187 (2020) 202–209, https://doi.org/10.1016/j scriptamat.2020.06.022.

[35] H. Li, R. Yuan, H. Liang, W.Y. Wang, J. Li, J. Wang, Towards high entropy alloy with enhanced strength and ductility using domain knowledge constrained active learning, Mater. Des. 223 (2022), https://doi.org/10.1016/j.matdes.2022.111186 111186.

[36] J. Cheney, K. Vecchio, Prediction of glass-forming compositions using liquidus temperature calculations, Mater. Sci. Eng. A 471 (2007) 135–143, https://doi.org/10.1016/j.msea.2007.02.120.

[37] A.I. Fernández, C. Barreneche, M. Belusko, M. Segarra, F. Bruno, L.F. Cabeza, Considerations for the use of metal alloys as phase change materials for high temperature applications, Sol. Energy Mater. Sol. Cells 171 (2017) 275–281, https://doi.org/10.1016/j.solmat.2017.06.054.

[38] Y. El Karim, Y. Grosu, A. Faik, R. Lbbib, Investigation of magnesium-copper eutectic alloys with high thermal conductivity as a new PCM for latent heat thermal energy storage at intermediate-high temperature, J. Storage Mater. 26 (2019), https://doi.org/10.1016/j.jest.2019.100974 100974.

[39] N. Gokon, C.S. Jie, Y. Nakano, S. Okazaki, T. Kodama, T. Hatamachi, S. Bellan, Phase change material of copper-germanium alloy as solar latent heat storage at high temperatures, Front. Energy Res. 9 (2021), https://doi.org/10.3389/fenrg.2021.696213 696213.

[40] J. Qi, A.M. Cheung, S.J. Poon, High entropy alloys mined from binary phase diagrams, Sci Rep. 9 (2019) 15501, https://doi.org/10.1038/s41598-019-50015-4.

[41] H. Jiang, K. Han, X. Gao, Y. Lu, Z. Cao, M.C. Gao, J.A. Hawk, T. Li, A new strategy to design eutectic high-entropy alloys using simple mixture method, Mater. Des. 142 (2018) 101–105, https://doi.org/10.1016/j.matdes.2018.01.025.

[42] K. Chen, Z. Xiong, M. An, T. Xie, W. Zou, Y. Xue, X. Cheng, Machine learning correlated with phenomenological mode unlocks the vast compositional space of eutectics of multi-principal element alloys, Mater. Des. 219 (2022), https://doi.org/10.1016/j.matdes.2022.110795 110795.

[43] J.J. Bhattacharyya, S.B. Innan, M.A. Wischhusen, J. Qi, J. Poon, J.R. Scully, S.R. Agnew, Lightweight, low cost compositionally complex multiphase alloys with optimized strength, ductility and corrosion resistance: discovery, design and mechanistic understandings, Mater. Des. 228 (2023), https://doi.org/10.1016/j.matdes.2023.111831 111831.

[44] L. Wondraczek, L. Beunet, Liquidus prediction in multicomponent lithium aluminosilicate glasses, J American Ceramic Society. 91 (2008) 1309–1311, https://doi.org/10.1111/j.1551-2916.2007.02237.x.
[45] C. Dreyfus, G. Dreyfus, A machine learning approach to the estimation of the liquidus temperature of glass-forming oxide blends, J. Non Cryst. Solids 318 (2003) 63–78, https://doi.org/10.1016/S0022-3093(02)01859-8.

[46] I. Baskin, A. EPShtein, Y. Ein-Eli, Benchmarking machine learning methods for modeling physical properties of ionic liquids, J. Mol. Liq. 351 (2022), https://doi.org/10.1016/j.molliq.2022.118616 118616.

[47] Q.-J. Hong, S.V. Ushakov, A. van de Walle, A. Navrotsky, Melting temperature prediction using a graph neural network model: From ancient minerals to new materials, Proc. Natl. Acad. Sci. U.S.A. 119 (2022) e2209630119. https://doi.org/10.1073/pnas.2209630119.

[48] T. Abe, K. Hashimoto, Y. Goto, Y. Sawada, K. Hirose, CPDDB, (2007). https://doi.org/10.48505/NIMS.3060.

[49] W. Cao, S.-L. Chen, F. Zhang, K. Wu, Y. Yang, Y.A. Chang, R. Schmid-Fetzer, W.A. Oates, PANDAT software with PanEngine, PanOptimizer and PanPrecipitation for multi-component phase diagram calculation and materials property simulation, Calphad 33 (2009) 328–342, https://doi.org/10.1016/j.calphad.2008.08.004.

[50] G. Deffrennes, K. Terayama, T. Abe, E. Ogamino, R. Tamura, Datasets from “A framework to predict binary liquidus by combining machine learning and CALPHAD assessments, Purendere Data (2023), https://doi.org/10.17632/rj6gmnmnmt1.1.

[51] A.T. Dinsdale, SGTE data for pure elements, Calphad 15 (1991) 317–425, https://doi.org/10.1016/0364-5916(91)90030-N.

[52] S. Bigdeli, Q. Chen, M. Selleby, A new description of pure C in developing the third generation of calphad databases, J. Phase Equilib. Diffus. 39 (2018) 832–840, https://doi.org/10.1007/s11669-018-0679-3.

[53] L. Ward, A. Dunn, A. Faghaniainia, N.E.R. Zimmermann, S. Bajaj, Q. Wang, J. Montoya, J. Chen, K. Bystrom, M. Dylla, K. Chard, M. Asta, K.A. Persson, G.J. Snyder, I. Foster, A. Jain, Matminer: an open source toolkit for materials data mining, Comput. Mater. Sci 152 (2018) 60–69, https://doi.org/10.1016/j.commatsci.2018.05.018.

[54] L. Ward, A. Agrawal, A. Choudhary, C. Wolverton, A general-purpose machine learning framework for predicting properties of inorganic materials, Npj Comput. Mater. 2 (2016) 16028, https://doi.org/10.1038/npjcompumats.2016.28.

[55] A.R. Miedema, F.R. De Boer, R. Boom, Predicting heat effects in alloys, Physica B +C. 103 (1981) 67–81, https://doi.org/10.1016/0378-4363(81)91003-2.

[56] A. Takeuchi, A. Inoue, Mixing enthalpy of liquid phase calculated by miedema's scheme and approximated with sub-regular solution model for assessing forming ability of amorphous and glassy alloys, Intermetallics 18 (2010) 1779–1789, https://doi.org/10.1016/j.intermet.2010.06.003.

[57] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, É. Duchesnay, Scikit-learn: machine learning in python, J. Mach. Learn. Res. 12 (2011) 2825–2830.

[58] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, T.-Y. Liu, LightGBM: A Highly Efficient Gradient Boosting Decision Tree, in: I. Guyon, U.V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc., 2017.

[59] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D.G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, X. Zheng, TensorFlow: A system for large-scale machine learning, OSDI'16: Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation. (2016).

[60] T. Akiba, S. Sano, T. Yanase, T. Ohta, M. Koyama, Optuna: A Next-generation Hyperparameter Optimization Framework, in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM, Anchorage AK USA, 2019: pp. 2623–2631. https://doi.org/10.1145/3292500.3330701.

[61] H. Bakker, Enthalpies in alloys, Trans Tech Publications (1998), https://doi.org/10.4028/www.scientific.net/MSfo.1.

[62] R.F. Zhang, X.F. Kong, H.T. Wang, S.H. Zhang, D. Legut, S.H. Sheng, S. Srinivasan, K. Rajan, T.C. Germann, An informatics guided classification of miscible and immiscible binary alloy systems, Sci. Rep. 7 (2017) 9577, https://doi.org/10.1038/s41598-017-09704-1.

[63] G. Lemaître, F. Nogueira, C.K. Aridas, Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning, J. Mach. Learn. Res. 18 (2017) 1–5.

[64] N.V. Chawla, K.W. Bowyer, L.O. Hall, W.P. Kegelmeyer, SMOTE: synthetic minority over-sampling technique, Jair. 16 (2002) 321–357, https://doi.org/10.1613/jair.953.

[65] H. Han, W.-Y. Wang, B.-H. Mao, Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning, in: D.~.-S. Huang, X.~.-P. Zhang, G.~.-B. Huang (Eds.), Advances in Intelligent Computing, Springer, Berlin Heidelberg, Berlin, Heidelberg, 2005, pp. 878–887, https://doi.org/10.1007/11538059_91.

[66] B.W. Mott, Immiscibility in liquid metal systems, J Mater Sci. 3 (1968) 424–435, https://doi.org/10.1007/BF00550987.

[67] N. Mattern, Y. Yokoyama, A. Mizuno, J.H. Han, O. Fabrichnaya, M. Richter, S. Kohara, Experimental and thermodynamic assessment of the La-Ti and La-Zr systems, Calphad 52 (2016) 8–20, https://doi.org/10.1016/j.calphad.2015.10.015.

[68] R.J. Pérez, B. Sundman, Thermodynamic assessment of the CR-SN binary system, Calphad 25 (2001) 59–66, https://doi.org/10.1016/S0364-5916(01)00030-X.

[69] J. Johansson, A. Rosengren, Generalized phase diagram for the rare-earth elements: calculations and correlations of bulk properties, Phys. Rev. B. 11 (1975) 2836–2857, https://doi.org/10.1103/PhysRevB.11.2836.

[70] G. Cacciamani, S. De Negri, A. Saccone, R. Ferro, The Al–R–Mg (R=Gd, Dy, Ho) systems, Part II: Thermodynamic modelling of the binary and ternary systems, Intermetallics. 11 (2003) 1135–1151, https://doi.org/10.1016/S0966-9795(03)00151-1.

[71] A.D. Pelton, M. Blander, Thermodynamic analysis of ordered liquid solutions by a modified quasichemical approach—application to silicate slags, MTB. 17 (1986) 805–815, https://doi.org/10.1007/BF02657144.

[72] T.M. Besmann, J. Schorne-Pinto, Developing practical models of complex salts for molten salt reactors, Thermo. 1 (2021) 168–178, https://doi.org/10.3390/themo1020012.

[73] S.L. Wang, J.L. Li, L.L. Qiu, L.J. Zhou, Y.H. Li, C.Q. Ren, C.Y. Fu, Y.H. Lin, C.P. Wang, X.J. Liu, Thermodynamic assessments of the Au-Tb and Au-Lu systems, J. Phase Equilib. Diffus. 37 (2016) 319–326, https://doi.org/10.1007/s11669-016-0460-4.

[74] S.G. Fries, H.L. Lukas, R. Konetzki, R. Schmid-Fetzer, Experimental investigation and thermodynamic optimization of the Y-Cu binary system, JPE. 15 (1994) 606–614, https://doi.org/10.1007/BF02647621.

[75] G. Rider, McMaster's GoldRich rare-earth-gold solid solutions, Trans. Metall. Soc. AIME. 233 (1965) 1488–1496.

[76] O.D. McMaster's, K.A. Gschneidner, G. Bruzzone, A. Palenzona, Stoichiometry, crystal structures and some melting points of the lanthanide-gold alloys, Journal of the Less Common Metals. 25 (2) (1971) 135–160.

[77] A. Saccone, D. Macciò, S. Delfino, R. Ferro, The phase diagram of the terbium-gold alloy system, Intermetallics 8 (2000) 229–237, https://doi.org/10.1016/S0966-9795(99)00099-0.

[78] M. Massalski, Bennett, Binary Alloy Phase Diagrams, American Society for Metals, Baker, 1986.

[79] G. Qi, K. Itagaki, A. Yazawa, High Temperature Heat Content Measurements of Cu&ndash;ash;RE (RE=Y, La, Ce, Pr, Nd) Binary Systems, Mater. Trans., JIM. 30 (1989) 273–282, https://doi.org/10.2320/matertrans1989.30.273.

[80] F. Wald, R.W. Stormont, Investigations on the constitution of certain binary boron-metal systems, Journal of the Less Common Metals. 9 (1965) 423–433, https://doi.org/10.1016/0022-5088(65)90126-8.

[81] X. Li, K. Cheng, X. Yuan, D. Zhao, J. Xin, W. Wang, C. Zhang, Y. Du, Thermodynamic assessment of the Ga-X (X=B, Ca, Sr, Ba) systems supported by first-principles calculations, Calphad 43 (2013) 52–60, https://doi.org/10.1016/j.calphad.2013.09.002.

[82] A. Watson, Re-assessment of phase diagram and thermodynamic properties of the Al-Ga system, Calphad 16 (1992) 207–217, https://doi.org/10.1016/0364-5916(92)90009-M.

[83] J. Gröbner, D. Mirkovic, M. Ohno, R. Schmid-Fetzer, Experimental investigation and thermodynamic calculation of binary Mg-Mn phase equilibria, J Phs Eqil and Diff. 26 (2005) 234–239, https://doi.org/10.1007/s11669-005-0110-8.

[84] Y.-B. Kang, A.D. Pelton, P. Chartrand, P. Spencer, C.D. Fuerst, Critical evaluation and thermodynamic optimization of the binary systems in the Mg-Ce-Mn-Y system, J Phs Eqil and Diff. 28 (2007) 342–354, https://doi.org/10.1007/s11669-007-9095-9.

[85] J. Zhang, E.A. Lahti, W. Zhou, Thermodynamic properties of actinides and rare earth fission products in liquid cadmium, J Radioanal Nucl Chem. (2015), https://doi.org/10.1007/s10967-014-3827-1.

[86] B. Skolyszewska-Kühberger, T.L. Reichmann, R. Ganesan, H. Ipser, Thermodynamic study of the cerium-cadmium system, Calphad 44 (2014) 14–20, https://doi.org/10.1016/j.calphad.2013.07.005.

[87] A.I. Goldman, T. Kong, A. Kreyssig, A. Jesche, M. Ramazanoglu, K.W. Dennis, S.L. Bud'ko, P.C. Canfield, A family of binary magnetic icosahedral quasicrystals based on rare earths and cadmium, Nature Mater. 12 (8) (2013) 714–718.

[88] X. Liu, R. Xu, Achieving ultra-high hardness of Mg-Sm-Ca alloy with the unique nanostructure, Mater. Sci. Eng. A 825 (2021), https://doi.org/10.1016/j.msea.2021.141929 141929.

[89] L. Zhang, H. Chen, X. Tao, H. Cai, J. Liu, Y. Ouyang, Q. Peng, Y. Du, Machine learning reveals the importance of the formation enthalpy and atom-size difference in forming phases of high entropy alloys, Mater. Des. 193 (2020), https://doi.org/10.1016/j.matdes.2020.108835 108835.

[90] R. Feng, M. Gao, C. Lee, M. Mathes, T. Zuo, S. Chen, J. Hawk, Y. Zhang, P. Liaw, Design of light-weight high-entropy alloys, Entropy 18 (2016) 333, https://doi.org/10.3390/e18090333.