# Computational Screening of New Perovskite Materials Using Transfer Learning and Deep Learning
## 2 Materials and Methods
### GBR Machine Learning Model for Formation Energy Prediction  
It has been shown [33] that training a perovskite specific formation energy prediction model using structural information can achieve high accuracy comparable to that of a DFT calculation. Instead of using artificial neural networks and elemental features only, as done before, we propose to use GBR with both structural and elemental features. Gradient Boosting Regressor  
Boosting is a family of algorithms that can promote weak learners to strong learners, and its performance is significantly better than other basic classifiers. Adding new models to the collection in turn is the main idea for improvement. In each particular iteration, a new, basic learner model is trained by the entire integration error that has been learned so far. Its gradient promotion, like other lifting methods, builds the model in stages and uses arbitrary loss functions. It uses the gradient descent method to solve the minimization problem and generates a predictive model in the form of a  
Figure 2: Framework for the computational screening of perovskite materials. Abbreviations: RFR, random forest regression; SVR, support vector regression; GBR, gradient boosting regressor; Crystallographic Information File (CIF). set of weak predictive models (usually decision trees). Boosting can be used for both regression and classification problems while this study uses it for regression. The main architecture of GBR includes three elements: the loss function, the weak learner (for prediction), and the addition model. The main idea of this algorithm is to construct a new basic learner, which is most correlated with the negative gradient of the loss function and is related to the whole. Any loss function can be used. In general, the choice of the loss function depends on the user of the algorithm. So far, there have been various loss functions [34]. The mathematical formula for GBR is as follows:  
\[F(x)=\sum_{m=1}^{M}\gamma_{m}l_{m}(x), \tag{1}\]  
where \(h_{m}(x)\) is a basic function and is often referred to as a weak learner. GBR uses a fixed-size decision tree as a weak learner. Decision trees have the ability to process mixed-type data and the ability to model complex functions. Like other enhancement algorithms, GBR builds the addition model in a step-by-step manner, with the following formula:  
\[F_{m}(x)=F_{m-1}(x)+\gamma_{m}h_{m}(x), \tag{2}\]  
At each stage, the decision tree \(h_{m}(x)\) selects the minimized loss function \(L\), giving the current model \(F_{m-1}\) and \(F_{m-1}(x_{i})\), as follows:  
\[F_{m}(x)=F_{m-1}(x)+\operatorname*{argmin}_{h}\sum_{i=1}^{n}L(y_{i},F_{m-1}(x _{i})-h_{m}(x)), \tag{3}\]  
The initial model \(F_{0}\) is determined for a particular problem, and for least square regression, the average of the target values is typically chosen. Given any divisible loss function \(L\), the algorithm starts with the initial model. GBR solves this minimization problem by gradient descent numerical methods. The gradient descent direction is the negative gradient of the loss function on the current model \(F_{m-1}\) and can be calculated for any divisible loss function. The formula is:  
\[F_{m}(x)=F_{m-1}(x)+\gamma_{m}\sum_{i=1}^{n}\
abla_{F}L(y_{i},F_{m-1}(x_{i})), \tag{4}\]  
In the gradient-lifting regression tree, there are multiple hyperparameters. The values of the hyperparameters used in this study are:  
max_depth = 6; n_estimators = 500; min_samples_split = 0.5; subsample = 0.7; alpha = 0.1; learning_rate = 0.01; loss = ls  
To evaluate the performance of GBR, we also evaluated several mainstream machine learning algorithms as the baselines, including random forest regression (RFR), support vector regression (SVR), and least absolute shrinkage and selection operator (Lasso) with the same dataset using the same set of features. Random forest regression is also a commonly used algorithm in boosting. It is composed of many decision trees trained on each subsample of the dataset and uses their averages to improve prediction accuracy and control overfitting.