XGBoost
{\displaystyle {\hat {g}}_{m}(x_{i})=\left[{\frac {\partial L(y_{i},f(x_{i}))}{\partial f(x_{i})}}\right]_{f(x)={\hat {f}}_{(m-1)}(x)}.} h ^ m ( x i ) = [ ∂ 2 L ( y i , f ( x i ) ) ∂ f ( x i ) 2 ] f ( x ) = f ^ ( m − 1 ) ( x ) . {\displaystyle {\hat {h}}_{m}(x_{i})=\left[{\frac {\partial ^{2}L(y_{i},f(x_{i}))}{\partial f(x_{i})^{2}}}\right]_{f(x)={\hat {f}}_{(m-1)}(x)}.} Fit a base learner (or weak learner, e.g. tree) using the training set { x i , − g ^ m ( x i ) h ^ m ( x i ) } i = 1 N {\displaystyle \displaystyle \{x_{i},-{\frac {{\hat {g}}_{m}(x_{i})}{{\hat {h}}_{m}(x_{i})}}\}_{i=1}^{N}} by solving the optimization problem below: φ ^ m = arg  min φ ∈ Φ ∑ i = 1 N 1 2 h ^ m ( x i ) [ − g ^ m ( x i ) h ^ m ( x i ) − φ ( x i ) ] 2 .