Question: how xgboost works in parallel?
Answer: 1 Answer. Xgboost doesn't run multiple trees in parallel like you noted, you need predictions after each tree to update gradients. Rather it does the parallelization WITHIN a single tree my using openMP to create branches independently. To observe this,build a giant dataset and run with n_rounds=1.