Move argsort and some other stuff which don't need to do for each tree to forest class.