# MicroForest: Lightweight Bottleneck Prediction for Manufacturing Processes on Edge Devices
## 2 Related Work  
Random forest-based models are widely [4, 5, 6, 7] used as they deliver strong predictive performance. However, their high resource requirements impede deployment on embedded systems and mobile devices where compute and memory budgets are tight. Many works have tried to preserve the accuracy of random forests while improving their efficiency in terms of model size and inference speed. One representative method is pruning, which removes some leaf nodes or sub-trees with a contribution to prediction that is marginal [8, 9, 10, 11]. However, these schemes often leave inefficient or non-informative nodes at intermediate levels untouched, and the overall optimization effect can be limited.  
A second approach exploits feature-importance statistics obtained during training or by post hoc analysis. Importance can be measured by impurity reduction or by the drop in accuracy when a feature is permuted [12, 13, 14]. As a result, only the top-N features are retained, and a smaller forest is re-trained on this subset, yielding compact models and potential speed-ups by eliminating low-relevance or redundant variables.  
In addition, gradient boosting-based ensembles such as LightGBM [1] and XGBoost [2] can build trees with efficiency as an another design goal. They adopt leaf-wise expansion that splits the leaf with the largest contribution in reducing loss. LightGBM combines this with Gradient-based One-Side Sampling (GOSS) and Exclusive Feature Bundling (EFB) to train faster than a conventional forest [1]. XGBoost achieves a speed-up of up to a ten times by approximate split finding and aggressive parallelization [2]. Also, there are explicit compression schemes and model-distillation methods that cut model size without significant accuracy loss [15; 16].  
These works aim to improve the performance of random forests by adopting better training method or by making forest efficient. On the other hand, there is a line of work that aim to transfer the knowledge of a large, accurate model (the teacher) into a smaller, more efficient student, which are called knowledge-distillation approaches. In this teacher-student architecture, the student is trained to mimic the teacher's soft labels rather than the hard class assignments, thereby exploiting richer information [3; 17; 18; 19]. A straightforward method is to set a trained random forest as the teacher and a single decision tree as the student; the tree is fitted so that, for any input, it reproduces the forest's soft outputs. One study identifies decision paths with the largest Gini-impurity reductions and encodes whether a sample satisfies each path as a binary feature; these features are then used to train a separate model (e.g., a decision tree) [20; 21]. Another work extracts core nodes, which are jointly based on impurity reduction and balance of split, and converts each into a stump, and feeds the stump outputs into a logistic regression layer that learns their relative importance, yielding a compact yet accurate surrogate [22].  
Overall, many works have been proposed to make random forests simple and efficient, but optimizing individual forests still falls short when a system must predict multiple tasks at the same time. First, even those optimized forests need one model per task, so the set of multiple models consume significant memory on embedded hardware. Second, separate models cannot effectively integrate cross-task information, leaving untapped the potentially informative relationship among tasks [23].  
We address these gaps with MicroForest: a compressed tree model distilled from multiple source forests. This unified model enables fast bottleneck prediction across all tasks in manufacturing process on resource-constrained devices.