# Unbiased Isotonic Regression Tree for Discovering Hidden Heterogeneity in Monotonicity Constraints
###### Abstract  
Integrating domain knowledge is increasingly recognized as vital for improving the relevance and reliability of machine learning models. This integration is often implemented through specific types of constraints that reflect real-world conditions or theoretical insights. Within the family of regression trees, the isotonic regression tree is used to incorporate a monotonicity constraint between a predictor and a response variable. However, the isotonic regression tree could be susceptible to split selection bias, as it selects both its split variable and cutpoint simultaneously. This study first explores the possibility of selection bias on split variables in the isotonic regression tree and proposes an unbiased isotonic regression tree that mitigates the issue of the selection bias problem. The results of the simulation and case study demonstrate the effectiveness of the proposed approach and the ability to discover hidden heterogeneous monotonic constraints. s +
Footnote †: journal: Applied sciences  
1  
e1e30  
with monotonicity and concavity constraints [9]. Sensitivity Direction Learning (SDL) is introduced to analyze neural networks by embedding user-defined relationships, such as monotonicity and convexity, between input features and outputs [10]. SDL achieves this by applying soft shape constraints that embody domain knowledge, and the model enables us to adhere to meaningful patterns defined by the user. Furthermore, several studies have advanced constrained Gaussian process modeling, which includes various shape constraints as well as other types of limitations such as bounds and censoring [11; 12; 13; 14]. By embedding expert knowledge specific to a domain, models can be fine-tuned to better reflect the real-world dynamics and specific requirements of various fields. Within the broader trend of integrating domain knowledge, the incorporation of monotonicity constraints holds particular significance in enhancing model performance and reliability in real-world applications [15]. These constraints enforce logical and consistent directional relationships between inputs and outputs, reflecting prior knowledge and improving generalization [16; 17]. By introducing structural biases, monotonicity reduces overfitting, enhances predictive power, and supports robust learning in data-scar scenarios [18]. Additionally, these constraints promote transparency and interpretability, which are essential for trust in machine learning systems [19]. They are particularly critical in domains like finance and healthcare, where fairness, ethical concerns, and safety require consistent and logical decision-making [20]. The most recent regression tree to handle a monotonicity constraint is the isotonic regression tree (IRT) [21]. As a regression tree, IRT is also a flexible and non-parametric model that fits a monotonically increasing/decreasing curve to samples by minimizing the mean squared error on the dataset. Basically, IRT is designed to solve bid landscape forecasting problems, but it is also applicable to other domains where enforcing a monotonicity constraint between input and output variables is critical. Known for their non-parametric and uniformly consistent properties, isotonic regression models can represent monotone relationships effectively when adequate data are provided [22; 23]. An example of IRT is represented in Figure 1. In Figure 1, there is one split covariate (patient's condition), one predictor (medication dosage), and the response variable (the recovery score of patients). A non-decreasing relationship between the medication dosage and recovery score is expected because higher doses should generally lead to higher recovery scores. Isotonic regression effectively captures this trend by fitting a model that strictly enforces the monotonic increase in the recovery score with dosage without assuming linearity. However, the degree to which there is a monotonic increase may vary depending on the patient's condition.