One potential drawback of the regression-as-classification approach is the discretization of the output space, which may lead to information loss and reduced accuracy, especially in cases where the output distribution is continuous and complex. This discretization can limit the model's ability to capture the nuances of the output distribution, particularly in scenarios with multiple modes or heavy tails. To address this limitation, techniques like adaptive binning or dynamic bin creation based on the data distribution can be implemented. This would allow for a more fine-grained representation of the output space, enabling the model to better capture the complexity of the distribution.
Another limitation is the reliance on a single loss function, which may not always capture the full complexity of the relationship between inputs and outputs. To address this, a combination of loss functions or a more sophisticated loss function that considers the ordinal structure of the output space could be explored. Additionally, incorporating regularization techniques to prevent overfitting and improve generalization can help mitigate the limitations of the regression-as-classification approach.