While risk-sensitive RL offers a unique perspective on handling uncertainty and risk in reinforcement learning problems, it also comes with certain limitations and drawbacks compared to other risk-aware objectives like Conditional Value at Risk (CVaR) or certainty-equivalency.
One limitation of risk-sensitive RL is the complexity of the exponential form objective function, which may lead to challenges in optimization and convergence. The exponential form introduces non-linearity and can make the learning process more computationally intensive compared to other risk-aware objectives.
Another drawback is the sensitivity of the risk sensitivity coefficient in risk-sensitive RL. The optimal value of the coefficient can significantly impact the performance of the algorithm, and determining this value accurately can be challenging. In contrast, other risk-aware objectives like CVaR may offer more straightforward interpretations and parameterizations.
Additionally, risk-sensitive RL may struggle with scalability and generalization to complex environments. The exponential form objective function may not always capture the full spectrum of risk preferences and may not be easily adaptable to different tasks and domains.
Overall, while risk-sensitive RL has its advantages in capturing risk attitudes and uncertainties, it is essential to consider these limitations when choosing between different risk-aware objectives for reinforcement learning tasks.