The insights from learning Reward Machines (RMs) can indeed provide valuable guidance for designing more interpretable reward functions in reinforcement learning beyond just representing non-Markov dynamics. By leveraging the structured and symbolic nature of RMs, researchers can design reward functions that capture complex dependencies and patterns in the environment.
One way to utilize these insights is to incorporate domain knowledge and expert input when designing reward functions. By encoding domain-specific rules and constraints into the RM structure, we can create reward functions that align with the desired behavior and objectives in a more interpretable manner.
Additionally, the concept of hidden triggers and reward dependencies in RMs can inspire the development of modular and decomposable reward functions. By breaking down the reward function into smaller components that interact based on specific triggers, we can create more transparent and understandable reward structures.
Furthermore, the interpretability of RMs can guide the design of reward shaping techniques that provide meaningful feedback to the learning agent. By shaping the reward signal based on the learned dependencies and triggers, we can steer the agent towards desired behaviors while maintaining transparency and interpretability in the learning process.