An Algorithmic Approach for Optimal Stopping of Random Walks and Its Extensions