We propose two algorithms for episodic stochastic shortest path problems with linear function approximation. The first is computationally expensive but provably obtains (\sqrt{B_\star^3 d^3 K/c_{min , where B_\star is a (known)upper bound on the optimal cost-to-go function, d is the feature dimension, K is the number of episodes, and c_{min . Both algorithms are based on an optimistic least-squares version of value iteration analogous to the finite-horizon backward induction approach from Jin et al. 2020. To the best of our knowledge, these are the first regret bounds for stochastic shortest path that are independent of the size of the state and action spaces .
We propose two algorithms that use linear function approximation (LFA) for stochastic shortest path (SSP) and bound their regret over K episodes. When all stationary policies are proper, our first algorithm obtains sublinear regret (K^{3/4 , is computationally efficient, and uses stationary policies. This is the first LFA algorithm with these three properties, to the best of our knowledge. Our second algorithm improves the regret to \sqrt{K . Both algorithms are special cases of a more general one, which has \sqrt{K the first for SSP with function approximation .
R
for episodic
that use linear function approximation (LFA) for
D
problems with linear function approximation. The first is computationally expensive but provably obtains
R
(\sqrt{B_\star^3 d^3 K/c_{min
(SSP) and bound their regret over K episodes. When all stationary policies are proper, our first algorithm obtains sublinear regret (K^{3/4
R
where B_\star is a (known)upper bound on the optimal cost-to-go function, d is the feature dimension, K is the number of episodes, and c_{min
is computationally efficient, and uses stationary policies. This is the first LFA algorithm with these three properties, to the best of our knowledge. Our second algorithm improves the regret to \sqrt{K
R
based on an optimistic least-squares version of value iteration analogous to
special cases of a more general one, which has \sqrt{K
R
finite-horizon backward induction approach from Jin et al. 2020. To the best of our knowledge, these are the first regret bounds for stochastic shortest path that are independent of the size of the state and action spaces
first for SSP with function approximation