Venue: UAI 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept, e.g., novelty, outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: Q1 Summary and contributions: The work investigates an offline version of the standard stochastic shortest path (SSP) problem, giving two algorithms for it and analyzing their properties.

Q3 Main strengths: This is a solid technical study of offline SSP and two of its algorithms.

Q4 Main weakness: Presentation of the work is quite abstract, leaving much of the background unexplained.

An experimental study would strengthen the work.

Q5 Detailed comments to the authors: It would be good to clarify (in the introduction) the differences between "offline shortest path" and the two components (I) constructing a conventional MDP/SSP from historical data, and (II) applying conventional (non-learning) MDP/SSP algorithms. Is this (only?) about the scarceness of the historical data, which makes it difficult/impossible to do (I) well?

There are other readability issues. E.g. the "data" in "The Offline SSP task" seems to be a collection of sequences of state, action, cost, but it would be good to confirm this assumption.

Bibliography: Two dots missing in the "o" in "Gyorgy" and ' in the lower case "a" in "Andras".

Bibliography should also otherwise be cleaned up. E.g. things that should be upper case is in lower case, including "RL", "Markov"

Q7 Justification for your score: 
The paper is very technical, a bit exhausting, and would significantly benefit from improved presentation as well as better placement in the context provided by earlier works.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area of AI or moderate to high impact on more than one area of AI, with good to excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations. 
review_content: Q1 Summary and contributions: The paper studies the problem of learning policies to minimize the cost of reaching a specific state in an MDP in an offline setting. This means that a policy is learned based on historical data and not by interacting with the environment. This is motivated by real-world situations which involve the control of systems where the cost of making a mistake is high. The authors present algorithms for policy learning and evaluation and prove instance-based bounds on their performance. 
Q3 Main strengths: The problem tackled by the authors appears to be well-motivated, important and intuitive. The problem formulation appears to be original, though I am not very confident in those assessments as this is far from my research area. The paper is also quite clear at the high level, even though the content is very technical. The authors do a good job structuring their results in a way that makes them easy to understand and not too hard to verify, though due to time constraints I had to skip verifying some of the lemmas.
Q4 Main weakness: While the overall structure of the paper is well laid out, some paragraphs are confusing as the authors try to condense too much information in too little space. The authors cite a lot of previous work, but it is not clear which of those papers are most relevant to the present work.
Q5 Detailed comments to the authors: - p. 5, left middle - reply-> rely

last sentences in Remark 6.2 unclear

since both V̄ and P e s,π
? (s) depend on data, therefore
Bernstein concentration cannot be directly applied - very confusing 
Q7 Justification for your score: The paper addresses an important problem and delivers a clear result. I am not an expert in the field, however, so I do not have a lot of confidence in my assessment.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate to high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations. 
review_content: Q1 Summary and contributions: The paper looks at the offline version of the stochastic shortest path problem. SSP is representative for goal-oriented reinforcement learning having a wide range of applications. The offline version of SSP is applicable to real-world situations where online interactions are infeasible or too expensive. The main contribution consists of offline policy evaluation as well as offline policy learning algorithms with strong statistical guarantees.

Q3 Main strengths: The main strength of the paper is an extensive theoretical analysis of offline policy evaluation as well as offline policy learning algorithms for the offline SPP problem.

Q4 Main weakness: The presentation is quite dense using a relatively heavy notation which makes it inaccessible to the wider AI community. It is actually quite difficult to follow the technical details without some supporting running examples.

Q5 Detailed comments to the authors: I was actually quite surprised  that the paper doesn't include an empirical evaluation. It is typically the case these days to evaluate reinforcement learning algorithms on standard benchmarks in order to provide some evidence on their performance in practice. 

The offline SSP problem appears to be new and therefore I think some sort of evaluation would strengthen the paper considerably.

Q2(4) Quality of experiments (Optional): 1: Poor: The experimental evaluation is flawed or the results fail to adequately support the main claims.
Q7 Justification for your score: The paper provides a strong theoretical contribution but lacks an empirical evaluation of the proposed algorithms.
review_title: null

Meta-review: Meta Review: Ultimately this paper ended up just above the threshold of UAI acceptance--the authors gave a very nice response.  In the end, two reviewers requested experimental validation of the result to provide evidence of practical performance of the proposed algorithm -- this would strengthen the paper since constants can matter for practical implementation.  As a minor remark, I note that some of the motivating examples (Mujoco) with seemingly continuous state and action spaces can be misleading if the authors require finite state and action spaces; and returning to the experimental concern, it's always nice to evaluate on the problems that one motivates with.
Decision: Accept (Poster)