In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
Q: We present a new method for estimating the expected return of a POMDP from experi ence. The estimator does not assume any knowledge of the POMDP, can estimate the returns for finite state controllers, allows ex perience to be gathered from arbitrary se quences of policies, and estimates the return for any new policy. We motivate the estima tor from function-approximation and impor tance sampling points-of-view and derive its bias and variance. Although the estimator is biased, it has low variance and the bias is of ten irrelevant when the estimator is used for pair-wise comparisons. We conclude by ex tending the estimator to policies with mem ory and compare its performance in a greedy search algorithm to the REINFORCE algo rithm showing an order of magnitude reduc tion in the number of trials required.
A:
Policy Improvement for POMDPs using Normalized Importance Sampling