Ensure policy validated marker files are generated.

Close #2659, #2694