Ensure policy marker files are generated

Close #2659, #2694