Improve flag action to keep flag info between removals