Revert "Clear dirty attrs on encrypt too"

This reverts commit d6d92666c1c20bddf6b835430c3553c951de3ee8.