(fix) Remove unsafety in set_mut by refactoring Modifier

Modifier::modify now receives `&mut F`, since the previous
version was exception unsafe and could cause double-frees
during unwinding.

As this changes the signature of Modifier::modify:

[breaking-change]