Fixup unsafe struct ref/set operations to work with callable structs