Fix syncing of deleted attributes for existing tasks.

Can't think of a good way to test this sadly. Would be good if the loader took
configuration as an argument, and did not have the YAML read hardcoded into it.