Remove directory refresh preference.

This is present on the contact screen now, so there's no longer
any need for it here.