gfapi: dict_unref() xattr_req in fop finish instead of dict_destroy()

The current way of calling dict_destroy() at the end of an API fop assumes
that xattr_req is not stored/ref'd by any translators in the stack. However
when translators like DHT store xattr_req in dht_local_t with a dict_ref()
and perform dict_unref() in the unwind path, things get subject to a race.

The race is between the woken up thread (by syncop_wake) i.e, the gfapi
invoking thread and the thread where the FOP was unwound. As the C stack
of STACK_UNWIND unwinds back, dht_local_unref() gets invoked within the
DHT_STACK_UNWIND macro. This thread attempts dict_unref, which would
be "safe" if it wins the race against the gfapi invoking thread. However
if the gfapi invoking thread wins the race, it will perform dict_destroy()
first and therefore make dict_unref() within dht_local_unref perform
a double free.

This is the embarrassing on-screen bug which showed up in a roomful of people
during the gluster dev summit demo of qemu/libgfapi integration.

Change-Id: I284c93de87cdc128d5801f42c84aa87f753090d4
BUG: 839950
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4644
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>