fix up a few g_return*fail calls

This commit fixes some g_return_val_if_fail and g_return_if_fail
confusion.