Fix 'r' is used uninitialized when compiled

When line:529 "if" condition is true, "r" is uninitialized and returned, causing compile error under LLVM

engine/serialize.c:529:6: error: variable 'r' is used uninitialized whenever 'if'
      condition is true [-Werror,-Wsometimes-uninitialized]
        if (exp_xsum != act_xsum) {