Remove unnecessary const_cast

Now that an const T overload of xdr() has been added back, the cast is
no longer necessary.