invoke getmetatable(val).__tojson() if present

This allows you to provide custom serialization.
It mimics the same feature in dkjson.