Obey default DecodeFormat in prefill API.

Also does some cleanup, including switching to use
a builder and re-naming a few classes.