Tcl_ExternalToUtf appends a terminating NUL to its encoded results.
Perhaps this is a welcome convenience for some callers, but not for Tcl's
I/O system, which has no need for that.  Added a new flag value
TCL_ENCODING_NO_TERMINATE that callers can use to suppress this behavior.
This means buffers don't require so much padding, and a tiny bit of processing
is saved.  Update I/O callers to use the feature.
With no padding needed for a terminating NUL, there no need for a distinction
between the dstNeeded and dstLimit values.
Add to Tcl_ExternalToUtf() a capability to impose a limit on the number
of chars produce in the encoding result.  When the flag
TCL_ENCODING_CHAR_LIMIT is set and dstCharsPtr is not NULL, then the
initial value of *dstCharsPtr is taken as the max number of chars to produce.
The limit is imposed in a way that does not require the assistance of the
encoding's driver procs, but the flag is passed on to them in case they
can do better when they know they should.  No callers updated yet.
No drivers updated yet.

One difficulty is that this necessarily imposes a pre-translation limit,
and the I/O system has a history of wanting to impose only a post-translation
limit.
merge trunk
merge trunk
merge trunk
Use the new TCL_ENCODING_CHAR_LIMIT flag to have the encoding system manage
the max chars to read constraint.
Support TCL_ENCODING_CHAR_LIMIT in the BinaryProc driver.
Support TCL_ENCODING_CHAR_LIMIT in the UtfToUtfProc driver.
Support TCL_ENCODING_CHAR_LIMIT in the UnicodeToUtfProc driver.
Support TCL_ENCODING_CHAR_LIMIT in the Iso88591ToUtfProc driver.
Support TCL_ENCODING_CHAR_LIMIT in TableToUtfProc and EscapeToUtfProc drivers.
Add two new (undocumented) flags to the Tcl_ExternalToUtf() interface.
TCL_ENCODING_NO_TERMINATE rejects the default behavior of appending a
terminating NUL byte to the produced Utf output.  This permits use of all
of the dstLen bytes provided, and simplifies the buffer size calculations
demanded from callers.  Perhaps some callers need or appreciate this default
behavior, but for Tcl's own main use of encodings - conversions within I/O -
this just gets in the way.
TCL_ENCODING_CHAR_LIMIT lets the caller set a limit on the number of chars
to be output to be enforced by the encoding routines themselves.  Without
this, callers have to check after the fact for going beyond limits and
make multiple encoding calls in a trial and error approach.
Full compatibility is supported.  No defaults are changed, and the flags
have their effect even if an encoding driver has not been written to support
these flags (but greater efficiency is enjoyed if they do!).
All of Tcl's own encoding drivers are updated to support this.  Other encoding
drivers may exist somewhere, but I cannot point to any.
A TIP to document this and make it officially supported may come in time.
merge trunk
merge novem