NEWS updates.

  * Forgot to add a NEWS update to my recent commit involving the
internal-error logic.

  * And clarify that only vectors of boxed items may be stack-
allocated on PPC.
code/room: Improve type-format database initialization for simple vector types.

  * There has been a longstanding FIXME comment on a piece of code
which contains a hand-maintained list of specialized vector types
and the shift count for converting the length from elements to
octets.

  * It turns out that all of this information, plus the type names
that we currently do a song-and-dance with INTERN, SUBSEQ, and
MISMATCH to obtain, plus information for the string types, is
available from *SPECIALIZED-ARRAY-ELEMENT-TYPE-PROPERTIES*.  And
*S-A-E-T-P* is guaranteed to be up-to-date, as it's too central to
our implementation of UPGRADED-ARRAY-ELEMENT-TYPE and MAKE-ARRAY
for it to be allowed to break.

  * So, replace nasty KLUDGE of an initialization for simple
vector types with something more principled, making it explicit
which properties need to be derived and which are simply already
available, and picking off the one specialized array type that
needs to be handled differently (SIMPLE-ARRAY-NIL).
code/room: Completely rewrite MAP-ALLOCATED-OBJECTS.

  * The old version of M-A-O consisted of bizaare toplevel logic,
a scheme for figuring out what each heap object was and its size
that did not parallel what the garbage collector used and may or
may not have been correct, and relied heavily on inlining to
reduce consing.

  * This new version of M-A-O uses straightforward toplevel logic,
a scheme for figuring out what each heap object is and its size
that directly parallels what the garbage collector uses and is
verifiably correct, and relies heavily on the aligned unboxed
pointer to fixnum equivalence to reduce consing.

  * The new interface to M-A-O no longer includes the optional
"careful" argument, as it gains us nothing once the underlying
mechanism is so obviously correct.  sb-introspect has been updated
appropriately.

  * The way the new implementation walks the heap and page table
requires direct access to a "static" global variable in gencgc.c,
so the "static" attribute has been removed.

  * This implementation has been lightly tested on an x86-64 and
PPC, and it seems to work quite well, but there are still some
fairly obvious non-optimalities in terms of generated code (as
seen in the trace-file output from the cross compiler).  It does
pass the two test cases that exhausted the heap on PPC with the
previous implementation.
early-alieneval: Fix package-related thinko with saved-fp-and-pc logic.

  * Forgot a package prefix for GET-LISP-OBJ-ADDRESS, because the
function symbol was exported from SB!ALIEN-INTERNALS yet defined
in SB!ALIEN, and the prefix wouldn't have been necessary from
SB!ALIEN-INTERNALS.

  * Thanks to Stas Boukarev for the heads-up.
Fix describe-object for characters.

Don't prefix lines with ":_".
loop: remove code size-estimation.

Loop has a facility to determine whether it's ok to duplicate variable
initialization and stepping code when the variable preceding it has
different initialization and stepping forms. The code which determines
code size is quite strange and it may have been relevant 20 years ago
on primitive implementations, but not anymore, and people who really
care about code size would use functions, which will also improve code
readability.

As a side effect, it fixes a bug which was present in the
estimate-code-size function.

Fixes lp#1178989.
Add a stub for %other-pointer-p.

Otherwise the VOP isn't translated on literal objects, and
sb-sequence:do-sequence stops working on literal vectors. Having a
stub allows constant folding to work.

Reported by adeht on #lisp.
Recover full backtraces with generic arithmetic on x86 and x86-64

 * Errors in generic arithmetic (or comparisons) used to hide the caller
   in the backtrace: it was replaced with a frame in the anonymous
   assembly stub.

 * Regression since 1.0.24.35, fixes lp#800343.

 * Also remove a misleading FIXME in typed-accessor-definitions
   (reported by Matt Novenstern in lp#1171646).
COMPILE-FILE shouldn't "attempt to dump invalid structure" anymore

 * When CAST nodes detect definite type mismatch, they are replaced
   with debugging instrumentation to provide source locations at
   compile and run -time. When code is generated internally, the
   source can include literal internal data structures. Skip those
   when recovering source locations.

 * Fixes lp#943953 and a bunch of equally baffling duplicates.
More robust erroneous local call detection

 * When possible, convert known bad calls into calls to error-signaling
   stubs.

 * Fixes lp#504121 (and likely other occurrences of
   "failed AVER (ZEROP (HASH-TABLE-COUNT ...))."
Insert typechecks before RAW-INSTANCE-INIT in structure constructors

 * Usually, FTYPE declarations ensure that happens, but multiple
   inlining of the same structure constructor cause strangeness.

 * Fixed lp#1177703, reported by Jan Moringen.
More efficient MASK-SIGNED-FIELD

 Word => signed-word and {word, signed-word} => fixnum conversions
 are implemented with unchecked move VOPs.
Complete cut-to-width

 * Insert logand/mask-signed-field even around references to variables
   in modular arithmetic: avoid recursive rewriting by disabling the
   transform when the destination is a direct logand/mask-signed-field
   combination.

 * Fixes lp#1026634 (reported by Anton Marsden on sbcl-devel).
update to unicode 6.2
MORE COMMENT regarding the careful format of the encoded UCD data
Incorporate some decomposition information in ucd table

Oh boy.  This one is quite intricate.  We have two bytes free in
the 8-byte entries for information about characters, so use one of
them to indicate if the character has a decomposition, and if so of
what kind it is.  Adapt the ucd.lisp tools-for-build code to
parse and preserve that information.

However, this causes there to be more than 256 distinct possible
classes of character known to the system: not a problem in principle,
but Teemu Kalvas' implementation of the double indirection depended on
having a one-byte index.  But since Unicode characters are limited to
21 bits, with a careful packing scheme we can in fact steal 3 more bits
for the index, at the cost of needing to do an extra memory reference
and some arithmetic to reconstruct the index.  (In the process, change
the endianness of the ucd.dat filesystem representation, because it's
easier that way).

But wait, there's more.  Before, there were only two kinds of
lower-case characters: those whose upper-case transformation
lowercase back to the original character, and those where there is
no round-trip.  (The former are cl:lower-case-p, the latter aren't).
This gave rise to straightforward implementations of lower-case-p
and friends; in the new world, where there are multiple different
kinds of lower-case characters (with various decomposition classes)
we need to adjust the implementations, still fairly straightforward,
of lower-case-p and related functions.

The extra information provided in the ucd table by this commit
is largely useless on its own; the next step is to incorporate
the actual decomposition data.  Stay tuned.
delete now-unused code from ucd.dat
beginnings of decomposition

Store enough information in output from ucd.lisp to be able to actually
decompose individual characters.  Include proof-of-concept implementation
of decomposition, not hooked into anything yet.
work-in-progress towards full normalization support
handle Hangul syllable decomposition

Entries for the codepoint range (#xac00 -- #xd7a3) have 1 for
their decomposition-info, a decomposition length of 2 or 3, but
a zero decomposition index (the decomposition is handled
algorithmically instead).