Previous commit wrongly claimed that both sanity tests passed. This was wrong, since 8-bit test did not pass.
This has been fixed by properly configuring uc.c to handle conditional assembly for both 8 and 32 bit cases.
Also fix error in compiling utf8.c for 8-bit case
Fix some errors in unicode support to properly deal with
length of generated utf-8 encoded strings.
Add several small test programs: copy.spt to copy file,
reverse.spt to reverse each line in a file, and chars,spt
to build character frequency table. All of these were tested on simple file containing cyrillic (russian) characters.
Add test/ru.txt, a small file containing some cyrillic text.
Extern the string hash algorithm, which assumed there were several characters in a word, to support unicode.
This is done by shifting the individual characters before combining them in the hashcode.
Change architecture names in makfile to use '-' and not '_', giving
ARCH=x32=8 and ARCH=x32-32, for example.
Increase length of temporary buffers used for conversion to and from unicode from
512 to 2048. This is to allow Unicode lines of up to 512 characters for which utf-8 conversion
will take more than 512 bytes. Until adding unicode support, the same buffer length could be used
for both Minimal-format and c-format strings.