* src/manp.c (check_and_add): New function.
  (add_nls_manpath): Use check_and_add to reduce repetition. When
  processing ll_CC.CODESET, try ll.CODESET after ll_CC.CODESET and
  ll_CC but before ll.
* src/man.c (maybe_update_file): Only recache pages automatically if
  --update was given.
merge from trunk
* src/man.c (add_candidate): Skip candidates that match an existing
  candidate for the same language even if they are for a different
  country, codeset, etc.
* lib/decompress.c (decompress_open, decompress_fdopen): Don't start
  the returned pipeline.
* lib/decompress.h (decompress_open, decompress_fdopen): Note that
  the caller must start the returned pipeline.
* src/lexgrog.l (find_name): Start the decompressor pipeline.
* src/man.c (display): Likewise.
* src/ult_src.c (ult_src): Likewise.
* src/zsoelim.l (open_file): Likewise.
* src/straycats.c (check_for_stray): Append 'col -bx' to the
  decompressor pipeline rather than connecting two pipelines
  together. Don't call pipeline_wait by hand as pipeline_free will
  do this for us now.
Add limited character set aliasing support.

* src/encodings.c (charset_alias_table): New table for common
  character set aliases.
  (get_canonical_charset_name): New function to inspect
  charset_alias_table.
  (get_page_encoding): Use get_canonical_charset_name. Adjust FHS
  commentary, since we now support its craziness.
  (get_standard_output_encoding): Use get_canonical_charset_name.
  (get_locale_charset): Likewise.
* src/encodings.h: Prototype get_canonical_charset_name.

* src/encodings.c (directory_table): Use EUC-CN rather than GB2312
  and BIG5HKSCS rather than BIG5-HKSCS (canonical names according to
  glibc's /usr/lib/gconv/gconv-modules).
  (charset_table): Likewise.
  (compatible_encodings): Likewise.
merge from trunk
Back out the last two non-merge revisions on this branch. Instead:

* src/man.c (duplicate_candidates): New function to determine
  whether candidates are substantially duplicated, and if so in
  which direction. The logic is better than before: it now knows how
  to compare locales and pick the best available candidate for each
  language, and prefers UTF-8 candidates if everything else is
  equal.
  (add_candidate): Move duplication logic to duplicate_candidates.
  Adjust insertion handling so that we can drop old candidates in
  favour of new candidates.
* src/manp.c (check_and_give): Remove.
  (unpack_locale_bits): New function, inspired by _nl_explode_name
  in libintl.
  (free_locale_bits): New function.
  (add_nls_manpath): Instead of trying individual directories (which
  doesn't work so well with character set aliasing), iterate over
  all subdirectories of each manpath entry and add all those
  matching the current locale's language and (if specified)
  territory. This now includes ll.CODESET as well as more
  conventional locale names.
* src/manp.h: Declare 'struct locale_bits'. Prototype
  unpack_locale_bits and free_locale_bits.
* src/man.c (lang_dir): Move to ...
* src/util.c (lang_dir): ... here.
* src/lexgrog.l (long_options, args, main): New -E/--encoding
  option.
  (usage): Document it.
  (cs_name, hu_name, ja_name, ru_name): Recode to UTF-8.
  (find_name): Add encoding argument. Guess the page's encoding from
  its filesystem location (or use encoding if supplied), and add an
  iconv command to recode it to UTF-8.
* man/man1/lexgrog.man1 (SYNOPSIS, OPTIONS): Document --encoding.
* src/straycats.c (check_for_stray): Guess the page's encoding from
  its filesystem location, and add an iconv command to recode it to
  UTF-8.
* include/manconfig.h.in (find_name): Update prototype.
  (lang_dir): Add prototype.
* src/Makefile.in (mandb, lexgrog): Add encodings.o.
* docs/TODO: Update.
reorganise changelog
* src/whatis.c (display): Convert output from UTF-8 to locale
  charset.
  (main): Initialise iconv descriptor.
* configure.ac: Call AM_ICONV.
* src/Makefile.in (whatis, apropos): Add encodings.o and
  $(LIBICONV).
* docs/TODO: Update. I've decided not to bump the database version.
* src/whatis.c (usage): Document -L/--locale.
* man/man1/apropos.man1 (SYNOPSIS, OPTIONS): Likewise.
* man/man1/whatis.man1 (SYNOPSIS, OPTIONS): Likewise.
* src/encodings.c (directory_table): Add Serbian -> ISO-8859-5.
* src/lexgrog.l (cs_name, hu_name): Expand character classes
  containing non-ASCII characters into the form (a|b|...); flex
  doesn't understand Unicode well enough to deal with them
  otherwise.
  (cs_name, ru_name): Add more possible translations.
  (ko_name, sk_name, sr_name, tr_name, zh_CN_name, zh_TW_name): Add.
  (find_name_decompressed): Ignore exit status from decompressor
  pipeline; iconv may exit non-zero on misencoded pages, but if we
  manage to extract a whatis from it anyway then it doesn't really
  matter.
* src/whatis.c (main): Sync locale handling with src/man.c, in order
  to look up per-locale databases correctly. Based on a patch by
  Nicolas François (Debian bug #281811).
* docs/TODO: Update. (Now unchanged in this branch relative to trunk.)
polish docs/ChangeLog and docs/NEWS
* src/whatis.c (simple_convert): New function to simplify iconv
  handling.
  (display): Use it.
Create and use databases for non-English manual hierarchies (Debian
bug #29448). Extensive changes elsewhere to support this by means of
better UTF-8 handling and knowledge of more languages.

* src/check_mandirs.c (make_database_directory): New function.
  (create_db, update_db): Use it.
  (purge_missing): Check that the database exists before starting.
* src/mandb.c (process_manpath): New function, split out from main.
  Make the single_filename check aware that the file might be in a
  per-locale subdirectory that we aren't processing right now.
  (main): Use it. Move man_owner, purged, and strays to file scope.
  Build databases for all per-locale subdirectories.
* src/straycats.c (straycats): Constify manpath argument.
* src/whatis.c (main): Sync locale handling with src/man.c, in order
  to look up per-locale databases correctly. Based on a patch by
  Nicolas François (Debian bug #281811).
* include/manconfig.h.in (straycats): Update prototype.

* src/man.c (duplicate_candidates): New function to determine
  whether candidates are substantially duplicated, and if so in
  which direction. The logic is better than before: it now knows how
  to compare locales and pick the best available candidate for each
  language, and prefers UTF-8 candidates if everything else is
  equal.
  (add_candidate): Move duplication logic to duplicate_candidates.
  Adjust insertion handling so that we can drop old candidates in
  favour of new candidates.
* src/manp.c (check_and_give): Remove.
  (unpack_locale_bits): New function, inspired by _nl_explode_name
  in libintl.
  (free_locale_bits): New function.
  (add_nls_manpath): Instead of trying individual directories (which
  doesn't work so well with character set aliasing), iterate over
  all subdirectories of each manpath entry and add all those
  matching the current locale's language and (if specified)
  territory. This now includes ll.CODESET as well as more
  conventional locale names.
* src/manp.h: Declare 'struct locale_bits'. Prototype
  unpack_locale_bits and free_locale_bits.

* src/man.c (lang_dir): Move to ...
* src/util.c (lang_dir): ... here.
* src/lexgrog.l (long_options, args, main): New -E/--encoding
  option.
  (usage): Document it.
  (cs_name, hu_name, ja_name, ru_name): Recode to UTF-8.
  (find_name): Add encoding argument. Guess the page's encoding from
  its filesystem location (or use encoding if supplied), and add an
  iconv command to recode it to UTF-8.
* man/man1/lexgrog.man1 (SYNOPSIS, OPTIONS): Document --encoding.
* src/straycats.c (check_for_stray): Guess the page's encoding from
  its filesystem location, and add an iconv command to recode it to
  UTF-8.
* src/whatis.c (simple_convert): New function to simplify iconv
  handling.
  (display): Convert output from UTF-8 to locale charset.
  (main): Initialise iconv descriptor.
* include/manconfig.h.in (find_name): Update prototype.
  (lang_dir): Add prototype.
* configure.ac: Call AM_ICONV.
* src/Makefile.in (mandb, lexgrog): Add encodings.o.
* src/Makefile.in (whatis, apropos): Add encodings.o and
  $(LIBICONV).

* src/encodings.c (directory_table): Add Serbian -> ISO-8859-5.
* src/lexgrog.l (cs_name, hu_name): Expand character classes
  containing non-ASCII characters into the form (a|b|...); flex
  doesn't understand Unicode well enough to deal with them
  otherwise.
  (cs_name, ru_name): Add more possible translations.
  (ko_name, sk_name, sr_name, tr_name, zh_CN_name, zh_TW_name): Add.
  (find_name_decompressed): Ignore exit status from decompressor
  pipeline; iconv may exit non-zero on misencoded pages, but if we
  manage to extract a whatis from it anyway then it doesn't really
  matter.

* src/whatis.c (usage): Document -L/--locale.
* man/man1/apropos.man1 (SYNOPSIS, OPTIONS): Likewise.
* man/man1/whatis.man1 (SYNOPSIS, OPTIONS): Likewise.

* docs/NEWS: Document this.
* man/de_DE.88591, man/es_ES.88591, man/fr_FR.88591,
  man/it_IT.88591, man/ja_JP.eucJP: Rename to ...
* man/de.UTF-8, man/es.UTF-8, man/fr.UTF-8, man/it.UTF-8,
  man/ja.UTF-8: ... these. Recode all contents to UTF-8.
* man/Makefile.in (NLS): Adjust directory search.
* man/THANKS: Update credits.
* .bzrignore: Update for manual directory renaming.
* GNUmakefile.in (nls_man): Likewise.