Check for stale db locks when opening write-cursors

- During long-running transactions its entirely possible for some
  other player to come and go leaving stale locks behind and cause
  the transaction to get stuck until the cavalry comes along in the
  form of somebody else opening the rpmdb, clearing the blockage.
- Presumably dbenv->failchk() is not entirely free of cost so we only
  do this for writes which are way more critical and also more prone to
  getting stuck.
- dbenv->failchk() could return DB_RUNRECOVER in which case we should
  abort everything but we lack a mechanism to do it... just add
  a reminder comment for now.
Warn when whitespace is missing before macro body

This will now issue a warning when macro definition is possibly
incorrect or ambigous, such as the one found in FC18 lvm2.spec:

%define util-linux_version 2.22.1
warning: Macro %util needs whitespace before body
Stricter macro substitution syntax

This change introduces a separate routine to parse for valid macro
names.  Valid macro names are either regular 3+ character identifiers,
or special names: "S", "P", "0", "#", "*", "**", macro options such as
"-o" and "-o*", and macro arguments such as "1".  Other names are not
valid.  This fixes a number of bugs seen earlier due to sloppy name
parsing: "%_libdir*" and "%01" were not expanded (these are now expanded
to e.g. "/usr/lib64*" and "<name>1", as expected).  This also fixes
bugs in as-is substitution: "%!foo" was expanded to "%foo", and likewise
"%!!!" was expanded to "%" (and to "%<garbage>" at EOL).

Also, bad names in %name and %{name...} substitutions are now handled
differently.  In %name form, the name is parsed tentatively; a silent
fall-back to as-is substitution is provisioned when no valid name can
be obtain.  In %{name...} form, a failure to obtain a valid name is now
a syntax error.  Furthermore, only 3 variants are syntactically valid:
%{name} proper, %{name:...}, and %{name ...}.  This renders invalid
ambiguous macro substitutions such as the one found in FC18 lvm2.spec:

Requires: util-linux >= %{util-linux_version}
error: Invalid macro syntax: %{util-linux_version}
Warn when macro options evaporate silently

In the %name form, when the name is invalid or when the macro does
not exist, a silent fall-back is provisioned to as-is substitution.
On the contrary, the %-o form has %?name semantics, that is, implies
an existent test.  Hence at the top level, i.e. in specfile sections,
options always evaporate silently.  Given the ambiguity of the %-o form,
this deserves a warning.

For example, the following line found in FC18 xfce4-taskmanager.spec:
- Add patch to fix 0%-CPU bug
gets actually expanded to:
- Add patch to fix 0PU bug
This will now produce the warning:
warning: %-CPU parsed as %{-C}PU
Check for undefined macros

This change introduces a generalized routine rpmExpandMacros() to expand
macros on behalf of user input, with improved diagnostic facilities.
In particular, one of its arguments is a callback function which is
called whenever an undefined macro is encountered in user input.

When specfile parser calls this routine, it supplies a custom function
to handle undefined macros.  This function is keenly aware of specfile
syntax: valid specfile tokens, such as %prep and %setup, when found in
the right context, are not treated as undefined macros.  Otherwise,
there are 3 possibilities.
- Undefined macros in preamble and %pre/%post/... scriptlets raise
  an error, unless a non-build parse is performed; this is to prevent
  malformed packages from being built.
- In other cases, a warning is issued.
- Some tokens are permitted beyond their context (e.g. %ghost in
  %changelog), to comply with traditional/sloppy usage.  Also, undefined
  macros found in comments normally do not produce a warning.

Update:
- When checking for comments, allow leading whitespace.
- In copyNextLineFromOFI, eliminate rstrlcpy completely: instead of
  truncating the expanded output, simply replace spec->lbuf.
Improve macro table performance

In the existing implementation, when a new macro is added, the whole
table has to be sorted again.  Hence the cost of adding n macros is
worse than O(n^2), due to arithmetic progression.

This change drops all qsort(3) stuff altogether, by carefully preserving
table in sorted order.  In findEntry routine, bsearch(3) is replaced
with customized binary search which tracks position for insertion.
In the addMacro routine, if a matching entry is not found, this
position is used for direct insertion, after the rest of the elements
are "shifted to the right" with memmove(3).  Likewise, in delMacro
routine, the elements are shifted back to the left when the last macro
definition is popped.  Technically, shifting half of the array with
memmove(3) is still O(n^2); however, modern CPUs process contiguous
memory in a very efficient manner, and glibc provides a fine-tuned
memmove(3) implementation.

Also, macro table entries are now allocated in a single chunk.

This change reduces rpm startup costs by factor of 6 (see callgrind
annotations below).  Also, this change improves specfile parser
performance by a factor of 2 (e.g. the parse time of texlive.spec
is reduced from 67s to 35s).

Callgrind annotations for 'rpm --eval %_libdir', previous commit:
88,193,084  PROGRAM TOTALS
27,669,049  glibc-2.16-75f0d304/stdlib/msort.c:msort_with_tmp.part.0'2
24,880,649  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strcmp.S:__GI_strcmp
11,111,276  rpmio/macro.c:compareMacroName
 7,048,036  glibc-2.16-75f0d304/string/../sysdeps/x86_64/memcpy.S:__GI_memcpy
 1,943,118  glibc-2.16-75f0d304/stdlib/msort.c:msort_with_tmp.part.0
 1,903,867  glibc-2.16-75f0d304/elf/dl-lookup.c:do_lookup_x
 1,701,781  rpmio/macro.c:rpmLoadMacroFile
 1,491,535  rpmio/macro.c:doDefine
 1,439,321  rpmio/macro.c:sortMacroTable
 1,254,448  glibc-2.16-75f0d304/malloc/malloc.c:_int_malloc
   683,782  glibc-2.16-75f0d304/malloc/malloc.c:_int_free
   482,748  glibc-2.16-75f0d304/elf/dl-lookup.c:_dl_lookup_symbol_x
   397,935  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strcmp.S:strcmp'2
   307,673  glibc-2.16-75f0d304/elf/dl-misc.c:_dl_name_match_p
   306,247  rpm-4.10.2/lib/rpmrc.c:doReadRC
   300,710  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strcpy.S:__GI_strcpy
   263,309  glibc-2.16-75f0d304/malloc/malloc.c:malloc
   254,657  rpm-4.10.2/lib/rpmrc.c:doReadRC'2
   240,893  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strlen.S:__GI_strlen
...

Callgrind annotations for 'rpm --eval %_libdir', this commit:
14,813,520  PROGRAM TOTALS
1,903,161  glibc-2.16-75f0d304/elf/dl-lookup.c:do_lookup_x
1,701,781  rpmio/macro.c:rpmLoadMacroFile
1,491,535  rpmio/macro.c:doDefine
  932,161  glibc-2.16-75f0d304/malloc/malloc.c:_int_malloc
  900,004  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strcmp.S:__GI_strcmp
  520,864  glibc-2.16-75f0d304/malloc/malloc.c:_int_free
  482,756  glibc-2.16-75f0d304/elf/dl-lookup.c:_dl_lookup_symbol_x
  397,574  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strcmp.S:strcmp'2
  307,322  glibc-2.16-75f0d304/elf/dl-misc.c:_dl_name_match_p
  306,247  rpm-4.10.2/lib/rpmrc.c:doReadRC
  286,734  glibc-2.16-75f0d304/string/../sysdeps/x86_64/memcpy.S:__GI_memcpy
  254,657  rpm-4.10.2/lib/rpmrc.c:doReadRC'2
  240,179  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strlen.S:__GI_strlen
...
Optimize argvCount function

Callgrind annotations for 'rpmspec -P anaconda.spec', previous commit:
380,151,860  PROGRAM TOTALS
232,943,797  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strlen.S:__GI_strlen
 63,770,642  rpmio/argv.c:argvCount
 13,702,038  build/parseSpec.c:readLine
...

Callgrind annotations for 'rpmspec -P anaconda.spec', this commit:
364,226,458  PROGRAM TOTALS
232,943,797  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strlen.S:__GI_strlen
 47,845,240  rpmio/argv.c:argvCount
 13,702,038  build/parseSpec.c:readLine
...
Reimplemented argvJoin function

Joining lines with consecutive rstrscat() calls was bad idea, because,
to append n-th argument, rstrscat() first has to calculate the length
of its previous result, that is, in a sense, the length of all its (n-1)
arguments.  Thus the complexity was O(n^2), due to arithmetic progression.

This change introduces a highly optimized version of argvJoin.  In
particular, strlen() call for each argv element is issued only once.
Also, the inner loop is tuned for the special case of empty or
single-character separator, to avoid extra memcpy per-element call.

Callgrind annotations for 'rpmspec -P anaconda.spec', previous commit:
363,974,889  PROGRAM TOTALS
232,675,214  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strlen.S:__GI_strlen
 47,845,240  rpmio/argv.c:argvCount
 13,702,038  build/parseSpec.c:readLine
...

Callgrind annotations for 'rpmspec -P anaconda.spec', this commit:
130,717,478  PROGRAM TOTALS
47,862,184  rpmio/argv.c:argvCount
13,702,038  build/parseSpec.c:readLine
...
Optimize expandMacro function

This change brings the following optimizations:
- In the beginning of the routine, the input string needs to be copied
  only in non-terminated case.
- When mb->buf is NULL, expensive xcalloc call is replaced with xmalloc.
  It suffices to ensure that mb->buf is (initially) null-terminated at
  mb->tpos, and that mbAppend* routines follow this rule.
- The inner per-character loop is replaced with call to strchr(s, '%').
  Also note that the "trailing %" case is no longer handled by this part
  of the code (this is now the case of no valid macro name).
- New routine mbAppendMem now complements mbAppendStr.  Also, mbGrow
  routine was factored to improve possibilities for inlining.

Callgrind annotations for 'rpmspec -P selinux-policy.spec', previous commit:
160,979,831  PROGRAM TOTALS
58,348,925  rpmio/argv.c:argvCount
14,378,832  build/parseSpec.c:readLine
10,710,181  glibc-2.16-75f0d304/string/../sysdeps/x86_64/memset.S:__GI_memset
 5,445,134  rpmio/macro.c:mbAppend
 5,147,651  glibc-2.16-75f0d304/malloc/malloc.c:_int_malloc
 3,490,055  rpm-4.10.2/lib/header.c:copyData
...

Callgrind annotations for 'rpmspec -P selinux-policy.spec', this commit:
140,610,489  PROGRAM TOTALS
58,348,925  rpmio/argv.c:argvCount
14,378,832  build/parseSpec.c:readLine
 3,893,942  glibc-2.16-75f0d304/malloc/malloc.c:_int_malloc
 3,490,055  rpm-4.10.2/lib/header.c:copyData
...
Optimize readLine routine

This change introduces the following optimizations:
- Line chunking used to be done with per-character loop; it now uses
  strchr(line, '\n') call.
- Likewise, the "balancing check" has been enhanced with strpbrk(3).

Callgrind annotations for 'rpmspec -P glibc.spec', previous commit:
94,796,525  PROGRAM TOTALS
33,242,383  rpmio/argv.c:argvCount
 9,886,925  build/parseSpec.c:readLine
 3,964,158  glibc-2.16-75f0d304/malloc/malloc.c:_int_malloc
...

Callgrind annotations for 'rpmspec -P glibc.spec', this commit:
88,013,018  PROGRAM TOTALS
33,242,383  rpmio/argv.c:argvCount
 3,964,158  glibc-2.16-75f0d304/malloc/malloc.c:_int_malloc
...

Update: on the second thought, it occurred to me that two more
optimizations related to the "balancing check" can be implemented:
- Note that the check actually does two different things: it tracks
  line concatenation with trailing backslash; and it further checks
  for balanced brackets and parens, but only within a macro substitution.
  Unless a macro has been started, the check for brackets/parens can be
  skipped.  Likewise, unless a trailing backslash has been seen, there
  is no need to search for a newline.  Thus, strpbrk() can be first
  called with a reduced set of characters: only "\\" and "%" have to be
  tracked down.  When a trailing backslash is found, or when a macro
  substitution is started which needs a closing bracket/paren, the set
  of character is changed to a full set of "\\\n%{}()".
- The balancing check can also be used to find out whether the
  rpmExpandMacros() call is needed at all. Note that, on average, only
  about 30% of specfile lines have "%" character in them (among the
  remaining 70% are empty lines with trailing "\n", which nontheless
  still have to be processed).  Thus most of the rpmExpandMacros() calls
  and the associated buffer allocation/copying can be skipped altogether.

Callgrind annotations for 'rpmspec -P glibc.spec', updated commit:
80,831,100  PROGRAM TOTALS
33,244,684  rpmio/argv.c:argvCount
 3,244,420  glibc-2.16-75f0d304/string/../sysdeps/x86_64/strcmp.S:__GI_strncmp
...
Fix horrible bug in freeing macro arguments

After a macro which takes options/arguments is called, such as %systemd_post,
freeArgs() frees ALL macros *previously defined in specfile*.  For example,
in amanda.spec, after scriptlet sections which invoke %systemd_* macros are
processed, %amanda_user and %amanda_group in %files sections become undefined,
resulting in malformed owner/group file metadata.

$ rpm -qp --qf '[%{FILEUSERNAME}:%{FILEGROUPNAME}\n]' amanda-3.3.2-2.fc18.x86_64.rpm |
> sort | uniq -c | sort -n | tail
      3 %amanda_user:%amanda_group
    104 root:root

This change installs additional name check in freeArgs(), so that, upon
finishing macro call, only macro arguments are freed.