Move generalized arithmetical primitives to 'generalize-arithm' template
(code refactoring)

* src/Makefile.am (EXTRA_DIST): Add generalize-arithm.template.
* src/Makefile.am (nobase_private_HEADERS): Add generalize-arithm.h.
* src/Makefile.am (atomic_ops/generalize-arithm.h): New rule (similar
as generalize-small.h but using generalize-arithm.template).
* src/atomic_ops/generalize-arithm.template: New file.
* src/atomic_ops/generalize-small.template: Move compare_and_swap
(only which is based on fetch_compare_and_swap), fetch_and_add,
fetch_and_add, fetch_and_add1, fetch_and_sub1, and/or/xor template
primitives to generalize-arithm.template).
* src/atomic_ops/generalize-arithm.h: Regenerate (new file).
* src/atomic_ops/generalize-small.h: Regenerate.
* src/atomic_ops/generalize.h: Include generalize-arithm.h (just after
inclusion of generalize-small.h).
Enable generalization of all variants of CAS via fetch_compare_and_swap

* src/atomic_ops/generalize.h (AO_compare_and_swap_full,
AO_compare_and_swap_acquire, AO_compare_and_swap_release,
AO_compare_and_swap, AO_TS_COMPARE_AND_SWAP_FULL,
AO_TS_COMPARE_AND_SWAP_ACQUIRE, AO_TS_COMPARE_AND_SWAP_RELEASE,
AO_TS_COMPARE_AND_SWAP): Define (in this file) only if required for
AO_test_and_set_X generalization.
* src/atomic_ops/generalize-arithm.template
(AO_XSIZE_compare_and_swap_write, AO_XSIZE_compare_and_swap_read,
AO_XSIZE_compare_and_swap_release_write,
AO_XSIZE_compare_and_swap_acquire_read,
AO_XSIZE_compare_and_swap_dd_acquire_read): Add missing variant of
generalized template CAS primitives based fetch_compare_and_swap; add
the corresponding AO_HAVE_XSIZE_compare_and_swap_X macros.
* src/atomic_ops/generalize-arithm.h: Regenerate.
Add AO_load primitives generalization based on CAS

* src/atomic_ops/generalize-small.template (AO_XSIZE_load_acquire,
AO_XSIZE_load, AO_XSIZE_load_read, AO_XSIZE_load_acquire_read,
AO_XSIZE_load_full, AO_XSIZE_load_dd_acquire_read,
AO_XSIZE_store_release, AO_XSIZE_store, AO_XSIZE_store_write,
AO_XSIZE_store_release_write, AO_XSIZE_store_full): Move primitive
down to be after CAS defintion section.
* src/atomic_ops/generalize-small.template (AO_XSIZE_load_read,
AO_XSIZE_load_full, AO_XSIZE_load_acquire, AO_XSIZE_load): Add
generalization template primitive based on CAS.
* src/atomic_ops/generalize-small.h: Regenerate.
Fix template-based headers regeneration order in src/Makefile

* src/Makefile.am (BUILT_SOURCES): Define to force regeneration
of generalize-arithm.h, generalize-small.h (if the corresponding
source template is changed) before compilation of .c files (that use
atomic_ops.h).
Group all X_acquire_release_volatile.h and X_[aligned_]atomic_load_store.h
in separate folder

* src/Makefile.am (nobase_private_HEADERS): Update path to
acquire_release_volatile.h, aligned_atomic_load_store.h,
atomic_load_store.h, char_acquire_release_volatile.h,
char_atomic_load_store.h, int_acquire_release_volatile.h,
int_aligned_atomic_load_store.h, int_atomic_load_store.h,
short_acquire_release_volatile.h, short_aligned_atomic_load_store.h,
short_atomic_load_store.h entries (add "loadstore" to prefix).
* src/atomic_ops/sysdeps/all_acquire_release_volatile.h: Likewise.
* src/atomic_ops/sysdeps/all_aligned_atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/all_atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/gcc/alpha.h: Likewise.
* src/atomic_ops/sysdeps/gcc/mips.h: Likewise.
* src/atomic_ops/sysdeps/hpc/hppa.h: Likewise.
* src/atomic_ops/sysdeps/acquire_release_volatile.h: Move file to
src/atomic_ops/sysdeps/loadstore folder.
* src/atomic_ops/sysdeps/loadstore/aligned_atomic_load_store.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/loadstore/char_acquire_release_volatile.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/char_atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/loadstore/int_acquire_release_volatile.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/int_aligned_atomic_load_store.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/int_atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/loadstore/short_acquire_release_volatile.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/short_aligned_atomic_load_store.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/short_atomic_load_store.h:
Likewise.
* src/atomic_ops/sysdeps/all_acquire_release_volatile.h: Reformat
comment.
* src/atomic_ops/sysdeps/all_aligned_atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/all_atomic_load_store.h: Likewise.
Generate 'loadstore' headers from templates

* src/Makefile.am (EXTRA_DIST): Add acquire_release_volatile.template,
aligned_atomic_load_store.template, atomic_load_store.template
entires.
* src/Makefile.am (BUILT_SOURCES): Add acquire_release_volatile.h,
aligned_atomic_load_store.h, atomic_load_store.h,
char_acquire_release_volatile.h, char_atomic_load_store.h,
int_acquire_release_volatile.h, int_aligned_atomic_load_store.h,
int_atomic_load_store.h, short_acquire_release_volatile.h,
short_aligned_atomic_load_store.h, short_atomic_load_store.h entires.
* src/Makefile.am
(atomic_ops/sysdeps/loadstore/acquire_release_volatile.h,
atomic_ops/sysdeps/loadstore/char_acquire_release_volatile.h,
atomic_ops/sysdeps/loadstore/int_acquire_release_volatile.h,
atomic_ops/sysdeps/loadstore/short_acquire_release_volatile.h,
atomic_ops/sysdeps/loadstore/aligned_atomic_load_store.h,
atomic_ops/sysdeps/loadstore/int_aligned_atomic_load_store.h,
atomic_ops/sysdeps/loadstore/short_aligned_atomic_load_store.h,
atomic_ops/sysdeps/loadstore/atomic_load_store.h,
atomic_ops/sysdeps/loadstore/char_atomic_load_store.h,
atomic_ops/sysdeps/loadstore/int_atomic_load_store.h,
atomic_ops/sysdeps/loadstore/short_atomic_load_store.h): New rules (to
generate header from the corresponding template).
* src/atomic_ops/sysdeps/loadstore/acquire_release_volatile.template:
New template header (code copied from the corresponding header of
"loadstore" folder and parameterized with XSIZE and XCTYPE); update
header comment.
* src/atomic_ops/sysdeps/loadstore/aligned_atomic_load_store.template:
Likewise.
* src/atomic_ops/sysdeps/loadstore/atomic_load_store.template:
Likewise.
* src/atomic_ops/sysdeps/loadstore/acquire_release_volatile.template
(AO_GCC_BARRIER): Add TODO item.
* src/atomic_ops/sysdeps/loadstore/acquire_release_volatile.h:
Regenerate.
* src/atomic_ops/sysdeps/loadstore/char_acquire_release_volatile.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/int_acquire_release_volatile.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/short_acquire_release_volatile.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/aligned_atomic_load_store.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/int_aligned_atomic_load_store.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/short_aligned_atomic_load_store.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/loadstore/char_atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/loadstore/int_atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/loadstore/short_atomic_load_store.h:
Likewise.
Add missing 'const' in aligned-atomic AO_XSIZE_load implementation

* src/atomic_ops/sysdeps/loadstore/aligned_atomic_load_store.template
(AO_XSIZE_load): Add missing 'const' (when casting away volatile).
* src/atomic_ops/sysdeps/loadstore/aligned_atomic_load_store.h:
Regenerate.
* src/atomic_ops/sysdeps/loadstore/int_aligned_atomic_load_store.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/short_aligned_atomic_load_store.h:
Likewise.
Remove X_aligned_atomic_load_store headers and template
(code refactoring)

* src/Makefile.am (EXTRA_DIST): Remove
aligned_atomic_load_store.template entry.
* src/Makefile.am (BUILT_SOURCES, nobase_private_HEADERS): Remove
aligned_atomic_load_store.h, int_aligned_atomic_load_store.h,
short_aligned_atomic_load_store.h entries.
* src/Makefile.am
(atomic_ops/sysdeps/loadstore/aligned_atomic_load_store.h,
atomic_ops/sysdeps/loadstore/int_aligned_atomic_load_store.h,
atomic_ops/sysdeps/loadstore/short_aligned_atomic_load_store.h):
Remove rules.
* src/atomic_ops/sysdeps/all_aligned_atomic_load_store.h: Refine
comment; include X_atomic_load_store.h instead of the corresponding
X_aligned_atomic_load_store.h file.
* src/atomic_ops/sysdeps/all_aligned_atomic_load_store.h
(AO_ACCESS_CHECK_ALIGNED, AO_ACCESS_short_CHECK_ALIGNED,
AO_ACCESS_int_CHECK_ALIGNED): New macros (defined before inclusion
of the corresponding X_atomic_load_store.h file).
* src/atomic_ops/sysdeps/loadstore/aligned_atomic_load_store.h:
Remove.
* src/atomic_ops/sysdeps/loadstore/aligned_atomic_load_store.template:
Likewise.
* src/atomic_ops/sysdeps/loadstore/int_aligned_atomic_load_store.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/short_aligned_atomic_load_store.h:
Likewise.
* src/atomic_ops/sysdeps/loadstore/atomic_load_store.template
(AO_XSIZE_load, AO_XSIZE_store): Add conditional assertion check of
(*addr) alignment (enabled only if AO_ACCESS_<XSIZE>_CHECK_ALIGNED).
* src/atomic_ops/sysdeps/loadstore/short_atomic_load_store.h:
Regenerate.
* src/atomic_ops/sysdeps/loadstore/int_atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/loadstore/char_atomic_load_store.h: Likewise.
* src/atomic_ops/sysdeps/loadstore/atomic_load_store.h: Likewise.
Do not duplicate BUILT_SOURCES entries in nobase_private_HEADERS (Makefile)

* src/Makefile.am (nobase_private_HEADERS): Add "$(BUILT_SOURCES)"
instead of specifying generalize-arithm.h, generalize-small.h,
acquire_release_volatile.h, atomic_load_store.h,
char_acquire_release_volatile.h, char_atomic_load_store.h,
int_acquire_release_volatile.h, int_atomic_load_store.h,
short_acquire_release_volatile.h, short_atomic_load_store.h entries
directly.
Eliminate redundant underscore symbols in sed arguments in Makefile
(revert commit 75b9e67 partially)

* src/Makefile.am (atomic_ops/generalize-small.h,
atomic_ops/generalize-arithm.h,
atomic_ops/sysdeps/loadstore/char_acquire_release_volatile.h,
atomic_ops/sysdeps/loadstore/int_acquire_release_volatile.h,
atomic_ops/sysdeps/loadstore/short_acquire_release_volatile.h,
atomic_ops/sysdeps/loadstore/char_atomic_load_store.h,
atomic_ops/sysdeps/loadstore/int_atomic_load_store.h,
atomic_ops/sysdeps/loadstore/short_atomic_load_store.h): Remove
redundant '_' (underscore) symbol in sed "search" and "replace-to"
strings.
Add generic implementation of AO_double_load primitives

* src/Makefile.am (nobase_private_HEADERS): Add
double_atomic_load_store.h entry.
* src/Makefile.am (atomic_ops/generalize-small.h):
Generate primitives for AO_double_t as well (from the template).
* src/atomic_ops/generalize-small.h: Regenerate.
* src/atomic_ops/sysdeps/generic_pthread.h (AO_double_load_full): New
primitive.
* src/atomic_ops/sysdeps/loadstore/double_atomic_load_store.h: New
header (most code copied from atomic_load_store template).
* tests/test_atomic_include.template (test_atomicXX): Add test of
AO_double_loadXX.
Implement AO_double_load for gcc/arm

* src/atomic_ops/sysdeps/armcc/arm_v6.h (AO_load_ex): Add 'const'
qualifier to argument pointer type.
* src/atomic_ops/sysdeps/gcc/arm.h (AO_fetch_and_add,
AO_fetch_and_add1, AO_fetch_and_sub1, AO_compare_and_swap): Remove
useless comment.
* src/atomic_ops/sysdeps/gcc/arm.h (AO_double_load): Implement (only
for ARMv7+).
* src/atomic_ops/sysdeps/armcc/arm_v6.h (AO_double_load): Likewise.
Remove redundant 'cc' clobber for LDREXD instruction (gcc/arm)
(revert commit 07e9ea9 partially)

* src/atomic_ops/sysdeps/gcc/arm.h (AO_fetch_compare_and_swap): Remove
redundant "cc" clobber for assembly code that consists of a single
LDREXD instruction.
Implement AO_double_load/store based on guaranteed x86 access atomicity

* src/atomic_ops/sysdeps/gcc/x86.h: Define
AO_ACCESS_double_CHECK_ALIGNED and include double_atomic_load_store.h
after inclusion of standard_ao_double_t.h (only if AO_t is 32-bit).
* src/atomic_ops/sysdeps/msftc/x86.h: Likewise.
* src/atomic_ops/sysdeps/sunc/x86.h: Likewise.
Fix missing compiler barrier in AO_nop_full for uniprocessor ARM

* src/atomic_ops/sysdeps/armcc/arm_v6.h (AO_nop_full): Insert
AO_compiler_barrier for AO_UNIPROCESSOR case.
* src/atomic_ops/sysdeps/gcc/arm.h (AO_nop_full): Likewise.
* src/atomic_ops/sysdeps/msftc/arm.h (AO_nop_full): Likewise.
* src/atomic_ops/sysdeps/msftc/arm.h (AO_nop_full): Refine comment.
Fix missing abort() usage in atomic_ops_malloc and tests on WinCE

* src/atomic_ops_malloc.c (abort): Define to _exit(-1) if _WIN32_WCE
or __MINGW32CE__ (since there is no abort() in WinCE).
* tests/run_parallel.h (abort): Likewise.
* tests/test_stack.c (abort): Likewise.
Implement AO_nop_full/write using 'dmb' instruction if available (gcc/arm)

* src/atomic_ops/sysdeps/gcc/arm.h (AO_ARM_HAVE_DMB): New macro
(defined for ARMv6M and ARMv7+).
* src/atomic_ops/sysdeps/gcc/arm.h (AO_nop_full): Update comment;
use "dmb" instruction if AO_ARM_HAVE_DMB (and not AO_UNIPROCESSOR).
* src/atomic_ops/sysdeps/gcc/arm.h (AO_nop_write): Implement using
"dmb st" instruction and define AO_HAVE_nop_write (only if
AO_ARM_HAVE_DMB and not AO_UNIPROCESSOR).
Fix return type of AO_char/short/int_load_read() in read_ordered.h

* src/atomic_ops/sysdeps/read_ordered.h (AO_char_load_read,
AO_short_load_read, AO_int_load_read): Change return type (and the
type of "result" local variable) from AO_t to the type matching the
name of the function (i.e., unsigned char/short/int, respectively).
Move definitions of ordered loads/stores to inner separate headers
(code refactoring)

* src/Makefile.am (nobase_private_HEADERS): Add ordered_loads_only.h,
ordered_stores_only.h entries.
* src/atomic_ops/sysdeps/loadstore/ordered_loads_only.h: New file.
* src/atomic_ops/sysdeps/loadstore/ordered_stores_only.h: Likewise.
* src/atomic_ops/sysdeps/ordered_except_wr.h: Include
ordered_stores_only.h.
* src/atomic_ops/sysdeps/ordered_except_wr.h (AO_store_write,
AO_store_release, AO_char_store_write, AO_char_store_release,
AO_short_store_write, AO_short_store_release, AO_int_store_write,
AO_int_store_release): Move to ordered_stores_only.h (with the
corresponding AO_HAVE_x macros).
* src/atomic_ops/sysdeps/read_ordered.h: Include ordered_loads_only.h.
* src/atomic_ops/sysdeps/read_ordered.h (AO_load_read,
AO_load_acquire, AO_char_load_read, AO_char_load_acquire,
AO_short_load_read, AO_short_load_acquire, AO_int_load_read,
AO_int_load_acquire): Move to ordered_loads_only.h (with the
corresponding AO_HAVE_x macros).
Generate ordered_loads/stores_only headers from templates
(code refactoring)

* src/Makefile.am (EXTRA_DIST): Add ordered_loads_only.template,
ordered_stores_only.template entries.
* src/Makefile.am (BUILT_SOURCES): Add ordered_loads_only.h,
ordered_stores_only.h entries (move from nobase_private_HEADERS).
* src/Makefile.am (atomic_ops/sysdeps/loadstore/ordered_loads_only.h,
atomic_ops/sysdeps/loadstore/ordered_stores_only.h): New rule (to
generate header from the corresponding template).
* src/atomic_ops/sysdeps/loadstore/ordered_loads_only.h: Regenerate
(from the corresponding template).
* src/atomic_ops/sysdeps/loadstore/ordered_stores_only.h: Likewise.
* src/atomic_ops/sysdeps/loadstore/ordered_loads_only.template: New
template header (code copied from the corresponding header of
"loadstore" folder and parameterized with XSIZE and XCTYPE).
* src/atomic_ops/sysdeps/loadstore/ordered_stores_only.template:
Likewise.