[osm2pgsql] Allow osm2pgsql to be compiled on mac osx

As reported in #4544 osm2pgsql was not portable to mac osx.

Add additional configure checks for functions that are not available
on mac osx and either disable the functionality or replace it through defines



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28646 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
Keep tags that match --hstore-columns. Fixes #4547.

git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28671 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Fix previous commit that caused a segmentation fault with --hstore



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28683 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
Fix a logic bug in r28671 that made the return value of pgsql_filter_tags() incorrect when --hstore-column was set.

git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28686 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
Fix a bug that caused --hstore-match-only to be ignored when --hstore-column is set.

git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28687 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
Extend regression tests to compare results between runs.

git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28688 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Remove debug check from binarysearcharray 
 
The binary search array had a sanity check in the add function 
to check that the ordering invariant holds true. This turned addition 
into a O(n) operation rather than an O(log(n)) 
 
Remove this sanity check as a performance optimisation



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28700 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Replace async I/O in flat-node-cache with posix_fadvise 
 
The idea behind using async I/O was to try and have more than a single I/O operation 
in flight, which can greatly increase performance of raid arrays or SSDs but also has some 
advantages on normal disks. 
 
The glib aio functions however did not work correctly, and were unnecessarily complex. 
 
This patch now replaces all of the async code with a simple call to posix_fadvise(WILLNEED) 
for all of the nodes in a get_nodes_list call. 
 
On my 32Gb SSD, the use of posix_fadvise improves performance of node retrieval from 
3.5k nodes/s to 22k nodes/s under ideal benchmark conditions. In real world usage, 
the speedup is likely to be a lot less, as less I/O can be predicted (most get_node_list calls 
only need to load a single 4k block) and thus less I/O can be parallelised.



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28701 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Improve cache management on initial writeout of flat-node-cache 
 
1) fsync the flat-node-cache file in regular intervals to make sure not too 
   much dirty data is in the disk cache which can then stall the whole system 
   during write out. However, instead of fsync, it uses the Linux specific function 
   sync_file_range 
 
2) After writing out the data, drop the information from the disk cache. 
   If one has enough RAM, then one should rather set the -C parameter high enough 
   which is its own ram cache for nodes. It is then a waste of 16GB RAM 
   to also cache the flat-node-cache in disk cache. 
   If one doesn't have enough cache, then chances are that the wrong 
   data is in the disk cache by the time one needs it again, so it is 
   probably also better to drop it from disk cache



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28702 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Enable 64bit IDs by default

Databases generated in 64bit and 32bit ID mode are not compatible with each other,
including applying diffs.

It is therefore necessary to reload the database to switch from 32bit ID mode to 64bit.

Also bump version number to 0.81.1



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28712 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Add an --exclude-invalid-polygon option 
 
Polygons that don't pass the ST_IsValid test are excluded with this switch



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28719 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
gazetteer: add aerialway (closes #4499), 
 move place to extratags only when boudary tag is there (fixes #4558),
 move name:prefix to extratags


git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28746 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Implement a ways_get_list function to retrieve multiple ways at once. 
 
For the postgresql back-end this issues all way requests of a relation 
in a single sql call and then matches the result back to the list of 
ways in osm2pgsql. 
 
This saves round trip time / context switches to postgresql and potentially 
allows postgresql to be more efficient in retrieving the data. 
 
On tests with small extracts, this nearly doubled performance of the 
relation processing stage and showed increases of up to 25% for the complete 
import. On larger imports this speedup is likely smaller, but hopefully 
still noticeable. 
 
This function currently does not yet combine loading of nodes from 
multiple ways in ways_get_list, which might in future provide additional 
speed up.



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28747 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
gazetteer: ignore buildings without name or address


git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28748 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
use way_get_list function in gazetteer


git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28761 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
+ add office=*


git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28788 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Change the default node ram cache allocation strategy on 32bit platforms to sparse

Chunked and optimized both cause problems on platforms with limited virtual memory (i.e. 32bit platforms),
so change the default to sparse, which doesn't have as much virtual memory overhead for accounting and indexing.



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28789 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Move code back to using ANSI C standard

In an attempt to get osm2pgsql compiled for windows, move
back to using ANSI C standard, as msvc doesn't support C99.

Also clean up some compiler warnings.



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28862 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Fix bug #4651 - Failed fall back for when fork doesn't succeed

In multi-process mode, if the fork did not succeed "Iterate over pending ways"
and "Iterate over pending relations" produced the wrong data, as the fall back
mechanism didn't correctly work.

The information of the reduced number of successful helper processes was only
conveyed to the first process, and therefore all other helper processes used
the wrong stride length to go through the pending ways/relations array. Therefore
many ways/relations were not correctly processed.

Using a mmaped structure, make sure all helper processes know how many processes
correctly started for processing.



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28864 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4
[osm2pgsql] Fix Bug #3607 - Function index is not available in MinGW

The function index() is deprecated in the posix standard and superseded
by the strchr function. As index is not available in MinGW, replace
it with strchr to make osm2pgsql compile.



git-svn-id: http://svn.openstreetmap.org/applications/utils/export/osm2pgsql@28865 b9d5c4c9-76e1-0310-9c85-f3177eceb1e4