[Touch Diamond, MDA Compact IV General] low.flying.pigs:
Separated from post 2293 for clarity and ease of future reference.

There are no known openly accessible sources of compatible, bulk, up to date tiles in OSM tile format, intended for public usage, rendered from raw data, not ripped from Tile Servers by abusing MOBAC or similar tools. Ironically, this is mainly due to the prevalence and pervasiveness of MOBAC and similar tools that historically have scraped pre rendered tiles from Tile Servers.

Furthermore, the nature of the OSM tile format is that it is efficient in limited use, but not in bulk use. This because it was not intended for bulk offline mapping. It was intended for online Tile Server based mapping for applications utilising a 'slippy map', where only the tiles as required for display on screen at any given time are requested from and provided by the server on demand, and that server subjected to the intended usage would never receive request or supply in excess of that.

Whilst many applications support OSM tile format, for the efficiency and additional functionality benefits, most utilise alternative formats for bulk offline mapping.

Options previously detailed are summarised below, with additional detail where applicable:

1. Use the features already in GPS Cycle Computer:

1a. Download within GPS Cycle Computer. For users that have no requirement of bulk mapping, or can slowly accumulate over time, briefly, copy osm_servers.txt attached to post 2281 to the GPS Cycle Computer installation directory, having backed up any existing file and map folders if desired or required. As most devices support ActiveSync connection and thereby allow download of mapping within GPS Cycle Computer directly on to device / memory card for later off line use, when connected via ActiveSync to an Internet connected computer, mapping of the area at the zoom levels actually required is then quickly obtained by simply scrolling around the map screen. Select 'OpenStreetMap' for 'Mapnik' style mapping.

Note this 'pre-downloading' or 'pre-caching' cannot be done with Cycle Map, Outdoor Map, Landscape Map or Public Transport Map, as it is expressly prohibited. See here. These can only be downloaded in actual usage, for example whilst riding, over a data connection.

Abuse of the above will result in Tile Servers blocking individuals or GPS Cycle Computer itself, affecting all users. Act responsibly.

1b. Scan from paper based mapping. Refer to Readme.

2. Purchase commercial mapping. An abundance of suppliers exist that will provide OpenStreetMap based mapping rendered in any required style, custom or otherwise, in an array of formats, including OSM Tile format directly compatible already with GPS Cycle Computer.

3. Render your own mapping tiles from raw data. As detailed in post 2289, there are a number of rendering tools, some of which are detailed below. Further information regarding rendering will follow in a separate post in due course.

3a. Maperitive is currently one of the easier to use (that does not mean it is necessarily easy, it means most ordinary users will find the other currently readily available rendering software even harder, or produces incompatible output, requiring conversion (if even readily possible), which creates further difficulties). It is a standalone application that produces output in OSM Tile format directly compatible already with GPS Cycle Computer, and does not need to be used in conjunction with anything else.

3b. TileMill also produces OSM Tile format directly compatible already with GPS Cycle Computer, but is harder to use than Maperitive.

3c. MOBAC, it has recently been announced, will, in a forthcoming version, add 'integrated mapsforge renderer' functionality. It can be foreseen, if not immediately, then eventually, functionality will allow raw OpenStreetMap data to be converted to be converted to Mapsforge .map format and output in that format. To provide output in other formats, the .map data would be rendered using Mapsforge (which can only render .map data) to create a 'mapsource', which is then output in any other supported format. See here.

4. Wait for OpenStreetMap and other service providers to implement workable secure and sustainable free access to their Tile Servers, and compliant non abusive tools to be developed for accumulating pre rendered tiles from them for offline usage. Effort has been made in previous posts to demonstrate free access can remain when implemented securely and sustainably, as it should have been from the outset.

Maperitive, in the interim, provides Tile Server tile downloading in OSM Tile format directly compatible already with GPS Cycle Computer, from a range of Tile Servers, including those from which previous and current versions of MOBAC and similar tools are blocked, in a manner that attempts to manage, control, and minimise demand placed on the Tile Server. Act responsibly.

5. Wait for GPS Cycle Computer to integrate support for OpenStreetMap and other mapping legitimately obtained and usable and or freely available, where the technical details of the file format, are generally, in large part, known, and have been determined via legitimate research and analysis:

5a. Memory-Map™ .qct format Ordnance Survey mapping detailed in post 2155. An example of an application that already integrates the format is here. Memory-Map™ 2800 and 3500 *

5b. Garmin .img format. As detailed in post 2281. See here, here, and here. Mapping is readily available, for example see here. An example of an application that already integrates the format is here.

5c. Magellan .rmp map format. See here, here and here. Magellan eXplorist 310, eXplorist 610 and possibly others * Magellan GIS eXplorist Pro 10 **

5d. Mapsforge .map format. See here and here. Mapping is readily available, for examples see here, here and here. It is widely used, for example see here. Viewer software is available for Microsoft Windows, see here. Raw OpenStreetMap data can be converted to this format. See here. MOBAC, in a forthcoming version, may support the format. See 3c

This is a vector format, the mapping displayed on screen is rendered on device. To enable this, it may be possible to use the Open Source Java Library as is or with modification in conjunction with a Windows Mobile Java Virtual Machine, Java ME (formerly J2ME), see here, here and here. Alternatively, it may be possible to convert the library to C#.

5e. SQLite database based formats. See here, here and here. Some include OSMAND SQLite, Osmdroid SQLite, MBTiles (can be rendered by Maperitive), and RMaps SQLite.

Bryton Rider 50 and 60 maps are SQLite based. *

Medion S3857 and S3867 maps are SQLite based. *

The map format of the following devices may be similar and are currently being investigated. Notes about this to be added when available:

A-Rival Teasi One and Teasi Pro *

Blaupunkt BikePilot *

Falk LUX, IBEX and PANTERA *

Mio Cyclo™ 300, 305, 305 HC, 310, 315, 315 HC, 500, 505, 505 HC *

* Devices are Windows CE based / ** Devices are Windows Mobile 6.5 based: It is therefore at least a technical possibility that GPS Cycle Computer could run on Windows CE based devices, as it does on Medion S3857 and S3867, and relatively certain it would run on Windows Mobile based devices. It would therefore be advantageous if the mapping already on device could be used with GPS Cycle Computer. Further information regarding devices will follow in a separate post in due course.

5x. There may be other formats.

6. Wait for GPS Cycle Computer to offer dedicated Tile Server for online mapping use. This may be somewhat superfluous if 7 is implemented.

7. Wait for GPS Cycle Computer to offer dedicated Pre Prepared Downloads for offline mapping use. There are a multitude of issues with this and 6. If 4 or 5 are implemented, necessity for either 6 and 7 decreases, but both would still be useful, not least to reduce dependence on 4 and 5.

Regarding 7: When GPS Cycle Computer supports efficient alternative mapping formats, proof of concept maps for these will be created. As GPS Cycle Computer currently supports OSM tiles, a proof of concept map has been rendered in this format using Maperitive as per 3a.

The Sub Region chosen was: Europe / Germany / Nordrhein-Westfalen / Regierungsbezirk Köln. See here.

Levels 7 to 15 are complete comprising 67,424 tiles in 689 folders, totalling size 626MB, generated from raw data at the time totalling 96.4MB. Actual disk space required varies dependent on file system. On a default FAT32 formatted Micro SD memory card, size on disk is 1.41GB.

Screen grabs:

1 to 3 ............. Tiles being rendered in zoom levels 7 to 15 from bulk raw data.

4 to 11 ........... Rendered tiles in usage in GPS Cycle Computer 4.12beta22.

12 ................... Tiles being rendered in zoom levels 16 to 19 from raw data of smaller area to reduce time.

13 to 16 ......... Rendered tiles in usage in GPS Cycle Computer 4.12beta22.

17 ................... Default OpenStreetMap Standard 'Mapnik' style map and tiles for comparison. See here.

Level 16 was being rendered in full and would have had in total 201,600 tiles. It was over 60% complete. At 10% complete it was working at approximately 1.4 tiles per second. At 30% complete it had slowed to slowed to approximately 1 tile per second where at that speed it would take in total approximately 56 hours (over 2 days) to complete. At 50% complete it has slowed to 0.7 tiles per second. At 60% complete it had slowed to 0.5 tiles per second.

Unfortunately the process failed, having created 123,508 tiles in 672 folders, totalling size 823MB. Failure I believe was due to my own error. However is was becoming progressively slower and slower before my error. Unfortunately again, there does not appear any way to resume from that point. The level would have to be started from the beginning again.

Each level has approximately 4 times the number of tiles of the previous level, and rendering times increase accordingly. Level 17 would approximately have 806,400 tiles taking 224 hours (over 9 days) at 1 tile per second, level 18 would approximately have 3,225,600 tiles taking 896 hours (over 37 days) at 1 tile per second, and level 19 would approximately have 12,902,400 tiles taking 3,584 hours (over 149 days) at 1 tile per second.

Therefore levels 16 to 19 were rendered from raw data of a smaller area, shown in screen grab 12, to save time, comprising 16,664 tiles in 218 folders totalling 194MB in size. Level 19 is the OpenStreetMap maximum, and the maximum of Maperitive.

Rendering times vary dependent on size of area being rendered, data therein, rendering rules, output format, computer software optimisation and computer hardware resources available. It may be possible to improve the efficiency of the rendering. Testing was carried out on entirely unoptimised equipment, as part of the test, to simulate that which an ordinary user might have.

RAM is more important than processing speed. Relatively low memory and low processing speed machines can render relatively quickly if the amount of raw data and levels are in proportion.

When an area, and / or size of raw data, and / or levels being rendered is within the capability of the hardware, the process is relatively fast. However, due to the ever increasing number of tiles, the process, dependent upon hardware capability, slows as levels increase. When the limit of hardware capability is reached, the process continues, but at a slow rate. Due to the number of tiles, this slow rate can produce inordinate durations to completion.

Alternative methods of rendering may increase the capability of the hardware. For example, command line tools, that do not require Windows to be running, immediate free the resources on the machine Windows would otherwise have consumed.

Before making any changes, the same bulk raw data was being rendered in MBTiles format using Maperitive, to enable fair comparison.

The results were comparable. To save time, as before, a smaller area was instead rendered. It could be observed that as MBTiles is a single file, that file size, or more specifically, size on disk is reduced. Furthermore, unnecessary tiles are eliminated. For instance, if a tile is a single colour, a river, or forest for example, and zoom levels above that tile would be more tiles of a single colour, these superfluous tiles and not stored. Dependent on the mapping being rendered, the size differences can be substantial.

For the purposes of distribution, and offline bulk mapping, MBTiles therefore has the aforementioned advantages over OSM tiles format.

Users that do not need, require, want, or will not use proposed functionality will not use it and are therefore not adversely affected, as existing functionality will not be adversely affected.

Recent posts in brief give an overview of some of the expanded capabilities in areas related to mapping further to earlier posts.

All trademarks and registered trademarks are the property of their respective owners.

low.flying.pigs