ToDoMe/ToDoMe-V2.0 — Issue #3: Data packages
Author: Fodaro
Created: 2012-01-27 15:28:39+00:00
State: open

To reduce server load, we need a data packaging system, where the database is divided into packages.

Packages
--------
Each package will contain all of the data for a certain geographic area. These areas will be defined either by grid references (e.g. a 1° by 1° 'square') or by administrative boundaries (e.g. county). I am in favour of the first option as this is far easier to implement and organise. The size of these areas will have to be chosen carefully as they will influence the size of the packages.

Preferably, the data will be compressed.

The Server
----------
The server will store all of the packages that may be needed by the clients, and will deliver those packages on request. Assuming that we use a grid reference system, the packages will be organised by grid reference.

The server need not be very complex for this implementation. It could easily be an FTP or HTTP server.

The Local Cache
---------------
When the client first enters a geographic area, it will download the whole package for that area into a cache, preferably on an SD card, and extract the data from the compressed package. If the size of this cache exceeds a certain value, the oldest data (by when it was used?) will be deleted.

If the client is near the boundary of its area, the other nearby packages should probably be downloaded too.

User configuration (or not)
---------------------------
All of this working should be invisible to the average user. The maximum cache size should be chosen automatically (possibly as a percentage of the available storage space), and the scheduling of package downloads may require some intelligence (to avoid high data charges, for example). These options should probably still be user-configurable, but be hidden away in an 'Advanced Settings' screen.