artemp/trac2github — Issue #795: Revisit NULL and Boolean types handling
Author: artemp
Created: 2011-10-09 20:51:24+00:00
State: open
Labels: Core_Library

Over time mapnik value_type has gained the ability to store null types and boolean types, but full support appears unfinished at least in the postgis plugin (https://github.com/mapbox/carto/issues/75).

Across datasources that support nulls and booleans we should be able to do:

Match nulls:
{{{
[field] is null
}}}

Maybe: Match nulls and other types that evaluated to false
{{{
not [field]
}}}

Match non nulls:
{{{
not [field] is null
# and after #796:
[field] is not null
}}}

Note: this should likely not result in hits because in postgres you have to use `is` to trigger null matching (but this is an acceptable regression for now I feel)
{{{
[field] != null
# and
[field] = null
}}}

Match booleans:
{{{
[field] = true
# or
[field] != false
or
[field] != true
# or
[field] = false
}}}

* Initial NULL support landed in r657. This started skipping features with null values and broke osm.xml usage at the time (http://lists.berlios.de/pipermail/mapnik-devel/2008-February/000500.html)

* Fixing rendering of zeros (#89) lead to a refactor of null support in r693, but postgis driver was not updated to stop skipping null value attributes even though `mapnik::value_null` was added.

* And filter support for NULL's was never finished: #318

* Boolean support first landed in r660. However we still lack Boolean support for attribute_descriptors for datasources (#795), and featureset support in datasources that support boolean types (postgres).

Overall issues:

1) how to allow proper filtering on boolean and null types consistently across all datasource plugins

2) whether to make optimizations like not storing features attributes with null values

3) mapnik does not currently offer much support for writing data, but if/when it does we need to insure the proper formatting and maintenance of types from feature output (e.g avoiding collapsing boolean types to integer of 0/1 in feature attribute display).

4) conversion of booleans and null to proper types in bindings (e.g. NULL to python None - r3019)

Actionable:

* NEEDS REVIEW - Datasource plugins need to start using `mapnik::value_null` (currently nothing uses it)

* RESOLVED - Python bindings need to support conversion of NULL to None (r3019), and bool to bool (r3022)

* RESOLVED - SQLite driver overall needs to store mapnik::value_null instead of writing empty strings.

* RESOLVED - PostGIS featureset.cpp needs to stop ignoring null values

* RESOLVED - PostGIS datasource.cpp needs to report booleans as booleans not integers

* RESOLVED - Need to support reporting attributes of Boolean type (#795)