ckan/ckan — pull request #1967 by aliceh75 (2014-10-09 12:02:02+00:00)
Title: unicode_or_json_validator should assume literals are unicode rather than json

As per #1966:

When using the recline grid, attempting a full text search with an integer causes the grid to hang forever.

This is what happens:

1. The full text search is sent to the datastore API using the `q` parameter, which is set to (say) `1234`;
2. `q` is validated using `ckanext.datastore.logic.schema.unicode_or_json_validator`;
3. In turns this calls `json_validator` which does the validation by attempting to parse the string as json. This uses `json.loads`. This implementation considers literals to be valid json [*], so `u'1234'` becomes `int(1234)`
4. Later in the process, `ckanext.datastore.plugin.DatastorePlugin.datastore_validate`, checks whether `q` is a `dict` or a `string`. It doesn't allow other types - so the search fails on validation.

The fact that `unicode_or_json_validator` accepts literals has other implications - for instance a double quoted string `"hello"` would be accepted, and returned without the quotes. So I suggest the best option is to change `unicode_or_json_validator` to not accept JSON literals.

I will create a PR for this.

[*] [RFC 4267](http://www.ietf.org/rfc/rfc4627.txt) says that a JSON document should contain an array or a string, however the [ECMA](http://www.ecma-international.org/publications/standards/Ecma-404.htm) variation of the standard does not impose this restriction. In practice most implementations accept litterals.