django/django — pull request #2485 by mjtamlyn (2014-03-26 17:20:22+00:00)
Title: django.contrib.postgres.fields.ArrayField - WIP

This is a first draft of Array fields. The basic field definition is there, with the required functionality to handle arrays of almost any type. I've also written the lookups/transforms specific to array fields.

Work still to do:
- [ ] Docs
- [ ] Form fields (naive and admin) and data cleaning
- [ ] Handling dimensions

The last of these is a particularly interesting case. Postgres has a "casual relationship" with the definition of an array field. You can create `integer[]`, `integer[][]`, `integer[3][4]` etc, but [postgres docs](http://www.postgresql.org/docs/9.3/static/arrays.html#ARRAYS-DECLARATION) state that this is basically just documentation as it is not enforced at all. We have a couple of options here:

- Force single dimensional, unbounded arrays always. This would be pretty boring.
- Allow `max_size=4` and do python side only validation. We'd still pass the correct `[4]` to postgres, but it won't enforce integrity.
- Allow a complex `dimensions` flag to be passed allowing for any option. I think this isn't needed as if you want a 2-dimensional array you could do `ArrayField(ArrayField(IntegerField()))`. This also makes the code path much easier as all the functions which delegate to the `base_field` don't have to worry about its dimensions.

In the absence of strong opinion otherwise, I'm going to do option 2.

Other notes for reviewers:
- Related fields are banned. For M2M this is quite obviously necessary, but I've done so for FKs as well as they currently do not support referential integrity, which is what Django FKs try to enforce. Otherwise just use an integer.
- Postgres uses 1-based indexing, but I'm converting this in the lookups from 0-based indexing. If someone is used to writing a lot of raw pg queries directly, this will be confusing, but to a normal python user we expect 0-based indexing everywhere.
- At present I have not implemented `contained_by`, which is `contains` with the arguments reversed. It's basically a "is subset" operator. Thinking about it as I'm writing this, I think it does have use cases so I should add it in.
- String based lookups (`__iexact`, `startswith` etc) continue to be accepted, even though they are largely useless. `contains` has been overloaded with a more sensible implementation. This is on the principle that date based fields accept them, and the query is functional (casts everything to text). Personally, I would like fields to only support the lookups which make sense on them now that is easily done, but this is a backwards incompatible change. I may open it up as a ticket when working on refactoring `__year` etc into transforms.
- The approach for handling test models is copied from gis. As Anssi said on IRC, it might be nice if runtests didn't need to know about this, but it'll do for now.
- There's a bit of hackery with the `deconstruct` method which means the `__init__` accepts two formats for the base field. I wonder whether this could be avoided if there is a suitable hook in `migrations.writer` to allow me to pass a string containing the correct field definition for the `base_field` from `deconstruct`. This would make the migration files look less weird. @andrewgodwin is this sensible? Also should I have explicit tests that migrations work, and if so what would that look like?