astropy/astropy — pull request #305 by astrofrog (2012-06-27 14:55:00+00:00)
Title: Initial implementation of Table I/O infrastructure

This is an initial implementation of a system to register readers/writers with Table so as to be able to do:

t = Table.read('some_file.fits')
t.write('some_file.vot')

as in ATpy.

The aim of this is to not have any actual I/O code in ``astropy.table`` but instead to allow other components of Astropy, other packages, or the users themselves, to register readers and/or writers.

Then ``astropy.io.fits``, ``astropy.io.wcs``, and ``astropy.io.ascii`` could, if they wish, register readers and writers with ``astropy.table`` (but this is not included in this PR). If you look at the initial documentation, you'll see a simplistic example to see how it works.

One of the nice features is the ability to specify identifying functions which can be used for example to look at filename extensions, but could also look at the first few bytes of a file, etc. It's as generic as possible (unlike ATpy, which made some pretty stringent assumptions) and the whole system isn't limited to files, but can include web resources, databases, etc.

If we go ahead with this, I'll complete the documentation (to talk about writers) and will write tests.

There are some remaining things to consider that @taldcroft and I talked a bit about off-list:

* We may want to have some kind of guidelines as to how much guessing readers/writers can do - for example if multiple tables are present in a file (note that as in the simplistic example, readers/writers can support options to make this unambiguous).

* Not all formats will support all numpy types that ``Table`` supports, especially objects columns.

* Not all formats can store the same kind of meta-data, and there is currently no standard for how to even store simple things like keyword/value pairs in the ``Table`` meta-data object.

While the code in this PR should not be concerned about the latter two issues, which are up to the readers/writers, we may want to work on some guidelines, at least for the internal readers/writers, to ensure that as much data and meta-data can be converted between formats as possible.

I'm cc-ing @ebressert since he will probably not see this PR otherwise.