astropy/astropy — pull request #3090 by astrofrog (2014-11-07 23:54:47+00:00)
Title: WIP: Phase 1 of improvements to I/O infrastructure

(I had already started working on improving the I/O infrastructure in https://github.com/astropy/astropy/pull/2063 but decided to tackle it in smaller pieces)

This PR implements delayed loading of read/write/identifier functions by allowing strings to be registered instead of functions, and when the functions are needed they are loaded. This should speed up the import of astropy.table by not importing all the rest of astropy when Table is needed.

Now this won't work as-is, and I was hoping to start a conversation about it. Specifically, ``astropy.io.ascii`` auto-registers many ASCII formats via a meta-class. At the moment, I have therefore not implemented lazy loading of these because this would require hard-coding all the formats in ``registry.py`` instead of using the meta-class. Instead I just import ``io.ascii`` inside the ``_register_builtins`` function. This works if the first thing I do is import ``Table`` then call ``read`` - it will auto-register all the formats. However, this causes an issue when doing

```
python -c 'import astropy.io.ascii'
```

because then it calls the meta-classes, registers readers/writers, but during the registration of the registration, ``_update__doc__`` gets called, and then ``_register_builtins`` gets called for the first time and tries to import ``io.ascii``, raising an error because ``io.ascii`` isn't loaded yet.

So one option would be to try and make it that the documentation for ``Table.read`` and ``Table.write`` gets built only at the very last minute. One way to do that would be to check if ``read`` or ``write`` are accessed on the e.g. table class and if so to then generate the docstring on-the-fly. It might make more sense to do this because otherwise the docstring gets re-generated every time a format is added, rather than just when it's needed. But I'm not sure if that's the simplest?

@taldcroft - it would be great if you could share your thoughts on this briefly

@embray - maybe you also have some thoughts on this?

Note that in Phase 2, I plan to work on the idea that was suggesting before of making use of I/O classes instead of functions, but wanted to do things one step at a time.