ioam/topographica — pull request #567 by jlstevens (2014-07-02 15:30:34+00:00)
Title: Independent weight initialization with time_dependent=True

**Summary**

This commit allows ``time_dependent=True`` to be enabled throughout Topographica, allowing reproducible, well controlled ``numbergenNumberGenerator``, ``imagen.PatternGenerator`` and ``TransferFn`` objects to be used everywhere.

This is particularly important for reproducible random number streams and will enable parallel weight generation when initializing a model (as random patterns are no longer affected by their order of instantiation).

This PR is only for Topographica with the other necessary changes already pushed to master of the relevant submodules (namely ``numbergen`` and ``imagen``).

The two main changes in this PR are:

* Pattern generation in the constructor of ``ConnectionField``.

* A helper function in ``topo/__init__.py`` to toggle time-dependent everywhere.

**Design Overview**

Objects with state that should depend on some notion of time (simulation time, real time whatever) should inherit from [``numbergen.TimeAware``](https://github.com/ioam/param/blob/45963dad1bb429b7681ec0a01fa5e59523c27971/numbergen/__init__.py#L21-68) or [``numbergen.TimeDependent``](https://github.com/ioam/param/blob/45963dad1bb429b7681ec0a01fa5e59523c27971/numbergen/__init__.py#L70-86). The essential difference between these classes is that the latter is always ``time_dependent`` whereas the former can be toggled (see their respective docstrings).

Objects that inherit from these classes will have access to ``time_fn`` which reports the time value. By default, the ``time_fn`` is a global time object that is an instance of [``param.Time``](https://github.com/ioam/param/blob/45963dad1bb429b7681ec0a01fa5e59523c27971/param/__init__.py#L86-291) which provides facilities for navigating the timeline. If you want a global notion of time, it is necessary that all objects that use time reference the same object.

**Controlling random state**

This depends on time-dependent hashing using [``numbergen.Hash``](https://github.com/ioam/param/blob/45963dad1bb429b7681ec0a01fa5e59523c27971/numbergen/__init__.py#L179-244) whereby the hash is taken of the object name (which sets the random 'identity' of the object), the value returned by the ``time_fn`` and the global ``param.seed``. This hash is then used to seed the random state per call.

I found this functionality was being repeated in [``numbergen.RandomDistribution``](https://github.com/ioam/param/blob/45963dad1bb429b7681ec0a01fa5e59523c27971/numbergen/__init__.py#L350-396), [``imagen.random.RandomGenerator``](https://github.com/ioam/imagen/blob/ad72c3096449167e22bf25710ac6363bbcc50179/imagen/random.py#L26-91) and [``topo.transferfn.TransferFnWithRandomState``](https://github.com/ioam/topographica/blob/47463e971c97417018e85740b4f848a6051129b5/topo/transferfn/__init__.py#L303-340) which is why I've introduced a new abstract class to numbergen,[``numbergen.TimeAwareRandomness``](https://github.com/ioam/param/blob/45963dad1bb429b7681ec0a01fa5e59523c27971/numbergen/__init__.py#L247-346) which is used by all three of these classes.

**Weight generation**

Assuming an time-dependent imagen pattern is passed into a ``connect`` call, this PR simply uses the ``param.Time`` context manager to temporarily set the global time to zero and then sets a coordinate-dependent name for each pattern used to generate the weights of each CF. For a given value of ``param.random_seed`` and CF positions, each weight pattern is therefore completely independent and constrained (which should eventually allow for parallel initialization).

**Outstanding Issues**

* Our old models require ``time_dependent=False`` if their results are to remain unchanged. I propose adding setting ``time_dependent=True`` which will generate many warnings in older models (because it is unlikely that the name parameter will have been set for numbergen instance etc). Then by calling ``topo.time_dependent(False)`` at the start of the ty file, the old behaviour would be enabled without warnings.

* Currently the ``scriptrepr`` test is failing due to my recent commits (mentioned below) with the following traceback:

```
File "/tmp/tmpzABXUa/script_repr_test.ty", line 38
random_generator=,
^
SyntaxError: invalid syntax
```

This is unsurprising as random state has no succinct ``repr`` (how is this handled currently?). This problem is actually solved with ``time_dependent=True`` as any random state will do (as it is tightly controlled by the hash seed). This means if the ``repr`` of the ``random_generator`` parameter value is simply ``random.Random()`` or ``np.random.RandomState()`` (as appropriate) the subsequent streams will still be correct. We still need to get the ``scriptrepr`` test passing again (and I'm not quite sure how to fix it).

**Notes**

* Objects with random state need an identity, provided by the name parameter. If you want two objects to generate the same random stream, you simply need to give those two objects the same name. This does mean there lots of warnings will be generated if the name is not set (as the default name depends on the order of instantiation which is not well controlled). In general, if we switch to ``time_dependent=True``, we will have to start giving our objects names.

**Relevant commits**

These commits have already been pushed and test. All tests pass for each respective repository except for the ``scriptrepr`` problem (described above) in Topographica.

Param - ioam/param@45963dad1
Imagen - ioam/imagen@ad72c309
Topographica - ioam/topographica@47463e971c9