Repository: lisa-lab/pylearn2

Issue:
test_video.test_spatiotemporal_cubes is randomly failing
The test `utils.tests.test_video.test_spatiotemporal_cubes` seems to be randomly failing on some Travis builds. Both @jych and @daemonmaker have experienced this. From [comments in the code](https://github.com/lisa-lab/pylearn2/blob/master/pylearn2/utils/tests/test_video.py#L27-L28) it seems like this is not the first time. Was a reason ever discovered?

test_video.test_spatiotemporal_cubes is randomly failing
The test `utils.tests.test_video.test_spatiotemporal_cubes` seems to be randomly failing on some Travis builds. Both @jych and @daemonmaker have experienced this. From [comments in the code](https://github.com/lisa-lab/pylearn2/blob/master/pylearn2/utils/tests/test_video.py#L27-L28) it seems like this is not the first time. Was a reason ever discovered?



Discussion:
One of the reasons was non-determinism in the code itself, which used sets and dicts, which resulted in different random patches being drawn each time. So, from time to time, the coverage was not complete. I fixed that specific instance, so I reactivated the test in #1162, but apparently there is another reason for failures, or another commit re-activated that.
Is `six.iteritems` on an `OrderedDict` deterministic?

`six.iteritems` would indeed be deterministic.
However, I just realized that in https://github.com/lisa-lab/pylearn2/blob/master/pylearn2/utils/video.py#L121 we create an RNG with a deterministic seed, but we are not using it, and use [`numpy.random`](https://github.com/lisa-lab/pylearn2/blob/master/pylearn2/utils/video.py#L123) instead!

Fixed in #1269 

This issue should be reopend. Using the following bash script, I found that It still fails randomly.

``` bash
for i in {1..100}
do
    theano-nose pylearn2/utils/tests/test_video.py:test_spatiotemporal_cubes
done
```

One of the reasons was non-determinism in the code itself, which used sets and dicts, which resulted in different random patches being drawn each time. So, from time to time, the coverage was not complete. I fixed that specific instance, so I reactivated the test in #1162, but apparently there is another reason for failures, or another commit re-activated that.
Is `six.iteritems` on an `OrderedDict` deterministic?

`six.iteritems` would indeed be deterministic.
However, I just realized that in https://github.com/lisa-lab/pylearn2/blob/master/pylearn2/utils/video.py#L121 we create an RNG with a deterministic seed, but we are not using it, and use [`numpy.random`](https://github.com/lisa-lab/pylearn2/blob/master/pylearn2/utils/video.py#L123) instead!

Fixed in #1269 

This issue should be reopend. Using the following bash script, I found that It still fails randomly.

``` bash
for i in {1..100}
do
    theano-nose pylearn2/utils/tests/test_video.py:test_spatiotemporal_cubes
done
```


Resolving patch:
diff --git a/pylearn2/utils/video.py b/pylearn2/utils/video.py
--- a/pylearn2/utils/video.py
+++ b/pylearn2/utils/video.py
@@ -131,8 +131,8 @@ def spatiotemporal_cubes(file_tuples, shape, n_patches=numpy.inf, rng=None):
         # fall within frame.
         last_row = video_height - patch_height
         last_col = video_width - patch_width
-        row = numpy.random.random_integers(0, last_row)
-        col = numpy.random.random_integers(0, last_col)
+        row = rng.random_integers(0, last_row)
+        col = rng.random_integers(0, last_col)
         patch_slice = (slice(frame_no, frame_no + patch_length),
                        slice(row, row + patch_height),
                        slice(col, col + patch_width))


Test patch: