Repository: mila-udem/fuel
Pull request #271 (2015-12-03 16:10:39)

Added Random2DRotation transformer and corresponding test.
I haven't looked into Cython implementations for 2D image rotation yet, so this implementation is still (very) slow. Furthermore, I currently use `scipy.ndimage.interpolation.rotate` for rotation, which (as far as I know) can only rotate per-image. As such, `transform_source_batch` simply calls `transform_source_example` for every image in the batch.

Timeit example in which we rotate 3x256x256 images: 
```python
>>> import timeit                                                                                                                                                
>>> setup = """                                                                     
... from collections import OrderedDict                                             
... import numpy                                                                    
... from fuel.datasets.base import IndexableDataset                                 
... from fuel.schemes import SequentialScheme, SequentialExampleScheme              
... from fuel.streams import DataStream                                             
... from fuel.transformers.image import Random2DRotation                            
...                                                                              
... batch = numpy.zeros((100, 3, 256, 256), dtype=int)                              
... img = numpy.arange(3*256*256, dtype=int).reshape(3, 256, 256)                   
... batch[:] = img                                                                  
...                                                                                 
... data = OrderedDict([('image', batch)])                                          
... dataset = IndexableDataset(                                                     
...     data, axis_labels={'image': ('batch', 'channel', 'height', 'width')})       
... stream1 = DataStream(dataset, iteration_scheme=SequentialExampleScheme(1))      
... stream1 = Random2DRotation(stream1, which_sources=('image',))                   
... stream2 = DataStream(dataset, iteration_scheme=SequentialScheme(100, 100))      
... stream2 = Random2DRotation(stream2, which_sources=('image',))                   
... """                                                                             
>>> main_statement = """                                                            
... stream1.transform_source_example(img, source_name='image') """      
>>> # example by example                                                                       
>>> t = timeit.Timer(main_statement, setup)                                         
>>> print t.repeat(5, 1000)                                                         
[38.28747606277466, 38.34248900413513, 38.31103992462158, 38.42443799972534, 38.43785381317139]                                                                                 
>>> # equivalent, but per batch                                                                       
>>> main_statement = """                                                            
... stream2.transform_source_batch(batch, source_name='image') """
>>> t = timeit.Timer(main_statement, setup)                                         
>>> print t.repeat(5, 10)                  
[37.95794892311096, 37.86270499229431, 37.8510000705719, 37.841578006744385, 37.87645411491394]
```

Lastly, any pointers on output correctness checks are greatly appreciated! I haven't seen many tests for which the expected output is a hard-coded array, so if this is regarded as undesirable or just plain ugly, I'm more than willing to change it to something more readable.  

I realize that in its current state, this transformer is probably not ready to be merged just yet. Still, random rotation is (in my opinion) a useful transformer for data augmentation, so I hope this PR generates some discussion on its implementation. 

vdumoulin: @aukejw Very sorry for the delay! I'll try to have a look at that by the end of the week.
bartvm: Overall the code looks good to me. I don't see an issue with the hardcoded arrays; I can't think of another good way to test things.

The current code loops over the channels in order to rotate each one independently, you don't need to do that. The `rotate` command takes an `axes` argument (which defaults to `(1, 0)`) which you can just set to `(2, 1)` instead in order to rotate the entire array in one go (it's not any faster though).

I also benchmarked Pillow's rotate method, and that looks promising (and it's already a dependency). It would be nice if the resampling method could be an argument to the transformer so that the user can choose the quality/speed trade-off.

```ipython
In [118]: import numpy

In [119]: import scipy

In [120]: from PIL import Image

In [121]: image = numpy.random.randint(0, 256, size=(3, 256, 256)).astype('uint8')

In [122]: %timeit scipy.ndimage.interpolation.rotate(image, numpy.pi / 2, reshape=False, axes=(2, 1))
10 loops, best of 3: 23.1 ms per loop

In [123]: %timeit numpy.array(Image.fromarray(image.transpose((1, 2, 0))).rotate(45, resample=Image.BICUBIC))
100 loops, best of 3: 5.88 ms per loop

In [124]: %timeit numpy.array(Image.fromarray(image.transpose((1, 2, 0))).rotate(45))
1000 loops, best of 3: 1.26 ms per loop
```
dwf: API adaptation trick: you can use "gettattr(Image, resample.upper())" in
some places in order to support using a nicer string argument of "bicubic",
"nearest", "linear", etc. rather than have the user know the byzantine
module-level constants.

>

aukejw: @bartvm : Thanks for the feedback. You're right, looping over channels is unnecessary. If Image.rotate turns out to be the better option, I'll replace the scipy rotate call, and add the resampling method as argument. 

@dwf Agreed. 
aukejw: I took inspiration for the `resample` argument from the `MinimumImageDimensions` transformer constructor -- although this transformer seems [not to use the argument](https://github.com/mila-udem/fuel/blob/master/fuel/transformers/image.py#L169) (?)

Repeat of the timeits shows that the speedup when using `Image.rotate` is considerable:
```
>>> # example by example                                                                       
>>> t = timeit.Timer(main_statement, setup)                                         
>>> print t.repeat(5, 1000)
[27.805537939071655, 28.955289125442505, 29.08978796005249, 29.47601008415222, 29.370649099349976]
>>> # equivalent, but per batch                                                                       
>>> main_statement = """                                                            
... stream2.transform_source_batch(batch, source_name='image') """
>>> t = timeit.Timer(main_statement, setup)                                         
>>> print t.repeat(5, 10)
[28.193249940872192, 27.960711002349854, 27.922738075256348, 28.211444854736328, 28.142256021499634]
```

I could make the transformer a bit more flexible by adding an auxiliary function (like in `MinimumImageDimensions`) that performs the rotation, rather than having `transform_source_example` do the work. 
bartvm: One of those Travis errors should be fixed by https://github.com/mila-udem/fuel/pull/286 (it's a mysterious non-deterministic error), the other one is new but my guess is that it's just a Travis hiccup. I'll try clearing the Travis cache otherwise.

Seems like you're right, the `resample` argument somehow got looked over in #196. (@dwf)

Having a separate method do the rotation would be slightly cleaner perhaps, but I think it would be fine to merge either way.
aukejw: Actually, `tranform_source_example` triggered unnecessary `verify_axis_labels` checks, so the auxiliary. function is IMO necessary. This also allows me to generate the rotation angles for all items in the batch simultaneously. 

If you want me to squash some commits, let me know.
bartvm: Squashing everything would be great. Otherwise LGTM.

Patch:
diff --git a/fuel/converters/svhn.py b/fuel/converters/svhn.py
index 9e7169d..e0eaeb4 100644
--- a/fuel/converters/svhn.py
+++ b/fuel/converters/svhn.py
@@ -236,9 +236,7 @@ def convert_svhn_format_1(directory, output_directory,
                 h5file['features'].dims[0]['shapes'][index] = image.shape
                 for field in BoundingBoxes._fields:
                     name = 'bbox_{}'.format(field)
-                    h5file[name][index] = numpy.maximum(0,
-                                                        getattr(bounding_boxes,
-                                                                field))
+                    h5file[name][index] = getattr(bounding_boxes, field)
                     h5file[name].dims[0]['shapes'][index] = [num_boxes, 1]
 
                 # Replace label '10' with '0'.
diff --git a/fuel/datasets/base.py b/fuel/datasets/base.py
index c598b66..e85a14b 100644
--- a/fuel/datasets/base.py
+++ b/fuel/datasets/base.py
@@ -40,14 +40,6 @@ class Dataset(object):
     example_iteration_scheme : :class:`.IterationScheme` or ``None``
         The iteration scheme the class uses in order to produce a stream of
         examples.
-    default_transformers: It is expected to be a tuple with one element per
-        transformer in the pipeline. Each element is a tuple with three
-        elements:
-            - the Transformer subclass to apply,
-            - a list of arguments to pass to the subclass constructor, and
-            - a dict of keyword arguments to pass to the subclass
-              constructor.
-
 
     Notes
     -----
diff --git a/fuel/datasets/hdf5.py b/fuel/datasets/hdf5.py
index 4ce64af..9ac7167 100644
--- a/fuel/datasets/hdf5.py
+++ b/fuel/datasets/hdf5.py
@@ -1,4 +1,3 @@
-import numbers
 from itertools import product
 from collections import defaultdict
 
@@ -10,7 +9,6 @@ from six.moves import zip, range
 
 from fuel.datasets import Dataset
 from fuel.utils import do_not_pickle_attributes, Subset
-from fuel.schemes import SequentialExampleScheme
 
 
 @do_not_pickle_attributes('nodes', 'h5file')
@@ -190,12 +188,6 @@ class H5PYDataset(Dataset):
         kwargs.setdefault('axis_labels', self.default_axis_labels)
         super(H5PYDataset, self).__init__(**kwargs)
 
-        # It is really important to do it here, because self.num_examples
-        # call will cause a crash if done before calling
-        # super(...).__init__
-        self.example_iteration_scheme = SequentialExampleScheme(
-            self.num_examples)
-
     def _parse_dataset_info(self):
         """Parses information related to the HDF5 interface.
 
@@ -542,11 +534,8 @@ class H5PYDataset(Dataset):
             data, shapes = self._out_of_memory_get_data(state, request)
         for i in range(len(data)):
             if shapes[i] is not None:
-                if isinstance(request, numbers.Integral):
-                    data[i] = data[i].reshape(shapes[i])
-                else:
-                    for j in range(len(data[i])):
-                        data[i][j] = data[i][j].reshape(shapes[i][j])
+                for j in range(len(data[i])):
+                    data[i][j] = data[i][j].reshape(shapes[i][j])
         return tuple(data)
 
     def _in_memory_get_data(self, state=None, request=None):
@@ -560,7 +549,7 @@ class H5PYDataset(Dataset):
         return data, shapes
 
     def _out_of_memory_get_data(self, state=None, request=None):
-        if not isinstance(request, (numbers.Integral, slice, list)):
+        if not isinstance(request, (slice, list)):
             raise ValueError()
         data = []
         shapes = []
diff --git a/fuel/datasets/mnist.py b/fuel/datasets/mnist.py
index 38e31e4..9898f4b 100644
--- a/fuel/datasets/mnist.py
+++ b/fuel/datasets/mnist.py
@@ -19,7 +19,7 @@ class MNIST(H5PYDataset):
     ----------
     which_sets : tuple of str
         Which split to load. Valid values are 'train' and 'test',
-        corresponding to the training set (60,000 examples) and the test
+        corresponding to the training set (50,000 examples) and the test
         set (10,000 examples).
 
     """
diff --git a/fuel/datasets/toy.py b/fuel/datasets/toy.py
index 34e90b4..e09aaa2 100644
--- a/fuel/datasets/toy.py
+++ b/fuel/datasets/toy.py
@@ -4,7 +4,6 @@ import numpy
 
 from collections import OrderedDict
 
-from fuel import config
 from fuel.datasets import IndexableDataset
 
 
@@ -51,19 +50,18 @@ class Spiral(IndexableDataset):
     """
     def __init__(self, num_examples=1000, classes=1, cycles=1., noise=0.0,
                  **kwargs):
-        seed = kwargs.pop('seed', config.default_seed)
-        rng = numpy.random.RandomState(seed)
         # Create dataset
-        pos = rng.uniform(size=num_examples, low=0, high=cycles)
-        label = rng.randint(size=num_examples, low=0, high=classes)
-        radius = (2 * pos + 1) / 3.
+        pos = numpy.random.uniform(size=num_examples, low=0, high=cycles)
+        label = numpy.random.randint(size=num_examples, low=0, high=classes)
+
+        radius = (2*pos+1) / 3.
         phase_offset = label * (2*numpy.pi) / classes
 
         features = numpy.zeros(shape=(num_examples, 2), dtype='float32')
 
         features[:, 0] = radius * numpy.sin(2*numpy.pi*pos + phase_offset)
         features[:, 1] = radius * numpy.cos(2*numpy.pi*pos + phase_offset)
-        features += noise * rng.normal(size=(num_examples, 2))
+        features += noise * numpy.random.normal(size=(num_examples, 2))
 
         data = OrderedDict([
             ('features', features),
@@ -115,21 +113,20 @@ class SwissRoll(IndexableDataset):
     """
     def __init__(self, num_examples=1000, noise=0.0, **kwargs):
         cycles = 1.5
-        seed = kwargs.pop('seed', config.default_seed)
-        rng = numpy.random.RandomState(seed)
-        pos = rng.uniform(size=num_examples, low=0, high=1)
+
+        pos = numpy.random.uniform(size=num_examples, low=0, high=1)
         phi = cycles * numpy.pi * (1 + 2*pos)
-        radius = (1 + 2 * pos) / 3
+        radius = (1+2*pos) / 3
 
         x = radius * numpy.cos(phi)
         y = radius * numpy.sin(phi)
-        z = rng.uniform(size=num_examples, low=-1, high=1)
+        z = numpy.random.uniform(size=num_examples, low=-1, high=1)
 
         features = numpy.zeros(shape=(num_examples, 3), dtype='float32')
         features[:, 0] = x
         features[:, 1] = y
         features[:, 2] = z
-        features += noise * rng.normal(size=(num_examples, 3))
+        features += noise * numpy.random.normal(size=(num_examples, 3))
 
         position = numpy.zeros(shape=(num_examples, 2), dtype='float32')
         position[:, 0] = pos
diff --git a/fuel/server.py b/fuel/server.py
index 4c01d51..0ab36b2 100644
--- a/fuel/server.py
+++ b/fuel/server.py
@@ -7,6 +7,7 @@ from numpy.lib.format import header_data_from_array_1_0
 from fuel.utils import buffer_
 
 logger = logging.getLogger(__name__)
+logging.basicConfig(level='INFO')
 
 
 def send_arrays(socket, arrays, stop=False):
@@ -108,8 +109,6 @@ def start_server(data_stream, port=5557, hwm=10):
         receiving end as well.
 
     """
-    logging.basicConfig(level='INFO')
-
     context = zmq.Context()
     socket = context.socket(zmq.PUSH)
     socket.set_hwm(hwm)
diff --git a/fuel/transformers/image.py b/fuel/transformers/image.py
index 3a523ca..f79c8d0 100644
--- a/fuel/transformers/image.py
+++ b/fuel/transformers/image.py
@@ -270,3 +270,100 @@ class RandomFixedSizeCrop(SourcewiseTransformer, ExpectsAxisLabels):
             off_w = 0
         return example[:, off_h:off_h + windowed_height,
                        off_w:off_w + windowed_width]
+
+
+class Random2DRotation(SourcewiseTransformer, ExpectsAxisLabels):
+    """Randomly rotate 2D images in the spatial plane.
+
+    Parameters
+    ----------
+    data_stream : :class:`AbstractDataStream`
+        The data stream to wrap.
+    maximum_rotation : float, default `math.pi`
+        Maximum amount of rotation in radians. The image will be rotated by
+        an angle in the range [-maximum_rotation, maximum_rotation].
+    resample : str, optional
+        Resampling filter for PIL to use to upsample any images requiring
+        it. Options include 'nearest' (default), 'bilinear', and 'bicubic'.
+        See the PIL documentation for more detailed information.
+
+    Notes
+    -----
+    This transformer expects to act on stream sources which provide one of
+
+     * Single images represented as 3-dimensional ndarrays, with layout
+       `(channel, height, width)`.
+     * Batches of images represented as lists of 3-dimensional ndarrays,
+       possibly of different shapes (i.e. images of differing
+       heights/widths).
+     * Batches of images represented as 4-dimensional ndarrays, with
+       layout `(batch, channel, height, width)`.
+
+    The format of the stream will be un-altered, i.e. if lists are
+    yielded by `data_stream` then lists will be yielded by this
+    transformer.
+
+    """
+    def __init__(self, data_stream, maximum_rotation=math.pi,
+                 resample='nearest', **kwargs):
+        if maximum_rotation <= 0 or maximum_rotation > math.pi:
+            raise ValueError('maximum_rotation ({:.5f}) must be in the range '
+                             '(0, math.pi]'.format(maximum_rotation))
+        self.maximum_rotation = numpy.rad2deg(maximum_rotation)
+        try:
+            self.resample = getattr(Image, resample.upper())
+        except AttributeError:
+            raise ValueError("unknown resampling filter '{}'".format(resample))
+
+        self.rng = kwargs.pop('rng', None)
+        self.warned_axis_labels = False
+        if self.rng is None:
+            self.rng = numpy.random.RandomState(config.default_seed)
+        kwargs.setdefault('produces_examples', data_stream.produces_examples)
+        kwargs.setdefault('axis_labels', data_stream.axis_labels)
+        super(Random2DRotation, self).__init__(data_stream, **kwargs)
+
+    def transform_source_batch(self, source, source_name):
+        self.verify_axis_labels(('batch', 'channel', 'height', 'width'),
+                                self.data_stream.axis_labels[source_name],
+                                source_name)
+        rotation_angles = self.rng.uniform(-self.maximum_rotation,
+                                           self.maximum_rotation,
+                                           len(source))
+        if isinstance(source, list) and all(isinstance(b, numpy.ndarray) and
+                                            b.ndim == 3 for b in source):
+            return [self._example_transform(im, angle)
+                    for im, angle in zip(source, rotation_angles)]
+        elif isinstance(source, numpy.ndarray) and source.dtype == object and \
+                all(isinstance(b, numpy.ndarray) and b.ndim == 3 for b in
+                    source):
+            out = numpy.empty(len(source), dtype=object)
+            for im_idx, (im, angle) in enumerate(zip(source, rotation_angles)):
+                out[im_idx] = self._example_transform(im, angle)
+            return out
+        elif isinstance(source, numpy.ndarray) and source.ndim == 4:
+            return numpy.array([self._example_transform(im, angle)
+                                for im, angle in zip(source, rotation_angles)],
+                               dtype=source.dtype)
+        else:
+            raise ValueError("uninterpretable batch format; expected a list "
+                             "of arrays with ndim = 3, or an array with "
+                             "ndim = 4")
+
+    def transform_source_example(self, example, source_name):
+        self.verify_axis_labels(('channel', 'height', 'width'),
+                                self.data_stream.axis_labels[source_name],
+                                source_name)
+        if not isinstance(example, numpy.ndarray) or example.ndim != 3:
+            raise ValueError("uninterpretable example format; expected "
+                             "ndarray with ndim = 3")
+        rotation_angle = self.rng.uniform(-self.maximum_rotation,
+                                          self.maximum_rotation)
+        return self._example_transform(example, rotation_angle)
+
+    def _example_transform(self, example, rotation_angle):
+        dt = example.dtype
+        im = Image.fromarray(example.transpose(1, 2, 0))
+        example = numpy.array(im.rotate(rotation_angle,
+                                        resample=self.resample)).astype(dt)
+        return example.transpose(2, 0, 1)
diff --git a/fuel/utils/__init__.py b/fuel/utils/__init__.py
index 887b619..a1607c0 100644
--- a/fuel/utils/__init__.py
+++ b/fuel/utils/__init__.py
@@ -1,6 +1,5 @@
 import collections
 import os
-import numbers
 
 import h5py
 import six
@@ -236,12 +235,12 @@ class Subset(object):
         """
         # Translate the request within the context of this subset to a
         # request to the indexable object
-        if isinstance(subset_request, numbers.Integral):
+        if isinstance(subset_request, int):
             request, = self[[subset_request]]
         else:
             request = self[subset_request]
         # Integer or slice requests can be processed directly.
-        if isinstance(request, numbers.Integral) or hasattr(request, 'step'):
+        if isinstance(request, int) or hasattr(request, 'step'):
             return indexable[request]
         # If requested, we do fancy indexing in sorted order and reshuffle the
         # result back in the original order.
diff --git a/setup.cfg b/setup.cfg
deleted file mode 100644
index 5aef279..0000000
--- a/setup.cfg
+++ /dev/null
@@ -1,2 +0,0 @@
-[metadata]
-description-file = README.rst
diff --git a/setup.py b/setup.py
index 74d6d04..5c655c1 100644
--- a/setup.py
+++ b/setup.py
@@ -27,7 +27,6 @@ setup(
     description='Data pipeline framework for machine learning',
     long_description=LONG_DESCRIPTION,
     url='https://github.com/mila-udem/fuel.git',
-    download_url='https://github.com/mila-udem/fuel/tarball/v' + version,
     author='Universite de Montreal',
     license='MIT',
     # See https://pypi.python.org/pypi?%3Aaction=list_classifiers


Test patch:
diff --git a/tests/test_cifar10.py b/tests/test_cifar10.py
index c2dac95..4a930fc 100644
--- a/tests/test_cifar10.py
+++ b/tests/test_cifar10.py
@@ -33,5 +33,7 @@ def test_cifar10():
 
     assert_raises(ValueError, CIFAR10, ('valid',))
 
-    assert_raises(ValueError, CIFAR10,
-                  ('train',), subset=slice(50000, 60000))
+    dummy = CIFAR10(('train',), subset=slice(50000, 60000))
+    handle = dummy.open()
+    assert_raises(ValueError, dummy.get_data, handle, slice(0, 10000))
+    dummy.close(handle)
diff --git a/tests/test_hdf5.py b/tests/test_hdf5.py
index 88695ad..c4e0dbe 100644
--- a/tests/test_hdf5.py
+++ b/tests/test_hdf5.py
@@ -232,20 +232,6 @@ class TestH5PYDataset(object):
         assert_raises(TypeError, dataset.get_data, handle, [7, 4, 6, 2, 5])
         dataset.close(handle)
 
-    def test_in_memory_example_scheme(self):
-        dataset = H5PYDataset(
-            self.h5file, which_sets=('train',), load_in_memory=True)
-        iter_ = dataset.get_example_stream().get_epoch_iterator()
-        assert_equal(next(iter_), (self.features[0], self.targets[0]))
-        assert_equal(next(iter_), (self.features[1], self.targets[1]))
-
-    def test_out_of_memory_example_scheme(self):
-        dataset = H5PYDataset(
-            self.h5file, which_sets=('train',), load_in_memory=False)
-        iter_ = dataset.get_example_stream().get_epoch_iterator()
-        assert_equal(next(iter_), (self.features[0], self.targets[0]))
-        assert_equal(next(iter_), (self.features[1], self.targets[1]))
-
     def test_value_error_on_unequal_sources(self):
         def get_subsets():
             return H5PYDataset(self.h5file, which_sets=('train',)).subsets
@@ -269,8 +255,7 @@ class TestH5PYDataset(object):
 
     def test_value_error_out_of_memory_get_data(self):
         dataset = H5PYDataset(self.h5file, which_sets=('train',))
-        assert_raises(ValueError, dataset._out_of_memory_get_data, None,
-                      dict())
+        assert_raises(ValueError, dataset._out_of_memory_get_data, None, True)
 
     def test_index_split_out_of_memory(self):
         features = numpy.arange(50, dtype='uint8').reshape((10, 5))
@@ -409,23 +394,3 @@ class TestH5PYDataset(object):
             assert_equal(val, truth)
         assert_equal(rval[1], expected_targets)
         dataset.close(handle)
-
-    def test_vlen_in_memory_example_scheme(self):
-        dataset = H5PYDataset(
-            self.vlen_h5file, which_sets=('train',), load_in_memory=True,
-            sort_indices=False)
-        iter_ = dataset.get_example_stream().get_epoch_iterator()
-        assert_equal(next(iter_),
-                     (self.vlen_features[0], self.vlen_targets[0]))
-        assert_equal(next(iter_),
-                     (self.vlen_features[1], self.vlen_targets[1]))
-
-    def test_vlen_out_of_memory_example_scheme(self):
-        dataset = H5PYDataset(
-            self.vlen_h5file, which_sets=('train',), load_in_memory=False,
-            sort_indices=False)
-        iter_ = dataset.get_example_stream().get_epoch_iterator()
-        assert_equal(next(iter_),
-                     (self.vlen_features[0], self.vlen_targets[0]))
-        assert_equal(next(iter_),
-                     (self.vlen_features[1], self.vlen_targets[1]))
diff --git a/tests/test_utils.py b/tests/test_utils.py
index 0a0b060..fef61bc 100644
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -284,5 +284,5 @@ def receive_integers(socket):
 
 def test_producer_consumer():
     assert (producer_consumer(partial(send_integers, n=2000),
-                              receive_integers) ==
-            sum(i ** 2 for i in range(2000)))
+                              receive_integers)
+            == sum(i ** 2 for i in range(2000)))
diff --git a/tests/transformers/test_image.py b/tests/transformers/test_image.py
index ba6c7b4..3abe221 100644
--- a/tests/transformers/test_image.py
+++ b/tests/transformers/test_image.py
@@ -1,7 +1,7 @@
 from collections import OrderedDict
 from io import BytesIO
 import numpy
-from numpy.testing import assert_raises
+from numpy.testing import assert_raises, assert_equal
 from PIL import Image
 from picklable_itertools.extras import partition_all
 from six.moves import zip
@@ -11,7 +11,8 @@ from fuel.schemes import ShuffledScheme, SequentialExampleScheme
 from fuel.streams import DataStream
 from fuel.transformers.image import (ImagesFromBytes,
                                      MinimumImageDimensions,
-                                     RandomFixedSizeCrop)
+                                     RandomFixedSizeCrop,
+                                     Random2DRotation)
 
 
 def reorder_axes(shp):
@@ -283,3 +284,154 @@ class TestFixedSizeRandomCrop(ImageTestingMixin):
 
         assert_raises(ValueError, bstream.transform_source_batch,
                       numpy.empty((5, 3, 4, 2)), 'source1')
+
+
+class TestRandom2DRotation(ImageTestingMixin):
+    def setUp(self):
+        source1 = numpy.zeros((2, 3, 4, 5), dtype='uint8')
+        source1[:] = numpy.arange(3 * 4 * 5, dtype='uint8').reshape((3, 4, 5))
+
+        source2 = numpy.empty(2, dtype=object)
+        source2[0] = numpy.arange(3 * 4 * 5, dtype='uint8').reshape((3, 4, 5))
+        source2[1] = numpy.arange(3 * 4 * 6, dtype='uint8').reshape((3, 4, 6))
+
+        source3 = [source2[0], source2[1]]
+
+        self.source1 = source1
+        self.source2 = source2
+        self.source3 = source3
+
+        axis_labels = {'source1': ('batch', 'channel', 'height', 'width'),
+                       'source2': ('batch', 'channel', 'height', 'width'),
+                       'source3': ('batch', 'channel', 'height', 'width')}
+        self.dataset = \
+            IndexableDataset(OrderedDict([('source1', source1),
+                                          ('source2', source2),
+                                          ('source3', source3)]),
+                             axis_labels=axis_labels)
+        self.common_setup()
+
+    def test_format_exceptions(self):
+        estream = Random2DRotation(self.example_stream,
+                                   which_sources=('source2',))
+        bstream = Random2DRotation(self.batch_stream,
+                                   which_sources=('source2',))
+        assert_raises(ValueError, estream.transform_source_example,
+                      numpy.empty((5, 6)), 'source2')
+        assert_raises(ValueError, bstream.transform_source_batch,
+                      [numpy.empty((7, 6))], 'source2')
+        assert_raises(ValueError, bstream.transform_source_batch,
+                      [numpy.empty((8, 6))], 'source2')
+
+    def test_maximum_rotation_invalid_exception(self):
+        assert_raises(ValueError, Random2DRotation, self.example_stream,
+                      maximum_rotation=0.0,
+                      which_sources=('source2',))
+        assert_raises(ValueError, Random2DRotation, self.example_stream,
+                      maximum_rotation=3.1416,
+                      which_sources=('source2',))
+
+    def test_invalid_resample_exception(self):
+        assert_raises(ValueError, Random2DRotation, self.example_stream,
+                      resample='nonexisting')
+
+    def test_random_2D_rotation_example_stream(self):
+        maximum_rotation = 0.5
+        rng = numpy.random.RandomState(123)
+        estream = Random2DRotation(self.example_stream,
+                                   maximum_rotation,
+                                   rng=rng,
+                                   which_sources=('source1',))
+        # the C x X x Y image should have equal rotation for all c in C
+        out = estream.transform_source_example(self.source1[0], 'source1')
+        expected = numpy.array([[[0,  0,  0,  2,  3],
+                                 [0,  0,  1,  7,  8],
+                                 [0,  5,  6, 12, 13],
+                                 [0, 10, 11, 17, 18]],
+                                [[0,  0,  0, 22, 23],
+                                 [0, 20, 21, 27, 28],
+                                 [0, 25, 26, 32, 33],
+                                 [0, 30, 31, 37, 38]],
+                                [[0,  0,  0, 42, 43],
+                                 [0, 40, 41, 47, 48],
+                                 [0, 45, 46, 52, 53],
+                                 [0, 50, 51, 57, 58]]], dtype='uint8')
+        assert_equal(out, expected)
+
+    def test_random_2D_rotation_batch_stream(self):
+        rng = numpy.random.RandomState(123)
+        bstream = Random2DRotation(self.batch_stream,
+                                   maximum_rotation=0.5,
+                                   rng=rng,
+                                   which_sources=('source1',))
+        # each C x X x Y image should have equal rotation for all c in C
+        out = bstream.transform_source_batch(self.source1, 'source1')
+        expected = numpy.array([[[[0,  0,  0,  2,  3],
+                                  [0,  0,  1,  7,  8],
+                                  [0,  5,  6, 12, 13],
+                                  [0, 10, 11, 17, 18]],
+                                 [[0,  0,  0, 22, 23],
+                                  [0, 20, 21, 27, 28],
+                                  [0, 25, 26, 32, 33],
+                                  [0, 30, 31, 37, 38]],
+                                 [[0,  0,  0, 42, 43],
+                                  [0, 40, 41, 47, 48],
+                                  [0, 45, 46, 52, 53],
+                                  [0, 50, 51, 57, 58]]],
+                                [[[0,  0,  1,  0,  0],
+                                  [0,  5,  6,  2,  3],
+                                  [0, 10, 11,  7,  8],
+                                  [0, 15, 16, 12, 13]],
+                                 [[0, 20, 21,  0,  0],
+                                  [0, 25, 26, 22, 23],
+                                  [0, 30, 31, 27, 28],
+                                  [0, 35, 36, 32, 33]],
+                                 [[0, 40, 41,  0,  0],
+                                  [0, 45, 46, 42, 43],
+                                  [0, 50, 51, 47, 48],
+                                  [0, 55, 56, 52, 53]]]], dtype='uint8')
+        assert_equal(out, expected)
+
+        expected = \
+            [numpy.array([[[0,  0,  0,  2,   3],
+                           [0,  0,  1,  7,   8],
+                           [0,  5,  6,  12, 13],
+                           [0,  10, 11, 17, 18]],
+                          [[0,  0,  0,  22, 23],
+                           [0,  20, 21, 27, 28],
+                           [0,  25, 26, 32, 33],
+                           [0,  30, 31, 37, 38]],
+                          [[0,  0,  0,  42, 43],
+                           [0,  40, 41, 47, 48],
+                           [0,  45, 46, 52, 53],
+                           [0,  50, 51, 57, 58]]], dtype='uint8'),
+             numpy.array([[[0,  0,  1,  2,  0,   0],
+                           [0,  6,  7,  8,  3,   4],
+                           [12, 13, 14, 15, 9,  10],
+                           [18, 19, 20, 15, 16, 17]],
+                          [[0,  24, 25, 26,  0,  0],
+                           [0,  30, 31, 32, 27, 28],
+                           [36, 37, 38, 39, 33, 34],
+                           [42, 43, 44, 39, 40, 41]],
+                          [[0,  48, 49, 50,  0,  0],
+                           [0,  54, 55, 56, 51, 52],
+                           [60, 61, 62, 63, 57, 58],
+                           [66, 67, 68, 63, 64, 65]]], dtype='uint8')]
+
+        rng = numpy.random.RandomState(123)
+        bstream = Random2DRotation(self.batch_stream,
+                                   maximum_rotation=0.5,
+                                   rng=rng,
+                                   which_sources=('source2',))
+        out = bstream.transform_source_batch(self.source2, 'source2')
+        assert_equal(out[0], expected[0])
+        assert_equal(out[1], expected[1])
+
+        rng = numpy.random.RandomState(123)
+        bstream = Random2DRotation(self.batch_stream,
+                                   maximum_rotation=0.5,
+                                   rng=rng,
+                                   which_sources=('source3',))
+        out = bstream.transform_source_batch(self.source3, 'source3')
+        assert_equal(out[0], expected[0])
+        assert_equal(out[1], expected[1])