Theano/Theano — Issue #130: Allow pickle of SharedVariable on the gpu
Author: nouiz
Created: 2011-10-20 16:52:28+00:00
State: open

Currently if we pickle a shared variable on the GPU, we can unpickle it only on a computer with a GPU.

We need to separate the SharedVariable type(tensor/sparse, dtype, broadcastable flags) from where the data it located. This will allow to unpickle in the described case.

There is currently 2 approached we found:

1) James workspace/strange proposal that only him understand
2) Fred/Arnaud proposal

SharedVariable:
_value =
_storage = None

(This of course ignores a bunch of fields like type, shape and whatnot.)

We would also add a new argument to theano.function called device
which takes its default value from the config.device variable.

Also, at compile time, a function would look at all its shared
variable and for all of them that still have a None _storage, it would
assign them one based on the device parameter. Otherwise, if they
already have a storage, it would either use it or insert the necessary
transfer ops. We will discard the original value used and keep only the actual storage.

When we pickle/unpickle, we pickle the current value, but not where the storage is. New theano function compilation will choose again where to put it. This will prevent pickle/unpickle theano function without checking/changing the shared variable/theano function during unpickle. We also need to think about pickle function in different process and reloading them together. This proposal suppose we won't support pickling of compiled theano function, just of theano graph.

The device argument would also influence which optimizations get used
so that device='gpu' would try to do stuff on the GPU.

This imply that if we compile 2 theano.function like this:
theano.function(...., device='cpu')
theano.function(...., device='gpu')
both using the same shared variable, it won't store it on the
GPU for the second function, because its _storage was defined to be on CPU.

Later we could change this to have the data at possible multiple inputs/outputs and transfer it at the begining of the function. This seam complicated and probably not very useful. So this proposal don't include it, but it would be possible.