Checkpoint didn't support list output

username: JindongJiang
cooked: <p>Currently, checkpoint function allows only <code>Tensor</code>/<code>Variable</code> output. But some architectures might have more than one output, such as the encoder of a standard FCNs, which might have several outputs from the mainstream and skip connection, or architectures that have both text and label outputs.</p>
<p>Now, my solution is to concatenate the tensors in the function and unroll them outside checkpoint, which is not so convenient.</p>
<p>So are there any neat way to do that? Or would it be <code>list</code> supports in the future?</p>
<p>Thank you so much, really appreciate your work !</p>

username: albanD
cooked: <p>Hi,</p>
<p>I’m not sure to understand, I though checkpoint was takes an arbitrary number of input tensor and output an arbitrary number of output tensors. Doesn’t that work? Are you returning a list containing Tensor or multiple tensor from the python function?</p>

username: JindongJiang
cooked: <p>It actually didn’t work. It sure can input many input as the <code>*args</code>, but list output causes error. For example, my output was like,</p>
<pre><code class="lang-python">return [tensor1, tensor2, tensor3, tensor4, tensor4]
</code></pre>
<p>and the error would be like,</p>
<pre><code class="lang-bash">TypeError: CheckpointFunctionBackward.forward: expected Variable (got list) for return value 0
</code></pre>
<p>I hope I am using it right. <img src="https://discuss.pytorch.org/images/emoji/apple/face_with_monocle.png?v=5" title=":face_with_monocle:" class="emoji" alt=":face_with_monocle:"></p>

username: albanD
cooked: <p>Yes I don’t think it support list as output. Can you change your code to: <code>return tensor1, tensor2, tensor3, tensor4, tensor4</code> (without the list) ?</p>

username: JindongJiang
cooked: <p>It actually works! <img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=5" title=":joy:" class="emoji" alt=":joy:"> <img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=5" title=":joy:" class="emoji" alt=":joy:"> <img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=5" title=":joy:" class="emoji" alt=":joy:"></p>
<p>Thank you so much!</p>

username: neostrider
cooked: <p>What could one do, if a custom layer (implemented by deriving from autograd.Function) has a forward function that has to return a set of tensors, where the number of tensors depends on an input parameter. List of tensors could help but it looks like list as output is not supported. I get the following error: “TypeError: ripsLayerBackward.forward: expected Variable (got list) for return value 0”, where ripsLayer is the name of my layer.</p>

username: albanD
cooked: <p>You can return as many Tensors as you want (it can be different from one forward to the next actually). You just need to make sure that your backward will handle this variable number of grad output.</p>

username: neostrider
cooked: <p>Thank you for your reply. Since, I need to return a variable number of tensors, could I create a tuple and then return that tuple?</p>

username: albanD
cooked: <p>The output of your function should be a bunch of Tensors. Tuples are a special construct in python in the sense that when you do <code>return a, b</code> you actually return a tuple containing a and b.<br>
So I guess it should work yes.</p>

username: neostrider
cooked: <p>Thanks for your clarification. In case there is a more elegant way to handle a variable number of tensors, please, do share it.</p>

username: albanD
cooked: <p>I don’t think there is I’m afraid <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

username: neostrider
cooked: <p>No problem <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> It is understandable.</p>