question: How to apply a nn.Module (i.e. CNN) across an axis (i.e. Video input) in a parallelizable way
context: Hi, I’m trying to apply CNN to each image in a video. Currently, my implementation uses a for loop and torch.cat where I take each image and apply the CNN module in the loop. But clearly, this is sequential and I don’t see why it can’t be parallelized in theory since all images are independent from each other.
  However, I’m not sure how this can be accomplished. I couldn’t find any built-in function for PyTorch. Is there a way to do this in parallel in PyTorch Lightning?
  My video input shape looks like this: (batch_size, seq_len, channel, height, width) and CNN takes input shape of (batch_size, channel, height, width).
  Thanks in advance for your help!
answer: You can simply convert your (batch_size, seq_len, channel, height, width) tensor into an (batch_size*seq_len, channel, height, width) tensor, run your model and then reshape your output back:
  batch_size, seq_len, channel, height, width = 5, 10, 3, 28, 28 # just random picked
  input = torch.randn(batch_size, seq_len, channel, height, width)
  input = input.reshape(batch_size * seq_len, channel, height, width)
  output = model(input) 
  # split the batch dimension back into the original batch size and sequence length
  output = output.reshape(batch_size, seq_len, *output.shape[1:])