Yes, but the code would much more complicated. The perform would need to do what the perform of GpuAdv..Subtensor1 do and it is more complicated.

But this type of optimization lower the overhead. So it is not sure if you will benefic from it. How much time do you spent per call in the GpuAdv..Subtensor1?