Cumulative sum with decay factor

username: A_A
cooked: <p>Hello,</p>
<p>I need to have following operation:</p>
<blockquote>
<p>y[i] = w * \sum_{j=1}^{i} x[j] + x[i]</p>
</blockquote>
<p>Which is equivalent to torch.cumsum() if w := 1. However, in cases where we need to decay the summation by having 0 &lt; w &lt; 1, cumsum() won’t work. Any suggestions about how to go ahead in solving this problem?</p>
<p>Thanks,<br>
Ahmed</p>

username: KFrank
cooked: <p>Hello Ahmed!</p>
<aside class="quote no-group quote-modified" data-username="A_A" data-post="1" data-topic="69788" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/7ea924/48.png" class="avatar"> A_A:</div>
<blockquote>
<p>I need to have following operation:</p>
<blockquote>
<p>y[i] = w * \sum_{j=1}^{i} x[j] + x[i]</p>
</blockquote>
<p>Which is equivalent to torch.cumsum() if w := 1.</p>
</blockquote>
</aside>
<p>As written, this doesn’t actually have a decay factor.  (Each value of<br>
<code>y</code> only has at most a single power of <code>w</code>.)</p>
<p>I think you mean to write something like</p>
<pre><code class="lang-plaintext">y[i] = w * y[i - 1] + x[i]
</code></pre>
<aside class="quote no-group">
<blockquote>
<p>However, in cases where we need to decay the summation by having 0 &lt; w &lt; 1, cumsum() won’t work.</p>
</blockquote>
</aside>
<p>You can get what (I think) you want by pre-multiplying <code>x</code> with powers<br>
of the weight, and then using <code>cumsum()</code>, all using pytorch tensor<br>
operations, and no explicit loops.</p>
<p>Here is a (pytorch version 0.3.0) demonstration script:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
torch.__version__

# weighted cumulative sum with tensor operations
def wtCumSum (vector, wt):
    wts = wt**((len (vector) - 1.0) - torch.FloatTensor (range (len (vector))))
    return torch.cumsum (wts * vector, dim = 0) / wts

torch.manual_seed (2020)

vector = torch.randn (10,)
wt = 0.5

wcs = wtCumSum (vector, wt)

vector
wt
wcs

# check against non-tensor iterative calculation
wcs2 = []
wcs2.append (vector[0])
for  i in range (len(vector) - 1):
    wcs2.append (wt * wcs2[i] + vector[i+1])

wcs2

wcs - torch.FloatTensor (wcs2)
</code></pre>
<p>And here is the output:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'0.3.0b0+591e73e'
&gt;&gt;&gt;
&gt;&gt;&gt; # weighted cumulative sum with tensor operations
... def wtCumSum (vector, wt):
...     wts = wt**((len (vector) - 1.0) - torch.FloatTensor (range (len (vector))))
...     return torch.cumsum (wts * vector, dim = 0) / wts
...
&gt;&gt;&gt; torch.manual_seed (2020)
&lt;torch._C.Generator object at 0x000001BEE6816630&gt;
&gt;&gt;&gt;
&gt;&gt;&gt; vector = torch.randn (10,)
&gt;&gt;&gt; wt = 0.5
&gt;&gt;&gt;
&gt;&gt;&gt; wcs = wtCumSum (vector, wt)
&gt;&gt;&gt;
&gt;&gt;&gt; vector

 1.2372
-0.9604
 1.5415
-0.4079
 0.8806
 0.0529
 0.0751
 0.4777
-0.6759
-2.1489
[torch.FloatTensor of size 10]

&gt;&gt;&gt; wt
0.5
&gt;&gt;&gt; wcs

 1.2372
-0.3418
 1.3706
 0.2775
 1.0193
 0.5626
 0.3564
 0.6559
-0.3480
-2.3229
[torch.FloatTensor of size 10]

&gt;&gt;&gt;
&gt;&gt;&gt; # check against non-tensor iterative calculation
... wcs2 = []
&gt;&gt;&gt; wcs2.append (vector[0])
&gt;&gt;&gt; for  i in range (len(vector) - 1):
...     wcs2.append (wt * wcs2[i] + vector[i+1])
...
&gt;&gt;&gt; wcs2
[1.2372283935546875, -0.34178459644317627, 1.370636761188507, 0.2774530053138733, 1.0193056166172028, 0.5625850260257721, 0.3564097583293915, 0.6558640152215958, -0.34795021265745163, -2.3229051791131496]
&gt;&gt;&gt;
&gt;&gt;&gt; wcs - torch.FloatTensor (wcs2)

 0
 0
 0
 0
 0
 0
 0
 0
 0
 0
[torch.FloatTensor of size 10]
</code></pre>
<p>Please note that this illustrates the idea, but is written only for a<br>
one-dimensional vector.</p>
<p>Also note that if <code>wt</code> differs much from one and <code>vector</code> is rather<br>
long, then the powers of <code>wt</code> will underflow to zero, and you will<br>
get <code>0 / 0</code> <code>nan</code>s in the result of this version.</p>
<p>So for real work, you should write out the loop with the conventional<br>
iterative formula, losing autograd and the performance benefit of<br>
tensor operations (or implement a tensor version of weighted<br>
<code>cumsum()</code>).</p>
<p>Best.</p>
<p>K. Frank</p>

username: A_A
cooked: <p>Thanks for your response.</p>
<p><strong>1. Factoring out decay factor:</strong></p>
<p>So yes in principle the idea is correct to implement weighted cumulative sum as:</p>
<pre><code class="lang-auto">y[i] = (y[i-1] + x[i]/(w^i)) / w^i
</code></pre>
<p>But the array is rather long so can have a loss of precision.</p>
<p><strong>2. For loops:</strong><br>
The second way of using for loops is also viable but I have to see if there are much performance drop (I do not need backpropagation anyway).</p>
<p><strong>3. Implement tensor version:</strong><br>
Have never done this before, but would like to give it a try. Can you recommend how to give it a shot?</p>
<p>Thanks!</p>

username: KFrank
cooked: <p>Hello Ahmed!</p>
<aside class="quote no-group" data-username="A_A" data-post="3" data-topic="69788" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/7ea924/48.png" class="avatar"> A_A:</div>
<blockquote>
<p><strong>2. For loops:</strong><br>
The second way of using for loops is also viable but I have to see if there are much performance drop (I do not need backpropagation anyway).</p>
</blockquote>
</aside>
<p>Since you don’t need your weighted cumulative sum for<br>
backpropagation, I assume that you are using it for neither<br>
training nor inference.</p>
<p>As such, any loss of performance shouldn’t matter much to<br>
your overall workflow, so I wouldn’t worry about it.</p>
<p>Good luck!</p>
<p>K. Frank</p>

username: A_A
cooked: <p>Hello,</p>
<p>Half of your assumption is wrong <img src="https://discuss.pytorch.org/images/emoji/apple/stuck_out_tongue.png?v=9" title=":stuck_out_tongue:" class="emoji" alt=":stuck_out_tongue:"> I use it during inference.</p>
<p>Thanks,<br>
Ahmed</p>

username: anton
cooked: <p>Check out a package I made for this purpose: <a href="https://github.com/toshas/torch-discounted-cumsum" class="inline-onebox" rel="noopener nofollow ugc">GitHub - toshas/torch-discounted-cumsum: Fast Discounted Cumulative Sums in PyTorch</a></p>

username: pchvykov
cooked: <p>Just set up a neet little function based on the above idea - and also reversing the direction of the cumsum, as is typical in most RL applications:</p>
<pre><code class="lang-auto">def discount_cumsum(vector, discount, device='cpu'): 
    wts = discount**torch.arange(len(vector), dtype = torch.float64, device=device) # pre-compute all discounts
    x = wts*vector # weighted vector
    cum_sum = torch.cumsum(x, dim=0) #forward cumsum
    re_cum_sum = x - cum_sum + cum_sum[-1:] #reversed cumsum
    return re_cum_sum/wts
</code></pre>
<p>As mentioned above, beware of precision issues when vectors get long and decay factors go to zero. Also note that the method for reversed cumsum here is faster than using torch.flip</p>