Speeding Up Loops on GPU

username: dgn2
cooked: <p>Hello,</p>
<p>I have been trying to use PyTorch to speed up some simple embarrassingly parallel computations with little success. I am looking for some guidance as to how to speed up the following simple code. Any help would be very much appreciated.</p>
<p>The following functions are to create data to use in the simple example further below:</p>
<pre><code class="lang-auto">import numpy
import math
import torch
import pandas
import timeit
from timeit import default_timer as timer

def assetPathsCPU(S0,mu,sigma,T,nRows,nPaths):
    dt = T/nRows
    nudt = (mu-0.5*sigma**2)*dt    
    sidt = sigma*math.sqrt(dt)
    increments = nudt + sidt*numpy.random.randn(int(nRows),int(nPaths))
    x=numpy.concatenate((math.log(S0)*numpy.ones((1,int(nPaths))),increments))
    pricePaths=numpy.exp(numpy.cumsum(x,axis=0))

    return pricePaths

def assetPathsGPU(S0,mu,sigma,T,nRows,nPaths,dtype,device):
    dt = T/nRows
    nudt = (mu-0.5*sigma**2)*dt
    sidt = sigma*torch.sqrt(dt)
    pricePaths=torch.exp(torch.cumsum(torch.cat((torch.log(S0)*torch.ones(1,nPaths,dtype=dtype,device=cuda0),
        torch.distributions.Normal(nudt,sidt).sample((nRows, nPaths)).squeeze()), dim=0),dim=0))
    
    return pricePaths
</code></pre>
<p>These are the simple functions - one for the CPU and one for the GPU:</p>
<pre><code class="lang-auto">def emaNPathsCPU(pricePaths,lookback):
    # find T and nPaths
    T,nPaths=pricePaths.shape
    # create output array
    ema=numpy.zeros([int(T),int(nPaths)])
    # compute the smoothing constant
    a = 2.0 / ( lookback + 1.0 )
    # iterate over each price path
    for pathIndex in range(0,int(nPaths)):
        # iterate over each price path
        ema[0,pathIndex] = pricePaths[0,pathIndex]
        # iterate over each point in time and compute the EMA 
        for t in range(1,T):
            ema[t,pathIndex]=a * (pricePaths[t,pathIndex]-ema[t-1,
                pathIndex]) + ema[t-1,pathIndex]
    return ema

def emaNPathsGPU(pricePaths,lookback,dtype,device):
    # find T and nPaths
    T,nPaths=pricePaths.shape
    # create output array
    #ema=numpy.zeros([int(T),int(nPaths)])
    ema=torch.zeros(T,nPaths,dtype=dtype,device=device)  
    # compute the smoothing constant
    a = 2.0 / ( lookback + 1.0 )
    ema[0,:] = pricePaths[0,:]    
    # iterate over each price path
    for pathIndex in range(nPaths):
        # iterate over each point in time and compute the EMA 
        for t in range(1,T):
            ema[t,pathIndex]=a * (pricePaths[t,pathIndex]-ema[t-1,
                pathIndex]) + ema[t-1,pathIndex]
    return ema
</code></pre>
<p>Here is how I call them:</p>
<pre><code class="lang-auto">cuda0 = torch.device('cuda:0')
dtype=torch.float64
lookbackGPU=torch.tensor(90.0,dtype=dtype,device=cuda0)

# start timer (EMA)
ts_emaGPU = timer()
# EMA on paths
emaGPU=emaNPathsGPU(pricePathsGPU[:,0:1000],lookbackGPU,dtype,cuda0)
# end timer (prices)
te_emaGPU = timer()
# compute time elasped
timeElasped_emaGPU=te_emaGPU-ts_emaGPU
# display time elasped
print('EMA Time Elasped (GPU): '+str(timeElasped_emaGPU))
</code></pre>
<pre><code class="lang-auto">pricePathsGPU_CPU=pricePathsGPU.cpu().numpy()
lookbackCPU=90.0
# start timer (EMA)
ts_emaCPU = timer()
# EMA on paths
emaCPU=emaNPathsCPU(pricePathsGPU_CPU[:,0:1000],lookbackCPU)
# end timer (EMA)
te_emaCPU = timer()
# compute time elasped
timeElasped_emaCPU=te_emaCPU-ts_emaCPU
# display time elasped
print('EMA Time Elasped (CPU): '+str(timeElasped_emaCPU))
</code></pre>
<p>The link to the example notebook is here:</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/dnokes/pytorch_examples/blob/master/simpleGpuVsCpuExample.ipynb" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/dnokes/pytorch_examples/blob/master/simpleGpuVsCpuExample.ipynb" target="_blank" rel="nofollow noopener">dnokes/pytorch_examples/blob/master/simpleGpuVsCpuExample.ipynb</a></h4>
<pre><code class="lang-ipynb">{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy\n",
    "import math\n",
    "import torch\n",
    "import pandas\n",
    "import timeit\n",
    "from timeit import default_timer as timer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
</code></pre>

  This file has been truncated. <a href="https://github.com/dnokes/pytorch_examples/blob/master/simpleGpuVsCpuExample.ipynb" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Why is the GPU version so slow?</p>
<p>I was expecting the ‘path’ loop to run in parallel because it is completely independent. The utilization on the Titan GPU is about 25% while it is running, but is very slow.</p>
<p>What can I do to make the PyTorch version faster?</p>
<p>Any help would be greatly appreciated.</p>
<p>Many thanks,<br>
Derek</p>

username: tom
cooked: <p>The parallelism offered by the GPU is primarily per-op parallelism (there is something to be said about cuda streams, but that probably isn’t what you want to do here). The best way to speed this up is to vectorize over your Monte Carlo paths (possibly serializing over time).<br>
As you appear to be looking into financial mathematics: I did something like that for the <a href="https://github.com/t-vi/interesting-rates/blob/master/interest-rate-models/Hull-White-1f.ipynb" rel="nofollow noopener">Hull-White-Model</a> a long time ago (when we had <code>Variable</code>…).</p>
<p>Best regards</p>
<p>Thomas</p>

username: dgn2
cooked: <p>Thank you Thomas for your response. The referenced notebook is helpful.</p>
<p>I was hoping that PyTorch would function in a way similar to numba.</p>

username: tom
cooked: <p>Numba is great, but you have to think about the order of your computations nonetheless.<br>
What you would need is the “holy grail of all <a href="https://en.wikipedia.org/wiki/Polytope_model" rel="nofollow noopener">polyhedral optimizers</a>” or so that would rewrite the order of your entire computation. There are projects like TVM or TensorFlow MLIR that are working on pushing some boundaries, but so far spending a few thoughts yourself is the best way to get these things faster.</p>
<p>Best regards</p>
<p>Thomas</p>