BLAS performance on macOS vs Linux vs Lua Torch

florin: <SCODE>Allocation:   5.921
Torch Blas:   7.277
Numpy Blas:   7.841
Torch cuBlas: 0.205
<ECODE> <SCODE>Allocation:   4.030
Torch Blas:   21.112
Numpy Blas:   21.82
Torch cuBlas: 0.121
<ECODE> <SCODE>Allocation:   4.557
Torch Blas:   19.646
Numpy Blas:   20.155
Torch cuBlas: 0.155
<ECODE> <SCODE>openblas_lapack_info:
  NOT AVAILABLE
lapack_opt_info:
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/opt/anaconda3/include']
    library_dirs = ['/opt/anaconda3/lib']
    libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'iomp5', 'pthread']
blas_mkl_info:
    ...
blas_opt_info:
    ...
lapack_mkl_info:
    ...
<ECODE> The code I am using: <SCODE>import time
import torch
import numpy
torch.set_default_tensor_type("torch.FloatTensor")

w = 5000
h = 40000
is_cuda = torch.cuda.is_available()
start = time.time()

a = torch.rand(w,h)
b = torch.rand(h,w)
a_np = a.numpy()
b_np = b.numpy()
if is_cuda:
    a_cu = a.cuda()
    b_cu = b.cuda()

allocation = time.time()
print("Allocation ", allocation - start)

c = a.mm(b)
th_blas = time.time()
print("Torch Blas ", th_blas - allocation)

c = a_np.dot(b_np)
np_blas = time.time()
print("Numpy Blas ", np_blas - th_blas)

if is_cuda:
    c = a_cu.mm(b_cu)
    cu_blas = time.time()
    print("Torch cuBlas ", cu_blas - np_blas)

print("Final", time.time() - start)
<ECODE> edit: For comparison here are the results of the same script on Lua Torch on the last machine from above: <SCODE>Allocation:  	4.426
Torch Blas: 	2.777
Torch cuBlas: 	0.097
<ECODE>

apaszke: 

florin: Did that, it didn’t help. Is the variable relevant for runtime only? Or I should also try to recompile torch with it exported? MKL seems properly configured: <SCODE>>> mkl.get_max_threads()
>> 20
<ECODE> Can someone else run the script above and report the numbers?

colesbury: I’m seeing similar behavior. I ran your script from both PyTorch built from source and the Conda installation: built from source: <SCODE>Allocation  6.476427316665649
Torch Blas  4.414772272109985
Numpy Blas  2.665677547454834
Torch cuBlas  0.14812421798706055
Final 13.705262184143066
<ECODE> conda: <SCODE>Allocation  5.521166086196899
Torch Blas  39.35049605369568
Numpy Blas  39.40145969390869
Final 84.42150139808655
<ECODE> It looks like something is wrong with Conda.

florin: Thanks for taking the time to check this. Also for the tip on benchmarking cuda. I’ll try to build it again altghough I did this several times with same result.

apaszke: 

florin: 

florin: 

apaszke: Ok, MKL is found. We’ll have to look into it. Thanks for the report.

florin: Let me know if there’s anything else I can look into, especially if you can’t reproduce this situation.

ngimel: What fixed things for me was adding “iomp5” to FindMKL.cmake: <SCODE>diff --git a/torch/lib/TH/cmake/FindMKL.cmake b/torch/lib/TH/cmake/FindMKL.cmake
index e68ae6a..7c9325a 100644
--- a/torch/lib/TH/cmake/FindMKL.cmake
+++ b/torch/lib/TH/cmake/FindMKL.cmake
@@ -50,7 +50,7 @@ ENDIF ("${SIZE_OF_VOIDP}" EQUAL 8)
 IF(CMAKE_COMPILER_IS_GNUCC)
   SET(mklthreads "mkl_gnu_thread" "mkl_intel_thread")
   SET(mklifaces  "gf" "intel")
-  SET(mklrtls)
+  SET(mklrtls "iomp5")
 ELSE(CMAKE_COMPILER_IS_GNUCC)
   SET(mklthreads "mkl_intel_thread")
   SET(mklifaces  "intel")
<ECODE>

apaszke: 

florin: