Is All Pairs Shortest Path with PyTorch / CUDA possible?

joshualmitchell: <p>I don’t know enough about the implementation details to know whether this is a good idea, but would the following be possible / feasible / efficient?</p>
<p>Imagine you have a big graph G = (V, E), and you want to find the shortest path from a to b for all a, b in V. Since this is embarrassingly parallel and has tons of individual threads, is there a known way to use PyTorch/CUDA to find all these paths?</p>

gapcoder: <p>In my opinion, it is possible, not feasible, not that efficient.</p>
<p>Textbooks algorithms for all-pairs shortest paths imply that either you use a dynamic programming approach or a particular version of matrix multiplication. <a href="https://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm" rel="nofollow noopener">https://en.wikipedia.org/wiki/Floyd–Warshall_algorithm</a></p>
<p>In both cases, you work with matrices/tensors; it is doable with PyTorch.</p>
<p>But you don’t need to calculate gradients or use calculation graphs to solve the problem, that makes it not that feasible and efficient for PyTorch.</p>
<p>One stop library for graph algorithms is Networkx:<br>
<a href="https://networkx.github.io/documentation/stable/reference/algorithms/shortest_paths.html" class="onebox" target="_blank" rel="nofollow noopener">https://networkx.github.io/documentation/stable/reference/algorithms/shortest_paths.html</a></p>
<p>If your graphs don’t fit in memory, take a look at Spark.GraphX: <a href="https://spark.apache.org/graphx/" rel="nofollow noopener">https://spark.apache.org/graphx/</a> it is specifically designed to distribute the graphs, and work in Pregel-like (message passing) algorithm design.</p>

Jakub_Adamczyk: <p>I know this is an old question, but it shows as one of the first results in Google, so I will correct the existing answer.</p>
<p>Computing APSP on GPU is possible, feasible and very efficient, and it’s a well-researched topic. There are dedicated algorithms for this, most prominently all-pair shortest-paths R-Kleene algorithm, see [1] and [2]. It uses parallelized, in-place version of fast matrix multiplication routines, rendering APSP problem as a set of recursive matrix-matrix multiplications on a tropical closed semiring. The basic version requires keeping the distance matrix in memory, but is extremely fast, orders of magnitude faster than typical CPU-based methods. But in [3] and [4] authors show that if you use condensed adjacency lists as a graph representation, you can use BFS-style algorithm, you can scale this up further, at the cost of random access and overall slower computation. There are also parallelized variants of Floyd-Warshall algorithm that work well on GPUs, if you implement them well.</p>
<p>None of those papers are implemented in PyTorch and publicly available, as far as I’m aware. There are CUDA-based versions on GitHub, using various approaches:</p>
<ul>
<li><a href="https://github.com/mmitalidis/APSP-CUDA" class="inline-onebox" rel="noopener nofollow ugc">GitHub - mmitalidis/APSP-CUDA: All-Pairs Shortest Path implementation on GPU, using CUDA.</a></li>
<li><a href="https://github.com/tanjary21/APSP_GPU/" class="inline-onebox" rel="noopener nofollow ugc">GitHub - tanjary21/APSP_GPU: GPU implementation of Floyd-Warshall and R-Kleene algorithms to solve the All-Pairs-Shortest-Paths(APSP) problem on Graphs. Code includes random graph generators and benchmarking/plotting scripts.</a></li>
<li><a href="https://github.com/ihollywhy/APSP_CUDA" class="inline-onebox" rel="noopener nofollow ugc">GitHub - ihollywhy/APSP_CUDA: CUDA code for apsp problem</a></li>
</ul>
<p>[1] D’alberto, Paolo, and Alexandru Nicolau. “R-Kleene: A high-performance divide-and-conquer algorithm for the all-pair shortest path for densely connected networks.” <em>Algorithmica</em> 47 (2007): 203-213.<br>
[2] Buluç, Aydın, John R. Gilbert, and Ceren Budak. “Solving path problems on the GPU.” <em>Parallel Computing</em> 36.5-6 (2010): 241-253.<br>
[3] Czech, Wojciech, and David A. Yuen. “Efficient graph comparison and visualization using gpu.” <em>2011 14th IEEE International Conference on Computational Science and Engineering</em> . IEEE, 2011.<br>
[4] Harish, Pawan, and Petter J. Narayanan. “Accelerating large graph algorithms on the GPU using CUDA.” <em>International conference on high-performance computing</em> . Berlin, Heidelberg: Springer Berlin Heidelberg, 2007.</p>