2012-001 Co-design of Extremely Scalable Algorithms / Architecture for 3-Dimensional Linear Transforms

Report Date: Written Language: Any Other Identifying Information of this Report: Distribution Statement: Supplementary Notes: The University of Aizu Aizu-Wakamatsu Fukushima 965-8580 Japan 7/2/2012 English First Issue: 10 copies Stanislav G. Sedukhin Co-design of the Extremely Scalable Algorithms/Architecture for 3-Dimensional Linear Transforms 3-dimensional linear transforms, multilinear tensor-by-matrix multiplication, computational index space, cubical data decomposition, modular scheduling, data reusing, torus interconnect, extremely scalable algorithms, algorithm/architecture co-design The 3-dimensional (3D) forward/inverse separable discrete transforms, such as Fourier transform, cosine/sine transform, Hartley transform, and many others, are frequently the principal limiters that prevent many practical applications from scaling to the large number of processors. Existing approaches, which are based on 1D or 2D data decomposition, do not allow the 3D transforms to be scaled to the maximum possible number of computer nodes. Based on the newly proposed 3D decomposition of an N×N×N initial data into P×P×P blocks, where P = N/b and b ∈ [1, 2, . , N ] is the blocking factor, we systematically design unified, highly scalable algorithms for parallel implementation of any forward/inverse 3D transform on the one-to-one correspondent P×P×P torus network of computer nodes. All designed algorithms require 3P “compute-and-roll” time-steps, where each step is equal to the time of execution in each node b4 fused multiply-add (fma) operations and concurrent movement of O(b3) scalar data between nearest-neighbor nodes. The proposed 3D orbital algorithms gracefully avoid a required 3D data transposition and can be extremely scaled up to the maximum number of N3 simple nodes (fma-units) which is equal to the size of initial data. Revised and extended version of the Technical Report 2011-001, the University of Aizu, 2011, 24 p. Distributed Parallel Processing Laboratory Co-design of Extremely Scalable Algorithms/Architecture for 3-Dimensional Linear Transforms Stanislav G. Sedukhin Distributed Parallel Processing Lab. , The University of Aizu, Aizuwakamatsu City, Fukushima 965-8580, Japan e-mail: sedukhin@u-aizu.ac. jp July 2, 2012 Abstract The 3-dimensional (3D) forward/inverse separable discrete transforms, such as Fourier transform, cosine/sine transform, Hartley transform, and many others, are frequently the principal limiters that prevent many practical applications from scaling to the large number of processors. Existing approaches, which are based on 1D or 2D data decomposition, do not allow the 3D transforms to be scaled to the maximum possible number of computer nodes. Based on the newly proposed 3D decomposition of an N×N×N initial data into P ×P ×P blocks, where P = N/b and b ∈ [1, 2, . , N ] is the blocking factor, we systematically design unified, highly scalable algorithms for parallel implementation of any forward/inverse 3D transform on the one-to-one correspondent P ×P ×P torus network of computer nodes. All designed algorithms require 3P “compute-and-roll” time-steps, where each step is equal to the time of execution in each node b fused multiply-add (fma) operations and concurrent movement of O(b) scalar data between nearest-neighbor nodes. The proposed 3D orbital algorithms gracefully avoid a required 3D data transposition and can be extremely scaled up to the maximum number of N simple nodes (fma-units) which is equal to the size of initial data.The 3-dimensional (3D) forward/inverse separable discrete transforms, such as Fourier transform, cosine/sine transform, Hartley transform, and many others, are frequently the principal limiters that prevent many practical applications from scaling to the large number of processors. Existing approaches, which are based on 1D or 2D data decomposition, do not allow the 3D transforms to be scaled to the maximum possible number of computer nodes. Based on the newly proposed 3D decomposition of an N×N×N initial data into P ×P ×P blocks, where P = N/b and b ∈ [1, 2, . , N ] is the blocking factor, we systematically design unified, highly scalable algorithms for parallel implementation of any forward/inverse 3D transform on the one-to-one correspondent P ×P ×P torus network of computer nodes. All designed algorithms require 3P “compute-and-roll” time-steps, where each step is equal to the time of execution in each node b fused multiply-add (fma) operations and concurrent movement of O(b) scalar data between nearest-neighbor nodes. The proposed 3D orbital algorithms gracefully avoid a required 3D data transposition and can be extremely scaled up to the maximum number of N simple nodes (fma-units) which is equal to the size of initial data.