page_title: Collective operation
section_title: Reduce
hierarchical_section_title: Collective operation / Reduce
caption_attribution_description: English: Information from all nodes is combined into a final result alpha, which is computed by an associative operator and stored at one node.
context_page_description: Collective operations are building blocks for interaction patterns, that are often used in SPMD algorithms in the parallel programming context. Hence, there is an interest in efficient realizations of these operations.
  A realization of the collective operations is provided by the Message Passing Interface.
context_section_description: The reduce pattern is used to collect data or partial results from different processing units and to combine them into a global result by a chosen operator. Reduction can be seen as an inverse version of broadcast (§ Broadcast). Given  processing units, message  is on processing unit  initially. All  are aggregated by  and the result is eventually stored on . The reduction operator  must be associative at least. Some algorithms require a commutative operator with a neutral element. Operators like , ,  are common.
  Since reduce can be interpreted as an inverse broadcast, equal implementation considerations apply (§ Broadcast). For pipelining on binary trees the message must be representable as a vector of smaller object for component-wise reduction.
  Pipelined reduce on a balanced binary tree is possible in .
caption: Information flow of Reduce operation performed on three nodes. f is the associative operator and α is the result of the reduction.
exif: {}