A distributed processing device configured to be used for machine learning and data mining, wherein the device activates at least one of a plurality of processes activated by a plurality of distributed processing devices wherein said at least one process comprises: collective communication means for executing local collective communication among a plurality of processes within one process group among a plurality of process groups generated by partitioning said plurality of processes by a first partitioning pattern, said first partition pattern being configured to form one or more process groups in which local collective communication is executed, by partitioning said plurality of processes; and communication control means for selecting, after the local collective communication, a second partitioning pattern different from said first partitioning pattern used in said local collective communication and being configured to form at least one process group that includes at least one process other than the processes included in the process groups that are formed by the first partitioning pattern, and allowing said collective communication means to execute a next local collective communication using data based on a result of said local collective communication among a plurality of processes within one process group among a plurality of process groups generated by partitioning said plurality of processes by said second partitioning pattern, wherein after said local collective communication is repeated a predetermined number of times, said communication control means retains such a selection order of a plurality of partitioning patterns including said first and second partitioning patterns as to allow data reflecting a plurality of pieces of target data that said plurality of processes have to be shared among said plurality of processes and selects a partitioning pattern to be used for said next local collective communication based on said selection order. collective communication means for executing local collective communication among a plurality of processes within one process group among a plurality of process groups generated by partitioning said plurality of processes by a first partitioning pattern, said first partition pattern being configured to form one or more process groups in which local collective communication is executed, by partitioning said plurality of processes; and communication control means for selecting, after the local collective communication, a second partitioning pattern different from said first partitioning pattern used in said local collective communication and being configured to form at least one process group that includes at least one process other than the processes included in the process groups that are formed by the first partitioning pattern, and allowing said collective communication means to execute a next local collective communication using data based on a result of said local collective communication among a plurality of processes within one process group among a plurality of process groups generated by partitioning said plurality of processes by said second partitioning pattern, wherein after said local collective communication is repeated a predetermined number of times, said communication control means retains such a selection order of a plurality of partitioning patterns including said first and second partitioning patterns as to allow data reflecting a plurality of pieces of target data that said plurality of processes have to be shared among said plurality of processes and selects a partitioning pattern to be used for said next local collective communication based on said selection order.