## 3 PSO Parallelization Based on Apache Spark and RDD Applied to Training Neural Networks
### Distributed Asynchronous PSO  
The distinction between DAPSO and DSPSO lies in their differing approaches to synchronization. Unlike DSPSO, DAPSO does not employ iterations that link all particles together. In DAPSO, each particle is evaluated and moves independently of the other particles. This serves to further increase its degree of independence. This section will present the implemented DAPSO variant, which is distributed and asynchronous and uses Spark RDDs to parallelise the updating of particle positions and velocities. Spark is employed with its low granularity transformations, with the result that the swarm is divided into several subswarms, which are then evaluated in parallel. The particles are updated according to the current state of the entire swarm. This entails modifying both the position and the velocity of each particle as soon as the fitness function is evaluated. In this process, the best global position thus far is taken into account. This results in the complete independence between the particles, which move to their subsequent position with the information available to them at the time of each particle's evaluation. The DAPSO distributed algorithm follows the master-worker paradigm, whereby a master node is responsible for coordinating the remaining worker nodes, which are tasked with performing the computations sent by the master. This is illustrated in the workflows presented in Figure 3.  
Each particle in the swarm is capable of autonomous movement and evaluation of the fitness function. The Master node process consists of the following steps:  
1. Initializing the particle parameters, positions, and velocities;
2. Initializing the state of the swarm, as well as that of a particle queue to send to the worker nodes;
3. Loading the initial particles into the queue;
4. Distributing the particles from the queue to the workers;
5. Waiting for the results of all evaluated particles (with the calculated fitness function) of the subswarm and the personal best position \(P_{id}\) for this iteration;
6. Updating the velocity vector \(V_{id}\) and position vector \(X_{id}\) of each particle based on the personal best position \(P_{id}\) and the global best position \(P_{bg}\) found by all the particles;
7. Placing the particles back in the queue and returning to step 4 if the stop condition is not satisfied;
8. Returning the best global position \(P_{bg}\). The process of worker nodes is similar to the synchronous variant, but it now processes particle subswarms. In order to enhance efficiency, the DAPSO algorithm employs an abstraction called SuperRDD, which comprises a set of interdependent particles operating within the cluster as a unified subswarm. In the context of Spark, a SuperRDD is constituted by the aggregation of multiple individual RDD particles, which are subsequently evaluated. Figure 3: (**a**) Master node flowchart with subswarms of size 1. (**b**) Flowchart of a worker node. In contrast with the previous DSPSO algorithm, whereby each particle was sent to the cluster independently, the particles are now grouped into subswarms of variable size, i.e. between one and the total number of particles, denoted by variable \(S\). As the size of the subswarm increases, the asynchrony of the algorithm is reduced, as a greater number of particles become dependent on each other. Nevertheless, this modification enhances the efficiency of the algorithm by removing some of the computational overhead associated with communication between nodes. In this way, and as the number of particles in the RDD increases, fewer Spark jobs need to be scheduled, reducing the communication overhead. However, this results in the particles having to wait for the rest of the RDD to be evaluated by the node. As for the implementation in Scala and Spark, two fundamental concepts are essential for optimal functionality: an accumulator that receives the evaluated particle data (or SuperRDDs) from the worker nodes and a mechanism for distributing these particles to the worker nodes.