# _teaching_ history = model.fit(train_dataset,epochs=epochs, callbacks= callbacks)
#### 4.2.4 Methodology Supplement  
Research with this application was only conducted for the scenario in which a parallel application operates in Docker containers, with its processes distributed among many containers in such a way that each container runs a maximum of one process. Such a distribution strategy forces the MPI implementation to engage the network stack for inter-process communication. There is a need to pass two environmental variables to the script. When running a program through docker exec in a container shell, it is possible to define custom environmental variables for it using the -e argument, as in Listing 7. The PYTHONHASHSEED variable sets the seed for the hash() hash generator. Assigning it a specific constant is one of the necessary steps to ensure reproducibility of neural network training. The TF_CPP_MIN_LOG_LEVEL variable allows filtering the messages printed by the TensorFlow module on the console screen. Assigning a value of two to it means silencing regular information and warnings, while errors will still be displayed. ```
dockercomposeexec-ituuser-ePYTHONHASHSEED=0-e TF_CPP_MIN_LOG_LEVEL=2 master<command>
```  
**Listing 6**. Horovod initialization and configuration. Based on [45] and documentation. To ensure that the given environmental variables are properly propagated to all parallel processes run by mpirun, it is necessary to use the appropriate option in the command line. In MPICH, this is achieved with the -genvlist argument, after which a list of variable names separated by commas is provided (Listing 8), and in OpenMPI, it is the -x argument, after which a single variable can be listed (Listing 9). To run a single test, the command from Listing 10 is used. The script conducts neural network training on data from the data.csv file. The provided options have the following meanings:  
- --distributed--distributed mode (initiates the Horovod environment and enables MPI support),  
- --epochs100--one hundred training epochs,  
- --batch64--batch size of sixty-four sequences,- --seed 1234--fixed setting of pseudorandom number generators' seed (to ensure reproducibility of the resulting neural network),
- --no-early-stopping--no early termination of training before the set number of epochs,
- --no-output-model--the model will not be saved to a file (not needed for research),
- --silent--turning off the display of information about the progress of training. Running a test for the \"MPICH Network\" scenario. Propagation of environmental variables. mpirun -np ${i} -genvlist TF_CPP_MIN_LOG_LEVEL,PYTHONHASHEED < program>  
Running a test for the \"OpenMPI Network\" scenario. Propagation of environmental variables.  
mpirun -np ${i} -TF_CPP_MIN_LOG_LEVEL -x PYTHONHASHEED <program >  
Executing a single test with the anomaly detection application. python src/train.py --distributed --epochs 100 --batch 64 --seed 1234 --no-early-stopping --no-output-model --silent data.csv  
The chart in Figure 4 shows the average execution times of the ML application for the MPICH and OpenMPI implementations. The time gain achieved due to distributed computing was not as satisfying as with applications for prime number searching and integration. With the MPICH implementation, neural network training averaged the shortest duration, with two processes, and with OpenMPI, with three processes (Table 2). For larger numbers of processes, the average test times noticeably increased. MPICH performed worse in this regard, as the tests started to take longer from four processes upwards, compared to running exactly one process of the tested application. The same can be said for OpenMPI, but from seven processes upwards. In terms of performance, MPICH performed better than OpenMPI, especially with one and two processes. According to Table 11, training with MPICH was almost 41% faster in those cases. For numbers of processes from three to eight, the average test times for both implementations were practically the same.