## 5 IO Data Management and Data Sharing
### Distributed SQL Databases with RPC
#### 5.1.1. Replicated Data Bases and Tables
Pre-processing of the input data (creation of training and test data, data normalization, filtering). 3. Model training (here, a CNN), generally the major contribution to the overall computation time. 4. Test and verification. Each shell worker process allocated about 400 MB of working memory for executing all phases. The entire image data set and CNN input data required 1900 \(\times\) 64 \(\times\) 64 \(\times\) 3 \(\times\) 4  
Figure 14: Results of the total computation times, with shell worker processing of multi-model CNN-ML tasks with a different number of workers (PN). S: speeding up, \(\sigma\): scaling up, HPG3: NC = 4, HPZ620: NC = 12. (**a**) Input data loading times from the SQL data base; (**b**) data pre-processing times; (**c**) training times. Figure 9.0 The distributed-parallel worker architectures used for multi-model ML training: (**a**) parallel web workers managed by a WorkBook; (**b**) distributed-parallel shell workers; (**c**) distributed web WorkShell workers; WB: WorkBook, WS: WorkShell, WWS: Web WorkShell. (float32) = 84 MB. Therefore, it is recommended to use 64-bit versions of node.js to access more than 2 GB of memory and computers with at least 2-4 GB of memory/core. The best single-instance training performance was achieved on the embedded HPG3 platform (4 cores) with a 44 ms/iteration/image data volume. The 2 CPU 6-Core HP Z620 workstation shows higher computational times for single-instance training by about 25%. This experiment, with the same ANN architecture and input data, was conducted with the native code Tensorflow 2.0 framework, using the CPU back-end (and an HP Z620 comparable host platform). The single-instance training time was about 20 ms/iteration/image data volume set in this setup. The JS VM processing can compete with this, even if processed in web browsers, shown in Figure 9.1 The experiments considered parallel processing on one physical node utilising multi-core CPU processing. Up to the number of cores Web browsers show appropriate scaling efficiency about 0.7. A distributed-parallel worker-tree with different physical nodes would also scale linearly, except for the data transfer via LAN. Maximal speeding-up \(S=PN\) is achieved in this dynamically growing size problem if the overall computation time keeps constant with an increasing \(PN\), under the assumption that all processes have the same workload (computation time). The scaling up (speeding-up  
Figure 9.2 Results of the total computation times, with web worker processing of multi-model CNN-ML tasks with a different number of workers PN. CF-LX3/LX6: NC = 2. (**a**) Input data loading times from SQL data base; (**b**) data pre-processing times; (**c**) training times. with respect to the number of parallel worker processes, _PN_) is nearly linear, up to the maximal number of CPU cores, i.e., \(S\approx\)_NC_. The total loading time of the input data (1800 images) does not increase significantly up to _PN_ = _NC_ and is independent of the SQL server location (local or remote). The remote SQL data base access only increases the loading time by a factor of two (when using 1 Gb/s LAN; the data consumer and source nodes were connected by the same LAN switch). Note that about 10-30 CNN training iterations (epochs) were required to achieve a good prediction accuracy of about 80-90%. Processing on the Raspberry PI3B platform only allowed one training process, due to memory constraints (a worker requires about 300-400 MB of memory). The loading time of the input data from a remote SQL data base (using 1 Gb/s LAN; the data consumer and source nodes were connected by the same LAN switch) required about 8.0 s, the pre-processing of the input data, about 8.3 s, and the training, about 670 s. This slow down relates approximately to the measured (j)dhrystone value ratio of 1:11. It is well known that node.js underperforms on ARM platforms (the V8 core engine was designed and optimized for x86/x64 platforms).