## 8 Implementation and Experiments
### Reproducing Experiments (or Running New Ones)  
Figure 28 presents the list of main settings that can be imposed on an experiment using the experimenter.py script. The very first variable EXPERIMET_ID decides whether an existing experiment is going to be reproduced (in which case its 10-digit identifier is expected) or a new one executed (in which case the None value is expected). The example setting shown in the figure, EXPERIMET_ID = \"1224383397\", corresponds to the 8 th experiment carried out on MNIST data set (refer back to Table 2). We remind the reader that log files from all reported experiments are available in the repository ([[https://github.com/pklesk/hmdl](https://github.com/pklesk/hmdl)]([https://github.com/pklesk/hmdl](https://github.com/pklesk/hmdl))) in the folder experiments/ (accessed on 17 October 2024). Identifiers of those experiments are visible in the file names and also inside the logs. Once the EXPERIMET_ID value is set, the user can carry out an experiment using the standard Python command, i.e., python experimenter.py or python3 experimenter.py.  
The two subsequent variables, UNPICKLE_DATA and PICKLE_DATA, are provided for the user's convenience and they allow, respectively, for reading and writing data arrays from/to a binary file (based on Python's pickle module). Data sets such as MNIST and CIFAR-10  
Figure 28: Main settings for an experiment in script experimenter.py.  
are fairly large in size, and the setting UNPICKLE_DATA = False causes those original data sets to be imported from within the keras.datasets module and read via mnist.load_data() or cifar10.load_data(), which may be time-consuming. By setting UNPICKLE_DATA = True, one gets the data arrays faster as they are read from pre-prepared pickled binary files.  
It is also worth explaining the settings prefixed by COPY_INITIAL_WEIGHTS_ and ending with either KERAS_TO_HMDL or HMDL_TO_KERAS. They let the user initialize (or read) weights for a certain network architecture in one of the frameworks and then copy those exact weights to the same architecture in the second framework. This allows for two kinds of comparisons. One is to check whether forward passes of the two networks (from Keras and from hmdl) for the same batch of data are consistent, e.g., how many significant digits of the floating-point response from final output layers agree. The other kind of comparison is to see for how long the learning processes initiated from the same weights remain consistent. We believe the names of other main settings in Figure 28 are self-explanatory.  
Figure 29 exemplifies how a structure for a neural network becomes described in the hmdl project (within the __main_ function of the script experimenter.py). The description is formed as a list of tuples. The first tuple defines the general settings for the SequentialClassifier. The other tuples define the subsequent layers, each described by its class name (e.g., Conv2D, MaxPool2D, Dropout, Flatten, and Dense) and a dictionary of its properties.  
Finally, Figure 30 shows the code excerpt (in experimenter.py) reponsible for the choice of which data set to experiment with and, in particular, the choice of the randomization _seed_. The same value for the randomization seed implies the same experimental results for all executions on a fixed GPU device (and independently of the CPU).  
Figure 29: Description of a network structure in the hmdl project, declared in experimenter.py.  
data_description = {  \"name\": \"mnist\", # possibilities: \"mnist\", \"olivetti\", \"cifar-10\"  \"m_train_per_class_limit\": None, # specify None if unlimited  \"m_test_per_class_limit\": None, # specify None if unlimited  \"uniform_scaling\": True,  \"seed\": #,  \"n_repetitions\": 1 } ```
**Figure 30**. Choice of data set, randomization seed, and other settings in the script experimenter.py.