# Blind Image Quality Assessment with Deep Learning:
## 3 Experimental Setup
### Replicability Pipeline  
As previously stated, due to the limited resources of a Google Colab free plan, we were not able to replicate the KonCept512 model using the exact same experimental setup of the original authors in [16]. Consequently, we adjusted the provided code for Python3 (www.python.org, accessed on 25 November 2022) Google Colab notebook on our local machine. We updated the code to run with the most recent version of Python (3.8) and TensorFlow (2.6) that includes the Keras API. Unlike the setup used in the original paper, our local machine only provides one GPU (RTX 3080 ti) with the latest CUDA Toolkit (11.4) ([[https://developer.nvidia.com/cuda-toolkit](https://developer.nvidia.com/cuda-toolkit)]([https://developer.nvidia.com/cuda-toolkit](https://developer.nvidia.com/cuda-toolkit)), accessed on 25 November 2022) and GPU-accelerated libraries versions. As the authors do not provide a requirements list with all the needed libraries to run on a local machine, we performed an analysis of the code in order to identify them. We then proceeded to installing their most recent stable versions on the local machine. In the training process, a custom learning rate was applied by first training the training set for 40 epochs with a learning rate of \(10^{-4}\), monitoring the PLCC on the validation set, and saving the best performing model. After training these 40 epochs, the best model is loaded and trained for another 20 epochs with lower learning rate (of \(5\times 10^{-5}\)). The authors then follow the same procedure for another 10 epochs with learning rate of \(10^{-5}\). They do not specify the batch size for the KonCept512 model; however, they used the largest batch size images that fit on the available GPU memory in [9], so we inferred that the batch size would be 16. However, our GPU architecture can not handle a batch size images of 16. We used this apparent limitation as a drive for understanding the effect of the batch size in the training phase. We trained the model three times, with different batch sizes (of 2, 4 and 8). We made another experiment by changing the loss function. In the training process, we trained a model by maximizing the PLCC with a loss functions based on the PLCC metric instead of MSE loss function. MSE loss is a loss function widely-used in BIQA to minimize the error of the trained model. However, in recent studies [19; 20; 21] a PLCC-induced loss has been applied together with other loss functions in order to train models. By using these type of loss functions the models could converge faster and better than with MSE loss [20]. After training this model, a model was trained by decreasing the batch size from 8 to 4, to understand the impact of the batch size in case of using a PLCC-induced loss function. It is important to note that we contacted the authors and asked them to clarify the computations that were performed in the validation stage. Their response was quick and detailed. We also requested for an implementation of the code that would not rely on Google Colab. The authors replied with an updated version of the code, with minor adjustments in terms of libraries versions. However, the updated code also needed to be running within Google Colab and the authors stated they have no experience with running the code outside these computing infrastructures. The updated code that the authors shared were consistent with the initial assumptions we made in order to be able to run the code on our local machine. Nonetheless, we trained two more models using the updated libraries, exactly as proposed by the authors. We followed their code by training the models following the same procedure as the first trained. It should be noted that these models were not trained with same number of epochs as described in [9], instead the authors excluded the final 10 epochs with learning rate of \(10^{-5}\) of the training process.