# On-the-Fly Machine Learning for Improving Image Resolution in Tomography
## 5 Discussion  
The experimental results demonstrate the feasibility of applying our combined acquisition scheme and on-the-fly machine learning technique to improve the resolution of tomographic reconstructions. Although our approach already achieves substantial improvement in image resolution, we believe our approach can still be improved. In this section, we discuss improvements for the training and architecture of the neural network, and how the acquisition geometry may be adapted to less self-similar objects.  
We believe that training the neural network can be accomplished faster and that the training procedure can be made less sensitive to small changes in alignment. In its current form, the computations of the learning step take considerably longer than the reconstruction step. In fact, the reconstruction for both the simulated data and the experimental data took less than fifteen minutes, while the learning step took up to two days. By using a validation set, for instance, the training can be cut short when the validation error does not improve. Sacrificing part of the training set for validation appears possible without making the training set too small for the network to learn a useful transformation from low- to high-resolution data. Based on the results in Figure 11, we expect that training for a considerably shorter period is possible, and will lead to results comparable to those reported in this paper. The neural networks were trained using the mean square error, which is sensitive to small changes in alignment. Hence, the alignment of the low-resolution and high-resolution reconstructions is critical, but can in practice not always be guaranteed. Therefore, using an error metric that does not depend on the exact alignment of the input and target data could improve robustness. Resolving this problem has received considerable attention, for instance by learning the loss function in an adversarial setting [17; 44]. Finally, training time could be further improved in batch settings by pre-training the network with an object that is similar to the object under investigation.  
The MS-D network architecture could be changed to obtain better results. Since we have not found the MS-D network to overfit to the training data, a deeper MS-D network with the capacity to exploit more input features might be able to increase resolution even more. Furthermore, we believe that a fully three-dimensional convolutional neural network might be able to achieve even better results on this task. In the synthetic data, we have found some qualitative visual difference in the output of the networks with a single input slice and those with nine input slices. This suggests that three-dimensional information can be useful for transforming low-resolution data to high-resolution data.  
The experimental results indicate that, if objects are self-similar, super-resolution can be obtained using our method, thus providing a super-resolution approach that does not rely on a training set  
Figure 11: Oatmeal dataset, method B: the mean square error for MS-D and U-net calculated on the central slice of the training and test set. The training error decreases as the networks are trained longer. The test error of the U-net increases, whereas the test error of the MS-D network remains stable.  
of similar objects. There are cases, however, where not all parts of the object have similar structure. For example, an object may consist of multiple parts with different characteristics. In that case, the network may be trained on multiple ROI reconstructions in these parts to ensure that it is able to improve the resolution of the various structures within the object. The areas where the local structure are different can usually be identified using just the low-resolution reconstruction.