## 8 Implementation and Experiments
### Executions of Forward and Backward Computations  
During the forward computations, every input_ is mapped to an output_, possibly with the use of weights_ and an invocation of activation_. As regards the backward flow, firstly, the delta_ tensor becomes computed based on activation_d_ and delta_backward_input_ (the latter stores weighted sums of subsequent \(\delta\)s) in compliance with the inductive Rule 1. Then, the gradient_ becomes computed based on delta_ and input_ in compliance with Rule 2. Finally, the responsibility of each layer is to prepare a delta_backward_output_ tensor, using delta_ and weights_, so that it constitutes a delta_backward_input_ from the perspective of the preceding layer.  
To suitably handle the described forward and backward flows, each layer is equipped with function pointers named do_forward_function, do_backward_function, do_backward, and _output_function. The design of hmdl allows for the provision of more than one implementation for either of those. An actual implementation to be executed can be chosen either manually by the user or automatically via the framework itself in an attempt to improve efficiency (taking into account the sizes of tensors, filters, etc.). The source code shown below (Figure 13) presents two main functions that execute forward and backward computations from within the abstract class Layer. Inside the functions, one can note suitable invocations performed via pointers do_forward_function, do_backward_function, and do_backward_output_function, assumed to be defined by subclasses.  
Figure 12: UML class diagram of _hdml_ project ([[https://github.com/pklesk/hmdl/blob/main/uml/classes.pdf](https://github.com/pklesk/hmdl/blob/main/uml/classes.pdf)]([https://github.com/pklesk/hmdl/blob/main/uml/classes.pdf](https://github.com/pklesk/hmdl/blob/main/uml/classes.pdf))) (accessed on 17 October 2024).  
Moreover, it should be remarked that layer instances belonging to the same class but present in different places of a network structure are allowed to execute different implementation variants of their forward/backward computations. For example, in the Conv2D class, there exist 7 implementation variants that can be inserted into the do_forward_function pointer. They differ with respect to computational resources (CPU: numpy or numba.jit; GPU: numba.cuda) or the algorithmic approach (direct, tiles, or via FFT). We discuss them in Section 8.4.  
Networks built with hmdl are instances of the SequentialClassifier class. Its add(...) method allows the user to append successive layers in a manner similar to [PERSON]'s API [68]. The class is additionally equipped with the fit, predict, and predict_proba methods, following the convention known from the sklearn module. In fact, the SequentialClassifier inherits from two classes, BaseEstimator and ClassifierMixin, that come from sklearn.base. It is also worth remarking that a classifier instance initializes every backward flow induction by calling the self.loss_d_(...) function, a pointer to the categorical_crossentropy_d or square_loss_d. Figure 14 presents the source code of the fit function for SequentialClassifier, whereas Figure 14 presents the core parts of its forward and backward functions involved in the process of fitting.  
Figure 14: The core of the fit function of SequentialClassifier. The main training loop (over epochs and batches) executes forward and backward computations through layers for each batch.  
Figure 13: Functions executing forward and backward computations from the abstract class Layer.