# Modular Dynamic Neural Network: A Continual Learning Architecture
## 3 Modular Dynamic Neural Network
### Modular Dynamic Classification
In the future, this classifier will be the object of further studies in a way to improve the scalability of the network and save memory. It is important to justify at this point, the reason behind having a neuron representing the false value when it is not used. The reason is quite trivial: it is much more straightforward to implement and train a binary classifier this way, as this allows for the use of standard backpropagation for the true and false classes as if they were two normal classes. Nevertheless, in the future, the false values will be taken into account and also become part of the decision-making algorithms. Figure 4 shows a more detailed view of the MDNN architecture (complementing Figure 2): the left side shows the ResNet50 [28] feature extraction, with the 2048 vector of extracted features, and the right side shows the network's flexibility and how the number of sub-modules that a _node_ can have is not limited, including sub-modules of sub-modules. The figure also shows the use of the maximum value for deciding which sub-module should be used to continue the process. For the proposed network to meet its purpose of distinguishing between classes, the minimum number of classes for initialization is two. Therefore, the root module will always be a _node_, as _endpoints_ only ever represent one class and _nodes_ are the only modules that can contain more modules. It is worth remembering that whenever a binary classifier is called upon, no matter its position in the network, the input values will always be the same feature values (2048 length vector) which are extracted once from the data sample being classified. The positions of modules within other modules are symbolic and serve only to decide if and which other modules are to be used, but the extracted feature data are never altered. This is important to clarify because, when visualizing tree-like structures that are based on NN, one might easily mistake the presented architecture for a single network with a tree format, similar to what is shown in [29], where everything is backwards dependent. Therefore, it must be emphasized that the \"tree\" serves only as a representation of the order in which things are done and which data are used by each binary classifier. The main takeaway from this is that the feature data are not altered intermediately, i.e., the input data for the last modules are the same as those for the first modules. The nesting of modules within modules means that the classification process can be recursive. This is because, when applied to a _node_, the results dictate whether it should be re-applied to another sub-_node_, i.e., the classification process is (i) initially applied to the network's root _node_ and then, potentially, (ii) recursively applied to more _nodes_ depending on the intermediate results. So, to make a classification, the first step is for all the binary classifiers in the current _node_ to process the extracted features and then for all their outputs to be evaluated. Keeping in mind that these binary classifiers indicate the certainty that the input sample data belong to the module they represent, the application of the input sample's extracted feature values to each of the binary classifiers (displayed in Figure 4 as \(BC\), with \(BC(X)\in[0,1]\)), results ina group of values between 0 and 1 (because of the final layer's sigmoid activation function) that represent the likelihood of the input sample belonging to each of the modules in the _node_ being analyzed. Then, the highest value from these results is sought out, which indicates which module is the most likely to include the correct class. This selected module can then either be an _endpoint_ or a _node_, where each one has a different next step. If the chosen module is of the _endpoint_ type, then the classification process ends here, and the output prediction is the class represented by that _endpoint_.