# Modular Dynamic Neural Network: A Continual Learning Architecture
## 3 Modular Dynamic Neural Network  
Throughout the literature, the main goals for building CL architectures are very similar: to solve the catastrophic forgetting problem while at the same time reducing memory consumption and allowing for scalability. In the present framework, those goals are considered to be of great importance, but for a CL method to be functional and applicable to real-world problems, avoiding catastrophic forgetting is considered to be the most important objective to follow (the same happens in the majority of the papers that deal with CL). With the elimination of catastrophic forgetting being the main focus (and to consider memory and network size constraints as secondary goals), the presented approach prioritizes accuracy. The modular dynamic neural network (MDNN) architecture is comprised of modular sub-networks that, as it learns continuously, continuously grows and re-arranges itself. It is structured in such a way that the modules function independently of one another. This configuration means that, as new classes are learned, only certain sub-networks are modified, making it so that old knowledge is retained. The network is divided into two main components: (a) feature extraction and (b) modular dynamic classification. Figure 1 shows a global flowchart of the MDNN training and production processes, which are later explained in detail in the present section, where it can be seen that the feature extraction component is general to both training and \"production\" (inference) and how, based on decisions made by algorithms that are discussed in the paper, some recursive processes can occur. For the first component of the network, (a) feature extraction, a pre-trained (ImageNet) ResNet50 [1] was used as a backbone, which is known to provide excellent results for image classification; see [24]. This component is only used for extracting generic \"low-level\" features, while smaller and more basic (modular) networks exist for extracting class-specific features in the dynamic classification component. The feature extraction part of the presented architecture is the only component that is never altered when new classes are being learnedbecause the modules in the next component rely on it to be consistent, so if the feature extraction part changed, they would be invalidated. Figure 1: A global flowchart of the MDNN architecture (see the text for a detailed explanation). The second component of the network, the (b) modular dynamic classification component, is made up of numerous small modules. These modules serve the purpose of classifying specific classes or groups of classes that, as new ones are learned, are automatically split into groups of modules and sub-modules based on their class's similarities. This grouping of classes can also be done manually if desired, but the standard operation of the network is to place them automatically based on the algorithms explained in Section 3.2. Figure 2 demonstrates how the modules fit into the proposed architecture: the modules with information inside brackets contain their own sub-modules (e.g., \([X_{1},X_{2},\ldots,X_{n_{x}}]\)) or groups of sub-modules in the case of nested brackets (e.g., \([[X_{1},X_{2},\ldots,X_{n_{x}}],\ldots]\)), where \(X_{1}\) to \(X_{n_{x}}\) are modules with no children and \(n_{x}\) represents the number of sub-modules belonging to their parent. The modules containing their own sub-modules are designated as _node_ modules and they have one binary classifier per direct child module. The modules with no children are named _endpoint_ modules and contain nothing but feature data obtained during the training process. As the network learns new classes, the extracted feature values are stored within the _endpoint_ modules for future use, specifically for when some binary classifiers have to be retrained in order to avoid confusion with a new class.