# Reducing Model Complexity in Neural Networks by Using Pyramid Training Approaches
## 4 Experiments
### Experimental Configurations  
Different network configurations were tested to examine the impact of feature size reduction and quantization:  
* **BaseNetwork**: this configuration serves as a control, comprising Block 1 and Block 2 combined without any feature size reduction steps.
* **ReducedNetwork**: incorporates a feature size reduction step between Conv2 and Conv3 layers. This network undergoes iterative pyramid training, progressively refining the model and reducing its complexity by minimizing redundant feature representations.
* **Quantization**: both BaseNetwork and ReducedNetwork are subjected to quantization to compress model size and enhance computational efficiency.
* **Reduced+Quantization**: this setup investigates the synergistic effect of combining feature size reduction with quantization, focusing on the rate of model size reduction and operational efficiency.
* **U-Net**: employed for semantic segmentation, this network's architecture is designed to efficiently segment images by progressively reducing and then expanding the resolution of intermediary representations.
* **U-Net-Reduced**: a simplified variant of U-Net, where feature size reduction techniques are applied to decrease the complexity and computational demands of the network while aiming to maintain effective segmentation performance.
* **Faster R-CNN**: utilizes a two-stage approach for object detection, combining region proposal networks with a CNN classifier. The backbones used in these experiments include CustomBackbone and VGG16, adapted for this framework.  
In the classification task in Table 4, BaseNetwork achieves a test set accuracy of 77.31% with a model size of 0.65 MB and 23.08M FLOPS. ReducedNetwork, designed to decrease complexity, has a slightly lower accuracy of 74.77% but significantly reduces the model size to 0.17 MB and FLOPS to 12.7M. When quantization is applied, the BaseNetwork's model size is halved to 0.32 MB, maintaining similar FLOPS, while the Reduced+Quantization setup shows the smallest size at 0.10 MB and retains the reduced FLOPS of 12.7M.  
For the segmentation task in Table 5, the U-Net model presents a train loss of 1.004, test loss of 1.062, and validation loss of 0.8484, with a model size of 31 MB and 48.4B FLOPS. The U-Net-Reduced model, implementing feature size reduction, achieves a lower train loss of 0.6522 and test loss of 1.027 but has a slightly higher validation loss of 0.8945. The model size is significantly reduced to 7 MB, and FLOPS is also reduced to 12.1B, demonstrating the efficiency of the reduction technique.  
In the object detection task in Table 6, the CustomBackbone model shows an overall loss of 1.14, with the specific losses for box regression and classifier being 0.3894 and 0.4018, respectively. This model requires 15.84 GB of memory and has 8.4B FLOPS. The CustomBackbone-Reduced variant, though it incurs a slightly higher overall loss of 1.223, reduces memory allocation to 10.31 GB and FLOPS to 5.6B, indicating better efficiency. The VGG16 model, used for comparison, achieves lower losses but requires significantly more resources, with 214.2B FLOPS and 16.04 GB of memory. The VGG16-Reduced model, while showing an increase in losses, brings down the FLOPS to 208B and memory usage to 9.96 GB, reflecting the trade-offs involved in model size reduction.  
The FLOPS and model size results across all tasks highlight the efficiency gains from applying the proposed feature size reduction techniques. These reductions in FLOPS and model sizes illustrate the computational savings, which are particularly beneficial in environments with limited resources. The balance between model accuracy, size, and computational efficiency demonstrates the practicality and effectiveness of the proposed methods in various machine learning tasks.