# Hardware Platform-Aware Binarized Neural Network Model Optimization
## 4 Proposed Architecture Search Solution
### Architectural Search via Deepbit Method
During the binary searching process, the model with the width being the midpoint of the range is trained. If the accuracy is better than the accuracy threshold, the upper half of the searching range is selected for the subsequent searching round. By contrast, the lower half of the range would be selected for the subsequent searching round. The final output \(optimal\_uniform\_width\), corresponding to the certain number of layers, is stored to use for the second phase. Figure 6: An example of accuracy deviation on RRAM devices (40 nm, 20 nm, 10 nm), effected by IR drop and sneak path, in which Bittline number represents the number of channels on each layer, and read inaccuracy is accuracy deviation: (**a**) accruacy deviation with a number of bitline from 1 to 8; (**b**) accuracy deviation with a number of bitline from 1 to 32. In the second phase, our method tries to find the optimal width for each layer. This method is performed on the output of phase 1. In this phase, the uniform width of our network is reduced to the minimum width of the network found in phase 1. A binary search is used again to find the optimal width for the first layer, while the width for the other layers is set constant to the optimal uniform width. Once the optimal width for the first layer is found, searching the optimal widths of the rest layers is performed with the same process. During the search for a typical layer, the widths of the rest layers need to be kept with the initial value, even for those that have performed the optimal width search. All optimal width values for all convolution layers are stored and resumed for the next phase of the searching method. The algorithm for phase 2 is described in Algorithm 2. Similar to Algorithm 1, the searching process is performed in an initial range, in which \(high\_width\) is the \(optimal\_uniform\_width\) from phase 1, and \(low\_width\) is zero. However, as mentioned before, the searching is executed for each layer while keeping other ones with the \(optimal\_uniform\_width\). ```
Result:\(optimal\_uniform\_width\) procedureuniform_width_search(\(model\),\(thresh\),\(max\_width\),\(layers\)) \(high\_width\leftarrow\)\(max\_width\); \(low\_width\gets\)\(0\); while\(high\_width-low\_width>1\)do \(mid\_width\leftarrow\lfloor(high\_width-low\_width)/2\rfloor+low\_width\); fork=1 tolayersdo \(model[k].width\leftarrow\)\(mid\_width\);  end while \(best\_accuracy\gets 0\); fori=1 to10 do \(model\leftarrow\)\(train(model)\); \(acc\leftarrow\)\(test(model)\); ifacc>best_accuracythen \(best\_accuracy\leftarrow\)\(acc\);  end if  end if ifbest_acc>threshthen \(high\_width\leftarrow\)\(mid\_width\); \(optimal\_uniform\_width=\)\(mid\_width\) else \(low\_width\leftarrow\)\(mid\_width\);  end if  end if
```  
**Algorithm 1**Optimal uniform channel search for a given BNN model  
In the third phase, which is also the final phase of the searching method, the \(optimal\_width\) of each layer from the second phase is used to define the optimal BNN model. Since all the layers were set to a uniform width while searching for an optimal width for a specific layer, the desirable accuracy degradation can be expected when combining all of the \(optimal\_width\)s of each layer. To compensate for the accuracy loss, the width of each layer is increased by two more channels for each layer. If the width of each layer is greater than the model's \(optimal\_uniform\_width\) in the first phase, the \(optimal\_uniform\_width\) is applied for this layer instead of using the optimal value plus 2. The next step is the training process on the new BNN model with reconfigured width values. If the accuracy matches the threshold accuracy, the model is selected as the optimal BNN model with the given initial depth. If the accuracy is below the threshold, the width of each layer continues to increase one, and the model is retrained to check the accuracy. This process is repeated until the optimal model with accuracy matching the threshold is achieved, or the width of any layer is equal to the \(optimal\_uniform\_width\) in phase 1.