# Hardware Platform-Aware Binarized Neural Network Model Optimization
## 4 Proposed Architecture Search Solution
### Architectural Search via Deepbit Method
This process is repeated for all the depth values in set L. Consequently, a series of BNN models with optimal width configurations and variable depths is produced. In our experiments, we keep the depth of the network from 3 to 8. The algorithm for phase 3 is explained in Algorithm 3. More specifically, at the beginning, the width of each layer is updated with \(min(optimal\_width[k]+1,optimal\_uniform\_width)\). In the while loop, the width of each layer is increased one to ensure increasing two for initialization. After training 100 times, the best model and corresponding accuracy are stored. If the best accuracy is less than the accuracy threshes, the training process is repeated until we obtain the model with better accuracy than the accuracy threshold. ```
Result:\(optimal\_width\) procedureoptimal_width_search(\(model,thresh,optimal\_uniform\_width,layers\)) fork=1 tolayersdo \(high\_width\gets optimal\_uniform\_width\); \(low\_width\gets 0\); forj=1 tolayersdo \(model[j].width\gets optimal\_uniform\_width\);  end forwhile\(high\_width-low\_width>1\)do \(mid\_width\leftarrow\lfloor(high\_width-low\_width)/2\rfloor+low\_width\); \(model[k].width\gets mid\_width\); \(best\_accuracy\gets 0\); fori=1 tol0 do \(model\gets train(model)\); \(acc\gets test(model)\); ifacc>best_accuracythen \(best\_accuracy\gets acc\);  end if  end for ifbest_acc>threshthen \(high\_width\gets mid\_width\); else \(low\_width\gets mid\_width\);  end if  end if\(optimal\_width[k]\gets high\_width\);  end for
```  
**Algorithm 2**Optimal width search for a given BNN model  
In summary, the entire **Deepbit** algorithm is summarized in Algorithm 4. More specifically, based on the initial inputs, including the number of layers, accuracy threshold, and initial max-width, the \(uniform\_width\_search\) procedure is used to explore \(optimal\_uniform\_width\) (phase 1). Then, the \(optimal\_uniform\_width\) is used to find the \(optimal\_width\) for each layer by using the \(optimal\_width\_search\) procedure (phase 2). Finally, the \(optimal\_widths\) of all layers are used to construct a unique model, and this model is continuously optimized with the \(optimal\_bnn\_search\) procedure until we have the minimum width for each layer, and this final model satisfies the accuracy threshold (phase 3). ```
Result:\(Optimal\_BNN\) procedureoptimal_BNN_search(\(model,thresh,optimal\_width,layers\)) fork=1 to Layersdo \(model[k].width\leftarrow min(optimal\_width[k]+1,optimal\_uniform\_width)\); end  acc \(\leftarrow\) 0; best_accuracy \(\leftarrow\) 0; whilebest_acc \(<\) threshdo fork=1 to Layersdo \(width\leftarrow model[k].width\); \(model[k].width\leftarrow min(width+1,optimal\_uniform\_width)\);  end for fori=1 to100 do \(model\leftarrow train(model)\); acc \(\leftarrow\) test(model); if(acc \(>\)best_accuracy)then  best_accuracy \(\leftarrow\) acc;  optimal_BNN \(\leftarrow\) model;  end if  end for
```  
**Algorithm 3**Optimal BNN search based on optimal width configuration  
The output of the Deepbit algorithm is a list of optimal BNN models. Each model corresponds to a certain number of layers in the L input list. To find the most hardware-friendly model based on this series of optimal BNN models, the target hardware costs for each model are estimated according to the hardware cost estimation method introduced in the previous subsection. Typically, every model uses a uniform width in the hardware cost charts to represent the number of channels per layer. This means the number of channels per layer is the same as the model's uniform width. Hence, to evaluate the hardware cost for an optimal model, the average number of channels per layer can play the same role as the model's optimal width and can be used to search the optimal model based on the hardware estimation charts. Because the estimation charts do not show all hardware costs for all optimal width values, the linear regression method can be further used to estimate the target hardware cost for each model.