## 4 Algorithms
### Uniform Quantization-Aware Training  
To train our structured subnetworks for low-precision quantization, we use the straight-through estimator (STE) [39] to approximate the gradients of our rounding function such that \(\
abla_{x}\lfloor x\rceil=1\), but \(\lfloor x\rceil\
eq x\). To apply asymmetric quantization to our hidden activations \(\{\mathbf{h}_{i}\}_{i=1}^{L}\), we adaptively fit our per-tensor scaling factor \(s_{i}\) and zero-point \(z_{i}\). To apply symmetric quantization to our weights \(\{\mathbf{w}_{i}\}_{i=1}^{L}\), we adaptively fit our per-channel scaling factor \(\mathbf{s}_{i}\) where \(s_{i,c}\) denotes the scaling factor for channel \(c\in\{1,\cdots,C_{i}\}\). For each layer \(i\), we estimate the upper and lower bounds of activation \(\mathbf{h}_{i}\) and the maximum magnitude of weight \(\mathbf{w}_{i}\) using moving average statistics, similar to our technique in Algorithm 1. Following the work of [28], we summarize the adaptive asymmetric and symmetric quantization algorithms used in this paper in Algorithms 2 and 3, respectively. ```
Input:\(\left(l_{i}^{(l)},u_{i}^{(l)}\right):=\) estimated bounds on hidden activation \(\mathbf{h}_{i}\) at time step \(t\) Output: Quantized activation \(\hat{\mathbf{h}}_{i}\); Updated bounds \(\left(l_{i}^{(l+1)},u_{i}^{(l+1)}\right)\)
1\(l_{i}^{(l+1)}\leftarrow\left(l_{i}\cdot t+\min(\mathbf{h}_{i})\right)/(t+1)\)
2\(u_{i}^{(l+1)}\leftarrow\left(u_{i}\cdot t+\max(\mathbf{h}_{i})\right)/(t+1)\)
3\(s_{i}\leftarrow(u_{i}^{(l+1)}-l_{i}^{(l+1)})/2^{b}\)
4\(z_{i}\leftarrow\lfloor-l_{i}^{(l+1)}/s\rceil\)
5\(\hat{\mathbf{h}}_{i}\leftarrow\text{clip}\Big{(}\lfloor\mathbf{h}_{i}/s_{i}\rceil+z_{i };0,2^{b}-1\Big{)}\)
```  
**Algorithm 2**Our adaptive asymmetric quantization algorithm for our activations \(\mathbf{h}_{i}\) using per-tensor scaling factors. We use moving average statistics over hidden activation \(\mathbf{h}_{i}\) to estimate the bounds on its dynamic range for the purpose of deriving scaling factor \(s_{i}\) and zero-point \(z_{i}\) for layer \(i\). Figure 5: We depict the differences between the standard stepwise pruning schedule (**a**) and our layerwise pruning schedule (**b**). While stepwise pruning algorithms iteratively increase sparsity globally in the network with each step, our layerwise pruning algorithm iteratively increases sparsity layer-by-layer with each step. Throughout the training progress (horizontal flow), gray nodes denote visible neurons that are not pruned, red nodes denote hidden neurons that have been identified to be pruned in a given step, and white nodes denote hidden neurons that remain active. ```
Input:\(s^{(t)}_{i,c}:=\) estimated scaling factor \(s\) for channel \(c\) of weight \(\mathbf{w}_{i}\) at time \(t\) Output: Quantized weight \(\hat{\mathbf{w}}_{i}\); updated scaling factor \(s^{(t+1)}_{i,c}\)
1 for\(c\gets 1\)to\(C_{i}\)do
2\(s^{(t+1)}_{i,c}\leftarrow\left(s^{(t)}_{i,c}\cdot t+\frac{\max(\mathbf{w}_{i})}{2^{b -1}}\right)/(t+1)\)
3 end for
4\(\hat{\mathbf{w}}_{i}\leftarrow\text{clip}(\lfloor\mathbf{w}_{i}/s_{t+1}\rceil;-2^{b-1},2^{b-1}-1)\)
```  
**Algorithm 3**Our adaptive symmetric quantization algorithm for the set of weights \(\mathbf{w}_{i}\) for layer \(i\) using per-channel scaling factors. We use moving average statistics to estimate the maximum weight magnitude for each channel \(c\) to derive our per-channel scaling factors \(s_{c}\).