# ClipQ: Clipping Optimization for the Post-Training Quantization of Convolutional Neural Network
## 3 Method
### Weight Clipping Optimization
 \tag{6}\]  
ClipQ obtains the optimal clipping thresholds based on particle swarm optimization. For the target convolution kernel \(w\), we initialize the positions and velocities of several particles (as shown in Figure 3b). The position of the kth particle \(P^{k}=(x_{1},x_{2})\) represents the minimum clipping threshold and maximum clipping threshold. Its movement speed \(V^{k}=(v_{1},v_{2})\) indicates the search direction and step size. The initial position and initial velocity of the particle are defined as (6), where \(\Delta\) is a parameter that controls the range of the particle's initial position. We can obtain the quantization deviation of each particle at the current position according to (5). It will function as the fitness function of the particle swarm to guide the particles to converge to the global optimal position. Specifically, the quantization deviation is calculated at the initial positions \((P^{1},P^{2},\cdots,P^{k})\) of each particle at first. The lowest quantization deviation is recorded, which is defined as the global optimal position \(G_{best}\). In addition, the optimal position \((P^{1}_{best},P^{2}_{best},\cdots,P^{k}_{best})\) of each particle is also recorded. After that, their speed is adjusted based on \(G_{best}\) and \(P^{k}_{best}\):  
\[ \begin{split} V_{k}=\gamma\times V^{k}+c_{1}\times r_{1}\times(P ^{k}_{best}-P^{k})\\ +c_{2}\times r_{2}\times(G_{best}-P^{k})\end{split}  \tag{7}\]  
Figure 2: Numerical relationship between quantization deviation and clipping thresholds. Among them, \(\gamma\) is the inertia factor of the original speed, indicating that the particle will retain the initial motion trend to a certain extent. We define it as  
\[\gamma=0.4+0.5\times\frac{T-t}{T} \tag{8}\]  
\(T\) and \(t\) represent the total number of iterations and the current iteration period of the particle swarm, respectively. This means that in the early stage of the particle swarm optimization, a larger \(\gamma\) makes the movement direction of the particles more dispersed, which improves the global optimization ability of the particle swarm. In the later stage, a smaller \(\gamma\) allows the particle swarm to conduct a more granular search near the optimal point. \(P_{best}^{k}-P^{k}\) and \(G_{best}-P^{k}\) represent the distance from the kth particle to its optimal position and the distance from the kth particle global optimal position, respectively. \(c_{1}\) and \(c_{2}\) are learning factors. The larger \(c_{1}\) is, the more the particle tends to learn the experience of itself, approaching its optimal position. The larger the \(c_{2}\) is, the more the particles tend to learn the experience of the whole particle swarm and move to global optimal position. It reflects the cooperation and knowledge sharing among the particles. \(r_{1}\in(0,1)\) and \(r_{2}\in(0,1)\) are random numbers, which provide randomness to the movement of particles and help particle swarms avoid getting stuck in local optima. \[P^{k}=P^{k}+V^{k} \tag{9}\]  
After the particles move to new positions with the new velocity in line with (9) we recalculate the quantization deviation for each particle to update \(P_{best}^{k}\) and \(G_{best}\). Then, the velocity can be re-determined. In the end, most of the particles will converge to the global optimal position as shown in Figure 3c. The global optimal position of the particle swarm is the optimal weight clipping threshold \(G_{best}\rightarrow[w_{min},w_{max}]\).