## 3 Construction  
### Privacy-Preserving CNN Inference  
The high-level framework of our privacy-preserving CNN inference protocol is depicted in Figure 2. As previously discussed, we aim to allow IoT devices to securely offload the linear operations of the convolutional layers and the fully connected layers, which count for over 99.9% of the overall computation of the inference task. The non-linear layers are executed locally on IoT devices. Therefore, the protocol is interactive. More specifically, an IoT device encrypts the inputs of each convolutional layer or fully connected layer and sends the encrypted inputs to the edge server. The latter computes the scalar products of the encrypted inputs and the corresponding weight parameters of the CNN network. The computed scalar products are, therefore, in encrypted forms, which are returned to the IoT device. The IoT device then locally decrypts the returned results and proceeds with non-linear layers, if there are any. This process is repeated until the output layer of the CNN structure is reached. The overall process is depicted in Algorithm 1. ```
Data: Input Data \(\mathcal{X}\), Pre-trained model \(\mathcal{W}\) Result: Inference result \(\mathcal{Y}\) forach layer in \(\mathcal{W}\)do  Layer.Input = \(\mathcal{X}\); ifLayer = Fully-Connected Layer or Convolution Layerthen \(C\) interacts with \(S\) to execute Algorithm 2 and obtains output \(\mathcal{X}^{\prime}\); \(\mathcal{X}=\mathcal{X}^{\prime}\); else \(C\) computes the non-linear functions locally and obtains output \(\mathcal{X}^{\prime}\); \(\mathcal{X}=\mathcal{X}^{\prime}\)  end if  end for
```  
**Algorithm 1**Privacy-Preserving CNN Inference  
In the privacy-preserving CNN inference protocol, one important component is the secure outsourcing of scalar products, which demands the corresponding data encryption/decryption algorithm be _linear_, _lossless_, and _efficient_. In particular, the \"linear\" property is in place to assure that the edge server is able to perform the learning operations as in plaintext models, e.g., it shall be compatible with tensor operations using existing libraries, such as TensorFlow, and conveniently support GPU executions for efficiency. The \"lossless\" property requires that the decryption does not introduce any distortion of the corresponding scalar products, so that the model performance is not impaired by the CNN inference protocol. Last but not least, \"efficiency\" is in terms of not only computation and communication, but also storage available considering contemporary IoT devices,  
Figure 2: Framework of the proposed secure CNN inference protocol. edge servers and wireless networks. To achieve these design goals, we introduce a new interactive privacy-preserving scalar product (iPPSP) primitive that can securely evaluate the scalar product in the four stages depicted in Figure 3: _(1) Initialization_, _(2) Pad Generation_, _(3) Encryption_, and _(4) Report_. In the _Initialization_ phase, the IoT device owner prepares for the initial secrets necessary for the subsequent operation of the protocol. Specifically, our protocol generates a constant-size _random key pool_ to be loaded to the IoT device; a so-called _pre-computation table_, which is also of constant size, is loaded on the IoT device as well. The two types of initial secret will be used as master keys to generate real-time data encryption and decryption keys, respectively. During the _Pad Generation_ phase, the IoT device will randomly generate the encryption keys using the random keys in the key pool; as the encryption is similar to a one-time pad, we call these encryption keys as _encryption pads_. In the _Encryption_ phase, the IoT device encrypts the message with the pad and sends the encrypted message to the edge server, which will compute the scalar product (over the encrypted data) for the IoT device.