# iCaps-Dfake: An Integrated Capsule-Based Model for Deepfake Image and Video Detection
## 3 Materials and Methods
### Classification--Capsule Network  
Regular CNNs are built based on scalar neuron representations that are capable of identifying probabilities of existence of particular features. To better detect a feature, CNNs require lots of different variants of the same type of data entity which in turn needs extra neurons leading to size and parameters expansion of the entire network. On top of that, pooling operations done by CNNs only propagate features that stand out and drop all other features thus losing a lot of spatial information. Unlike CNNs, CapsNets exploits all the spatial information to identify both the probability of feature existence as well as its location relative to others (pose information) making it viewpoint-invariant thus requiring less data. First introduced by [PERSON] et al. [58], a capsule is a group of neurons in a shape of a vector encoding the pose information whose length represents the probability of an entity existence (activation probability). This makes deriving a part-whole relation given the embedded information in one computational unit representing the part easier. For example, a face's components (eye, mouth, nose...etc.) would be embedded in the lower capsule levels along with their relative position. The routing algorithm then tries to connect each lower-level capsule to a single higher level one containing its specifications. In EM Routing [59], [PERSON] et al. represented the pose by a matrix and the activation probability was determined by the EM algorithm. The proposed model used the inverted dot-product attention routing algorithm [69] that applies a matrix-structured pose in a capsule. As demonstrated in Figure 8, our capsule network consists of one primary, two convolutional and two class capsules one for each class. The extracted features are fed to the primary capsule that is responsible for creating the first low-level capsules. The primary capsule applies one convolutional layer on to the extracted features then normalize the output and reshape it to create matrix-capsules of size \(\mathbb{R}^{\sqrt{d_{L}}\ x\sqrt{d_{L}}}\) where \(d_{L}\) represents the grouped number of hidden layers that define a capsule. Capsules in the primary layer (children) are used to update capsules in the first convolutional capsule layer (parents) that in turn update their parents and so forth. The convolutional capsules layers are formed using Equations (1)-(5), each containing 32 capsules of size 4 \(\times\) 4. Figure 8: Capsule Network Structure. To route a child capsule \(i\) in layer \(L\) (\(p_{i}^{L}\)) to a parent capsule \(j\) in layer \(L+1\) (\(p_{j}^{L+1}\)), a vote \(v_{ij}^{L}\) is created per child for each parent by applying weights assigned between them \(w_{ij}^{L}\). Initially, all parents' poses \(p_{j}^{L+1}\) are set to zero. \[v_{ij}^{L}=w_{ij}^{L}\cdot p_{i}^{L} \tag{1}\]  
By applying the dot-product similarity, the agreement \(a_{ij}^{L}\) between each parent all children are calculated using their votes \(v_{ij}^{L}\). \[a_{ij}^{L}=p_{j}^{L+1^{T}}\cdot v_{ij}^{L} \tag{2}\]  
A routing coefficient \(r_{ij}^{L}\) is calculated by passing the agreements scores through a softmax function. \[r_{ij}^{L}=\exp\left(a_{ij}^{L}\right)/\sum_{j^{\prime}}\exp\left(a_{i^{ \prime}j^{\prime}}^{L}\right) \tag{3}\]  
Each child contributes in updating parents poses according to its vote \(v_{ij}^{L}\) and routing coefficient \(r_{ij}^{L}\). \[p_{j}^{L+1}=\sum_{i}r_{ij}^{L}v_{ij}^{L} \tag{4}\]  
Finally, a Normalization layer [70] is added to enhance the routing's convergence. \[p_{j}^{L+1}=\text{LayerNorm}\left(p_{j}^{L+1}\right) \tag{5}\]  
Equations (1)-(5) show the calculation steps of the capsule layers using the inverted algorithm. The first iteration is sequential, where the values of all but the first capsule layer is computed. The following iterations are concurrent thus resulting in an improved training performance.