# Gap between Prediction and Truth: A Case Study of False-Positives in Leakage Detection

Authors: Pengbo Wang; Ming Tang; Shoukun Xiang; Yaru Wang; Botao Liu
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Since leakage detection was introduced as a popular side-channel security assessment, it has been plagued by false-positives (a.k.a. type I errors). To fix this error, the previous solutions set detection thresholds based on an assumption-based prediction of false-positive rate (FPR). However, this study points out that such a prediction (of FPR) may be inaccurate. We notice that the prediction in EuroCrypt2016 is much smaller than (approximately1/779 times) the true FPR. The gap between prediction and truth, called underpredicted false-positives (UFP), leads to severe false-positives in leakage detection. Then, we check the statistical distribution of test statistics to analyze the cause of UFP. Our analysis indicates that the overlap between cross-validation (CV) blocks gives rise to an assumption error in the distribution of the CV-based estimates of ρ-statistics, which is the root cause of UFP. Therefore, we tackle the UFP by eliminating the overlap between blocks. Specifically, we propose a profiling-shared validation (PSV) and utilize this validation to improve the detection of any-variate any-order leakages. Our experiments show that the PSV solves the UFP and saves more than 75% of the test time costs. In summary, this article reports a potential flaw in leakage detection and provides a complete analysis of the flaw for the first time.


---

## Body

## 1. Introduction

Side-channel attack (SCA) utilizes the physical leakages (such as execution time [1], power consumption [2], and electromagnetic radiation [3]) of a running device to retrieve some secrets (e.g., the private key) inside the device. Since it was proposed by Kocher [1], such an attack has seriously threatened the security of cryptographic modules, including smart cards [2, 3] and FPGAs [4, 5]. Thus, side-channel security assessments have been developed to evaluate the security of these modules against SCA [6].Leakage detection is a side-channel security assessment that determines whether the leakages depend on the data (e.g., plaintext or ciphertext) accessible to an attacker [7]. In most cases, it uses a hypothesis test to identify the data-dependent leakages by comparing the test statistics with a certain threshold [8–10]. If no statistics exceed the threshold, the module is allowed to pass the test; otherwise, the module is rejected. In contrast to the attack-based assessment [6], leakage detection exploits the dependency between leakage and data rather than the key recovery. Hence, it has advantages in computational complexity and works well in black-box environments [10]. A typical example is the test vector leakage assessment (TVLA) [8]. This assessment utilizes Weltch’s t-test to compare t-statistic with a threshold of 4.5 and identifies the leakages dependent on the plaintext. In EuroCrypt2016, Durvaux et al. found that TVLA failed to detect the plaintext-dependent leakages [10]. Then, they put forward a correlation-based leakage detection (ρCV-test) to identify the hard-to-detect leakages for TVLA. The ρCV-test takes advantage of cross-validation (CV) to obtain a well-estimated ρ-statistic r^z,CV and then compares the r^z,CV with a threshold of 5.0 to assess any-variate any-order leakages. Based on the ρCV-test, more assessments have emerged, such as the χ2-test [11], T2-test (and its simplification D-test) [12], KS-test [13], ANOVA [14], and DL-based test [15].However, as discussed in [10], some of the identified plaintext-dependent leakages are in fact not related to the plaintext. In other words, leakage detection is challenged by false-positives (a.k.a. type I errors). To address this challenge, the frequently used solutions carefully set the threshold for leakage detection according to an acceptable false-positive rate (FPR) [16, 17]. Such solutions rely on an assumed distribution of the test statistics to make a fast prediction of FPR. For example, Durvaux et al. at EuroCrypt2016 chose 5.0 as the threshold of the ρCV-test based on the assumption that r^z,CV follows the standard normal distribution (when ρ-statistic rz=0) [10]. This choice corresponds to a theoretical p value (i.e., the predicted FPR of a single test) below 10−6 and reduces the predicted FPR (PFPR) of 104 tests to 0.0057 [17].Obviously, the above solutions demand that the assumption must be close enough to the true distribution of the test statistics; otherwise, an assumption error (AE) in the distribution may lead to an inaccurate prediction of FPR. As far as we know, the potential risk of AE has not been studied in previous work.This work studies the false-positives in theρCV-test and focuses on the potential AE in [10]. We first propose an estimation algorithm that statistically approximates the true FPR of the ρCV-test. After running the algorithm, we notice the underpredicted false-positives (UFP) that the true FPR (of ρCV-test) is about 779 times the prediction (in [10] at the threshold of 5.0). The UFP implies that the previous prediction of FPR may be unreliable. Second, we discover an AE in the statistical distribution of r^z,CV. Due to the overlap between the training and the test blocks, there is a nonnegligible error between the assumed distribution and the true distribution (of r^z,CV). This error explains well why the PFPR of the ρCV-test deviates from the truth. Third, we present a new time-efficient validation, named profiling-shared validation (PSV) to tackle the UFP (and AE). The PSV splits the samples into m+k nonoverlapping subsets and assigns these subsets to different blocks—m subsets are allocated to the training block, and the other k subsets are mutually exclusive to k test blocks. The experiments show that our PSV not only solves the UFP (and AE) but also reduces the time cost by more than 75%.The rest of this study is organized as follows. In Section2, we introduce the underpredicted false-positives. Then, the UFP and AE are analyzed in Section 3. Next, we elaborate on the PSV-based leakage detection in Section 4. Section 5 applies the PSV to high-order leakage detection, and Section 6 summarizes the whole study and draws our conclusions.


## 2. Underpredicted False-Positives

In [10], FPR of the ρCV-test was predicted based on an assumed standard normal distribution. This section shows that the prediction in [10] is not accurate. In this section, we first review the details of the ρCV-test. Then, we introduce our estimation algorithm and describe the underpredicted false-positives.


### 2.1. Correlation-Based Leakage Detection

The correlation-based leakage detection (ρ-test) takes advantage of a ρ-statistic rz to identify the plaintext-dependent leakage by comparing the absolute value rz to a detection threshold Tdet. In [10], Durvaux et al. utilized the k-fold CV to estimate the statistic rzτ at time point τ as follows.First, an assessor randomly splits the acquired tracesℒ into k nonoverlapping subsets ℒi of approximately the same size.Then, forjth1≤j≤k fold of CV, the assessor defines the profiling block ℒpj:=ℒ/ℒj and the test block ℒtj:=ℒj. Thus, a CV-based model model^τ,CVj is profiled from ℒpj:(1)model^τ,CVjX⟵ℒpj.The sample mean isμ^τ,CVjX, where X is the associated data.Next, compute the Pearson correlation coefficient between the modelmodel^τ,CVj and the test leakages Ltjτ⊂ℒtj:(2)r^CVjτ=ρ^Ltjτ,model^τ,CVjXtj,where Xtj is the data associated with the test leakages Ltj. An unbiased estimate r^CVτ of the correlation coefficient is obtained by combining the k coefficients r^CV1,…,r^CVk:(3)r^CVτ=1k∑j=1kr^CVjτ.Finally, after Fisher’sz-transformation and normalization, the CV-based estimate of rzτ is(4)r^z,CVτ=N−32ln1+r^CVτ1−r^CVτ.where N is the size of the set ℒ. Based on the assumption that r^z,CV can be interpreted by a normal distribution Nrz,1, the FPR of (a single) the ρCV-test can be predicted by(5)α˜Tdet=2×1−CDFN0,1Tdet.where CDFN0,1⋅ is the cumulative function of the standard normal distribution N0,1.


### 2.2. Statistical Estimation of FPR

In statistical estimation, FPR is calculated as the ratio of the numberNFP of false-positives to the number NAN of actual negatives:(6)α=NFPNAN.However, neitherNFP nor NAN is known to the evaluators in leakage detection. Therefore, we define a data-independent leakage set (DILSET) to statistically approximate the true FPR of the ρCV-test. DILSET is a leakage set, where each leakage L is independent of the associated data X′, L╨X′. Thus, rzτ≡0, and all the L are negatives for the ρCV-test. By rewriting (6), FPR of the ρCV-test can be estimated:(7)α^Tdet=Nrz>TdetNρ.where Nρ is the number of ρCV-tests on DILSET, and Nrz>Tdet is the number of (false) positives rz>Tdet in the Nρ test results.The statistical estimation is formally described in Algorithm1. Taking the trace set ℒ and the data vector X as input, the algorithm first creates a DILSET ℒ,X′, where the ρ-statistic rz≡0. In each loop of Algorithm 1, the function GenLeakℒ generates a leakage vector L based on ℒ, and GFRndX produces a random plaintext vector X′ on the Galois field GFX, so that X′╨L. Then, the function CorrTest⋅ performs the ρCV-test on ℒ,X′ and returns an estimate r^z,CV to the vector rz. After Nρ loops, the algorithm counts the number of r^z,CV>Tdet in rz and outputs the ratio α^ as the estimated FPR (EFPR) of the ρCV-test. According to the law of large numbers, α^ will converge to the true FPR of the ρCV-test as Nρ increases.Algorithm 1: Statistical estimation of FPR.
Input: Trace set ℒ, Data vector X, Number of CV folds k, Detection threshold Tdet, Number of ρ-tests Nρ.Output: Estimate of FPR α^.fori←1:NρL←GenLeakℒ;X′←GFRndX;rz1:Nρ←CorrTestL,X′,k;end forNFP←CountPositivesrz;α^=NFP/Nρ;


### 2.3. Assumption-Based Prediction vs. Statistical Estimation

In this subsection, we run Algorithm1 on three different DILSETs to check the prediction of FPR in [10] (i.e., equation (5)).


#### 2.3.1. Data-Independent Leakage Set

The three DILSETs used to check the prediction are constructed as follows.DILSET_1 is a simulated DILSET where all the leakages are Gaussian noise. Accordingly, in each loop of Algorithm1, the function GenLeak⋅ generates a leakage vector L composed of 5,000 random leakages l∼N0,1, and GFRnd⋅ randomly produces 5,000 8 bit variables to form the associated data vector X′, X′╨L.DILSET_2 and DILSET_3 are two measured DILSETs where all the leakages come from DPA contests [18]. In each loop of Algorithm 1, the function GenLeakℒ randomly selects a time point τ from ℒ and returns the leakages Lτ at τ as L, and GFRnd⋅ produces 5,000 8 bit random variables to form X′. The DPA contests are detailed.DPA contest v2 targets an unprotected FPGA implementation of the advanced encryption standard (AES) algorithm and provides three databases for side-channel evaluation [19]. In this study, we use the first 5,000 traces in the template base to build DILSET_2.DPA contest v4.2 uses a rotating SBOX masking to protect the AES implementation on a smart card and collects 5,000 traces in each zip file [20]. This study decompresses the first zip file, corresponding to the SHA1sums: f711206b413b8d02f595d5861996ff61a1711f3d, to build DILSET_3.


#### 2.3.2. Parameter Setting

The number of foldsk has an impact on the bias and variance of the CV-based estimation. A larger value of k gives a less biased estimate but means a higher procedure’s variance. In [21], James et al. showed that k=5 and k=10 make a good trade-off between the bias and the variance. We choose 5, 10, 15, and 20 as the candidates for k to support the universality of our work.The detection thresholdTdet determines the false-positives in leakage detection. Goodwill et al. suggested a threshold of 4.5 for TVLA [22]. However, as the number of points (or tests) increases, the test statistics may be so large that a leak-free device cannot pass the test with Tdet=4.5. Therefore, Tdet=5.0 was recommended for longer traces in [9] and adopted as the threshold of the ρCV-test in [10]. In our experiments, we set Tdet to the empirical values of 4.5 and 5.0, respectively.A larger number ofρCV-tests Nρ improves the estimation accuracy of FPR, but means more running time of Algorithm 1. Because α˜4.5=6.7953×10−6 and α˜5.0=5.733×10−7, we set Nρ=107 to make a trade-off between the estimation accuracy and the time overhead.


#### 2.3.3. Comparison Experiments

We run Algorithm1 on DILSETs and record EFPRs of the ρCV-test, as given in Table 1. By comparing EFPRs to PFPRs, we notice that all EFPRs are much higher than PFPRs. For example, in the case of DILSET_1 and k=10, EFPRs α^4.5=1.5255×10−3 and α^5.0=4.465×10−4, whereas PFPRs α˜4.5=6.7953×10−6 and α˜5.0=5.733×10−7, α^4.5/α˜4.5≈224 and α^5.0/α˜5.0≈779. The gap between EFPR and PFPR reports an underpredicted false-positives (UFP) that the FPR of the ρCV-test was underpredicted in [10].Table 1
The PFPR vs. EFPR of theρCV-test, Nρ=10,000,000.
FPRDILSETk-fold CVTdet=4.5Tdet=5.0PFPR——6.7953×10−65.733×10−7EFPRDILSET_1k=51.4371×10−34.000×10−4k=101.5255×10−34.465×10−4k=151.5472×10−34.494×10−4k=201.5666×10−34.643×10−4DILSET_2k=51.4147×10−34.115×10−4k=101.5335×10−34.391×10−4k=151.5368×10−34.564×10−4k=201.5814×10−34.612×10−4DILSET_3k=51.4009×10−33.935×10−4k=101.4958×10−34.267×10−4k=151.5354×10−34.524×10−4k=201.5727×10−34.544×10−4


## 2.1. Correlation-Based Leakage Detection

The correlation-based leakage detection (ρ-test) takes advantage of a ρ-statistic rz to identify the plaintext-dependent leakage by comparing the absolute value rz to a detection threshold Tdet. In [10], Durvaux et al. utilized the k-fold CV to estimate the statistic rzτ at time point τ as follows.First, an assessor randomly splits the acquired tracesℒ into k nonoverlapping subsets ℒi of approximately the same size.Then, forjth1≤j≤k fold of CV, the assessor defines the profiling block ℒpj:=ℒ/ℒj and the test block ℒtj:=ℒj. Thus, a CV-based model model^τ,CVj is profiled from ℒpj:(1)model^τ,CVjX⟵ℒpj.The sample mean isμ^τ,CVjX, where X is the associated data.Next, compute the Pearson correlation coefficient between the modelmodel^τ,CVj and the test leakages Ltjτ⊂ℒtj:(2)r^CVjτ=ρ^Ltjτ,model^τ,CVjXtj,where Xtj is the data associated with the test leakages Ltj. An unbiased estimate r^CVτ of the correlation coefficient is obtained by combining the k coefficients r^CV1,…,r^CVk:(3)r^CVτ=1k∑j=1kr^CVjτ.Finally, after Fisher’sz-transformation and normalization, the CV-based estimate of rzτ is(4)r^z,CVτ=N−32ln1+r^CVτ1−r^CVτ.where N is the size of the set ℒ. Based on the assumption that r^z,CV can be interpreted by a normal distribution Nrz,1, the FPR of (a single) the ρCV-test can be predicted by(5)α˜Tdet=2×1−CDFN0,1Tdet.where CDFN0,1⋅ is the cumulative function of the standard normal distribution N0,1.


## 2.2. Statistical Estimation of FPR

In statistical estimation, FPR is calculated as the ratio of the numberNFP of false-positives to the number NAN of actual negatives:(6)α=NFPNAN.However, neitherNFP nor NAN is known to the evaluators in leakage detection. Therefore, we define a data-independent leakage set (DILSET) to statistically approximate the true FPR of the ρCV-test. DILSET is a leakage set, where each leakage L is independent of the associated data X′, L╨X′. Thus, rzτ≡0, and all the L are negatives for the ρCV-test. By rewriting (6), FPR of the ρCV-test can be estimated:(7)α^Tdet=Nrz>TdetNρ.where Nρ is the number of ρCV-tests on DILSET, and Nrz>Tdet is the number of (false) positives rz>Tdet in the Nρ test results.The statistical estimation is formally described in Algorithm1. Taking the trace set ℒ and the data vector X as input, the algorithm first creates a DILSET ℒ,X′, where the ρ-statistic rz≡0. In each loop of Algorithm 1, the function GenLeakℒ generates a leakage vector L based on ℒ, and GFRndX produces a random plaintext vector X′ on the Galois field GFX, so that X′╨L. Then, the function CorrTest⋅ performs the ρCV-test on ℒ,X′ and returns an estimate r^z,CV to the vector rz. After Nρ loops, the algorithm counts the number of r^z,CV>Tdet in rz and outputs the ratio α^ as the estimated FPR (EFPR) of the ρCV-test. According to the law of large numbers, α^ will converge to the true FPR of the ρCV-test as Nρ increases.Algorithm 1: Statistical estimation of FPR.
Input: Trace set ℒ, Data vector X, Number of CV folds k, Detection threshold Tdet, Number of ρ-tests Nρ.Output: Estimate of FPR α^.fori←1:NρL←GenLeakℒ;X′←GFRndX;rz1:Nρ←CorrTestL,X′,k;end forNFP←CountPositivesrz;α^=NFP/Nρ;


## 2.3. Assumption-Based Prediction vs. Statistical Estimation

In this subsection, we run Algorithm1 on three different DILSETs to check the prediction of FPR in [10] (i.e., equation (5)).


### 2.3.1. Data-Independent Leakage Set

The three DILSETs used to check the prediction are constructed as follows.DILSET_1 is a simulated DILSET where all the leakages are Gaussian noise. Accordingly, in each loop of Algorithm1, the function GenLeak⋅ generates a leakage vector L composed of 5,000 random leakages l∼N0,1, and GFRnd⋅ randomly produces 5,000 8 bit variables to form the associated data vector X′, X′╨L.DILSET_2 and DILSET_3 are two measured DILSETs where all the leakages come from DPA contests [18]. In each loop of Algorithm 1, the function GenLeakℒ randomly selects a time point τ from ℒ and returns the leakages Lτ at τ as L, and GFRnd⋅ produces 5,000 8 bit random variables to form X′. The DPA contests are detailed.DPA contest v2 targets an unprotected FPGA implementation of the advanced encryption standard (AES) algorithm and provides three databases for side-channel evaluation [19]. In this study, we use the first 5,000 traces in the template base to build DILSET_2.DPA contest v4.2 uses a rotating SBOX masking to protect the AES implementation on a smart card and collects 5,000 traces in each zip file [20]. This study decompresses the first zip file, corresponding to the SHA1sums: f711206b413b8d02f595d5861996ff61a1711f3d, to build DILSET_3.


### 2.3.2. Parameter Setting

The number of foldsk has an impact on the bias and variance of the CV-based estimation. A larger value of k gives a less biased estimate but means a higher procedure’s variance. In [21], James et al. showed that k=5 and k=10 make a good trade-off between the bias and the variance. We choose 5, 10, 15, and 20 as the candidates for k to support the universality of our work.The detection thresholdTdet determines the false-positives in leakage detection. Goodwill et al. suggested a threshold of 4.5 for TVLA [22]. However, as the number of points (or tests) increases, the test statistics may be so large that a leak-free device cannot pass the test with Tdet=4.5. Therefore, Tdet=5.0 was recommended for longer traces in [9] and adopted as the threshold of the ρCV-test in [10]. In our experiments, we set Tdet to the empirical values of 4.5 and 5.0, respectively.A larger number ofρCV-tests Nρ improves the estimation accuracy of FPR, but means more running time of Algorithm 1. Because α˜4.5=6.7953×10−6 and α˜5.0=5.733×10−7, we set Nρ=107 to make a trade-off between the estimation accuracy and the time overhead.


### 2.3.3. Comparison Experiments

We run Algorithm1 on DILSETs and record EFPRs of the ρCV-test, as given in Table 1. By comparing EFPRs to PFPRs, we notice that all EFPRs are much higher than PFPRs. For example, in the case of DILSET_1 and k=10, EFPRs α^4.5=1.5255×10−3 and α^5.0=4.465×10−4, whereas PFPRs α˜4.5=6.7953×10−6 and α˜5.0=5.733×10−7, α^4.5/α˜4.5≈224 and α^5.0/α˜5.0≈779. The gap between EFPR and PFPR reports an underpredicted false-positives (UFP) that the FPR of the ρCV-test was underpredicted in [10].Table 1
The PFPR vs. EFPR of theρCV-test, Nρ=10,000,000.
FPRDILSETk-fold CVTdet=4.5Tdet=5.0PFPR——6.7953×10−65.733×10−7EFPRDILSET_1k=51.4371×10−34.000×10−4k=101.5255×10−34.465×10−4k=151.5472×10−34.494×10−4k=201.5666×10−34.643×10−4DILSET_2k=51.4147×10−34.115×10−4k=101.5335×10−34.391×10−4k=151.5368×10−34.564×10−4k=201.5814×10−34.612×10−4DILSET_3k=51.4009×10−33.935×10−4k=101.4958×10−34.267×10−4k=151.5354×10−34.524×10−4k=201.5727×10−34.544×10−4


## 2.3.1. Data-Independent Leakage Set

The three DILSETs used to check the prediction are constructed as follows.DILSET_1 is a simulated DILSET where all the leakages are Gaussian noise. Accordingly, in each loop of Algorithm1, the function GenLeak⋅ generates a leakage vector L composed of 5,000 random leakages l∼N0,1, and GFRnd⋅ randomly produces 5,000 8 bit variables to form the associated data vector X′, X′╨L.DILSET_2 and DILSET_3 are two measured DILSETs where all the leakages come from DPA contests [18]. In each loop of Algorithm 1, the function GenLeakℒ randomly selects a time point τ from ℒ and returns the leakages Lτ at τ as L, and GFRnd⋅ produces 5,000 8 bit random variables to form X′. The DPA contests are detailed.DPA contest v2 targets an unprotected FPGA implementation of the advanced encryption standard (AES) algorithm and provides three databases for side-channel evaluation [19]. In this study, we use the first 5,000 traces in the template base to build DILSET_2.DPA contest v4.2 uses a rotating SBOX masking to protect the AES implementation on a smart card and collects 5,000 traces in each zip file [20]. This study decompresses the first zip file, corresponding to the SHA1sums: f711206b413b8d02f595d5861996ff61a1711f3d, to build DILSET_3.


## 2.3.2. Parameter Setting

The number of foldsk has an impact on the bias and variance of the CV-based estimation. A larger value of k gives a less biased estimate but means a higher procedure’s variance. In [21], James et al. showed that k=5 and k=10 make a good trade-off between the bias and the variance. We choose 5, 10, 15, and 20 as the candidates for k to support the universality of our work.The detection thresholdTdet determines the false-positives in leakage detection. Goodwill et al. suggested a threshold of 4.5 for TVLA [22]. However, as the number of points (or tests) increases, the test statistics may be so large that a leak-free device cannot pass the test with Tdet=4.5. Therefore, Tdet=5.0 was recommended for longer traces in [9] and adopted as the threshold of the ρCV-test in [10]. In our experiments, we set Tdet to the empirical values of 4.5 and 5.0, respectively.A larger number ofρCV-tests Nρ improves the estimation accuracy of FPR, but means more running time of Algorithm 1. Because α˜4.5=6.7953×10−6 and α˜5.0=5.733×10−7, we set Nρ=107 to make a trade-off between the estimation accuracy and the time overhead.


## 2.3.3. Comparison Experiments

We run Algorithm1 on DILSETs and record EFPRs of the ρCV-test, as given in Table 1. By comparing EFPRs to PFPRs, we notice that all EFPRs are much higher than PFPRs. For example, in the case of DILSET_1 and k=10, EFPRs α^4.5=1.5255×10−3 and α^5.0=4.465×10−4, whereas PFPRs α˜4.5=6.7953×10−6 and α˜5.0=5.733×10−7, α^4.5/α˜4.5≈224 and α^5.0/α˜5.0≈779. The gap between EFPR and PFPR reports an underpredicted false-positives (UFP) that the FPR of the ρCV-test was underpredicted in [10].Table 1
The PFPR vs. EFPR of theρCV-test, Nρ=10,000,000.
FPRDILSETk-fold CVTdet=4.5Tdet=5.0PFPR——6.7953×10−65.733×10−7EFPRDILSET_1k=51.4371×10−34.000×10−4k=101.5255×10−34.465×10−4k=151.5472×10−34.494×10−4k=201.5666×10−34.643×10−4DILSET_2k=51.4147×10−34.115×10−4k=101.5335×10−34.391×10−4k=151.5368×10−34.564×10−4k=201.5814×10−34.612×10−4DILSET_3k=51.4009×10−33.935×10−4k=101.4958×10−34.267×10−4k=151.5354×10−34.524×10−4k=201.5727×10−34.544×10−4


## 3. Root Cause Analysis

Root cause analysis (RCA) identifies the root cause of UFP, which helps prevent the underprediction from recurring. This section uses hypothesis test tools to determine the root cause of UFP. In this section, we first introduce the test tools and then analyze the UFP and the AE in [10].


### 3.1. Hypothesis Test Tools

The tests used for our root cause analysis (RCA) are given in Table2.Table 2
Test tools in this study.
Hypothesis test toolThe null hypothesis in this studyOne-samplet-testThe meanμ=0.Chi-squared testThe varianceσ2=1.Kolmogorov–Smirnov testThe samples come from a standard normal distributionN0,1.Pearson’s correlation testThe correlation coefficientρ=0.


#### 3.1.1. One-Samplet-Test

The one-samplet-test compares the sample mean to a prespecified value to test for the deviation from this value. The null hypothesis of the test assumes that no difference exists between the true mean μ and the comparison value μ¯, while the alternative assumes that the difference exists. The test statistic for a one-sample t -test is denoted by the letter t, which is calculated using the following formula:(8)t=μ^−μ¯σ^/n.σ^ is the sample standard deviation, and n is the sample size. Then, the p value of the t-test can be computed:(9)p=2×1−CDFtt,v,where CDFt⋅ is the cumulative function of Student’s t-distribution, and v=n−1 is the degree of freedom. As v increases, Student’s t-distribution gets close to a standard normal distribution N0,1 [23].


#### 3.1.2. Chi-Squared Test

The chi-squared test (χ2-test) can be used to test whether the true variance σ2 is equal to a specified value σ¯2. Assuming σ2=σ¯2, the test statistic of the χ2-test is(10)χ2=n−1×σ^2σ¯2.Thep value of the null hypothesis is(11)p=CDFx2χ2,n−1,where CDFx2⋅ is the cumulative function of a χ2 distribution:(12)CDFx2x,v=∫x∞fx,vdx,fx,v=xv/2−1e−x/22v/2Γv/2x>0,0x≤0,where Γ⋅ denotes the gamma function [23].


#### 3.1.3. Kolmogorov–Smirnov Test

The Kolmogorov–Smirnov test (KS-test) is a nonparametric test that can be used to compare a sample with a reference probability distribution. Its null hypothesis assumes that the samples come from a specified distributionΘ¯. In this test, an empirical cumulative function Fn for n independent and identically distributed ordered observations Xi is defined as(13)Fnx=1n∑i=1nI−∞,xXi.I−∞,xXi is the indicator function which returns 1 if Xi≤x and returns 0 otherwise. The KS statistic Dn for a given function Fx is(14)Dn=supFnx−Fx.where sup⋅ is the supremum of the set of distances. If the sample comes from Θ¯, Dn converges to zero in the limit when n goes to infinity. Its p value, that is, the probability that the null hypothesis holds can be calculated regarding [24].


#### 3.1.4. Pearson’s Correlation Test

Pearson’s correlation test is used to test whether there is a relationship between two variables.Its null hypothesis assumes no correlation between the observed phenomenaρ=0. Given an estimate ρ^ of ρ, the test statistic can be estimated:(15)t=ρ^1−ρ^n−2.Thep value of the test is determined by Student’s t-distribution with degrees n−2 of freedom [25]. Note that Pearson’s correlation test is different in definition from the ρ-test in Section 2.1.


### 3.2. Error in Assumed Distribution

As shown in Figure1, the UFP implies a nonnegligible error between the true distribution of r^z,CV and the assumed distribution N0,1. Regarding the description in [26], we name this error the assumption error in leakage detection and use the p values of the tests on r^z,CV to quantify the AE—the smaller the p value, the stronger the null hypothesis should be rejected and the more significant the AE in the ρCV-test. The results of the hypothesis tests are given in Table 3. At the significance level of 0.01 (an accepted threshold in [27]), the p values of t-tests exceed 0.01, which means the null hypothesis μ=0 is accepted. In contrast, the p values of χ2-tests and KS-tests are less than 0.01, that is, the null hypotheses σ2=1 and Θ∼N0,1 are rejected. Hence, it is confirmed that the assumed N0,1 is not the true distribution of r^z,CV.Figure 1
The frequency distribution ofr^z,CV (in case of DILSET_1). The curves of r^z,CVk=5, r^z,CVk=10, r^z,CVk=15, and r^z,CVk=20 roughly coincide but deviate from the assumed distribution N0,1.Table 3
Thep values of the hypothesis tests on r^z,CV.
DILSETk-fold CVt-testχ2-testKS-testDILSET_1k=50.252707.1675×10−36k=100.972001.4230×10−29k=150.274102.4038×10−36k=200.963001.6397×10−30DILSET_2k=50.978503.5003×10−30k=100.145703.4179×10−34k=150.303403.3193×10−33k=200.933507.3287×10−32DILSET_3k=50.274605.5938×10−32k=100.740603.2925×10−32k=150.843303.1929×10−33k=200.207503.0201×10−36


### 3.3. Correlation between Cross-Validation Blocks

To determine the root cause of a problem, RCA establishes an event timeline from the normal situation up to the time the problem occurs. Based on equations (1)–(4), we create the timeline from ℒ input to UFP (or AE) occurrence, as shown in Figure 2. To facilitate RCA, intrablock estimate r^z,CVj is obtained by performing Fisher’s z-transform and normalization on r^CVj.(16)r^z,CVjτ=N/k−32ln1+r^CVjτ1+r^CVjτ,where N/k is the number of traces in the test block ℒtj.Figure 2
Event timeline used for the RCA of UFP. The solid line is the timeline fromℒ input to UFP occurrence (i.e., r^z,CV output), and the dashed line is the operation to obtain the intrablock estimate r^z,CVj.We first check the distribution ofr^z,CVj to locate the source of AE. Table 4 provides the p values of the tests on r^z,CVj (when k=5). Since all the p values exceed the level of 0.01, the intrablock estimates r^z,CVj pass the tests for N0,1, which excludes (1) and (2) from the possible source of AE. In [28], Bengio et al. proved that between-block correlation leads to a biased variance of the CV-based estimation (in equation (3)). Therefore, we utilize Pearson’s correlation test to examine the correlation between r^z,CVj1,r^z,CVj2 (j1≠j2). In the case of DILSET_1 and k=5, the results of correlation tests are given in Table 5. Since p≡0, the null hypothesis ρ=0 is rejected. The correlation between CV blocks is confirmed and may be the cause of the AE in the ρCV-test.Table 4
Thep values of the hypothesis tests on r^z,CVj (k=5).
DILSET_1DILSET_2DILSET_3r^z,CVjt-testχ2-testKS-testt-testχ2-testKS-testt-testχ2-testKS-testr^z,CV10.14580.14050.20980.82680.96260.99570.82090.66170.9697r^z,CV20.83360.55410.99890.59290.14300.26720.26060.03100.1622r^z,CV30.46760.32570.75210.66380.90340.92940.72150.43730.2853r^z,CV40.86380.77020.88150.21650.64100.65070.69130.56910.7465r^z,CV50.45710.26620.67680.72060.34440.52940.70580.12460.6759Table 5
Thep values of Pearson’s correlation test between r^z,CVj (DILSET_1, k=5).
r^z,CV1r^z,CV2r^z,CV3r^z,CV4r^z,CV5r^z,CV1—0000r^z,CV20—000r^z,CV300—00r^z,CV4000—0r^z,CV50000—


### 3.4. Overlapping Cross-Validation Blocks

According to Corollary 3 in [28], the between-block correlation in cross-validation stems from the overlap between the training and the test blocks. In this subsection, we present a nonoverlapping validation (NOV) and compare the NOV-based ρ-test (ρNOV-test) with the ρCV-test to check the impact of the overlap on FPR. The ρNOV-test works as follows.First, the tracesℒ are split into k nonoverlapping blocks ℒj of approximately the same size. For each block ℒj, 1/m of the traces are randomly selected as the test subset ℒtj, and the others are left as the profiling subset ℒpj. Then, the model model^τ,NOVj is profiled:(17)model^τ,NOVjX←ℒpj.Finally, an NOV-based estimater^z,NOV of ρ-statistic can be calculated by equations (18)–(20):(18)r^NOVjτ=ρ^Ltjτ,model^τ,NOVjXtj,(19)r^z,NOVjτ=N/m×k−32ln1+r^NOVjτ1−r^NOVjτ,(20)r^z,NOVτ=1k∑j=1kr^z,NOVjτ.We run Algorithm1 on an extended DILSET_1 (ExDILSET_1) to approximate the true FPR of the ρNOV-test. Specifically, in each loop of the algorithm, the function GenLeak⋅ produces 50,000 random leakages l∼N0,1 to form the leakage vector L, and GFRnd⋅ randomly generates a vector X′ composed of 50,000 8 bit variables, so that X′╨L. Then, the function CorrTest⋅ performs the ρNOV-test on ℒ,X′ and returns an NOV-based estimate r^z,NOV. After 10,000,000 loops, 10,000,000r^z,NOVj (for each j) and 10,000,000r^z,NOV are obtained. Finally, Algorithm 1 counts the number of false-positives and outputs the EFPR of the ρNOV-test.The results of Pearson’s correlation test betweenr^z,NOVj1,r^z,NOVj2 (m=10, k=5) are given in Table 6. At the significance level of 0.01, all the pairs pass Pearson’s correlation test, which means that there is no correlation between the NOV blocks. Table 7 provides the p values of the distribution tests on r^z,NOV and the EFPRs of ρNOV-test. On the one hand, since p>0.01, r^z,NOV pass the test for the distribution N0,1. On the other hand, compared with the EFPRs of the ρCV-test (Table 1), EFPRs of the ρNOV-test are much closer to PFPRs α˜4.5=6.7953×10−6 and α˜5.0=5.733×10−7. Therefore, both the AE and the UFP are solved by eliminating the between-block overlap. It is concluded that the overlap between the training and test blocks is the root cause of AE and UFP in the ρCV-test.Table 6
Thep values of Pearson’s correlation test between r^z,NOVj (ExDILSET_1, m=10, k=5).
j2=1j2=2j2=3j2=4j2=5j1=1—0.42150.56700.27910.0839j1=20.4215—0.35910.77180.0463j1=30.56700.3591—0.03720.7828j1=40.27910.77180.0372—0.9380j1=50.08390.04630.78280.9380—Table 7
The test results for the AE and UFP in theρNOV-test (ExDILSET_1, m=10).
DILSETk-fold NOVt-testχ2-testKS-testTdet=4.5Tdet=5.0ExDILSET_1k=50.63460.03420.27676.4×10−66×10−7k=100.08160.16580.17165.0×10−61×10−7k=150.40120.18170.19576.8×10−61.0×10−6k=200.84110.53300.81876.0×10−68×10−7


## 3.1. Hypothesis Test Tools

The tests used for our root cause analysis (RCA) are given in Table2.Table 2
Test tools in this study.
Hypothesis test toolThe null hypothesis in this studyOne-samplet-testThe meanμ=0.Chi-squared testThe varianceσ2=1.Kolmogorov–Smirnov testThe samples come from a standard normal distributionN0,1.Pearson’s correlation testThe correlation coefficientρ=0.


### 3.1.1. One-Samplet-Test

The one-samplet-test compares the sample mean to a prespecified value to test for the deviation from this value. The null hypothesis of the test assumes that no difference exists between the true mean μ and the comparison value μ¯, while the alternative assumes that the difference exists. The test statistic for a one-sample t -test is denoted by the letter t, which is calculated using the following formula:(8)t=μ^−μ¯σ^/n.σ^ is the sample standard deviation, and n is the sample size. Then, the p value of the t-test can be computed:(9)p=2×1−CDFtt,v,where CDFt⋅ is the cumulative function of Student’s t-distribution, and v=n−1 is the degree of freedom. As v increases, Student’s t-distribution gets close to a standard normal distribution N0,1 [23].


### 3.1.2. Chi-Squared Test

The chi-squared test (χ2-test) can be used to test whether the true variance σ2 is equal to a specified value σ¯2. Assuming σ2=σ¯2, the test statistic of the χ2-test is(10)χ2=n−1×σ^2σ¯2.Thep value of the null hypothesis is(11)p=CDFx2χ2,n−1,where CDFx2⋅ is the cumulative function of a χ2 distribution:(12)CDFx2x,v=∫x∞fx,vdx,fx,v=xv/2−1e−x/22v/2Γv/2x>0,0x≤0,where Γ⋅ denotes the gamma function [23].


### 3.1.3. Kolmogorov–Smirnov Test

The Kolmogorov–Smirnov test (KS-test) is a nonparametric test that can be used to compare a sample with a reference probability distribution. Its null hypothesis assumes that the samples come from a specified distributionΘ¯. In this test, an empirical cumulative function Fn for n independent and identically distributed ordered observations Xi is defined as(13)Fnx=1n∑i=1nI−∞,xXi.I−∞,xXi is the indicator function which returns 1 if Xi≤x and returns 0 otherwise. The KS statistic Dn for a given function Fx is(14)Dn=supFnx−Fx.where sup⋅ is the supremum of the set of distances. If the sample comes from Θ¯, Dn converges to zero in the limit when n goes to infinity. Its p value, that is, the probability that the null hypothesis holds can be calculated regarding [24].


### 3.1.4. Pearson’s Correlation Test

Pearson’s correlation test is used to test whether there is a relationship between two variables.Its null hypothesis assumes no correlation between the observed phenomenaρ=0. Given an estimate ρ^ of ρ, the test statistic can be estimated:(15)t=ρ^1−ρ^n−2.Thep value of the test is determined by Student’s t-distribution with degrees n−2 of freedom [25]. Note that Pearson’s correlation test is different in definition from the ρ-test in Section 2.1.


## 3.1.1. One-Samplet-Test

The one-samplet-test compares the sample mean to a prespecified value to test for the deviation from this value. The null hypothesis of the test assumes that no difference exists between the true mean μ and the comparison value μ¯, while the alternative assumes that the difference exists. The test statistic for a one-sample t -test is denoted by the letter t, which is calculated using the following formula:(8)t=μ^−μ¯σ^/n.σ^ is the sample standard deviation, and n is the sample size. Then, the p value of the t-test can be computed:(9)p=2×1−CDFtt,v,where CDFt⋅ is the cumulative function of Student’s t-distribution, and v=n−1 is the degree of freedom. As v increases, Student’s t-distribution gets close to a standard normal distribution N0,1 [23].


## 3.1.2. Chi-Squared Test

The chi-squared test (χ2-test) can be used to test whether the true variance σ2 is equal to a specified value σ¯2. Assuming σ2=σ¯2, the test statistic of the χ2-test is(10)χ2=n−1×σ^2σ¯2.Thep value of the null hypothesis is(11)p=CDFx2χ2,n−1,where CDFx2⋅ is the cumulative function of a χ2 distribution:(12)CDFx2x,v=∫x∞fx,vdx,fx,v=xv/2−1e−x/22v/2Γv/2x>0,0x≤0,where Γ⋅ denotes the gamma function [23].


## 3.1.3. Kolmogorov–Smirnov Test

The Kolmogorov–Smirnov test (KS-test) is a nonparametric test that can be used to compare a sample with a reference probability distribution. Its null hypothesis assumes that the samples come from a specified distributionΘ¯. In this test, an empirical cumulative function Fn for n independent and identically distributed ordered observations Xi is defined as(13)Fnx=1n∑i=1nI−∞,xXi.I−∞,xXi is the indicator function which returns 1 if Xi≤x and returns 0 otherwise. The KS statistic Dn for a given function Fx is(14)Dn=supFnx−Fx.where sup⋅ is the supremum of the set of distances. If the sample comes from Θ¯, Dn converges to zero in the limit when n goes to infinity. Its p value, that is, the probability that the null hypothesis holds can be calculated regarding [24].


## 3.1.4. Pearson’s Correlation Test

Pearson’s correlation test is used to test whether there is a relationship between two variables.Its null hypothesis assumes no correlation between the observed phenomenaρ=0. Given an estimate ρ^ of ρ, the test statistic can be estimated:(15)t=ρ^1−ρ^n−2.Thep value of the test is determined by Student’s t-distribution with degrees n−2 of freedom [25]. Note that Pearson’s correlation test is different in definition from the ρ-test in Section 2.1.


## 3.2. Error in Assumed Distribution

As shown in Figure1, the UFP implies a nonnegligible error between the true distribution of r^z,CV and the assumed distribution N0,1. Regarding the description in [26], we name this error the assumption error in leakage detection and use the p values of the tests on r^z,CV to quantify the AE—the smaller the p value, the stronger the null hypothesis should be rejected and the more significant the AE in the ρCV-test. The results of the hypothesis tests are given in Table 3. At the significance level of 0.01 (an accepted threshold in [27]), the p values of t-tests exceed 0.01, which means the null hypothesis μ=0 is accepted. In contrast, the p values of χ2-tests and KS-tests are less than 0.01, that is, the null hypotheses σ2=1 and Θ∼N0,1 are rejected. Hence, it is confirmed that the assumed N0,1 is not the true distribution of r^z,CV.Figure 1
The frequency distribution ofr^z,CV (in case of DILSET_1). The curves of r^z,CVk=5, r^z,CVk=10, r^z,CVk=15, and r^z,CVk=20 roughly coincide but deviate from the assumed distribution N0,1.Table 3
Thep values of the hypothesis tests on r^z,CV.
DILSETk-fold CVt-testχ2-testKS-testDILSET_1k=50.252707.1675×10−36k=100.972001.4230×10−29k=150.274102.4038×10−36k=200.963001.6397×10−30DILSET_2k=50.978503.5003×10−30k=100.145703.4179×10−34k=150.303403.3193×10−33k=200.933507.3287×10−32DILSET_3k=50.274605.5938×10−32k=100.740603.2925×10−32k=150.843303.1929×10−33k=200.207503.0201×10−36


## 3.3. Correlation between Cross-Validation Blocks

To determine the root cause of a problem, RCA establishes an event timeline from the normal situation up to the time the problem occurs. Based on equations (1)–(4), we create the timeline from ℒ input to UFP (or AE) occurrence, as shown in Figure 2. To facilitate RCA, intrablock estimate r^z,CVj is obtained by performing Fisher’s z-transform and normalization on r^CVj.(16)r^z,CVjτ=N/k−32ln1+r^CVjτ1+r^CVjτ,where N/k is the number of traces in the test block ℒtj.Figure 2
Event timeline used for the RCA of UFP. The solid line is the timeline fromℒ input to UFP occurrence (i.e., r^z,CV output), and the dashed line is the operation to obtain the intrablock estimate r^z,CVj.We first check the distribution ofr^z,CVj to locate the source of AE. Table 4 provides the p values of the tests on r^z,CVj (when k=5). Since all the p values exceed the level of 0.01, the intrablock estimates r^z,CVj pass the tests for N0,1, which excludes (1) and (2) from the possible source of AE. In [28], Bengio et al. proved that between-block correlation leads to a biased variance of the CV-based estimation (in equation (3)). Therefore, we utilize Pearson’s correlation test to examine the correlation between r^z,CVj1,r^z,CVj2 (j1≠j2). In the case of DILSET_1 and k=5, the results of correlation tests are given in Table 5. Since p≡0, the null hypothesis ρ=0 is rejected. The correlation between CV blocks is confirmed and may be the cause of the AE in the ρCV-test.Table 4
Thep values of the hypothesis tests on r^z,CVj (k=5).
DILSET_1DILSET_2DILSET_3r^z,CVjt-testχ2-testKS-testt-testχ2-testKS-testt-testχ2-testKS-testr^z,CV10.14580.14050.20980.82680.96260.99570.82090.66170.9697r^z,CV20.83360.55410.99890.59290.14300.26720.26060.03100.1622r^z,CV30.46760.32570.75210.66380.90340.92940.72150.43730.2853r^z,CV40.86380.77020.88150.21650.64100.65070.69130.56910.7465r^z,CV50.45710.26620.67680.72060.34440.52940.70580.12460.6759Table 5
Thep values of Pearson’s correlation test between r^z,CVj (DILSET_1, k=5).
r^z,CV1r^z,CV2r^z,CV3r^z,CV4r^z,CV5r^z,CV1—0000r^z,CV20—000r^z,CV300—00r^z,CV4000—0r^z,CV50000—


## 3.4. Overlapping Cross-Validation Blocks

According to Corollary 3 in [28], the between-block correlation in cross-validation stems from the overlap between the training and the test blocks. In this subsection, we present a nonoverlapping validation (NOV) and compare the NOV-based ρ-test (ρNOV-test) with the ρCV-test to check the impact of the overlap on FPR. The ρNOV-test works as follows.First, the tracesℒ are split into k nonoverlapping blocks ℒj of approximately the same size. For each block ℒj, 1/m of the traces are randomly selected as the test subset ℒtj, and the others are left as the profiling subset ℒpj. Then, the model model^τ,NOVj is profiled:(17)model^τ,NOVjX←ℒpj.Finally, an NOV-based estimater^z,NOV of ρ-statistic can be calculated by equations (18)–(20):(18)r^NOVjτ=ρ^Ltjτ,model^τ,NOVjXtj,(19)r^z,NOVjτ=N/m×k−32ln1+r^NOVjτ1−r^NOVjτ,(20)r^z,NOVτ=1k∑j=1kr^z,NOVjτ.We run Algorithm1 on an extended DILSET_1 (ExDILSET_1) to approximate the true FPR of the ρNOV-test. Specifically, in each loop of the algorithm, the function GenLeak⋅ produces 50,000 random leakages l∼N0,1 to form the leakage vector L, and GFRnd⋅ randomly generates a vector X′ composed of 50,000 8 bit variables, so that X′╨L. Then, the function CorrTest⋅ performs the ρNOV-test on ℒ,X′ and returns an NOV-based estimate r^z,NOV. After 10,000,000 loops, 10,000,000r^z,NOVj (for each j) and 10,000,000r^z,NOV are obtained. Finally, Algorithm 1 counts the number of false-positives and outputs the EFPR of the ρNOV-test.The results of Pearson’s correlation test betweenr^z,NOVj1,r^z,NOVj2 (m=10, k=5) are given in Table 6. At the significance level of 0.01, all the pairs pass Pearson’s correlation test, which means that there is no correlation between the NOV blocks. Table 7 provides the p values of the distribution tests on r^z,NOV and the EFPRs of ρNOV-test. On the one hand, since p>0.01, r^z,NOV pass the test for the distribution N0,1. On the other hand, compared with the EFPRs of the ρCV-test (Table 1), EFPRs of the ρNOV-test are much closer to PFPRs α˜4.5=6.7953×10−6 and α˜5.0=5.733×10−7. Therefore, both the AE and the UFP are solved by eliminating the between-block overlap. It is concluded that the overlap between the training and test blocks is the root cause of AE and UFP in the ρCV-test.Table 6
Thep values of Pearson’s correlation test between r^z,NOVj (ExDILSET_1, m=10, k=5).
j2=1j2=2j2=3j2=4j2=5j1=1—0.42150.56700.27910.0839j1=20.4215—0.35910.77180.0463j1=30.56700.3591—0.03720.7828j1=40.27910.77180.0372—0.9380j1=50.08390.04630.78280.9380—Table 7
The test results for the AE and UFP in theρNOV-test (ExDILSET_1, m=10).
DILSETk-fold NOVt-testχ2-testKS-testTdet=4.5Tdet=5.0ExDILSET_1k=50.63460.03420.27676.4×10−66×10−7k=100.08160.16580.17165.0×10−61×10−7k=150.40120.18170.19576.8×10−61.0×10−6k=200.84110.53300.81876.0×10−68×10−7


## 4. Improvedρ-Test

Section3 has shown that NOV is an effective solution to the UFP in the ρCV-test, but requires more samples to construct the k nonoverlapping blocks. In this section, we introduce the profiling-shared validation to efficiently solve the UFP.


### 4.1. Profiling-Shared Validation

Profiling-shared validation shares the same profiling samples and the same profiled model for all the test blocks. In a PSV-basedρ-test (ρPSV-test), the evaluator randomly splits the traces into m+k nonoverlapping subsets ℒi with approximately same size. Choosing the first m subsets as the profiling block ℒp:=ℒi|1≤i≤m, the model at time point τ can be profiled:(21)model^τ,PSVX←ℒp.Then, forjth test block ℒtj:=ℒj+m, 1≤j≤k, the Pearson correlation coefficient between the model and the test leakages Ltjτ⊂ℒtj is computed:(22)r^PSVjτ=ρ^Ltjτ,model^τ,PSVXtj.Next, after Fisher’sz-transformation and normalization, the intrablock estimates r^z,PSVj,(23)r^z,PSVjτ=N/m+k−32ln1+r^PSVjτ1−r^PSVjτ.Finally, averaging the intrablock estimatesr^z,PSVj and then normalizing the mean to obtain a PSV-based estimate r^z,PSVτ of ρ-statistic,(24)r^z,PSVτ=1k∑j=1kr^z,PSVjτ.


### 4.2. Effectiveness and Efficiency

#### 4.2.1. Effectiveness

We perform theρPSV-test on DILSET_1, 2, and 3 and then test the estimates r^z,PSVj and r^z,PSV. Since each PSV block is independent of other blocks, ρr^z,PSVj1,r^z,PSVj2≡0, j1≠j2. For example, in the case of DILSET_1 and m,k=10,5, the obtained results of correlation tests between r^z,PSVj are given in Table 8. At the significance level of 0.01, the null hypothesis ρ=0 holds. Table 9 provides the p values of the distribution tests on r^z,PSV and EFPRs of the ρPSV-test. Since all the p values exceed the acceptable level of 0.01, the PSV-based estimates r^z,PSV pass the test for N0,1. In addition, compared to EFPRs of the ρCV-test (Table 1), EFPRs of ρPSV-tests are much closer to PFPRs. Hence, PSV is an effective solution to the AE and UFP.Table 8
Thep values of Pearson’s correlation test between r^z,PSVj (DILSET_1, m=10, k=5).
j2=1j2=2j2=3j2=4j2=5j1=1—0.34710.61850.91310.0447j1=20.3471—0.76030.01510.7986j1=30.61850.7603—0.01390.8772j1=40.91310.01510.0139—0.6937j1=50.04470.79860.87720.6937—Table 9
The test results for the AE and UFP in theρPSV-test.
DILSETm,k-PSVt-testχ2-testKS-testTdet=4.5Tdet=5.0DILSET_15,50.36630.17370.54756.6×10−65×10−75,100.19690.82440.63057.5×10−69×10−710,50.96050.63950.83677.6×10−66×10−710,100.64260.55360.37595.9×10−67×10−7DILSET_25,50.44440.19160.03666.7×10−65×10−75,100.87940.38180.98118.8×10−69×10−710,50.11020.52670.06236.5×10−61.1×10−610,100.74680.32700.98346.1×10−65×10−7DILSET_35,50.34690.41180.69946.9×10−66×10−75,100.19370.50680.34758.0×10−65×10−710,50.83390.75770.93928.1×10−67×10−710,100.95650.15420.63267.0×10−65×10−7


#### 4.2.2. Efficiency

Different from the CV which requiresk-profiled models for the validation, PSV shares the same model for all test blocks, which means the ρPSV-test spends less time in the profiling phase than the ρCV-test. We compare the execution time of the ρPSV-test and ρCV-test on an HP EliteBook 735 G6 (AMD Ryzen 5 PRO 3500U CPU @ 2.1 GHz, 8 GB RAM) with Windows 10 as the operating system. The time overhead of 1,000,000 tests on DILSET_1 is shown in Figure 3. Compared with the time cost of 1,000,000 ρCV-tests of at least 676.94 seconds when k=5, the maximum time cost of 1,000,000 ρPSV-tests is 144.98 seconds when m,k=10,5, saving about 79% of the time costs. In short, the ρPSV-test is more time-efficient than the ρCV-test.Figure 3
Time cost of theρPSV-test vs. ρCV-test. The solid line is the time cost of the ρPSV-test, and the dashed line is the time cost of the ρCV-test.


### 4.3. Measured Experiments

We verify whether theρPSV-test can identify the plaintext-dependent leakages in the captured power traces. In our measured experiments, the ρPSV-test assesses the dependency between the leakages and first 4 plaintext bytes in DPA contest v4.2. Let m,k=5,5,5,10,10,5, and 10,10 respectively; we run the ρPSV-test on DPA contest v4.2 and plot the estimates r^z,PSV at the time domain 375μs,390μs, as shown in Figure 4. When Tdet=5.0, the 4 plaintext bytes are leaked in the intervals 379.96μs,381.02μs, 380.80μs,382.05μs, 381.64μs,382.89μs, and 382.48μs,383.73μs. In other words, the plaintext-dependent leakages are successfully identified by the ρPSV-test from the captured traces.Figure 4
Univariate 1st-order ρPSV-tests on DPA contest v4.2. The dashed line is the detection threshold 5.0. (a) m=5,k=5. (b) m=5,k=10. (c) m=10,k=5. (d) m=10,k=10.
(a)(b)(c)(d)


## 4.1. Profiling-Shared Validation

Profiling-shared validation shares the same profiling samples and the same profiled model for all the test blocks. In a PSV-basedρ-test (ρPSV-test), the evaluator randomly splits the traces into m+k nonoverlapping subsets ℒi with approximately same size. Choosing the first m subsets as the profiling block ℒp:=ℒi|1≤i≤m, the model at time point τ can be profiled:(21)model^τ,PSVX←ℒp.Then, forjth test block ℒtj:=ℒj+m, 1≤j≤k, the Pearson correlation coefficient between the model and the test leakages Ltjτ⊂ℒtj is computed:(22)r^PSVjτ=ρ^Ltjτ,model^τ,PSVXtj.Next, after Fisher’sz-transformation and normalization, the intrablock estimates r^z,PSVj,(23)r^z,PSVjτ=N/m+k−32ln1+r^PSVjτ1−r^PSVjτ.Finally, averaging the intrablock estimatesr^z,PSVj and then normalizing the mean to obtain a PSV-based estimate r^z,PSVτ of ρ-statistic,(24)r^z,PSVτ=1k∑j=1kr^z,PSVjτ.


## 4.2. Effectiveness and Efficiency

### 4.2.1. Effectiveness

We perform theρPSV-test on DILSET_1, 2, and 3 and then test the estimates r^z,PSVj and r^z,PSV. Since each PSV block is independent of other blocks, ρr^z,PSVj1,r^z,PSVj2≡0, j1≠j2. For example, in the case of DILSET_1 and m,k=10,5, the obtained results of correlation tests between r^z,PSVj are given in Table 8. At the significance level of 0.01, the null hypothesis ρ=0 holds. Table 9 provides the p values of the distribution tests on r^z,PSV and EFPRs of the ρPSV-test. Since all the p values exceed the acceptable level of 0.01, the PSV-based estimates r^z,PSV pass the test for N0,1. In addition, compared to EFPRs of the ρCV-test (Table 1), EFPRs of ρPSV-tests are much closer to PFPRs. Hence, PSV is an effective solution to the AE and UFP.Table 8
Thep values of Pearson’s correlation test between r^z,PSVj (DILSET_1, m=10, k=5).
j2=1j2=2j2=3j2=4j2=5j1=1—0.34710.61850.91310.0447j1=20.3471—0.76030.01510.7986j1=30.61850.7603—0.01390.8772j1=40.91310.01510.0139—0.6937j1=50.04470.79860.87720.6937—Table 9
The test results for the AE and UFP in theρPSV-test.
DILSETm,k-PSVt-testχ2-testKS-testTdet=4.5Tdet=5.0DILSET_15,50.36630.17370.54756.6×10−65×10−75,100.19690.82440.63057.5×10−69×10−710,50.96050.63950.83677.6×10−66×10−710,100.64260.55360.37595.9×10−67×10−7DILSET_25,50.44440.19160.03666.7×10−65×10−75,100.87940.38180.98118.8×10−69×10−710,50.11020.52670.06236.5×10−61.1×10−610,100.74680.32700.98346.1×10−65×10−7DILSET_35,50.34690.41180.69946.9×10−66×10−75,100.19370.50680.34758.0×10−65×10−710,50.83390.75770.93928.1×10−67×10−710,100.95650.15420.63267.0×10−65×10−7


### 4.2.2. Efficiency

Different from the CV which requiresk-profiled models for the validation, PSV shares the same model for all test blocks, which means the ρPSV-test spends less time in the profiling phase than the ρCV-test. We compare the execution time of the ρPSV-test and ρCV-test on an HP EliteBook 735 G6 (AMD Ryzen 5 PRO 3500U CPU @ 2.1 GHz, 8 GB RAM) with Windows 10 as the operating system. The time overhead of 1,000,000 tests on DILSET_1 is shown in Figure 3. Compared with the time cost of 1,000,000 ρCV-tests of at least 676.94 seconds when k=5, the maximum time cost of 1,000,000 ρPSV-tests is 144.98 seconds when m,k=10,5, saving about 79% of the time costs. In short, the ρPSV-test is more time-efficient than the ρCV-test.Figure 3
Time cost of theρPSV-test vs. ρCV-test. The solid line is the time cost of the ρPSV-test, and the dashed line is the time cost of the ρCV-test.


## 4.2.1. Effectiveness

We perform theρPSV-test on DILSET_1, 2, and 3 and then test the estimates r^z,PSVj and r^z,PSV. Since each PSV block is independent of other blocks, ρr^z,PSVj1,r^z,PSVj2≡0, j1≠j2. For example, in the case of DILSET_1 and m,k=10,5, the obtained results of correlation tests between r^z,PSVj are given in Table 8. At the significance level of 0.01, the null hypothesis ρ=0 holds. Table 9 provides the p values of the distribution tests on r^z,PSV and EFPRs of the ρPSV-test. Since all the p values exceed the acceptable level of 0.01, the PSV-based estimates r^z,PSV pass the test for N0,1. In addition, compared to EFPRs of the ρCV-test (Table 1), EFPRs of ρPSV-tests are much closer to PFPRs. Hence, PSV is an effective solution to the AE and UFP.Table 8
Thep values of Pearson’s correlation test between r^z,PSVj (DILSET_1, m=10, k=5).
j2=1j2=2j2=3j2=4j2=5j1=1—0.34710.61850.91310.0447j1=20.3471—0.76030.01510.7986j1=30.61850.7603—0.01390.8772j1=40.91310.01510.0139—0.6937j1=50.04470.79860.87720.6937—Table 9
The test results for the AE and UFP in theρPSV-test.
DILSETm,k-PSVt-testχ2-testKS-testTdet=4.5Tdet=5.0DILSET_15,50.36630.17370.54756.6×10−65×10−75,100.19690.82440.63057.5×10−69×10−710,50.96050.63950.83677.6×10−66×10−710,100.64260.55360.37595.9×10−67×10−7DILSET_25,50.44440.19160.03666.7×10−65×10−75,100.87940.38180.98118.8×10−69×10−710,50.11020.52670.06236.5×10−61.1×10−610,100.74680.32700.98346.1×10−65×10−7DILSET_35,50.34690.41180.69946.9×10−66×10−75,100.19370.50680.34758.0×10−65×10−710,50.83390.75770.93928.1×10−67×10−710,100.95650.15420.63267.0×10−65×10−7


## 4.2.2. Efficiency

Different from the CV which requiresk-profiled models for the validation, PSV shares the same model for all test blocks, which means the ρPSV-test spends less time in the profiling phase than the ρCV-test. We compare the execution time of the ρPSV-test and ρCV-test on an HP EliteBook 735 G6 (AMD Ryzen 5 PRO 3500U CPU @ 2.1 GHz, 8 GB RAM) with Windows 10 as the operating system. The time overhead of 1,000,000 tests on DILSET_1 is shown in Figure 3. Compared with the time cost of 1,000,000 ρCV-tests of at least 676.94 seconds when k=5, the maximum time cost of 1,000,000 ρPSV-tests is 144.98 seconds when m,k=10,5, saving about 79% of the time costs. In short, the ρPSV-test is more time-efficient than the ρCV-test.Figure 3
Time cost of theρPSV-test vs. ρCV-test. The solid line is the time cost of the ρPSV-test, and the dashed line is the time cost of the ρCV-test.


## 4.3. Measured Experiments

We verify whether theρPSV-test can identify the plaintext-dependent leakages in the captured power traces. In our measured experiments, the ρPSV-test assesses the dependency between the leakages and first 4 plaintext bytes in DPA contest v4.2. Let m,k=5,5,5,10,10,5, and 10,10 respectively; we run the ρPSV-test on DPA contest v4.2 and plot the estimates r^z,PSV at the time domain 375μs,390μs, as shown in Figure 4. When Tdet=5.0, the 4 plaintext bytes are leaked in the intervals 379.96μs,381.02μs, 380.80μs,382.05μs, 381.64μs,382.89μs, and 382.48μs,383.73μs. In other words, the plaintext-dependent leakages are successfully identified by the ρPSV-test from the captured traces.Figure 4
Univariate 1st-order ρPSV-tests on DPA contest v4.2. The dashed line is the detection threshold 5.0. (a) m=5,k=5. (b) m=5,k=10. (c) m=10,k=5. (d) m=10,k=10.
(a)(b)(c)(d)


## 5. Higher-Order Leakage Detection

Higher-order leakage detection evaluates the security of the protected implementation against higher-order side-channel analysis. In this section, we analyze the performance of the PSV method in detecting higher-order leakages.


### 5.1. Combining Function

In higher-order side-channel analysis, a combining function maps the leakages of multiple shares to a simple univariate leakage. The work of Prouff et al. has demonstrated the central product function:(25)Cτ=∏j=1dLτj−1N∑i=1NLiτj,where Liτj is the ith leakage at time point τj in trace set ℒ and is optimal for the Hamming weight leakage scenario represented by the smart card platform [29]. In our experiments, the central product function is selected to preprocess the leakages of the masking. According to the classification in [30], we analyze the performance of PSV under the following combining functions:(1)
The central product function with the number of variablesd=1 and the order of statistical moment o=2,(26)C12τ=Lτ−1N∑i=1NLiτ2.(2)
The central product function withd=1 and o=3,(27)C13τ=Lτ−1N∑i=1NLiτ3.(3)
The central product function withd=2 and o=2,(28)C22τ=∏j=12Lτj−1N∑i=1NLiτj.(4)
The central product function is withd=3 and o=3:(29)C33τ=∏j=13Lτj−1N∑i=1NLiτj.


### 5.2. Effectiveness and Efficiency

#### 5.2.1. Effectiveness

As described in Section3, the p values of the distribution tests on r^z quantify the AE and the UFP in the ρ-test. Thus, we test the distribution of r^z,PSV to demonstrate the effectiveness of PSV in higher-order leakage detection, where the r^z,PSV is calculated from the combined DILSETs with different o,d. For example, when o,d=2,2, we compute the estimates r^z,CV and r^z,PSV as follows. In addition to the vector X′, two sets of leakages l∼N0,1 are randomly generated, so that Lτ1╨Lτ2╨X′. Then, the leakages Lτ1 and Lτ2 are combined by the central product function (28). Finally, we perform the ρCV-test or ρPSV-test on the combined leakages to obtain r^z,CV or r^z,PSV. For each o,d, Table 10 provides the results of the distribution tests on r^z,CV and r^z,PSV. At the significance level of 0.01, r^z,CV is rejected by the test of σ2=1 and N0,1, while r^z,PSV passes. It is proved that the AE and UFP in the higher-order ρCV-test can be solved by the PSV method.Table 10
Test results for the AE and UFP in the higher-orderρ-test.
Validation2nd-order univariate ρ-test2nd-order bivariate ρ-testt-testχ2-testKS-testt-testχ2-testKS-test5-fold CV0.142007.4164×10−290.119003.6605×10−3910-fold CV0.830603.2023×10−320.263502.3100×10−3615-fold CV0.554002.2630×10−370.176903.3470×10−4020-fold CV0.189903.0929×10−370.598603.5635×10−335,5-PSV0.57190.19140.81200.33070.24940.21215,10-PSV0.40670.04770.58990.02340.35180.091610,5-PSV0.23510.38990.35630.05550.60040.114610,10-PSV0.32650.24400.49410.02130.09940.0212Validation3nd-order univariate ρ-test3nd-order 3-variate ρ-testt-testχ2-testKS-testt-testχ2-testKS-test5-fold CV0.697402.3462×10−340.637208.8311×10−3510-fold CV0.900301.1018×10−290.172506.6531×10−3515-fold CV0.314908.2116×10−360.850902.6746×10−3320-fold CV0.997901.6713×10−300.741401.2006×10−335,5-PSV0.62060.88540.64930.65120.91850.91235,10-PSV0.11660.72950.39530.77610.03270.498210,5-PSV0.79670.19010.48320.46070.30870.569010,10-PSV0.55510.85650.76780.99650.20550.2865


#### 5.2.2. Efficiency

The time costs of 1,000,000 2nd-order ρ-tests are shown in Figure 5. As shown in Figure 5(a), compared with the time cost of 1,000,000 univariate 2nd-order ρCV-tests of at least 743.85 seconds, the maximum time cost of the 1,000,000 ρPSV-test is 161.31 seconds, saving about 78% of the time overhead of detections. In Figure 5(b), compared with the minimum time cost of the bivariate 2nd-order ρCV-test of 767.80 seconds, the ρPSV-test takes at most 187.23 seconds, saving about 76% of the time overhead. In other words, the higher-order ρPSV-test is more time-efficient than the higher-order ρCV-test.Figure 5
Time cost of the 2nd-order ρPSV-test vs. 2nd-order ρCV-test. The solid line is the time cost of the ρPSV-test, and the dashed line is the time cost of the ρCV-test. (a) o=2,d=1. (b) o=2,d=2.
(a)(b)


### 5.3. Measured Experiments

We use (26) to preprocess DPA contest v4.2 and run the ρPSV-test on the preprocessed traces. Figure 6 shows the results of the ρPSV-test between the traces and the first 4 plaintext bytes. When Tdet=5.0, the first 4 bytes are leaked in the intervals 379.97μs,381.00μs, 380.81μs,381.81μs, 381.65μs,382.69μs, and 382.49μs,383.59μs, which means that the univariate 2nd-order leakages in the measured power consumption are identified by the ρPSV-test.Figure 6
Univariate 2nd-order ρPSV-tests on DPA contest v4.2. The dashed line is the detection threshold 5.0. (a) m=5,k=5. (b) m=5,k=10. (c) m=10,k=5. (d) m=10,k=10.
(a)(b)(c)(d)


## 5.1. Combining Function

In higher-order side-channel analysis, a combining function maps the leakages of multiple shares to a simple univariate leakage. The work of Prouff et al. has demonstrated the central product function:(25)Cτ=∏j=1dLτj−1N∑i=1NLiτj,where Liτj is the ith leakage at time point τj in trace set ℒ and is optimal for the Hamming weight leakage scenario represented by the smart card platform [29]. In our experiments, the central product function is selected to preprocess the leakages of the masking. According to the classification in [30], we analyze the performance of PSV under the following combining functions:(1)
The central product function with the number of variablesd=1 and the order of statistical moment o=2,(26)C12τ=Lτ−1N∑i=1NLiτ2.(2)
The central product function withd=1 and o=3,(27)C13τ=Lτ−1N∑i=1NLiτ3.(3)
The central product function withd=2 and o=2,(28)C22τ=∏j=12Lτj−1N∑i=1NLiτj.(4)
The central product function is withd=3 and o=3:(29)C33τ=∏j=13Lτj−1N∑i=1NLiτj.


## 5.2. Effectiveness and Efficiency

### 5.2.1. Effectiveness

As described in Section3, the p values of the distribution tests on r^z quantify the AE and the UFP in the ρ-test. Thus, we test the distribution of r^z,PSV to demonstrate the effectiveness of PSV in higher-order leakage detection, where the r^z,PSV is calculated from the combined DILSETs with different o,d. For example, when o,d=2,2, we compute the estimates r^z,CV and r^z,PSV as follows. In addition to the vector X′, two sets of leakages l∼N0,1 are randomly generated, so that Lτ1╨Lτ2╨X′. Then, the leakages Lτ1 and Lτ2 are combined by the central product function (28). Finally, we perform the ρCV-test or ρPSV-test on the combined leakages to obtain r^z,CV or r^z,PSV. For each o,d, Table 10 provides the results of the distribution tests on r^z,CV and r^z,PSV. At the significance level of 0.01, r^z,CV is rejected by the test of σ2=1 and N0,1, while r^z,PSV passes. It is proved that the AE and UFP in the higher-order ρCV-test can be solved by the PSV method.Table 10
Test results for the AE and UFP in the higher-orderρ-test.
Validation2nd-order univariate ρ-test2nd-order bivariate ρ-testt-testχ2-testKS-testt-testχ2-testKS-test5-fold CV0.142007.4164×10−290.119003.6605×10−3910-fold CV0.830603.2023×10−320.263502.3100×10−3615-fold CV0.554002.2630×10−370.176903.3470×10−4020-fold CV0.189903.0929×10−370.598603.5635×10−335,5-PSV0.57190.19140.81200.33070.24940.21215,10-PSV0.40670.04770.58990.02340.35180.091610,5-PSV0.23510.38990.35630.05550.60040.114610,10-PSV0.32650.24400.49410.02130.09940.0212Validation3nd-order univariate ρ-test3nd-order 3-variate ρ-testt-testχ2-testKS-testt-testχ2-testKS-test5-fold CV0.697402.3462×10−340.637208.8311×10−3510-fold CV0.900301.1018×10−290.172506.6531×10−3515-fold CV0.314908.2116×10−360.850902.6746×10−3320-fold CV0.997901.6713×10−300.741401.2006×10−335,5-PSV0.62060.88540.64930.65120.91850.91235,10-PSV0.11660.72950.39530.77610.03270.498210,5-PSV0.79670.19010.48320.46070.30870.569010,10-PSV0.55510.85650.76780.99650.20550.2865


### 5.2.2. Efficiency

The time costs of 1,000,000 2nd-order ρ-tests are shown in Figure 5. As shown in Figure 5(a), compared with the time cost of 1,000,000 univariate 2nd-order ρCV-tests of at least 743.85 seconds, the maximum time cost of the 1,000,000 ρPSV-test is 161.31 seconds, saving about 78% of the time overhead of detections. In Figure 5(b), compared with the minimum time cost of the bivariate 2nd-order ρCV-test of 767.80 seconds, the ρPSV-test takes at most 187.23 seconds, saving about 76% of the time overhead. In other words, the higher-order ρPSV-test is more time-efficient than the higher-order ρCV-test.Figure 5
Time cost of the 2nd-order ρPSV-test vs. 2nd-order ρCV-test. The solid line is the time cost of the ρPSV-test, and the dashed line is the time cost of the ρCV-test. (a) o=2,d=1. (b) o=2,d=2.
(a)(b)


## 5.2.1. Effectiveness

As described in Section3, the p values of the distribution tests on r^z quantify the AE and the UFP in the ρ-test. Thus, we test the distribution of r^z,PSV to demonstrate the effectiveness of PSV in higher-order leakage detection, where the r^z,PSV is calculated from the combined DILSETs with different o,d. For example, when o,d=2,2, we compute the estimates r^z,CV and r^z,PSV as follows. In addition to the vector X′, two sets of leakages l∼N0,1 are randomly generated, so that Lτ1╨Lτ2╨X′. Then, the leakages Lτ1 and Lτ2 are combined by the central product function (28). Finally, we perform the ρCV-test or ρPSV-test on the combined leakages to obtain r^z,CV or r^z,PSV. For each o,d, Table 10 provides the results of the distribution tests on r^z,CV and r^z,PSV. At the significance level of 0.01, r^z,CV is rejected by the test of σ2=1 and N0,1, while r^z,PSV passes. It is proved that the AE and UFP in the higher-order ρCV-test can be solved by the PSV method.Table 10
Test results for the AE and UFP in the higher-orderρ-test.
Validation2nd-order univariate ρ-test2nd-order bivariate ρ-testt-testχ2-testKS-testt-testχ2-testKS-test5-fold CV0.142007.4164×10−290.119003.6605×10−3910-fold CV0.830603.2023×10−320.263502.3100×10−3615-fold CV0.554002.2630×10−370.176903.3470×10−4020-fold CV0.189903.0929×10−370.598603.5635×10−335,5-PSV0.57190.19140.81200.33070.24940.21215,10-PSV0.40670.04770.58990.02340.35180.091610,5-PSV0.23510.38990.35630.05550.60040.114610,10-PSV0.32650.24400.49410.02130.09940.0212Validation3nd-order univariate ρ-test3nd-order 3-variate ρ-testt-testχ2-testKS-testt-testχ2-testKS-test5-fold CV0.697402.3462×10−340.637208.8311×10−3510-fold CV0.900301.1018×10−290.172506.6531×10−3515-fold CV0.314908.2116×10−360.850902.6746×10−3320-fold CV0.997901.6713×10−300.741401.2006×10−335,5-PSV0.62060.88540.64930.65120.91850.91235,10-PSV0.11660.72950.39530.77610.03270.498210,5-PSV0.79670.19010.48320.46070.30870.569010,10-PSV0.55510.85650.76780.99650.20550.2865


## 5.2.2. Efficiency

The time costs of 1,000,000 2nd-order ρ-tests are shown in Figure 5. As shown in Figure 5(a), compared with the time cost of 1,000,000 univariate 2nd-order ρCV-tests of at least 743.85 seconds, the maximum time cost of the 1,000,000 ρPSV-test is 161.31 seconds, saving about 78% of the time overhead of detections. In Figure 5(b), compared with the minimum time cost of the bivariate 2nd-order ρCV-test of 767.80 seconds, the ρPSV-test takes at most 187.23 seconds, saving about 76% of the time overhead. In other words, the higher-order ρPSV-test is more time-efficient than the higher-order ρCV-test.Figure 5
Time cost of the 2nd-order ρPSV-test vs. 2nd-order ρCV-test. The solid line is the time cost of the ρPSV-test, and the dashed line is the time cost of the ρCV-test. (a) o=2,d=1. (b) o=2,d=2.
(a)(b)


## 5.3. Measured Experiments

We use (26) to preprocess DPA contest v4.2 and run the ρPSV-test on the preprocessed traces. Figure 6 shows the results of the ρPSV-test between the traces and the first 4 plaintext bytes. When Tdet=5.0, the first 4 bytes are leaked in the intervals 379.97μs,381.00μs, 380.81μs,381.81μs, 381.65μs,382.69μs, and 382.49μs,383.59μs, which means that the univariate 2nd-order leakages in the measured power consumption are identified by the ρPSV-test.Figure 6
Univariate 2nd-order ρPSV-tests on DPA contest v4.2. The dashed line is the detection threshold 5.0. (a) m=5,k=5. (b) m=5,k=10. (c) m=10,k=5. (d) m=10,k=10.
(a)(b)(c)(d)


## 6. Conclusions

Assumption error (AE) invalidates side-channel security assessment. This study finds that the false-positive rate of leakage detection might be mispredicted due to potential errors between the assumed distribution and the true distribution of the estimated test statistics. We notice underpredicted false-positive (UFP) in [10]. This underprediction, interpreted as the AE in the statistical distribution of the estimates of ρ-statistics, is caused by the overlap between the training and the test blocks in cross-validation. In addition, we propose the profiling-shared validation (PSV) to improve the detection of any-variate any-order leakages and show that the UFP and AE can be addressed by eliminating the between-block overlap. Compared with the ρCV-test, our ρPSV-test overcomes the UFP and only takes less than 25% of the time cost. To the best of our knowledge, this article is the first empirical study of the false-positives in leakage detection. In future work, we will refine our tools, including the estimation algorithm and the distribution tests, to preevaluate other security assessments.



---

*Source: 6904232-2022-02-12.xml*