SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Is SPQR really affecting independence? We perform independence hypothesis testing and visualize the spectral distribution to investigate whether SPQR really affects the independence of the Q-ensemble. First, we analyze the performance gain of SPQR from a spike reduction perspective. We train a Q-ensemble using SAC-Min and SPQR-SAC-Min on the hopper-random dataset and test it on the hopper-full-replay dataset. Figure 6 and Table 1 show the evaluation results for the spike-reducing phenomenon of SPQR, which deviates less from Wigner's semicircle law. Compared to SPQR-SAC-Min with  $\beta = 0.1$ , SAC-Min has significantly more spikes, in particular, 2434 spikes compared to 2182 spikes over 25600 data. For extreme regularization gain,  $\beta = 100$ , the spike is drastically reduced to 698,  $71\%$  less than the baseline algorithm. We also plot a spike histogram for another diversification method, EDAC. EDAC increases spikes to  $54\%$  more than SAC-Min, which can be interpreted as SPQR being the orthogonal method with EDAC.
We also demonstrate the  $\chi^2$  test with null hypothesis  $H_0: P(Q_i, Q_j) = P(Q_i)P(Q_j)$  and significant level  $\alpha = 0.025$  for independence hypothesis testing on the same experimental setting above. Since SPQR promotes being "more" independent, we use the proportion of data that can accept an independent hypothesis as a measure of more or less independence. In Table 1, we report the proportion of trials over 1000 tests that accept the null hypothesis  $H_0$ , meaning that the Q-ensemble is independent. These empirical studies can be interpreted as SPQR having Q-ensemble independence regularization power since the acceptance rate increases monotonically as  $\beta$  increases. Orthogonality between SPQR and EDAC is also found from an independence testing perspective. However, reducing spikes and being more independent does not guarantee optimal performance since there is an optimal conservatism for each task. We can find the proper conservatism by controlling the independence regularization power,  $\beta$ . More details can be found in Appendix E.
<https://github.com/watchernyu/REDQ>
$^{2}$ https://github.com/snu-mllab/EDAC
<sup>3</sup>https://github.com/aviralkumar2907/CQL
Figure 6: Histogram of spikes for Wigner's semicircle (black line), SAC-Min (blue), EDAC (orange), and SPQR-  $\beta = 100.0$  (green). For effective visualization, we plot only the eigenvalues that violate Wigner's semicircle.
Table 1: Number of spikes, independence hypothesis testing acceptance ratio, and performance of SAC-Min, EDAC, and SPQR $\beta = 0.01, 0.1, 1.0, 10.0, 100.0$ . We reproduce EDAC- $\eta = 1.0$ . The reported value of EDAC- $\eta = 1.0$  is given in the Section H of [2]
Algorithm	Spike	Performance	Accept ratio
SAC-Min	2434	31.3 ± 0.0	30.4%
EDAC-η = 1.0	3751	10.3 ± 0.2	0.0%
SPQR-β = 0.01	2360	33.6 ± 1.8	59.0%
SPQR-β = 0.1	2182	35.6 ± 1.4	80.4%
SPQR-β = 1.0	1844	32.6 ± 0.4	91.4%
SPQR-β = 10.0	1603	32.7 ± 0.4	92.3%
SPQR-β = 100.0	698	32.9 ± 0.8	97.6%
Online Reinforcement Learning. We implement SPQR-SAC-Ens and SPQR-REDQ with UTD ratio 1 and 20, and compare them to SAC-Ens and REDQ to show how SPQR improves performance. We evaluate each algorithm on the OpenAI Gym MuJoCo tasks. Figure 7 shows the training curves for SAC-Ens, REDQ, SPQR-SAC-Ens, and SPQR-REDQ. Since the two main contributions of REDQ, UTD ratio, and subset minimization, are orthogonal methods to each other, we experiment with SPQR-REDQ for UTD ratio 1 and 20. A high UTD ratio can be interpreted as frequent gradient updates, thereby resulting in better performance. For each algorithm, we plot the average return as a solid curve and the standard deviation as a shaded area. The implementation of SPQR yields notable improvements in the performance of SAC-Ens and REDQ across all tasks, as well as under both UTD ratio 1 and 20 conditions. Specifically, SPQR-SAC-Ens performs 2 times better than SAC-Ens on the Hopper task. Also, SPQR-REDQ-UTD=1 and SPQR-REDQ-UTD=20 show 1.1 and 1.3 times better than REDQ-UTD=1 and REDQ-UTD=20 for the Walker2d and Ant tasks.
Figure 7: Average returns of SPQR-SAC-Ens, SPQR-REDQ-UTD=1, and SPQR-REDQ-UTD=20 on the MuJoCo environment. Averaged over 4 random seeds.
To investigate why SPQR performs better, we evaluate the bias of the Q-value computed by the Monte-Carlo algorithm. The average of the Q-bias decreases with SPQR, which can be interpreted as the relationship between the distribution of the ensemble becoming more independent. We also evaluate the ablation study for the UTD ratio. SPQR-SAC-Ens shows performance improvement and bias alleviation with low standard deviation as the UTD ratio increases although high UTD SAC causes large and unstable bias. In terms of diversity, we also evaluate the standard deviation with different  $\beta$  for SPQR. As  $\beta$  increases, the standard deviation for explored state-action also increases in all training timesteps. In summary, Q-ensemble independence is the key factor in improving the performance of SPQR. Further details and experimental results can be found in Appendix E.
Offline Reinforcement Learning. SAC-Min is formally constructed and evaluated in [2]. We follow the reported value and evaluation protocol from the paper. We consider baseline behavior cloning (BC), CQL, EDAC, and SAC-Min. The evaluation results of SPQR-SAC-Min compared with SAC-Min and other baselines are shown in Table 2. For most tasks, SPQR outperforms all baselines well. Specifically, for *-random dataset, SPQR improves SAC-Min by about  $16\%$  when CQL and EDAC fail for *-random dataset. It implies that SPQR is robust for the quality of data. Since SPQR gives more conservatism, which shows identical behavior with increasing ensemble size  $N$ , SPQR-SAC-Min needs a smaller number of ensemble networks to outperform SAC-Min. For the hopper-medium, SAC-Min uses 500 Q-networks and SPQR uses only 50 Q-networks, which is about  $90\%$  efficient in terms of computational cost. It implies that the independence regularization of the Q-ensemble becomes computationally efficient.
We also implement SPQR to CQL  $(\mathcal{H})$ , denoted as SPQR-CQL-Min. Referring to the implementation details in the paper and the original baseline code of CQL, the original CQL uses a twin Q-function trick, which is equivalent to SAC-Min with  $N = 2$ . We modified CQL as CQL-Min to use  $N$
Table 2: Normalized average returns of SPQR-SAC-Min and SPQR-CQL-Min on D4RL Gym, Franka Kitchen, and Antmaze tasks, averaged over 4 random seeds. Except for SPQR- $*$ , the evaluated values of baselines are reported refer to [2, 21], and reproduced for  $*$ -full-replay tasks of CQL-Min. Reproduced values are indicated as italic font.
Gym	BC	CQL-Min	EDAC	SAC-Min	SPQR-SAC-Min
walker2d-random	1.3 ± 0.1	7.0	16.6 ± 7.0	21.7 ± 0.0	24.6 ± 1.1
walker2d-medium	70.9 ± 11.0	74.5	92.5 ± 0.8	87.9 ± 0.2	98.4 ± 2.0
walker2d-expert	108.7 ± 0.2	121.6	115.1 ± 1.9	116.7 ± 0.4	115.2 ± 1.3
walker2d-medium-expert	90.1 ± 13.2	98.7	114.7 ± 0.9	116.7 ± 0.4	118.2 ± 0.7
walker2d-medium-replay	20.3 ± 9.8	32.6	87.1 ± 2.3	78.7 ± 0.7	87.8 ± 2.5
walker2d-full-replay	68.8 ± 17.7	98.96	99.8 ± 0.7	94.6 ± 0.5	101.1 ± 0.4
halfcheetah-random	2.2 ± 0.0	35.4	28.4 ± 1.0	28.0 ± 0.9	33.5 ± 2.5
halfcheetah-medium	43.2 ± 0.6	44.4	65.9 ± 0.6	67.5 ± 1.2	74.8 ± 1.3
halfcheetah-expert	91.8 ± 1.5	104.8	106.8 ± 3.4	105.2 ± 2.6	112.8 ± 0.3
halfcheetah-medium-expert	44.0 ± 1.6	62.4	106.3 ± 1.9	107.1 ± 2.0	114.0 ± 1.6
halfcheetah-medium-replay	37.6 ± 2.1	46.2	61.3 ± 1.9	63.9 ± 0.8	69.7 ± 1.4
halfcheetah-full-replay	62.9 ± 0.8	82.07	84.6 ± 0.9	84.5 ± 1.2	88.5 ± 0.5
hopper-random	3.7 ± 0.6	7.0	25.3 ± 10.4	31.3 ± 0.0	35.6 ± 1.4
hopper-medium	54.1 ± 3.8	86.6	101.6 ± 0.6	100.3 ± 0.3	100.2 ± 1.3
hopper-expert	107.7 ± 9.7	109.9	110.1 ± 0.1	110.3 ± 0.3	112.0 ± 0.2
hopper-medium-expert	53.9 ± 4.7	111.0	110.7 ± 0.1	110.1 ± 0.3	112.5 ± 0.3
hopper-medium-replay	16.6 ± 4.8	48.6	101.0 ± 0.5	101.8 ± 0.5	104.9 ± 0.7
hopper-full-replay	19.9 ± 12.9	104.85	105.4 ± 0.7	102.9 ± 0.3	109.1 ± 0.4
Average	49.9	70.9	85.2	84.5	89.6
Franka Kitchen	BC	SAC-Min	BEAR	CQL-Min	SPQR-CQL-Min
kitchen-complete	33.8	15.0	0.0	43.8	47.9 ± 12.3
kitchen-partial	33.8	0.0	13.1	49.8	54.2 ± 7.2
kitchen-mixed	47.5	2.5	47.2	51.0	55.6 ± 7.9
Average	38.4	5.8	20.1	48.2	52.6
Antmaze	BC	SAC-Min	BEAR	CQL-Min	SPQR-CQL-Min
umaze	65.0	0.0	73.0	74.0	93.3 ± 4.7
umaze-diverse	55.0	0.0	61.0	84.0	80.0 ± 0.0
medium-play	0.0	0.0	0.0	61.2	80.0 ± 8.2
Average	40.0	0.0	44.7	73.1	84.4
Q-networks to compute the Bellman target. Hence, the original CQL can be called CQL-Min with  $N = 2$ . We evaluate SPQR-CQL-Min on D4RL Franka Kitchen and Antmaze, one of the most complex and sparse tasks in D4RL.
In Table 2, we evaluate SPQR-CQL-Min with other baseline methods. SPQR-CQL-Min outperforms all baseline algorithms, notably 2.6 times better than BEAR. For Franka Kitchen tasks, SPQR consistently improves CQL by  $10\%$  for all three tasks, which can be interpreted as robustness over the quality of data. Surprisingly, SPQR achieves the maximum score on 3 different Antmaze tasks, which outperforms all existing algorithms. Further details and results are presented in Appendix D and E.
Can other diversification methods benefit from the SPQR? Since SPQR regularizes Q-ensemble independence, which is the orthogonal method with other diversification methods, we implement SPQR in EDAC to benefit from the combination of two orthogonal methods. We evaluate SPQR-EDAC for the tasks where EDAC outperforms the baseline algorithm SAC-Min. SPQR and SPQR-EDAC show large performance gains over SAC-Min, as shown in Table 3.
Table 3: Normalized average returns of SPQR-SAC-Min and SPQR-EDAC on D4RL Gym tasks, averaged over 4 random seeds. Except for SPQR- $*$ , the evaluated values of baselines are reported. Refer to [2].
Gym	EDAC	SAC-Min	SPQR-SAC-Min	SPQR-EDAC
walker2d-medium	92.5 ± 0.8	87.9 ± 0.2	98.4 ± 2.0	94.8 ± 1.0
walker2d-medium-replay	87.1 ± 2.3	78.7 ± 0.7	87.8 ± 2.5	89.6 ± 1.8
walker2d-full-replay	99.8 ± 0.7	94.6 ± 0.5	101.1 ± 0.4	102.2 ± 0.2
halfcheetah-expert	106.8 ± 3.4	105.2 ± 2.6	112.8 ± 0.3	110.3 ± 0.2
halfcheetah-full-replay	84.6 ± 0.9	84.5 ± 1.2	88.5 ± 0.5	86.8 ± 0.2
hopper-medium	101.6 ± 0.6	100.3 ± 0.3	100.2 ± 1.3	103.8 ± 0.1
hopper-medium-expert	110.7 ± 0.1	110.1 ± 0.3	112.5 ± 0.3	111.7 ± 0.0
hopper-full-replay	105.4 ± 0.7	102.9 ± 0.3	109.1 ± 0.4	108.2 ± 0.5
We propose SPQR, a Q-ensemble independence regularization framework for ensemble Q-learning by applying random matrix theory and a spiked random model. We theoretically propose a tractable loss for Q-ensemble independence. The proposed theoretical approach does not require any prior assumption on the Q-function distribution, such as uniform distribution or i.i.d. In addition, we present the difference between diversity and independence and empirically show the orthogonality of the diversification method and SPQR. Furthermore, SPQR serves as a conservatism control framework for ensemble Q-learning by effectively regularizing independence with various empirical evidence. We also empirically show that SPQR is computationally efficient and outperforms many existing ensemble Q-learning algorithms on various online and offline RL benchmark tasks.
We appreciate Minyoung Kim for creating an illustration for our paper, as well as Jaehoon Choi and Yejin Kim for writing advice. We also thank the anonymous reviewers for their insightful feedback and reviews. This work is in part supported by National Research Foundation of Korea (NRF, 2021R1A2C2014504(20%), 2021M3F3A2A02037893(20%)), Institute of Information & communications Technology Planning & Evaluation (IITP, 2021-0-00106(20%), 2021-0-01059(20%), 2021-0-02068(20%)) grant funded by the Ministry of Science and ICT (MSIT), INMAC and BK21 FOUR Program.
[33] Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
[34] Eugene P. Wigner. Characteristic vectors of bordered matrices with infinite dimensions. Annals of Mathematics, 62(3):548-564, 1955. ISSN 0003486X.
A Review for Detection Theory and Spiked Random Model
We call strong detection when we perfectly detect the presence of a spike signal and weak detection when we detect the presence of a spike signal with some error by hypothesis testing. A formal definition for a strong detection can be constructed under the notion of contiguity by [23]. The contiguity of the spiked Wishart model is proven by the second moment method, which means that we cannot distinguish without error whether data is pure noise or contains some signal.
Definition A.1 (Contiguity). Let  $P_{n}$  and  $Q_{n}$  be two sequences of probability measures defined on the sequence of measurable spaces  $(\Omega_{n},\mathcal{F}_{n})$ .  $Q_{n}$  is contiguous with respect to the sequence  $P_{n}$  if  $P_{n}(A_{n})\to 0$  implies  $Q_{n}(A_{n})\to 0$  as  $n\to \infty$  for every sequence of measurable sets  $A_{n}\in \mathcal{F}_{n}$ , denoted as  $Q_{n}\triangleleft P_{n}$ . The sequences  $P_{n}$  and  $Q_{n}$  are mutually contiguous if both  $Q_{n}\triangleleft P_{n}$  and  $P_{n}\triangleleft Q_{n}$  holds, denoted as  $Q_{n}\triangleleft \triangleright P_{n}$
Lemma A.2 (Le Cam's First Lemma). Let  $P_{n}$  and  $Q_{n}$  be two sequences of probability measures defined on the sequence of measurable spaces  $(\Omega_n, \mathcal{F}_n)$ . Then the following statements are equivalent: 1.  $Q_{n} \triangleleft P_{n}$
2. If  $dP_{n} / dQ_{n}\xrightarrow{Q_{n}}U$  along a subsequence, then  $P(U > 0) = 1$
3. If  $dQ_{n} / dP_{n}\xrightarrow{P_{n}}V$  along a subsequence, then  $\mathbb{E}[V] = 1$
4. For any statistics  $T_{n}:\Omega_{n}\mapsto \mathbb{R}^{k}$ , If  $T_{n}\xrightarrow{P_{n}}0$ , then  $T_{n}\xrightarrow{Q_{n}}0$
Lemma A.3 (Second moment method). Let  $P_{n}$  and  $Q_{n}$  be two sequences of probability measures defined on the sequence of measurable spaces  $(\Omega_{n},\mathcal{F}_{n})$ , and Radon-Nikodym derivative of  $P_{n}$  with respect to  $Q_{n}$  as  $L_{n}\equiv dP_{n} / dQ_{n}$ . If  $\lim \sup \mathbb{E}_{Q_n}[L_n^2 ] < \infty$ , then  $P_{n}\triangleleft Q_{n}$
Lemma A.4. For spiked wishart model following definition 3.2,  $K_u^4 K_v^4 \psi^2 < 1$ , the families of distributions  $\mathbb{P}_0$  and  $\mathbb{P}_{\psi}$ , indexed by  $M, N$ , are mutually contiguous in the limit  $N \to \infty$ ,  $M / N \to \omega$ .
Hypothesis Testing for Weak Detection. To distinguish whether spiked Wishart model  $Y$  follows  $\mathbb{P}_{\psi}$  or  $\mathbb{P}_0$ , detection error is inevitable since contiguity holds. Even if the perfect test is not available, it needs to be better than a random guess, which is achievable by constructing a hypothesis test. Let  $(\Omega, \mathcal{F}, \hat{\rho})$  be a probability space,  $(\mathcal{X}, \mathcal{B})$  be a topological space with Borel  $\sigma$ -algebra  $\mathcal{B}$ , and  $\Theta$  to be a set of parameters. For observed random variable  $X: \Omega \mapsto \mathcal{X}$  with  $X \sim P_{\theta}$ ,  $\theta \in \Theta$ , and fixed  $\hat{\theta} \neq 0$ , Objective of hypothesis testing is to distinguish between two hypotheses, called null hypothesis  $H_0$  and alternative hypothesis  $H_1$
$$
H _ {0}: \theta = 0, H _ {1}: \theta = \hat {\theta} \tag {3}
$$
Constructing a measurable function  $T: \mathcal{X} \mapsto \{0,1\}$  to represent test,  $T(X) = 0$  means accept  $H_0$  and  $T(X) = 1$  means accept  $H_1$ . The error of test  $T$  is the sum of Type-I error, false positive, and Type-II error, false negative, stated as,
$$
e r r (T) = P _ {\hat {\theta}} (T (X) = 0) + P _ {0} (T (X) = 1) \tag {4}
$$
Optimal test  $T^{*}$  is defined as minimizing Type-II error subject to the constraint that Type-I error is no greater than a predetermined level  $\alpha$ .  $T^{*}$  must have a form of likelihood ratio test by following Neyman-Pearson Lemma and propositions from [19].
Lemma A.5 (Neyman-Pearson Lemma). Let null hypothesis  $H_0$  with probability measure  $p_0$  and density  $f_0$ , alternative hypothesis  $H_1$  with probability measure  $p_1$  and density  $f_1$ , density ratio  $r(x) = f_1(x) / f_0(x)$ , and constant  $k \in \mathbb{R}$  such that  $p_0\{r(x) > k\} \leq \alpha$  for a given significance level  $\alpha$ . There exist an optimal test function  $T^* : \mathbb{R}^D \to \mathbb{R}$ , with a mild continuity assumption, such that
$$
T ^ {*} (X) = \mathbb {1} \left(f _ {1} (x) / f _ {0} (x) > k\right) = \mathbb {1} (r (x) > k)
$$
To construct an optimal test by applying Neyman-Pearson Lemma, we define a likelihood ratio, Radon-Nikodym derivative between  $\mathbb{P}_{\psi}$  and  $\mathbb{P}_0$  as  $L(Y;\psi)$  by the following equations. For convenience, let Hamiltonian  $H:\mathbb{R}^{N + M}\to \mathbb{R}$  be as follows:
$$
- H (u, v) = \sum_ {i, j} \sqrt {\frac {\psi}{N}} Y _ {i j} u _ {i} v _ {j} - \frac {\psi}{2 N} u _ {i} ^ {2} v _ {j} ^ {2} \tag {5}
$$
$$
d P _ {u} ^ {\otimes N} (u) = d P _ {u} \left(u _ {1}\right) \otimes \dots \otimes d P _ {u} \left(u _ {N}\right) \tag {6}
$$
where  $\otimes$  represents a tensor product,  $L(Y;\psi)$  has form of
$$
L (Y; \psi) = \frac {d \mathbb {P} _ {\psi}}{d \mathbb {P} _ {0}} = \int e ^ {- H (u, v)} d P _ {u} ^ {\otimes N} (u) d P _ {v} ^ {\otimes M} (v) \tag {7}
$$
The optimal test for a spiked Wishart model is given as follows:
Lemma A.6. Let us assume that a spiked Wishart model is defined in definition 3.2. Let null hypothesis  $H_0: Y \sim \mathbb{P}_0$ , alternative hypothesis  $H_1: Y \sim \mathbb{P}_{\psi}$ , likelihood ratio  $L$  between  $\mathbb{P}_{\psi}$  and  $\mathbb{P}_0$ , and define tests  $T: \mathbb{R}^{N \times M} \to \{0,1\}$ . By Lemma A.5, optimal test is,
$$
T ^ {*} (Y) = \mathbb {1} (\log L > 0) = \left\{ \begin{array}{l l} H _ {0}: Y \sim \mathbb {P} _ {0}, & i f \log L \leq 0 \\ H _ {1}: Y \sim \mathbb {P} _ {\psi}, & i f \log L > 0 \end{array} \right.
$$
Proof of A.6 is given in Section B.
Corollary A.7. For optimal test  $T^{*}$ , Error of  $T^{*}$  is
$$
e r r _ {M, N} ^ {*} (T ^ {*}) = P _ {0} (\log L (Y; \psi) > 0) + P _ {\psi} \log L (Y; \psi) \leq 0) = 1 - D _ {T V} (\mathbb {P} _ {\psi}, \mathbb {P} _ {0})
$$
For  $\psi \geq 0$  such that  $K_u^4 K_v^4\psi^2 < 1$
$$
\lim  _ {N \rightarrow \infty} e r r _ {M, N} ^ {*} (T ^ {*}) = 1 - \lim  _ {N \rightarrow \infty} D _ {T V} (\mathbb {P} _ {\psi}, \mathbb {P} _ {0}) = e r f c \left(\frac {1}{4} \sqrt {- \log (1 - \psi^ {2})}\right)
$$
where  $erfc(x) = \frac{2}{\sqrt{\pi}}\int_{x}^{\infty}e^{-t^{2}}dt$
The error for optimal test  $T^{*}$  is stated at Corollary A.7. Likelihood ratio  $\log L$  is used for the optimal test to distinguish whether data  $Y$  in ensembles are sampled from i.i.d prior  $\mathbb{P}_0$ . However,  $\log L$  is intractable since information about  $\psi$  and  $u, v$  is not given. Therefore, we need an alternative criterion for optimal test  $T^{*}$  that does not require prior knowledge to make it tractable with ensemble Q-learning. Intuitively, although we cannot know the exact form of  $\mathbb{P}_{\psi}$  and  $\mathbb{P}_0$ , its eigenvalue distribution can be described by ESD and Wigner's semicircle law. The expectation of  $\log L$  becomes KL divergence between  $\mathbb{P}_{\psi}$  and  $\mathbb{P}_0$ , followed by the definition of  $\log L$ . By integrating these intuitions, intractable criterion  $\log L$  can be restated as KL divergence between ESD and Wigner's semicircle law, which we prove in Section B.
Proof of Lemma A.2. The proof is exactly same as the Lemma 6.4 of [33]
Proof of Lemma A.3. For any event  $A \in \mathcal{F}_n$ , by Cauchy-Schwarz inequality,
$$
\begin{array}{l} P _ {n} (A) = \int \mathbb {1} _ {A} (w) d P _ {n} (w) = \int \mathbb {1} _ {A} (w) L _ {n} (w) d Q _ {n} (w) \\ \leq \sqrt {\int L _ {n} ^ {2} d Q _ {n}} \cdot \sqrt {\int \mathbb {1} _ {A} d Q _ {n}} = \sqrt {\mathbb {E} _ {Q _ {n}} [ L _ {n} ^ {2} ]} \cdot \sqrt {Q _ {n} (A)} \\ \end{array}
$$
$\therefore$  If  $\lim \sup \mathbb{E}_{Q_n}[L_n^2] < \infty$ , then  $Q_{n}(A_{n})\to 0$  implies  $P_{n}(A_{n})\to 0$ , concludes to  $P_{n}\triangleleft Q_{n}$
Proof of Lemma A.4. The proof is exactly same as the Theorem 4 of [4]
B.1 Technical Lemmas and Proof
Before proving our theoretical results, we present some lemmas to clear the description.
Lemma B.1 (Convergence). Let  $\psi \geq 0$  such that  $K_u^4 K_v^4\psi^2 < 1$ , then in the limit  $N \to \infty$ , where  $\rightsquigarrow$  denotes convergence in distribution,
$$
\log L (Y; \psi) \rightsquigarrow \mathcal {N} \left(\pm \frac {1}{4} \log \left(1 - \psi^ {2}\right), - \frac {1}{2} \log \left(1 - \psi^ {2}\right)\right) \tag {8}
$$
The sign + means  $Y\sim \mathbb{P}_0$  -means  $Y\sim \mathbb{P}_{\psi}$
Lemma B.2 (Concentration). for fixed  $u^{*}, v^{*}$ ,  $Y = \sqrt{\frac{\psi}{N}} u^{*}v^{*^T} + W$ , for every  $t \geq 0$
$$
P r (| \log L (Y; \psi) - \mathbb {E} [ \log L (Y; \psi) ] | \geq N t) \leq 2 \exp (- N t ^ {2} / 2 \psi K _ {u} ^ {2} K _ {v} ^ {2})
$$
Lemma B.3 (Weyl's Lemma). If  $X$  is rotational,  $P[X] = g(Tr(X), Tr(X^2), \ldots, Tr(X^N))$
Lemma B.4 (Vandermonde determinant). For  $GOE Y = O\Lambda OT$ , the relationship between joint probability density function of matrix entity and eigenvalues is given as,
$$
P \left(Y _ {1 1}, Y _ {1 2}, \dots , Y _ {N N}\right) d Y _ {1 1} d Y _ {1 2} \dots d Y _ {N N} = P \left(\lambda_ {1}, \lambda_ {2}, \dots , \lambda_ {N}, \{O \}\right) d \lambda_ {1} d \lambda_ {2} \dots d \lambda_ {N} d \{O \}
$$
$$
P \left(\lambda_ {1}, \lambda_ {2}, \dots , \lambda_ {N}, \{O \}\right) = P \left(Y _ {1 1}, Y _ {1 2}, \dots , Y _ {N N}\right) | J (Y \rightarrow \lambda , \{O \}) |
$$
$$
w h e r e J (Y \rightarrow \lambda , \{O \}) = d Y _ {1 1} d Y _ {1 2} \dots d Y _ {N N} / d \lambda_ {1} d \lambda_ {2} \dots d \lambda_ {N} d \{O \}
$$
Then vandermonde determinant become
$$
| J (Y \rightarrow \lambda , \{O \}) | = | J (Y \rightarrow \lambda) | = | d Y / d \lambda | = \prod_ {i <   j} | \lambda_ {i} - \lambda_ {j} |
$$
Proof of Lemma B.1. The proof is exactly same as the Theorem 3.1 of [11]
Proof of Lemma B.2. The proof is exactly same as the Lemma 3.14 of [11]
Proof of Lemma B.3. The proof is exactly same as the Lemma 1.1 of [9]
Proof of Lemma B.4. The proof is exactly same as the procedure in Chapter 3 of [26]
Proof of lemma A.6. Using Lemma B.1 and Lemma A.5,  $\log L$  can distinguish at  $\log L = 0$  as  $N\to \infty$  because of symmetry of Gaussian mean, i.e  $k = 1$  for Neyman-Pearson Leamma. Therefore, optimal test  $T^{*} = \mathbb{1}(\log L > 0)$
B.2 Proof of Main Theorem
Proof. By Lemma B.3 and Lemma B.4, For spiked wishart model  $Y \sim \mathbb{P}_{\psi}$
$$
\begin{array}{l} p _ {\psi} (\lambda) = P \left(\lambda_ {1}, \lambda_ {2}, \dots , \lambda_ {N}\right) = \int d O P \left(\lambda_ {1}, \lambda_ {2}, \dots , \lambda_ {N}, \{O \}\right) \\ = \int d O P (Y _ {1 1}, Y _ {1 2}, \dots , Y _ {N N}) | J (Y \rightarrow \lambda , \{O \}) | \\ = | J _ {\psi} | \mathbb {P} _ {\psi} (Y) \int d O, \text {w h e r e} \int d O = \text {c o n s t a n t (v o l u m e)} > 0 \\ \end{array}
$$
If  $D_{KL}(p_{\psi}(\lambda), p_{sc}(\lambda)) < \varepsilon$
$$
\begin{array}{l} D _ {K L} \left(\mathbb {P} _ {\psi}, \mathbb {P} _ {0}\right) = \left| J _ {\psi} \right| D _ {K L} \left(p _ {\psi} (\lambda), p _ {s c} (\lambda)\right) + \left| J _ {\psi} \right| \log \frac {C \left| J _ {\psi} \right|}{\left| J _ {0} \right|}, C > 0 \\ \Rightarrow D _ {K L} \left(\mathbb {P} _ {\psi}, \mathbb {P} _ {0}\right) <   | J _ {\psi} | \varepsilon + | J _ {\psi} | \log \frac {C | J _ {\psi} |}{| J _ {0} |} = \tilde {\varepsilon}, \quad \because | J _ {\psi} | > 0 \\ \end{array}
$$
Let  $\Delta = \log L - \mathbb{E}[\log L],\hat{t} = Nt.$  By Lemma B.2,
$$
P r (| \Delta | \geq \hat {t}) \leq 2 \exp (- \hat {t} ^ {2} / 2 \psi N K _ {u} ^ {2} K _ {v} ^ {2})
$$
Let  $\delta = 2\exp (-\hat{t}^2 /2\psi NK_u^2 K_v^2)$ . Then, with probability at least  $1 - \delta$
$$
\Delta \in [ - \Delta_ {m a x}, \Delta_ {m a x} ]
$$
$$
\Delta_ {m a x} = \sqrt {2 N \psi K _ {u} ^ {2} K _ {v} ^ {2} \log (2 / \delta)}
$$
Since  $D_{KL}(\mathbb{P}_{\psi},\mathbb{P}_0) < \tilde{\varepsilon}$ , then,
$$
\log L = \mathbb {E} [ \log L ] + \Delta = D _ {K L} (\mathbb {P} _ {\psi}, \mathbb {P} _ {0}) + \Delta <   \tilde {\varepsilon} + \Delta \tag {9}
$$