SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
f (x) = \sum_ {i = 1} ^ {m} y _ {i} \log \left(\frac {y _ {i}}{\alpha (A x) _ {i}}\right) + \alpha (A x) _ {i} - y _ {i}, \tag {142}
$$
for  $A \in \mathbb{R}^{m \times n}$ . Note that as done in Bauschke et al. [2017], denoting  $(a_i)_{1 \leq i \leq n}$  the columns of  $A$ , we assume that  $a_i \neq 0_m$  and  $\forall 1 \leq j \leq m, \sum_{i=1}^n a_{i,j} > 0$  such that  $Ax \in \mathbb{R}_{++}^m$  if  $x \in \mathbb{R}_{++}^n$ . This is verified for  $A$  representing the blur with circular boundary conditions with the (normalized) kernels used in Section 5.2.
We first check Assumptions 1.
(ii)  $f$  is real analytic and thus subanalytic on  $\mathbb{R}_{++}^n$ .
(iii) It is shown in [Bauschke et al., 2017, Lemma 7] that  $f$  verifies the NoLip assumption, i.e.  $L_{f}h - f$  is convex on  $\mathbb{R}_{++}^{n}$ , for  $L_{f} \geq ||y||_{1}$ .  $y$  stands for the Poisson degraded observation appearing in the definition of  $f$ .
(iv) First,  $h$  is strongly convex everywhere on its domain except in  $+\infty$ . For  $C$  a bounded subset of  $\mathbb{R}_{++}^n$ , as  $\nabla^2 h(x) = \frac{1}{x^2}$  Id, we have  $\forall x \in C, \forall d \in \mathbb{R}^n$ ,  $\langle \nabla^2 h(x)d,d\rangle > \frac{1}{\sup_{x\in C}x^2} ||d||^2$  indicating that  $h$  is strongly convex on bounded subsets of  $\mathbb{R}_{++}^n$ . Second,  $h$  and  $f$  are Lipschitz continuous everywhere on  $\mathbb{R}_{++}^n$  except close to 0. For both B-RED and B-PnP  $\Psi (x)\to +\infty$  when  $x\to 0$  and  $\{\Psi (x)\leq \alpha \}$  avoids the case  $x\rightarrow 0$ .
(v) We remind the parameterizations  $\mathcal{B}_{\gamma} = \mathrm{Id} - \nabla g_{\gamma}$  and  $g_{\gamma}(y) = \frac{1}{2} ||x - N_{\gamma}(x)||^{2}$  with a U-Net  $N_{\gamma}$  (with softplus activations). The softplus activation function being real analytic, using the fact that the composition and sum of real analytic functions are real analytic [Krantz and Parks, 2002],  $N_{\gamma}$  and then subsequently  $g_{\gamma}$  are real analytic functions.
For the convergence of B-PnP, we also need to prove that  $\phi_{\gamma}$  is real analytic (and thus subanalytic) on its domain. For this assumption to be verified, we require in Theorem 2,  $\psi_{\gamma} \circ \nabla h^{*}$  not only to be convex, but to be strictly convex. Indeed, using the expression (64)  $\mathcal{B}_{\gamma}(y) = \nabla (\psi_{\gamma} \circ \nabla h^{*}) \circ \nabla h(y)$ , we have for  $y \in \operatorname{int} \operatorname{dom}(h)$
$$
\begin{array}{l} J _ {\mathcal {B} _ {\gamma}} (y) = \nabla (\nabla \left(\psi_ {\gamma} \circ \nabla h ^ {*}\right) \circ \nabla h (y)) (143) \\ = \nabla^ {2} h (y). \nabla^ {2} \left(\psi_ {\gamma} \circ \nabla h ^ {*}\right) (\nabla h (y)). (144) \\ \end{array}
$$
By bijectivity of  $\nabla h^{*}$  between int  $\mathrm{dom}(h^{*})$  and int  $\mathrm{dom}(h)$ , we have
$$
\psi_ {\gamma} \circ \nabla h ^ {*} \text {s t r i c t l y c o n v e x o n i n t d o m} \left(h ^ {*}\right) \tag {145}
$$
$$
\Leftrightarrow \forall z \in \operatorname {i n t} \operatorname {d o m} \left(h ^ {*}\right), \forall u \in \mathbb {R} ^ {n}, \langle \nabla^ {2} \left(\psi_ {\gamma} \circ \nabla h ^ {*}\right) (z) u, u \rangle > 0 \tag {146}
$$
$$
\Leftrightarrow \forall y \in \operatorname {i n t} \operatorname {d o m} (h), \forall u \in \mathbb {R} ^ {n}, \langle \nabla^ {2} (\psi_ {\gamma} \circ \nabla h ^ {*}) (\nabla h (y)) u, u \rangle > 0. \tag {147}
$$
Now, using the fact that  $\nabla^2 h(y)$  is positive definite, it follows
$$
\Leftrightarrow \forall y \in \operatorname {i n t} \operatorname {d o m} (h), \forall u \in \mathbb {R} ^ {n}, \langle \nabla^ {2} h (y). \nabla^ {2} \left(\psi_ {\gamma} \circ \nabla h ^ {*}\right) (\nabla h (y)) u, u \rangle > 0 \tag {148}
$$
$$
\Leftrightarrow \forall y \in \operatorname {i n t} \operatorname {d o m} (h), \forall u \in \mathbb {R} ^ {n}, \langle J _ {\mathcal {B} _ {\gamma}} (y) u, u \rangle > 0 \tag {149}
$$
$$
\Leftrightarrow J _ {\mathcal {B} _ {\gamma}} \text {p o s i t i v e d e f i n e o n i n t} \operatorname {d o m} (h). \tag {150}
$$
By the inverse function theorem [Krantz and Parks, 2002], as  $\forall y\in \operatorname {int}\operatorname {dom}(h),J_{\mathcal{B}_{\gamma}}(y) > 0,$ $\mathcal{B}_{\gamma}^{-1}$  is then real analytic on  $Im(\mathcal{B}_{\gamma})$  . We finally obtain, using the expression (12) of  $\phi_{\gamma}$  on  $Im(\mathcal{B}_{\gamma})$  , again by sum and composition, that  $\phi_{\gamma}$  is real analytic (and thus subanalytic) on its domain.
Eventually,  $g_{\gamma}$  is non-negative and we now prove that we have  $\forall y \in \mathbb{R}^n$ ,  $\phi_{\gamma}(y) \geq g_{\gamma}(y)$ . If  $y \notin \operatorname{int} \operatorname{dom}(h)$ , as  $\operatorname{Im}(\mathcal{B}_{\gamma}) \subset \operatorname{int} \operatorname{dom}(h)$ ,  $\phi_{\gamma}(y) = +\infty$ , this is verified. If  $y \in \operatorname{dom}(h)$ ,
as  $D_{h}(y,y) = 0$ , we have
$$
\begin{array}{l} \phi_ {\gamma} (y) = \phi_ {\gamma} (y) + D _ {h} (y, y) \\ \geq \phi_ {\gamma} (\mathcal {B} _ {\gamma} (y)) + D _ {h} (\mathcal {B} _ {\gamma} (y), y) \tag {151} \\ = g _ {\gamma} (y), \\ \end{array}
$$
where the inequality comes from (13) and the last equality from (12). Therefore  $\phi_{\gamma}$  is also lower-bounded.
Second, we verify Assumption 2 required for the convergence of B-RED.
(i)  $[0,R]^n$  is a non-empty closed, bounded, convex and semi-algebraic subset of  $\mathbb{R}_{++}^n$ .
(ii) With the parametrization  $g_{\gamma}(y) = \frac{1}{2} ||x - N_{\gamma}(x)||^{2}$  with a neural network  $N_{\gamma}$ .  $g_{\gamma}$  can be shown to have Lipschitz gradient (see Hurault et al. [2021, Appendix B] for a proof). We can not show a global NoLip property for  $g_{\gamma}$ . However, as  $\nabla g_{\gamma}$  is  $Lip(g_{\gamma})$ -Lipschitz, we have  $\forall x \in (0,R]^n, \forall d \in \mathbb{R}^n$ ,
$$
\langle \nabla^ {2} g _ {\gamma} (x) d, d \rangle \leq L i p (g _ {\gamma}) \| d \| ^ {2} \leq L i p (g _ {\gamma}) R ^ {2} \sum_ {i = 1} ^ {n} \frac {d _ {i} ^ {2}}{x _ {i} ^ {2}} = L i p (g _ {\gamma}) R ^ {2} \langle \nabla^ {2} h (x) d, d \rangle , \tag {152}
$$
which proves that, for  $L_{\gamma} = Lip(g_{\gamma})R^{2}$ ,  $L_{\gamma}h - g_{\gamma}$  is convex on  $(0,R]^n$ .
B-PnP additional assumptions We finally discuss the additional assumptions required for the convergence of B-PnP in Theorem 2.
-  $\operatorname{Im}(\mathcal{B}_{\gamma}) \subseteq \operatorname{dom}(h) = \mathbb{R}_{++}^{n}$ . We train the denoiser  $\mathcal{B}_{\gamma}$  to restore images in  $[\epsilon, 1]^{n}$  (with  $\epsilon = 10^{-3}$ ), the denoiser is thus softly enforced to have its image in this range. In practice, we empirically verify during the iterations that we always get  $x_{k} > 0$ .
-  $\psi_{\gamma} \circ \nabla h^{*}$  convex on int  $\mathrm{dom}(h^{*})$ . This is discussed in detail in Appendix E.2.
-  $\operatorname{Im}(\nabla h - \lambda \nabla f) \subseteq \operatorname{dom}(\nabla h^*)$ . We now show that this condition is true if  $\lambda \| y \|_1 < 1$ . For  $x > 0$
$$
\nabla h (x) - \lambda \nabla f (x) = - \frac {1}{x} - \lambda \nabla f (x) = - \frac {x + x \lambda \nabla f (x)}{x}. \tag {153}
$$
Thus we need to verify that  $\forall 1\leq i\leq n,1 + \lambda x_{i}\nabla f(x)_{i} > 0$  . For  $f$  Poisson data-fidelity term, using  $(Ax)_j = \sum_{k = 1}^n a_{j,k}x_k$  , we have  $\forall 1\leq i\leq n$
$$
\nabla f (x) _ {i} = \sum_ {j = 1} ^ {m} - y _ {j} \frac {a _ {j , i}}{\sum_ {k = 1} ^ {n} a _ {j , k} x _ {k}} + \alpha a _ {j, i} \tag {154}
$$
$$
1 + \lambda x _ {i} \nabla f (x) _ {i} = 1 + \alpha \lambda \sum_ {j = 1} ^ {m} a _ {j, i} x _ {i} - \lambda \sum_ {j = 1} ^ {m} y _ {j} \frac {a _ {j , i} x _ {i}}{\sum_ {k = 1} ^ {n} , a _ {j , k} x _ {k}}. \tag {155}
$$
We assumed that  $A$  has positive entries and  $\sum_{j=1}^{m} a_{j,i} = r_i > 0$ . Therefore, using  $0 \leq \frac{a_{j,i} x_i}{\sum_{k=1}^{n} a_{j,k} x_k} < 1$ , we get
$$
1 + \lambda x _ {i} \nabla f (x) _ {i} \geq 1 - \lambda | | y | | _ {1} \tag {156}
$$
which is positive if  $\lambda \| y \|_1 < 1$ .
- The stepsize condition  $\lambda L_f < 1$ . Using the NoLip constant proposed in [Bauschke et al., 2017, Lemma 7],  $L_f = ||y||_1$ , the condition boils down to  $\lambda ||y||_1 < 1$ . The condition  $\lambda ||y||_1 < 1$  is too restrictive in practice, as  $||y||_1$  could get very big, especially for large images. This is due to the fact that the NoLip constant  $L_f \geq ||y||_1$  can be largely overestimated. Indeed, in the proof of [Bauschke et al., 2017, Lemma 7] as well as in the proof of the previous point, the upper bound
$$
\frac {a _ {j , i} x _ {i}}{\sum_ {k = 1} ^ {n} a _ {j , k} x _ {k}} <   1 \tag {157}
$$
can be very loose in practice. For B-RED this is not a problem, as we use automatic stepsize backtracking. However, for B-PnP, backtracking is not possible as the stepsize is fixed to  $\tau = 1$ . In order to still guarantee the convergence, we adopt the following backtracking-like strategy to adjust the regularization parameter  $\lambda$ :
- Choose an initial value for  $\lambda > 0$ .
- At each iteration of the B-PnP algorithm, check sufficient decrease of the objective function  $F = \lambda f + \phi_{\gamma}$  i.e.
$$
F \left(x _ {k}\right) - F \left(x _ {k + 1}\right) \leq \delta D _ {h} \left(x _ {k + 1}, x _ {k}\right). \tag {158}
$$
If at some iteration, this condition is not satisfied before convergence, we alert the user and restart the algorithm with  $\lambda \gets \eta \lambda$  for some  $\eta \in (0,1)$ . We also let the user know that, for optimal performance, it might be necessary to adjust the regularization parameter  $\gamma$  of the denoiser, in order to compensate for this decrease of  $\lambda$ .
In our deblurring experiments, for the proposed value of  $\lambda$ , over the variety of blur kernels and noise levels experimented, the sufficient decrease property was always verified and this backtracking algorithm was never activated. This illustrates that  $||y||_1$  is a rough approximation of the NoLip constant.
E.5 Experiments
We give here more details and results on the evaluation of B-PnP and B-RED algorithms for Poisson image deblurring. We present in Figure 5 the four blur kernels used for evaluation. Initialization is done with  $x_0 = A^T y$ . The algorithm terminates when the relative difference between consecutive values of the objective function is less than  $10^{-8}$  or the number of iterations exceeds  $K = 500$ .
(a)
Figure 5: The 4 blur kernels used for deblurring evaluation. (a) and (b) are real-world camera shake kernels from Levin et al. [2009]. (c) is a  $9 \times 9$  uniform kernel. (d) is a  $25 \times 25$  Gaussian kernel with standard deviation 1.6.
Choice of hyperparameters For B-RED, stepsize backtracking is performed with  $\gamma = 0.8$  and  $\eta = 0.5$ .
When performing plug-and-play image deblurring with our Bregman Score Denoiser trained with Inverse Gamma noise, for the right choice of hyperparameters  $\lambda$  and  $\gamma$ , we may observe the following behavior. The algorithm first converges towards a meaningful solution. After hundreds of iterations, it can converge towards a different stationary point that does not correspond to a visually good reconstruction. We illustrate this behavior in Figure 6 where we plot the evolution of the PSNR and of the function values  $f(x_{k})$  and  $g_{\gamma}(x_{k})$  along the algorithm.
(a) PSNR  $(x_{k})$
Figure 6: Evolution of the PSNR,  $f(x_{k})$  and  $g_{\gamma}(x_{k})$  when deblurring with B-RED with the initialization parameters from Table 2 and without hyper-parameter update after 100 iterations. We observe a first phase of fast decrease of both the data-fidelity term and regularization term values, resulting in a fast PSNR increase. After approximately 100 iterations, the regularization continues decreasing and the iterates converge towards a different stationary point with low PSNR.
(b)  $f(x_{k})$
(c)  $g_{\gamma}(x_k)$
This phenomenon can be mitigated by using small stepsize, large  $\gamma$  and small  $\lambda$  values at the expense of slowing down significantly the algorithm. To circumvent this issue, we propose to first initialize the algorithm with 100 steps with initial  $\tau$ ,  $\gamma$  and  $\lambda$  values and then to change this parameters for
the actual algorithm. Note that it is possible for B-RED to change the stepsize  $\tau$  but not for B-PnP which has fixed stepsize  $\tau 1 =$ . For B-PnP, as done in Hurault et al. [2022] in the Euclidean setting, we propose to multiply  $g_{\theta}$  by a parameter  $0 < \alpha < 1$  such that the Bregman Score denoiser becomes  $\mathcal{B}_{\gamma}^{\alpha}(y) = y - \alpha (\nabla^{2}h(y))^{-1}.\nabla g_{\gamma}(y)$ . The convergence of B-PnP with this denoiser follows identically. The overall hyperparameters  $\lambda, \gamma, \tau$  and  $\alpha$  for B-PnP and B-RED algorithms for initialization and for the actual algorithm are given in Table 2.
α	20	40	60
B-RED	Initialization τ = 1 γ = 50	λ = 1.5	λ = 2.	λ = 2.5
Algorithm τ = 0.05 γ = 500	λ = 0.5	λ = 0.5	λ = 0.5
B-PnP	Initialization α = 1 γ = 50	λ = 1.5	λ = 2.	λ = 2.5
Algorithm α = 0.05 γ = 500	λ = 0.025	λ = 0.025	λ = 0.025
Additional experimental results We provide in Table 3 a quantitative comparison between our 2 algorithms B-RED and B-PnP and 3 other methods. (a) PnP-PGD corresponds to the plug-and-play proximal gradient descent algorithm  $x_{k + 1} = D_{\sigma} \circ (\mathrm{Id} - \tau \nabla f)$  with  $D_{\sigma}$  the DRUNet denoiser (same architecture than B-RED and B-PnP) trained to denoiser Gaussian noise. (b) PnP-BPG corresponds to the B-PnP algorithm  $x^{k + 1} = D_{\sigma} \circ \nabla h^{*}(\nabla h - \tau \nabla f)(x_{k})$  with again the DRUNet denoiser  $D_{\sigma}$  trained for Gaussian noise. For both (a) and (b) the parameters  $\sigma$  and  $\tau$  are optimized for each noise level  $\alpha$ . (c) ALM Unfolded [Sanghvi et al., 2022] uses the Augmented Lagrangian Method for deriving a 3-operator splitting algorithm that is then trained specifically in an unfolded fashion for image deblurring with a variety of blurs and noise levels  $\alpha$ . The publicly available model being trained on grayscale images, for restoring our color images, we treat each color channel independently.
Note that contrary to the proposed B-PnP and B-RED algorithms, the 3 compared methods do not have any convergence guarantees. We observe that our algorithms perform the best when the Poisson noise is not too intense ( $\alpha = 40$  and  $\alpha = 60$ ) but that PSNR performance decreases for intense noise ( $\alpha = 20$ ). We assume that this is due to the fact that the denoising prior trained on Inverse Gamma noise is not powerful enough for such a strong noise. A visual example for  $\alpha = 20$  is given in Figure 7. As a future direction, we plan on investigating how to increase the regularization capacity of the deep inverse gamma noise denoiser to better handle intense noise.
Table 2: B-RED and B-PnP hyperparameters
α	20	40	60
PnP-PGD	23.81	24.41	24.45
PnP-BPG	23.85	24.26	24.71
ALM Unfolded [Sanghvi et al., 2022]	23.39	23.91	24.22
B-RED	23.58	24.54	24.90
B-PnP	23.29	24.54	24.80
Table 3: PSNR (dB) of Poisson deblurring methods on the CBSD68 dataset. PSNR averaged over the 4 blur kernels represented Figure 5 for each noise levels  $\alpha$ .
(b) Observed (14.91dB)
Figure 7: Deblurring from the indicated motion kernel and Poisson noise with  $\alpha = 20$ .
(c) B-RED (22.44dB)
(d) B-PnP (22.40dB)