Original: Automatic differentiation of nonsmooth iterative algorithms
Revised: Automatic differentiation of nonsmooth iterative algorithms

Original: Abstract
Revised: Abstract

Original: Differentiation along algorithms, i.e., piggyback propagation of derivatives, is now routinely used to differentiate iterative solvers in differentiable programming.
Revised: Differentiation along algorithms, i.e., piggyback propagation of derivatives, is now routinely used to differentiate iterative solvers in differentiable programming.

Original: Asymptotics is well understood for many smooth problems but the nondifferentiable case is hardly considered.
Revised: Asymptotics is well understood for many smooth problems but the nondifferentiable case is hardly considered.

Original: Is there a limiting object for nonsmooth piggyback automatic differentiation (AD)?
Revised: Is there a limiting object for nonsmooth piggyback automatic differentiation (AD)?

Original: Does it have any variational meaning and can it be used effectively in machine learning?
Revised: Does it have any variational meaning and can it be used effectively in machine learning?

Original: Is there a connection with classical derivative?
Revised: Is there a connection with classical derivative?

Original: All these questions are addressed under appropriate nonexpansivity conditions in the framework of conservative derivatives which has proved useful in understanding nonsmooth AD.
Revised: All these questions are addressed under appropriate nonexpansivity conditions in the framework of conservative derivatives which has proved useful in understanding nonsmooth AD.

Original: We characterize the attractor set of nonsmooth piggyback iterations as a set-valued fixed point which remains in the conservative framework.
Revised: We characterize the attractor set of nonsmooth piggyback iterations as a set-valued ﬁxed point which remains in the conservative framework.

Original: Among various consequences we have almost everywhere convergence of classical derivatives.
Revised: Among various consequences we have almost everywhere convergence of classical derivatives.

Original: Our results are illustrated on parametric convex optimization with forward-backward, Douglas-Rachford and Alternating Direction of Multiplier algorithms as well as the Heavy-Ball method.
Revised: Our results are illustrated on parametric convex optimization with forward-backward, Douglas-Rachford and Alternating Direction of Multiplier algorithms as well as the Heavy-Ball method.

Original:  Introduction
Revised:  Introduction

Original: [Equation]
Revised: [Figure]

Original: nonsmooth autodiff k Ñ `derivative? li m it ? automatic  of 
Revised: tence and meaning of ¯ as a derivative of ¯ x , compatible with automatic differentiation of the iterates p x k p ✓ qq k P N

Original: Differentiable programming.
Revised: Differentiable programming.

Original: We consider a Lipschitz function
Revised: We consider a Lipschitz function

Original: F : R p
Revised: F : R p

Original: ˆ R m ÞÑ R p , representing an iterative algorithm, parameterized by θ P R m , with Lipschitz initialization x 0 : θ ÞÑ x 0 p θ q andx k ` 1 p θ q “ F p x k p θ q ,
Revised: ˆ R m ﬁ Ñ R p , representing an iterative algorithm, parameterized by ✓ P R m , with Lipschitz initialization x 0 : ✓ ﬁ Ñ x 0 p ✓ q and

Original: [Equation]
Revised: [Equation]

Original: where F θ : “ F p¨ , θ q , under the assumption that x k p θ q converges to the unique fixed point of F θ : ¯ x p θ q “ fix p F θ q .
Revised: where F ✓ : “ F p¨ , ✓ q , under the assumption that x k p ✓ q converges to the unique ﬁxed point of F ✓ : ¯ x p ✓ q “ ﬁx p F ✓ q .

Original: Such recursion represent for instance algorithms to solve an optimization problem min x h p x q (e.g. empirical risk minimization), such as gradient descent: F p x, θ q “ x ´ θ ∇ h p x q .
Revised: Such recursion represent for instance algorithms to solve an optimization problem min x h p x q (e.g. empirical risk minimization), such as gradient descent: F p x, ✓ q “ x ´ ✓ r h p x q .

Original: But (1) could also be a fixed-point equation such as a deep equilibrium network [5].
Revised: But (1) could also be a ﬁxed-point equation such as a deep equilibrium network [5].

Original: In the last years, a paradigm shift occurred: such algorithms are now implemented in algorithmic differentiation (AD)-friendly frameworks such as Tensorflow [1],
Revised: In the last years, a paradigm shift occurred: such algorithms are now implemented in algorithmic differentiation (AD)-friendly frameworks such as Tensorﬂow [1],

Original: PyTorch [42] or JAX [13].
Revised: PyTorch [42] or JAX [13].

Original: For a differentiable F , it is possible to compute iteratively the derivatives of 
Revised: For a differentiable F , it is possible to compute iteratively the derivatives of 36th Conference on Neural Information Processing Systems (NeurIPS 2022).

Original: [Figure]
Revised: [Figure]

Original: Trend 
Revised: Trend Filtering

Original: [Figure]
Revised: [Figure]

Revised: ADMM)

Original:  x k with respect to θ using the differential calculus rules resulting in so called “piggyback” recursion:
Revised: [Figure] ¯ [Figure] x k with respect to ✓ using the differential calculus rules resulting in so called “piggyback” recursion:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (2) where BB θ x k is the Jacobian of x k with respect to θ .
Revised: ) where BB ✓ x k is the Jacobian of x k with respect to ✓ .

Original: In practice, automatic differentiation frameworks do not compute the full Jacobian, but compute either vector-Jacobian products in reverse-mode (or backpropagation) [48] or Jacobian-vector products in forward mode [53].
Revised: In practice, automatic differentiation frameworks do not compute the full Jacobian, but compute either vector-Jacobian products in reverse-mode (or backpropagation) [48] or Jacobian-vector products in forward mode [53].

Original: We rather consider the full Jacobian, and therefore, our findings apply to both modes.
Revised: We rather consider the full Jacobian, and therefore, our ﬁndings apply to both modes.

Original: We focus on two issues arising with nonsmooth recursions, illustrated in Figure 1.
Revised: We focus on two issues arising with nonsmooth recursions, illustrated in Figure 1.

Original: (i) what can be said about the chain rule (2) and its asymptotics when the function F is not smooth (for example a projected gradient step)?
Revised: (i) what can be said about the chain rule (2) and its asymptotics when the function F is not smooth (for example a projected gradient step)?

Original: (ii) how to interpret its asymptotics as a notion of derivative for ¯ x , the fixed point of F θ ?
Revised: (ii) how to interpret its asymptotics as a notion of derivative for ¯ x , the ﬁxed point of F ✓ ?

Original: We propose a joint answer to both questions, providing a solid theoretical ground to the idea of algorithmic differentiation of numerical solvers involving nonsmooth components in a differentiable programming context.
Revised: We propose a joint answer to both questions, providing a solid theoretical ground to the idea of algorithmic differentiation of numerical solvers involving nonsmooth components in a differentiable programming context.

Original: Related works.
Revised: Related works.

Original: Algorithmic use of the chain rule (2) to differentiate programs takes its root in [53], with forward differentiation, and later in reverse mode [35].
Revised: Algorithmic use of the chain rule (2) to differentiate programs takes its root in [53], with forward differentiation, and later in reverse mode [35].

Original: Along with the development of AD, convergence of the iterative sequence (2) was investigated, notably in the optimization community as reviewed in [28].
Revised: Along with the development of AD, convergence of the iterative sequence (2) was investigated, notably in the optimization community as reviewed in [28].

Original: This important survey paper gathers results in differentiable programming rediscovered/reused later: implicit differentiation [43, 45] and its stability [8], adjoint fixed point iteration [5] (a key aspect of the deep equilibrium network) and linear convergence of (2).
Revised: This important survey paper gathers results in differentiable programming rediscovered/reused later: implicit differentiation [43, 45] and its stability [8], adjoint ﬁxed point iteration [5] (a key aspect of the deep equilibrium network) and linear convergence of (2).

Original: Notably, linear convergence of Jacobians was investigated in [25, 27] for the forward mode and in [15, Theorem 2.3] for the reverse mode.
Revised: Notably, linear convergence of Jacobians was investigated in [25, 27] for the forward mode and in [15, Theorem 2.3] for the reverse mode.

Original: This was more recently investigated – for C 2 functions – in imaging for primal-dual algorithms [14, 9] and in machine learning for gradient descent [39, 36] and the
Revised: This was more recently investigated – for C 2 functions – in imaging for primal-dual algorithms [14, 9] and in machine learning for gradient descent [39, 36] and the

Original: Heavy-ball [39] method.
Revised: Heavy-ball [39] method.

Original: In the specific context where F solves a min - min problem, the authors in [2] proved the linear convergence of the Jacobians.
Revised: In the speciﬁc context where F solves a min - min problem, the authors in [2] proved the linear convergence of the Jacobians.

Original: The use of AD for nonsmooth functions was justified with the notion of conservative Jacobians [12, 11] with a nonsmooth version of the chain rule for compositional models.
Revised: The use of AD for nonsmooth functions was justiﬁed with the notion of conservative Jacobians [12, 11] with a nonsmooth version of the chain rule for compositional models.

Original: Correctness of AD was also investigated in [34] for a large class of piecewise analytic functions, and in [33] where a qualification condition is used to compute a Clarke Jacobian.
Revised: Correctness of AD was also investigated in [34] for a large class of piecewise analytic functions, and in [33] where a qualiﬁcation condition is used to compute a Clarke Jacobian.

Original: Along with AD, a natural way to differentiate a model (1) is by implicit differentiation, recently applied in several works [5, 3, 21].
Revised: Along with AD, a natural way to differentiate a model (1) is by implicit differentiation, recently applied in several works [5, 3, 21].

Original: In a nonsmooth context, an implicit function theorem [10] was proved for path-differentiable functions.
Revised: In a nonsmooth context, an implicit function theorem [10] was proved for path-differentiable functions.

Original: In terms of applications, nonsmooth piggyback derivatives are applied to hyperparameter tuning for inverse problems in [8] while the case of Lasso was investigated in [7].
Revised: In terms of applications, nonsmooth piggyback derivatives are applied to hyperparameter tuning for inverse problems in [8] while the case of Lasso was investigated in [7].

Original: Other relevant applications include plug-and-play denoising [32], parameter selection [19], bilevel programming [41]
Revised: Other relevant applications include plug-and-play denoising [32], parameter selection [19], bilevel programming [41]

Original: Contributions:
Revised: Contributions

Original: Under suitable nonexpansivity assumptions, our contributions are as follows
Revised: Under suitable nonexpansivity assumptions, our contributions are as follows

Original: • We address both questions illustrated in Figure 1 for nonsmooth recursions.
Revised: • We address both questions illustrated in Figure 1 for nonsmooth recursions.

Original: Set-valued extensions of the piggyback recursion (2) have a well defined limit: the fixed point of subset map (Theorem 1), it is conservative for the fixed point map ¯ x .
Revised: Set-valued extensions of the piggyback recursion (2) have a well deﬁned limit: the ﬁxed point of subset map (Theorem 1), it is conservative for the ﬁxed point map ¯ x .

Original: This is a nonsmooth “infinite” chain rule for AD (Theorem 2).
Revised: This is a nonsmooth “inﬁnite” chain rule for AD (Theorem 2).

Original: • For almost all θ , despite nonsmoothness, recursion (2) is well defined using the classical Jacobian and converges to the classical Jacobian of the fixed point ¯ x (Corollary 2).
Revised: • For almost all ✓ , despite nonsmoothness, recursion (2) is well deﬁned using the classical Jacobian and converges to the classical Jacobian of the ﬁxed point ¯ x (Corollary 2).

Original: This has implications for both forward and reverse modes of AD.
Revised: This has implications for both forward and reverse modes of AD.

Original: • For a large class of functions (Lipschitz-gradient selection), it is possible to give a quantitative rate estimate (Corollary 3), namely to prove linear convergence of the derivatives.
Revised: • For a large class of functions (Lipschitz-gradient selection), it is possible to give a quantitative rate estimate (Corollary 3), namely to prove linear convergence of the derivatives.

Original: • We show that these results can be applied to proximal splitting algorithms in nonsmooth convex optimization.
Revised: • We show that these results can be applied to proximal splitting algorithms in nonsmooth convex optimization.

Original: We include forward–backward (Proposition 2), as well Douglas–Rachford (Proposition 3) and ADMM, a numerical illustration of the convergence of derivatives is given in Figure 2.
Revised: We include forward–backward (Proposition 2), as well Douglas–Rachford (Proposition 3) and ADMM, a numerical illustration of the convergence of derivatives is given in Figure 2.

Original: • We also illustrate that, contrarily to the smooth case, nonsmooth piggy back derivatives of momentum methods such as Heavy-ball, may diverge even if the iterates converge linearly (Proposition 4).
Revised: • We also illustrate that, contrarily to the smooth case, nonsmooth piggy back derivatives of momentum methods such as Heavy-ball, may diverge even if the iterates converge linearly (Proposition 4).

Original: Notations.
Revised: Notations.

Original: A function f : R p Ñ R m is locally Lipschtiz if, for each x P R n , there exists a neighborhood of x on which f is Lipschitz.
Revised: A function f : R p Ñ R m is locally Lipschtiz if, for each x P R n , there exists a neighborhood of x on which f is Lipschitz.

Original: Denoting by R Ď R p , the full measure set where f is differentiable, the Clarke Jacobian [16] at x P R p is defined as
Revised: Denoting by R Ñ R p , the full measure set where f is differentiable, the Clarke Jacobian [16] at x P R p is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: The Clarke subdifferential, B c f is defined similarly.
Revised: The Clarke subdifferential, B c f is deﬁned similarly.

Original: Given two matrices A, B with compatible dimension, r A, B s is their concatenation.
Revised: Given two matrices A, B with compatible dimension, r A, B s is their concatenation.

Original: For a set X , conv X is its convex hull.
Revised: For a set X , conv X is its convex hull.

Original: The symbol B denotes a unit ball, the corresponding norm should be clear from the context.
Revised: The symbol B denotes a unit ball, the corresponding norm should be clear from the context.

Original:  Nonsmooth piggyback differentiation
Revised:  Nonsmooth piggyback differentiation

Original: We first show how the use of the notion of conservative Jacobians allow us to justify rigorously the existence of a nonsmooth equivalent of piggyback iterations in (2) that is compatible with AD.
Revised: We ﬁrst show how the use of the notion of conservative Jacobians allow us to justify rigorously the existence of a nonsmooth equivalent of piggyback iterations in (2) that is compatible with AD.

Original: Conservative Jacobians.
Revised: Conservative Jacobians.

Original: Conservative Jacobians were introduced in [12] as a generalization of derivatives to study automatic differentiation of nonsmooth functions.
Revised: Conservative Jacobians were introduced in [12] as a generalization of derivatives to study automatic differentiation of nonsmooth functions.

Original: Given a locally Lipschitz continuous function f : R p Ñ R m , the set-valued J : R p Ñ R m ˆ p is a conservative Jacobian for the path differentiable f if J has a closed graph, is locally bounded and nowhere empty with
Revised: Given a locally Lipschitz continuous function f : R p Ñ R m , the set-valued J : R p Ñ R m ˆ p is a conservative Jacobian for the path differentiable f if J has a closed graph, is locally bounded and nowhere empty with

Original: [Equation]
Revised: [Equation]

Original: for any γ : r 0 , 1 s Ñ R p absolutely continuous with respect to the Lebesgue measure.
Revised: for any  : r 0 , 1 s Ñ R p absolutely continuous with respect to the Lebesgue measure.

Original: Conservative gradients are defined similarly.
Revised: Conservative gradients are deﬁned similarly.

Original: We refer to [12] for extensive examples and properties of this class of function, key ideas are recalled in Appendix A for completeness.
Revised: We refer to [12] for extensive examples and properties of this class of function, key ideas are recalled in Appendix A for completeness.

Original: Let us mention that the classes of convex functions, definable functions, or semialgebraic functions are contained in the set of path differentiable functions.
Revised: Let us mention that the classes of convex functions, deﬁnable functions, or semialgebraic functions are contained in the set of path differentiable functions.

Original: Given D f : R p Ñ R p , a conservative gradient for f : R p Ñ R , we have:
Revised: Given D f : R p Ñ R p , a conservative gradient for f : R p Ñ R , we have:

Original: • ( Clarke subgradient ), for all x P R p , B c f p x q Ă conv p D f p x qq .
Revised: • ( Clarke subgradient ), for all x P R p , B c f p x q Ä conv p D f p x qq .

Original: • ( Gradient almost everywhere )
Revised: • ( Gradient almost everywhere )

Original: D f p x q “ t ∇ f p x qu for almost all x P R p .
Revised: D f p x q “ t r f p x qu for almost all x P R p .

Original: • ( Calculus ) differential calculus rules preserve conservativity, e g. sum and compositions of conservative Jacobians are conservative Jacobians.
Revised: • ( Calculus ) differential calculus rules preserve conservativity, e.g. sum and compositions of conservative Jacobians are conservative Jacobians.

Original: Finally, D f can be used as a first order optimization oracle for methods of gradient type [11].
Revised: Finally, D f can be used as a ﬁrst order optimization oracle for methods of gradient type [11].

Original: Piggyback differentiation of recursive algorithms.
Revised: Piggyback differentiation of recursive algorithms.

Original: The following is standing throughout the text.
Revised: The following is standing throughout the text.

Original: Assumption 1 (The conservative Jacobian of the iteration mapping is a contraction) F is locally Lipschitz, path differentiable, jointly in p x, θ q , and J F is a conservative Jacobian for F .
Revised: Assumption 1 (The conservative Jacobian of the iteration mapping is a contraction) F is locally Lipschitz, path differentiable, jointly in p x, ✓ q , and J F is a conservative Jacobian for F .

Original: There exists 0 ď ρ ă 1 , such that for any p x, θ q P R p ˆ R m and any pair r A, B s P J F p x, θ q , with 
Revised: There exists 0 § ⇢ † 1 , such that for any p x, ✓ q P R p ˆ R m and any pair r A, B s P J F p x, ✓ q , with A

Original: A P R p ˆ p and B P R p ˆ m , the operator norm of A is at most ρ .
Revised: P R p ˆ p and B P R p ˆ m , the operator norm of A is at most ⇢ .

Original: J x 0 is a conservative Jacobian for the initialization function θ ÞÑ x 0 p θ q .
Revised: J x 0 is a conservative Jacobian for the initialization function ✓ ﬁ Ñ x 0 p ✓ q .

Original: Under Assumption 1, F θ is a strict contraction so that p x k p θ qq k P N converges linearly to ¯ x p θ q “ fix p F θ q the unique fixed point of the iteration mapping F θ .
Revised: Under Assumption 1, F ✓ is a strict contraction so that p x k p ✓ qq k P N converges linearly to ¯ x p ✓ q “ ﬁx p F ✓ q the unique ﬁxed point of the iteration mapping F ✓ .

Original: More precisely, for all k P N , we have
Revised: More precisely, for all k P N , we have

Original: [Equation]
Revised: [Equation]

Original: Furthermore, for every k P N , let us define the following set-valued piggyback recursion:
Revised: Furthermore, for every k P N , let us deﬁne the following set-valued piggyback recursion:

Original: [Equation]
Revised: [Equation]

Original: We will show in Section 3 that (PB) plays the same role as (2) in the nonsmooth setting.
Revised: We will show in Section 3 that (PB) plays the same role as (2) in the nonsmooth setting.

Original: Note that one can recursively evaluates a sequence J k P J x k , k P N , through operations actually implemented in nonsmooth AD frameworks, as follows
Revised: Note that one can recursively evaluates a sequence J k P J x k , k P N , through operations actually implemented in nonsmooth AD frameworks, as follows

Original: [Equation]
Revised: [Equation]

Original: Remark 1 (Local contractions)
Revised: Remark 1 (Local contractions)

Original: Assumption 1 may be relaxed as follows: for all θ , the fixed point set fix p F θ q is a singleton ¯ x θ such that x k p θ q Ñ ¯ x p θ q as k Ñ 8 , and the operator norm condition on 
Revised: Assumption 1 may be relaxed as follows: for all ✓ , the ﬁxed point set ﬁx p F ✓ q is a singleton ¯ x ✓ such that x k p ✓ q Ñ ¯ x p ✓ q as k Ñ 8 , and the operator norm condition on J

Original: J F in Assumption 1 holds at the point p ¯ x p θ q , θ q .
Revised: F in Assumption 1 holds at the point p ¯ x p ✓ q , ✓ q .

Original: By graph closedness of J F , in a neighborhood of p ¯ x p θ q , θ q , F θ is a strict contraction and the operator norm condition on J F holds, possibly with a larger contraction factor ρ .
Revised: By graph closedness of J F , in a neighborhood of p ¯ x p ✓ q , ✓ q , F ✓ is a strict contraction and the operator norm condition on J F holds, possibly with a larger contraction factor ⇢ .

Original: After finitely many steps, the iterates p x k q k P N remain on some neighborhood and all our convergence results hold, due to their asymptotic nature.
Revised: After ﬁnitely many steps, the iterates p x k q k P N remain on some neighborhood and all our convergence results hold, due to their asymptotic nature.

Original: Remark 2 (Relation to existing work) For a smooth F a natural conservative Jacobian is the classical one.
Revised: Remark 2 (Relation to existing work) For a smooth F a natural conservative Jacobian is the classical one.

Original: The, hypotheses in [39, 36] for gradient descent ( F is C 1 ), are exactly the classical counterpart of Assumption 1.
Revised: The, hypotheses in [39, 36] for gradient descent ( F is C 1 ), are exactly the classical counterpart of Assumption 1.

Original: On the other hand [25, 27, 15] use a more general assumption on spectral radius, which allow to treat the Heavy-Ball method, e.g. in [39].
Revised: On the other hand [25, 27, 15] use a more general assumption on spectral radius, which allow to treat the Heavy-Ball method, e.g. in [39].

Original: However this does not generalize to sets of matrices, as shown in Section 5.
Revised: However this does not generalize to sets of matrices, as shown in Section 5.

Original: Hence Assumption 1 is on operator norm and not on spectral radius, which is sharp, contrary to the smooth case.
Revised: Hence Assumption 1 is on operator norm and not on spectral radius, which is sharp, contrary to the smooth case.

Original:  Asymptotics of nonsmooth piggyback differentiation 3.1 Fixed point of affine iterations
Revised:  Asymptotics of nonsmooth piggyback differentiation 3.1 Fixed point of afﬁne

Revised: iterations

Original: Gap and Haussdorf distance.
Revised: Gap and Haussdorf distance.

Original: Being given two nonempty compact subsets X , Y of R p , set gap p X , Y q “ max x P X d p x, Y q where d p x, Y q “ min y P Y } x ´ y } , and define the Hausdorff distance between X and Y by dist p X , Y q “ max p gap p X , Y q , gap p Y , X qq .
Revised: Being given two nonempty compact subsets X , Y of R p , set gap p X , Y q “ max x P X d p x, Y q where d p x, Y q “ min y P Y } x ´ y } , and deﬁne the Hausdorff distance between X and Y by dist p X , Y q “ max p gap p X , Y q , gap p Y , X qq .

Original: Note that gap p X , Y q “ 0 if, and only if, X Ď Y , whereas dist p X , Y q “ 0 if, and only if, X “ Y .
Revised: Note that gap p X , Y q “ 0 if, and only if, X Ñ Y , whereas dist p X , Y q “ 0 if, and only if, X “ Y .

Original: Moreover,
Revised: Moreover,

Original: X Ď Y ` gap p X , Y q B where B is the unit ball.
Revised: X Ñ Y ` gap p X , Y q B where B is the unit ball.

Original: It means that gap p X , Y q “measures” the default of inclusion of
Revised: It means that gap p X , Y q “measures” the default of inclusion of

Original: X in Y , see [46, Chapter 4] for more details.
Revised: X in Y , see [46, Chapter 4] for more details.

Original: Affine iterations by packets of matrices.
Revised: Afﬁne iterations by packets of matrices.

Original: Let J Ă R p ˆp p ` m q be a compact subset of matrices such that any matrix of the form r A, B s P J with A P R p ˆ p is such that A has operator norm at most ρ ă 1 .
Revised: Let J Ä R p ˆp p ` m q be a compact subset of matrices such that any matrix of the form r A, B s P J with A P R p ˆ p is such that A has operator norm at most ⇢ † 1 .

Original: We let J act naturally on the matrices of size p ˆ m as follows J : R p ˆ
Revised: We let J act naturally on the matrices of size p ˆ m as follows J : R p ˆ

Original: m Ñ R p ˆ m the function from R p ˆ m to subsets of R p ˆ m which is defined for each X P R p ˆ m as follows: 
Revised: m Ñ R p ˆ m the function from R p ˆ m to subsets of R p ˆ m which is deﬁned for each X P R p ˆ m as follows: [Equation] P

Original: [Equation]
Revised: [Equation]

Original: This defines a set-valued map through, for any X Ă R p ˆ m ,
Revised: This deﬁnes a set-valued map through, for any X Ä R p ˆ m ,

Original: [Equation]
Revised: [Equation]

Original: Recursions of the form (PB) generate sequences p X k q k P N of subsets of R p ˆ m satisfying
Revised: Recursions of the form (PB) generate sequences p X k q k P N of subsets of R p ˆ m satisfying

Original: [Equation]
Revised: [Equation]

Original: The following is an instance of the Banach–Picard theorem (whose proof is recalled in Appendix B).
Revised: The following is an instance of the Banach–Picard theorem (whose proof is recalled in Appendix B).

Original: Theorem 1 (Set-valued affine contractions)
Revised: Theorem 1 (Set-valued afﬁne contractions)

Original: Let J Ă R p ˆp p ` m q be a compact subset of matrices as above with ρ ă 1 .
Revised: Let J Ä R p ˆp p ` m q be a compact subset of matrices as above with ⇢ † 1 .

Original: Then there is a unique nonempty compact set fix p J q Ă R p ˆ m satisfying fix p J q “ J p fix p J qq , where the action of J is given in (6) .
Revised: Then there is a unique nonempty compact set ﬁx p J q Ä R p ˆ m satisfying ﬁx p J q “ J p ﬁx p J qq , where the action of J is given in (6) .

Original: Let p X k q k P N be a sequence of compact subsets of R p ˆ m , such that X 0 ‰ H , and satisfying the recursion (7) .
Revised: Let p X k q k P N be a sequence of compact subsets of R p ˆ m , such that X 0 ‰ H , and satisfying the recursion (7) .

Original: We have for all kP N
Revised: We have for all k

Original: [Equation]
Revised: [Equation]

Original: where dist is the Hausdorff distance related to the Euclidean norm on p ˆ m matrices.
Revised: where dist is the Hausdorff distance related to the Euclidean norm on p ˆ m matrices.

Original:  An infinite chain rule and its consequences
Revised: 3.2 An inﬁnite chain rule and its consequences

Original: Define the following set-valued map based on the fix operator from Theorem 1,
Revised: Deﬁne the following set-valued map based on the ﬁx operator from Theorem 1,

Original: [Equation]
Revised: [Equation]

Original: where ¯ x p θ q is the unique fixed point of the algorithmic recursion.
Revised: where ¯ x p ✓ q is the unique ﬁxed point of the algorithmic recursion.

Original: Since ¯ x p θ q “ fix p F θ q , we have equivalently that J pb ¯ x is the fixed-point of the Jacobian at the fixed-point: J
Revised: Since ¯ x p ✓ q “ ﬁx p F ✓ q , we have equivalently that J pb ¯ x is the ﬁxed-point of the Jacobian at the ﬁxed-point: J

Original: pb ¯ x : θ Ñ fix r J F p fix p F θ q , θ qs .
Revised: pb ¯ x : ✓ Ñ ﬁx r J F p ﬁx p F ✓ q , ✓ qs .

Original: We have the following (proved in Appendix C) and a consequence from Theorem 1.
Revised: We have the following (proved in Appendix C) and a consequence from Theo- rem 1.

Original: Theorem 2 (A conservative mapping for the fixed point map) Under Assumption 1, J pb ¯ x is welldefined, and is a conservative Jacobian for the fixed point map ¯ x .
Revised: Theorem 2 (A conservative mapping for the ﬁxed point map) Under Assumption 1, J pb ¯ x is welldeﬁned, and is a conservative Jacobian for the ﬁxed point map ¯ x .

Original: Corollary 1 (Convergence of the piggyback derivatives)
Revised: Corollary 1 (Convergence of the piggyback derivatives)

Original: Under Assumption 1, for all θ , the recursion (PB) satisfies
Revised: Under Assumption 1, for all ✓ , the recursion (PB) satisﬁes

Original: [Equation]
Revised: [Equation]

Original: Unrolling the expression of J x k , using (6) and (7), we can rewrite (8) with a compositional product:
Revised: Unrolling the expression of J x k , using (6) and (7), we can rewrite (8) with a compositional product:

Original: [Equation]
Revised: [Equation]

Original: In plain words, this is a limit-derivative exchange result: Asymptotically, the gap between the automatic differentiation of x k and the derivative of the limit is zero.
Revised: In plain words, this is a limit-derivative exchange result: Asymptotically, the gap between the automatic differentiation of x k and the derivative of the limit is zero.

Original: In particular the recursion (5) produces bounded sequences whose accumulation points are in J pb ¯ x .
Revised: In particular the recursion (5) produces bounded sequences whose accumulation points are in J pb ¯ x .

Original: Since conservative Jacobians equal classical Jacobians almost everywhere [12], we have convergence of classical derivatives.
Revised: Since conservative Jacobians equal classical Jacobians almost everywhere [12], we have convergence of classical derivatives.

Original: Corollary 2 (Convergence a.e. of the classical piggyback derivatives)
Revised: Corollary 2 (Convergence a.e. of the classical piggyback derivatives)

Original: Under Assumption 1, for almost all θ , the classical Jacobian BB θ x k p θ q , is well defined for all k and converges towards the classical Jacobian of ¯ x .
Revised: Under Assumption 1, for almost all ✓ , the classical Jacobian BB ✓ x k p ✓ q , is well deﬁned for all k and converges towards the classical Jacobian of ¯ x .

Original: That is
Revised: That is

Original: [Equation]
Revised: [Equation]

Original: for almost allθ .
Revised: for almost all

Revised: [Equation]

Original: Remark 3 (Connection to implicit differentiation)
Revised: Remark 3 (Connection to implicit differentiation)

Original: The authors in [10] proved a qualification-free version of the implicit function theorem.
Revised: The authors in [10] proved a qualiﬁcation-free version of the implicit function theorem.

Original: Assuming that for every r A, B s P J p ¯ x p θ q , θ q , the matrix
Revised: Assuming that for every r A, B s P J p ¯ x p ✓ q , ✓ q , the matrix

Original: I ´ A is invertible, we have that
Revised: I ´ A is invertible, we have that

Original: [Equation]
Revised: [Equation]

Original: is a conservative Jacobian for ¯ x .
Revised: is a conservative Jacobian for ¯ x .

Original: Under Assumption 1, one has J imp ¯ x p θ q Ă J
Revised: Under Assumption 1, one has J imp ¯ x p ✓ q Ä J

Original: pb ¯ x p θ q for all θ .
Revised: pb ¯ x p ✓ q for all ✓ .

Original: Unfortunately, as soon as F is not differentiable, the inclusion may be strict, see details in Appendix D.
Revised: Unfor- tunately, as soon as F is not differentiable, the inclusion may be strict, see details in Appendix D.

Original: 3.3 Consequence for algorithmic differentiation
Revised: 3.3 Consequence for algorithmic differentiation

Original: Given k P N , 9 θ P R m , ¯ w k P R p , the following algorithms allow us to compute 9 x k “ J k 9 θ using the forward mode of automatic differentation (Jacobian Vector Products, JVP), and ¯ θ Tk “ ¯ w Tk J k using the backward mode of automatic differentiation (Vector Jacobian Products, VJP).
Revised: Given k P N , 9 ✓ P R m , ¯ w k P R p , the following algorithms allow us to compute 9 x k “ J k 9 ✓ using the forward mode of automatic differentation (Jacobian Vector Products, JVP), and ¯ ✓ Tk “ ¯ w Tk J k using the backward mode of automatic differentiation (Vector Jacobian Products, VJP).

Original: In a compositional model 9 θ is the derivative of an inner functions controlling algorithm parameters θ , with another variable real variable λ P R , for example an hyper parameter.
Revised: In a compositional model 9 ✓ is the derivative of an inner functions controlling algorithm parameters ✓ , with another variable real variable  P R , for example an hyper parameter.

Original: The goal is to combine B θ p λ q B λ and
Revised: The goal is to combine B ✓ p  q B  and

Original: B x k p θ q B θ with the chain rule in a forward pass to obtain the total derivative B x k p θ p λ qq B λ .
Revised: B x k p ✓ q B ✓ with the chain rule in a forward pass to obtain the total derivative B x k p ✓ p  qq B  .

Original: On the other hand, in a compositional model, ¯ w k is typically the gradient of an outer loss functions ℓ evaluated at x k p θ q .
Revised: On the other hand, in a compositional model, ¯ w k is typically the gradient of an outer loss functions ` evaluated at x k p ✓ q .

Original: In this case the goal is to combine derivatives of iterates B x k p θ q B θ with ¯ w k “ B ℓ p x k q B x k in a backward pass to obtain B ℓ p x k p θ qq B θ .
Revised: In this case the goal is to combine derivatives of iterates B x k p ✓ q B ✓ with ¯ w k “ B ` p x k q B x k in a backward pass to obtain B ` p x k p ✓ qq B ✓ .

Original: Input: k P N , θ P R m , 9 θ P R m , ¯ w k P R p , initialization function x 0 p θ q , recursion function F p x, θ q , conservative Jacobians J F p x, θ q and J x 0 p θ q .
Revised: Input: k P N , ✓ P R m , 9 ✓ P R m , ¯ w k P R p , initialization function x 0 p ✓ q , recursion function F p x, ✓ q , conservative Jacobians J F p x, ✓ q and J x 0 p ✓ q .

Original: Initialize: x 0 “ x 0 p θ q P R p .
Revised: Initialize: x 0 “ x 0 p ✓ q P R p .

Original: Forward mode (JVP): 9 x 0 “ J 9 θ , J P J x 0 p θ q . for i “ 1 , . . .
Revised: Forward mode (JVP): 9 x 0 “ J 9 ✓ , J P J x 0 p ✓ q . for i “ 1 , . . .

Original: k do x i “ F p x
Revised: k do x i “ F p x

Original: i ´ 1 , θ q 
Revised: i ´ 1 , ✓ q 9 x i “ A

Original: 9 x i “ A i ´ 1 9 x i ´ [-
-]
Revised: i ´ 1 9 x i ´ [-
-]1 ` B i ´

Original: ` B i ´ 1 9 θ r A 
Revised: 1 9 ✓ r A i ´ 1 , B i ´

Original: i ´ 1 , B i ´ 1 s P J F p x
Revised: 1 s P J F p x

Original: i ´ 1 , θ q Return: 9 x k
Revised: i ´ 1 , ✓ q Return: 9 x k

Original: Reverse mode (VJP): ¯ θ k “ 0 .
Revised: Reverse mode (VJP): ¯ ✓ k “ 0 .

Original: for i “ 1 , . . .
Revised: for i “ 1 , . . .

Original: k do x i “ F p x
Revised: k do x i “ F p x

Original: i ´ 1 , θ q for i “ k, . .
Revised: i ´ 1 , ✓ q for i “ k, . .

Original: , 1 do ¯ θ k “ ¯ θ k ` B Ti ´ 1 ¯ w i ¯ w i ´ 

Revised: , 1 do ¯ ✓ k “ ¯ ✓ k ` B Ti ´ 1 ¯ w i ¯ w i ´

Original: “ A Ti ´ 1 ¯ w i r
Revised: “ A Ti ´ 1 ¯ w i r

Original: A
Revised: A

Original: i ´ 1 , B i ´ 1 s P J F p x
Revised: i ´ 1 , B i ´ 1 s P J F p x

Original: i ´ 1 , θ q ¯ θ k “ ¯ θ k ` J T ¯ w 0 , J P J x 0 p θ q Return: ¯ θ k
Revised: i ´ 1 , ✓ q ¯ ✓ k “ ¯ ✓ k ` J T ¯ w 0 , J P J x 0 p ✓ q Return: ¯ ✓ k

Original: Proposition 1 (Convergence of VJP and JVP) Let k P N , 9 θ P R m , ¯ w k P R p , x k P R p , 9 x k P R p , ¯ θ Tk P R m be as in Agorithm 1 under Assumption 1.
Revised: Proposition 1 (Convergence of VJP and JVP) Let k P N , 9 ✓ P R m , ¯ w k P R p , x k P R p , 9 x k P R p , ¯ ✓ Tk P R m be as in Agorithm 1 under Assumption 1.

Original: Then for almost all θ P R m , 9 x k Ñ B ¯ x B θ 9 θ .
Revised: Then for almost all ✓ P R m , 9 x k Ñ B ¯ x B ✓ 9 ✓ .

Original: Assume furthermore that, as k Ñ 8 , ¯ w k Ñ ¯ w (for example, ¯ w k “ ∇ ℓ p x k q for a C 1 loss ℓ ), then for almost all θ P R m , ¯ θ Tk Ñ ¯ w T B ¯ x B θ .
Revised: Assume furthermore that, as k Ñ 8 , ¯ w k Ñ ¯ w (for example, ¯ w k “ r ` p x k q for a C 1 loss ` ), then for almost all ✓ P R m , ¯ ✓ T k Ñ ¯ w T B ¯ x B ✓ .

Original: Remark 

Revised: Remark

Original: In addition to Proposition 1, in both cases, for all θ , all accumulation points of both 9 x k and ¯ θ Tk are elements of J pb ¯ x 9 θ and ¯ w T J
Revised: In addition to Proposition 1, in both cases, for all ✓ , all accumulation points of both 9 x k and ¯ ✓ Tk are elements of J pb ¯ x 9 ✓ and ¯ w T J

Original: pb ¯ x
Revised: pb ¯ x

Original: respectively.
Revised: respectively.

Original: This is a consequence of Corollary 2 combined with algorithmic differentiation arguments which proof is given in Appendix D.
Revised: This is a consequence of Corollary 2 combined with algorithmic differentiation arguments which proof is given in Appendix D.

Original: 3.4 Linear convergence rate for semialgebraic piecewise smooth selection function
Revised: 3.4 Linear convergence rate for semialgebraic piecewise smooth selection function

Original: Semialgebraic functions are ubiquitous in machine learning (piecewise polynomials, ℓ 1 , ℓ 2 norms, determinant matrix rank ... ).
Revised: Semialgebraic functions are ubiquitous in machine learning (piecewise polynomials, ` 1 , ` 2 norms, determinant matrix rank ... ).

Original: We refer the reader to [11] for a thorough discussion of their extensions, and use in machine learning.
Revised: We refer the reader to [11] for a thorough discussion of their extensions, and use in machine learning.

Original: For more technical details, see [17, 18] for introductory material on semialgebraic and o-minimal geometry.
Revised: For more technical details, see [17, 18] for introductory material on semialgebraic and o-minimal geometry.

Original: Lipschitz gradient selection functions.
Revised: Lipschitz gradient selection functions.

Original: Let F : R p ÞÑ R q be semialgebraic and continuous.
Revised: Let F : R p ﬁ Ñ R q be semialgebraic and continuous.

Original: We say that F has a Lipschitz gradient selection p s, F 1 , . . .
Revised: We say that F has a Lipschitz gradient selection p s, F 1 , . . .

Original: , F m q if s : R p ÞÑ p 1 , . . .
Revised: , F m q if s : R p ﬁ Ñ p 1 , . . .

Original: , m q is semialgebraic and there exists L ě 0 such that for i “ 1 . . .
Revised: , m q is semialgebraic and there exists L • 0 such that for i “ 1 . . .

Original: , m , F i : R p ÞÑ R p is semialgebraic with L -Lipschitz Jacobian, and for all x P R p , F p x q “ F s p x q p x q .
Revised: , m , F i : R p ﬁ Ñ R p is semialgebraic with L -Lipschitz Jacobian, and for all x P R p , F p x q “ F s p x q p x q .

Original: For any x P R p , set I p x q “ t i P t 1 , . . .
Revised: For any x P R p , set I p x q “ t i P t 1 , . . .

Original: , m u , F p x q “ F i p x qu .
Revised: , m u , F p x q “ F i p x qu .

Original: The set-valued map J sF : R p Ñ R p ˆ q given by
Revised: The set-valued map J sF : R p Ñ R p ˆ q given by

Original: J sF : x Ñ conv `␣ B F i 
Revised: J sF : x Ñ conv ` B F i B x p x q

Original: B x p x q , i P I p x q (˘ , is a conservative Jacobian for F as shown in [11].
Revised: i P I p [Equation] q (˘ , is a conservative Jacobian for F as shown in [11].

Original: Here B F i B x denotes the classical Jacobian of F i .
Revised: Here B F i B x denotes the classical Jacobian of F i .

Original: Let us stress that such a structure is ubiquitous in applications [11, 34].
Revised: Let us stress that such a structure is ubiquitous in applications [11, 34].

Original: Rate of convergence.
Revised: Rate of convergence.

Original: We may now strengthen Corollary 1 by proving the linear convergence of piggyback derivatives towards the fixed point.
Revised: We may now strengthen Corollary 1 by proving the linear convergence of piggyback derivatives towards the ﬁxed point.

Original: The following is a consequence of the fact that the proposed selection conservative Jacobians of Lipschitz gradient selection functions are Lipschitz-like (Lemma 4 in Appendix E.1).
Revised: The following is a consequence of the fact that the proposed selection conservative Jacobians of Lipschitz gradient selection functions are Lipschitz-like (Lemma 4 in Appendix E.1).

Original: Note that semialgebraicity is only used as a sufficient condition to ensure conservativity of the selection Jacobian together with this Lipschitz like property.
Revised: Note that semialgebraicity is only used as a sufﬁcient condition to ensure conservativity of the selection Jacobian together with this Lipschitz like property.

Original: It could be relaxed if it can be guaranteed by other means, in particular one could consider the broader class of definable functions in order to handle log-likelihood data fitting terms.
Revised: It could be relaxed if it can be guaranteed by other means, in particular one could consider the broader class of deﬁnable functions in order to handle log-likelihood data ﬁtting terms.

Original: Corollary 3 (Linear convergence of piggyback derivatives)
Revised: Corollary 3 (Linear convergence of piggyback derivatives)

Original: In addition to Assumption 1, assume that F has a Lipschitz gradient selection structure as above.
Revised: In addition to Assumption 1, assume that F has a Lipschitz gradient selection structure as above.

Original: Then, for any θ and ϵ ą 0 , there exists
Revised: Then, for any ✓ and ✏ ° 0 , there exists

Original: C ą 0 such that the recursion (PB) with J F “ J sF satisfies for all k P N , gap p J x k p θ q , J pb ¯ x p θ qq ď
Revised: C ° 0 such that the recursion (PB) with J F “ J sF satisﬁes for all k P N , gap p J x k p ✓ q , J pb ¯ x p ✓ qq §

Original: C p? ρ ` ϵ q k .
Revised: C p? ⇢ ` ✏ q k .

Original: Moreover, classical Jacobians in Corollary 2 converge at a linear rate for almost all θ .
Revised: Moreover, classical Jacobians in Corollary 2 converge at a linear rate for almost all ✓ .

Original:  Application to proximal splitting methods in convex optimization
Revised:  Application to proximal splitting methods in convex optimization

Original: Consider the composite parametric convex optimization problem, where θ P R m represents parameters and x P R p is the decision variable
Revised: Consider the composite parametric convex optimization problem, where ✓ P R m represents parameters and x P R p is the decision variable

Original: [Equation]
Revised: [Equation]

Original: The purpose of this section is to construct examples of functions F used in recursion (1) based on known algorithms.
Revised: The purpose of this section is to construct examples of functions F used in recursion (1) based on known algorithms.

Original: The following assumption will be standing throughout the section.
Revised: The following assumption will be standing throughout the section.

Original: Assumption 2 f is semialgebraic, convex, its gradient with respect to x for fixed θ , ∇ x f , is locally
Revised: Assumption 2 f is semialgebraic, convex, its gradient with respect to x for ﬁxed ✓ , r x f , is locally

Original: Lipschitz jointly in p x, θ q and L -Lipschitz in x for fixed θ .
Revised: Lipschitz jointly in p x, ✓ q and L -Lipschitz in x for ﬁxed ✓ .

Original: Semialgebraicity implies that ∇ x f is path-differentiable jointly in p x, θ q , we denote by J 2 f its Clarke Jacobian.
Revised: Semialgebraicity implies that r x f is path-differentiable jointly in p x, ✓ q , we denote by J 2 f its Clarke Jacobian.

Original: The function g is semialgebraic, convex in x for fixed θ , and lower semicontinuous.
Revised: The function g is semialgebraic, convex in x for ﬁxed ✓ , and lower semicontinuous.

Original: For all α ą 0 , we assume that
Revised: For all ↵ ° 0 , we assume that

Original: G α p x, θ q ÞÑ prox αg p¨ ,θ q p x q is locally Lipschitz jointly in p x, θ q .
Revised: G ↵ p x, ✓ q ﬁ Ñ prox ↵ g p¨ , ✓ q p x q is locally Lipschitz jointly in p x, ✓ q .

Original: Semialgebraicity implies that it is also path differentiable jointly in p x, θ q , we denote by J G α its Clarke Jacobian.
Revised: Semialgebraicity implies that it is also path differentiable jointly in p x, ✓ q , we denote by J G ↵ its Clarke Jacobian.

Original: This assumption covers a very large diversity of problems in convex optimization as most gradient and prox operations used in practice are semialgebraic (or definable).
Revised: This assumption covers a very large diversity of problems in convex optimization as most gradient and prox operations used in practice are semialgebraic (or deﬁnable).

Original: Under Assumption 2, we will provide sufficient conditions on f and g for Assumption 1, for different algorithmic recursions.
Revised: Under Assumption 2, we will provide sufﬁcient conditions on f and g for Assumption 1, for different algorithmic recursions.

Original: These will therefore imply convergence as stated in Corollary 1 and 2, Proposition 1, as well Corollary 3 in the piecewise selection case.
Revised: These will therefore imply convergence as stated in Corollary 1 and 2, Proposition 1, as well Corollary 3 in the piecewise selection case.

Original: The proofs are postponed to Appendix F.
Revised: The proofs are postponed to Appendix F.

Original: 4.1 Splitting algorithms
Revised: 4.1 Splitting algorithms

Original: In this section we provide sufficient condition for Assumption 1 to hold.
Revised: In this section we provide sufﬁcient condition for Assumption 1 to hold.

Original: The underlying conservative
Revised: The underlying conservative

Original: Jacobian is obtained by combining Clarke Jacobians of elementary algorithmic operations (gradient, proximal operator in Assumption 2), using the compositional rules of differential calculus [11] and implicit differentiation [10].
Revised: Jacobian is obtained by combining Clarke Jacobians of elementary algorithmic operations (gradient, proximal operator in Assumption 2), using the compositional rules of differential calculus [11] and implicit differentiation [10].

Original: Using [12], such Jacobians are conservative by semialgebraicity and their combination provide conservative Jacobians for the corresponding algorithmic recursion F .
Revised: Using [12], such Jacobians are conservative by semialgebraicity and their combination provide conservative Jacobians for the corresponding algorithmic recursion F .

Original: These objects are explicitly constructed in Appendix F.
Revised: These objects are explicitly constructed in Appendix F.

Original: Forward–backward algorithm.
Revised: Forward–backward algorithm.

Original: The forward–backward iterations are given for α ą 0 by
Revised: The forward–backward iterations are given for ↵ ° 0 by

Original: [Equation]
Revised: [Equation]

Original: Proposition 2 Under Assumption 2 with [-
 ă α ă-] 2 L , denote by F α : R p ˆ m Ñ R p the forwardbackward recursion in (10) .
Revised: Proposition 2 Under Assumption 2 with [-
 ă α ă-]0 † ↵ † 2 L , denote by F ↵ : R p ˆ m Ñ R p the forwardbackward recursion in (10) .

Original: For µ ą 0 , if either f or g is µ -strongly convex in x for all θ , then F α is a strict contraction and Assumption 1 holds.
Revised: For µ ° 0 , if either f or g is µ -strongly convex in x for all ✓ , then F ↵ is a strict contraction and Assumption 1 holds.

Original: It is well known that if f is µ -strongly convex, choosing α “ 2 {p L ` µ q provides a contraction factor ρ FB “ p τ ´ 1 q{p 1 ` τ q , where τ “ L { µ ě 1 is the condition number of the problem.
Revised: It is well known that if f is µ -strongly convex, choosing ↵ “ 2 {p L ` µ q provides a contraction factor ⇢ FB “ p ⌧ ´ 1 q{p 1 ` ⌧ q , where ⌧ “ L { µ • 1 is the condition number of the problem.

Original: Douglas–Rachford algorithm.
Revised: Douglas–Rachford algorithm.

Original: Given α ą 0 , the algorithm goes as follows
Revised: Given ↵ ° 0 , the algorithm goes as follows

Original: [Equation]
Revised: [Equation]

Original: (11) where R αf p¨ ,θ q “ 2prox αf p¨ ,θ q
Revised: (11) where R ↵ f p¨ , ✓ q “ 2prox ↵ f p¨ , ✓ q

Original: ´
Revised: ´

Original: I is the reflected proximal operator, which is 1 -Lipschitz (and similarly for g ).
Revised: I is the reﬂected proximal operator, which is 1 -Lipschitz (and similarly for g ).

Original: Following [6, Theorem 26.11], if the problem has a minimizer, then p y k q k P N converges to a fixed point of (11), ¯ y such that ¯ x “ prox αg p ¯ y q is a solution to the optimization problem.
Revised: Following [6, Theorem 26.11], if the problem has a minimizer, then p y k q k P N converges to a ﬁxed point of (11), ¯ y such that ¯ x “ prox ↵ g p ¯ y q is a solution to the optimization problem.

Original: Following [26, Theorem 1], if f is strongly convex, then R αf p¨ ,θ q is ρ -Lipschitz for some ρ ă 1 and our differentiation result applies to Douglas-Rachford splitting in this setting.
Revised: Following [26, Theorem 1], if f is strongly convex, then R ↵ f p¨ , ✓ q is ⇢ -Lipschitz for some ⇢ † 1 and our differentiation result applies to Douglas-Rachford splitting in this setting.

Original: Proposition 3 Under Assumption 2 with α ą 0 , denote by F α : R p ˆ m Ñ R p the Douglas-Rachford recursion in (11) .
Revised: Proposition 3 Under Assumption 2 with ↵ ° 0 , denote by F ↵ : R p ˆ m Ñ R p the Douglas-Rachford recursion in (11) .

Original: If f is µ -strongly convex in x for all θ , then F α is a strict contraction and
Revised: If f is µ -strongly convex in x for all ✓ , then F ↵ is a strict contraction and

Original: Assumption 1 holds.
Revised: Assumption 1 holds.

Original: Following [26, Proposition 3], choosing α “ 1 {?
Revised: Following [26, Proposition 3], choosing ↵ “ 1 {?

Original: Lµ provides a contraction factor of order ρ DR p?
Revised: Lµ provides a contraction factor of order ⇢ DR p?

Original: τ ´ 1 q{p?
Revised: ⌧ ´ 1 q{p?

Original: τ ` 1 q ă ρ FB , where again τ “ L { µ is the condition number of the problem.
Revised: ⌧ ` 1 q † ⇢ FB , where again ⌧ “ L { µ is the condition number of the prob- lem.

Original: In this respect Douglas-Rachford’s iterations provide a faster asymptotic rate than those of
Revised: In this respect Douglas-Rachford’s iterations provide a faster asymptotic rate than those of

Original: Forward-Backward, which may also impact the convergence of derivatives in the context of Corollary 3.
Revised: Forward-Backward, which may also impact the convergence of derivatives in the context of Corollary 3.

Original: Alternating Direction Method of Multipliers.
Revised: Alternating Direction Method of Multipliers.

Original: Consider the separable convex problem
Revised: Consider the separable convex problem

Original: [Equation]
Revised: [Equation]

Original: The alternating direction method of multipliers (ADMM) algorithm combines two partial minimization of an augmented Lagrangian, and a dual update:
Revised: The alternating direction method of multipliers (ADMM) algorithm combines two partial minimization of an augmented Lagrangian, and a dual update:

Original: [Equation]
Revised: [Equation]

Original: As observed in [23], the ADMM algorithm can be seen as the Douglas-Rachford splitting method applied to the Fenchel dual of problem (12) (see Appendix F.3 for more details).
Revised: As observed in [23], the ADMM algorithm can be seen as the Douglas-Rachford splitting method applied to the Fenchel dual of problem (12) (see Appendix F.3 for more details).

Original: More precisely,
Revised: More precisely,

Original: ADMM updates are equivalent to Douglas-Rachford iterations applied to the following problem
Revised: ADMM updates are equivalent to Douglas-Rachford iterations applied to the following problem

Original: [Equation]
Revised: [Equation]

Original: Therefore, if ϕ θ is strongly convex with Lipschitz gradient and A θ is injective, then ADMM converges linearly and one is able to combine derivatives of proximal operators to differentiate ADMM.
Revised: Therefore, if  ✓ is strongly convex with Lipschitz gradient and A ✓ is injective, then ADMM converges linearly and one is able to combine derivatives of proximal operators to differentiate ADMM.

Original: 4.2 Numerical illustrations
Revised: 4.2 Numerical illustrations

Original: We now detail how Figure 2 discussed in the introduction is obtained, and how it illustrates our theoretical results.
Revised: We now detail how Figure 2 discussed in the introduction is obtained, and how it illustrates our theoretical results.

Original: We consider four scenarios (Ridge, Lasso, Sparse inverse covariance selection and Trend filtering) corresponding to the four columns.
Revised: We consider four scenarios (Ridge, Lasso, Sparse inverse covariance selection and Trend ﬁltering) corresponding to the four columns.

Original: For each of them, the first line shows the empirical linear rate of the iterates x k and the second line shows the empirical linear rate of the derivative BB θ x k .
Revised: For each of them, the ﬁrst line shows the empirical linear rate of the iterates x k and the second line shows the empirical linear rate of the derivative BB ✓ x k .

Original: All experiments are repeated 100 times and we report the median along with the first and last deciles.
Revised: All experiments are repeated 100 times and we report the median along with the ﬁrst and last deciles.

Original: Forward–Backward for the Ridge.
Revised: Forward–Backward for the Ridge.

Original: The Ridge estimator is defined for θ ą 0 as ¯ x p θ q “ arg min
Revised: The Ridge estimator is deﬁned for ✓ ° 0 as ¯ x p ✓ q “ arg min

Original: [Equation]
Revised: [Equation]

Original: Among several possibilities to solve it, one can use the Forward–
Revised: Among several possibilities to solve it, one can use the Forward–

Original: Backward algorithm applied to f : p x, θ q ÞÑ 12 } Ax ´ b } 22 and g : θ } x } 22 .
Revised: Backward algorithm applied to f : p x, ✓ q ﬁ Ñ 12 } Ax ´ b } 22 and g : ✓ } x } 22 .

Original: Since g is strongly convex, the operator F α is strongly convex, and thus Proposition 2 may be applied.
Revised: Since g is strongly convex, the operator F ↵ is strongly convex, and thus Proposition 2 may be applied.

Original: Forward–Backward algorithm for the Lasso.
Revised: Forward–Backward algorithm for the Lasso.

Original: Consider the Forward–Backward algorithm applied to the Lasso problem [49], with parameter θ ą 0 , ¯ x p θ q P arg min x P R p 12 } Ax ´ b } 22 ` θ } x } 1 “ arg min x 12 L } Ax ´ b } 22 ` θL } x } 1 , where L is any upper bound on the operator norm of A T A .
Revised: Consider the Forward–Backward algorithm applied to the Lasso problem [49], with parameter ✓ ° 0 , ¯ x p ✓ q P arg min x P R p 12 } Ax ´ b } 22 ` ✓ } x } 1 “ arg min x 12 L } Ax ´ b } 22 ` ✓ L } x } 1 , where L is any upper bound on the operator norm of A T A .

Original: The gradient of the quadratic part is 1 -Lipschitz, so we may consider the forward backward algorithm (10), with unit step size and f : p x, θ q ÞÑ 12 L } Ax ´ b } 22 and g : p x, θ q ÞÑ θL } x } 1 .
Revised: The gradient of the quadratic part is 1 -Lipschitz, so we may consider the forward backward algorithm (10), with unit step size and f : p x, ✓ q ﬁ Ñ 12 L } Ax ´ b } 22 and g : p x, ✓ q ﬁ Ñ ✓ L } x } 1 .

Original: A well known qualification condition involving a generalized support at optimality ensures uniqueness of the Lasso solution [20, 37].
Revised: A well known qualiﬁcation condition involving a generalized support at optimality ensures uniqueness of the Lasso solution [20, 37].

Original: It holds for generic problem data [50].
Revised: It holds for generic problem data [50].

Original: Following [10, Proposition 5], under this qualification condition, the implicit conservative Jacobian J F is such that, at the solution x ˚ , the matrix set I
Revised: Following [10, Proposition 5], under this qualiﬁcation condition, the implicit conservative Jacobian J F is such that, at the solution x ˚ , the matrix set I

Original: ´ J F only contains invertible matrices.
Revised: ´ J F only contains invertible matrices.

Original: This means that there exists ρ ă 1 , such that any M P J F p x ˚ q has operator norm at most ρ .
Revised: This means that there exists ⇢ † 1 , such that any M P J F p x ˚ q has operator norm at most ⇢ .

Original: Following Remark 1, all our convergence results apply qualitatively.
Revised: Following Remark 1, all our convergence results apply qualitatively.

Original: Note that we recover the results of [7, Proposition 2] for the Lasso.
Revised: Note that we recover the results of [7, Proposition 2] for the Lasso.

Original: Douglas–Rachford for the Sparse Inverse Covariance Selection.
Revised: Douglas–Rachford for the Sparse Inverse Covariance Selection.

Original: The Sparse Inverse Covariance
Revised: The Sparse Inverse Covariance

Original: Selection [52, 22] reads ¯ x p θ q P arg min x P R n ˆ n tr p Cx q ´ log det x ` θ ř i,j | x i,j | , where C is a symmetric positive matrix and θ ą 0 .
Revised: Selection [52, 22] reads ¯ x p ✓ q P arg min x P R n ˆ n tr p Cx q ´ log det x ` ✓ ∞ i,j | x i,j | , where C is a symmetric positive matrix and ✓ ° 0 .

Original: It is possible to apply Douglas–Rachford method to f : p x, θ q ÞÑ tr p Cx q ´ log det x and g : p x, θ q ÞÑ θ } x } 1 , 1 .
Revised: It is possible to apply Douglas–Rachford method to f : p x, ✓ q ﬁ Ñ tr p Cx q ´ log det x and g : p x, ✓ q ﬁ Ñ ✓ } x } 1 , 1 .

Original: It is known that f is locally strongly convex, indeed x ÞÑ ´ log det x is the standard self-concordant barrier in semidefinite programming [40].
Revised: It is known that f is locally strongly convex, indeed x ﬁ Ñ ´ log det x is the standard self-concordant barrier in semideﬁnite programming [40].

Original: Following Remark 1, all our convergence results apply qualitatively.
Revised: Following Remark 1, all our convergence results apply qualitatively.

Original: ADMM for Trend Filtering.
Revised: ADMM for Trend Filtering.

Original: Introduced in [51] in statistics as a generalization of the Total
Revised: Introduced in [51] in statistics as a generalization of the Total

Original: Variation, the trend filtering estimator with observation θ P R p reads ¯ x p θ q “ arg min x P R p 12 } x ´ θ } 22 ` λ } D p k q x } 1 , where D p k q is a forward finite–difference approximation of a differential operator of order k
Revised: Variation, the trend ﬁltering estimator with observation ✓ P R p reads ¯ x p ✓ q “ arg min x P R p 12 } x ´ ✓ } 22 `  } D p k q x } 1 , where D p k q is a forward ﬁnite–difference approximation of a differential operator of order k

Original: (here k “ 2 ).
Revised: (here k “ 2 ).

Original: Using ψ θ : u ÞÑ λ } u } 1 , ϕ θ : v ÞÑ } v ´ θ } 22 (strongly convex), A θ “ ´ I (injective), B θ “ D p k q , and c θ “ 0 , we can apply the ADMM to solve trend filtering.
Revised: Using ✓ : u ﬁ Ñ  } u } 1 ,  ✓ : v ﬁ Ñ } v ´ ✓ } 22 (strongly convex), A ✓ “ ´ I (injective), B ✓ “ D p k q , and c ✓ “ 0 , we can apply the ADMM to solve trend ﬁltering.

Original: Failure of automatic differentiation for inertial methods

Original: This section focuses on the Heavy-Ball method for strongly convex objectives, in its global linear convergence regime.
Revised: This section focuses on the Heavy-Ball method for strongly convex objectives, in its global linear convergence regime.

Original: For C 2 objectives, piggyback derivatives converge to the derivative of the solution map [28, 39, 36].
Revised: For C 2 objectives, piggyback derivatives converge to the derivative of the solution map [28, 39, 36].

Original: However, we provide a C 1 , 1 strongly convex parametric objective with path differentiable derivative, such that piggyback derivatives of the Heavy Ball algorithm contain diverging vectors for a given parameter value.
Revised: However, we provide a C 1 , 1 strongly convex parametric objective with path differentiable derivative, such that piggyback derivatives of the Heavy Ball algorithm contain diverging vectors for a given parameter value.

Original: In this example, other conservative differentiation means (implicit differentiation, piggyback on gradient descent), avoid this divergent behaviors.
Revised: In this example, other conservative differentiation means (implicit differentiation, piggyback on gradient descent), avoid this divergent behaviors.

Original: 5.1 Heavy-ball algorithm and global convergence
Revised: 5.1 Heavy-ball algorithm and global convergence

Original: Consider a function f : R p ˆ R m Ñ R , and β ą 0 , for simplicity, when the second argument is fixed, we write f θ : x ÞÑ f p x, θ q .
Revised: Consider a function f : R p ˆ R m Ñ R , and  ° 0 , for simplicity, when the second argument is ﬁxed, we write f ✓ : x ﬁ Ñ f p x, ✓ q .

Original: Set for all x, y, θ , F p x, y, θ q “ p x ´ ∇ f θ p x q ` β p x ´ y q , x q , consider the Heavy-Ball algorithmp x k ` 1 , y k ` 1 q “ F p x k , y k , θ q for k P N .
Revised: Set for all x, y, ✓ , F p x, y, ✓ q “ p x ´ r f ✓ p x q `  p x ´ y q , x q , consider the Heavy-Ball algorithm

Original: If f θ is µ -strongly convex with L -Lipschitz gradient, then, choosing α “ 1 { L and β ă ˆ µ

Original: [Equation]
Revised: [Equation]

Original:  L 
Revised: If f ✓ is µ -strongly convex with L -Lipschitz gradient, then, choosing ↵ “ 1 { L and  † ˆ

Original: [Equation]
Revised: [Equation]

Original: ˙ , the algorithm will converge globally at a linear rate to the unique solution, ¯ x p θ q [24, Theorem 4], local convergence is due to Polyak [44].
Revised: ˙ , the algorithm will converge globally at a linear rate to the unique solution, ¯ x p ✓ q [24, Theorem 4], local convergence is due to Polyak [44].

Original: Furthermore, if in addition f is
Revised: Furthermore, if in addition f is

Original: C 2 forward propagation of derivatives converge to the derivative of the solution [28, 29, 39].
Revised: C 2 forward propagation of derivatives converge to the derivative of the solution [28, 29, 39].

Original: 5.2 A diverging Jacobian accumulation
Revised: 5.2 A diverging Jacobian accumulation

Original: Details and proof of the following result are given in Section G.
Revised: Details and proof of the following result are given in Section G.

Original: Proposition 4 (Piggyback differentiation fails for the Heavy Ball method) Consider f : R 2 Ñ R , such that for all θ P R , f p x, θ q “ x 2 { 2 if x ě 0 and f p x, θ q “ x 2 { 8 if x ă 0 .
Revised: Proposition 4 (Piggyback differentiation fails for the Heavy Ball method) Consider f : R 2 Ñ R , such that for all ✓ P R , f p x, ✓ q “ x 2 { 2 if x • 0 and f p x, ✓ q “ x 2 { 8 if x † 0 .

Original: Assume that α “ 1 and β “ 3 { 4 .
Revised: Assume that ↵ “ 1 and  “ 3 { 4 .

Original: Then the heavy ball algorithm converges globally to 0 and ∇ f is path differentiable.
Revised: Then the heavy ball algorithm converges globally to 0 and r f is path differ- entiable.

Original: The Clarke Jacobian of F with respect to p x, y q at p 0 , 0 , 0 q is J F p 0 , 0 , 0 q “ conv t M 1 , M 2 u , where the product M 1 M 1 M 2 M 2 has eigenvalue ´ 9 { 8 .
Revised: The Clarke Jacobian of F with respect to p x, y q at p 0 , 0 , 0 q is J F p 0 , 0 , 0 q “ conv t M 1 , M 2 u , where the product M 1 M 1 M 2 M 2 has eigenvalue ´ 9 { 8 .

Original: The presence of an eigenvalue with modulus greater than 1 may produce divergence in (PB).
Revised: The presence of an eigenvalue with modulus greater than 1 may produce divergence in (PB).

Original: Set f 1 : p x, θ q ÞÑ " x 2 {if x ěx 2 {if x ă 0 . f 2 : p x, θ q ÞÑ " x 2 {if x ąx 2 {if x ď 0 .
Revised: Set [Equation]

Original: Note that f 1 and f 2 are both equivalent to f as they implement the same function.
Revised: Note that f 1 and f 2 are both equivalent to f as they implement the same function.

Original: With initializations x p θ q “ y p θ q “ θ , we run a few iterations of the Heavy Ball algorithm for θ “ 0 , and implement (PB) alternating between two steps on f 1 and two steps on f 2 and differentiate the resulting sequence p x k q k P N with respect to θ using algorithmic differentiation.
Revised: With initializations x p ✓ q “ y p ✓ q “ ✓ , we run a few iterations of the Heavy Ball algorithm for ✓ “ 0 , and implement (PB) alternating between two steps on f 1 and two steps on f 2 and differentiate the resulting sequence p x k q k P N with respect to ✓ using algorithmic differentiation.

Original: The divergence phenomenon predicted by Proposition 4 is illustrated in Figure 3, while the true derivative is 0 (the sequence is constant).
Revised: The divergence phenomenon predicted by Proposition 4 is illustrated in Figure 3, while the true derivative is 0 (the sequence is constant).

Original:  Conclusion
Revised:  Conclusion

Original: We have developed a flexible theoretical framework to describe convergence of piggyback differentiation applied to nonsmooth recursions – providing, in particular, a rigorous meaning to differentiation of nonsmooth solvers.
Revised: We have developed a ﬂexible theoretical framework to describe convergence of piggyback differentiation applied to nonsmooth recursions – providing, in particular, a rigorous meaning to differentiation of nonsmooth solvers.

Original: The relevance of our approach is illustrated on composite convex optimization
Revised: The relevance of our approach is illustrated on composite convex optimization

Original: [Figure]
Revised: [Figure]

Original: through widely used methods as forward-backward, Douglas-Rachford or ADMM algorithms.
Revised: through widely used methods as forward-backward, Douglas-Rachford or ADMM algorithms.

Original: Our framework allows however to consider many other abstract algorithmic recursions and provides thus theoretical ground for more general problems such as variational inequalities or saddle point problems as in [14, 9].
Revised: Our framework allows however to consider many other abstract algorithmic recursions and provides thus theoretical ground for more general problems such as variational inequalities or saddle point problems as in [14, 9].

Original: As a matter for future work we shall consider relaxing Assumption 1 to study a wider class of methods, e g., when F is not a strict contraction
Revised: As a matter for future work, we shall consider relaxing Assumption 1 to study a wider class of methods, e g., when F is not a strict contraction

Original: Acknowledgments
Revised: Acknowledgments

Original: J. B. and E. P. acknowledge the financial support of the AI Interdisciplinary Institute ANITI funding under the grant agreement ANR-19-PI3A-0004, Air Force Office of Scientific Research, Air Force
Revised: J. B. and E. P. acknowledge the ﬁnancial support of the AI Interdisciplinary Institute ANITI funding under the grant agreement ANR-19-PI3A-0004, Air Force Ofﬁce of Scientiﬁc Research, Air Force

Original: Material Command, USAF, under grant numbers FA9550-19-1- and ANR MaSDOL 19-CE230017-01.
Revised: Material Command, USAF, under grant numbers FA9550-19-1-7026, FA8655-22-1-7012 and ANR MaSDOL 19-CE23-0017-01.

Original: J. B. also acknowledges the support of ANR Chess, grant ANR-17-EURE-0010, TSE-P and the Centre Lagrange.
Revised: J. B. also acknowledges the support of ANR Chess, grant ANR-17EURE-0010, TSE-P and the Centre Lagrange.

Original: S. V. acknowledges the support ANR GraVa, grant ANR-18-CE40-0005.
Revised: S. V. acknowledges the support ANR GraVa, grant ANR-18-CE40-0005.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] Our submission follows the order mentionned in the abstract and introduction.
Revised: [Yes] Our submission follows the order mentionned in the abstract and introduction.

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] The two main assumptions are discussed in the main text.
Revised: [Yes] The two main assumptions are discussed in the main text.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] We believe that our work, being fully theoretical, does not bring negative societal impacts, especially points 1 to 7 of the ethics guidelines of NeurIPS 2022.
Revised: [N/A] We believe that our work, being fully theoretical, does not bring negative societal impacts, especially points 1 to 7 of the ethics guidelines of NeurIPS 2022.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] Points 1 to 5 are satisfied.
Revised: [Yes] Points 1 to 5 are satisﬁed.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] Our analysis requires a small number of assumptions: assumption 1 on contraction and assumptionon semialgebraic regularity.
Revised: [Yes] Our analysis requires a small number of assumptions: assumption 1 on contraction and assumptionon semialgebraic regularity.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] Most of the appendices are dedicated to them.
Revised: [Yes] Most of the appendices are dedicated to them.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] The (self-contained) jupyter notebook to generate the figures is submitted in supllemental material.
Revised: [Yes] The (self-contained) jupyter notebook to generate the ﬁgures is submitted in supllemental material.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See appendix H. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] See appendix H. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] See appendix H.
Revised: [Yes] See appendix H.

Original: Every curves report the median over 100 repetitions, along with first and last deciles.
Revised: Every curves report the median over 100 repetitions, along with ﬁrst and last deciles.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See appendix H. 4.
Revised: [Yes] See appendix H. 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See appendix I. (b) Did you mention the license of the assets?
Revised: [Yes] See appendix I. (b) Did you mention the license of the assets?

Original: [Yes] See appendix I. (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] See appendix I. (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: No new assets.
Revised: No new assets.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A]
Revised: [N/A]

Original: No nonpublic data used.
Revised: No nonpublic data used.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] No personally identifiable information.
Revised: [N/A] No personally identiﬁable information.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]