A functional framework for nonsmooth autodiff with {\it maxpooling} functions
Authors: Bruno Després
Venue: Accepted by TMLR ()

Abstract: We make a  comment on the recent work  by Boustany, by showing that the  Murat-TrombettiTheorem provides  a simple and efficient mathematical  framework for nonsmooth automatic differentiation of {\it maxpooling} functions. In particular it gives a  the chain rule formula which  correctly defines the composition of Lipschitz-continuous functions which are   piecewise $C^1$. The formalism is applied  to four basic examples, with some  tests in PyTorch. A self contained proof of an important Stampacchia formula is in the appendix.

Published in Transactions on Machine Learning Research (04/2025)
A functional framework for nonsmooth autodiff with mar- pooling functions
Bruno Després
bruno botr
Sorbonne Université, Université Paris Citée, CNRS, LJLL, F-75005 Paris, France MEGAVOLT, INRIA, Paris, France
Reviewed on OpenReview: https: openreview. net/ forum? id= waw.ooohtthhWhhhtee1////..//22
Abstract
We make comment the recent contribution Boustany (2024), by showing that the Murat & Trombetti (2003) Theorem provides simple and efficient mathemalical framework for nonsmooth automatic differentiation of mazpooling functions. In particular it gives a the chain rule formula which correctly defines the composition of Tipschitz-contimooss functions which additionnaly are piecewise-C*. Ct. The formalism is applied to four basic examples, with some tests PyTorc.. A self contained proof of an important Stampacchia formula in the appendix.
1 Introduction
In this work we make comment on the recent contribution by Boustany (2024). We will show that the Murat & Trombeuti (2003) Theorem gives a natural framework for nonsmooth automatic differentiation (nonsmooth autodiff). More generally we believe that the methods developed hereafter offer a comprehensive mathematical functional framework for the description of nonsmooth autodiff. Recent contributions on
mathematical issues for nonsmooth autodiff are developed in Bolte & Pauwels (2021); Bertoin al. (2021
2023 Boustany 2024 and references therein, based the Clarke's generalized derivative (Clarke 1990
The historical remark in Bolte & Pauwels (2021) provides insights related topics.
Notations and developments in this self contained article are kept to the minimum.
The first part introduces the notation and provides two examples which illustrate the apparent paradox discussed in this work. The second part focuses on a theoretical justification of chain rule manipulations for non smooth functions. Similar problems have been identified in the past in the partial differential community. Seminal references are Stampacchia (1963). Ambrosio & Dal Maso (1990) and Kinderlchrer & Stampacchial (2000). In the context of this work, the most important reference is the Murat & Trombetti (2003) Theorem which can be very conveniently adapted to describe the mathematics of nonsmooth autodiff. A preliminary work Berner et al. (2019) has already highlighted the potential interest of the Murat-Troun Trombetti Theorem for nonsmooth autodiff. However the discussion was restricted to activation functions only, so to the best of our understanding, the scope in Berner al. (2019) was limited to fully-connected feed-forward neural network function where the main difficulty can be ruled out by mcans of trivial simplification using R'(0) 0 where R is the ReLU activation function. Quite surprisingly the main illustrative example in the Murat- Trombettti contribution has exactly the structure of modern basic convolutional neural network function (CNN) [Begiio et al. 2017) with marpooling function.
That is why we provide a detailed and self contained proof of the Murat-Trombell Theorem. The original part this work is Theorem 2 and Proposition [ which explicit the chain rule for composition of many Lipschitz-contimious and piccewise-C" functions. We hope that this discussion will contribute to popularize this approach among the Machine Learning scientific community and to show its potential for the description of many problems that intervene in nonsmooth automatic different iation.

---

Published in Transactions on Machine Learning Research (04/2025)
The third part is devoted to some structural properties of the set of Lipschitz-coottinoous functions which are piecewise-C*. This set is evidently an additive algebra and, more important, it is an associative algebra for composition and nonsmooth automatic differentiation. In the fourth part, we compare with other frameworks for nonsmooth differentiation, such as Clarke's generalized diffcrential and alike and discuss the case of differentiation with respect the the weights of the Neural Network view of minimization of a Loss function. In the fifth part, we will describe the solution of basic problems the context of the Murat- Trombetit Theorem. A general conclusion will be that the gradient constructed through nonsmooth autodiff in PyTorch is systematically equal to an associated gradient in the sense of Murat- Trombetii.
The author warmly thanks François Murat for his deep explanations on mathematical methods for the differentiation of nonsmooth functions. He also thanks Hervé Le Dret for valuable help in manipulating Borel sets.
2 Notation and examples
A fully-connected feed-forward neural network function f :Ro + - can written as
f o fi-1 St-1 .... fi o fo
(1)
where the parameter & is identified with the number of hidden layers. The functions f. for i = o....,e are affine functions with varying input and output dimensions (4o, .1...., a2+1) E N with > 0, for = O..., More precisely, f((x,) = W,,, + b, E R for all R The intermediate functions S, for = 1.....f are nonlinear activation functions (Sharma et al. 2017). We adopt the normalization that < (x) S for almost all æ R. The ReLU function S; =
R(x) ma.
is extremely popular activation function used in production codes (Bengio et al. 2017). The mathematical issues discussed in this work come from the fact that the ReLU function is not differentiable at x = 0. The moint-wise non differentiability is shared with other basic functions in Neural Networks. For example mazpooling used in convolutional neural networks (Bengio et al. 2017 is based on the maximum function
(a, b) + max(aa b)
which is also not uniquely differentiable for b. In practice mazpooling is activated blocks windows of numbers on tensors of arbitrary dimensions. With adapled natural notations from Pintore & Després (2024), the representation T holds for CNN as well by taking fe to be the identity and S₃ to be softmar function [Bengio et al. 2017
A function ʃ : Ra + R Tipschitz-continuuus if there exists L 2 0 such that II[(>) - s(x')l S L| a'll for all 2, x' Ra. Therefore all these the activation functions S,, and by extension marpooling functions and all similar functions, are Lipschitz-comtinnous by assumption.
Since linear functions Ja are clearly Lipschitz-comtimmooss and the activation functions described above are also Lipschitz-contimmous,, then the food-forward function [ is Lipschitz-cootimuous by composition, that is f E Lip(R R2*+1), Any intermediate step is also Lipschitz-cootinuuus, that is
o ....o fi o S o So E Lip(R :R*++1) for 0 0srse
as well S, .... Si see also Morrey Jr (2009), states that the functions f, fr and S, for 0 S r & are differentiable almost everywhere (that is up to sets of zero measure). So the gradient of f, written matrix, is bounded Vf ( Ma (R)). Similarly one has
Lip Rox) for 0 < < l. The Rademacher 19199) Theorem,
A, = o o...
. ..... Si0 L Ma (R))
and
B, VS, S So 1 (R Ma,.ag
2

---

Published in Transactions on Machine Learning Research (04/2025)
If all functions are smooth enough, the matrix-valued functions A, and B, are defined without ambiguity. However these matrix-valued functions are ambiguous because the regularity is only Tipschitz-contimoous in our case. Then a natural mathematical question is to interpret the chain the rule
Vf(z)
(2)
which is product of matrix-valuer functions which are ambignously defined. For feed-forward functions. our notations imply that (x) = v2, (W,,,-b,) = W, is constant matrix so it is trivially non ambiguous. The difficulty is then concentrated in the matrices (x).)
Remark 1 (On notations). It must be noted that nonsmooth automatic differentiation with respect to the weights and biases leads to the same difficulties. For ezample the function x + f(x) in 7 would be noted + fa( where the variable 0 denotes all the paramelers of Neural Network (lypically all paramelers W and b, of the linear functions). The differentiability must be taken with respect to 0.
Fundamental is the ménimization of functions used for the identification of the parameters. Consider that one knows a finile dataset DD C Ro0 Ro made with pairs of inpul-oulpul variahles. Then supervised learning con be formulated as the minimization problem
= argmingJ where J(0)
E lfo(x) - yy
(a.y)ED
With our hypotheses the function + J(0) is Lipschitz continuous, however it is known to be highly non convex. The lools developed in this work can be used to describe the differentlabiilty properlies of VoJ, see Section
For the sake of coherence of the notations of this work, we restrict the notations to differentiabllity with respect to the input variable z, and let the reader do the easy generalization to differentiability with respect to the porameter 0.
To illustrate the issue and the apparent paradox, we consider two examples.
2.1 First example
The first example is degenerate in a sense. We take fo(x) wor where the weight wy E R and S1(x)
R(x). Then f(x)= R(wx2) and 2 becomes
('(x) = R'(wor)wo-
(4)
where (g) for y 0, R'(y) for y 0, but (0) is not defined. problem shows up if wo = 0 because the function x: R' R'(woz) is not defined in this case. Of course, one argue that the multiplication by wo 0 is enough to obtain the correct solution I' = O. However this operation is not correct solid mathematical grounds because the function R'(wor) is not mathematically defined for wo 0.
2.2 Second example
The second example is more problematic. It comes from Murat & Trombetti (0033) but is rewritten here with Neural Networks notation. Consider two functions fo and S,. The function fo + R22 is linear
fo(x) (x) Wo with Wo (1, while the second function S₁ is marpooling function over two values
(v) = max y2), where y = (V1, E R
(5)
Then f = S o fo is the identity f(x) for all E R, so that f = is of course bounded. The gradient of fo is f.fo = Wo. The gradient of S is defined almost everywhere, There are three cases
1
if if > V/
((i(v)
0
0
(6)
if < V2
VS) =
1
if yf
VS (v) is not defined.
3

---

Published in Transactions on Machine Learning Research (04/2025)
Now since f(x) z;, the chain rule formula
2 writes
VS(()()))Wo.
But fo(x) (z) so only the third case matters in 6 therefore (f)(f( is nowhere defined. Since Wo + 0, one can not argue as in the first example that the product with Wo is enough recover the correct solution. One obtains paradox since the left hand side equal to is perfectly known while the right hand side is not even correctly defined function.
2.3 General case
In general, the chain rule formula (2 does not make sense because the matrix-valued functions in the right hand side are not defined in non ambignous manner. In our opinion, it is related to some seemingly erratic behavior of nonsmooth autodiff related to the Boustany (2024) example detailed in Section 6.3
3 The Murat- Trombetio Theorem
The Murat Trombetti (2003) Theorem offers natural solution the apparent paradox explained the previous examples. is written in the sequel for the context Neural Networks. We slightly adapt the functional setting and notations from Murat & Trombetti (2003) and use two different notations for the gradient V and for an associated gradient. V.. The examples will show that associated gradients correspond gradients calculated with autodiff.
We will need the notion of Lipschitz and piccewiso-C' functions R R which is defined follows.
Consider a finite decomposition of I in Borel sets or Borel pieces pa
R UP".
pa n for #
for finite number of values of a, that S a S Omax < 00.
For simple functions, the pieces pa are usually piecewise affine with Lipschitz regularity. It means that they correspond to polygons, polyhedrons, lines, half lines, points and all finite unions and intersections of such objects in any dimension. This intuitive condition is evident in our examples so we do not detail it.
Definition 1. We say that pieces pa and functions fa E Lip(R :R") nc C R²) for 1 I a S Omax with the representation
function f : Ra + R² is piecewise-C" if there erists Borel
f(x) = 1pa Vo RR..
(8)
Here the notation 1, denoles the indicalrie function of a set w, that is l=(x) = for x Ew and I (=) for x & w. By definition (x) f(x) for all pa and f (R" Mo,a n (o Mo,((R)).
The main idea in Murat & Trombetti 2003) is the following.
Definition 2. We say that the gradient associated to the representation
S is
Vs(z)= = ((())=Era p (x) E Mo.a
Vr E T.o.
(9)
brief we refer to it as an associated gradient.
The associated gradient is a real b matriz defined everywhere, that is for all Ra It not unique since it depends on the representation { which is not unique. Corollary 7 will show that it is equal to the gradient almost everywhere.
Theorem 1
Consider two functions. The first one u E Lip(R R²) is Lipschitz- continuous. The second one v E Lip : is Lipschitz-contiinn contituoos and piecewise-Ci with a representation and with an associated gradient
9 Then the chain rule identäty holds in I (RR Maa (R))
V(vou)= Voou Vu

---

Published in Transactions on Machine Learning Research (04/2025)
where Vu denotes the function such that Vvou(x) Vo(u(z)) for all RR.
Hint of the proof. The key part of the proof is the third step in (Murat & Trombetti 2003 page 590) that we reproduce mutatis mutandi within the convenient functional setting. The two first steps in Murat. &
Trombetti (2003) are evident for and Lipschitz. The fourth and fifth steps concern additional properties.
Since is piecewise- C² it admits a piecewise smooth approximations denote as v" E (RR) for all a. Since V o u E CR (R9)) is a continuous function, the chain rule is applied without difficulty
V'0" o u) Ve ou Vu.
(10)
This identity holds almost everywhere (a.e.) with respect to E RR. Let Ua be the set
U === = u(z) E pd).
(11)
Since U the pre-image of the Borel set pa by the continuous function u, that Ua (Pa() then U is also Borel set so it is mcasurable set.
One notes that
v(u(x)) = v (u())) a.c. EUU
(12)
To use this identity one notes w = vow- ua Then one uses an intuitive but non trivial important properly from Stampacchia (1963); Kinderlehrer & Stampacchia 2000 (a self contained proof is in the appendix). It writes
Vw(x)= 0 a.e. x x ( R w(x) =0.
(13)
Since U C far RR w(x) 0}, it yields using 110
v(vou)(z) = Vo o u(x) Vu(x) a.e. tEU
(14)
One also has
(x) = 1p (u(x)) a.e. ZER
(15)
Consider the difference of the two termss in the claim A(x) = V(vou)(z) Voou Vu. It writes also
A(x) = ((vou) (Ea 1p- (u(z)) (u)z)
= (2a (u(x)))
Ea 1p (u(x))
1ya (x) (V(v Ve*(u(z)
(use 15
Ea 1u- (2)
(use
where all manipulations holds a.e. with respect to x. Therefore A(x) = a.e. which the claim.
Corollary 1. The associaled gradient of Desinition 2 is equal to the gradient Vf almost everywhere with respect to x. That is = Vf in the space 1 (IR. :Maa (R)).
Proof. Write Theorem with f and u(x)= = that Vu 7 is the identity matrix.
We now consider the composition of Lipschitz-continnoor and piecewise-C' functions, many desired. With respect to the literature nonsmooth differentiation discussed in Section 5 it seems that the next result is the first of its kind since it has no equivalent neither in Clarke 1990 nor even in Murat & Trombetti 2003).
Theorem 2. Consider Neural Network function I defined by the composition formula i where all func- tions fr and S, are Lipschitz for < r < L. Assume moreover that all fr and S, are piecewise-C' so that they admit associated gradients
A,
o Jo 1 (R Ma (TR)
5

---

Published in Transactions on Machine Learning Research (04/2025)
and
B, o....... fo E (RR" aa (RR).
Then the chain rule can be written as a product of associated gradients
vf = B
(x)) a.c. x E R
(16)
where the right hand side is defined for all RR
Proof. The proof is based on iterations of Theorem I
The first step is based f ft (Se fi-1 fo). It yields
f V Se fe-1 0 ................................ fo VS. o fe-1 0...
The gradient f is expressed as the product of one associated gradient and one gradient.
The second step is based on St ft-10... 1....... fo Se (fe-1 fo) One obtains
VS, fe-1
fo = vs fe- o .......... o V o .... of o
One substitutes this expression in the expression for VS which is now expressed the product of two associated gradients and one gradient.
Then iterations yields the result. The right hand side of the claim is defined for all a by definition of the associated gradients.
If the functions fr are linear one can simplify using (x)x) = W, which is constant matrix. One obtains the representation Vʃ ( ... Wi Bi Wo x E R2 where the right hand side is defined for all x E I
4 The algebra of Lipschitz-contimuous and piecewise- C' functions
We show that the set of functions which are Lipschitz-continuouss and piecewise-C" functions is algebra for the operation of composition, provided of course that the dimensions of the functions match. This property is the consequence of the next. Lemma.
Lemma 1. Consider two functions u E Lip(R" R²) and V E Lip(R": IR)). Assume u is piecewise- Ca with a representation u(x) = Ea p (x)) (x) for all x E IR. Assume v is piecewise-C" with a representation u(x) = Ee qqa (p) for all E Rbb. Then f =uvve Lip(Te Ro) piecewise-C* with represenlation
f(x) = Σ 1 (z) for all x ER.
(17)
a.
where Ra.8 {a E pa u(x)= Q33 and fa. v aa for all a and B.
Moreover the gradient Vf associated to the representation 17 salisfies
Ff(z) = Voou(z) for all x E 12.
=
(18)
Proof. One has Ra. = pa ( Since u is contimuous, then Ra.9 is the intersection of two Borel sets,
so it is also a a Borel set.
A proof that R Ro's"' 0 0 for (a, + (3', ) is by contraposilive. Assume R R .8' * there exists IR such that E pa.8 and x E Ra .g
Then Therefore I pa pa so a'. Denote y = w(x). One has y E Q8 nQ 3 B. So R Ra .0' #0 (a, = (d',8)) which is the contrapositive.
Therefore the sets Ra.9 are piecewise decomposition of Ra.
Consider the function g(z) = f(x) - En,s 1 o (x) and let us show it vanishes. Take æ E Ra.3 then g(x) f(x d (x). Since E pa, then n (x: u(x). Moreover y (x) w(x) Qo,

---

Published in Transactions on Machine Learning Research (04/2025)
v (y) =(y). It yields that g(z) f(æ) vo u(x). Since it holds for all and B, then g is the mull function which is the first part of the claim.
The final part is shown as follows. Consider the difference B(x) gradients are defined from their corresponding representation. One has
where the associated
B(x)
Σ 1 Ren (2) - - 10 (e 1p a
11pa (1
For R then
B(x) = (zaa((=) (u(()))))
V(ve (x) - Vo (ul(()))VVu((
Vu o (z - Vo (<(
But Ra C pa so (x) = u(x). Therefore B(x) 0 for Ra. Since it is true for all and B, one obtains the second part (18) of the claim.
The next Lemma shows that the previous algebra is associative. We need some notations to state the result..
We consider three functions u E Lip(R* R²), e Lip( R9) and Lip( Rd), all of them being piecewise-C"
u(x) E (x)
BER R", x E R². E R
v(x) Eg 3 (x)
w(x) E, I L (x)
with their corresponding associated gradient. For these functions and for the other ones below, only write the representation of the functions since the representation atio of their gradient is immediale.
The function w U u admits two formally different representations, the first one assembled in the order w o (vou), u), the second one assembled the order (wor)ou. u. It might yield two different associated gradients. Actually this not the case and both representations and their associated gradients are the same as shown in Lemma for which we need some preliminary notations.
Let us firstly detail the representation 17 of f = von. We write it
f(x)
1 1ga.s fa (x) for all e R
where ga.B {a Pa, u(x) E Q9) and ya. v n Invoking one more time Temma 1 the function g w (vo w o f is Lipschit and piecewise-C" with a representation deduced from the one of w and the one of f. We write it
9(x)= = Σ (x) o (x)
(19)
a.S.y
where g(a.8),7 = {x Sa.8 (x) R
The second possibility is to start from = w o u which admits the representation 17
with an associated
gradient (18 We write
p(x) Σ
1pa (x) for all &
8,7
where T fx RT and p n o v Finally q (woz) pou has the representation
((x)
Σ I (z) (x)
(20)
a.9,7
where (9.) (x pa, u(x) To.0).

---

Published in Transactions on Machine Learning Research (04/2025)
Lemma 2. The representation of 19 and the representation of 20 are identical. More preciscly
the Borel sets are equal S(a.9),7 = Ta.33) for all a, 8,
the composed functions are equal w faud = ppo our for all a, 3:71 and the functions are equal g(x) = q(x) for all x E Re. The associated gradients of g and q are identicol as well.
Proof. Expansion of all terms shows that S(a.9).7 = Ta.(9.)) = for E pa, "(()) E QQ, vo E R and that w o fa.g = p o x² = w² v² ou" so the representations are the same. The associated gradients are equal as consequence of 18
The previous results are used to propose another version of Theorem 2
Proposition 1. Make the assumptions of Theorem Then two properties hold: Sunction I ilself is conlinuous-Lpischee and piecemisseC",; the left hand side of 16 can be written
(z) f B()))) (1)
for all x E E
(21)
where a representation formula for the associated gradient f is determined from the multiplication in any order of the associated gradients in the right hand side.
Proof. Use Lemma and Lemma 2
Note that the addition of two Lipschitz-comtinuouu continuous and piecewise-C" C* functions is also Lopscmos and piecewise-C* C function. This evident property will be used in the numerical Section.
Lemma 3. Consider two functions
E Lip(R" Re) where u is piecewise-C' with a representation
u(x) = Ea (p-1p(r))t (x) for all x E Ro and is piecewise-C" with representation v(z) = En 1Q= (z) for all Rb. Then f = ut Lip(R 1R)) is piecewise-C" with representation
f(x)= = (z) (z) for all Ro
where B P" n Qa and fa. un + 8 for all and B, and the associated gradients additive ff Vu Ve.
Proof. Ra.9 the intersection of two Borel sets, so it is also a a Borel set. The rest of the proof is evident.
The multiplication of Lipschitz-comniinous and piecewise-CC C function by real number IR gives function Af which evidently is also Lipschitz-comtimmous and piccewiso-C!.
5
Additional theoretical considerations
This Section contains some reflexions about the positioning of the previous results with respect to the literature non smooth differentiation. The discussion below does not pretend to be exhaustive.
5.1 Comparison with other nonsmooth differentiation frameworks
We base the discussion on two corpus of references, which are on the one hand the works issued from Clarke's generalized gradients (Clarke] 1975; 1990) and on the other hand the recent contribution Li al. (2020)
and references therein which compare different notion of gradients for the characterization of stationary
8

---

Published in Transactions on Machine Learning Research (04/2025)
points in nonsmooth optimization. We restrict the discussion to functions which are Lipschitz--contimous and piecewise-C*.
Non smooth optimization has a rich mathematical history concerning the development of generalized gradi- ents beyond the classical derivative. The first notion is sub-differential for convex functions (Clarke 1990): where typically the sub-gradient at = of the absolute value function x + f(x)= = | is the closed interval 0f(0) = [-1,1] R. This interval contains all slopes a E such that ar S f(x) for all x. The definition of sub-differential is restricted to convex functions.
A more general notion is the generalized gradient in the sense of Clarke (Clarke 1990). also referred to Clarke's 's subdifferential. It can be shown that the Clarke's generalized gradient is the convexification of the local directions of differentiability (called the Bouligand differential), that is (Li et al. 2020)
dos(x) = conv (On[(x)).
This definition applics to concave functions such as z + g(x) = - -1ø|. It has the important interest that 0cg(0)= = [-1,1] [-1,1 so 0 E (C)(() which is a characterization of the fact that x = 0 is an stationary point (i.e. extremal point) of g.
However Clarke's gencralized gradients are not the panacea to treat all situations. A paradoxical situation
taken from Li et al. (2020) is for the function x ++ h(x) + x² sin(1/x)) The derivative is 1 for < 0, and is equal to 2z s1/(1/ c1/21/ for 0. The classical derivative at = 0 is clearly ((())
limg-z,, y h()) while it can be checked that the Clarke's gradient is fc = [0. 2] which is of
course not satisfactory. Also Clarke's gradient does not satisfy the sum rule but only an embedding rule
0c(S + 52) c ach + 8c
(22)
The example in Li et al. 2020 is as follows. Take f1(x) = max( 0) (the ReLU function) and f2(x) = min(x, 0). Then (f1 + f2) (z) x so one can check that 0c(fi + (2) = {1} while fc + f2
[01] + [0, 1] 2]. So the embedding (22 loose even in elementary siluutions. Clarke's generalized
gradients of product of functions is also loose embedding (Clarke 1990 Proposition 2.3.13).
Another point is that Clarke's generalized gradicnts can be used for the composition of two functions but with restrictions. Typically at least one over the two functions must. be either smooth (that is differentiable in the classical sense) or regular which boils down to be essentially either concave or convex (Clarke 1990 Theorem 2.3.10 Chain II). Up to the knowledge of the author of this contribution, Clarke's generalized gradient does not manage to describe the gradient of the composition of more than two nonsmooth general functions. The partial-iiferentia--quual the composition of two functions.
based theory Ambrosio & Dal Maso 1990) is also restricted to
On the contrary the associated gradient in the sense of Murat- Trerat--Tomeetti is function defined everywhere which is equal the gradient in the sense of distribution. This equality holds up to a set of measure zero since it holds in the sense of distribution. With respoct to the examples just discussed above in the Section, striking properties are that the Mural-Trombelti itself, formula (16) in Theorem 2 and formula (21) in Proposition
are all equalities. The set of Lipschitz-continnonn and piecewise-C" functions introduced in this work is algebra. Of course this thcoretical gain has its own price, typically an associated gradient is not unique. However the applicative examples in next Section will show that this non uniqueness can also be seen as an excellent property, because it corresponds to what is observed in non smooth automatic infferentiation in standard softwares (such as Pytortch, Tensorflow, Ja..
5.2 Differentiation with respect to the weights and application to minimization
All previous considerations about differentiation with respect to the input variable æ have immediate ge- neralization ion for differentiation with respect to the parameters 0 of the Neural Network, see Remark [ The main difference concerns the sets of measure zero evoked for example either in the proof of Theorem I or in in Formula (6). Indeed these sets of mcasure zero are now to be considered as embodded in the parameter space. Since it theoretically difficult to determine the influence of these sets in general, let us consider instead practical scenario.
9

---

Published in Transactions on Machine Learning Research (04/2025)
This practical scenario is basic gradient descent method for the Loss function
(t)) = (((()
(23)
where we assume that the gradient of the Loss is a certain type of modified gradient (note that softwares necessarily provide a numerical gradient for all 0, if not the code would not run in all situations). We write the modified gradient as
J( = 2 E (Jo(m)- vololt))
(a.N))
Clearly the history of the gradient descent (23 depends on the initial guess (0) = 0b and on the choice of the modified gradient because a modified gradient not unique.
Being able reach any conclusion on the influence of modified gradients on training sessions is task of formidable importance for the theoretical study of practical training sessions.
6 Examples
We illustrate the interest of the Murat-Trombelli Theorem on simple examples where the number of layers is limited for the sake of simplicity and the differentiation is taken with respect to a given variable which can represent cither the input z or the parameter 0.
6.1 Back to the first example
The issue is the value of the derivative of the ReLU function at the origin.
One can simply use three Borel pieces pl = (-oc, 0), p2_ = (0, oc) and p3 {0) to construct the associated derivative R'. The three smooth functions are f', f² and få. By definition (x) R(x) = 0 for x E P! then (z)) 0 for E pl. Similarly f2(z) R(x) = for p2, then (z))(z) for p2. The only constraint on f is (0) = O, so (f²)(0) can be any real number. Let us note (f²)(0) = IR.
So the associated derivative is
[z) = 0 for for 0, (z)z) = for 2 x> 0, (z) = z for x = 0.
Clearly the associated derivative depends on the choice of
Most the studics about the influence of the derivative of the ReLU at the origin are restricted to 2 = 0
that is to (0( = 0, see Boustany 2024g: : Berner et al. 2019 ; Bertoin et al. (2221) The previous detailed analysis shows that = # 0 is also possible.
Whatever the value of 2, then (4) becomes J" Rt' wp which is non ambiguous for wp 0 since R²' is correctly defined. Note that the derivative of ff is written
associated gradient (associated derivative in this case) thanks to Lemma [ However, once again, the use of an associated gradient is not mandatory in this case since there is no real difficulty for wp 00
6.2 Back to the second example
To construct an associated gradient for the mazpooling function S
5 one can distinguish three Borel
pieces which are pl = (1)1,2) E R² 31 < ya). p2 = {(1.. 32) E R 31 > 12 and p3 = {(11,8)) e
R : 91 = )- Then the smooth functions are fl, f² and f Clearly (v1. = y1 in pl, so that (f' U2) = (1, in pl. For similar reasons, Vf 92) = 10 in p2.
The critical situation concerns p3. The construction principle {8) yields that y1 y2 = ( U2) p3 It can be written y (v, for all y E R. The function fd being continuously differentiable, one has necessarily 1 0,, f v) + 0ge o (y. for all y, that is
1 Oy, (v1, y2) + 2 (v1, p3.
(24)
10

---

Published in Transactions on Machine Learning Research (04/2025)
Let now examine what is the meaning of the modified chain rule formula which replaces the initial one This modified chain rule formula can be taken from Theorem
(s(()) for all x e R.
(25)
The key observations are that Wo = (1, and that (=)()()))== Vf since fo(z) E P Then 25 reduces the identity (24) which holds by definition. So the paradox does not show up again.
Remark A simple geometrical interpretation emerges from the fact that (25) reduces
24 Actually
the gradient of f can take any value in the direction normal the line pa while it takes the correct value in the direction tangent to p3.
6.3 The Boustany example
This example is proposed in Boustany 2024) to exemplify the issues at stake with nonsmooth autodiff with mazpoooling functions. The example is directly implemented in PyTorch. One defines a first maximum function max, for a vector æ E Iº of arbitrary size a 2 1, together with second maximum function maxş
which is a PyTTrch function. The scripts taken from Boustany (2024) are in Table 1 Then for given & E R one defines the function L + f(t) = max (tx) (lx)
def max (x):
res x0]] for i in range a): if x[i] > res: res x [i]
def maxz (x): return morch.max
return res
Table 1: Script of the functions max, and maxe
By construction f is the mull function.
However it is reported (Boustany 2024 Table 1) that the derivative calculated with aulodiff in Py Porch is not zero. More precisely take x = 21 3, 4), then ft( is (mumerically) zero everywhere except at t = 0 where the derivative As 1.5. Our own tests reported in Table confirm this observation.
t derivative of f
-1
-0.5
-0.01
0
0.01
0.5
1
0
-1.5
0
Table Values of the derivative of calculated with autodiff within PyTorrh
The explanation in the context of the Murat-Trombeetii representation formula 19 is as follows. The function
max; calculated with autodiff is given in Table It yields that the associated gradient of max, is calculated accordingly to the decomposition ion of the space in Borel pieces
p1 {a E
I 2 max (#2, F3, x4)}.
f1 F1,
p2 {ir E R4
I 32 and 22 2 max x4)h,
f²(x) 32,
(26)
p3 {ir R
max(21, < 23 and I3 2 zsh.
(() = F3
pi {ir E R
max(x11.22) .3) < #4).
M(xx) = T4
where x = (I1, T4). Then the associated gradient is
Vmax (1,0,0,0) in pl, 10,1,0,0) in P2, (0,0, 1,0) in p3, (0,0,0,1) in p4.
(27)
11

---

Published in Transactions on Machine Learning Research (04/2025)
The representation that propose for max² is different. It is based on different Borel pieces
Q' {ir R+
I; > max(
f'(x) Fit
1,
Q { R4]
21
32 max (13, 344
{$(x) (I1 + x2)
Q6 {I R+|
21 23 max 22, 244)
f((x) (I1 +x3) xz3/
Q? = {Ir E R-1
21
max [22, 333
f²(x) = (#1 + +x4) /2,
Q = {Ir E R-1
22 = > max(11, 24)
f*(x) = (æ2 + +xs) x2)/22
Q = {xr E RR1
4/2
> max [11, 13) /:
(9(x) = ( + +x4)/2
(28)
Q10 {x E R41
23 = ΣΑ > max [11, 22
(1)(z) = (#3 + x4)/2,
Q { E R+1
21 Σ2 Σ3 > Z4/,
fl (21 + 22 22 +z3//3,
Q12 { E R.
(12 (x1 + + r4)/3,
21 Σ2
84 > 23
Q13 { E Rd
(L3 (x1 + 23 r4)
21 Σ3
> 22
Ql4 {x E R
(4) (22 + + x4)
12
Σ3
Z1
Q {x E R+
21 = I
241
(1)(I) [1 + 22 + D3 IA1/4.
It yields
1
Vmax2 22,
(y1. 92. 93. V4)
(29)
N (Z Ø2, Ø3, x4)
where
Na1,, 23, .4) the number of xs (1 S 4) equal to the maximum value,
V1 = if x; = max( F2, F3, va), and z; = if a, < max(#1, 22, F3, x4).
Remark 3. The eramination of (28) shows that Vaxxa is symmetrized. One has for example
1 maxx -0) (1,
(0,0,0,0).
(30)
This value corresponds to the piece Q which is a line in a space of dimension 4.
More precisely, the associated gradient maxx is invariant under the action of permutations among the volues equal to the marimum.. This is confirmed by numerical evidence based on elementary tests in Py
Now let us calculate the associated derivative of f at ('(0) max (1, 21,33 31, 46) t=0 at maxx(1, 21, 31,,314444 40) that is f'(0) Vmax, (0) 21, 2, 3, 4)
with the rules of automatic differentiation
Vmaxx(0)
2, 4), where the notation of the associated derivative is justified using Lemmas [ and 3 One obtains
(())) 01. 0) (1, 4) (1.1,
(1,2,3.4) (1,,2,33)11
10/4 1.5 which is the value reported in
Boustany 2024) and in Table 2
6.4 A simplified Boustany example
Finally we prepare another simple example in the spirit of Boustany (2024), but where the maximum is caleulated with the MarPoolid function of PyTorch. Maxpooling is important and popular operation in modern neural networks.
We assemble the function f(t) = maxpoolld(t, 4t) maxpoolld(4t, t). Some values implemented in PyTorch are given in Table 3 The explanation of the value of (0) is follows. Actually all numerical test show that the PyTorch function maxpoolld torch. mn. wa...aooollllll MaxPoolld(2, stride = 1) with window of 2 elements has the same associated gradient the function maxı presented in Table 1 Therefore Vmaxppolldd(000) 0) (1,
Then (0) = (1, (1, - (4, ()(, (1, 4) = -3 which the value in Table observed in the numerical
tests.
7 Conclusion
The Murat- Trombettt Theorem provides a simple functional framework which allows to manipulate composi- tion of Lipschitz-contimious and piecewise-C* functions. It constructs associated gradient which is defined
12

---

Published in Transactions on Machine Learning Research (04/2025)
t
-1
-0.5
-0.01
0.01
0.5
1 0
derivative of
0
Table 3: Values of the derivative calculated with autodiff within PyTorch
for all values of the input variable. An associated gradient not unique nevertheless. We have observed that the gradient obtained from nonsmooth autodiff in PyTorch is systematically equal to an associated gradient in the sense of Murat-Trombetti. This approach also provides non ambignous chain rule formula, that was actually the key in the original paper Murat & Trombetti (2003) Then we defined the framework of Lipschitz-contimmous contimuous and piecewise-C" C functions which is an associative algebra for the composition of functions. Connections with the method of breakpoints described in Daubechies et al. (2019) is a priori possible. We mention some problems which could be the subject for future research. Some of them are already evoked in Berner et al. (2019)
Evaluation of the Lipschitz constant of function modeled with Neural Network: The numerical evaluation and use of the Lipschitz constant of given Neural Network function where the weights W, and the biaises by are given has the subject of recent. research (Combettes & Pesquet 2020 Virmaux & Scaman,
2018 Pintore & Després associaled gradients.
2024
Béthune 2024). More solid foundations for these works can be obtained with
The associated gradient has been introduced and studied in this work on examples with limited number of layers and with limitod number of variables. In practice a Neural Network function is defined with respect. to space variables (lypically in ([) and to parameters (lypically W, and b, in (.). Then it raises the mathematical question of the definition of an associated gradient with respect to all variables. There is major practical interest in designing an associated gradient with respect to the parameters W, and b, only. This has been evoked in Section 5.2 It can be used to describe a functional setting for training sequences and to compare with the numerical tests in Boustany (022)).
More variables and training:
SciML: SciML a new discipline that seeks to use ML for solving PDE (partial diffcrential equation) models that show up physies and engineering (Klawonn & al., 2024 EMS TAG), (Després al.) 2024 is recent event). J. An important example is PINNs (Physically Informed Neural Networks), refer to Mishra & Molinaro (0233) and references therein. It is reasonable to foresee that some integral formulations used in SciML can be justified with Theorem
Acknowledgments
This work received fundings by Agence Nationale de la Recherche, program France 2030, reference ANR-23- PEIA-0004.
References
Luigi Ambrosio and Gianni Dal Maso. A general chain rule for distributional derivatives. Proceedings of the American Mathemotical Society, 108(3))691-702, 1990.
Yoshua Bengio, Ian Goodfellow, and Aaron Courville. Deep learning, volume 1. MIT press, 2017.
Julius Berner, Dennis Elbrächter, Philipp Grohs, and Armulf Jentzen. Towards regularity theory for relu networks-chain rule and global error estimates. In 2019 13th International conference Sampling Theory and Applications (Sump TA), pp. 1-5. IEEF, 2019.
David Bertoin, Jérôme Bolte, Sébastien Gerchinovitz, and Edouard Pauwels. Numerical influence of relu'(0) on backpropagation. Advances in Newral Information Processing Systems, 479, 2021.
David Bertoin, Jérôme Bolte, Sébastien Gerchinovitz, and Edouard Pauwels. Erratum: Numerical influence of r0) on backpropagation. Technical report, 2023. URL httpss///aallcccenoo//aaaa22222 File/Impact.p of Re___eLU____ BacllU_prim___e .dff
Louis Béthune. Deep learning with Lipschitz constraints. PhD thesis, Université de Toulouse, 2024.
13

---

Published in Transactions on Machine Learning Research (04/2025)
Jéröme Bolte and Edouard Pauwels. Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning. Mathematical Programmingg 188:19-5 188:19-51, 2021.
Ryan Boustany. On the numerical reliability of nonsmooth autodiff: a maxpool case study, Transactions on Machine Learning Research, 2024. URL https:///axxvv.rrg abs//401/2....... 027366
Frank H. Clarke. Generalized gradients and applications. Transactions of the American Mathematical Society, 205:247-262, 1975.
Frank H Clarke. Optimization and nonsmooth analysis. SIAM, 1990.
Patrick Combettes and dhan-Christophe Pesquet. Lipschitz certificates for layered network structures driven by averaged activation operators. SIAM J. Math. of Data Science, 2(2):529-537, 2020.
Ingrid Daubechies, Ronald DeVore, Simon Foucart, Boris Hanin, and Guergana Petrova. Nonlinear approximation and (deep) relu neural networks. Constructive Approzimation, 55:127-172, 2019.
Bruno Després, Victorita Dolean, Emnanuel Franck, Stéphane Lanteri, Victor Michel-Dansac, and Laurent Navoret. Sciml 2024, 2024. CRL attpp://irmnt math, unistra wtt. http:://////mooooaaaaaaaaaaaa/// html
Lawrence Craig Evans. Measure theory and fine properties of functions. Routledge, 2018.
David Kinderlehrer and Guido Stampacchia. An introduction to variational inequalities and their applica- tions. SIAM, 2000.
Axel Klawonn and al. Topical Activity Group Scientific Machine Learning of the EMS, 2024. URL https: lens-tag-sciml. wwt.ss.mmm....b......m.. 10
Jialin Li, Anthony Man-Cho So, and Wing-Kin Ma. Understanding notions of stationarity in nonsmooth optimization: A guided tour various constructions of subdifferential for nonsmooth functions. 1EEE Signal Processing Magazine, 37:18 31, 2020.
Siddhartha Mishra and Roberto Molinaro. Estimates on the generalization error of informed neural networks for approximating pdes. IMA Journal of Numerical Analysis, 43(1):1-43, 2023.
Charles Bradfield Morrey Jr. Multiple integrals in the calculus of variations. Springer, 2009.
François Murat and Cristina Trombetti. A chain rule formula for the composition vector-valued function by a piecewise smooth function. Bollettino Uninne Matematica Italiana, 6(3):581-595, 2003.
Moreno Pintore and Bruno Després. Computable lipschitz bounds for deep neural networks. Technical report, 2024. URL htttss://iniitii hal. wtt.aaaaeaecaeee////000060611111
Hans Rademacher. Über partielle und totale differeenzierbarkeitt funktionen mehrerer variabeln und über die transformation der doppelintegrale. Mathematische Annalen, 7944):340-359, 1919.
Sagar Sharma, Simone Sharma, and Anidhya Athaiya. Activation functions in neural networks. Towards Data Sci, 6(12):310-316, 2017.
Guido Stampacchia. Equations elliptiques du second ordre à coefficients discontinus. Séminaire Jean Leray, (3):1-77, 1963.
Aladin Virmaux and Kevin Scaman. Lipschitz regularity of deep neural networks: analysis and efficient estimation. Advances in Newral Information Processing Systems, 31, 2018.
14

---

Published in Transactions on Machine Learning Research (04/2025)
A A self contained proof of the Stampacchia property
The Stampacchia property Stampacchia (9633 w E Lip(RR) is such that
Kinderlehrer & Stampacchia (2000) states that a function
Vw(x) = a.e. I x fx w(xx 0}.
(31)
A simple proof comes from a regularization technique. See also Evans (2018)
First regularization. Consider x | = vz² + for
The derivative is de [o|s Vz*+e"
Thanks to the Rademacher (1919) Theorem, w admits a gradient Vw E (R RR). Also |ue| admits a gradient V/wl L [Rt IR) as well because |w] is also Lipschitz. Take a vectorial smooth test function with compact support up E C (R" IR)) The integration by part formula holds
V (w](()) p(r)dr =- |ul(z)V -Llllca p(z)d=== =- lim [ullc(t)V (((())))V(((dd.. ((r)dr.
--0+
There is no difficulty in passing to the limit because is continuous. A reverse integration by parts shows that
- [wl ((x)ddx = V/wle(>) (d(r)do = /u(z)++e
wix)
Vw(x)) ((x))dr
=lu(e)s0 w(x)
Vw(x)=+ Vw(x) p(z)dxx + Su(or Vu(x)248 Vw(a) ((x)dr Su(o)= Vw(a)
w(x)
w(x) Vu(()) ((())dz
In the right hand side, the last integral vanishes of course. In the first integral one has the boundedness
m(xx Vw((x) ((x) s (w((x) ((x) where the right hand side the function x - |w(xx) (p(x) defines function in L (R*). One also has pointwise convergence almost everywhere with respect to æ w(x) Vw(a)) (xx)) + sign(u(z)) p(x) a.c. x. Therefore the Lebesgue dominated convergence
Vw(a))+s
Theorem yields
w(x) Vw(x) :(()dd== / Vw(z) p(z)dx.
lim 8-0 w(x)s0 V /x(x)²
ww())0
Similarly lom Ju(e)co Vu(a)tE Vw(x) ((x)dd = Su(o)e Vu(()) (p(x)d It yields the formula
m(x)
V 7/w[(x) - p(x)dr= = / Jutoaoo Vw(x) p(x)ddx - mn(a)<o
Vw(x) p(r)dr.
(32)
Second regularization. Let us now redo the calculation but starting from a different regularization of the absolute value. We take + |0| = v (I va)² + for E > 0, with derivative |
#tve
V(a+
One has f (zw) p(x)dr - 1 (=) p(x)dr lim (w)l(()) ppx)de and [w) p(x)dr V 10 (x) p(o)de - w(2)+VE Vw(x) =<x)da
Viule)+VEjate
Ju(a)SO (w(())-E))++ Vu(x) ((x)dx + Su(E)eo ((w(a)+v8)443
w(x)+vE
w(x)+VE Vw(x) (
Su(Ga==
((m(a)+ Vw(x) c(x)dx w(x)tvE
((u(t)-Va)2+e Vu(x) ((x)dr + Su(ereo ((u(s)+ V2 (w(a)=o Vw(ax) (fr)ddr
Su(a)>0
w(x)tVE - c(x)drr
The Lebesgue dominated convergence Theorem yields the same limit before for the two first integrals. However the third integral remains. One obtains
[
Vw(x) ((x)dxx+ + /2 / /mt) Vw(x) ((r)dd. (33)
V [w]( p(x)dx=
J
Final part of the proof. Comparison of 32 and
33 yields Su(r)= Vw(x) :p<x)r 0 for all e
(IR.: RR). This is equivalent to the Stampacchia property
31 because the test function yp arbitrary.
15


Reviews:
[{"review_id": "hqfD3TXSvc", "reviewer": "Reviewer_jYJx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- The paper tackles an important topic, often overlooked in deep learning: how to differentiate non-smooth functions. Overall the examples are convincing, and the community could greatly benefit from these theoretical tools\n\n\nWeaknesses:\n- **The clarity**. The contribution of the paper is to present the Murat-Trombetti Thoreom to the machine learning community, however, in its current form, the manuscript cannot be understood by machine learning practitioners. I think the paper is very important and I will champion it, **but it will require massive changes in order to have value for the machine learning community**", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The purpose of the paper is to provide a correct chain-rule formula for Lipschitz functions, which are piecewise $ C1$ functions.\nFirst, the authors provide failures of the standard chain rule formula on Lipchitz piecewise $C^1$ functions, including Relu and maxpool functions. The authors then recall the Murat-Trombetti Theorem that provides a \"correct\" chain rule formula for this class of functions. The authors then show that the class of Lipchitz piecewise $ C1$ functions is an algebra for the composition operation. Finally, they provide an example of how to apply the Murat-Trombetti to machine learning-related functions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Clarity:\n- page 1. what is the goal of this introduction? The into felt very unstructured to me and should be rewritten.\n- page 2. I would recall Rademacher 1919 Theorem in a proper Theorem environment\n- \"give a meaning to the chain rule formula\" this is too vague of a formulation\n- page 2 Can you clarify the following? \"For feed-forward functions, our notations imply that Ar (x) = Wr is constant with respect to x, so there is no difficulty.\"\n- page 2 Can you clearly point toward the difficulties? \"Nonsmooth autodiff with respect to the weights and biases leads to similar difficulties\"\n- The first example is not very convincing, the authors themselves admit. I would remove it for clarity\n- Section 2.3, what do you mean by exactly \"chain rule formula (2) has no clear interpretation\"?.\n- The second example is the main motivation of the paper, **this example should be much better polished**. The example is very simple but the current display makes it hard to understand. I would group some of the equations in math mode.\n- Part 3 is very clear until Theorem 1 is included. The hints of the proof of theorem 1 are currently not clear enough for me. This should be entirely re-written (or moved to appendix).\n- Is Section 4 an original contribution? This is very unclear to me.\n- In Section 5, would it be possible to only present in detail the simplified Boustany example? **In order to avoid the 15 Borel pieces in equation 25?**", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Cu5ZLDw1pb", "reviewer": "Reviewer_bcK4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. The scope of the paper is clear. This kind of study is necessary when an entire community (Deep Learning) relies heavily on autodiff.\n\n2. The adaptation of the Murat–Trombetti Theorem gives a natural framework for nonsmooth autodiff.\n\n3. The illustrative PyTorch experiments confirm that widely adopted frameworks implicitly align with the proposed framework. This suggests immediate practical value in understanding “mysterious” derivative behaviors.\n\n4. Because many activation and pooling functions are piecewise linear, the concept of an associated gradient extends naturally to various deep learning components.\n\n**Weaknesses**\n\n1. The current notations make it difficult to distinguish between the input $x$ and trainable parameters $\\theta$. Defining $f(\\theta, x)$ explicitly—as is standard in machine learning with an objective $\\min_{\\theta} f(\\theta, x)$ would help. Equation (2) focuses on $\\nabla_x f$, and it is unclear why gradients with respect to the weights are not shown. In Example 2.1, $w_0$ is treated like a weight, but the authors derive $\\frac{d}{dx}f$; clarifying or showing $\\nabla_{w_0} f$ could tie the example more closely to training. \n\n2. It lacks a comparison with alternative nonsmooth optimization methods (e.g., Clarke subgradients, conservative gradients, generalized subdifferential, etc..). This would clarify how the “associated gradient” concept aligns with or differs from standard approaches to nonsmooth autodiff.\n\n3. While the paper briefly mentions differentiating with respect to network parameters, it does not fully demonstrate how this approach applies to large-scale backpropagation in practical scenarios. More explicit examples or a short discussion illustrating parameter gradients in typical training loops would reinforce its relevance for the machine learning community. \n\n4.  While generally well-written, a few changes could improve the clarity of the manuscript. The proposed changes are outlined below in the requested changes section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a rigorous approach to nonsmooth automatic differentiation by applying the Murat–Trombetti Theorem to define an “associated gradient” for Lipschitz, piecewise-C^1 functions such as ReLU and max-pooling. The authors propose a mathematical framework in which:\n\n1.\tFor Lipschitz, piecewise-C^1 functions, the authors define a surrogate gradient called \"associated gradient\", which remains well-defined at every point of the domain. \n\n2.\tA chain rule formula holds for these associated gradients, thereby resolving apparent contradictions that arise when we compose nonsmooth functions or differentiate them at points of nondifferentiability.\n\n3.\tConcrete examples illustrate how “standard” autodiff can yield paradoxical results—yet those are interpreted via associated gradients in the Murat–Trombetti setting.\n\n4.\tThe authors also demonstrate that nonsmooth autodiff systematically yields one valid “associated gradient” in the sense of the theorem, thus offering a novel explanation for how famous frameworks (PyTorch, Tensorflow, JAX) handle partial derivatives at nondifferentiable points.\n\nFrom a theoretical standpoint, the Stampacchia property, the Rademacher Theorem, and classical results on composition algebras of piecewise-C^1 functions all come together to show that the Murat–Trombetti formalism is both rigorous and practically relevant. Overall, this work provides new clarity on how to manage nonsmooth autodiff.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Using citation commands \\citep when it is necessary and standard punctuation would help the paper read more smoothly.\n\n- A brief discussion on how the “associated gradient” differs from or complements classical subgradient or Clarke gradient methods would clarify its place in nonsmooth optimization.\n\n**Typos**\n\n- Fix mismatches such as “The functions $f_i$ for $i = 1,\\dots,L$\" vs. the use of $f_0$ in other sections. \n- Equation (1) suggests the nonlinear activation functions are $S_r$ for $r=1, \\dots, \\ell$. But Example 2.1 introduces $S_0$.\n- “So the gradient of f is written as a matrice…” --> matrix \n- 1966 Stampacchia (1963). I don't understand the 1966. \n- piecewise $C^1$ --> piecewise-$C^1$\n- researches --> research \n- A space between Murat-Trombetti and Theorem in the openreview abstract.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "xdrY5BLMTI", "reviewer": "Reviewer_rA3G", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "While the paper makes an interesting connection to the associated gradients from Murat and Trombetti, the paper lacks experimental demonstrations to validate the contribution. My perspective is that the connection, while interesting, is not sufficient on it's own without further connecting to the practical relevance of experimentally improving non-smooth autodiff.\n\nThe non-differentiability of ReLU and max operations in neural networks is well-known. In many cases, these non-smooth regions only occur in a measure-zero set that is still subdifferentiable and are therefore not a significant issue during training. The Boustany (2024) paper does a good job at experimentally showing cases where this does not hold, and could maybe provide an experimental starting point for the paper under submission..", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Update [March 15, 2025] --- after the discussion, the revised version addresses all of my original concerns. I have updated my score to say the claims are supported and it is relevant to the TMLR audience.\n\n----\n\nThe paper studies non-smooth autodiff, focusing on differentiating ReLU and max operations arising in neural networks, making a connection to the associated gradient perspective from Murat and Trombetti (2003). S2 provides examples for the ReLU and max functions. S3 summarizes the results from Murat and Trombetti (2003). S4 expands on the algebra of Lipschitz-continuous and C^1 piecewise functions. And S6 revisits the examples.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The work would be strengthened by experimental results showing where the associated gradients add a significant advantage to a standard neural network setting", "broader_impact_concerns": "No concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper provides a theoretical framework for nonsmooth automatic differentiation using the Murat–Trombetti theorem, introducing the notion of an “associated gradient” that remains valid at nondifferentiable points. The reviewers recommended acceptance, highlighting the work’s relevance on understanding how deep learning frameworks handle nonsmooth functions like ReLU and max pooling. Suggestions were made to improve clarity and accessibility for a broader ML audience, the authors have addressed key concerns.
The paper meets TMLR std for correctness and scientific value.
Decision: Accept (poster)