A Theoretical Framework for Inference Learning
Authors: Nicholas Alonso
Beren Millidge
Jeffrey Krichmar
Emre Neftci
Venue: NeurIPS 2022 Accept (2022)
Keywords: Predictive Coding
Backpropagation
Synaptic Plasticity
Local Learning
Inference Learning

Abstract: Backpropagation (BP) is the most successful and widely used algorithm in deep learning. However, the computations required by BP are challenging to reconcile with known neurobiology. This difficulty has stimulated interest in more biologically plausible alternatives to BP. One such algorithm is the inference learning algorithm (IL). IL trains predictive coding models of neural circuits and has achieved equal performance to BP on supervised and auto-associative tasks. In contrast to BP, however, the mathematical foundations of IL are not well-understood. Here, we develop a novel theoretical framework for IL. Our main result is that IL closely approximates an optimization method known as implicit stochastic gradient descent (implicit SGD), which is distinct from the explicit SGD implemented by BP. Our results further show how the standard implementation of IL can be altered to better approximate implicit SGD. Our novel implementation considerably improves the stability of IL across learning rates, which is consistent with our theory, as a key property of implicit SGD is its stability. We provide extensive simulation results that further support our theoretical interpretations and find IL achieves quicker convergence when trained with mini-batch size one while performing competitively with BP for larger mini-batches when combined with Adam.

A Theoretical Framework for Inference Learning
Nick Alonso 1- Beren Millidge", Jeff Krichmar' 1.2 Emre Neftei 12,4
Depaarmeen of Cognitive Science, UC Irvine *Department of Computer Science, UC Irvine
MRC Brain Network Dynamics Unit, University of Oxford
Electrical Engineering and Information Technology. RWTH Aachen, Germany and Peter Grünberg Institute, Forschungszentrum Jülich, Germany
Abstract
Backpropagation (BP) is the most successful and widely used algorithm deep learning. However, the computations required by BP are challenging recon- cile with known neurobiology. This difficulty has stimulated interest in more biologically plausible alternatives BP. One such algorithm is the inference learn- ing algorithm (IL). IL trains predictive coding models of neural circuits and has achieved equal performance BP on supervised and auto-asser associative tasks. con- trast to BP, however, the mathematical foundations of IL are not -ell-undersoood. Here, we develop a novel theoretical framework for II. Our main result that closely approximates an optimization method known implicit stochastic gradient descent (implicit SGD), which is distinct from the explicit SGD implemented by BP. Our results further show how the standard implementation of IL can be altered to better approximate implicit SGD. Our novel implementation considerably improves the stability of IL across learning rates, which is consistent with our theory, a key property of implicit SGD is its stability. We provide extensive simulation results that further support our theoretical interpretations and find IL achieves quicker convergence when trained with mini-batch size one while performing competitively with BP for larger mini-batchon when combined with Adam.
Introduction
Backpropagation (BP) [33]. the most successful and ubiquitously used learning algorithm in deep learning, has long been criticized as being incompatible with known neurobiology [I0 21]. For exam-
ple, unlike the highly interconnected, recurrent circuits in the brain, BP performs credit assignment through a separate feedback stream that does not alter feedforward (FF) signals [10] 21 Addition- ally, standard implementations of BP are largely incompatible with energy efficient, neuromorphic hardware, due, in part, BP's non-local gradient computations. These concerns have motivated neuroscientists and engineers to search for alternatives to BP that use more biologically compatible, local learning rules.
One alternative to BP is the inference learning algorithm (II.) learning algorithm used train predictive coding (PC) models of biological neural circuits (e.g. 4 7 Within neuroscience, PC models have grown in popularity, where some even propose PC may be canonical biological circuit [5 14 IL works by first minimizing an energy function, known as free energy, w.r.t. neuron activities using the recurrent computations involved in predictive coding. At convergence, IL updates weights to further minimize energy. Unlike standard BP. IL uses local learning rules and recurrent circuits instead of separate feedback stream to perform
32
II is the standard
32
39
"Conrespondence to ailoosoo@@ui.eeem
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

credit assigmment. In addition to evidence the brain does predictive coding (and thus may do IL), recent empirical work has found evidence the brain may learn by first inferring target neural activities through recurrent processing then updating synapses to consolidate those neural activities, similar to II. [38. II. has also been used to train deep networks for machine learning tasks. II., for example, achieved comparable classification and recall accuracy to BP trained networks on supervised and auto-associative tasks (e.g. [43 36 35], see also below).
Despite IL's wide use in biological models and success on machine learning tasks, a rigorous formal understanding of IL's optimization properties and their differences from BP is lacking. Providing such a characterization could yield several useful contributions: 1) A basis for novel theory of how optimization and credit assigmment may work in the brain, 2) formal basis for improving. developing, and applying IL to machine learning tasks and neuromorphic hardware.
In this paper, we develop novel theoretical framework for II. that describes its optimization properties and their similarities and differences from BP. More specifically, 1) we expand beyond current literature by deriving a general form of the IL algorithm, which we call Generalized IL (G-IL), 2) We demonstrate our main result, which is that G-IL closely approximates an optimization method known as implicit stochastic gradient descent (implicit SGD), where the approximation closest in the case of mini-batch size 1. Implicit SGD is distinct from explicit SGD (the standard form of SGD used in machine learning and the sort implemented by BP). 3) We provide theoretical guarantees optimization. 4) We identify learning
concerning IL's stability and connection to rule and variable settings needed in order for IL equal implicit SGD and use this result to develop novel implementation of II. called pr-- 5) Finally, we present extensive simulation results that support our theoretical findings, and for the first time show certain performance advantages of II. over BP, such as better stability across learning rates and improved convergence speed with biologically realistic online learning tasks. These results collectively suggest that, not only does IL better fit biological constraints, but it is also mathematically justified and has performance advantages over BP in more biologically realistic training scenarios.
2 Related Works
Inference Learning and Backpropagation Whittington and Bogacz [43] proved that parameter updates performed by the IL algorithm approach those of BP as the global loss approaches zero and optimized activities approach FF activities. This proof also implies, however, that non-zero IL updates essentially never equal those of BP, and suggests IL poorly approximates BP early in training when the loss is large and optimized activities deviate significantly from FF activities. This leaves open the question of whether there is a better description of IL's optimization strategy and the question of how IL is able to minimize the loss in a stable manner early in training. Other work has shown IL. is formally related to variational expectation maximization (25 [2], which is a learning algorithm with convergence guarantees (27]. This analysis is step forward though leaves open the question of how IL relates to BP and the broader framework of baaeed methods that are the backbone of deep learning. The focus of the current work is develop insights into these open questions. More recently, several works have altered IL to more closely approximate SGD and BP These variants significantly change the original II. algorithm, and some of the alterations seem hard to reconcile with neurobiology [26] 371. Alternatively, instead of altering IL to better fit SGD and BP, we show the standard implementation of IL already closely approximates a working optimization method known as implicit SGD, and we identify the variable settings under which the standard implementation of IL better approximates implicit SGD.
26 37
Similar Algorithms Algorithms similar to II. have been proposed. For example, the alternating minimization algorithm [9] which was developed independently of the predictive coding and IL framework from neuroscience. updates weights to minimize local prediction errors similar to IL (see equation 4 below). However, [9] do not connect their algorithm to implicit SGD, A more recent variant of this algorithm by Qiao et al. (31) show some formal links to proximal operators, an optimization process related to implicit SGD [30] (see appendix B.I.. However, they do not interpret the weight updates as performing the proximal update or implicit SGD as we do here with IL. Proximal operators have also recently been incorporated into learning algorithms for deep networks. Frerix et al. [12] developed a BP, proximal algorithm hybrid. Lau et al. [17] developed an algorithm that combined block coordinate descent with the proximal operator. Other works use proximal operators to reduce noise or perform other tasks, then BP is used to to update at convergence
2

---

All of these algorithms,, however, are distinct from IL, as they use BP or other non-local gradient information to compute weight updates local targets.
23 45
Target Propagation Finally, II. has some relation to target propagation algorithms, in which local target activities computed and used compute local error gradients update weights. Target propagation [5] and difference target propagation [19] utilize approximate Gauss-Newton updates on activities create local targets [24 6]. while other algorithms compute targets using gradient updates on activities [8] 29
3 16. These algorithms have important differences from IL, e.g., they use different learning rules than IL, which we discuss further below. In sum, previous works have not developed the same mathematical interpretation of IL that we do here and have not developed the same mathematical descriptions of the differences between IL and BP. Further, none of the works mentioned above produced the same empirical findings we do in our simulations below.
3 Background
3.1 Notation
Term
Description
W
Weight Matrix, pre-synaptic layer Feedforward Activity layer n, hn = Wn Wf
hn
hn
Optimized/Target Activity layer n
Ahn
Optimized Activity Change layer n, Ahn = hn - hn
Local Prediction layer n. Pr. W (n-1f((nn1)
Pn
Local Error, layer n, either e,2 hn hn en Pnn Pr
en
Table Notation
Notation describing multi- luyli- feed-forward (FF) network (MLP) is summarized in table To simplify notation, we assume that the bias is stored an extra column on each weight matrix W,, and we assume a is concatenated to the end of the pre-synaptic activity vector (h,, or hn depending). All the following results should hold with and without biases.
3.2 Generalized Backpropagation
Consider a set of parameters g(b) at training iteration b with global loss function An explicit at SGD iteration subtracts the gradient of the loss from the parameters: (bb-1) ((b) a Sp with step size BP 34 is the common algorithm for implementing SGD in deep networks. Here, to allow for easier comparison to IL, we consider an alternative implementation of SGD in deep networks that involves computing local target activities at hidden and output layers, similar [18]. Under this regime, each weight matrix W,, is updated using the gradient local loss Ilhh+1 - h 1| = $lle*+1ll" Here hn+ is the local target, while hn+1 is the FF activity. The target at the output layer computed hN hN computed
3
29
h
1
aL. At hidden layer n, the local target is Thx
OL 04 hn Oh hn = ha Ohn
It can be shown hn
8lat1 Sh.
AL by first computing hw-1: hn 8hn
ØNN
Øl θhN hx EN
ŌhN
hN-1 1N-1
bn -1
0hx-1
Oh 0hx-1
Øhx-1
(2)
0L
OhN
aL
= =bx-1 -hh
hw)
hN-1
OhN
ōhN-1
Ōhx-1
This same process can applied recursively to show the remaining hn are also global loss gradient steps over hn. See appendix of (3] for similar proof.
3

---

Updates to weights W, also use the gradient of the local loss such that AW, = -a aw. If we use the result from the above derivation that عhا global loss gradient step over the weights:
we can see this update is equivalent to at
8hn."
0ln+1 0h a OL 0h
OL afn-1f(hm)")
AW, x2 -a 0h oWn
(3)
0hx+1 oWn aw,
This local target formulation of SGD similar other target propagation (TP) 5 19 algorithms. which use essentially the same learning rule but compute targets slightly differently. For example, a well-known variant TP is difference target propagation (DTP) [9)] which computes local targets by approximating Gauss-Newton update on activities rather than gradient updates (24] 6
hn RJ hn - J N. where JN is the pseudo-inverse of the Jacobian of the forward network from layer to N and eN = y hx- Recent attempts to improve DTP generally do so by making DTP updates more similar to SGD updates (e.g. [24 I1]
Additionally, variants of BP algorithms, like random feedback alignment (20] and sign symmetric feedback alignment [44], approximate SGD. Since all these methods approximate the (explicit) SGD update over weights and use similar learning rules, we consolidate them all under the heading of generalized BP (G-BP). The target-based formulation of G-BP is shown algorithm
3.3 Generalized Inference Learning
We now present general description of the IL algorithm, which call Generalized Inference Learning (G-IL), A summary the algorithm is presented in algorithm] 2 At each layer n of an MLP, G-IL stores two variables: hn and Pn- At initialization these variables are set to feed-forward activity values: hn = Pr = hn. Then hn are altered over time to minimize the energy function F:
N
N-1
decay 1
F= P= hx))
hhn
Pm 1m + Σ
| 11 2
In
mm
n=1
where L is the global loss, 1 11hm 2 are local losses, aecay il 2 is an optional regu- larization term, and "y are positive scalar weighting terms. The prediction pn. is computed as Pn = W, W/-i(((m-1)) (Note Pn may also be computed as pn. = ((Wn-1hm-)) Our theoretical results use the first formulation, but we find little difference in performance between the two in practice. See simulations below.) G-IL first minimizes F w.r.t. to activities h (inference phase). While optimizing h, predictions also change since Pa+1 = Wn (hr)) After argming F is estimated, F is again minimized, now w.r.t. the weight, i.c. argminw F. Typical IL algorithms perform the inference phase with either the LMS rule (below) or Adam optimizers to update weights (e.g. (32] 43). G-IL, the other hand, is general w.r.t. the optimization processes that approximate argminw F and argming F. We consider two weight updates in our analysis and simulations below. First, is local error gradient update, which is equivalent to the least-mean squares (LMS) rule:
SGD to
0lp+1 Wn + af argminw, F W + AWn = W,, - OW,
(5)
where Cn+1 hat1 - Pn-1 and local loss 4x+1 = bllent1l12. The LMS rule does not solve argming F but only approximates it. The next rule, which is equivalent to the normalized least mean squared rule (NLMS) 28 is
argminn, F= WW=W. AW, = W. + 2 (ho))
(6)
The NLMS rule performs a gradient update but with an inverse squared & norm of the pre-synaptic activities as its step size instead of a saapee-prrameter The NLMS rule is the minimum-nomm solution argminw! F' in the case where we use mini-batch size (see proposition B.I.. The LMS rule closely approximates the NLMS update in the sense the two are proportional. Comparing these rules to the update rule for G-BP (equation 33 we see the main difference is the pre-synaptic term used in the rule: BP uses FF activity hn while uses optimized/target activity hn This difference leads to significantly different stability properties between the algorithms, note below.

---

Algorithm 1: Generalized BP
begin
Algorithm 2: Generalized IL
// Feedforward Pass
begin
hg xxh))
// Feedforvard Pass
for n = to N - do
ho + x:(b) for to N do Pu+1, t (h)(h,) end
An+11 Wn [
end
hy f- -y)
// Compute Local Targets for to N do h1 h Sh.
1/ Compute Local Targets (h,,...hx) argmin; F 1/ Update Weight Matrices (Wo. WN-1) x argminw F, Eqn. 4,5
end
// Update Weight Matrices
end
Eqn.
end
4 Theoretical Results
4.1 Implicit Gradient Descent
Let the set of MLP weight parameters ((0) (W
( N-14 where the input data is x((), the output target is (0), and b is the current training iteration. The explicit SGD update uses the loss gradient produced by the current parameters:
((++1) 0 (T((M(), x000) ,0()),((1)))
00
where step size and L is the loss measure. This update explicit because the gradient can be readily computed given g(6), ..(6), y()), The implicit SGD update uses the loss gradient of the parameters at the next training iteration:
((b-1) g(b) aL(O(++1), (1000(11,,,,))1)))))))))))))))))
00(++)
This is an implicit update because (1++1) shows up on both sides of the equation. Unlike explicit SGD, ((++1) cannot be readily computed using available quantities. However, the implicit gradient update is equivalent the solution of the following optimization problem (see appendix equation 14
(00+1) = argming (100, y ()) +
1 10 000011)
2a
This update is equivalent to the proximal operator/update 30
This proximal update changes parameters (+6) in way that minimizes the loss L and the magnitude of the update, which helps keep g(b+1) in the proximity g(b), (For more details on the proximal update and its relation implicit SGD see appendix B.I.
4.2
Main Results
We now present our main result, which to show G-IL is equivalent to implicit SGD in certain limits that are approximated well in practice. We focus the case where single data-points (mini-batch size 1) are presented each training iteration. This scenario is more biologically realistic than mini-batching and is the case where IL best approximates implicit SGD. We discuss the case of mini-batching in appendix B.3 We first define a kind of algorithm that minimizes the proximal loss (equation 9 w.r.t. neuron activities and show G-IL approaches this algorithm in certain limits.
Definition 4.1. Proximal Inference Learning (II-prox). An algorithm identical to G-IL (algorithh
2 except activities are optimized according to argming prox = argmin, (I((0)) + 1 -|00 ((0) and the NLMS rule is used to update weights.
2a
5

---

Initialize
Wo
W.
W,
Implicit SGD
LdO, 8
go+4 anggenng (L(8) 3el 800 10))
WG
W
W
Inference
Inference Phase
Phasc
Le y)
We
argmin or oming (L(0") + 11100" 010
W2
L y)
Weight Update
Weight
Wi
Update
W
W aryunim, F
W
OWN
Figure 1: A depiction of IL. Target activities are initialized to FF activities h then updated to minimize energy F during the inference phase. Afterward, weights updated to further minimize F. Weight updates "align' with h activities so h become the new FF activities given the same data point. h are activities that will both improve the loss L and minimize || This process is equivalent to minimizing the proximal objective and approximates implicit SGD.
Here b is the current training iteration and 0 are the parameters updated with the NLMS rule. IL-prox the same G-IL except during the inference phase, minimizes the proximal loss w.r.t. activities h instead the energy F. Intuitively, minimizing the proximal loss w.r.t. activities will result in h that yields small loss after weights are updated and small weight update norms. Weight update norm 2 10 g(6) || is known during the inference phase because the NLMS learning rule is used and known explicitly:
N Σ 75
N
2 110 0000
2 AWW, 1 2 2 (lane
where O'n = (h, Thus, can be minimized w.r.t. hn. As we explain in the appendix (see
lemma B.1 and proposition B.I.. the loss T.(0*) is also explicitly known during the inference phase and can be optimized w.r.t. hn.
Now, let a 'global' learning rate hyper-parameter, and a, be layer-wise learning rates used in the actual weight update AWn-
Theorem 4.1. Let an = [hm))) and assume mini-batch size the limit where Inn
deccy +
||e++1| (en+111(11 2) IN + aN-1, and 7/n - for all n < N, it is the case that argming
argming L( Lall ||12 Hence, in these limits G-IL equivalent IL-prox.
The proof can be found in supplementary materials theorem B.1 This theorem states that when using the NLMS rule, G-IL is increasingly similar to IL--prxx, as the y weighting terms in the free energy (equation 4 approach the scalar values specified in the above theorem. In these limits, argmin, pror argming F and G-IL equivalent to IL-prox. The intuitive explanation theorem goes follows: minimizing the proximal loss (equation requires minimizing L w.r.t. H" while also minimizing the update norm 2n 1 B(b) 1|2. The inference phase of II. minimizes w.r.t. 8* by initializing output layer activity hy FF output hy then shifting toward global target y. Upon weight update this will yield 0* that produces smaller loss L. See figure [.]] for visualization. The magnitude of weight update ||en=1 (h.,) intuitively depends magnitude of local prediction error 6x+1- is a sum the magnitudes errors (see equation minimizing F minimizes the magnitudes of errors and consequently weight update norms Za | 1100~000) 1(0) Importantly, the settings under which theorem 4.1 holds can be computed exactly in practice and are of reasonable magnitude that is easily approximated in practice (e.g.. they do not approach 0 and typically approach positive scalars). This is opposed to the limits under which
this
the

---

Init.
Full Clamp
Rand. Init.
Rand.
Full Clamp
600
600
IL-5GD
IL-prox
2
400
400
((0)
1
200
200
h(nitiiil)
hiFinal)
b(initiaa)
hiFinal)
Minitial
hitinnal
hünitial)
i(Final)
Figure 2: We measure the proximal objective during the inference phase, where F is minimized w.r.t activities in a two hidden layer MLP with ReLU activations trained on MNIST. We test under four different initializations of h. First, all are initialized to (FF Init.). Second, hidden layer are initialized to h, while output layer hN = y (FF + Full Clamp). For both of these initializations, the proximal loss is reduced near zero for both the IL-prox and IL-SGD variants IL (defined below). We next initialized h randomly (Rand Init. and Rand + Full Clamp). Both networks reduce the proximal loss, but nowhere near the global minima, which may help explain why the å needs to be initialized to FF activities when F is minimized with SGD. These results support theorem 4.1 which states that minimizing F w.r.t. h also minimizes the proximal objective.
IL approximates BP/explicit SGD, which is the limit where hn hn and thus AW, 0 condition that does not generally occur in practice.
43
Theorem 4.2. Let (b)) be a set of MLP parameters at training iteration Let 0 argmingL(0) + 1.2 00 102 Let g(b+1) 4.1 and 0(b+2) the parameters updated by G-IL under values in theorem 4.1 Assume mini-batch size 1. Under this assumption, it is the case g(b+1) -((++) IL-prox = =0/1 (bb+1)
IL--prox be the parameters updated by IL-prox (see def.
This theorem states that the IL-prox update is equivalent to the proximal update (equation 9 and thus implicit SGD. follows that, under the y settings theorem 4. the G-IL. update is also equivalent to the proximal update/implicit gradient update.
Further theoretical results developed in the appendix. A novel closed form description of IL targets is developed in C.5 The closed form description shows local targets approximate a regularized Nauss-Newton update FF activities (see section C). We call this closed form approximation IL-GN. We study the stability of IL-GN when using the LMS learning rule, and compare to an analogous BP network, which uses Gauss-Newton updates (BP-GN). We show that IL-GN weight updates push output layer activities down their loss gradients for any positive learning rate, i.e., for any positive learning rate (1 - f
ør. where j a positive scalar (theorem D.1. We show BP-GN only (
has this property for a small range of learning rates (theorem D.2.. We develop a partial explanation of why IL is more stable than BP that is based on differences in their learning rule (for details see section E}. These results suggest that the differences the way IL and BP compute and use local targets in their weight updates can explain much of their differences in stability across learning rate. The II. learning rule and target computations are advantageous over that of G-BP in this sense.
Experiments
In this section, we compare the performance of BP based algorithms to IL algorithms. BP algorithms compute and use explicit gradients to update weights, while IL algorithms compute and use approxi- mate implicit gradients to perform updates. We test BP models that either perform simple gradient Adam optimizers (BP-Adam). IL-SGD is simple implementation of uses the LMS rule (local error gradient update)
descent (BP-SGD)
II. that is similar to implementations of 32
4
to update weights and optimizes h using SGD to near convergence (25 update steps). IL-Prox is our novel algorithm that is based on definition 4.1 IL-prox uses the NLMS rule update weights. Following theorem[4.] the learning rate is only used to determine how much the output layer activ-
ities hy are pushed toward the target y. As - co, hy y and 0, hy - hy. IL-Prox
7

---

Reconstruction BCE (mean=istd.)
Data
BPD
BP-Adam
IL-SGD
Ipox
Ip-o fast
F-NNIST
.303
.282
.306
.286
.286
CIFAR-10
.631
.622
.652
.611
.611
Table 3: Best test reconstruction loss after one epoch of training, mini-batch size 1. BCE was averaged
across pixels and data-points. Top two scores highlighted in bold. All standard deviations S .002.
optimizes h using SGD to near convergence (25 update steps). -r-P Fast is the same as the IL-Prox algorithm except it truncates the optimization of l to only 12 iterations. IL-Adam computes the weight gradients using IL-SGD, then uses Adam optimizers to update weights. -L-prox Adam computes the normalized weight gradient using IL-prox and uses Adam optimizers to update weights. All simulations average results over least seeds of each model type. All models use ReLU activations at hidden layers. Softmax is used at output layer for classification, while sigmoid is used on the autoencoder task.
Mini-batch Training Previous work has shown that IL algorithms can achieve the same accuracy as BP on small scale data sets (e.g.. MNIST and Fashion-MNIST),, in the case where Adam optimizers and medium sized mini-batches (e.g. 64) are used [3 2]. Here we train our BP and IL algo- rithms on CIFAR-I 10 with mini-batch size 64. Results are summarized ta- b Consistent with previous results, IL algorithms, including our novel IL-prox achieve near equal accuracy as BP-SGD and BP-Adam when us- ing Adam optimizers. However, un- like previous studies, we also test IL- SGD and IL-p algorithms without Adam, and find they do not perform as well as BP algorithms in this scenario. IL algorithms without Adam tend converge more quickly but to shallower local minima. Adam, with its momentum and adaptive learning rate, may help prevent II. algorithms from getting caught in shallow local minima (see discussion).
CIFAR-10 Test Accuracy (mean=std.))
Model
m-batch size=1
m-batch size=64
BP-SGD
36.834(+.4778
57.044 (+.144)
IL-SGD
43.894 (+.371)
46.924( +2248))
IL--rrxx
42.060(+.411
19.844(#..3.
IL-prox Fast
42.984 (+.530)
46.094(1.228)
BP-Adam
42.102(+.770)
56.466 (+.228)
IL-Adam
40.724 (6.0
54.066( 1900))
IL-prox + Adam
42.802 1900)
55.572(+.172)
Table 2: Best test accuracy CIFAR-10 supervised task with mini-batch size after 50,000 iterations (I epoch) and with mini-bated size 64 with 77000 iterations.
Online Learning We find IL algorithms tend to converge significantly quicker and to more similar loss/accuracies as BP in a more biologically realistic, online learning scenario, where single data- point is presented each training iteration and each data point is seen only once. This especially apparent with CIFAR-10, 10, shown in tables 2 and 3 and figure 3 IL-SGD and IL-prox algorithms perform similar to and even slightly better than BP-Adam, despite not using momentum or stored, parameter-wis adaptive learning rates like Adam. As far significant speedups were also seen on MNIST and Fashion-MNIST classification tasks (see figure 6). On autoencoder tasks, IL-prox algorithms, in particular, decreased loss significantly quicker than BP-SGD and even slightly quicker than BP-Adam (figure 3).
know, this is novel result. Less
Stability Test: CIFAR-10 Test Accuracy
Model
Ir=01
Ir=.1
Ir=1
Ir=2.5
Ir=10
Ir=100
BP-SGD
35.48(=1.66
IL-SGD
41.66(+1.66
(6.268(+.271)
BP-prox
38.39(=2.91)
23.978(+1.57)
12.89(+1.1)
12.10(=1.61)
(2.21(+11611)
IL-prox
34.97(#.67)
33.47(+2.56)
37.38 (1.46)
37.1
37.01(+1.01) 37.64(t.74)
Table 4: Accuracies after 50,000 training iterations CIFAR-10, mini-batch size 1. Fully connected
networks with layer sizes 17723x1104-111
8

---

F-MNIST Accuracy
F-MNIST Reconst, Loss
CIFAR-10 Accuracy
CIFAR-10 Reconst. Loss
0.50
80
BP-SGD
40
0.45
Test Accurasy (%)
60
II-SGD
Test Accuracy
Il-prox
Test DCL
0.66
II prox Fast
0.35
20-
0.64
20
0.62
0.30
0
2000
4000
20000
40000
20000
40000
20000
40000
Iteration
Eeration
Iteration
Reration
Figure 3: Training runs for mini- size (online) scenario, for classification and autoencoder tasks. MLP size 784-2x500-10 and autoencoder dimension 784-256-100 100-2667847 are trained F-MNIST for 50000+ iterations. MLP size 3072-3x 1024-10 and autoencoder 3072-1024-500 100- 500-1024-3072 trained on CIFAR-10 for 50000+ iterations. The first 4000 training iterations shown for F-MNIST on left. Full training run are shown for other simulations in right three plots.
Learning Under Data Constraints We also find that IL algorithms can converge to better accuracy than BP highly data constrained scenarios. We trained IL algorithms using only 10, 100, 500 data points from each category on F-MNIST and CIFAR-10, with mini-batch size 100. II algorithms achieved better accuracies than BP-SGD the 10 datapoint scenario on F-MNIST and CIFAR, and better accuracy on the 100 data point scenario on CIFAR, while performances evened out in the 500 data point scenario (see appendix table for details). IL-prox also tended to converge significantly quicker than other algorithms on Cifar- TO (figure 4
Stability Test Explicit SGD is well-known to be highly sensitive to learning rate. Implicit SGD, the other hand, is highly insensitive to learning rate and is unconditionally stable, i.e., is able to discount the loss in stable manner for nearly any positive learning rate [4]]. We compare performance of BP-SGD, II-SGD, and Ilr- on MNIST 5 and CIFAR-10 (table classification task across different learning rates. As control, we train hybrid of BP and II--prox (BP-prox). BP-prox, like IL-p, uses the learning rate only to adjust the target at the output layer and uses the NLMS rule to update weights. Unlike IL-prox, BP-prox computes local targets using the G-BP algorithm rather than the IL algorithm. Results can be found in table 5 and 4 Blank entries are those with accuracies below 12% IL-SGD more stable than BP-SGD, and IL- IL-prox is more stable than BP-prox. This suggests that the way IL computes and uses targets contributes to stability. Additionally, IL-p and -r-x algorithms are more stable than SGD algorithms, showing that using the normalized gradient to update weights and a to adjust the output layer target (rather than scale weight updates) also contributes to stability.
F-MNIST (n=10)
F-MNIST (n=100)
F-MNIST (n=500)
Test Test Accuracy (%)
80
70
80
BP-SGD
70
75
BP-Adam
IL-SGD
60
70
IL-prox
55
60
65
500
1000
1500
2000
4000
5000
10000
15000 20000
CIFAR- 10 (n=10)
CIFAR-10 (n=100)
CIFAR 10 (n=500)
Test Accuracy (%))
25
30
40
20
15
20
20
10
250
500
750
1000
2000
4000
10000
20000
lteraaion
lteration
Iteration
Figure 4: Training runs for subset of the algorithms the data constrained scenarios. Each model trained on n data points from each category.

---

6 Limitations
Our theoretical results show that algorithms closely approximate implicit SGD in the case where only single data-point (i.e., mini-batch size 1) is presented each training iteration. This approxima- tion is looser in the case where large bini-batches are used (see appendix [B.5). This no problem from a biological point of view, since the brain does not train with large mini-batches. Additionally, when Adam optimizers are used IL still performs similarly to BP. However, it may still be desirable to further study the optimization properties of IL with mini- to improve performance on machine learning tasks where mini-batches are typically used.
7 Discussion
In this paper, found that IL closely approximates implicit SGD, which is distinct from the explicit SGD performed by BP. This theoretical result, along with results that help to characterize how and BP behave differently, suggest hypothesis, which as far we know is novel: learning in the brain is more similar to implicit gradient descent than explicit gradient descent. This hypothesis would imply the learning characteristics of the brain behave differently from BP and explicit SGD, more generally. More empirical work will be needed to test this theory. However, recent evidence directly supports the idea the brain does something similar IL (38]. Evidence that neural circuits perform predictive coding provides further, though more indirect, evidence for the idea.
We also found that IL often converged more quickly than BP and achieved similar test losses and accuracies in online and data constrained scenarios. However, in large mini-batch scenarios without data constraints and without the use of Adam optimizers, IL sometimes converged to worse accuracies than BP classification tasks. With the use of Adam, IL's performance more closely matched that of BP. One possible explanation of this empirical finding may be related to the fact that II., and implicit SGD more generally, tend to take smaller steps over parameters and a more direct path toward local minima than BP (e.g., see figures and table D.. More direct paths to minima often lead to faster convergence. Smaller update magnitudes may also act as kind of built-in regularization that aids in difficult, noisy learning scenarios, such as small mini-batch and data constrained scenarios. In cases where regularization is less advantageous (e.g.. no data constraints and large mini-batches), IL's small update magnitudes may provide less of an advantage, and its more direct optimization paths may push parameters into shallow, nearby minima. Adam optimizers, with their momentum and adaptive learning rates, may help push parameters out of shallow minima. More work will be needed fully characterize and explain these behavioral differences, but we believe the work done here provides a good formal basis for doing so.
These results further suggest IL may have advantages over BP in more biologically realistic machine learning tasks, such as lifelong, online learning scenarios, where streams of data are received by the neural network, a single datapoint at a time, and the model must adapt quickly to changes in the environment and minimize the magnitude of weight updates prevent catastrophic forgetting. This possibility is especially interesting given that, unlike BP, IL's weight updates are both local in space and time and may thus be more compatible with energy efficient neuromorphic hardware, which is well suited for autonomous, embedded systems that run on batteries and receive data streams through sensors, similar the brain.
8 Broader Impacts
This work is largely theoretical. As such, we do not see any potential negative societal impacts.
9 Acknowledgements
This work was partially supported by the National Science Foundation (NSF) under grant 1652159.
References
[1] Rick Adams, Stewart Shipp, and Karl J Friston. Predictions not commands: active inference in the motor system. Brain Structure and Function, 218(3)6611-64) 2013.
10

---

[2] Nicholas Alonso and Emre Neftci. Tightening the biological constraints on gradient- based predictive coding. In International Conference on Neuromorphic Systems 2021, pages 1 1-9, 2021.
[3] Sergey Bartunov, Adam Santoro, Blake A Richards, Luke Marris, Geoffrey E Hinton, and Timothy Lillicrap. Assessing the scalability of biologically deep learning algorithms and architectures. arXiv preprint arXiv: 18Xivv887700458777 2018.
[4] Andre M Bastos, W Martin Usrey, Rick A Adams, George R Mangun, Pascal Fries, and Karl J Friston. Canonical microcireuits for predictive coding. Neuron, 76(4):695-7) -111999--11,--- 2012.
[5] Yoshua Bengio. How auto-encoders could provide credit assignment deep networks via target propagation. arXiv preprint arXiv: /407. 7906, 2014.
[6] Yoshua Bengio. Deriving differential target propagation from iterating approximate inverses. arXiv preprint urXiv:2007. 15139, 2020.
[7] Rafal Bogacz. A tutorial on the free-energy framework for modelling perception and learning. Journal of mathematical psychology, 76:198 2017 2017.
[8] Hyeonwoo Cho, Chang Woo Lee, and Sang Woo Kim. Derivation of a new normalized least mean squares algorithm with modified minimization criterion. Signal Processing, 89(4):692- 695, 2009.
[9] Anna Choromanska, Benjamin Cowen, Sadhana Kumaravel, Ronny Luss, Mattia Rigotti, Irina Rish, Paolo Diachille, Viatcheslav Gurev, Brian Kingsbury, Ravi Tejwani, et Beyond backprop: Online alternating minimization with auxiliary variables. In International Conference on Machine Learning, pages 1193-1202. PMLR, 2019.
[10] Francis Crick. The recent excitement about neural networks. Nature, 337(6203):129-1133 1989.
[11] Maxence Emoult, Fabrice Normandin, Abhinav Moudgil, Sean Spinney, Eugene Belilovsky, Irina Rish, Blake Richards, and Yoshua Bengio. Towards scaling difference target propagation by learning backprop targets. arXiv preprint arXiv:2201.13415 2022.
[12] Thomas Frerix, Thomas Möllenhoff, Michael Moeller, and Daniel Cremers. Proximal back- propagation. arXiv preprint (oXiv:/706.046:3 2017.
[13) Karl Friston. A theory of cortical responses. Philosophical transactions of the Royal Society B: Biological sciences, 360(1456):815-836,, 2005.
[14] Karl Friston. The free-energy principle: a unified brain theory? Nature reviews neuroscience, 11(2):127-138, 2010.
[15] Karl Friston and Stefan Kiehel. Predictive coding under the free-energy principle. Philosophical transactions of the Royal Sociery B: Biological sciences, (64(1121):1211-222 2009.
[16] Axel Laborieux, Maxence Emouut, Benjamin Scellier, Yoshua Bengio, Julie Grollier, and Damien Querlioz. Scaling equilibrium propagation to deep convnets by drastically reducing its gradient estimator bias. Frontiers neuroscience, 15:129, 2021.
[17] Tim Tsz-Kit Lau, Jinshan Zeng, Baoyuan Wu, and Yuan Yao. proximal block coordinate descent algorithm for deep neural network training, arXiv preprini 1803, 883009982, 2018.
[18] Yann LeCun, D Touresky, G Hinton, and T Sejnowski. A theoretical framework for back- propagation. In Proceedings of the 1988 connectionist models summer school, volume 1, pages 21-28, 1988.
[19] Dyug Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propaga- tion. In Joint european conference on machine learning and knowledge discovery in databases, pages 498-515. Springer, 2015.
[20] Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synap- tie feedback weights support error backpropagation for deep learning. Nature communications, 7(1):1-10, 2016.
11

---

[21] Timothy P Lillicrap, Adam Santoro, Luke Marris, Colin Akerman, and Geoffrey Hinton. Backpropagation and the brain. Nature Reviews Neuroscience. 21(6):335-346, 2020.
[22] Joseph Marino. Predictive coding, variational autoencoders, and biological connections. Neural Computation, 34(1): 2022.
[23] Tim Meinhardt, Michael Moller, Caner Hazirbas, and Daniel Cremers. Learning proximal operators: Using denoising networks for regularizing inverse imaging problems. In Proceedings of the IEEE International Conference on Computer Vision, pages 1781-1790, 2017.
[24] Alexander Meulemans, Francesco S Carzaniga, Johan AK Suykens, João Sacramento, and Benjamin F Grewe. A theoretical framework for target propagation. arXiv preprint arXiv:2006. 14331, 2020.
[25] Beren Millidge, Anil Seth, and Christopher L Buckley, Predictive coding: theoretical and experimental review. arXiv preprint arXiv:2107.12979 2021.
[26] Beren Millidge, Alexander Tschantz, and Christopher L. Buckley. Predictive coding approxi- mates backprop along arbitrary computation graphs. arXiv preprint urXiv:2006.04182 2020.
[27] Todd K Moon. The expectationmaxximizaaoonn algorithm. IEEE Signal processing magazine, 13(6):47-60, 1996.
[28] Jin-Ichi Nagumo and Atsuhiko Noda. A learning method for system identification. Transactions on Automatic Control, 12(3):282 287,278888 1967.
IEEE
[29] Alexander G Ororbia and Ankur Mali. Biologically motivated algorithms for propagating local target representations. In Proceedings of the aaai conference on artificial intelligence, volume 33, pages 4651-4658, 2019.
[30] Neal Parikh and Stephen Boyd. Proximal algorithms. Foundations and Trends in optimization, 13)):127-23,, 2014.
[31] Linbo Qiao, Tao Sun, Hengyue Pan, and Dongsheng Li. Inertial proximal deep learning alternating minimization for efficient neutral network training. In ICASSP 202 2221- IEFE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 3895- 3899. IEEE, 2021.
[32] Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1):79-87, 1999.
2():)999
[33] David E Rumelhart, Richard Durbin, Richard Golden, and Yves Chauvin. Backpropagation: The basic theory. Backpropagation: Theory, architectures and applications, pages 1-34, 1995.
[34] David E Rumelhart, Geoffrey E. Hinton, and Ronald J Williams. Learning representations by back-propase prokkapppggg errors. nature, (236668)) 1986.
[35] Tommaso Salvatori, Luca Pinchetti, Beren Millidge, Yuhang Song, Rafal Bogacz, and Thomas Lukasiewicz. Learning on arbitrary graph topologies via predictive coding. arXiv preprint 1:X11:2001.13/80. 2022.
[36] Tommaso Salvatori, Yuhang Song, Yujian Hong, Lei Sha, Simon Frieder, Zhenghua Xu, Rafal Bogacz, and Thomas Lukasiewicz. Associative memories via predictive coding. Advances in Neural Information Processing Systems, 34, 2021.
[37] Yuhang Song, Thomas Lukasiewicz, Zhenghua Xu, and Rafal Bogacz. Can the brain do backpropagation? exact implementation of backpropagation in predictive coding networks. Advances in neural information processing systems, 33:22566, 2020.
[38] Yuhang Song, Beren Gray Millidge, Tommaso Salvatori, Thomas Lukasiewicz, Zhenghua Xu, and Rafal Bogacz. Inferring neural activity before plasticity: A foundation for learning beyond backpropagation. bioRxiv, 2022.
12

---

[39] Michael W Spratling. Predictive coding as a model of biased competition in visual attention. Vision research, 48(12):1391-1408. 2008
[40] Panagiotis Toulis, Edoardo Airoldi, and Jason Rennie. Statistical analysis of stochastic gradient methods for generalized linear models. In International Conference on Machine Learning, pages 667-675. PMLR, 2014.
[41] Panos Toulis and Edoardo M Airoldi. Implicit stochastic gradient descent for principled estimation with large datasets. ArXiv e-prints, 2014.
[42] Panos Toulis and Edoardo M Airoldi. Asymptotic and finite-sample properties of estimators based on stochastic gradients. The Annals of Statistics, 45(4):169411727, 2017
[43] James CR Whittington and Rafal Bogacz. An approximation of the enror backpropagation algo- rithm in a predictive coding network with local hebbian synaptic plasticity. Neural computation, 29(5):1229-122) 2017.
[44] Will Xiao, Honglin Chen, Qianli Liao, and Tomaso Poggio. algorithms can scale to large datasets. arXiv preprint 1rX1v:1811.03367, 2018.
learning
[45] Dong Yang and Jian Sun. Proximal dehaze-net: A prior learning-based deep network for single image dehazing. In Proceedings of the european conference on computer vision (ECCV), pages 702-717, 2018.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See section
(c) Did you discuss any potential negative societal impacts your work? [Yes] See section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] For all theorems, propositions, and lemmas assumptions were stated in the statement. (b) Did you include complete proofs of all theoretical results? [Yes] All proofs are in appendix
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] This information in supplementary material and link to code
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] This information is reported in supplementary material
(c) Did you report error bars (e.g., with respect to the random seed after running exper- iments multiple times)? [Yes] Standard deviations over multiple seeds are shown in tables and in plots using error bars.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Is included in supplementary material
4. If you are using existing assets (c.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
13

---

(d) Did you discuss whether and how consent was obtained from people whose data you' re using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "jof6b3juoE", "reviewer": "Reviewer_nxGR", "summary": "This paper proposes a general theoretical framework for inference learning. To this end, the authors propose the Generalized Inference Learning (G-IL) algorithm. They are then able to show that G-IL closely approximates the implicit stochastic gradient descent (implicit SGD), which corresponds to the proximal update of the weight parameters and is distinct from the explicit SGD often used by backpropagation. In addition, the authors compare the performance of several algorithms based on backpropagation with those based on inference learning and establish that the latter converges more quickly on smaller mini-batches.", "questions": "While the authors claim that inference learning \"has achieved equal performance to BP on supervised learning and auto-associative tasks\", there is a clear discrepancy between the test accuracy of BP-SGD and IL-SGD with a mini-batch size of 64. While the authors mention this in their results, I was wondering where this non-negligible discrepancy comes from and what its implications for the comparison between inference learning and backpropagation would be.", "limitations": "The main limitation of the paper is that it mainly explores the case where a single data point is present in each mini-batch. The authors bring up this limitation in their work and provide a justification as to why it is not significant from a biological point of view. As such, I believe the authors adequately address the limitations of their work. However, I am not sure whether exploring the properties of larger mini-batch sizes in more detail could still yield some interesting results.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "To the best of my knowledge, this paper offers a novel perspective on inference learning by establishing the connection between G-IL and implicit SGD. The authors do a good job of motivating the similarities between inference learning and biological learning processes. In addition, they provide both theoretical justifications and experimental results for their claims.\n\nAs for the weaknesses, I believe that the authors could explain or provide some ideas for some of the experimental results they observe in more detail (such as IL-SGD performing worse than BP-SGD, but IL-Adam and BP-Adam performing more similarly on CIFAR-10).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "i82DJb1sVg9", "reviewer": "Reviewer_an3X", "summary": "The paper gives a theoretical framework and justification for the biologically plausible \"inference learning\" neural-network training algorithm, a generalization of predictive coding. The paper shows that in the low-N regime, IL closely approximates implicit stochastic gradient descent: it steps in the gradient direction while also staying close to the current weights. Based on this theorem, the paper then improves IL to more closely approximate implicit SGD, which improves the stability and convergence of IL across learning rates and tasks.", "questions": "Have the authors tested against implicit SGD with a backprop-based gradient estimator?", "limitations": "I can't quite see where this paper fits in.  Bioplausible training of artificial neural networks in the small-batch (eg: N=1) regime remains, to my knowledge, not a very common task.  If the idea is that the brain might do something like IL, can the authors point to any experiments specifically picking out IL as distinct from predictive coding in the brain?  If the idea is to train ANNs for tasks, why not just use BP and large batch-sizes?  Have the authors run any experiments in memory-constrained settings showing that IL does better there?\n\nThe authors have addressed these questions to my satisfaction in their response.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* Strong bioplausibility motivation\n* Beats backpropagation in the small-batch setting\n\nWeaknesses:\n* Beaten by backprop in the midsize-batch (N=64) setting", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "aun7VMkESHm", "reviewer": "Reviewer_XkhF", "summary": "This paper proposes an alternative to gradient back propagation in the form of inference learning. This algorithm has the advantage of being more compatible with a biophysical implementation, and the paper aims to develop a theoretical framework for this learning. The expected results are to obtain convergence results for this algorithm and to show the performance compared to classical BP algorithms. The abstract makes a very strong assumption about the result by stating that this algorithm can be superior to the BP algorithm.\n", "questions": "I have several questions about this work.\nFirstly, from equation (1) the cost function used to calculate an optimal prediction is always an L2 norm. How can you justify this, and have you verified this result experimentally by looking at the distribution of errors in learning?\nSecondly, in generalized inference learning, you introduce a decay term in equation (3). Could you justify more accurately the evidence for this finding in the simulation data?\nThird, could you substantiate a comparison of your algorithm with hierarchical predictive coding algorithms that include both a sparseness regularization or a feedback term?\n", "limitations": "Overall, the paper suffers from a major limitation between the conclusions that are put forward in the abstract and the theoretical but also simulation results that are presented in the paper. It would surely have been more correct to make less far-reaching assumptions but fully validated by the theoretical and experimental results.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "A main strength of the paper is to show the state of the art in machine learning algorithms, and in particular to show the comparison between stochastic gradient descent type models and inference learning models. The model is well presented, and the notations are relatively clear. In particular, the main limitation of the back propagation algorithm is well highlighted by showing how to implement a local target estimation algorithm.\n\nA first weakness of the paper is that it relies on details that are supplied in the (40 pages) appendix and that relying on this does not allow an independent evaluation of the results in the paper. Secondly, even if the results seem encouraging, they are still clearly insufficient to show the generality of the algorithm. Indeed, the only benchmark used is that of CIFAR and the results are obtained with results superior to BP only in the very restrictive case of a batch size equal to one. This simulation result does not therefore justify the conclusion made in the summary about the generality of the results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept