Associated Learning: an Alternative to End-to-End Backpropagation that Works on CNN, RNN, and Transformer
Authors: Dennis Y.H. Wu
Dinan Lin
Vincent Chen
Hung-Hsuan Chen
Venue: ICLR 2022 Poster (2022)
Keywords: pipeline training
parallel training
backpropagation
associated learning

Abstract: This paper studies Associate Learning (AL), an alternative methodology to the end-to-end backpropagation (BP).  We introduce the workflow to convert a neural network into a proper structure such that AL can be used to learn the weights for various types of neural networks.  We compared AL and BP on some of the most successful types of neural networks -- Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Transformer.  Experimental results show that AL consistently outperforms BP on various open datasets.  We discuss possible reasons for AL's success and its limitations.

Published as a conference paper at ICLR 2022
ASSOCIATED LEARNING: A METHODOLOGY TO DE- COMPOSE END-TO-END BACKPROPAGATION ON CNN, RNN, AND TRANSFORMER
Dennis Y. Wu! Di-Nan Lin', Vincent F. Chen', Hung-Hsuan Chen?,
Institute of Information Science, Academia Sinica
2 Department of Computer Science, National Central University hibbaiis mibbsiis.sinich sini edu. tw, lindi nan934301 @aaa300@@@mmmmll gmail com Anncent0110811 sew.siiniiii sinica. edu. tw, hhchengg ncu. edu. tw
ABSTRACT
We study associated learning (AL), an alternative methodology to end-to-end backpropagation (BP). We introduce the workflow to convert a neural network into an AL-form network such that AL can be used to learn parameters for various types of neural networks. We compare AL and BP on some of the most success- ful neural networks: convolutional neural networks, recurrent neural networks, and Transformers. Experimental results show that AL consistently outperforms BP on open datasets. We discuss possible reasons for AL's success, its limita- tions, and AL's newly discovered properties. Our implementation is available at https://g wttp::////iubuouooooo Ehub. bbb.:m/m//bbbbb1bb1
INTRODUCTION
Backpropagation (BP) is the keystone of modern deep learning. Although BP is the standard way to learn network parameters, it is far from ideal. Some of the most discussed issues of BP are optimization difficulties (e.g., vanishing gradients and exploding gradients (Hochreiter et al.) 2001)) and training performance (e.g., backward locking (Jaderberg al. (017
It appears that custom network structures may be needed for different types of learning tasks. Among the various forms, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and Transformer networks (along with their extensions, e.g., LSTM (Hochreiter & Schmidhuber) 1997) and VGG (Simonyan & Zisserman, 2015 are particularly useful. These networks have been suc- cessfully applied in fields as varied computer vision, natural language processing, signal process- ing, and others (Goodfellow et al. 2016 Deng & Yu 2014).
This paper studies new learning approach associated learning (AL) alternative to end-to- end backpropagation learning. AL decomposes BP's global end-to-end training strategy into several small local optimization targets such that each layer has an isolated gradient flow. However, since most layers in AL. do not interact with the final loss directly, we would expect AL-training models be less accurate than BP. Surprisingly, the original AL paper compares AL and BP using the CNN network (and its extensions, e.g., VGG) and shows impressive results based on image clas- sification datasets (MNIST, CIFAR-10, and CIFAR-100) (Kao & Chen 2021). We continue this line of study in two ways. First, we discover more interesting properties of AL. Second, we show how to apply AL on different network structures, including VGG (Simonyan & Zisserman 2015), LSTM (Hochreiter & Schmidhuber 1997). and Transformer (Vaswani et al.) (017).. Eventually, we compare the networks learned via AI. and via BP on various tasks (image classification, sentiment analysis, and topic classification) based public datasets (CIFAR-10 10, CIFAR-I00 100, IMDB movie reviews, AG's News, Stanford Sentiment Treebank (SST), and DBpedia). We find that AL consis- tently outperforms BP in most datasets. Additionally, AL requires fewer epochs than BP when using early stopping but still yields excellent accuracy. These results suggest that AL is a strong alternative to BP, AL is effective for various tasks and various network structures.
The rest of the paper is organized as follows. In Section 2 we introduce associated learning and its properties. Section 3 presents the experiments, including comparison of the model accuracies and

---

Published as a conference paper at ICLR 2022
convergence speed of AL and BP VGG, LSTM, and Transformer, generalization test of AL and BP, and an ablation study. rectionnrevveee previous work on backpropagation alternatives and the network structures that may look similar to AL. Finally, conclude our contribution in Section 5
2
ASSOCIATED LEARNING
2.1 AN OVERVIEW OF AL: AL FORM, TRAINING, AND INFERENCE
h
f2
fa
df
$1
82
83
y
8/2
0/s
dcl
00
80
28
Figure 1: A classification neural network with 3 hidden layers.
82
$3
f2
gradiont
81
82
$3
bg
y
t
t/
t3
hn
hss
t2
t
gradiont flow
t/2
MSE
E2
Figure 2: A 3-layer neural network in the AL form required for AL learning. Red arrows: the inference path; green arrows: the objective function; blue arrows: gradient flow; orange arrows: appended functions not in BP; black arrows: preserved functions in BP. We stop the gradient flow from an AL layer to its previous layer to ensure each layer is trained isolation.
To apply AL, must transform the network into different structure, which we call fL below. Instead of learning a function to map a feature vector I to a target y. the AL form performs metric learning by searching for functions to transform x and into latent representations such that the distance between these two latent representations close.
Algorithm shows the procedure to convert neural work into AL form. We Figure [ neural network with hidden layers) as an example to show the process. We can regard functions fi, / fa the encoders convert the input feature vector into latent representation and cl as a classifier to transform the latent representation into a target. When converting this network to its AL form (referring to Figure P we only keep the encoders fi-s and extend the model architecture by adding a bridge function b; and an autoencoder for each layer i. Let s;-1 and f be the inputs of layer (assuming 8g = x and to = y), functions and 9i convert 8,-1 and f.-1 into latent convolu-
representations s; and f,. The function fi can be any type of forwarding block, such tion layer, an LSTM layer, or residual block in ResNet. Each bridge function b, projects 8; (the latent representation of x at layer i) to be close to (the latent representation of y at layer i), i.e., b.(f((;)1)) z (tt-))). Meanwhile, serves the encoder for autoencoder such that h, (g(g(t;-11) the AL-form networks in our experiments, each of the functions g, and h, is constructed by linear transform followed by non-linear activation function (e.g., tanh), and by is linear transformation
not only extracts the latent representation of y. but also
t,-1 (h, is the decoder). For all
2

---

Published as a conference paper at ICLR 2022
of the input vector input matrix. We discuss the number of neurons for the input, output, and hidden layers of the autoencoders in Section
To summarize, minimize the following objective for each layer during training:
-AL= LA - -1,,(((((ss.)))) 9i(44-1)]| (144(99(((()))))) 4-1 -4-1
where Ss denotes the associated loss of layer i, and L indicates the autoencoder loss of layer
Algorithm 2 gives the training algorithm for AL. Since each layer has its objective function (Equa- tion 3) we can parallelly update the parameters of different layers using pipeline to increase the training throughput (details Section 2.2.3).
For inference, if an AL network has L layers, an input goes through ..,....J ST. to generate x's latent representation (sr.), which transformed to y's latent representation (4r.) via br. Next, lr. is converted y via hy, ....... h1. The autocncoders' encoding functions 9.-8 not used during inference. Take Figure as an example, the inference path (fi - f2 - fs - by + h3 + h2 - h1). ). Although it could be unintuitive concatenate unconnected transformation functions as an inference path (i.e., h followed by h; = 1, 3), is valid because h;+1 (1++1) = t R t.
Algorithm Converting a neural network to its AL. form
Input neural network N (f1. f2, ft.), target y E R², feature vector r E R Output an AL network A
1: So 2; 6 y
2: for = to L
3:
5; f((ss1))
4: insert function gr, where 4; = (li-)
5: insert function b,, where 5;
(8:)
adding bridge function
6: insert function hus where t1= (t,)
> adding a decoder for an autoencoder b associated loss and autoencoder loss
7: CAL = MSE(s s, (4) + MSS((t;;,,
8: return A (ai...., at), a; = {frn bis bi, Mi 1i
Algorithm 2 Training AL network
Input an AL network A (a1....,,ap aL), a. = (f,,b, hi training features and targets
(X, Y)
Output fine-tuned AL network A
1: repeat
2:
Sample x, from X.Y.
3:
8o I; y
4:
for = 1 to do
5:
8;
fi(si-1)
6:
t + (i(li-1)
7:
b((ss)
8:
4-1 + (ta)t)
9:
C + MSF((),
associated loss > autoencoder loss
10:
C + MSS
Update the parameter f.. b. 9: according VL Update the parameter of gs, h4 according to VLA
12:
13:
until converges
2.1.1
APPLYING AL TO DIFFERENT NEURAL NETWORK STRUCTURES
This section introduces details on converting some of the most successful neural network structures into their corresponding AL forms. We discuss vanilla CNN and VGG (used the representative models for CNN-based models), LSTM (used as the representative model for RNN-based models), and Transformer. We also discuss how to integrate word embeddings into AL-form network.
3

---

Published as a conference paper at ICLR 2022
The AL form of an CNN architecture (here, vanilla CNN and VGG) uses fi-s to convert input image x into latent representations by convolutions, just like regular CNN or a VGG does. The associated loss at layer i is defined the distance between (s,)) (transforming the flattened feature map at layer into 4 shape) and i; (the latent representation of y).
CNN
RNN The AL form of RNN architecture (here, LSTM and Bi-LSTM) uses the internal state to iteratively process each element in the input sequence I and generates the new internal state, just like a regular RNN does. After reading the entire input sequence z, we define s; the latent representation of x: at layer by the final internal state. Consequently, the associated loss is defined as the distance between (8:) (transforming the final internal state at layer into 4/s shape) and I; (the latent representation of y). Details are shown in Section 6.2
Transformer The AL form of a Transformer encodes the input sequence x into a list of vec- tors, regular Transformer does. We define 8; (the latent representation of by computing the mean-pooling on the encoded vectors. The associated loss is defined as the distance between (s:) (transforming the mean-pooling output into 44's shape) and I; (the latent representations of y)- Details are shown in Section 6.2
Word Embedding Word embeddings are frequently used the input of LSTM or Transformer for NLP tasks. We use mean-pooling to aggregate all token's 's word embeddings
2.2
PROPERTIES OF AL
This section presents three properties of AL that do not exist in BP: forward shortcuts, dynamic layer accumulation, and pipelines.
2.2.1 FORWARD SHORTCUTS
Forward shortcuts enable faster inference. However, we can also leverage "shortcut paths" for faster inference. As shown in Algorithm given an integer (1 < S T.), the bridge function by can serve as shortcut transform 9t to s/y, which should be close to ty when the network is well trained. As a result, we can skip fj. bj. and hy for all j > E to reduce the length of the inference function. In other words, we have multiple inference functions based
single model. When inference time is
critical, we can select shorter inference path, e.g., f1 inference time is unimportant, we can dynamically adjust the model complexity by modifying the number of AL layers used at the inference phase to reduce overfitting underfitting.
81 b1 t hi y. On the other hand, if
2.2.2 DYNAMIC LAYER ACCUMULATION
An AL-form network allows the dynamic creation of new layers during training. Specifically, we initially create an AI-formm network with & AL. layers and train the network based on the strategy introduced in Section 2.1 If the model still underfits, we simply add the (k + 1)-th AL layer on top of the first AL layers. We may choose to fix the parameters the first & AL layers and train only the parameters in the (k 1) AL layer. In contrast, when using standard neural network, adding layers dynamically is more complicated.
2.2.3 PIPELINE
In an AL-form network, the network parameters different AL layers can be trained simultaneously via pipelines because each AL layer i has local objective function. In contrast, it is difficult to simultaneously update the parameters of different layers for a standard BP-based neural network.
Given n training instances in standard neural network with L hidden layers, if we apply BP for training, the time complexity is ((L)) because each instance must complete the entire forward (L+1 transformations) and backward (L+11 transformations) process. Thus, for such an architecture, the time complexity increases linearly with the total number of layers L. However, since each AL layer has its local objective, each layer update its parameters without waiting for the gradient from preceding layers. As result, for an AL-form network with AL layers, using computation units make the parameter update process fully pipelined, explained below.
4

---

every dataset. Published as a conference paper at ICLR 2022
We denote task; as the task of computing both forward and backward propagation for an AL layer i based the j-th instance. If an AL layer finishes task; at time step i, at the next time step + 1, AL layers i can continue performing task? since all parameters in this layer are updated based on lask.. Meanwhile, AL layer i performs lask j-sk,-1- Consequently, training based the first instance requires (()), and each of the following n instances requires extra O(1) time units. The time complexity of one training epoch eventually reduces to ( + L).
EXPERIMENTS
To evaluate how AL works different network architectures, conducted experiments on image classification with vanilla CNN and VGG and on sentiment and topic classification with LSTM and Transformer (using pre-trained word embeddings as input). In addition, we conducted case study on AL's generalization ability, evaluated by fitting AL models and BP models on different portions of noisy labels. Finally, we discuss possible reasons for AL's success based on ablation studies. We report more detailed settings in the Appendix.
3.1 DATASETS
We used four datasets (IMDB Review, AG's News corpus, DBpedia Ontology, and the Stanford Sen- timent Treebank) to compare AL and BP on LSTM and Transformer, and two datasets (CIFAR-10 and Rashion-MNISTT to compare AL and BP on CNN and VGG. Dataset details are in Appendix 6.4
3.2 TEXT CLASSIHCATION (LSTM AND TRANSFORMER)
We compared BP and AL on LSTM and Transformer based the text classification task.
For LSTM, we used a 2-layer bidirectional LSTM with pre-trained GloVe word embeddings (Pen- nington al. 2014) as input. For Transformer, we used 2-layer Transformer encoder, again with pre-trained Glo Ve word embeddings. During training, we recorded the accuracies of the validation set for every epoch and saved that model with the best performance on the validation set for testing. For each model, we repeated the process five times and reported the average accuracy and standard deviation. For fair comparison, we forced the AL parameter counts to not exceed those of the compared standard neural network by reducing AL's hidden dimension size.
Table I: Accuracy (mean +: standard deviation) of neural networks and corresponding AL versions on text classification datasets. Tran denotes Transformer, <X>-AI-full denotes full path inference on AL-form network <X>, and <X>-AL-SCi denotes shorteut inference through bridge function b,41- EMB-AL-SC indicates inference through the shortcut in the embedding layer, i.., after computing mean-pooling the word embeddings, the output is passed to bridge function.
Method LSTM
IMDB
AGNews
SST
DBpedia
86.25 0.63
90.32 1 0.23
82.23 1 0.93
97.34 1 0.11
LL-M-AL-full
86.41 0.61
91.53 0.20
81.33 1 0.31
98.30 0.01
ASTM-AL-SCE
86.16 0.22
91.42 0.17
80.83 + 0.42
98.21 + 0.06
EMB-AL-S
87.80 0.13
91.03 + 0.28
80.57 + 0.81
96.95 + 0.03
Tran
83.45 -033
90.71 1 0.28
76.48 0.55
97.37 0.06
Tran-AL-full
85.65 0.77
91.17 0.43
81.16 1 0.11
97.55 0.06
AL--AL-SCI
86.25 1.34
91.49 0.09
80,58 - 0.67
96.53 - 0.29
EMB-AL-SC
86.98 0.72
91.43
80.12 + 0.57
97.14 0.11
We report the accuracies of L.SM and Transformer trained via BP and AL in Table We also report the accuracies based on the predictions using shorter inference paths in the same Table. Although AL is trained by isolated gradient flows, AL-form LSTMs outperform their standard BP-trained counterparts most datasets. Transformer-AL also outperforms Transformer Also, we observe that standard Transformer easily overfits smaller datasets like SST and IMDB, resulting in poor test accuracies, but Transformer-AL does not seem to have this weakness.
In Table 2 and Table 3 we report the epoch that yielded the best results. We discover that LSTM- AL converges much faster than BP, suggesting that training time will be improved if applying early
5

---

Published as a conference paper at ICLR 2022
Table 2: Accuracy (mean + standard deviation) and epoch (mean + standard deviation) reach the best accuracy using different RNN models and their AL versions on IMDB and AGNews.
Method
IMDB
AGNews
Epoch
Accuracy
Epoch
Accuracy
LSTM
15.4 2.60
86.25 1 0.63
25.4 1 13.42
90.32 1 0.23
LSTM-AL
4 1.22
86.41 + 0.61
2.4 + 0.89
91.53 + 0.20
Tran
1 6.63
83,45 1 0.33
3.75 0.96
90.71 1 0.28
Tran-AL
4.4 0.55
85.65 0.77
20.25 12.39
91.17 0.43
Table 3: Accuracy (mean + standard deviation) and epoch (mean + standard deviation) to reach the best accuracy using different RNN models and their AL versions on SST and DBPedia.
Method
SST
DBpedia
Epoch
Accuracy
Epoch
Accuracy
LSTM
16 2.65
82.33 = 0.93
31.8
5.26
97.34 0.11
LSTM-AL
2.25 - 1.89
81.33 = 0.31
14.6 2.97
98.30 0.01
Tran
6 2.1
76.48 1 0.55
6.75 + 1.71
97.37 1 0.06
Tran-AL
9.67 = 0.58
81.16 = 0.11
13.75 2.22
97.55 0.06
stopping. For the Transformer-AL, although the best validation and test accuracy come roughly the 20th epoch on AGNews, Transformer-AL reaches high validation and test accuracy within the first 5 epochs. Since we report only the epoch that reaches the best validation and test accuracies, the required epochs look large. Particularly, Transformer-AL achieved 99% of the best validation set accuracy before the 5th epoch. More details can be found in Appendix 6.7
3.3 IMAGE CLASSIFICATION (VANILLA CNN AND VGG)
We also reproduced the experiments in Kao & Chen (2021) determine whether AL-form CNNs also converge faster. In image classification, we report performance on the CIFAR- 10 and Fashion- MNIST benchmarks. We selected FashionMNIST instead of MNIST because MNIST is likely too simple to show the power of different models: CIFAR- more challenging than FashionMNIST since it contains three channels, whereas FNSHionMNIST is limited grayscale images. Table 4 shows the results: all AL models converge faster than BP. Figure 6 in Appendix 6.66 shows the re- lationship between epochs and test accuracies on CNN and CNN-AL, Also, AL models perform slightly better in terms of accuracy, consistent with the results demonstrated in Kao & Chen 2021
Table 4:
Accuracy (mean + standard deviation) and epoch (mean + standard deviation) to reach the best accuracy using different CNN models and their AL versions.
CIFAR-10
Fashion MNIST
Epoch
Accuracy
Epoch
Accuracy
CNN
36.8 = 4.60
82.64 = 0.61
58.4
9.32
91.57 = 0.11
CNN-AL
33.6 4.97
85.16 : 0.11
1.67
92.07 + 0.18
VGG
47.2 7.82
92.14
= 0.17
32 4.34
94.18 + 0.04
VGG-AL
35.6 = 2.19
92.48 = 0.15
25 i 3.83
93.80 + 0.08
3.4 GENERALIZATION
To assess the generalizability of model, a common approach is to empirically compare the errors on the training data and test data. Garg et al. (222)) show another way to evaluate generalizability: a model is prone to overfitting if closely fits a dataset with random labels. We use both methods to show that AL-trained model may be more general than that trained by BP.
We took f proportion of data, in which we changed the labels randomly. We used different propor- tions to train models. We report both the training and test accuracies clean test data.
Table 5 shows the results. First, as shown the last column (for a label noise rate of 0 0), the training accuracy of the CNN is larger than that of the CNN-AL, but CNN-AL outperforms CNN

---

Published as a conference paper at ICLR 2022
CNN random la bel with 6-1.0
CNN random label with 6-0.2
CNN random label with 6-0.6
Man a / WWwggy MM
3M
6.5
train accuracy
accurecy
6.55
6.48
epochs
opachs
opochs
Figure 3: Training accuracy when 0 = 1.0 (left) and test accuracies when equals 0.2 (middle) and 0.6 (right) with different epochs.
in terms of test accuracy. Second, with 0 > O. the vanilla CNN network with BP training fits the randomly labeled training data much better than the AL-form CNN. However, if we evaluate model accuracies based on the clean test data, the AL-form CNN performs better. Taken together, the above empirical evidence suggests that the new model may generalize better (Garg et al.) (202T). The relationship between accuracy and training epochs (for 1.0, 0.2, and 0.6) is given in Figure
Table 5: Accuracies of different label noise rates.
0
1.0
0.8
0.6
0.4
0.3
0.2
0.1
0.0
CNN
68.75
64.84
82.03
93.75
89.84
96.88
96.09
97.48
Training
CNN-AL
8.59
41.41
50.00
65.63
74.22
78.13
85.94
96.26
CNN
9.96
23.25
39.23
56.30
63.60
71.18
77.04
82.64
Testing
CNN-AL
10.00
38.23
64.74
76.77
78.70
82.19
83.40
85.16
3.5
WHAT MAKES AL WORK AN ABLATION STUDY
We empirically validates that AL generates great models, converges fast, and generalizes well. Be- low we report ablation studies using AG's news dataset to understand why AL performs excellently.
First, we ensure the gradient flow in each AL layer does not influence the gradients in other layers via the gradient stopping mechanism However, to optimize the objective function (Equation 3). either allowing or blocking the gradient flow between AL layers is acceptable. We sought to determine whether allowing gradient flow between layers improves model accuracy (although such a design may make pipelining infeasible). Hence, we disabled all the gradient stops (allowing gradients to flow through different AL layers). The results are denoted as "LSTTMAL w/o grad. stops" in Table 6 accuracy is nearly unchanged. Therefore, given enough computing nodes, applying pipelined AL increases training throughput without sacrificing model accuracy.
Second, to the effect of associated loss, disabled the gradient stops and removed the asso- ciated loss (except the last AL layer) from the model. The second row of Table shows the result: although the accuracy is similar to that with associated loss, far more epochs are needed to converge. So, the bridge functions likely facilitate model training by gradually bringing s and &; closer.
With the third experiment, we sought investigate the effect of the autoencoder components in the AL model. To do this, we passed the output of a normal Bi-LSTMM to a bridge function, whose output & was fed the bottleneck layer of an autoencoder, which decodes t back to the target y- Compared with the original Bi-LSTM classifier, the new design results in higher accuracy, seen in the third (LSTM + AE*1) 1) in Table 6 This suggests that replacing the standard classifier with a decoder of AE and training it with AL objectives could produce better model. Additionally, experimented with removing all autoencoders and using b; as the classifier to predict y for each AL layer (similar to the early exit technique in Teerapitayanon et al. (2016)). We use "LSTM-AL w/o AE (SC-1)" and "LSTM-AL w/o AE (full)" Table 6 to denote "early exit at the first layer" and "using by the last bridge as the classifier". respectively. The results show that using autoencoders
This supported by T'ensor.detach () in PyTorch and t wat.t__grradeeeee () in TensorFlow.

---

Published as a conference paper at ICLR 2022
gives better predicting model. We suspect that each autoencoder perhaps performs some kind of feature extraction and regularization. Particularly, although it is easy to transform one-hot encoded to t to decrease associated loss and transform to fo to decrease the autoencoder loss, the network still needs to learn more challenging tasks: converting 4 to 44+1 and 14+1 to i for > to reduce the associated loss and autoencoder loss of layer i. Consequently, adding autoencoders may prevent overfitting for the components beyond layer 1.
Finally, for LSTM-AL, the inference path goes through transformation functions (b3 - hi + h2 + h1) after the LSTM module. To ensure a fair comparison, we also tested a network with a 3-layer MLP classifier after the standard LSTM model. The result is denoted as LSTM MLP*3 in Table [ As shown, replacing the MLP from one layer to three layers does not produce better model, i.e., the number of layers is not why LSTM-AL. is better model.
Table 6: Ablation studies AG's News dataset to study why AL works.
Method
Accuracy
Convergence epochs
LSTM-AL w/o grad. stops
91.39 0.24
2.2 0.45
LSTM-AL w/o LA (i I,...., L 1) and w/o grad. stops
91.48 = 0.15
23.8 + 6.83
LSTM + AE
90.75 = 0.13
27.5 = 7.57
LSTM w/o AE (SC-1)
89.30 0.88
7.75 = 7,51
LSTM w/o AE (full)
89.39 0.99
7.75 7.51
LSTM + MLP*3
90.15 0.15
37.6 + 2.89
LSTM
90.32 = 0.23
25,4 13,42
LSTM-AL
91.53 0.20
2.4 0.89
4 RELATED WORK
4.1 BACKPROPAGATION ALTERNATIVES
Backpropagation is fundamental deep learning. However, backpropagation suffers from opti- mization and performance issues. Related work usually concerns methodologies that better imitate the signal transmission process of biological neural networks, since biological neural networks are highly efficient in analyzing visual, audio, textual, and other signals (Lee et al. 2015 Kao & Chen, 2021 Nokland 2016 Ororbia et al.) 2018: Ororbia & Mali 2019). Alternatives to end-to-end back- propagation can be categorized into three types: proxy objective, target propagation, and synthetic gradients (Duan & Principe 2021). Below we introduce each kind and the representative works.
The first type-proxy objective- uses local objective function for layer i to learn 0,, the parameters for the current layer i. The model then fixes 6,, adds one more layer + with another objective function, and learns 0:+ the parameters for the new layer. Consequently, proxy objective decouples the end-to-end backpropagation since the parameters in each layer are trained via local
objective function (Nøkland & Eidnes 2019 Belilovsky et al. 2019 2020
Löwe al. 2019).
The second every layer i. To update O,, the parameters of layer i, the model backpropagates the target (not gradient) through function 9;- Consequently, target propagation handles the situation where the relationship between the parameters and the cost is highly non-linear (e.g., discrete), which is oth- erwise difficult to solve using backpropagation (Lee et al. 2015 Meulemans et al. 2020).
propagation-approxiiiini approximates g,, the inverse of the downstream layers for
The third type is synthetic gradients, which learns function to approximate the gradient for each layer and utilizes the approximated gradients as the true gradients to update the parameters. This method thus decouples the layer-wise parameters (Jaderberg et al.] 2017 Czarnecki et al., 2017 Lansdell al. 2019).
Associated learning is different from these methods in several ways. First, both proxy gradient and target propagation propagate the signal (loss target) from the output layer directly to each layer. Biologically, this could be implausible because when in deep networks, hidden layers near the inputs unlikely to receive direct signals from the output layer that are far away. Associated learning, on the other hand, does not require each component to receive signals directly from the output layer. Additionally, although proxy gradient allows each layer to have local objective, most models of
8

---

Published as a conference paper at ICLR 2022
this type still learn the parameters in a layer- fashion, it may still be challenging to learn the parameters using a pipeline. One exception is Greedy InfoMax (GIM) (Löwe et al. 2019). which does not require direct signal from the output layer. However, GIM uses contrastive loss as the loss functionn, so GIM can mainly be applied to self-supervised learning tasks. As a result, for supervised learning tasks, AL is likely a more natural choice than GIM. As for synthetic gradients, empirical ex- periments show that gradients are difficult to predict, so model training based on synthetic gradients usually yields much worse accuracies than those for models trained via backpropagation.
4.2 NEURAL NETWORKS WITH STRUCTURES SIMILAR TO AL
The structure of an AL-form model may look similar to ResNet, as the bridge functions in AL appear analogous to the residual connections in ResNet. Therefore, some may argue that AL's superiority is the result of its resemblance to ResNet. However, the residual connections in ResNet are implemented to mitigate accuracy saturation (He et al. (016)), and all the parameters are adjusted to fit a global objective function, whereas the bridge functions in AL are implemented to modularize the network such that the gradient flow in one AL layer does not pass to other layers. Therefore, the bridge functions and the residual connections are only structurally similar; their purposes and functionalities are entirely different.
An AL-form network may also appear similar to contrastive learning (CL). Particularly, CL trans- forms a pair of images i and j (along with their augmented images i' and j') into vectors with large distances between (1) images and j and (2) augmented images i' and j', with small dis- tances between (1) an image i and its augmented image i' and (2) an image and its augmented image j'. This process is related to AL because both models convert inputs to vectors and compare the distance between the transformed vectors. Following this line, we are interested in studying the relationship between AL and CL in two ways. First, we could apply AL to CL such that CL is trained more effectively. Second, we could extend the AL model such that the inputs include a pair of samples 2 and Z2 with different labels and design an objective function that maximizes the distance between the vectors transformed from 21 and 222 (since their labels are different) and simultaneously minimizes the current AI. loss function (Equation 3)
An AL network is also structurally similar to the models in (Lin al. 2014 Yeh et al. 2017). which also transform æ into a latent space and convert it to y via an autoencoder. However, the design motivations are very different from AL. The two works attempt to efficiently represent a multi-label target by low-dimensional latent representation c, so learning a function to convert x to may be easier. The motivation of AL is to design isolated loss functions so that the new design has favorable properties that BP does not have, such as short gradient flows, forward shortcuts, dynamic layer accumulation, and layer-wise pipeline learning. Isolated objectives are not implemented ([in) et al. 2014 Yeh et al. 2017).
Ladder network is another structurally similar network (Rasmus et al. 2015), which adds skip con- nections between each encoder to its corresponding decoder. While the ladder network may look similar to AL structurally, at least two manifest differences exist. First, the residual between y and y in the ladder network is propagated through all layers update the parameters on the inference path, but in AL, each layer has a local objective function, and most layers do not receive signals from the output layer. Second, the skip connection in the ladder network connects encoder layer to a decoder layer. In contrast, the bridge function in AL connects a hidden representation of x to the bottleneck layer of an autoencoder to create the local loss.
5 CONCLUSION
This paper studies associated learning, an alternative methodology to end-to-end backpropagation. Since backpropagation is the cornerstone of today's 's deep learning but still suffers from optimization and training performance issues, it is appropriate to study alternatives to backpropagation. We dis- cussed the unique properties of AL that BP does not have. AL provides multiple inference paths; it allows dynamic layer accumulation during training; it supports pipeline training, and, perhaps most importantly, its prediction power is comparable to and frequently better than backpropagation on many of the most successful neural network structures. Ablation studies disclose why AL may work and demonstrate AL's effectiveness and architecture flexibility in training and testing.

---

Published as a conference paper at ICLR 2022
REFERENCES
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving. Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg. Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, llya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wat- tenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learn- ing on heterogencous systems, 2015. URL https: https:://ww/ //wp://wwwww...mm www.wrtessoffoo.. org/ Software available from tensorflow.org.
Sanjeev Arora, Nadav Cohen, and Elad Hazan. On the optimization of deep networks:
Implicit acceleration by overparameterivation. In International Conference on Machine Learning, pp- 244-253. PMLR, 2018.
Eugene Belilovsky, Michael Eickenberg, and Edouard Oyallon. Greedy layerwise learning can scale to ImageNet. In International Conference on Machine Learning, pp. 583-593, PMLR, 2019.
Eugene Belilovsky, Michael Eickenberg, and Edouard Oyallon. Decoupled greedy learning of CNNs. In International Conference on Machine Learning, pp. 736-745, PMLR, 2020.
Pu Chen and Hung-Hsuse Hsuan Chen. Accelerating matrix factorization by overparaneterizztion.. In International Conference Deep Learning T'heory and Applications, pp. 89-97, 2020.
Wojeiech Marian Czarnecki, Grzegorz Šwirszez, Max Jaderberg, Simon Osindero, Oriol Vinyals, and Koray Kavukcuoglu. Understanding synthetic gradients and decoupled neural interfaces. In International Conference on Machine Learning, pp. 904 912. PMLR, 2017.
Li Deng and Dong Yu. Deep learning: Methods and applications. Foundations and Trends in Signal Processing, 7(3-4):177-377, 2014.
Shiyu Duan and Jose C Principe. Training deep architectures without end-to-end backpropagation: A brief survey, arXiv preprint arXiv: 001.000011 2021.
Saurabh Garg, Sivaraman Balakrishnan, J Zico Kolter, and Zachary C Lipton. RATT: Leveraging unlabeled data to guarantee generalization. arXiv preprint arXiv:2105.0033 2021.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT press, 2016.
Florian Graf, Christoph D. Hofer, Mare Niethammer, and Roland Kwitt. Dissecting supervised contrastive learning, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp- 770-778, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8): 1735-1780, 1997.
Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber, et al. Gradient flow in recurrent nets: The difficulty of learning long-term dependencies, 2001.
Max Jaderberg, Wojciech Marian Czarnecki, Simon Osindero, Oriol Vinyals, Alex Graves, David Silver, and Koray Kavukcuoglu. Decoupled neural interfaces using synthetic gradients. In Inter- national Conference on Machine Learning, pp. 1627-1635. PMLR, 2017.
Yu-Wei Kao and Hung- Chen. Associated learning: Decomposing end-to-end backpropaga- tion based on autoencoders and target propagation. Neural Computation, 33(1):174-193, 2021.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Pro- cessing Systems, volume 33, pp. 18661-18673. Curran Associates, Inc., 2020.
10

---

Published as a conference paper at ICLR 2022
Benjamin James Lansdell, Prashanth Ravi Prakash, and Konrad Paul Kording. Learning to solve the credit assignment problem. arXiv preprint arXiv:1906. 00889, 2019.
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propagation. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 498-515. Springer, 2015.
Zijia Lin, Guiguang Ding, Mingqing Hu, and Jianmin Wang. Multi-label classification via feature- aware implicit label space encoding. In Proceedings the 3 stt International Conference on Inter- national Conference on Machine Learning Volume 32, ICML'14, pp. 11-325-11-333 J.L.R.ogg.
2014.
Sindy Löwe, Peter O'Connor, and Bastiaan Veeling. Putting end to end-to-end Gradient-isolated learning of representations. In Advances in Neural Information Processing Systems, pp. 3039- 3051, 2019.
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142-150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics.
Alexander Meulemans, Francesco S Carzaniga, Johan AK Suykens, João Sacramento, and Ben- jamin F Grewe. A theoretical framework for target propagation. arXiv preprint arXiv:2006.1433: 14331, 2020.
Arild Nøkland. Direct feedback alignment provides learning in deep neural networks. In Advances in Neural Information Processing Systems 29; Annual Conference on Neural Information Pro- cessing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 1037-1045, 2016.
Arild Nøkland and Lars Hiller Fidnes. Training neural networks with local error signals. In Inter- national Conference Machine Learning, pp. 4839-4850 PMLR, 2019.
Alexander G Ororbia and Ankur Mali. Biologically motivated algorithms for propagating local tar- get representations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 4651-4655 2019.
Alexander G Ororbia, Ankur Mali, Daniel Kifer, and Lee Giles. Conducting credit assignment by aligning local representations. arXiv preprint arXiv: / /803 0/ 0/34, 2018.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary De Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-Bin Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024-8035. Curran Associates, Inc., 2019.
Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word rep- resentation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532-1543, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.31 1L.1155/1/D14-111
Antti Rasmus, Harri Valpola, Mikko Honkala, Mathias Berglund, and Tapani Raiko. Semi- supervised learning with ladder networks. NIPS' 15, pp. 2015. MIT Press.
Cambridge, MA, USA,
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale im- age recognition. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631-1642, Seattle, Washington, USA, October 2013. Association for Computa- tional Linguistics.
11

---

Published as a conference paper at ICLR 2022
Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsune Kung. Branchynet: Fast inference via early exiting from deep neural networks. 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2469-2469. IEEE, 2016.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems, pp. 5998-6008, 2017.
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and
Léon Bottou. Stacked denoising autoencoders: Learning useful representations in deep network with a local denoising criterion. Journal of machine learning research, 11(12), 2010.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07405 2018.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: A novel image dataset for bench- marking machine learning algorithms, 2017.
Chih-Kuan Yeh, Wei-Chieh Wu, Wei-Jen Ko, and Yu-Chiang Frank Wang. Learning deep latent spaces for multi-label classification. In Proceedings of the Thirty-First AAAI Conference on Arti- ficial Intelligence, AAAI 17, pp. 2838-2844. AAAI Press, 2017.
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In NIPS, 2015.
Xiang Zhang. Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text clas- sification, 2016.
12

---

Published as a conference paper at ICLR 2022
APPENDIX
6.1 AL INFERENCE ALGORITHM
Algorithm shows the inference algorithm for AL networks. supports forward shortcuts by going through only of the T, AI layers.
From Algorithm 3 it is clear that the functions 9i-S are not used inference, and only one bridge function out of the I b;-s are used in inference. Therefore, the number of parameters of inference is smaller than the number of training parameters.
Algorithm 3 Using AL form network for inference
Input AL network A (01, (2,...aL), (fi, b, 9s, hi input data z, inference length
,, 1 S L
Output predicted label
1: s
2: for i = 1 to l
do 5 fs(ss)
3:
4: ++ by(s)
5: t
Same Figure
6: for = l to do
7:
i' *+ (t''))
8: return
6.2 CNN, RNN, AND TRANSFORMER THEIR AL FORMS
Here extend 2.1.J for further explanation. In AL, the process in bridge layers did influence the actual forward path in AL. Therefore, the process in 2.1.1 is only used to calculate associated loss and will not effect the original representation shape. Figure and 5 that when we take the last hidden state in the sequence input calculate associated loss, the original sequence representation remains the same when propagate to the next layer.
b2
W2
W2
02
W2a
$2
S/
f2
b
b
S1
S
R8
4
unfold
$
(1)
((xl)
S
4
f
F1
01
+0
x22
x (IrD
Figure A closer look on how AL works on RNN.
6.3 IMPLEMENTATION ENVIRONMENT DETAILS
For LSTM and Transformer, we used the PyTorch package (Paszke et al., 2019). For CNN, we used the Tensorflow Abadi et al. (2015) package to build the AL models (vanilla CNN and a vi- sual geometry group (VGG) network). We trained the models with GeForce 3090 GPUs on Ubuntu Linux 20.04. Since current hardware development does not support actual pipeline training, we sim- ulated pipeline training by blocking gradient flows between AL layers with Tensor. detach () in Torch and with f.f. stop_grad ent () in TensorFlow.
13

---

Published as a conference paper at ICLR 2022
52
$2
Mean-pooling
52
Attention
82
(2)
(+l
Mean-pooling
S
fi
fi
unfold
Attention
81
x
ORD
Figure 5: closer look on how AL works on Transformer.
6.4 DATASET STATISTICS
The IMDB Review dataset (Maas al. 2011 is binary sentiment classification dataset contain- ing 25k training data and 25k test data. AG's News corpus dataset (Zhang et al.) 2015) includes four news topics: world, sports, business, and sci/tech. It has 120k training data and 7.6k test data. The DBpedia Ontology classification dataset (Zhang et al. 2016) was constructed by picking 14 non-overlapping topic classes from DBpedia 2014, containing 560k training samples and 70k test samples. The Stanford Sentiment Treebank (SST2) dataset (Socher et al.] (013) is binary senti- ment classification dataset from GLUE (Wang al., 2018). The dataset contains 67k training sam- ples, 872 validation samples, and 1.8k test samples. The CIFAR-10 dataset contains color images from classes; 50k are training samples, and 10k are test samples. Finally, the Fashion-MNISS dataset (Xiao et al., (017)) is grayscale image classification dataset with 10 classes. It consists of 60k training samples and 10k test samples.
6.5 TEXT CLASSIFICATION
During training, the most frequent 30k words in the training corpus were included in the word em- bedding. We replaced the remaining words with the [UNK] token. We trained all text classification experiments with 50 epochs. We used the Adam optimizer.
6.5.1 BI-LSTM
For Bi-LSTM, we applied gradient clipping with the value 5. For the standard version of Bi-LSTM, experimented with learning rates of 0.0005 and 0.0001 and reported that with better accuracy. We also adjusted the learning rate for AL, but the learning rate has little influence on the accuracy and the convergence speed. We used batch size 64 for all Bi-LSTM experiments. For IMDB and DBpedia, we set the max sequence length to 500. For AG's News and SST, we set the max sequence length to the longest sequence length; the exact length settings were applied to the Transformer experiments. The objective of the standard Bi-LSTM was cross-entropy loss. Details are given in Table 7
Table 7: Bi-LSTM settings.
emb dim
Istml dim
lstm2 dim
g dim
h dim
Ir
LSTM LSTM-AL
300
350
350
0.0001
300
300
300
128
128
0.0001
14

---

Published as a conference paper at ICLR 2022
6.5.2 TRANSFORMER
For the standard Transformer model, we used negative log-likelihood the objective with multi- head attention (6 heads). We set the batch size to 128 because the standard Transformer converges only with a large batch size in our experiment. Table 8 gives the details.
Table 8: Transformer settings.
emb dim
layerl dim
layer2 dim
head
dim
h dim
Ir
Tran
300
512
512
6
0.00025
Tran-AL
300
256
256
6
128
128
0.00025
6.6 IMAGE CLASSIFICATION
We normalized the image pixels for both FashionMNIST and CIFAR-10 10 datasets as a preprocessing step. Since CIFAR-10 is more challenging. we also performed data augmentation on this dataset by resizing, random cropping, random flipping, and adjusting the brightness of the images. We set the batch size to 128. We set the initial learning rate to 0.0001 and applied scheduled decay until converging. We used Adam as the optimizer and cross-entropy loss for the standard version.
Vanilla CNN Model We implemented the vanilla CNN by modifying the previous code. The CNN contains 13 hidden layers and an output layer. The first 8 hidden layers are convolution layers; fully connected layer with 1280 neurons. The subsequent layers consist of feedforward layers, followed by an output layer with 10 classes. We set the initial learning rate to 0.0001 and applied scheduled decay.
the last flattened
CNN and Accuracy CIFAR1O
Test Accuracy FashionMNIST
0.920
0.80
1601 Accuacc
0.75
0.905
Test
0.900
0.895
2.65
0.890
OUNJAL
0
10
20
30
40
50
70
0
20
40
60
100
Epochs
Epochs
Figure 6: Testing accuracy vs. epochs for CNN and CNN-AL on CIFAR- and FashionMNIST.
6.7 TRANSFORMER-AL CONVERGENCE SPEED
Training an LSTM, CNN, VGG based on AL requires much fewer epochs when compared with training them by BP; this trend is less clear when training a Transformer-Al model. However, found that after training the Transformer-AL for roughly 5 epochs, the validation set accuracy (90.95) can already achieve 99% of the best validation set accuracy (91.31), and the test accuracy (90.98) very close the best test accuracy (91.17, 17, i.e., training Transformer-AL. for 20 epochs for AG News). So, we could reduce the number of epochs by sacrificing slight accuracy. Transformer- AL on DBpedia has similar trend: when training ALL on DBpedia for epochs, its validation set accuracy achieves 96.0902, which is very close to the best validation set accuracy (96.6134). This result shows that Transformer-AL is efficient on convergence when training on DBpedia and AGnews.
15

---

Published as a conference paper at ICLR 2022
6.8 ASSOCIATED LOSS vs. LAYER DEPTH
This section shows that, under the AL network structure, the latent representations in the lower layer appear to help generate the latent representations in an upper layer.
We experimented with the AL layer number i vs. the associated loss ||s; t4ll2 using different datasets on different network structures. The following tables show that the associated loss at an upper layer (i.e., the layers closer to the output) smaller than at lower layer (i.e., the layers closer to the input). Consequently, the shortcuts are likely doing some kind of curriculum learning.
Layer
Associated Loss on DBPedia
Associated Loss AGNews
Embedding layer
5.6061 X 10
1.4802 x 10
LSTM-1
3.7440 10
5.3281 X 10 55
LSTM-2
1.7786 x 10-7
4.8651 x 10-6
Table 9: Layer depth vs. associated loss on LSTM-AL
Layer
Associated Loss on DBPedia
Associated Loss AGNews
Embedding layer
5.8318 x 10
3.3179 x 10
Layer-1
3.0053 10-6
1.2561 X 10
Layer-2
7.4727 10-8
1.3976 X 10-5
Table 10: Layer depth vs. associated loss on Transformer-AL
6.9 THE DESIGN OF THE OVERCOMPLETE AUTOENCODERS IN AL
autoencoder's hidden layer usually consists of fewer neurons than the input and output layers. However, use overcomplete autoencoder (i.e., the hidden layer has more neurons than the input/output layers) (Vincent et al., 2010 in AL. An overcomplete autoencoder can reach zero re- construction loss (i.e., the autoencoder loss in Equation 3) by copying the neurons from the input layer to the output layer. However, in AL, Oi (the encoding function of the autoencoder at layer i) needs to minimize both L' the autoencoder loss and L the associated loss. As result, La can be regarded regularization term when minimizing CAEE-
Additionally, we hypothesize that mapping the one-hot encoded y to a high dimensional vector can accelerate the convergence speed. According to Graf al. (202I), the cross-entropy loss and su- pervised contrastive loss attain their minimum if and only if the features of each class collapse to the vertices of an origin-centered regular K 1 simplex, where K is the number of classes. In AL, since the autoencoder in layer-1 simply maps y to high dimensional space and back (i.e.. letting hi (gi (y)) as y). it is nearly guaranteed reach minimum loss. Therefore, can consider 91 (B) as an ideal prototype for AL to fit in, such that, b, (f(())) ~ 91 (3)). Therefore, the associ- ated loss directly optimizes for K- regular simplex in the representation space. which is the best configuration to minimize the supervised contrastive learning loss (Khosla et al.) 2020). Such an autoencoder setup probably helps AL find the best configuration the early stage of training- This may explain why AL converges so fast. Besides, an overparameterized model may accelerate optimization (Arora et al. 2018 Chen & Chen (2000). Empirical studies in Section 3.5 support our claim: using overcomplete autoencoders to replace classifiers performs better.
16

---

Published as a conference paper at ICLR 2022
6.10 PYTORCH PSEUDOCODE
Here provide PyTorch pseudocode of 5-layer AL network.
import torch
import torch.m.
class MLP (nn Module)
def
d1, :
super (uper()__init
self.f = mn Sequunttnt nn. Lineir (d1, d2), nn. Sigmiggidd
def (self, x): return felf.f(x)
class AL (nnnMoodue)):
def
class_num);
super (uper()._iit___
# we choose 10 as hidden dimension size for simplicity.
self.f Mn.Moculelist (MLP (10, 10) for i in range (5)
self.g mn.MoculeList (MLP (10, 10) for i in range (5)])
(MMP
self.b nn.Moculetiist [MLP (10, 10) for i in range (5)])
self.h an.ModuleList [MLP (10, 10) for i in range (5)
self.g[0] self.h[0] MLP (class_num, num, 10), MLP (10, class_num)
self.ce mn.MsEL (), mn.CrossEntropyloss ()
self.c class_num
def forward(self, x, y):
loss 0.0 s_i, t = x, nn.
oon_hot. (y, num_classes=seef.cc) float ()
# convert y into one-hot vector for linear input for i in range(5).
last_t_i, si = t_i, self.f[i) [[i] # encode input t_i, s_i_prime self.b[i] (s (s_i), (t_ bridge encode target
t_i_prime = self.h(i) (t_i) # decode target
if i == 0:
loss +- self.mse (t_i. (),, s i prime) self.ce (last_t_i, t p_ipr.
else:
loss += self. www.f.meeffeeee (t_i.detach (), s_i_prime)
self mse (last_t t_i_prime)
s_i, t = s detdchhch(() (), t_i. () # cut gradient
return loss
def inference (self, x):
for layer in self.f: layer
y self.b(-1]
for layer in (ser = layer (y)
return y
model AL
17

---

Published as a conference paper at ICLR 2022
X, y ttrch. rand (30, 10, dtype=torch. float), , randint (high=4, size= (30, ) loss, pred model (x, y), model. inference (x)
18


Reviews:
[{"review_id": "yFUZHADZyOh", "reviewer": "Reviewer_L9xu", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Thank you for this read. The results and the methodology are definitely compelling. Why I cannot accept the manuscript as is, is that:\n· The motivation is not clear enough. It is clear wrt why BP is not ideal. But it is not clear how you landed on this method specifically, as compared to many other attempts on finding more optimal neural network optimization methodologies.\n· Section 2 is very difficult to follow. I would spend some more effort explaining how your method works in the manuscript.\n· It would be nice to include an algorithm of how to implement AL.\n\nA selection of minor comments:\n· Some typos throughout the manuscript, e.g., in abstract \"associate\" and paragraph 4 in the introduction \"in Section 4 We\".\n· Notation must be introduced, e.g., f, y, etc. in Section 2 are not introduced properly in relation to Figure 1.\n· It is difficult to follow the difference in notation when using h, b, and f. I recommend you spend some more time on making this very clear to the reader. \n· I find the Table 2 epochs for AG News difficult to follow. There is a clear pattern that AL is faster, but then things changes radically for AG News? Would be nice with some further analysis into this.", "summary_of_the_paper": "This paper studies and benchmarks an alternative to back-prop named associated learning. They analyze the pros and cons.", "summary_of_the_review": "With some more clarification on how you ended up with this methodology and a clear algorithm for how to implement AL, the reviewer would be happy to accept the manuscript.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RG4NFnpye0f", "reviewer": "Reviewer_9Led", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors have resolved my concerns on the technical details of how AL is applied on RNNs and Transformers.\n\n================================\n\nThe paper is well written. Experiments show that in text classification and image classification, the proposed method outperform BP in some basic architecture setting.\n\nHere are my concerns:\n\n- It is unclear how the AL is applied on RNNs and Transformers. Section 2.1.1 only very briefly described them, but I could not figure out some of the details. For example, how is the temporal data processed in LSTM?\n\n-In CNN, when flatting the hidden representation, it also lost the spatial information in feature maps. Furthermore, how to convert si to si' if ti is also a 3d feature map when the spatial information is lost.\n\n- From the description in Section 2, it seems that AL introduces around double the parameters for a given neural network. What is the impact of the increased parameters in computation cost?\n\n- The experiments uses relatively simple network architecture for text classification. Does the same benefits carry over to large transformer models, and still beats currently popular models like BERT?\n\n- The architecture information on CNN in section 3.3 is missing.\n\n- If my understanding is correct, the proposed architecture would not work in sequence generation task like LSTM and transformers could do. Right?", "summary_of_the_paper": "This paper is a continuation of an original associated learning paper by Kao&Chen 2021. It attempts to propoose new learning approach associated learning as an alternative way to back-propagation. On top of the original paper, it discovers more interesting properties and extend AL to CNN, LSTM and transformers (though lacking sufficient details).", "summary_of_the_review": "In summary, I think though the paper proposes AL framework as an alternative to BP, it is actually a simple extension to a previous work, and does not proposes substantially new ideas. Some details are missing, and experiments are not extensive enough to cover state-of-the-art architectures.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "L0ezniPFaVi", "reviewer": "Reviewer_pfre", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "First, as in Figure 2, the paper proposes to map input x and output y into a latent space for metric learning ($f(x)=g(y)$) and auto-encoder learning ($y=h(g(y))$) are also investigated in multi-label classification[r1,r2], which are not discussed in this paper. In my opinion, the main difference is the design of multiple latent spaces compared with these multi-label classification methods.\n\n\nSecond, in traditional machine learning, we often map a high dimensional space to a low dimensional space for metric learning. It is unclear why maps the target $y$ to the intermediate layers in this paper. Given a high dimensional space (e.g., images), the inference model extracts useful features and filters unrelated features for metric learning. \n\nHowever, in this paper, I find that the authors conduct experiments on some single label classification (e.g., CIFAR10 and Fashion-MNIST). In this case, $y$ is a scalar or one-hot vector, I am curious about the exact form of $g_1$, $g_2$, $g_3$ in Figure 2. Does the proposed method map a low dimensional latent space to a high one? What is the motivation for expanding representation space? If $g_1(y)$ and $g_2g_1(x)$ are still in a low dimensional space or $g_i$ are very simple, do we really need inverse transformations from $Y$ to AL layers? In this case, we can simply fuse different AL layers to top layers for metric learning. For example, we can move $y$ after  $t_3$ and remove $h_1$, $h_2$, $h_3$ in Figure 2. Since $y$ is a specific label, it is unclear why we need to map to a high dimensional space.\n\nThe design of multi-label classification is reasonable to me because the target $y$ is complex (e.g., the multiple label vectors could miss some labels) and the multi-labels could be in a high dimensional space. In this case, one can map high dimensional space $X$ and $Y$ into a low dimensional latent space for metric learning.\n\n[r1] Learning Deep Latent Spaces for Multi-Label Classification, AAAI 2017.\n[r2] Multi-label Classification via Feature-aware Implicit Label Space Encoding, 2014.\n...\n\nThird, it would be better to set a baseline by moving $y$ after  $t_3$ and removing $h_1$, $h_2$, $h_3$ in Figure 2 for comparison.\n\nFourth, the architecture of AL layers is similar to Ladder networks. It is suggested to analyze the differences.\n\n[r3] Semi-Supervised Learning with Ladder Networks, 2015.", "summary_of_the_paper": "This paper proposes associated learning (AL) for CNN, RNN, and transformer. Different from back-propagation (BP), AL decomposes BP’s global end-to-end training strategy into several small local optimization targets such that each sub-networks has an isolated gradient flow. To achieve this, the paper proposes to map input $x$ and output $y$ into intermediate AL layers and performs metric learning (e.g., $t_1=b_1(s_1)$) and auto-encoder learning ($t_1=t_1^{‘}$), as shown in Figure 2. Moreover, Each AL layer can be optimized locally. The idea is interesting. The experiments demonstrate the effectiveness on (IMDB Review, AG’s News corpus, DBpedia Ontology, the Stanford Sentiment Treebank, CIFAR10, and Fashion-MNIST.", "summary_of_the_review": "(1) The proposed method can be optimized locally, and achieve competitive results. The proposed framework can be used for CNNs, RNNs, and transformers. The idea is interesting.\n\n(2) More analyses about the motivation and the necessity of inverse transformation for Y to latent space are needed.\n\n(3) The analyses and discussions about related works, such as multi-label classification and ladder networks, are missed.\n\n(4) Some experiments are suggested to support the authors' opinions (e.g. (2) and (3)) if possible.\n\n\n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HPa-YX2RP9L", "reviewer": "Reviewer_dmxT", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper clearly lays out the advantages of associated learning: faster inference, dynamic layer accumulation, and pipeline. The paper is clearly written with good figures. The experiments appear to be easy reproducible, too. The decrease in epochs needed for LSTMs is particularly impressive.\n\nI found the biological basis a little lacking. Perhaps, some type of curriculum learning or more exploration on what the various shortcuts are doing could make this argument stronger. The related works section neglects to mention other gradient-isolated methods like https://arxiv.org/abs/1905.11786. I think in some ways this work can be seen as encoder-decoder with additional regularization, too?", "summary_of_the_paper": "Associated Learning puts forth a template that can be applied to almost any network to achieve faster training and inference. They apply their template to several existing deep learning models and perform experiments that show they can achieve comparable if not better results with less training time. ", "summary_of_the_review": "I would recommend this paper to be accepted. While there are several issues, the empirical results are strong (particularly the LSTM reduction in epochs). I think is a lot more to explore with the dynamic layer accumulation and gradient isolation, too, that would be interesting to other researchers.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a method for associative learning as an alternative to back propagation based learning. The idea is to interesting. The coupling between layers are broken down into local loss functions that can be updated independently. The targets are projected to previous layers and the information is preserved using an auto-encoder loss function. The projections from the target side are then compared with the projections from input side using a bridge function and a metric loss. The method is evaluated on text and image classification tasks. The results suggest that this is a promising alternative to back propagation based learning.

Pros
+ A novel idea that seems promising
+ Evaluated on text and image classification tasks and demonstrated utility

Cons
- The impact of the number of additional parameters and the computation is not clarified (even though epoch's are lower)

The authors utilized the discussion period very well, running additional experiments that were suggested (especially ablation studies). They  also clarified all the questions that were raised. In all, the paper has improved substantially from the robust discussion.
Decision: Accept (Poster)