Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings
Authors: Edouard YVINEC
Arnaud Dapogny
Kevin Bailly
Venue: ICLR 2024 poster (2024)
Keywords: quantization
codebooks
hashlists
compression
efficient inference
deep learning
Primary area: general machine learning (i.e., none of the above)

Abstract: The massive interest in deep neural networks (DNNs) for both computer vision and natural language processing has been sparked by the growth in computational power. However, this led to an increase in the memory footprint, to a point where it can be challenging to simply load a model on commodity devices such as mobile phones. To address this limitation, quantization is a favored solution as it maps high precision tensors to a low precision, memory efficient format. In terms of memory footprint reduction, its most effective variants are based on codebooks. These methods, however, suffer from two limitations. First, they either define a single codebook for each tensor, or use a memory-expensive mapping to multiple codebooks. Second, gradient descent optimization of the mapping favors jumps toward extreme values, hence not defining a proximal search. In this work, we propose to address these two limitations. First, we initially group similarly distributed neurons and leverage the re-ordered structure to either apply different scale factors to the different groups, or map weights that fall in these groups to several codebooks, without any mapping overhead. Second, stemming from this initialization, we propose a joint learning of the codebook and weight mappings that bears similarities with recent gradient-based post-training quantization techniques. Third, drawing estimation from straight-through estimation techniques, we introduce a novel gradient update definition to enable a proximal search of the codebooks and their mappings. The proposed jointly learnable codebooks and mappings (JLCM) method allows a very efficient approximation of any DNN: as such, a Llama 7B can be compressed down to 2Go and loaded on 5-year-old smartphones.

Published as a conference paper at ICLR 2024
NETWORK MEMORY FOOTPRINT COMPRESSION THROUGH JOINTLY LEARNABLE CODEBOOKS AND MAPPINGS
Edouard Yvinect-2, Arnaud Dapogny", Kevin Bailly Sorbonne Université CNRS, ISIR, f-75005, 4 Place Jussieu 75005 Paris, France Datakalab2, 114 boulevard Malesherbes, 75017 Paris, France ey@datake 1. Lab. com
ABSTRACT
The massive interest in deep neural networks (DNNs) for both computer vision and natural language processing has been sparked by the growth in computational power. However, this led to an increase in the memory footprint, to a point where it can be challenging to simply load model on commodity devices such as mobile phones. To address this limitation, quantization is favored solution as it maps high precision tensors to low precision, memory efficient format. In terms of memory footprint reduction, its most effective variants are based on codebooks. These methods, however, suffer from two limitations. First, they either define a single codebook for each tensor, or use a memory-expensine mapping to multiple codebooks. Second, gradient descent optimization of the mapping favors jumps toward extreme values, hence not defining a proximal search. In this work, we pro- pose to address these two limitations. First, initially group similarly distributed neurons and leverage the re-ordered structure to either apply different scale factors the different groups, or map weights that fall in these groups several code- books, without any mapping overhead. Second, stemming from this initialization, we propose joint learning of the codebook and weight mappings that bears sim- ilarities with recent gradient- based post-training quantization techniques. Third, drawing estimation from straight-through estimation techniques, we introduce a novel gradient update definition to enable a proximal search of the codebooks and their mappings. The proposed jointly learnable codebooks and mappings (JLCM) method allows very efficient approximation of any DNN: such, a Llama 7B can be compressed down to 2Go and loaded on 5-ycar-old smartphones.
INTRODUCTION
Deep neural networks (DNNs) have reached point of hegemony in most areas of machine learning. This is most blatant in the context of computer vision Ren et al. (015): : Chen al. (2017 and natural language processing Devlin et al. (008). In particular, the transformer architectures Vaswani et al. (017)) have achieved the most impressive results in terms of performance and predictive relevance, with large language models Zhang et al. (2022a) in the lead. However, these deep architectures and their performance come at a price: their memory footprint. This cost has reached new heights with some architectures requiring multiple modern, high-end, devices to simply load such models. For example, to this day, the largest consumer grade graphies processing unit (GPU) has 24Go of VRAM, which is not enough to load even the smallest member of the Llama LLM family Touvron et al. (203) using the default floating point (fp32) format (28Go). This computational cost growth has been more and more hindering DNNs integration and deployment, especially on mobile devices.
In practice this limitation is often addressed using weight quantization techniques. The latter con- sists in mapping the original high-precision tensors to low-precision, memory efficient represen- tations. The resulting quantized model requires a significantly lower memory footprint and often leverages less costly individual operations. However, the quantization process itself may be costly, as the best performing such solutions (namely QAT, short for Quantization -wwar Training Zhang et al. (222 usually involves full retraining, sometimes with larger batch sizes and number of

---

Published as a conference paper at ICLR 2024
Standard Backpropagwtoon
JLCM (nuliiple codebooks)
JLGM (muliipe scales)
Hard indices
indxx Codebook: Index Gradient
Hard Indloos
Codecooks
Indox Grodlend
Hard
Indox
Codebook
Scales
Index Gradien Upduns
Updane
Upaane
12.23
11.25
102
+0.08
0:03
Figure I: Left side: in the standard approach, each weight is represented by soft index that points to a value within a single codebook. Assuming that the loss for this weight suggests increasing its value (highlighted in red), then the standard gradient update would be scaled by the codebook values, thus skipping 0.17 and directly increasing the weight of 0.96 in the codebook. Center/right side: a contrario, JLCM leverages multiple codebooks and/or scales and defines a novel gradient update term to allow proximal search within the codebook values.
epochs. To circumvent this and scale to any DNN size, post-training quantization has been intro- duced, with the most extreme scenario being the fully data-free setup, described in Nagel et al. (2019), however generally at a significant expanse in terms of accuracy of the quantized models. Noteworthy, beyond this fully data-free setting, the performance of quantization techniques can be greatly overhauled by considering very small calibration sets, as in the work of Nagel et al. (2020).
The primary aim of the aforementioned quantization techniques is reduce the latency of the net- work by performing arithmetic in lower precision formats. This may however introduce unnecessary constraints if the goal is rather limit the memory footprint. In order to do this, the most effective methods in terms of compression rates consists in storing in a codebook restricted set of values, onto which each weight shall be mapped using a more compact code. Nevertheless, these methods suffer from two major drawbacks, as illustrated on Figure First, the granularity problem: these approaches use single codebook for each tensor order avoid a more costly, conditional map- ping: intuitively, if multiple codebooks were to be used, one would need to store both the codebook and value indexes for each weight (vs. only the value index in a single codebook), which would dramatically increase the memory footprint. Second, we argue that the optimization of the mapping itself is not well-defined. Formally, for a given scalar weight value, a gradient- -based optimization would favor jumps towards extreme values rather than closer values in the target codebook. This would be in opposition with the desired proximal behavior of gradient descent optimization. As re- sult, codebooks and mappings cannot be learned in joint manner using gradient-based algorithms, limiting their efficiency.
In this work, we propose to tackle these two limitations, as pinpointed Figure First, we pro- pose to group similar neurons as defined by their weight distribution. We can then apply different scaling factors homogeneous weight values or, alternatively, we can assign multiple codebooks to the reordered weight matrix without additional memory overhead (as, in this case, the indexation to different codebooks depends implicitly on the weight index). This provides finer grained ini- tialization. Second, stemming from that initialization, we propose to jointly optimize the codebooks and mappings a setup similar to what done in gradient- based post training quantization. Third, drawing inspiration from ttraight-through estimation in DNN quantization, we propose to rewrite the gradient update for the codebook mappings, favoring proximal values in the codebooks. The proposed jointly Icarnable codebooks and mappings (JLCM) method can be summarized in:
neuron re-ordering based on weight distribution similarities. This can be leveraged to ap- ply different scale factors on homogeneous weight groups, or map these groups to multiple codebooks without any memory overhead over the mappings.
joint learning of the codebooks and mappings that bears similarities with gradient-based based post-training quantization techniques.
a new gradient update definition for the codebook assignment, which favors proximal val- ues over extreme values.
The proposed JLCM method achieves almost ternary compression of all the layers of large trans- formers with less than one percent performance degradation. Such compression rates are of par- ticular interest for large language models deployment at scale. For instance, JLCM enables to
2

---

Published as a conference paper at ICLR 2024
preserve the 95% of the performance of a Llama 7B model 2Go device such the iPhone 8 or a Samsung Galaxy A10.
RELATED WORK
In this section, we delve into the current state-of-the-art in the related DNN quantization techniques, well as the recent gradient-basel post-training quantization techniques. We also review existing codebook-baid based methods for DNN memory footprint compression.
2.1 QUANTIZATION
In post training quantization, the data-free setup first introduced in Nagel al. 2019). In this work, the core challenge at hand was to identify quantization ranges for both weights and activations without having access any data. To do this, the authors proposed use statistics stored during training by the mutch-normalization layers. With respect to the weight tensors, the proposed method leveraged baseline uniform operators Krishnamoorthi (2018) between floating points and integers. Since this work, many iterations Zhao et al. (2019); Cong et al. (2022) have been proposed, focusing on improving the weights encoding. However, all the aforementioned methods struggle retain good accuracy when weights are quantized using 4 or fewer bits, even on models that are nowadays deemed less challenging to quantize such as ResNet 50 addressed with non-uniform quantization. For instance, Li et al. (2019) propose to use the logarithm function to map weight values to an integer exponent: this changes the nature of the arithmetic used by the quantized networks, with multiplications becoming bit shifts. Yvnec et al. (202cc) argue that such change is hard to leverage on existing hardware in practice, and propose a power quantization method that preserve the nature of these operations. However, this method also requires custom implementation to provide latency boosts on existing hardware. Furthermore, on LLMs, fully data- free power quantization does not enable quantization below the 4 bits mark, even combined with group wise quantization techniques.
al. (2016). This shorteoming is often
2.2 GRADIENT-BRE BAASD POST-TRAINE QUANTIZATION
In order circumvent this limitation while maintaining the scalability of data-free quantization, Nagel et al. 2020) introduced AdaRound, the first gradient-based post-training quantization (GPTQ)
technique. This method and its iterations Li et (2021) Wei et al. 2202 ; Liu et al. (023): Y Yvinec et al. (o03ss in optimizing the quantized weights over a small calibration set excerpted from
the original training data. AdaRound optimizes each layer individually and sequentially in a self-
distillation Hinton et al. (2014) fashion, where the quantized model learns to imitate its original counterpart. This optimization framework has inspired other compression domains, such as pruning Kwon al. (2022). However, the primary goal of these methods is to reduce the latency of the quantized models: however, if the purpose is to reduce the memory footprint of the models, GPTQ methods struggle to scale very large networks, e.g. LLMs. In terms of raw memory footprint compression, the most effective methods are the more dedicated codebook-based methods.
2.3 CODEBOOK BASED APPROACHES FOR MEMORY FOOTPRINT REDUCTION
Codebook-based methods Chen al. (2155: Han al. (015 Eban et al. 2000 Jeon et al. (202): Javaheripi et al. ) vinec et al. (021) consist in mapping a continuous set of weight values to a finite set, and each weight is represented on fewer bits by an index referring to one of the codebook values. Among these methods, most focus on effective implementations Chen et al. (205): Jeon et al. (2020); Javaheripi et al.) (2022) and use naive approaches with respect to mapping step itself. On the flip side, other techniques propose to learn the codebook values Han ct al. 20135; Eban et al. (020). Ultimately, some of these techniques Eban et al. 2020); Yvine al. 2202 are only introduced as a stepping stone for further compression through pruning. All these methods bear number of limitations that render them ineffective for compression in large networks such as LLMs: first, these methods generally work at a coarse granularity level that does not allow much expressively. Second, to the best of our knowledge, there no method that jointly learns the codebook and weight mappings in an end-to-end manner, which, argue, is critical to
3

---

Published as a conference paper at ICLR 2024
Table 1: Relative footprint of the weights Table 2: Comparison between the latency (in ms)
of models fully loaded on a CPU (i7 13th gen), memory footprint Mar (in MB) of several GPU (A100) E.S. loaded on the fly from the disk deep neural network architectures.
Mw and activations MA (in %) and total
of several deep neural network architectures.
model
CPU
GPU
disk
model
Mw
MA
Myet
ResNet 18
9.32
1.99
18.45
ResNet 18
94.47
5.53
87.11
MobNet v2
9.29
3.51
22.12
MobNet v2
95.70
4.30
121.48
EffNet BO
12.28
5.40
31.22
EftNet BO
95.99
4.01
129.64
ViT b16
65.90
3.98
114.75
VIT b16
99.02
0.98
433.77
Llama 7B
704.28
53.22
23982.74
enable more efficient memory compression of DNNs, particularly for LLMs. In what follows, present the proposed JLCM method, which addresses all of these shortcomings.
METHODOLOGY
Let F denote a trained neural network comprising L parametric layers with weight tensors (W..)e Empirical evidence (see Table 7 from appendix A shows that near identical per- formance can be achieved using the floating point 16 (fp16) format as compared to the default floating point 32 (fp32). Consequently, we note Mier the reference memory footprint (in Megabits) of forward pass of F using fp16. Our goal is to reduce the memory footprint M of the final model F as much as possible. We note a Mat the compression goal. For instance, per-tensor quan- tization in 4 bits (W4) would result in a x 4. More precisely, we would get a < 4 as each scalar value is indeed stored using times less memory, but an overhead occurs from the presence of extra scaling parameters. In what follows, we will design JLCM such that its only hyperparameter will be a. Furthermore, as we focus on memory footprint during runtime, we need to evaluate the cost of each component of deep neural network during inference.
3.1 MOTIVATION: DEEP NEURAL NETWORKS MEMORY FOOTPRINT
At train time, intermediate activations computed during the forward pass need to stored for gra- dient backpropagation during the backward pass. Conversely, at inference time, these activations can be discarded as the subsequent layers are evaluated. Consequently, the memory footprint M of model at inference can be defined as the footprint of the weights Mw and the maximum footprint of intermediate features MA to store simultaneously. The second term MA is not simply defined by the largest intermediate feature, due the presence of skip connections in modern architectures. Nonetheless, it can be empirically computed fairly simply for any given architecture. In Table [ we provide the contributions of the weights and activations the memory footprint for several models. We observe that the weight systematically represent at least 94% of the memory footprint. For in- stance, the compression of the weight values by factor a R 7.2 of Llama 7B enables its loading on 2017 mobile device. We argue that these results motivate the focus on weight compression for memory efficient neural networks.
In Table 2 we show the latency cost of moving tensors from different memory caches. For instance, moving the weights from disk to RAM leads 38.51x x and 450.63 overhead for ViT bl6 and Llama 7b, respectively. Overall, these results emphasize the importance of memory-efficien weight encoding for efficient inference. Consequently, the proposed JLCM method will enable significant inference benefits by unlocking model caching. In practice, the compression objective is given by the size Mer of the trained neural network to run and the target hardware capacities Meapa: Mia. A mobile device with 2Go RAM requires a Llama 7B to be compressed by a = 7.2x in terms of memory footprint simply to fit the device properly.
Maps
In what follows, we assume a given weight tensor W E - (ie. one of the aforementioned Wi) with n. output neurons. Codebook- based memory compression methods suffer from two main shortcomings, namely the granularity problem and the difficulty to jointly learn the codebooks and mappings due gradient instability.
4

---

Published as a conference paper at ICLR 2024
3.2 SOLVING THE GRANULARITY PROBLEM WITH EFFICIENT MULTI-CODEBOOK FORMULATION
The first shortcoming of clustering techniques is that they usually assume single distribution for all weights of the whole tensor. Formally, for a given clustering technique C (e.g. k-means [Loyd (982 ) we transform the weight tensor W into a pair (C, of cluster centers C' and mapping indices such that Wa (C; La,). In other words, the clustering method C' is applied over W, viewed as a ng x n, samples of size The resulting memory footprint is defined as (CC) x 16 + (og(S((")) as compared to the original 16 x S2( footprint (assuming 16-bit representation of the codewords), with (1A) indicating the number of elements a tensor A. As a result, the larger the codebook C, the higher the memory cost.
Per-channel scaling factors: in order to operate finer granularity, we propose to draw in- spiration from quantization through the use of scaling factors. Formally, instead of using larger, common codebook, a first approach consists in sharing a smaller codebook across the weight tensor W with specific per-channel scaling factors. For instance, for per-channel granularity (one scaling factor for each row of the weight matrix), the scaling factor s reads:
(1) This solution leads to better control over the memory footprint Mw (S((C) + ((s)) x 16 + log(S((C))S as we can use smaller codebooks. Intuitively, the memory reduction comes from the fact that the dominant term is ())()) However, this approach still assumes that the different neurons of layer share similar distributions up to scaling term.
W;- = sC; T,)-
Weight matrix reordering: in order to alleviate the aforementioned shortcoming, we propose to use distinct codebooks C₁ rather than scaling factors. In practice, in a naive approach, the mapping I would need to encode both a codebook index (indicating to which codebook this weight shall point to) and value (the associated value in this specific codebook). Such formulation would increase Mw which, in practice, the dominant term, as
the dominant term the cost (W)))))))))( log())(C)) < 4 and S2(W)= n& X no 10000. To circumvent this limitation, systematically assign a codebook to a specific region of the weight tensor. This enables the mapping term 7 to only encode the codebook value, as the codebook index can be derived from the row index of the considered weight. The resulting compressed weight values are defined as
propose to
Wis (Cl 1xt J 443)-
In practice, the number of codebooks and the number of scaling factors are directly derived from a (the compression goal) as follows:
multi-scaling: 2(s)) (16 a log()(C))) x S(WW
16 x a
multi-codebooks: k = (16 a log()(C))) x S(W)
16 x (((C)
In order to set the codebook size I(C),, we propose to simply use 2 J, As result, the proposed coding can leverage the different distributions from different parts of the weight tensor (depending on the granularity), assuming that two closely related neurons (as defined by their row indexes) will have similar weight distributions. This is however not the case in practice, e.g. two neurons far from each other the weight matrix can be similarly distributed. In order leverage these similarities, we propose to re-order the neurons beforehand: formally, in the absence of a subsequent skip cxip-connection, we can re-order the output neurons of layer by re-ordering the input columns of the subsequent layer, as in Chen et al. (2021); Liang et al. (2018); ) Pool & Yu (2021). The re- ordering (a.): is given by a C clustering of the neurons, defined by their corresponding weight values. Formally, we apply C to W viewed as n. samples of size n,, The given clusters are ordered
such that neurons clustered together are contiguous in memory. We update equation 2 and get the final compressed weight values
W = (C) xi (T))
(4)
As a result, the initialization procedure reads as follows: first, we cluster the neurons and re-order them, second cluster the scalar weight values based on the hyperparameter books or scalings. These steps are summarized in Algorithm [ at lines 2 to 4. Stemming from this initialization, we propose to optimize both the codebooks and mappings through gradient descent.
into several code-
5

---

Published as a conference paper at ICLR 2024
3.3 JOINTLY LEARNING CODEBOOK AND MAPPINGS THROUGH GRADIENT DESCENT
In order to jointly optimize the codebook values and mappings, we adapt the sequential, layer per layer optimization introduced in Nagel al. (2020). For a given layer, we learn the two parameters C' and I such that they minimize the similarity loss
| |o((C x softmax(1)) -o
(5)
In this formulation, for each weight in W, softmax(T) denote soft assignment one of the code- book values. In order to fight overfitting (which, given the small size of the calibration set, is a major problem in GPTQ methods Hubara et al. (20TT) while driving this mapping hard assign- ment, we propose two regularization terms: the first one, II
|IC softmax WII Intuitively, Li encourages the final weight tensor to be similar to the original weight values. This a different solution to achieve similar behavior as in AdaRound where the optimized value is bounded by the quantized space step-size. The second one, specifically enforcing hard assignment, is defined as (T) 1 - 12 softmax( 11 The resulting total objective L reads
[(C,I) - -
-x x1 CI(C,T) + XI2(I)
In order to keep the solution simple, we only use a single hyperparameter to control the mapping sharpness. Such approach is standard in post-training optimizations Nagel et al. 2020): Li et al. [20I]] Wei et al. (2022); ; Liu et al. (2023) and we use the same scheduler as proposed in the work of Nagel et al. (2020). Note however that, practically speaking, in this state, the optimization fails.
In what follows, we delve into shortcomings of the gradient computation and propose an effective
solution for joint learning the codebooks and mappings.
3.4 ENABLING PROXIMAL SEARCI USING AN ALTERNATIVE GRADIENT TERM
During the backward pass, we compute the gradient updates, as
)(x0|)) x softmax(I)&
8C -[(F(()-((x11(3 081 (8) fx)11 x (3sotimaaa)
01 softxoftm
01
a(C a(Cxsulmax(I))
The first gradient update in equation well suited for optimization. Intuitively, It corresponds to the grouping of the standard weight update through stochastic gradient descent filtered by the soft mapping softmax(T), (T). However, on the flip side, we can see that the gradient update term in the second row of equation D is not adapted to the task. As T is learned, the goal is to assign a new index in the codebook C to a specific weight value. In other words, a large magnitude of the al]f(x)-Y113
X implies that a change may be required. However, in the standard chain rule, this term is multiplied by the values of the codebook C, which leads to an update that is proportional the codebook. This favors larger values in the codebook, regardless of the current indexing. For example, if we have an initial value of 0.12 for a weight and a ternary codebook C (-1.23 -0.09 0.17 0.96), with the corresponding mapping I (0 1 0 0). Then, for a negative gradient, it is likely that the indexing should shift from the second and to the third value. In practice, the chain rule would favor the update of the index corresponding to the value 0.96 (illustrated in Figure DD. This is due to the fact that the gradient update term is an increasing function of the distance between two codebook values C2- C,,. To allow for proximal behavior, we argue that we should instead use decreasing function of this quantity.
gradient X(Cxsollmxx
We draw inspiration from ttraight- estimation (STE), a method that was originally proposed for DNN quantization. In this context, STE consists in redefining custom gradient update to sidestep the zero gradients of the rounding function. In this vein, ideally, we would want the gradient update to consider the proximity of the codewords. Consider the following function D:
sign(C Jggn(Crgmmss Cargmux(())) 1.) C₁)
D 1 + 1C (I) - C55
(Tnic)) argmax [.).i
(8)

---

Published as a conference paper at ICLR 2024
where [íc] is the identity matrix, with the size of the codebook C' and the convention that sign(0) 1, with index for the weight value and j for the codebook value, respectively. With that convention, get the following custom gradient updates
0 YII
a(C soltmama))))))) x softmax(I)
8| 811 11
3(Cxsotmax(D)) X DX
01
This gradient update redefinition allows for proximal update of the codebook values, which, in turn, enables the joint learning of the codebooks and mappings (JLCM) by optimizing Equation The proposed approach summarized in Algorithm Appendix B In the following section, we provide an empirical validation of this method, as well as comparison with existing baselines.
equation @
EXPERIMENTS
In order to evaluate the proposed JLCM method, we considered both computer vision and natural language processing tasks and models. All details are available in Appendix C First, we compare multiple methods (clustering method, multiple scaling factors or codebooks) for the initialization, as detailed in Section 3.2 Second, we evaluate each component of the proposed JLCM method through an ablation study. Third, we compare the proposed method to the current state-of-the-art DNN memory compression techniques.
4.1 INITIALIZATION
Table 3: Evaluation of the clustering initialization method for the codebooks with a compression
goal of a 3.9 and codebooks of size 16.
Res18
VIT b16
Eff BO
Mob V2
Res18
VIT b16
Eff BO
Mob V2
Multi-Scales
Multi Collb-Codsesoks
random
0.100
0.100
0.100
0.100
0.100
0.100
0.100
0.100
K-means
64.641
80.572
50.174
9.277
62.536
80.622
46.165
20.222
B. K-means
28.297
78.879
42.582
0.220
4.005
75.272
40.545
0.630
Graph 38.328
72.903
7.965
0.098
0.100
0.144
0.100
0.112
Hierarchical 64.937
80.624
51.773
8.599
64.277
80.670
44.334
22.961
The proposed initialization has three hyperparameters: the compression goal the clustering tech- nique C and the choice of using multiple scales or multiple codebooks. The de facto standard centroid-based method C is the K-means clustering Lloyd (1982) and its iterations, such as bisecting k-means Wang et al. (1998). We also considered graph-based clustering [hi & Malik (2000) and hierarchical clustering Müllner (20TT) In Table D we report our results on ImageNet models. Our observations are two-fold. First, the clustering method plays crucial role, as random clustering leads to near accuracy for all the tested networks. Second, the hierarchical and k-means clus- tering achieve the highest performance, which can be explained by the fact that, for scalar clustering, they tend to behave similarly. Still, it appears the hierarchical clustering leads to slightly higher performance over the tested configurations. Consequently, we will use this initialization scheme for the upcoming experiments. Furthermore, it appears that the proposed multi-scaling approach performs best on convolutional layers. On the flip side, the multi- hashlists variants are more effec- tive on conneceede layers. Intuitively, we suggest that relative scales are more important for convolutional 2D kernels while scalar values nuance is more important for cully-connected layers. Consequently, for the remainder of the article, JL.CM will use multi-scalings for CNNs architectures and multi-hashlists for transformers.
4.2 ABLATION STUDY
In Table we evaluate the influence of each component of the optimization phase in JLCM. More specifically, learn C means that only update the codebook during optimization by applying the
7

---

Published as a conference paper at ICLR 2024
Table 4: Ablation of the optimization components of JLCM for a compression rale 7.5.
learn C
learn C' &
Proximal search
for ResNet 18
acc for ViT
X
X
20.555
60.691
X
X
22.149
63.912
/
X
20.343
57.800
/
41.779
69.996
Table Comparison to state-of-the-art compression techniques on ImageNet.
architecture
method
Compression rate (a)
optimization
integer
DFQ
5.33 (W3/A16)
0.442
SQuant
5.33 (W3/A16)
19.532
PowerQuant
5.33 (W3/A16)
3.909
RED+
5,33
X
14.124
JLCM (init)
5.33
X
41.133
Resnet 18
JLCM (init)
7.5
20.555 X
AdaRound
5.33 (W3/A16)
19.634
BrecQ
5.33 (W3/A16)
52.943
NUPES
5.33 (W3/A16)
x
53.000
JLCM
5.33
62.939
JLCM
7.5
41.779
DFQ
5.33 (W3/A16)
11.678
SQuant
5.33 (W3/A 16)
13.804
PowerQuant
5.33 (W3/A16)
/
75.330
RED+
5.33
75.743 X
JLCM (init)
5.33
X
76.316
VIT b16
JLCM (init)
7.5
60.691 x
AdaRound
5.33 (W3/A16)
61.154
BrecQ
5.33 (W3/A16)
73.123
NUPES
5.33 (W3/A16)
X
77.231
JLCM
5.33
X
80.558
JLCM
7.5
69.996
first row update in Equation equation This leads steady performance improvement. Fur- thermore, jointly optimizing the codebook and mappings (learn C' & T) by applying the whole equation leads decrease in performance: this is due to the limitations of the naive gradient update pinpointed in Section 3.4 Nevertheless, joint learning can be managed by using the pro- posed alternative gradient update term (Proximal search, equation R which dramatically enhances the accuracy of the compressed networks in both configurations. In what follows, we show that this framework allows to significantly outperform existing methods for memory compression of DNNs.
4.3 COMPARISON TO STATE- OF- THE-ART METHODS
In Table we report a comparison of JLCM to state-of- ofute-ofthhe-rtt post-training compression techniques on ImageNet. We distinguished convolutional neural networks (ResNet 18) and vision transformers (ViT) as well as the data usage (data-free (white) v.s. data-driven (light gray) methods) and report results at two different compression goals, 5.33 (float + int3) and 7.5 (float 16 - ~ int2), On CNNs, it appears that the data-free JLCM initialization already vastly outperforms other data-free methods Nagel et al. (2019); Cong et al. (2022); Yvnne et al. 2022 2023c) as well as AdaRound Nagel et al. (2020) at equal compression goal. This result maintained in the data-driven text, where JLCM improves the previous state-of th- by 9.989 points. On ViT b16, PowerQuant and RED++ already achieve strong results: consequently, improving these results is more chal-
8

---

Published as a conference paper at ICLR 2024
Table 6: Evaluation on Generative Al models.
(b) Average common sense reasoning performance of
Llama 7B (original score: 56.140).
method
compression a
acc
(a) CLIP Score of stable diffusion model (origi-
SQuant
5.33 (W3/A16)
32.725
nal score: 28. (8.5353) 2.0 on DiffusionDB.
LLM.int8(
2.14 (W8/A8)
55.208
method
compression
acc
PQuant
5.33 (W3/A16)
33,466
RED+
5.33
36.430
DFQ
3,9 (W4/A16)
25.369
OPTQ
5.33 (W3/A16)
49.938
SQuant
3.9 (W4/A16)
26.237
AWQ
5.8
47.678
PQuant
3.9 (W4/A16)
27.510
SqueezeLLM
5.8
48.169
RED+
3,9
27.498
JLCM (init)
3.9
56.153
JLCM (init)
3.9
28.471
JLCM (init)
5.33
45.314
JLCM
5.33
55.940
JLCM
7.0
53.081
JLCM (+ REx)
6.95
56.002
lenging. Still, JLCM manages to reach over 99% of the original model accuracy its data-driven variant. This corresponds to 3.327 points improvement over Yvnnec et al. (2023b).
Table 6a and db we report our results on larger, more recent generative models. Our results show that JLCM offers significant improvement in the data-free set-up. For instance, on the Stable Dif- fusion model, JLCM preserves 99.8% the original performance, which is 3.4% higher than the previous state-of-the-ee method (RED++). Similarly, on Llama 7B, observe 8. 8.884% improve- ment. This can be attributed to the finer-grained weight compression, which enables JLCC to better address the challenge of outlying values encoding, discussed by DDttmers al. 2022) In the data-driven setup. JLCM further increase the gap with previous techniques, which struggle to scale large models. Furthermore, on Llama, JLCM reaches 3. 3.34%% higher average score common sense reasoning tasks as compared to OPTQ Frantar et al. (2022) with 31 31.33% higher compres- sion rate. In addition, as compared to SqMeezeLLM Kim et (023). JLCM not only achieves higher fidelity but at higher compression rates which can be attributed to our more effective usage of codebooks and proximal learning. On top this, when JI.CM is combined with outlier specific compression from REX vinee et al. (2023a) (also used by SQueezelLLM) we reach the original performance similar compression rate, which is a major improvement over existing methods. Last but not least, also provide qualitative results diffusion models to complete this score-based evaluation (see Appendix Dj.
CONCLUSION
In this work, we pinpointed two major drawbacks of existing DNN memory compression techniques: the granularity (i.e. the ability to multiple codebooks per tensor with mapping overhcad) as well as the difficulty to optimize the mapping through stochastic gradient descent, as the process fa- vors extreme codebook values proximal ones. To cireumvent these limitations, a novel efficient codebook mapping which leverages tensor structure. To do so, we proposed to re-order neurons in order group them by distribution similarity. We then proposed either ap- ply different scaling factors (which, experimentally, works best for convolutional layers), or encode each group using separate codebook (which is more efficient for dense layers, e.g. transform- ers), removing the need for codebook indexing that causes memory overhead on the value mapping. We then propose joint learning of the codebooks and weight mappings (JL.C)) method that draws inspiration from gradient-based post-training quantization techniques. Last but not least, we pro- pose an alternative gradient update rule that allows a proximal search within codebook values. The proposed method significantly outperforms other approaches for memory-efficient inference of deep neural networks. As an example, JLCM compresses a Llama-7B model to on a 2Go device while retaining 95% of its original performance, which represents massive improvement previous -fate-of-the-ane and an important stepping stone for the deployment ever-growing deep neural networks.
introduced

---

Published as a conference paper at ICLR 2024
REFERENCES
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In AAAI, volume 34, pp. 7432-7439, 2020.
Liang-Chiel Chen et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. TPAMI, pp. 834-848, 2017.
Tianyi Chen, Bo Ji, Tianyu Ding, Biyi Fang, Guanyi Wang, Zhihui Zhu, Luming Liang, Yixin Shi, Sheng Yi, and Xiao Tu. Only train once: A one-shot neural network training and pruning framework. Advances in Neural Information Processing Systems, 34:19637-1961,, 2021.
Wenlin Chen, James Wilson, Stephen Tyree, Kilian Weinberger, and Yixin Chen. Compressing neural networks with the hashing trick. In International conference on machine learning, pp- 2285-2294, PMLR, 2015.
Christopher Clark, Kenton Lee, Ming-Wor Wei Chang, Tom Kwiatkowski. Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint 1rXiv:195..1000 2019.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the ai2 reasoning challenge. arXiv preprint arXiv:1803.0544 2018.
Guo Cong al. Squant: On-the-fly data-free quantization via diagonal hessian approximation. ICLR, 2022.
J. Deng, W. Dong, et al. ImageNet: A Lcrge-Scale Hierarchical Image Database. In CVPR, 2009.
Tim Dettmers, Mike Lewis. Younes Belkada, and Luke Zettlemoyer. Llm. int8 O: 8-bit matrix multiplication for transformers scale. arXiv preprint vrXiv:2288.07332 2022.
Jacob Devlin, Ming-Was Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 18.00488 2018.
Alexey Dosovitskiy et al. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
Elad Eban, Yair Movshovitz-Atiaa, Hao Wu, Mark Sandler, Andrew Poon, Yerlan Idelbayev, and Miguel A Carreira- Perpinán. Structured multi-ha hasting for model compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11902-11912, 2020.
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Optq: Accurate quantization for generative pre-trained transformers. In The Eleventh International Conference on Learning Representations, 2022.
Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28, 2015.
Kaiming He, Xiangyu Zhang, et Deep residual learning for image recognition. In CVPR, pp- 770-778, 2016.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXi::2104.08718 2021.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. NeurlPS, 2014.
Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Accurate post training quantization with small calibration sets. International Conference on Machine Learning, pp. 4466-4475. PMLR, 2021.
Mojan Javaheripi, Jung- Woo Chang, and Farinaz Koushanfar. Acchashtag: Accelerated hashing for detecting fault-injection attacks on embedded neural networks. ACM Journal on Emerging Technologies in Computing Systems, 19(1):1-20, 2022.
10

---

Published as a conference paper at ICLR 2024
Yongkweon Jeon, Baeseong Park, Se Jung Kwon, Byeongwook Kim, Jeongin Yun, and Dongsoo Lec. Biggemm: matrix multiplication with lookup table for binary-coding based quantized dnns. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1-14. IEEE, 2020.
Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W Mahoney, and Kurt Keutzer. Squeezellm: Dense-and-sparse quantization. arXiv preprint arXiv:2306.07629, 2023.
Raghuraman Krishnamoorthi Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.0823 2018.
Woosuk Kwon, Sehoon Kim, Michael W Mahoney, Joseph Hassoun, Kurt Keutzer, and Amir Gho- lami. A fast post-training pruning framework for transformers. Advances in Neural Information Processing Systems, 35:24101-24116 2022.
Tuanhui Li, Baoyuan Wu, et al. Compressing convolutional neural networks via factorized convo- lutional filters. In CVPR, pp. 3977-3986, 2019.
Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. Brecq: Pushing the limit of post-training quantization by block reconstruction. NeurIPS, 2021.
Ling Liang, Lei Deng, Yueling Zeng, Xing Hu, Yu Ji, Xin Ma, Guoqi Li, and Yuan Xie. Crossbar- aware neural network pruning. IEEE Access, 6:58324-588 01834,-533,,, 2018.
Jiawei Liu, Lin Niu, Zhihang Yuan, Dawei Yang, Xinggang Wang, and Wenyu Liu. Pd-quant: Post- training quantization based on prediction difference metrie. In CVPR, pp. 24427-2443) 2023.
Stuart Lloyd. Least squares quantization pem. IEEE transactions on information theory, 28(2): 129-137, 1982.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering, arXiv preprint arXiv:1809.022 2018.
Daniel Müllner. Modern hierarchical, agglomerative clustering algorithms. arXiv preprint 2011.
Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? adaptive rounding for post-training quantization. In ICML, pp. 7197-7206. PMLR, 2020.
Markus Nagel et al. Data-free quantization through weight equalization and bias correction. ICCV, pp. 1325-1334, 2019.
Jeff Pool and Chong Yu. Channel permutations for n:m sparsity. In M. Ranzato, A. Beygelz- imer, Y. Dauphin, P.S. Liang, and Wortman Vaughan (eds.), Advances in Neural In- formation Processing Systems, volume 34, pp. 13316-13327. Curran Associates, Inc.,
2021. URL htttps:/ neurips. cc/paper. files/paper/2021/ File/
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. NeurIPS, 28:91-99, 2015.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models, 2021.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adver- sarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106. 2021.
Mark Sandler, Andrew Howard, et al. Mobilenetv2: Inverted residuals and linear bottlenecks. In CVPR, pp. 4510-4520, 2018.
11

---

Published as a conference paper at ICLR 2024
Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions pattern analysis and machine intelligence, 22(8):888 905, 2000.
Mingxing Tan and Quoe V Le. Efficientnet: Rethinking model scaling for convolutional neural networks. ICML, pp. 6105-6114 14, 2019.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302 13971, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurlPS, 2017.
James Ze Wang, Gio Wiederhold, Oscar Firschein, and Sha Xin Wei. Content-based image indexing and searching using daubechies' wavelets, International Journal on Digital Libraries, 1:311-328, 1998.
Zijie Wang, Evan Montoya, David Munechika, Haoyang Yang. Benjamin Hoover, and Duen Horng Chau. Large-scale prompt gallery dataset for text-to-image generative models. arXiv:2210. 14896 [es], 2022.
14896
Xiuying Wei, Ruihao Gong, Yuhang Li, Xianglong Liu, and Fengwei Yu. Qdrop: randomly dropping quantization for extremely low-bit post-training quantization. ICLR, 2022.
Edouard Yvinec, Arnaud Dapogny, Matthieu Cord, and Kevin Bailly, Red: Looking for redundan- cies for data-free structured compression of deep neural networks. NeurIPS, 2021.
Edouard Yvinec, Arnau Dapogny, Matthieu Cord, and Kevin Bailly. Red++: Data-free pruning of deep neural networks via input splitting and output merging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4563):366433676 2022.
Edouard Yvinec, Amaud Dapgony, Matthieu Cord, and Kevin Bailly. Rex: Data-free residual quan- tization error expansion. NewrlPS, 2023a.
Edouard Yvinec, Arnau Dapogny, and Kevin Bailly. Nupes Non-uniform post- post-training quantiza- tion via power exponent search. arXiv preprint arXiv:2308.0560 2023b.
Edouard Yvinec, Arnaud Dapogny, Matthieu Cord, and Kevin Bailly. Powerquant: Automorphism search for non-uniform quantization. In ICLR, 2023c.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can ma- 2019.
chine really finish your sentence? arXiv preprint
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:205.00066 2022a.
Yichi Zhang, Zhiru Zhang, and Lukasz Lew. Pokebnn: A binary pursuit of lightweight accuracy. In CVPR, pp. 12475-12485, 2022b.
Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Chris De Sa, and Zhiru Zhang. Improving neural network quantization without retraining using outlier channel splitting. In ICML,, pp. 7543-7552, 2019.
12


Reviews:
[{"review_id": "Gscr8RywTf", "reviewer": "Reviewer_kEDS", "summary": "This paper compresses memory footprint in deep neural networks inference by a codebook-based approach, mainly solving the granularity problem and the training problem suffered by the previous works. This paper tried two methods to solve the granularity problem, setting per-channel scaling factors with one codebook or using multiple codebooks with weight matrix reordering. To enable proximal search thus solving the training problem, this method uses custom gradient updates inspired by STE.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. In the experiment section, this method shows advantages over baselines.\n\n2. This paper provides a detailed analysis of memory usage in different settings.", "weaknesses": "The paper has some unclear statements that require more explanation:\n\n1.  The settings for the number of codebooks and the number of scaling factors should be further explained. Although the authors mention in the paper that these values are determined by compression goal, the process needs more details and therelationship with the weight distribution needs to be explained as well.\n\n2. This paper does not explain how to cluster the neurons. There is also no data to prove that “two neurons far from each other in the weight matrix can be similarly distributed”.\n\n3. The meaning of the X-axis in Figure 1 is confusing. For different curves, the X-axis seems to have different meanings. These should be marked on the figure. \n\n4. For the LLM experiments in Table 6, how are the results compared with more recent works, e.g., SqueezeLLM, AWQ, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TyoOYeozXS", "reviewer": "Reviewer_P3Eg", "summary": "The authors propose a series of improvements to codebook-based weight compression schemes for deep neural networks with the aim of enabling larger models to fit in the limited on device storage for accelerators like GPUs. Their method is based on three core changes relative to existing methods. First, they apply a neuron re-ordering to group weights with similar distributions. This allows for finer-grained application of codebooks to weights without increasing the storage overhead of the compressed representation. Second, they propose to jointly learn the codebook and codebook mappings, similar to gradient-based post-training quantization schemes. Lastly, they modify the gradient update for the codebook mappings to enable more effective optimization.", "questions": "The neuron permutations you use in your method remind me of the channel permutations that are used by N:M sparsification methods [1]. Drawing this connection could be interesting in your related work section.\n\nIn a number of places you use “Go” as a unit - is this intentional? Was “Go” supposed to be “GB”?\n\nIn Table 2, I’m curious to understand how you implemented offloading of parameters to disk. Are transfers from disk to GPU pipelined with computation to hide as much transfer latency as possible? What batch sizes were used for each model?\n\n[1] https://proceedings.neurips.cc/paper_files/paper/2021/hash/6e8404c3b93a9527c8db241a1846599a-Abstract.html", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Despite my lack of experience with the topic of this paper I found the text reasonably easy to follow. I think the paper is well written and the methods appear to be sound to me. In particular, the impact of part 3 of the proposed method (improved gradient estimator) seems to be considerable based on the ablation results presented in Table 4.", "weaknesses": "I am not an expert on compression of neural network weights for storage optimization but I did not identify any particular weaknesses in the methodology. The technique appears to be reasonable and the results appear to be sound based on my review of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BFiqQlv2lk", "reviewer": "Reviewer_fJGw", "summary": "The novel ALAM framework introduced in this paper leverages average quantization and a simple sensitivity calculation method to reduce memory usage in LLMs without affecting their training efficacy. This approach minimizes gradient variance by compressing activations to their group average values, allowing for effective compression to less than 1 bit. Additionally, ALAM employs a novel sensitivity calculation that uses the L2 norm of parameter gradients, greatly reducing memory overhead. In testing, ALAM achieves up to a 12.5× compression rate for activation memory in LLMs without sacrificing accuracy.", "questions": "1.It would be better to explain why the second term $log(\\Omega(C))\\Omega(W)$dominates. It seems a common sense but I cannot see it in the context.\n\n2.Can you provide some evaluation results in inference or training or finetuning performance in efficiency (throughput or energy efficiency)?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.The paper applied a re-order method on neurons to reduce the memory overhead, which is quite novel.\n\n2.This paper jointly optimized the mapping and codebooks, and the proximal search method can be used to the modified gradient update method.\n\n3.The evaluation demonstrates the effectiveness.", "weaknesses": "1.It is hard to understanding the background part, especially the challenges of conventional approach. It would be better to provide a straightforward illustration with a figure or algorithm.\n2.Some typos. In the beginning of sec3.2, “the” should be “The“; “ram” in sec 3.1 should be “RAM”; Something is missing in Equation 3.\n3.The metric in evaluation part is accuracy and compression ratio. Can the proposed idea bring more benefits like accelerating the inference/training/finetuning?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)