Auto-tuning Matrix Multiplication and Convolution for Deep Learning on CPUs
Authors: Changbo Chen
Haoyu Chi
Venue: NeurIPS 2021 Submitted (2021)
Keywords: deep learning compiler
ideal cache model
loop tiling
auto-tuning
matrix multiplication
convolution

Abstract: Deep learning (DL) compilers have emerged aiming to reduce the gap between abundant, fast-growing DL models and the lag of high performance implementations of these models on diverse hardware devices. In this work, we introduce several optimization strategies, combining analytic ideal cache models with machine learning models trained with real hardware measures, and integrate them into a unified auto-tuning framework, called AutoMCL, to improve the performance of DL compilers on both the operation level and the end-to-end model inference.  We evaluate AutoMCL and compare it with state-of-the-art on multiple CPUs. End-to-end evaluations show that AutoMCL outperforms TensforFlow on fully connected and convolutional neural networks with respectively a geometric mean of $9.29\times$ and $1.54\times$ speedup. Over the baseline AutoTVM, on average, AutoMCL achieves respectively $1.37\times$  and $2.16\times $ speedup in inference and optimization time for fully connected neural networks and gains $2.55\%$ performance improvement in inference for convolutional neural networks with $1.91\%$ more optimization cost.

Auto-tuning Matrix Multiplication and Convolution for Deep Learning on CPUs
Anonymous Author(s)
Affiliation
Address email
Abstract
Deep learning (DL) compilers have emerged aiming to reduce the gap between abundant, fast-growing DL models and the lag of high performance implemen- tations of these models diverse hardware devices. In this work, introduce several optimization strategies, combining analytic ideal cache models with ma- chine learning models trained with real hardware measures, and integrate them into a unified auto-tuning framework, called AutoMCI, to improve the performance of DI compilers on both the operation level and the end-to-end model inference. We evaluate AutoMCL. and compare it with state-of-the-ar on multiple CPUs. End-to-end evaluations show that AutoMCL outperforms TensforFlow on fully connected and convolutional neural networks with respectively geometric mean of x9 and 1.54x speedup. Over the baseline AutoTVM, on average, AutoMCL achieves respectively 1.37 and 2.16x speedup in inference and optimization time for fully connected neural networks and gains 2.55% performance improvement inference for convolutional neural networks with 1.91% more optimization cost.
2
4
8
10
11
12
13
14
Introduction
18 Deep learning models have found wide applications in image and sound recognition, natural language translation, game playing, etc. The success of deep learning benefits greatly from the accessibility of DL frameworks, such as TensforFlow [4) PyTorch [19] and MXNet [8) which not only ease the
17
18
19 burden of coding but also provide high performance supports through efficient low-level libraries,
20 such as Intel oneMKL [2] or NVIDIA cuDNN B. However, it is difficult to make the library development, which requires tremendous manual engineering effort entangled with hardwares and often takes months or even years to finish, keep pace with the rapid innovation of DL models. As
21
23 a result, many newly introduced neural networks or operators may lack optimal implementation
24 support on the target hardwares, thus hindering the further innovation of DI. models. To address
25 this challenge, DL compilers (c.g. TVM 9 and TemsorComprehensioos (25)) emerged [16], whose goal is to automatically compile high-level declarations of DL operators into efficient low-level code across various hardware devices, including CPUs, GPUs, FPGAs, and ASICs.
27
28 To make the DL compilers appealing, it is essential to keep their performance competitive or even superior to that of DL frameworks or hand- aandiopt libraries. To achieve this, state-of-the -frthe-art DL compilers, such as TVM and its successor AutoTVM [10], extend the decoupled compute/schedule
29
30
31 principle of Halide 2 to separate target hardware intrinsics from computation description and optimization sequence specification composed of transform primitives to ease the process of high-
33 level optimization, and leverage machine learning to automate low-level optimizations. The success of DL compilers relies on high-quality schedules as well effective searching and learning strategies
35 to find optimal parameters. Recently, new progress have been made on automating the design of schedule primitives, enlarging the parameter space to expose more tuning opportunities and utilizing
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.

---

37
heuristic and learning approaches. in particular reinforcement learning, to explore the parameter space more effectively to find optimal candidates. Among these work, AdaTune [T5]. Ansor [29]. CHAMEL LEON [S. Flex Tensor [30] and Cortex [1] are built on top of TVM while the value function
38
39
40
method 23 and TIRAMISU 6 are respectively based on Halide and the polyhedral model.
41
Most of these optimizations have been focusing on the loop level optimizations, such as loop tiling, loop split and fuse, loop unroll, loop reordering, vectorization, etc. The algorithm level optimization, on the other hand, is hard to automate and still requires human's expertise. Moreover, while enlarging the tuning space may potentially include better candidates, it also calls more effort to find the optimal solution and often leads to getting suboptimal solution in limited budget. Thus, it remains a great challenge to prune the parameter space efficiently to avoid unnecessary exploration, which may also help increase the chance of optimal solutions to be picked earlier. A purely analytical modeling approach for optimizing convolutions 17 was recently proposed towards this direction.
42
43
44
45
46
47
48
49
In this work, we propose several new strategies aiming to leverage both analytic model and machine learning to generate more efficient code in shorter compilation time targeting on the CPU platforms, the ubiquity of which implies that great number of users can benefit from such improvement. Our main contributions are three-fold:
50
51
52
53
We introduce new strategies for initializing and filtering the tiling size space for matrix multiplicatiin and convolution based on analytic models.
54
55
We introduce several new competitive schedules for matrix multiplication and convolution in both algorithm and loop level enlarge the schedule space.
56
57
We integrate the proposed strategies into a new auto-tuning framework called AutoMCI. which leverages TVM's frontend computational graph optimization and backend code generation functionalities. We conduct operator level and end-to-end evaluations showing that the overall performance of AutoMCL is superior to MVM in both inference and optimization time on typical fully connected or convolutional neural networks.
58
59
80
61
2
Background
62
The operations matrix multiplication and convolution appear widely in many deep neural networks and improving their performance is critical to speed up the the training and inference. Matrix multi- plication has been implemented CPU in many basic linear algebra libraries, such ATLAS (27), GotoBLAS [3] and Intel oneMKL. (2). The convolution operation was also implemented on CPU in several standalone libraries, such Intel oneDNN (.. In the context of deep learning, there is strong demand to deploy a well-trained model to a great variety and amount of devices such that the model can infer in real time on the target hardwares. This offers new challenges and opportunities for auto-tuning the performance of these two operations for fixed size input tensors 28 [8]
64
65
66
67
68
69
70
71
Matrix multiplication and 2D-convolution operators. Mathematically, the matrix multiplica- tion operator matmul takes two matrices AMxk K and Bкх input and computes their prod-
73 uet matrix CMxN. N. In this paper, we would assume that the operator takes two matrices DM K
74 and WNNK and computes a new matrix CA by Cūj := Σk- Au Bzk- The cooooouuuu
75 operator comv2d, in its simplest form, takes a tensor D of dimensions B x IC DH DW,
76 a tensor W of dimensions OC x IC x KH x KW, two stride sizes 81, 82, and produces (DH K H)/s1 1 and OW- = (DW - KW)/ 882 + 1. Each element of C is computed according to the rule W k. In general, it may also takes two padding sizes PH, PW and two dilation sizes di, d2 and produces a tensor of dimen- sions B x OC x OH x OW, where OH = (DH + 2PH (KH 1)d1 1)/81 1 and
77 a tensor of dimensions B X OC x OH x OW, where OH
78
Chosa := Eizo >K kWW- Db....svtky.eezee
79
Lk.=0
80
81
OW = (DW + +PW - (KW 1)d2 - 1)/82 +
82
Idell cache model. The ideal cache model was introduced in 12 for studying the cache complexity 84 of algorithms. It assumes that the computer has two-level memory hierarchy consisting of an ideal
83
85 cache of words with cache line size C, where >> C, and an arbitrarily large main memory. To
86 access word in main memory, it first searches it in cache. If the word does not reside in the cache,
87 a cache miss occurs and a eache line containing the word loaded into the cache from the main memory. It assumes that the cache is fully associative and the line with furthest access in the future
2

---

89 will be replaced if new data is loaded into a full cache. The cache complexity counts the number of
90 cache misses. For instance, the (worst) cache complexity for scanning n words continuously stored in
91 an array is [n/C] + 1. In general, the cache complexity of an algorithm operating on tensor largely
92 depends on the layout of the tensor and the ordering for visiting the dimensions of the tensor.
Components of AutoMCL
93
94 We design a few optimization passes and evaluate the effectiveness of each optimization strategy 95 individually and only append experimentally proven working optimizations our framework. Fig. provides an overview of the framework, named AutoMCL.
Schodule
Opreator Declaration
Computational
Neural Network Model
Space
Gruph Optimization
Best Secheduue=CCnfiirattt
Optimized Schedule Space
Hardware Measurement
Historical Data
Performance Model
Configuration Space
Initialization
Configuration Space
Selected Configmtion
Code
Filtor
Generation
Schishule+Confiiuuatioon Space
Figure 1: Flow of AutoMCL with the new strategies introduced in this work highlighted.
96
Enlarging the space of schedules. The DL compiler TVM provides two default computes for the matrix multiplication operator, namely DNMM and RPMM and one default compute CONV for gen- eral cocoovoluuio.. We introduce another four alternative computes TMM, TTMM, DPMM, LPMM for matrix multiplication and two alternative computes Im2coIDNMMM33 and Im2coIRPMMM for convolution by converting convolution matrix multiplication in im2col manner. Table I summa- rizes the specification of each compute for matrix multiplication. The computes for convolution can be found in the supplemental material. We manually write schedule template for each new compute and improve the default schedule templates for DNMM, RPMM, CONV respectively as DNMM332 (single-level tiling double-level tiling), RPMMV (adding missing vectorization for some loop) and CONVOpt (loop reordering according to the cache complexity analysis in Theorem and its remark).
98
100
102
103
104
105
106
Table I: Compute specification for matrix multiplication
Name
Specification (M, K,, N₂ are parameters.)
TMM
Cy == K-1 Dy, Wa.k
TTMM
When = Wa,ki M Cy,z == K-1 Dy, Wh,z
DNMM
CCy,z,k Lk,=0 -K/K,-1 D Dykto W Ko Cy -1=>k-
Ek. CC CCy,z,k
LPMM
PDy..kii: = D DysaMetur ds Cy = Σk =0 PD // Jay mod M. * W
RPMM
PW W.zkk. W. Wa.aNetai Cz := Σk= K-1
Dy. PW
2/N,kk mod N₂
DPMM
DD Nook = D D M. Mi PWa..k.n := -..
Cy Ek- PD g/Ma.k.y mod M. PW k/N,,k. mod N.
107 We analyze the cache complexity with the ideal cache model for each schedule template, stated as Theorem and Theorem 2 whose detailed proof can be found in the supplemental material. Note 109 that all the nested loops will be tiled in the schedules. This would lead to a better cache complexity if the data required for computing tile all fit in cache. This assumption depends both the tile and cache size but should not depend on the input tensor size (with the kernel sizes as an exception since
111
112 they are usually small). Table 2 and summarize the assumptions.
113 Let Vz be the length of vectorization, C2 be the cache line size, be the cache size, and Dr be the size of tensor data type in bytes. Let V., := Vi/De. C.. == Cr/ De. Zw == Z/ De.
3

---

115
Theorem 1. Assume that Tm (M,, Ni)
C. (M,, K, N, Mr, Ke, N₂) for each schedule is listed as below:
<
Zu and M, Ki|K, N the cache complexity
TMM :
Mr
Kx
+
K
+
No
(1
Kx
+
+ (1 ( (1x1+1))
Mi No
Cw
Ke
Cu
Ke
TTMM:
K N.
Kt
+
No
(1
+ 1)
R. NG
+ M. N.
M. N.
(Me
(T
+1
R.
K
+
Kr
(r
C
Nr
++1
K
+ Mr (1 + +1))
DNMM:
Mr No
M. N.
K
1AMe + No)
C..
KL
+
+
Me
N.Ki + 1) + M1 (T + 1))
Cin
LPMM: :
Mr
M
CT KM.
C.
11 + M(I
C. 1
M
Cie
+
+ M N.
Mr
(TaM +1
K
No
Ke
+
+
KrMe
1 + 1))
M. N.
Ki
C
C
RPMM: :
N.
N (T
K.N.
CN
+
([
++1
McNi
Ci
+
M
No
N
Mr
(1
+1
+ K
Ri
MM (T
Ke
+
+
KeNe
Co
+ +1))
DPMM : NE
K.
Ciw
+11 + M
(I
[+1] 1)
N:
N.
Cs
+
+ N
Cae
+++
+ 1+
M N.
[M
[[a
+ +1)
K
Ky
+ 1 1+
Ki
+
1))
Cs
Mr No.
Cin
117 Theorem 2. Assume that (M,, Kr, N¹)= <
Zo Ee and OW; IC,JI,,
the cache com-
118 plexity for CONVOpt is:
B * (DH + 2PH)) IC * (DW + +2PW) 1+1+ B DH 1C
DW C
1
C
1C IC,
KH + KW IC,,
+0C*
([
+1)
(I)
Cw OC
+IC KH KW
OC
1+1++++++++++
B (B+
OC
OH
OW, * ([
OW
OW, + OC, 1+1) 11)
OC,
Co
OC,
+B+ OC
OW
([ 0Ct +)
OO
IC KH
KW
OC
OW.
C.
OC
* +CC + OO * OW
(82
(OW - 1) + (KW - 1) d2 + 1 +1
+B+ 8
KH
OC,
OW.
CN
OC
+OH x OW +( OC
OC
00,,
+ +B+ 8
1 1) +
B+OH + OW
* *[
OC
C
OC,
C
OW
OW
+ +B * OC OH
([
11111 1);
OW.
C
119
and the cache complexity of Im2col-CONV is:
IC (DH + 2PH) * (DW + 2PW),
B 10 DD DW
1+1+
1+1)
C..
C
B OW + +CC
IC. + KH KW
2+ IC
KH KW IC: 1 +1)
.[
+ +
(i
1C,
C.
IC:
On
+B OH OW IC KH * (0W,- - 1) 82 (KW 1) de
111)
C
+C.. OH * OW, IC KH & KW, OC, OW, 1C, * KH * KW,OCK,
OW
OW
OCt (001 1+1)
+B OC OH
+1)+
B BHOH OW
OC. (I
OW.
C
C..
120
Remark 1. For the cache complexity of CONV in TVM, we only need to replace the bold part in
Table with OC, IC, * KKW +1)+ KW IC, (IOC,/Co 1 + 1) and (7) in Theorem 2 by
KW
121
Co
122
OC * IC * KHH
KW
+ 1). It is usually larger than that of CONVOpt for the same tiling size. Co
123
Learning to choose schedules. We first evaluate the performance of each schedule on dataset
124
consisting matrices with sizes ranging from small to large. The experiments, reported in Section 4
125
show that each one can be exclusively the best for certain types of sizes. We then choose the top four best performed schedules as candidates and train boosted tree model by Xgboost [7], with the matrix size as input feature, to automatically select the best one for a particular size.
126

---

Table 2: Values of Tm for different schedules for matrix multiplication (from top to bottom: TMM, TTMM. DNMM. LPMM, RPMM, DPMM)
T [M, KG,ND No)
MxT
Kx
+ + (T(
+ ( No + 1)
Cw
Ki(T
N.
+1 max
(N([ K2 + 1), Mc(l K 1 1) M E. +1)
Ci
Mc(1
N.
+ 1) + max
(M No + 1), (M, + No) K. +1)
Ciu
1 + max
Mi1 + Mr
Ow
McN
Ce
+ max (Me( + No
Ce.
K1
+ 1) + KiM
+
1 + max
[[N+ + Ne,
M,Ne
1 max
N: Mc
+ 1),
Ma[
K:
+ +
K.
+
C=
C=
Cx
C
+ max
Mr Cw
+ Mr,
Ne
No.
McNe C
+ max
MM
No Cw
+ 1),
Ke Ox
+
K.Ne
+ 2
Cw
C
Table 3: Values T. for convolution schedules (top: CONVOpt, bottom: Im2col-CONV)
IC,, OCL)
OC, ( KH-KW-IC,
+ 1) + KH KW IC,
(1 OC
+ +1),
GN
C
([ + 1) KH IC, (1))(OW,.1)++KK
11,dg++1
max
+KH & KW * +1,, ([ [C
C..
C.. + 11,,
OW,,0
C
+11+ OW, ( + 1),
Cu
OW, (TOC,) + 1) + 00, (
OW:
+1)
C
OW, * ([ [[HH.
+ +1)+ +11++C,, ([ [OW,, +K -1)dd:+ + 1),
C
C
max
2 * OC2+ * ([ ([K .c,, (T..-WW.IC.
Cn 11,))
C..
1) OW, (1 ([oc: 1),
Tm (OW,, * KK * -KW
128
Initializing the tiling size space. Suppose that there are m dimensions to be tiled and the size of each dimension is X₃, i 1. ..... m. Then the number of valid one-level tilings is II X,. which is one billion for X; = 1000. Thus one has to set up reasonable initial tiling size space. For instance, in TVM, there are two basic strategies depending on the tiling size being a factor of X, or a power of 2. Suppose that there are m dimensions XI....,X, ..
129
130
131
132
to be tiled, and each dimension has a nested tiling of levels d,, = 1...., m. Then the initial configure space for the factor strategy is .....(())
133
direct product of the sets G, == {(x ,...,,,!"!!,.......... x(d))) II x(") Xi). I,... m. We adopt this factor strategy for 2D-convolution. For matrix multiplication, we propose more sophisticated strategy, motivated by both the factor strategy of TVM and the analytic model of (2] to balance cache locality and load balancing among parallel threads. This strategy is described by Algorithm [
134
135
136
137
Filtering the tiling size space. Let G(Z,, X,) be the initial tiling size space for the com- pute/schedule pair S), where Z,, Y,, X₁ denote the innermost tiling sizes for the tiled dimensions Z, Y, X. Let TX,,, Yr, X,) be the cache fit formula T- T,.. Let X be the dimension for vectoriza- tion and X, be the tiling size for this dimension. We would only consider the tiling size satisfying both X 2 min(X,, V.) and T < Zw/Cw and filter out the rest ones from G.
139
141
143
Learning to choose optimal configurations. Except for the default schedule CONV of TVM, the configuration space for all the schedules considered in this work is solely formed by different tiling sizes. The schedule CONV has another knob unroll kw to decide whether to unroll the for loop involving the kernel dimension KW. The size of the configuration space in our experiments is usually less than 10, 000 thanks to the initialization and filter strategies. For this moderate size, we find that the rather direct tuning strategy described by Algorithm 2 works quite well in practice.
145
146
147
4 Evaluation
150
We developed AutoMCL on top of TVM (0.6.0) and it will be released in open source. Three Intel CPUs (Intel i7-G9700F, Intel 17-9750H, Intel i9-9900) and one AMD CPU (AMD-RyzenD-3900X are used for evaluation. More detailed hardware information can be found in the supplemental material.
151
5

---

153
We first evaluate each optimization strategy individually based on TVM on randomly generated datasets consisting of tensors of various sizes, in order to see if a particular optimization can speed up either optimization time or inference time. Then we evaluate the whole integrated framework on both the operation and the end-to-end level for typical fully connected and convolutional neural networks. The maximum number of trials for the whole tuning and the early stopping are set respectively as 10, 000 and 400 for most of the experiments. The only exception is the end-to-end evaluation of CNNs, where we set the two numbers respectively 500 and 300.
154
156
158
Algorithm I: InitConfigSpace((.5))
Input:
A compute/schedule pair (O, S) for matmul, the number of parallel threads p. Output: The initial configure space G for tiling.
begin
if S has 1-level tiling then initialize G', Gzs Gy. Gk, Gya respectively as 0; for all factors Py of p do
3
5
Px := p/pr; let Gy and G2 be respectively all the factors of [M/pp] and [N/p=]: == ((,, No) M, Gy. No Gz))
let Gk be all the factors of K; G: = {(1, Mr, N, K.) (Me N1) e Gyr, K2 Ga else if S has 2-level tiling then
8
initialize G', Gz, Gy: Gk, Gyz respectively as 0: for all factors Py of do
10
Ps := p/py:
12
let Gy == [Mo. Mc) MMMMI 1::MM//y]::C..
Gz := ((No. N)): NoNp////p.l: [
Gyz {(Mo. Me, No, N.) (Mo, M,) Gy, (No,N) Gz}
let G2 be all the factors of K;
15
G = {(M,, Mr, No, N,, Kt) (Ma, Mr, No, N,) E Gya. K₁ E G+):
/* Due to limitation
TVM, it is additionally rquired that M, M for LPMM, N,IN for RPMM and M,M,, N for DPMM
16
return G
Algorithm 2: AutoConfig(O,,, m,
Input: The compute/schedule pair (O, S). the configuration space G for (O,S), S), the maximum number of trials m, the batch size n for restarting training, the batch size b for a parallel run.
Output: The optimal configuration.
begin
2
D : 0: 0; randomly pop n configurations from and put in N;
while true do
while N + do
choose b configurations B from N=
in parallel, run the code compiled from the tuple (O, C,,, e B, on hardware; add B examples labelled with (averaged) running timings D; := t [N];
8
if G # 1 and < m then
train ML model with D and predict the running timings of (O, S,c, G; pop the best (shortest predicted timing) configurations N from G;
10
11
else
break:
return the configurations in D with the shortest running time
180
Comparison of different schedules. To make a fair comparison, we create two testing datasets consisting of examples of various dimension sizes for matrix multiplication and convolution. For matrix multiplication, dimension size is chosen in three different scales, with small size in {1,8, 16). medium size in (64, 256} and large size in {1024, 4096), which creates 7 different combinations.
161
163
We remove 5 extreme size cases and add additional 120 examples with each dimension randomly

---

185
taking values in 1..4096. For convolution, we create dataset of the same size (458) as matrix mul- tiplication. The dimensions of each convolution example (DxxICx DH Dw, W Wocx 1XXXIXXXXXXXX
186
167
with stride and padding size randomly take values by the following rule: B E {1.32, 128
TC E (20..... 211], OC E {2 .,214), ). DH DW E {1 (1....26)) KH KW (1,3,5, 7, E (1, p [(K - 1).2j. In addition, we only keep examples with each dimension size less than 4096 in their im2col representations.
168
169
170
171
reports the proportions of examples with the shortest running time or the lowest cache misses meeasrred by the ideal cache model) for different schedules implementing matrix multiplication or convolution. The experiments show that each schedule can be exclusively the best for certain types of
172
173
174
tensor sizes. Here we allow 0.02 tolerance for being the best. Our manually improved schedule DNMM332, RPMMV and CONVOpt indeed work better than their counterparts. Moreover, the real and the theoretical measure correlate quite well for the "top performed" schedules, except for the two based on DNMM, which however have different vectorization dimension from the others.
175
176
perfiead
Exchuivaly Bestpa fattel
Backsdilly Roo parfereed
0.4
Dechuivdly mamtoocht-mmdd-mmaamm
Backudndly Dut casecaaeooeced
0.2
0.0
11649)
1000/M
D1P00
17949
15:4
110M
5.0
<WkV
Figure 2: Performance of different candidate schedules for matmul and conv2d.
177
178
Evaluation of automatic schedule chosen. With performing exclusively the best on at least 5% of the dataset as criterion, four "top performed" schedules DNMM332, RPMMV, LPMM, TMM are selected for matmul and three are selected for conv2d. For matmul, we adopt Xgboost to automatically choose the best schedule among the four for given problem size. The dataset is the same as the one in last subsection, from which 40 randomly chosen examples are reserved for the testing dataset and the rest for the training dataset. Fig. 3 reports the performance on the testing dataset, where AutoSchedule denotes the learned schedule and OptSchedule stands for choosing schedules in a static manner TVM but with DNMM and RPMM replaced respectively by DNMM332 and RPMMV. For conv2d, the learning approach does not work quite well and we instead use CONVOpt as the
180
182
184
4
V
Speedup
OptSchedule
AutoSchedule
Lis co (its))
12
Ore
(1025 (1025.002
(2588 possitiie
(25% (4)-
(64, (00 (000)
11620000000 (220
(16.8 230
(12266 1
c0095 250, o-
1994, 755. 1062)
(40021-77))
(8,08)
(1.0664)
(10.a0%) (104)
(258 (124))
cac9s, 1004,8
set iettse
(1ns
Figure 3: Performance of OptSchedule and AutoSchedule for matmul.
186
187
default implementation since it performs better than CONV while having the same advantage as CONV on leveraging NCHWe layout optimization [18 in the end-to-end inference.
188
189
Evaluation of tiling size space initialization and filter. Fig. illustrates how the two default schedules for matmul (DNMM when M < 16 and RPMM when M > 16) perform when being combined with different strategies for initializing the tiling size space. The left image shows the speedup over the base (factor). The middle and right images show the space swell ratio over the base (factor). Our strategy pfactor shrinks the tiling size space more than 40% for matrices with powers of 2 sizes without obvious performance loss. For the dataset consisting of matrices of prime number sizes, pfactor brings 1.2 speedup on average.
191
192
193
194
185
Fig.
5 show that the filter strategy further reduces tiling space size while not loosing performance.
7

---

2.0
-3
282
Specc Swelll
1.25
0.55
6.99
Seaplee
Gor
6a8-pitte
(AC-0))
Gamraa
Auaproed
Figure 4: Performance of different initialization strategy for matmul.
20
1.043
1019
1.012
Space Ratie
1.5
0.766
00:227
Speceuup
1
DNMM
RPMM
CONV
Goeman
DNMM
RFMM
CONY
Geomann
Figure 5: Performance of the proposed filter strategy for pruning the tiling size space.
197 Comparison of different configuration space exploiting strategies. Fig. 6 compares AVM'' exploration module (SA+RANK) and AutoMCL's performance model (REG) on tuning GEMMs of different sizes. The left and right image show the average performance of tuned matrix multiplications and the average tuning time.
198
250
SA /RANK
1.500
200
Oyamzzzten Titbe
1.00
R3S
Figure 6: Comparison between AuMoTVM and AutoMCL on exploring the configuration space.
200
Evaluation of AutoMCL the operation and the end-to-end level. Now evaluate the per-
202
formance of AutoMCL, which integrates all the optimization strategies introduced in Section 3 on optimizing matmul and conv2d for both fully connected neural networks (FCNNs) 26 and typical convolutional neural networks (CNNs) ResNet-50 (14), Inception-v3 [24,, and VGG16 [22]
204
25
mosard haxt
Infers Optimiootknn
Infezsee
Optintttionn
254
DAT
DAC
DTC
EC.M
DATH
25.14
27.14
0.5
Teici
Inferee
Opiritottos
Sporka
Figure 7: Evaluating the operations matmul and conv2d for FCNNs and CNNs.

---

Inference Time (Ens)
PC-5
Inference Time (mns
FC7
TenadFlow AZOTVV
100
TasofFlw NIRTVM AumMME
AnnoMEE
5
Optimiation Time (h)
FCS
FC7
10
AusTVM AsMMCC
Optimicationn Time
AuMTVM AsMMCL
10
Figure 8: End-to-end evaluation FCNNS with batch size=21, / 0, ..... on Intel CPU.
Tersceflow AutoTVM AutoMELL
AsITVM AssAMEL
1982
Oytnsorde Taurhaan)
400
300
100
Tmst
AND
AND
Figure 9: End-to-end evaluation on CNNs.
Ablation analysis. We analyze the effects of adding different optimizations on the performance, where OS and AS stand for using respectively the optimized and the automatically chosen schedules.
208
14, (266
110
250
300
200
90-
Gflops
Gflops
150
VM
AUBSTVM+OS
70
AuteTVM
AutoT VM+OS+AS
AusoTVMHOS
60
AunTTVMHS--hii-flee
AutuTTMM--S-AAA-ttiitttt-rr REG
AutTTVMHSS-lai-theer REG
200
250
Figure 10: Ablation analysis on a dense layer and a convolution layer from CNNs.
5
Conclusion
207
208
In this paper, we have introduced framework AutoMCL to auto-tune the matrix multiplication and the 2D-convolution operations in fully connected and convolutional neural networks by leveraging both analytic and machine learning models. Experiments show that it outperforms AutoTVM on both
210
inference speed and optimization cost for FCNNs and is competitive to AutoTTV for CNNs. In the
212
future, we plan to further improve performance by designing better strategies on automatically choosing the optimal schedule.

---

214 References
215
[1] Intel oncDNN.
216
[2] Intel ameap://oompmmantss/oomnllll..aa
217
218
[3]
NVIDIA cuDNN, https://devellpper www.iiiiiiii.... conleadnomm
219
[4]
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: A system for large-scale machine learning. In 12th { {USENIX) symposium on operating systems design and implementation ({OSDI) 16), pages 265 165-283, 2016.
221
223
[5] Byung Hoon Ahn, Prannoy Pilligundla, Amir Yazdanbakhsh, and Hadi Esmacilzadeh. Chameleon: Adaptive code optimization for expedited deep neural network compilation. arXiv preprint arXiv:2001.0874 2020.
224
225
226
[6] Riyadh Baghdadi, Abdelkader Nadir Debbagh, Kamel Abdous, Fatima Zohra Benhamida, Alex Renda, Jonathan Elliott Frankle, Michael Carbin, and Saman Amarasinghe. Tiramisu: A polyhedral compiler for dense and sparse deep learning, 2020.
227
228
229
[7] Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference knowledge discovery and data mining, pages 785-794, 2016.
230
231
232
[8] Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang. Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv preprint arXiv:1512.01224. 2015.
233
234
[9] Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. TVM: An automated end-to-end optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 578-594, 2018
236
238
239
[10] Tianqi Chen, Lianmin Zheng, Eddie Yan, Ziheng Jiang, Thierry Moreau, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. Learning optimize tensor programs. Advances in Neural Information Processing Systems, 31:3389-3400, 2018
240
241
242
Pratik Fegade, Tianqi Chen, Phil Gibbons, and Todd Mowry. Cortex: A compiler for recursive deep learning models. CORR, abs/201 1.01383, 2020.
243
244
[12] Matteo Frigo, Charles E. Leiserson, Harald Prokop, and Sridhar Ramachandran. Cache-oblivious algorithms. ACM Trans. Algorithms, 8(1), January 2012.
[13] Kazushige Goto and Robert A van de Geijn. Anatomy of high-performance matrix multiplication. ACM Transactions on Mathematical Software (TOMS), 34(3):1-25, 2008
247
248
[14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016
249
250
[15] Menghao Li, Minjia Zhang, Chi Wang, and Mingqin Li. Adatune: Adaptive tensor program compilation made efficient. Advances in Neural Information Processing Systems, 33, 2020.
252
253
[16] Mingzhen Li, Yi Liu, Xiaoyan Liu, Qingxiao Sun, Xin You, Hailong Yang, Zhongzhi Luan, Lin Gan, Guangwen Yang, and Depei Qian. The deep learning compiler: A comprehensive survey. IEEE Transactions on Parallel and Distributed Systems, 32(3):708-727, 2020.
254
255
256
[17] Rui Li, Yufan Xu, Aravind Sukumaran- -aanaarrn---ajaa,,,aa Atanas Rountev, and P. Sadayappan. Analytical characterization and design space exploration for optimization of cnns. In Proceedings of the 26th ACM International Conference Architectural Support for Programming Languages and Operating Systems, ASPLOS 2021, page 928 928-942, New York, NY, USA, 2021. Association for Computing Machinery.
257
259
10

---

261
[18] Yizhi Liu, Yao Wang, Ruofei Yu, Mu Li, Vin Sharma, and Yida Wang. Optimizing CNN model inference on cpus. In 2019 USENIX Anmual Technical Conference (USENIX ATC 19), pages 1025-1040, Renton, WA, July 2019. USENIX Association.
262
264
[19] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint 1eXivv/912.00 7030, 2019.
265
266
267
[20] Jonathan Ragan- Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, and Saman Amarasinghe. Halide: A language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. SIGPLAN Not., 48(6):519 19--539,,,,0 June 2013.
289
270
[21]
Yukinori Sato, Tomoya Yuki, and Toshio Endo. An autotuning framework for scalable execution of tiled code via iterative polyhedral compilation. ACM Transactions on Architecture and Code Optimization (TACO). 15(4):1-23, 2019.
271
272
[22] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Truck Proceedings, 2015.
274
275
276
277
[23] Benoit Steiner, Chris Cummins, Horace He, and Hugh Leather. Value learning for throughput optimization of deep learning workloads. In Proceedings of the 4th MLSys Conference, San Jose, CA, USA, 2021.
278
280
[24] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In 2016 JEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818- 2016
281
282
283
[25] Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary Vitoo, William S Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. Tensor comprehen- sions: Framework--gnostiic high-performance machine learning abstractions. arXiv preprint 18X 2018.
284
285
286
287
[26] Yu Emma Wang, Gu- Yeon Wei, and David Brooks Benchmarking tpu, gpu, and epu platforms for deep learning. arXiv preprint arXiv: 1907.10007 2019.
209
[27] R. Clinton Whaley and Jack J Dongarra. Automatically tuned linear algebra software. In SC'98: Proceedings of the 1998 ACM/IEEE conference on Supercomputing, pages 38-38 IEEE, 1998.
290
[28] Huaqing Zhang, Xiaolin Cheng, Hui Zang, and Dae Hoon Park. Compiler-level matrix multipli- cation optimization for deep learning. arXiv preprint 1rXiv: 2019.
292
293
[29] Lianmin Zheng, Chengfan Jia, Minmin Sun, Zhao Wu, Cody Hao Yu, Ameer Haj-Ali, Yida Wang, Jun Yang. Danyang Zhuo Koushik Sen, et al. Ansor: Generating high- perforarree tensor programs for deep learning. In 14th {USENIX} Symposium on Operating Systems Design and Implementation ( (OSDI)) 20). pages 863-879, 2020.
294
296
[30] Size Zheng, Yun Liang, Shuo Wang, Renze Chen, and Kaiwen Sheng. Flextensor: An automatic 297
298
schedule exploration and optimization framework for tensor computation on heterogeneous system. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, pages $59-873, 2020.
299
300
Checklist
301
302
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes], [No], or [N/A] You are strongly encouraged to include justification to your answer. either by referencing the appropriate section of your paper or providing a brief inline description. For example:
303
304
305
Did you include the license to the code and datasets? [Yes] See Section ??.
11

---

307
Did you include the license to the code and datasets?
[No]
The code and the data are
308
proprictary.
309
Did you include the license the code and datasets? [N/A]
310
Please do not modify the questions and only use the provided macros for your answers. Note that the
311
Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
312
313
1. For all authors...
314
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
316
(b) Did you describe the limitations of your work? [Yes] See Section 4 on "Evaluation of automatic schedule chosen".
317
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
320
321
2. If you are including theoretical results...
322
(a) Did you state the full of assumptions of all theoretical results? [Yes]
323
(b) Did you include complete proofs of all theoretical results? [Yes]. but only in the supplemental material due to space limit.
325
3. If you ran experiments..
326
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material URL)? [Yes]] in the supplemental material.
328
329
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]. in the supplemental material.
330
331
(c) Did you report error bars (e-g., with respect the random seed after running experi- ments multiple times)? [Yes] see Section
3 on "Ablation analysis"
333
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]. in the supplemental material.
334
335
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
336
(a) your work uses existing assets, did you eite the creators? [Yes]
337
(b) Did you mention the license of the assets? [Yes]. in the supplemental material.
338
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
339
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
340
341
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
343
5. If you used crowdsourcing or conducted research with human subjects...
344
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
345
346
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
347
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
349
12


Reviews:
[{"review_id": "qiPtbZTqwpb", "reviewer": "Reviewer_UVYp", "summary": "The paper proposes an auto-tuning framework for `matmul` and `2d convolution` operators targeted for CPUs. The authors talk about combining analytical cache models with ML trained with real hardware measurements to improve the final performance of running these operations on CPU. Finally, few pruning strategies was proposed to reduce the search space for scheduling.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The paper tackles the interesting problem of `auto-tuning machine learning operations on CPUs`. However, the paper is not clear about the contributions. For example, the authors talked about the analytic ideal cache model (which as mentioned it is not their contribution) with ML models trained with real hardware measurements. However, I couldn't clearly understand how the analytical model and ML models are integrated and used together or what is the performance implications of each of these.\n\n- It would be great if the authors summarize the last few figures in the paper and provide insights about the experiments. Without a proper explanation of the figures, it is not straightforward to understand what is the goal of each figure and what it presents. \n\n- In the `Evaluation of Automatic Schedule Chosen`, it was mentioned that `ConvOpt` is the default implementation. Does it mean that the search algorithm do no tune any parameters for convolution?\n\n- Do you have any explanation on why the speedup is better for the matrices with prime number sizes? Is it because of the data locality?\n\n- Which CPU platforms were used for the presented results? Are there any significant differences in the performance numbers across different CPU platforms?\n\n- Do you use the existing search algorithm in AutoTVM + proposed schedules or there are contributions on the search algorithm as well?\n\n- Figure 6 x-axis indicates different configurations? Can you provide some explanation on the high variance in SA+RANK and how REG reduces the variance in the results? How many times you ran the search?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "As mentioned earlier, the paper tackles an interesting problem, however without proper explanation of the main contributions it is not clear how the proposed approach can be used. It would be good if the results are explained in a more clear way to better understand the contributions. One of the main limitations of this work in my opinion is the applicability of the proposed approach across different CPU platforms with different caching hierarchy.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "m1Ra7XVlz3t", "reviewer": "Reviewer_4tmh", "summary": "The paper presents an application of the classic cache complexity analysis\nof the classic matrix multiplication and convolution algorithm used in deep learning workloads.\nThe paper provides a simple yet effective way to find the ideally optimal tile sizes\non both of the operators given a specific schedule.\nThe contribution includes:\n- A novel tile size initialization and search space pruning techniques.\n- A few schedule templates in AutoTVM to cover more search space.\n- Implemented the methodology into a new system called AutoMCL,\nwhich reuses the frontend and codegen backend of AutoTVM.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths.** This paper enumerates 6 different scheduling strategies for matmul,\nand another 6 for conv2d. The author calculated analytically on the closed form of potential\ncache misses in the each of the 6 strategies. Although very simple and straightforward,\nit does provide some insight on the effectiveness of the tile size choice.\nCompared with polyhedral models, this method does not necessarily require the affine map\nand solve a complicated optimization problem.\nAlthough loop skewing is not supported in this particularly case,\nwhich leads to smaller search space, \nit doesn't affect the tuning result.\n\n**Weaknesses.** The reviewer is wondering if such analysis can be formalized and conducted automatically.\nFor example, Ansor actually does cache-related feature extraction\n(see the bullet point \"Buffer Access Feature\" in Appendix B).\nIt is possible that the cache complexity can be analyzed and extracted automatically,\nas it is part of a simple loop analysis,\nand if it can be done, we are able to generalize this simple yet effective method to arbitrary workloads\nother than matmul and conv2d.\n\n**Correctness.** The paper is correct from the reviewer's perspective.\n\n**Relation to prior work.** The paper doesn't provide comprehensive enough comparison\nwith some of the related analytical or polyhedral models,\nincluding Tensor Comprehension [2] and MOpt [3].\nThe reviewer believes that it could be improvement in the rebuttal process.\n\n**Clarity.** Overall looks good. Some annotations need to be declared and explained\nbefore being used. For example, $T_m$ in Theorem 1.\n\n[1] Zheng, Lianmin, et al. \"Ansor: Generating high-performance tensor programs for deep learning.\" 14th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 20). 2020.\n\n[2] Vasilache, Nicolas, et al. \"Tensor comprehensions: Framework-agnostic high-performance machine learning abstractions.\" arXiv preprint arXiv:1802.04730 (2018).\n\n[3] Li, Rui, et al. \"Analytical characterization and design space exploration for optimization of CNNs.\" Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 2021.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1) Extending to real-world CPU architecture is an interesting future work.\nThe paper focuses only on an ideal cache model where the cache is single stage and fully associative,\nwith a simple replacement policy. Therefore, it is less realistic with the modern CPU architecture.\n2) Also, the paper didn't consider VNNI and other instructions for low-bit deployment,\nwhich plays an important role in deep learning deployment.\n3) It would be very interesting, though non-trivial, to extend the paper to other architectures,\ne.g. GPUs and TPUs, because of the different memory hierarchy and execution model.\n4) There are other interesting instructions to be used, for example, prefetching.\n\nThe reviewer is not aware of any negative social impact on this paper.\n", "review_text": "", "extra_scores": {"ethical_concerns": "The reviewer is not aware of any ethical concerns in this paper.\n", "needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "7 hours in total. 5 hours reading the paper, and another 2 hour for writing the reviews."}, "extra_text": {}}, {"review_id": "LjsUq1vdKAB", "reviewer": "Reviewer_uaRL", "summary": "This work introduces AutoMCL - a new auto-tuning framework aimed at making compiled output higher performing, while improving the compile-time as well though better pruning of the search space. Authors focus primarily on lower-level library primitives, specifically: fully connected and convolution layers. Their primary contribution is a mix of analytical and ML-driven models. More specifically authors introduce some analytical models, suited for assessing effectiveness of various tiling strategies on CPUs, with the help of ideal cache models and parameters learning/modeling cache complexity. These help both cutting down the search space with respect to prior work based on ML-based learning, and find in some cases better schedules as well.  Net gain on optimization/compile time is more significant (more than 2x over AutoTVM).  Gain in inference performance through better schedule generation at kernel level is less - about 37% for full-connected networks. CNNs show just about 2% improvement in throughput and optimization time.\n\n", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This submission's technical contributions are quite limited: a) applicable to CPUs only, b) primary contribution is some analytical models for cache subsystem which, in combination with learned parameters cut down search space and find better schedules, and c) very limited performance or throughput gains, best case only 37% for FCNNs.\n\nPaper is reasonably well-written. Section 3 - which is where most of the new content (analytical model) is explained - is not easy to read.\n\nExperimental evaluation is reasonable too - as it looks at both Intel and AMD CPUs. It would have been better to extend the work to ARM CPUs as well. Overall, this section (Sec 4) is the better-written and easier to read and follow.\n\nGain over AutoTVM is all that matters - since it is the primary and relevant prior-art for this work (not TensorFlow). And this gain is quite limited.\n\nFor observations like that in line 185: \"For conv2d, the learning approach does not work quite well\" - it would help the reader if you can offer more insight as to how/why the learned schedule does show much improved over static selection.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I cannot foresee how this work adds to or reduces potential negative societal use of the prior state-of-the-art they are contributing to.", "review_text": "", "extra_scores": {"ethical_concerns": "I cannot foresee how this work adds to or reduces potential ethical concerns of the prior state-of-the-art they are contributing to.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3-4 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers generally felt that this is an interesting and potentially useful paper. However, there are still outstanding issues. Limited scope (CPUs only), idealized scenario that may not generalize to real-world architectures, clarity issues (some of which were addressed in the discussion), the relatively minor improvement over AutoTVM, and a lack of comparison with other related models. Addressing these will help improve the paper.
Decision: Reject