DEX: Data Channel Extension for Efficient CNN Inference on Tiny AI Accelerators
Authors: Taesik Gong
Fahim Kawsar
Chulhong Min
Venue: NeurIPS 2024 poster (2024)
Keywords: TinyML
On-device ML
CNN
AI accelerator
Microcontroller
MCU
Primary area: machine_vision
TL;DR: We propose Data Channel EXtension (DEX) to improve CNN accuracy on tiny AI accelerators by using patch-wise sampling and channel-wise stacking, boosting accuracy by 3.5% without increasing inference latency.

Abstract: Tiny machine learning (TinyML) aims to run ML models on small devices and is increasingly favored for its enhanced privacy, reduced latency, and low cost. Recently, the advent of tiny AI accelerators has revolutionized the TinyML field by significantly enhancing hardware processing power. These accelerators, equipped with multiple parallel processors and dedicated per-processor memory instances, offer substantial performance improvements over traditional microcontroller units (MCUs). However, their limited data memory often necessitates downsampling input images, resulting in accuracy degradation. To address this challenge, we propose Data channel EXtension (DEX), a novel approach for efficient CNN execution on tiny AI accelerators. DEX incorporates additional spatial information from original images into input images through patch-wise even sampling and channel-wise stacking, effectively extending data across input channels. By leveraging underutilized processors and data memory for channel extension, DEX facilitates parallel execution without increasing inference latency. Our evaluation with four models and four datasets on tiny AI accelerators demonstrates that this simple idea improves accuracy on average by 3.5%p while keeping the inference latency the same on the AI accelerator. The source code is available at https://github.com/Nokia-Bell-Labs/data-channel-extension.

DEX: Data Channel Extension for Efficient CNN Inference on Tiny AI Accelerators
Taesik Gongl
Fahim Kawsar
Chulhong Min
Nokia Bell Labs
UNIST
University of Glasgow
taesik goaikkognnuuniss..e. ac.
{fahim. kawsar,
chulhong.. minn)mooki-belllllabs
Abstract
Tiny machine learning (TinyML) aims to ML models small devices and is increasingly favored for its enhanced privacy, reduced latency, and low cost. Recently, the advent of tiny Al accelerators has revolutionized the TinyML field by significantly enhancing hardware processing power. These accelerators, equipped with multiple parallel processors and dedicated per-processor memory instances, offer substantial performance improvements over traditional microcontroller units (MCUs). However, their limited data memory often necessitates downsampling input images, resulting in accuracy degradation. To address this challenge, we propose Data channel EXtension (DEX), a novel approach for efficient CNN execution on tiny AI accelerators. DEX incorporates additional spatial informa- tion from original images into input images through patch-wise even sampling and channel-wiss stacking, effectively extending data across input channels. By leveraging underutilized processors and data memory for channel extension, DEX facilitates parallel execution without increasing inference latency. Our evalua- tion with four models and four datasets on tiny AI accelerators demonstrates that this simple idea improves accuracy ference latency the same on the AI accelerator. The source code available at https: // /ttg//ggtttt/ttt www. Wwwm/Nokia--- Bell-Labs dataaaaaaaaaaaeeaaaa
average by 3.5%p while keeping the in-
Introduction
Tiny machine learning (TinyML) is active research field focused on developing and deploying machine learning models on extremely exsource-constringe devices, such as microcontroller units (MCUs) and small IoT sensors. Compared to cloud-based AI, TinyML on devices offers benefits in privacy preservation, low latency, and low cost. While research efforts in TinyML, such as model compression techniques [15, 17, 25, 27, 31, 32], have successfully reduced the size of AI models to fit into memory-constrainge MCUs, the fundamental limitation in the processing capability of MCUs leads long inference latency. This limitation hinders the widespread adoption of on-device AI, especially for real-time applications.
Recently, the advent of tiny Al accelerators like the Analog Devices MAX78000 [34] and Google Coral Micro [8] has revolutionized the TinyML field by dramatically boosting the model inference speed and leading
phase of on-device AL. For instance, the MAX78000 AI accelerator [34]
achieves 170x faster inference latency compared to an MCU processor (MAX32650 [33]).
To enable such acceleration, these tiny Al accelerators introduce several hardware optimization tech- niques. They often feature multiple convolutional processors (e.g.. 64 processors in MAX78000 [34])
This work was done entirely while the author was affiliated with Nokia Bell Labs.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Pooling Engine
2000
760
1500
FacelD
Emergy (102)
400
FaodlD
Coching
Latoocy
1000
754
200
500
350
Gonv
Engline
0.14
64 Parallel Processors
Data memory (512KB)
Weight memory (452KB)
MAX
MAX33650
STM
MAXT8000
MAX:2250
STMS2F1
Figure 1: The architecture of tiny Al Figure 2: Comparison between an AI accelerator (MAX78000) and MCUs (MAX32650 and STM32F7).
accelerator (MAX78000 [34]).
and parallelize per-channel CNN operations across these processors. For further optimization, the memory architecture allows each processor to have dedicated memory instance, i.e., per-processor memory instance. This design enables simultaneous memory access to multiple channels from different processors. While these hardware-level optimizations bring significant performance im- provements, we found that they also have several constraints at the expense of the optimizations. First, the per-processor memory architecture highly restricts the supported input image size because the data memory each processor can use for its input/output channels is limited to the capacity of its dedicated memory instance, which fraction of the total data memory divided by the number of processors. Consequently, most vision models for these accelerators are designed to support very small images, such 32 32 pixels. Given that images captured by cameras are often generated with higher resolutions, downsampling is inevitable, leading to accuracy degradation due to information loss from the original image. Second, we found that processors and data memory are underutilized for the input layer due to the per-processor memory architecture; since input images typically have low number of channels (e.g., RGB three channels). only limited number of processors tied to memory instances are utilized while the remaining processors remain idle. For instance, on the MAX78000, 61 of 64 processors and per-processor memory instances remain unused in the first layer.
In this work, we propose a novel approach, Data channel EXtension (DEX), to overcome these constraints while still benefiting from the acceleration power of tiny Al accelerators. The core idea is to boost accuracy by extending the data channels to incorporate additional image information into unused data memory instances and processors, instead of simple downsampling. Owing to the parallel processing and memory access capabilities of tiny AI accelerators, our method can achieve this accuracy improvement without compromising inference latency. Specifically, DEX involves two procedures: (1) wiie even sampling, where pixels from the original image are evenly sampled, and (2) channel-wise stacking, which arranges these samples across multiple channels.
To measure the impact of DEX on accuracy and resource utilization, we conducted experiments on the MAX78000 [34] and MAX78002 [37] tiny AI accelerator platforms. DEX was evaluated on four models, SimpleNet [1 [16], WideNet [16], EfficientNe [48], and MobileNetV2 [45], using four vision datasets: ImageNette [18], Caltech [11], Caltech256 [14], and Food101 [2]. Our results show that DEX improves average accuracy by 3.5%p compared to the original model with downsampling and 3.6%p compared to the existing coordinate augmentation approach (CoordConv [29]), without increasing inference latency. Additionally, DEX maximizes data memory and processor utilization, demonstrating its effectiveness in enhancing model performance on constrc-conttrained devices. In summary. DEX can significantly enhance the performance of neural networks on tiny Al accelerators, leading to more efficient and effective deployment of AI on resource-constrained devices.
2 Preliminary: tiny AI accelerators
The advent of tiny Al accelerators marks a pivotal shift towards on-device AI, greatly enhancing privacy and reducing latency. While a number of tiny-scale AI accelcrators have emerged recently, such as Analog Devices MAXT800MMXX7800 [34, 37J, Google Coral Micro [8], and GreenWaves GAP-8/GAP-9 [12], only few commercially available with access and control over their operations. In this paper, we focus on the MAX78000 [34] and MAX78002 [37] as our primary platforms since they are the most widely used tiny Al accelerator research platforms [1, 6, 13, 39, 40, 43] owing to the disclosed hardware details and open-source tools, enabling in-depth analysis and modification of their operations.

---

Architecture of tiny AI accelerators. The distinctive characteristic of tiny Al accelerators compared conven- tional microcontroller units (MCUs) is parallel processors that parallelize per-channel CNN operations across these processors. Figure| depicts an abstracted architecture of the MAX78000; MAX78002 has a similar architec- ture to MAX78000 with increased memory (1.3 MB data and 2 MB weight memory). Further details are in Ap- pendix A.1. It has 64 parallel convolutional processors, each capable of performing specific operations indepen- dently. To maximize performance, each processor has dedicated memory instance, i.e., per-processor memory instance that optimizes data transfer with parallel access. For each CNN layer, operations on individual channels are assigned to separate convolutional processors and executed simultancously, significantly reducing latency typically associated with convolutional algorithms. Each processor has a pooling engine, an input cache, and convolution engine that can handle by 3 kernels. The CNN accelerator includes 512 KB of data memory and 432 KB of weight storage memory. Within the 512 KB of data memory, an KB per-processor memory instance is allocated to each of the 64 processors. Figure 3 shows the utilization of the processors (Pry) for executing CNNs with varying sizes of the input channels. Each processor communicates with dedicated memory instance for each data channel. For example, given three-channel image, three parallel processors are utilized in the first layer.
Input
3che
Conv
82 32hhnnals
Conv
64 chonncl
Conv
Pri
Pri
Pri
in use
Prx
Pr
Prz
Prs
Prs
Prs
P%
Prap
Pris
Pryp
Prss
P1ga
Pts+
Pra
Prae
Figure 3: Processor utilizaaion with vary- ing input channels on the Al accelerator.
Performance gain over MCUs. A recent benchmark study [35] demonstrates the remarkable performance gain of the MAX78000 in terms of latency and energy consumption. Figure 2 shows that the MAX78000 significantly outperforms widely-used MCUs (MAX32650 with a Cortex-M4 at 120 MHz [33], and high-ppefforance MCU, the STM32F7 with Cortex-M7 at 216 MHz [47]) for face detection (FaceID) and keyword spotting (K (KW)). For KWS, lateney is drastically reduced only 2.0 ms, compared to 350 ms for the MAX32650 and 123 ms for the STM32F7. Accordingly, energy efficiency of the MAX78000 is also significant; it consumes only 0.40 mJ for FacelD, dramatically less than the 42.1 mJ and 464 mJ required by the MAX32650 and STM32F7, respectively.
3 DEX: Data channel extension for efficient CNN inference on AI accelerators
3.1 Constraints of per-processor memory instances in tiny Al accelerators for images
As mentioned in $2, tiny Al accelerators leverage per processoo memory instances for faster data transfer with parallel access. However, we disclose that this causes several constraints at the expense of rapid data access: (1) low image resolution and (2) underutilized processors and data memory.
Low image resolution due to limited per-processor memory size. MAX78000 [34] has 512 KB data memory which is divided into 64 segments of KB memory instances per processor, each storing the data of each input channel. This memory architecture highly restricts the supported input resolution. For instance, an input image with a shape 3 x 224 x 224 (channel, height, and weight), which is typical size of ImageNet [9], does not fit the MAX78000 even with Q7 format (one byte for each value), as memory limit for each channel 8 KB (224 x 224 50 KB KB). Thus, the current practice on tiny AI accelerators is to shrink the resolution of input images by downsampling and accordingly, to design small models to process lower-resolution images, e.g., 3 x 32 x 32. However, with this, it loses most of the information of the original image, which might lead to sub-optimal performance.
Underutilized processors and data memory for the input layer. Although per processessor memory instances allow simultancous memory access from different processors, it also brings inefficiency in data memory and processor utilization, especially the input layer. Specifically, given an input image 7 with the number of channels Cr, height H,, and width W, (e.g.. 3 x 224 x 224) as shown in Figure 4(a), Figure 4(b) illustrates the downsampled image with the number of channels Cr, height Ho, and width Wo (c.g.. 3 x 32 x 32), and its data memory usage in the Al accelerator.
3

---

(a) Original image! /
(b) Downsampling
DEX
We
W
W
Different samples
Ho
Ho
Procossors
Dota momery
Picecssors
Data momory
H
Fi
PG
Adk P10
memory limit
Pry
Figure 4: Comparison among different input data. (a) an original image that exceeds the data memory limit of the Al accelerator, (b) downsampled image that fits the data memory but does not fully utilize parallel processors and data memory, and (c) DEX-generated image that incorporates more information from original image by extending data across channels with full utilization of parallel processors and data memory instances.
Original image
Patch-wise even sampling (eg.
Channel-wise stacking (egg. Co
image
Pto
Ph
Roz
P12
We
P1
Wr
H2
Patch Puj
Oi
Ho
Cp
4x Wp,
b mod Wry
Figure S: Overview of DEX. DEX divides the original image into multiple patches. DEX then evenly samples pixels from each patch P.j and constructs output pixel Ozy by stacking samples across channels.
With three RGB channels, channel data separately stored for each data memory instances for parallel execution. As there N processors and corresponding data memory instances, it leaves the remaining N 3 processors and data memory instances idle. This provides opportunity to utilize these idle data memory instances and parallel processors, which we detail in the following section.
3.2 DEX procedure
As aforementioned, we note two key observations: (1) the input image needs to be downsampled due to the limited memory of tiny Al accelerators, which means most of the pixel information cannot be utilized, and (2) there exist idle data memory instances and processors that could process up to N channels parallel. Although several recent studies have found efficient model architectures on tiny AI accelerators [39, 43], existing studies lack considerations this inefficiency for input image processing in CNNs (further discussion on related work is in Appendix 5).
Based on our observations, we propose Data channel EXtension (DEX) for efficient CNN execution on tiny AI accelerators. The key intuition behind DEX is that we can utilize the remaining data memory to incorporate additional information from the original image into neural networks by extending the input data across channels. By utilizing this additional memory and processors, we can incorporate extra sample information for feature learning without sacrificing latency. Figure 4(c) shows the input data reshaped via DEX, where each channel contains different pixel information from the original image. With DEX extending data across channels (from C₁ to Co), it can fully utilize the data memory and associated parallel processors. Figure shows an overview of the procedure of DEX. Given an input image I with number of channels Cr, height Hf, and width W,, DEX generates an output image with an extended number of channels Co. height Ho. and width Wo (e.g., 64 x 32 x 32) via patch- patch-wise wise sampling and channel-wise stacking.
Patch-wise even sampling. The purpose of patch-wise even sampling select samples evenly spaced across the original image while keeping the spatial relationship among pixels. We first define

---

patch from the original image in which a corresponding output pixel is generated. We denote i-th row and j-th column of patch Pij in I as:
P.j = 1 [
H Ho_
H, Ho.
W Wo
W Wo ]
(i+1)
(++1)
(I)
where [: refers to 2-D array slicing operation, specifying the selection of rows and columns sequentially. The number of patches is determined by the resolution the output image, i.e., Ho x Wo- For each patch P.j. generate the corresponding output data Oij. This ensures that the spatial relationships among pixels in the input image are preserved the output, maintaining spatial consistency throughout the process.
The next step is to sample pixels within the patch considering the memory budget. Specifically, we define K - C2 as the number of samples be selected in each patch. Given the height and width of patch HP., and j-th column of output Oi can represented by:
11+
Ho i-
H1
Hy Ho
and WP,, (i 1)
Wr
- i Wz the i-th row Wo
Wa
O₂ { Po
4 WP.
4 mod We.)] 14 k- Hp. WP. - K-1
for t: 0, 1....... K
which means a collection of evenly distributed samples within each patch to encourage diverse information while minimizing the use of localized pixel information. With patch-wise even sampling, selected samples are evenly distributed both across patches and within each patch.
Channel-wise stacking. Channel-wise stacking arranges sampled data across multiple channels and keeps this procedure for all pixels to maintain data integrity. Channel- stacking is beneficial it maintains consistency within each channel, preserving the spatial and contextual relationships of the sampled data. Specifically, after patch-wise even sampling, the samples are stacked across the channel axis in ascending order of the index and this is repeated for each Oij- Note that 4 when K and this is identical traditional downsampling. If K Co. it fills up the target channel with P's data until the limit and discards the remaining channels. For instance, when using RGB channels (C,- 3) and if Co = 64 and = 22, it takes only the red channel for i = 21 and discards the remaining green and blue ehannels that exceed the channel limit of 64. Algorithm provides the pscudo-code that describes the procedure of DEX's channel extension algorithm.
Cj
Algorithm DEX Channel Extension Algorithm
Input: Source image / in shape (Cr, Hr, Wy)
Output: Reshaped image O in shape (Co, Ho, Wo)
1: O t- zeros(Con Ho, Wo)
2: for i + 0 to Ho - do
3: for + 0 to Wo do
4: start row, end row floor(i H1), floor( 1): Hi
5: start_col. end_col < floor(j wo
W. floor(() (j + 1) Wo
Wi
6:
Ps + |ta stwrttrooww row end_row, start start_cool: end _nd_col] K + ceil( Ce)
Patch P.y of
7:
> Number of samples to be selected in Ps Get channels of Oig from Ps
8:
for 1: + 0 to K - do end_row row - start_row start_col
9:
Hr, f- end
10:
Wrs Pr t- end_
4. = k floor( Mry We, WP,--
11:
k-l
12:
Olk Cr : (k: + 1) Cr, i, jl = Pul. floor( 12),, ), l mod Wr,))
13:
return O
3.3 Further analysis on DEX
5

---

Understanding how DEX leads to per- formance improvement. DEX's ability to incorporate additional pixel information from the original image can improve the accuracy of CNNs. The extended chan- nels provide further samples of adjacent areas in the original image, significantly broadening the receptive fields features in the initial CNN layer. This expansion allows the model to detect more complex and subtle features early in the processing pipeline, which is critical for the nuanced understanding and interpretation of visual data. Specifically, Figure 6 visualizes how the first CNN layer operates with DEX, where Leerac size and L refer to the kernel size and the output channel size of the first layer, respectively. It illustrates the application of the convolution operation across each enhanced channel (Co as opposed C1). where distinct kernel weights are applied each channel. This ensures that the additional information is integrated into the output feature maps, thereby en- riching the model's feature extraction capabilities. The convolutional layer processes the increased channel input, which is reflected in weight sums that construct output channels.
Murmaiaate
Image
1st layer output
Added
W
Conv
22
Kornets
Figure 6: The initial CNN layer's operation with DEX.
Impact of channel extension on the number of parameters. Given the first CNN layer's kernel size "'kernl_sszee and the first layer's channel output size L ooout the number of parameters required for the input layer can be calculated as Oc kernel L. out If Oc is 3, it is the same as the coutt.
traditional downsampling without our channel extension. Note that this channel extension does not incur additional inference latency on the Al accelerator. We found that the channel extension increases 3% of the total parameters as we show in our experiment $4.2. The rest of the layers remain the same. In addition to its simplicity, we have several reasons to change the first layer only, which we discuss further $6.
Utilization of the original image information. With traditional downsampling, the utilization of the original input is Ho-Wo Hy-W2" : but with DEX, this is extended to Co Ho-Wo For instance, given a Cj Wr-W;
3 x 256 x 256 input image, a downsampled image 3 x 32 x 32 utilizes only 1.6% of the original information, while with DEX and an output channel size Co 64, it can utilize 33.3% of the original information. DEX can accommodate all the information when Co C₁
H -,
Ho-Wo
Maximum number of output channels. Increasing the number of output channels allows DEX to accommodate the original image information. The number of output channels denoted as Oc. that can be extended without increasing latency on Al accelerators is limited by the number of data memory instances Dx, i.e., Oc < Dx For example, the MAX78000 has 64 data memory instances, allowing it to support up to Oc = 64 output channels without affecting inference latency.
4 Evaluation
4.1
Experimental settings
Here we explain experimental settings. Further details are in Appendix A.
On-device testbed. We evaluated DEX the off-the-shelf MAX78000 feather board [36] and MAX78002 Evaluation Kit [38], which are a development platform for the MAX78000 [34] and MAX78002 [37], respectively, as shown in Figure 9. In this paper, select these accelerators because they provide open-source tools for thorough analysis and modification of their internal processes, making them the most widely used tiny AI accelerator research platforms [1, 6, 13, 39, 40, 43].
Model training and deployment. In our experiment, we use four models officially supported in the Analog Devices MAX78000/78002 Training framework [20]: SimpleNet [16]. WideNet [16].

---

Table Average classification accuracy (%) and corresponding standard deviations over three runs for each dataset and method. Bold type indicates those of the highest classification accuracy.
Dataset
Method
SimpleNet
WideNet
MobileNetV2 EfficientNet
AVG (%)
Downsampling
57.8 = 1.2
61.8 0.2
51.3 + 0.5
62.0 0.7
58.2
CoordCony
58.0 1.1
61.7 0.2
0.1
61.6 + 0.3
58.3
ImageNette
CoordConv (r)
55.4 1.5
6.2
51.7
61.2 1.1
57.4
DEX (ours)
61.4
65.6 = 0.6
0..5
64.4 = 0.6
62.0
Downsampling
55.8 2.1
3,
51.4 1.6
50.1
CoordCoav
53.8 0.5
56.5
3..7
49.8 0.5
49.7
10
CoordConv (r)
56.0 1.7
38.2
49.7
49.1
DEX (ours)
56.9
61.1 + 1.4
45.9
53.3
54.3
Downsampling
20.8 + 0.5 2.8 0.5
0.6
14.7
2
19,4
CoordConv
0.8
2.87
19.6
Caltech256
CoordConv (r)
20.0
0.6
0.3
2..7
19.5
DEX (ours)
2..
2.9
0.9
2.5
22.6
Downsampling
16.0
0.7
12.1
2.64
17.1
CoordConv
1..1
0.3
1.10
2.3
16.9
Food101
CoordConv (r)
16.3
17.3
1.6
20.9 + 0.3
16.6
DEX (ours)
18.4
20.9
1.14
23.3 1.1
19.8
EfficientNetV2 [48], and MobileNetV2 [45]. The supported models from the framework were trained via quantization-awar training with 8-bit integers in PyTorrch [41]. We follow the official training
configuration (details in Appendix A.2). The checkpoints are synthesized as embedded codes for via the Analog Devices MAX78000/70002 Synthesis framework [19]. SimpleNet and WideNet developed for MAX78000 while EfficientNetV2 and MobileNetV2 are for MAX78002 considering the size of the models. All models originally designed to take 32 x 32 inputs, and DEX increases the number of the channels in the first layer to 64.
Datasets. We evaluated on four common vision datasets: (1) ImageNette [18], ten-class subset of ImageNet [9] with 9469/3925 train/test samples with the original image shape of 3 x 350 x 350, (2) Caltech101 [11] with 101 objects classes having 6941/1736 train/test samples with the original image shape of x 300 x 300, (3) Caltech256 [14] with 256 objects classes having 23824/5956 train/test samples with the original image shape of x 300 300, and (4) Food101 [2] with 101 food
categories with 75750/25250 train/test samples with the original image shape of 3 x 512 x 512.
Baselines. For baselines, compare with the Downsampling method which is a straightforward way to reduce the size of the input under image x 32 32. In addition, limitation of traditional CNNs that relied on RGB images for the coordinate transformation problem and introduced the augmentation of and j coordinates, which improved object detection efficiency by using two extra channels. The authors of CoordConv also introduced the third channel for an coordinate, where r V(i h/2)² + [j ww/2²² which they found effective in some experiments.
devices. It downsamples the input compare DEX with CoordCony [29] which pointed out the
4.2 Result
Overall accuracy. Table 1 shows the overall accuracy for four different datasets with the baselines and DEX. As shown, extending data channels to utilize additional input information improves accuracy in DEX. Specifically, DEX achieved 3.5%p higher accuracy compared to downsampling and 3.6% higher accuracy compared CoordConv across datasets. CoordConv shows lower accuracy compared with downsampling (0.1%p degradation average), showing they are not very effective solutions. This finding aligns with previous results indicating that CoordConv is useful for specific tasks such object detection, where coordinate information is important [29]. We found CoordConv (r) has a similar pattern to CoordConv. Overall, DEX's accuracy improvement shows the effectiveness of using extra information from the original image for feature learning.
Resource usage. Table compares the resource usage of the baseline and DEX. First, we found that, although DEX extends the number of channels in the first CNN layer to 64, its impact the

---

Table Model size (Size), utilization of the original image information (InfoRatio), accelerator's processor utilization for the first layer (ProcUtil), and inference latency on the accelerator (Latency) different models and methods averaged over three runs.
Model
Method
InputChan
Size (KB)
InfoRatio (x)
ProcUtil (%)
Latency (1u8)
Downsampling
3
162.6
1.0
2592
CoordConv
5
162.9
1.0
2592
SimpleNet
CoordConv (r)
6
163.0
1.0
2592 2
DEX (ours)
171.2
21.3
100.0
2591
Downsampling CoordConv
3 306.4
1.0
4.7
3820 +
5
306.9
1.0
7.8
3820 + 0
WideNet
CoordConv (r)
6
307.1
1.0
9,4
3819 + 1
DEX (ours)
64
319.3
21.3
100.0
3818 + 1
Downsampling
742.4
1.0
11688 2
CoordConv
5
743.0
1.0
11685 + 3
AfficientNetV2
CoordConv (r)
6
743.2
1.0
11689 +
DEX (ours)
64
759.6
21.3
100.0
11690 + 2
Downsampling
3
1317.8
1.0
4.7
3553 1
CoordCony CoordConv (r) DEX (ours)
5
1318.2
1.0
7.8
3554 1
MobileNetV2
6
1318.4
1.0
9.4
3554 1
64
1330.7
21.3
100.0
3552 +
SimpletNet
EtfiienNNeV2
MokikNNeV2
ImageNette
Caltech10)
Caltech256
Food101
Acouracy (%)
60
50
45
35
36
Figure 7: Accuracy of DEX varying the channel size. The shaded areas are standard deviations.
model size negligible (an average increment of 3.2% compared to channel extension). DEX utilizes 21.3x x more image information compared to downsampling, which is the primary reason for the accuracy improvement. As expected, DEX does increase on-device inference latency, even though it maximally utilizes the processors on the Al accelerators for information processing. This result consistent across the four datasets, as all the models are designed to take the same input size in MAX78000 and MAX78002.
Accuracy according to the channel size. We varied the size of the channels from (downsampling) to 6, 18, 36, and 64 with DEX to understand the impact of the channel size in terms of accuracy. Figure 7 shows the accuracy variation according to the channel size across the four datasets. As shown. it seems that higher number of channels increases accuracy in general. This means that selecting the highest channel size supported AI accelerators might be an effective strategy in practice, considering that it does not incur the latency increase. Still, there some cases where the accuracy of the highest channel size (64) is not the best among them. This means there might be an optimal number of channels tailored specific dataset and model architecture, which might be found in the model development process.
Resource usage according the channel size. We also measure resource usage varying the channel size. First, measured the model size and inference latency as shown in Table 3. The model size increment is negligible and inference latency remains the same across different numbers of channels. The model size and inference latency are the same for the four datasets as all the models are designed to take the same input size MAX78000 and MAX78002. Second, we measure the infor- mation utilization from the original image and processor utilization in the AI accelerators (Figure 8).

---

Table Model size (Size) with relative increment (%) compared the three channels and average inference latency on the accelerator (Latency) with standard deviations over three runs, varying the channel size.
Model
Chan = 3
Chan
Chan = 18
Chan 36
Chan 64
SimpleNet
162.6
163.0 (-0.3%)
164.7 (+1.3%
167.3 (+2.9%)
171.2 (+5.3%)
WideNet
306.4
307.1 (+0.2%)
309.6 (+1.0%)
313.4 (42.3%)
319.3 (+4.2%)
Size (KB)
EfficientNet12
742.4
743.2 (+0.1% 1%)
746.6 (+0.6%)
751.7 (+1.3)
759.6 (+2.3%)
MobileNetV2
1317.8
1318.4 (+0.0%)
1321.0 (+0.2%)
1324.8 (+0.5%)
1330.7 (+1.0%)
SimpleNet
2592 +:
2592 #
2591
2590
2591 +
WideNet
3820
3820 2
3825
3819 =
3818 =
Latency (us)
EfficientNetV2
11688 2
11691 =
11692
11691 +
11690 =
MobileNetV2
3553 4
3553
3552 *
3554 #
3552 +
The utilization of the original image infor- mation depends on the size of the orig-
ImageNNtte
100
Proc Util
Unilzatiion (%)
20
CalTech101
Uniizzttion (%)
inal data size, which grows linearly ac- cording to the channel size. We found a
CalTech256
Food101
10
correlation between information utilization
rate and accuracy improvement. For ex- ample, Caltech 101 and Caltech256 had uti- lization rates of 24.3%, improving accuracy by 4.2%p and 3. 3.2%p, respectively, while Food101 had an 8.3% utilization rate with 2.7%p accuracy improvement. The proces- utilization linearly increases until 100% with 64 channels size, which is the number of parallel processing units in the evaluated platforms.
3.6
36
0-
0.3
36
64
Channel Size
Charmel Size
(a) Information utilization.
(b) Processor utilization.
Figure 8: Resource usage varying the channel size.
Comparison of alternative data extension strategies in DEX. To understand the effec- tiveness of our patch-wise even sampling and channel-wise stacking. compared DEX with other possible data channel extension strategies. We compared with four strategies: repeating the same downsampled image across the chan- nels (Repetition), generating slightly different images through rotation (Rotation), dividing the original image into multiple tiles and stacking
Table 4: Comparison of data extension strategies.
Method
InputChan
InfoRatio (x) Accuracy
Downsampling Repetition
57.8
1.0
56.3 =
Rotation
64
1.0
5,74
Tile per channel
21.3
3.74
Patch-wise seq. Patch-wise rand.
21.3
61.0
64
21.3
60.4 1.0
DEX
21.3
61.4 0.6
those tiles across channels (Tile), patch-wise sequential sampling (Patch-wise seq.) that samples pixels sequentially within patch, and patch-wist random sampling (Patch-wise rand.) that randomly samples within a patch. Further implementation details are in Appendix A.5. In this experiment, we used SimpleNet and evaluated it on ImageNette. Table shows the results. Repetition does not improve accuracy over downsampling, indicating that merely increasing the number of kernels does not lead to performance gains. Rotation shows slight decrease in accuracy compared Repetition, which suggests that slight changes through rotation do not enhance performance. Interestingly. Tile shows low accuracy, demonstrating the importance of having complete view of the original image in each channel, rather than focusing specific regions. Both Patch-wise sequential and Patch-wise random samplings show lower accuracy than DEX's patch-wise even sampling, highlighting the importance of even sampling for better performance.
Related work
Tiny Machine Learning (TinyML) is emerging field that focuses on adapting machine leamning techniques for highly eesource-constrain devices, such as microcontroller units (MCUs). These devices often come with limited memory, typically hundreds of kilobytes of SRAM. Research in this area has mostly concentrated on reducing model size through various compression techniques, such as model pruning [15, 17, 25, 27, 31, 32), model quantization [7, 15, 42, 44, 49, 53, 54), and neural architecture search (NAS) [4, 5, 10, 24]. In addition, several studies have explored the
TinyML.

---

efficient utilization of memory resources (e.g., SRAM). Examples include optimizing on-device training processes [23, 28] and designing memory-efiiciet neural architectures [26, 52]. Unlike these approaches that primarily target MCUs, our research utilizes the distinctive architecture of tiny Al accelerators to enhance both memory efficiency and overall performance.
Tiny Al accelerators. Several studies have leveraged tiny Al accelerators for small-scale on-device AI applications. For instance, VinyissimoYOLO [39] offers quantized, memory-effiiiett and ultra-lighteiigt object detection network, showcasing its effectiveness on the MAX78000 plat- form. Additionally, KP2DTiny [43] introduces a quantized neural keypoint detector and descriptor specifically optimized for MAX78000 and Coral Al accelerators. Morcover, Synergy represents a multi-device collaborative inference platform across wearables equipped with tiny Al accelera- tors [13]. Another line of studies utilized tiny AI accelerators in battery-fire intermittent computing scenarios [1. 6]. Traditionally, hardware accelcrators on low-power AI platforms were capable of only one-shot atomic executions of neural network inference without intermediate result backups. study proposed a toolchain to address this that allows neural networks to execute intermittently on the MAX78000 platform [6]. To the best of our knowledge, there has been work that manipulates data and models to efficiently utilize computing resources considering the unique architecture of tiny Al accelerators.
Image channel extension in CNNs. Several studies have explored augmenting images with ad- ditional information construct multi-channel inputs for Convolutional Neural Networks (CNNs). Liu et al. proposed multi-modality image fusion approach, combining visible, mid-wave infrared, and motion images for enhanced object detection [30], while Wang et al. presented depth-aware CNN for image segmentation [50]. These approaches require extra sensing channels to acquire data, such as infrared cameras and depth cameras. Similarly, other research has incorporated location data to improve performance for segmentation [51] and object detection tasks [29]. For instance, CoordConv [29] pointed out the limitation of traditional CNNs that relied solely on RGB images for the coordinate transformation problem and introduced the augmentation of i and j coordinates, which improved object detection efficiency. However, these methodologies often necessitate additional sensor modalities are tailored for specific applications such as object detection, which restricts their general use. Nevertheless, adapting findings from those studies within DEX could be an interesting future direction.
6 Discussion and conclusion
We introduced DEX, a novel method to enhance CNN efficiency on tiny Al accelerators by augmenting input data across unused memory. Evaluations four image datasets and models showed that DEX improves accuracy without increasing inference latency. This method maximizes the processing and memory capabilities of tiny AI accelerators, making a promising solution for efficient Al model execution on resource-constrained devices.
Limitations and potential societal impacts. We modified only the initial CNN layer due to simplicity, effectiveness, and memory constraints. The first layer, representing image data in three channels (RGB), has the most unused processors after initial data assignment. Extending channels at the first layer significantly increases data utilization with minimal impact on model size. This approach aligns with the design of weight memory in tiny Al accelerators, which maximizes model capacity by collective use across processors, We think DEX might be less effective in certain tasks where incorporating more pixel information is not beneficial. In those cases, alternative data extension strategies might be used instead of patch-wise even sampling to utilize the additional channel budget. While our focus was on small models supported by the MAXX7000/MAX7880 evaluating larger models could be valuable, given rapid AI hardware advancements. Regarding societal impact, leveraging additional processors and memory to improve accuracy might increase carbon emissions [46], highlighting the need to balance accuracy improvements with environmental sustainability.
platforms,
10

---

References
[1] Abu Bakar, Rishabh Goel, Jasper de Winkel, Jason Huang, Saad Ahmed, Bashima Islam, Przemyslaw Pawelczak, Kasım Sinan Yildırm,, and Josiah Hester. Protean: An energy-efficien and heterogeneous platform for adaptive and Proceedings of the 20th ACM Conference Embedded Networked Sensor Systems, pages 207-221, 2022.
battery-free computing. In
[2] Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food- 101 - mining discriminative components with random forests. In European Conference Computer Vision, 2014.
[3] Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT 19th International Conference on Computational StatisticsPuris Frunce, August 22-27, 2010 Keynote, Invited and Contributed Papers, pages 177-186. Springer, 2010.
[4] Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. Once-for-all: Train one network and specialize it for efficient deployment. In International Conference on Learning Representutions, 2020.
[5] Han Cai, Ligeng Zhu, and Song Han. Proxylessnas: Direct neural architecture search on target task and hardware. In International Conference on Learning Representations, 2019.
[6] Luca Caronti, Khakim Akhunov, Matteo Nardello, Kasım Sinan Yıldırm, and Davide Brunelli. Fine-grained hardware acceleration for efficient batteryless intermittent inference on the edge. ACM Transactions on Embedded Computing Systems, 22(5):1-19, 2023.
[7] Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv:1805.00088. 2018.
[8] Google Coral Micro. May. 2024.
Accessed:: 20
[9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.
[10] Igor Fedorov, Ryan P Adams, Matthew Mattina, and Paul Whatmough. Sparse: Sparse architecture search for cnns resource-constrained microcontrollers. Advances in Neural Information Processing Systems, 32, 2019.
[11] Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested 101 object categories. Computer Vision and Puttern Recognition Workshop. 2004.
[12] Greenwaves Technology.
https://greanvare--techhee
com/
how-power-prone wwwppssor......... Accessed: 20 May. 2024.
[13] Taesik Gong, Si Young Jang, Utku Günay Acer, Fahim Kawsar, and Chulhong Min. Collab- orative inference via dynamic composition of tiny ai accelerators on meus. arXiv preprint arXiv:2401.08337, 2023.
[14] Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-255 object category dataset. 2007.
[15] Song Han, Huizi Mao, and William Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. International Conference on Learning Representations (ICLR), 2016.
[16) Seyyed Hossein Hasanpour, Mohammad Rouhani, Mohsen Fayyaz, and Mohammad Sabokrou. Lets keep simple, using simple architectures to outperform deeper and more complex archi- tectures. arXiv preprint arXiv: 1608.06037, 2016.
[17] Yihui He, Xiangyu Zhang, and Jian Sun. Channel pruning for accelerating very deep neural networks. Proceedings of the IEEE international conference on computer vision, pages 1389-1397, 2017.
11

---

[18] Jeremy Howard. Imagenette. httpp:///ggtttoo.ca////aaaaaaagg Accesesd 20 May. 2024.
[19] Analog Devices Inc. Ai8x synthesis repository, http::////gttth..cco///aa/lalgge ai8x- syythesssssss, 2024. Accessed: 20 May. 2024.
[20] Analog Devices Inc. Ai8x training repository. https: http::///gtttttt http://ghhhh.ccemmaaaa/n/ee/eeee
ai aiBx-training, 2024. Accessed: 20 May. 2024.
[21] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient pnteger-arihmmeti--ne inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2704-2713, 2018.
[22] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference Learning Representations (ICLR), San Diega, CA. USA, 2015.
[23] Young D Kwon, Rui Li, Stylianos I Venieris, Jagmohan Chauhan, Nicholas D Lane, and Cecilia Mascolo. Tinytrain: Deep neural network training at the extreme edge. arXiv preprint arXiv:330.009988 2023.
[24] Edgar Liberis, Lukasz Dudziak, and Nicholas D Lane. pmas: Constrained neural architecture search for microcontrollers. In Proceedings of the 1st Workshop on Machine Learning and Systems, pages 70-79, 2021.
[25] Edgar Liberis and Nicholas D Lane. Differentiable neural network pruning to enable smart applications on microcontrollers. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 6(4):1-19, 2023.
[26] Lin, Wei-Ming Chen, Han Cai, Chuang Gan, and Song Han. Mcunety2: Memory-efficient patch-based based inference for tiny deep learning. arXiv preprint 10..v:::1101135522 2021.
[27] Ji Lin, Yongming Rao, Jiwen Lu, and Jie Zhou. Runtime neural pruning. Advances in neural information processing systems, 30, 2017.
[28] Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan, and Song Han. On- device training under 256kb memory. Advances Neural Information Processing Systems, 35:22941-229551 2022.
[29] Rosanne Liu, Joel Lehman, Piero Molino, Felipe Petroski Such, Eric Frank, Alex Sergeev, and Jason Yosinski. An intriguing failing of convolutional neural networks and the coordcony solution. Advances in neural information processing systems, 31, 2018.
[30] Shuo Liu and Zheng Liu. Multi-channel enn-based object detection for enhanced situation awareness. arXiv preprint arXiv: 2017.
[31] Zechun Liu, Haoyuan Mu, Xiangyu Zhang, Zichao Guo, Xin Yang, Kwang-Time Cheng, and Jian Sun. Metapruning: Meta learning for automatic neural network channel pruning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3296-3305. 2019.
[32] Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang Learning efficient convolutional networks through network slimming. In Proceedings of the IEEE international conference on computer vision, pages 2736-2744,2011
cessed: 20 May. 2024. [33] Analog MAX32650.
https://www http::/wwwwaaaall...o. http://////oooouccs//aa/33555550 html. Ac-
[34] Analog MAX78000. cessed: 20 May. 2024.
https://www. http:://wwwwwaalg..ommo analog.
http:://ww..aallgoooo//aa//aa/ap cwt/////pproduttte/a///00...... Ac-
[35] Cutting the AI Power Cord: Technology to Enable True Edge Inference. https: //cms. wwiyml. hwtpppoooooetttutllaa//aaaaaaaaa and Rowooorettecseeell.tee 201027. .df.. Accessed: 20 May. 2024.
Mett0//tinnL__aaaae Talks
12

---

[36] Analog MAX7800FFHH,,
httpe://ww..aaalgg.coo/////o///
evaluation -aariorreeadaaeeeeeeee and- wwt.ta///eelllttooooorott/t///fe
html. Accessed: 20 May. 2024.
[37] Analog MAX78002. cessed: 20 May. 2024.
Ac-
[38] Analog MAX78002EVKIT.
ialluation-hardwane wwwpwwarr--nn--eeee -nd- wor-ssoftaarree htt.a////aaaalltoooooorerrttt//a html. Accessed: 20 May. 2024.
[39] Julian Moosmann, Marco Giordano, Christian Vogt, and Michele Magno. Tinyissimoyolo: A quantized, low-memory footprint, tinyml object detection network for low power microcon- trollers. In 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pages 1-5, IEEE, 2023.
[40] Arthur Moss, Hyunjong Lee, Lei Xun, Chulhong Min, Fahim Kawsar, and Alessandro Mon- tanari. Ultra-low power dnn accelerators for iot: Resource characterization of the max78000 In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pages 2022.
[41] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, al. Pytorch: An imperative style, sigh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[42] Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xntr-net: Imagenet classification using binary convolutional neural networks. In European conference on computer vision, pages 525-542 Springer, 2016.
[43] Thomas Rüegg, Marco Giordano, and Michele Magno. Kp2dtiny: Quantized neural keypoint detection and description on the edge. In 2023 IEEE 5th International Conference Artificial Intelligence Circuits and Systems (AICAS), pages 1-5. IEEE, 2023.
[44] Manuele Rusci, Alessandro Capotondi, and Luca Benini. demory-driven mixed low precision quantization for enabling deep network inference on microcontrollers. Proceedings of Machine Learning and Systems, 2:326-335, 2020.
[45] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang Chich Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510 451004520, 2018
[46] Roy Schwartz, Jesse Dodge, Noah A. Smith, and Oren Etzioni. Green ai. Commun. ACM, 63(12):54-63, nov 2020.
[47] STM32F7 Series. stm32f7-series. html. Accessed: 20 May. 2024.
https://ww, st http:///cccoottoooleeeeeerrrrrre
[48] Mingxing Tan and Quoc Le. Efficientnetv2: Smaller models and faster training. In International conference on machine learning, pages 10196-10106. PMLR, 2021.
[49] Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han. Haq: Hardware-aware automated quantization with mixed precision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8612-8620, 2019.
[50] Weiyue Wang and Ulrich Neumann. Depth-aware enn for rgb-d segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 135-150, 2018.
[51] Zhenyi Wang and Olga Veksler. Location augmentation for enn. 1807.000 2018.
arXiv preprint
[52] Hong-Sheng Zheng, Yu- Yuan Liu, Chen-Fong Hsu, and Tsung Tai Yeh. Streamnet: Memory- efficient streaming tiny deep learning inference on the mierocontroller. Advances in Neural Information Processing Systems, 36, 2024.
13

---

[53] Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv: 0600,, 2016.
[54] Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally. Trained ternary quantization. In International Conference on Learning Representations, 2016.
14

---

A Experimental details
For all experiments conducted in the paper, used three different random seeds (0, 1, 2) and reported the average accuracy with standard deviations.
A.1 Tiny Al accelerator platforms
MAX78002 (12mm x 12mm)
MAX78000 (8mm 8mm)
MAX78002
MAX78000 Feather Board
EV Kit
(a) MAX78000 Feather Board [36].
(b) MAX78002 Evaluation (EV) Kit [38].
Figure 9: Two tiny AI accelerator development platforms used our work. Note that although the development platform is bulky, the actual size of the accelerators is tiny (e.g., 1mmm&mm for MAX78000). All data storing and model inference is done only in the Al Accelerator part (MAX78000 and MAX78002).
Table 5: Comparison of MAX78000 and MAX78002.
Component
MAX78000 [34]
MAX78002 [37]
MCU Processor
Arm Cortex-Ma (100 MHz), RISC-V
Arm Cortex-M4 (120 MHz), RISC-V
Flash Memory
512 KB
2.5 MB
SRAM
128 KB
384 KB
CNN Processor
64 parallel CNN processors
64 parallel CNN processors
Data Memory
512 KB
1.3 MB
Weight Memory
432 KB
2 MB
Bias Memory
2 KB
8 KB
In this paper, we focus the MAX78000 [34] and MAX78002 [37] our primary platforms since they are the most popular research platforms [1, 6, 13, 39, 40, 43] owing to the disclosed hard- ware details and open-source tools, enabling in-depth analysis and modification of their operations. Figure 9 shows our testbed. Note that all operations required for the model inference are done under the AI accelerator part (highlighted with the red boxes), while the entire boards are bigger for development purposes. For on-device deployment and measurement, we used MAX78000 for SimpleNet and WideNet and MAX78002 [37] for EfficientNerV2 and MobileNetV2, following the memory requirement. For the sake of explanation. we assumed each processor is mapped to one memory instance in this paper, although MAX78000MAXX8880 together to communicate with four processors in reality. Our experiments were conducted on the actual implementations. Table 5 compares MAX78000 and MAX78002.
group four data memory instances
A.2 Model training
15

---

We followed the official model training code for MAX78000 and MAX78002 platforms [20]. Here, we detail the hyperparameters used in the official training code. We trained models with NVIDIA A40 GPUs
For all models, quantization-awant training is conducted with support for batch normalization after convolutional layers through batch normalization fusing [21]. This fusing operation integrates the effects of batch normalization directly into the parameters of the preceding convolutional layer by adjusting the weights and bias values. Consequently, after the fusing/folding process, the network no longer contains any batch normalization layers. Instead, the effects of batch normalization are reflected in the modified weights and biases of the preceding convolutional layers.
SimpleNet. SimpleNet [16] was trained for 300 epochs using the Adam optimizer [22] with an initial learning rate of 0.001 and batch size of 32. A multi-step learning rate scheduler was used with milestones set at epochs 100, 150, and 200, and a multiplicative factor of learning rate decay value of 0.25. Quantization-aware training (QAT) was introduced starting at epoch 240. During QAT, a shift quantile of 0.985 was applied to manage activation ranges. The weight precision was primarily set to 2 bits. However, exceptions were made for certain layers: the 1st convolutional layer utilized 8-bit weights, while the 2nd, 1 11th, 12th, 13th, and 14th convolutional layers used 4-bit weights.
WideNet. WideNet [16] was trained for 300 epochs using the Adam optimizer [22] with initial learning rate of 0.001 and a batch size of 100. A multí-step learning rate scheduler was used with milestones set at epochs 100, 150, and 200, and a multiplicative factor of learning rate decay value of 0.25. Quantization -awate aware training (QAT) was introduced starting at epoch 240. During QAT, a shift quantile of 0.985 was applied to manage activation ranges. The weight precision was primarily set to 2 bits. However, exceptions were made for certain layers: the I st convolutional layer utilized 8-bit weights, while the 2nd, 1 1th, 12th, 13th, and 14th convolutional layers used 4-bit weights.
EfficientNetV2. EfficientNetV2 [48] was trained for 300 epochs using the Adam optimizer [22] with an initial learning rate of 0.001 and a batch size of 100. A multi-step learning rate scheduler was used with milestones set epochs 50, 100, 150, 200, and 250 and multiplicative factor of learning rate decay value of 0.5. Duantization-awarl training (QAT) was introduced starting at epoch 210. During QAT, shift quantile of 0.995 was applied to manage activation ranges. The weight precision was primarily set to bits.
MobileNetV2. MobileNetV2 [45] was trained for 300 epochs using the stochastic gradient descent optimizer (SGD) [3] with an initial learning rate of 0.1 and batch size of 128. A multi-step learning rate scheduler was used with milestones set epochs 100, 150, 175, and 250 and a multiplicative factor of learning rate decay value of 0.235. Quantization-awan starting at epoch 200. During QAT, a shift quantile of 1.0 was applied to manage activation ranges. The weight precision was primarily set to 8 bits.
training (QAT) was introduced
A.3 Datasets
Imagenette [18] is a smaller, more manageable subset of ImageNet [9], containing 10 classes. These classes include tench, English springer, cassette player, chain saw, church, French horn, garbage truck, gas pump, golf ball, and parachute. ImageNette has 9469/3925 train/test samples with the original image shape of 3 x 350 350. All images were normalized with the ImageNet mean (0.485, 0.456, 0.406) and standard deviations (0.229, 0.224, 0.225), and then converted to Q7 format (one byte per data) to support on-device inference with the tiny Al accelerator platforms (MAX78000 and MAX78002).
ImageNette.
Caltech101. Caltech101 [11] is a dataset composed of images representing objects from 101 different categories, in addition to a background clutter category. Each image features a single object and is labeled accordingly. The number of images per category ranges from approximately 40 to 800, resulting in total of around 8677 images. 1altech101 has 6941/1736 train/test samples and the original image shape of 3 300 x 300. All images were normalized with the ImageNet mean (0.485, 0.456, 0.406) and standard deviations (0.229, 0.224, 0.225), and then converted to Q7 format (one byte per data) to support on-device inference with the tiny Al accelerator platforms (MAX78000 and MAX78002).
16

---

Caltech256. Caltech256 [14] built upon its previous version, Caltech101, offering enhancements such larger category sizes, additional and more extensive clutter categories, and increased overall difficulty. The dataset contains 29780 images across 256 classes after removing the clutter class. Caltech256 has 23824/5956 train/test samples with the original image shape of 3 x 300 x 300. All images were normalized with the ImageNet mean (0.485, 0.456, 0.406) and standard deviations (0.229, 0.224, 0.225), and then converted to Q7 format (one byte per data) to support on-device inference with the tiny Al accelerator platforms (MAX78000 and MAX78002).
Food101. Food101 [2] includes 101 food categories, each with 750 images for training and 250 images for testing, which is a total of 101000 images. The original images were rescaled to have maximum side length of 512 pixels. This dataset has 75750/25250 train/test samples and the original image with the shape of 3 512 x 512. All images were normalized with the ImageNet mean (0.485, 0.456, 0.406) and standard deviations (0.229, 0.224, 0.225), and then converted to Q7 format (one byte per data) to support on-device inference with the tiny Al accelerator platforms (MAX78000 and MAX78002).
A.4 Baseline details
Downsampling. Downsampling straightforward method that collects samples evenly distributed across the original image. This approach is equivalent to the case when the number of channels is equal three in DEX.
CoordConv. CoordConv [29] pointed out the limitation traditional CNNs that relied solely on RGB images for the coordinate transformation problem and introduced the augmentation of i and coordinates, which improved object detection efficiency. We referred to the Pytorch implementation of CoordConv² for implementing this baseline.
The authors of CoordConv also introduced the third channel for an r co- ordinate, where r = (ii - h/2)2 + (j w/2)22 which they found effective in some experiments. Similar to CoordConv, referred the Pytorch implementation of CoordConv for implementing this baseline.
CoordCony (with r).
A.5
Alternative data channel extension methods' details
(a) Repetition
Rotation
Tile
(d) Patch-wise
(e) Patch-wise
sequential sampling 1- 6 38 26-3
random sampling x=
Same
Ptj
Ptj
samplas
Figure 10: Visulaization of four alternative data extension methods.
Repetition. Repetition (Figure 10(a)) repeats the same downsampled image across the channels until it reaches the maximum possible number input channels, which is the same as the number of data memory instances (64).
Rotation. Rotation (Figure 10(b)) generates slightly different images through rotating images from the downsampled image. It makes rotated images until it reaches the maximum possible number of input channels. The angle rotation ranges from -30 to 30 degrees. For instance, given a downsampled three-channel image input and target channel size of 64, it generates rotated images with an angle linearly spaced between -30 to 30 degrees.
Tile. Tile (Figure 10(c)) divides the original image into multiple tiles and stacks those tiles across channels. Specifically, given the number of images take K [Ce it finds the nearest square
17

---

number S that is higher than or equal to K, (e.g., = 52 22 when K = 22). The original image then divided into equal-sized patches. Each patch is subsequently downsampled the target size. The downsampled patches are collected and concatenated along the channel dimension, forming new image with the desired number of channels. the total number of patches exceeds the target number of channels, the excess patches are discarded.
Patch-wist sequential sampling. Patch-wise sequential sampling (Figure 10(d) is similar to DEX but it involves sequential sampling within patch instead of even sampling. Specifically, it samples the first K' samples for each patch and follows the same channel-wis stacking procedure in DEX.
Patch-wise random sampling. Patch-wise random sampling (Figure 10(e)) is similar to DEX but it involves random sampling within a patch instead of even sampling. Specifically, it samples mundomly-selectee K samples for each patch and follows the same channel-wist stacking procedure DEX.
B
Additional Experiments
B.1 Overhead of the channel expansion on devices
Channel expansion latency. The latency of the channel expansion process depends on the pro- cessor's computational capability. During our evaluation, we pre-processed data on powerful server, and thus data processing was negligible. We additionally conducted data processing on the sorra-low-power
ultra-low -owerr MCU processor the board (Arm Cortex-M4) to understand the data processing overhead on less-capable devices. We measured the overhead of applying DEX to expand channels from 3 X 224 x 224 image (a typical size for ImageNet) to 64 x 32 32 (the highest channel expansion used in our accelerators) on the MAX78002's Arm Cortex-M4 (120MHz).
This process took 2.2 ms on the Arm Cortex-M4. In terms of memory, this addition took the SRAM memory of 62KB (64 x 32 x 32 Bytes - 3 x 32 x 32 Bytes) on the processor. However, since DEX extends data to size that the data memory in the Al accelerator can accommodate, this additional memory will not be an issue from the Al accelerator's perspective.
Impact on end-to-end inference performance. Note that the MCU processor and the Al accel- erator are independent processing components that run in parallel. This means that if the inference latency on the accelerator is higher than the data processing latency, data can be pre-processed for the next inference during the current inference (and thus data processing latency can be hidden). For inference, the inference latency of EfficientNet (11.7ms) is higher than the data processing latency of 2.2ms, and thus the inference throughput remains the same under continuous inference.
However, this depends on the scenario. The end-to-end tod impact of data processing latency depends on the processor's computational capability, the dimension of the data, and the size of channel expansion. For instance, in scenarios where data processing is done and transferred in more capable machines (e.g., cloud servers, smartphones, etc.) than the MCU processor on the tiny Al accelerator, the impact of data processing can be even more negligible.
B.2 Power consumption
We measured the power consumption of the inference on MAX78000 by varying the size of the channel extension with a Monsoon Power Monitor. The result is shown in Table 6. As the number of channels increased, power consumption increased accordingly. This because higher number of channels uses more processors in the Al accelerator, leading to increased power consumption.
Table 6: The power consumption of inference measured by varying the size of the channel extension with a Monsoon Power Monitor. All numbers are in milliwatts (mW).
Model
Chan = Chan = 6
Chan 18
Chan = 36
Chan = 64
SimpleNet
53.82
53.85
58.21
61.42
68.9 9
WideNet
60.74
61.37
63.76
67.92
77.14
18

---

C
Example images generated from DEX
Original image! I
Downsampled (k 0)
Downsampled (k 1)
Downsampled (ir = 2)
Downsampled (k 3)
Downsampled (k = 4)
Downsampled (k = 5)
Downsampled (k = 6)
Figure 11: Examples images generated from an original 3 x 350 x 350 image (ImageNette) to
3 x 32 downsampled image via DEX. k 0 to k 6 cases shown only. Each generated image contains different pixel information, which collectively enhances feature learning in CNNs.
Original image
Downsampled 0)
Downsampled 1)
Downsampled (k 2)
Downsampled 3)
Downsampled 4)
Downsampled 5)
Downsampled (k 6)
Figure 12: Examples images generated from original 3 x 350 x 350 image (ImageNette) to
3 x 32 x 32 downsampled image via DEX. & = 0 to k = 6 cases are shown only. Each generated image contains different pixel information, which collectively enhances feature learning in CNNs.
D License of assets
Datasets. ImageNette dataset (Apache-2.0 license), Caltech101 (CC BY 4.0), Caltech256 (CC BY 4.0), and Food101 dataset (MIT license).
19

---

Original image! I
Downsampled (k 0)
Downsampled (k = 1)
Downsampled (it = 2)
Downsampled (k 3)
Downsampled (k = 4)
Downsampled (k = 5)
Downsampled (k = 6)
Figure 13: Examples images generated from an original 3 x 350 x 350 image (ImageNette) to
3 x 32 x 32 downsampled image via DEX. k 0 to k 6 cases are shown only. Each generated image contains different pixel information, which collectively enhances feature learning in CNNs.
Codes. AI8X-training for MAX78000 and MAX78002 (Apache-2.)) 2.0 license), AI8X-synthesii for MAX78000 and MAX78002 (Apache-2.0 2.0 license), and PyTorch implementation of CoordConv (MIT license).
20

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiiaton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The main claims in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
21

---

Justification: See $6.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
No theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Experimental details are in $4 and Appendix A.
Guidelines:
The answer NA means that the paper does not include experiments.
22

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Yes. the source code is available https://github. com/
- Labs/data---aaaeaaa channel-extension
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: //nips.cc. wtwpsscccppbllcc///cccccc http//pbllccggueeeeeee guwdee/ /CodeSubmis http:////gguidee/ddeuuuuuiiiiii ssi ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
23

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
Experimental details are in $4 and Appendix A.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: See 4.2. We ran the experiments with three random seems (0.1,2) and reported the standard deviations.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-simma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See $2 $4, and Appendix A.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https:/ httppp:///eurips.... cc hpppppi//ttiissutiiiitt
Answer:
[Yes]
Justification:
We follow the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See $6.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: No such components.
Guidelines:
The answer NA means that the paper poses such risks.
25

---

Released models that have high risk for misuse or dual use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: See $4 and Appendix D.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No crowdsourcing conducted.
26

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No IRB required.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
27


Reviews:
[{"review_id": "DGrDzLRat3", "reviewer": "Reviewer_ySD7", "summary": "This paper indicates that current AI accelerators with limited data memory often require downsampling input images, which leads to reduced accuracy. Therefore, the proposed Data channel EXtension (DEX) includes additional spatial information from original images as informative input through two procedures: patch-wise even sampling and channel-wise stacking. This effectively extends data across input channels. As a result, DEX enables parallel execution without increasing inference latency. The numerical experiments consistently show improved model performance on four datasets.", "questions": "-\tThe primary concern of this paper is that the proposed image sampling approach is too simple to make a significant technical contribution. Moreover, the proposed method relies on the assumption of having unused per-processor memory instances to initiate the sampling process, a condition that may not always be met.", "limitations": "Included", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "•\tThe proposed method is easy to understand, with clearly written paragraphs and well-organized sections.\n\n•\tThe experiments conducted demonstrate the effectiveness of the proposed method.", "weaknesses": "-\tThe proposed data channel extension requires the assumption that only a limited number of processors tied to memory instances are utilized while the remaining processors remain idle. However, it is not ensured that such an assumption is always true to trigger the proposed method.\n-\tThe proposed method is as simple as an implementation trick; hence, the technical contribution is limited. \n * The compared channel extension methods are all proposed by the authors and hence failed to show a fair comparison.\n * It is curious what the performance of patch-wise random sampling could achieve.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your time and effort in providing us with thoughtful comments. We respond to your question in what follows. Please also refer to the *global response* we posted together.\n\n---\n*Weakness 1) The proposed data channel extension requires the assumption that only a limited number of processors tied to memory instances are utilized while the remaining processors remain idle. However, it is not ensured that such an assumption is always true to trigger the proposed method.*\n\n\nWe acknowledge that our work targets tiny AI accelerators that feature parallel processors and per-processor memory instances for rapid data access and parallelization, as we described in Section 2. **Tiny AI accelerators with these hardware-level optimizations are crucial** for performance improvement compared to traditional MCUs. We believe these tiny AI accelerators will be widely adopted in various small devices, such as recent AI-capable smart earbuds, patches, watches, glasses, wristbands, and shoes [r1, r2, r3, r4].\n\nGiven the extensive tinyML research on the STM32 MCU series and edge AI work on the NVIDIA Jetson series in the past years, we believe **these tiny AI accelerators will become a key enabling force for true on-device AI in tiny devices such as wearables**. In this context, we focus on the tiny AI accelerator platforms (MAX78000 and MAX78002) since they are not only the most widely used tiny AI accelerator research platforms [1, 6, 13, 39, 40, 43] but also feature these hardware-level optimizations. Our insights into utilizing parallel processors for accuracy improvement without sacrificing inference latency will remain valuable as long as future AI platforms continue to incorporate these tiny AI accelerators.\n\n\n\n[r1] Ananta Narayanan Balaji and Li-Shiuan Peh. 2023. AI-On-Skin: Towards Enabling Fast and Scalable On-body AI Inference for Wearable On-Skin Interfaces. Proceedings of the ACM on Human-Computer Interaction 7, EICS (2023), 1–34.\n\n[r2] OmniBuds - Sensory Earables powered by AI accelerators.\n\n[r3] Hearables with GAP9. TWS Processor with GAP9.\n\n[r4] Shift Moonwalkers.\n\n\n\n---\n*Weakness 2) The proposed method is as simple as an implementation trick; hence, the technical contribution is limited.*\n  -  *The compared channel extension methods are all proposed by the authors and hence failed to show a fair comparison.*\n  - *It is curious what the performance of patch-wise random sampling could achieve.*\n\n**Simplicity of the method**: While the proposed method might seem simple, we provide an in-depth analysis of its rationale, impact, utilization, and constraints in Section 3. Also, this simplicity allows our solution to be generally applicable to various types of models and AI accelerators. **We would like to mention that the other reviewers pointed out our simple and effective solution as a strength of our paper** (UAVo: “simple yet compelling idea”; k2BH: “simple yet effective solution”; Zbte: “both simple and effective”). Our approach is new and novel, specifically designed for emerging tiny AI accelerators, and we have shown its effectiveness. We believe many impactful papers, especially in the field of AI/ML, present simple yet impactful solutions. \n\n**Baselines**: This area has been hardly explored as tiny AI accelerators are new platforms, resulting in **few existing baselines**. In our original submission, **we did conduct a comparative study with existing channel manipulation methods proposed by prior art** such as Downsampling, CoordConv, and CoordConv (r) in Tables 1 and 2. While these baselines were not originally designed for our target platforms, we believe they provide meaningful comparisons that validate our design rationales—data channel extension to achieve accuracy improvement without extra latency. In addition, we compared with other possible channel extension strategies in Table 4 proposed by us (except for Downsampling which is widely used) due to the lack of proper baselines in the literature. We believe this is a fair comparison incorporating existing approaches and possible alternatives. \n\n\n**Patch-wise random sampling**: Thanks for the suggestion of comparison with patch-wise random sampling. Following your suggestion, we measured the performance of it and integrated it with Table 4 as shown below. DEX’s data extension algorithm outperformed the baselines, including patch-wise random sampling. We will incorporate this into our final draft.\n|             Method             | InputChan | InfoRatio (X) |    Accuracy    |\n|:------------------------------|:---------:|:-------------:|:--------------:|\n| Downsampling                   |         3 |           1.0 |   57.8 ± 1.2   |\n| Repetition                     |        64 |           1.0 |   56.3 ± 0.8   |\n| Rotation                       |        64 |           1.0 |   55.7 ± 0.6   |\n| Tile per channel               |        64 |          21.3 |   39.3 ± 0.9   |\n| Patch-wise seq.                |        64 |          21.3 |   60.4 ± 1.5   |\n| **Patch-wise random sampling** |        64 |          21.3 |   60.4 ± 1.0   |\n| DEX                            |        64 |          21.3 | **61.4 ± 0.6** |\n\n\n---\n\n*Question 1) The primary concern of this paper is that the proposed image sampling approach is too simple to make a significant technical contribution. Moreover, the proposed method relies on the assumption of having unused per-processor memory instances to initiate the sampling process, a condition that may not always be met.*\n\nPlease see our responses to Weakness 1 and 2 above.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MZUU1MYsZm", "reviewer": "Reviewer_UAVo", "summary": "Recent advancements in tiny ML accelerators, such as MAX 78000 and MAX 78002, have significantly boosted hardware processing power. On one hand, these accelerators feature 64 parallel processors with per-processor memory instances, enhancing CNN inference speed compared to traditional MCUs. On the other hand, downsampling of input images due to limited data memory can lead to accuracy degradation. To address this, this paper proposes DEX, which integrates patch-wise even sampling and channel-wise stacking to incorporate spatial information from original images into input images. Evaluation results demonstrate that DEX improves accuracy without introducing additional latency.", "questions": "1. Lines 111 to 112 mention that 512KB memory is divided into 64 segments, giving each core an 8KB dedicated memory instance (as shown in Figure 1). Is this scenario realistic? In other words, does each core physically have its own private 8KB memory? Typically, all processors share the same 512KB memory. Moreover, the Block Diagram from the MAX78000 specification in Ref [34] does not indicate these dedicated memory instances.\n2. Line 114 states that an image with an input shape of 3x224x224 may not fit MAX78000 even with Q7 format due to memory limitations for each channel. However, the MAX78000 product datasheet mentions \"input image size up to 1024x1024 pixels\" (Ref [34]). Could this be double-checked?\n3. What if the channels of the intermediate layers are significantly fewer than the number of cores? Would it still be feasible or straightforward to extend DEX to those layers?\n4. This work is similar to data augmentation but is designed for specific devices. How important or valuable is this hardware device in the context of TinyML? Even for the MAX78000/MAX78002, the contribution seems limited since DEX has only been applied to the first layer, and the accuracy improvement appears to be limited.\n\nRef [34]: https://www.analog.com/en/products/max78000.html", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ This paper presents a simple yet compelling idea to tackle CNN inference on a specific tiny ML accelerator.\n+ Figures, such as Figure 5, clearly illustrate the DEX procedure.\n+ The analysis in the paper provides a clear understanding of how DEX operates.", "weaknesses": "- This approach appears suitable only for specific small devices.\n- Some procedures are unclear and require further clarification. Detailed questions are listed below.\n- Limiting the approach to processing only the first layer for simplicity may be a limitation of this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your time and effort in providing us with positive and thoughtful comments. We respond to your question in what follows. Please also refer to the *global response* we posted together.\n\n---\n\n*Weakness 1) This approach appears suitable only for specific small devices.*\n\nPlease see our response to Question 4 below.\n\n---\n\n*Weakness 2) Some procedures are unclear and require further clarification. Detailed questions are listed below.*\n\nThank you for pointing this out. See our response below. We will incorporate the changes. \n\n---\n\n*Weakness 3) Limiting the approach to processing only the first layer for simplicity may be a limitation of this work.*\n\nPlease see our response to Question 3 below.\n\n---\n\n*Question 1) Lines 111 to 112 mention that 512KB memory is divided into 64 segments, giving each core an 8KB dedicated memory instance (as shown in Figure 1). Is this scenario realistic? In other words, does each core physically have its own private 8KB memory? Typically, all processors share the same 512KB memory. Moreover, the Block Diagram from the MAX78000 specification in Ref [34] does not indicate these dedicated memory instances.*\n\n**Each core cannot share the same 512KB memory.** This is due to the *per-processor memory instance* property in tiny AI accelerators, which allows for rapid data access and parallelization. The block diagram in Ref [34] provides only an abstract view of the memory architecture and does not illustrate this detail. For a more detailed explanation, please refer to the MAX78000 User Guide. To be precise, four processors share one data memory instance, as noted in our draft (lines 524-526). Nevertheless, parallelization occurs at the channel level. Hence, the data memory can be viewed as 64 segments in terms of parallelization.\n\n\n---\n\n*Question 2) Line 114 states that an image with an input shape of 3x224x224 may not fit MAX78000 even with Q7 format due to memory limitations for each channel. However, the MAX78000 product datasheet mentions \"input image size up to 1024x1024 pixels\" (Ref [34]). Could this be double-checked?*\n\nNote that 1024 x 1024 pixels = 1048 KB does not fit within the 512KB data memory size. The MAX78000 datasheet description, “Programmable Input Image Size up to 1024 x 1024 pixels,” is feasible using its “streaming mode.” According to the official documentation, “Streaming allows input data dimensions that exceed the available per-channel data memory in the accelerator.” This mode leverages special hardware support, such as the streaming queue in the MAX78000. However, this comes at the cost of increased inference latency. We did not cover this in the paper as it is a special implementation specific to the hardware and may not generalize to other types of hardware. The focus of our analysis was on the standard operation mode.\n\n---\n\n*Question 3) What if the channels of the intermediate layers are significantly fewer than the number of cores? Would it still be feasible or straightforward to extend DEX to those layers?*\n\n**Yes, it is both possible and straightforward to extend DEX to those layers by modifying the output channel size of those layers.** In this work, we focused on modifying the first CNN layer due to simplicity, effectiveness, and memory constraints. The first layer, representing image data in three channels (RGB), has the most unused processors after initial data assignment. Extending channels at the first layer significantly increases data utilization with minimal impact on model size. This approach aligns with the design of weight memory in tiny AI accelerators, which maximizes model capacity by collective use across processors. We discussed this in Lines 317-322 in our manuscript.\n\n---\n\n*Question 4) This work is similar to data augmentation but is designed for specific devices. How important or valuable is this hardware device in the context of TinyML? Even for the MAX78000/MAX78002, the contribution seems limited since DEX has only been applied to the first layer, and the accuracy improvement appears to be limited.*\n\nWhile our solution might look similar to data augmentation, it is specifically designed for tiny AI accelerators to maximize both processor utilization and accuracy improvement. We strongly believe that these **tiny AI accelerators are crucial platforms in the context of tinyML**. The advent of tiny AI accelerators is bringing AI closer to us than ever before, offering reduced latency, low power cost, and improved privacy. These accelerators with small form factors are becoming integrated into wearable devices recently, e.g., smart earbuds, patches, watches, glasses, wristbands, and shoes [r1, r2, r3, r4]. \n\nIn this paper, we focus on the MAX78000 and MAX78002 since they are the most widely used tiny AI accelerator research platforms [1, 6, 13, 39, 40, 43] thanks to their disclosed hardware details and open-source tools, enabling in-depth analysis and modification of their operations. **These tiny AI accelerators are common research platforms, similar to the STM32 series in MCU research and the NVIDIA Jetson series in Edge TPU research**.\n\nIn that context, we believe DEX is an important step in utilizing tiny AI accelerators within TinyML by improving accuracy without sacrificing inference latency. We identify inefficiencies in these accelerators and enhance accuracy through a novel data extension algorithm. We believe that **an average 3.1%p accuracy improvement is meaningful, especially for resource-constraint tiny devices**, and we focused on the first layer due to the reasons explained in our response to Question 3.\n\n\n[r1] Ananta Narayanan Balaji and Li-Shiuan Peh. 2023. AI-On-Skin: Towards Enabling Fast and Scalable On-body AI Inference for Wearable On-Skin Interfaces. Proceedings of the ACM on Human-Computer Interaction 7, EICS (2023), 1–34.\n\n[r2] OmniBuds - Sensory Earables powered by AI accelerators.\n\n[r3] Hearables with GAP9. TWS Processor with GAP9.\n\n[r4] Shift Moonwalkers.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tJ9AQigUky", "reviewer": "Reviewer_k2BH", "summary": "This paper introduces DEX, a novel technique designed to enhance the efficiency of DNN inference on resource-constrained tiny AI accelerators by extending the data channels. This approach aims to improve both resource utilization and inference accuracy by incorporating additional spatial information from the original image through patch-wise even sampling and channel-wise stacking. The authors identify that the limited memory budget on resource-constrained tiny AI accelerators requires downscaling the input image, which can degrade model quality and underutilize resources. By extending the data channel, the proposed method retains more information from the input image, thereby improving inference accuracy and maximizing resource utilization without increasing inference latency. Evaluations on real tiny AI accelerator devices demonstrate a 3.1% accuracy improvement with no additional inference latency.", "questions": "- What is the overhead of the channel expansion process in terms of computational and memory resources? How does this impact end-to-end inference performance? And on which hardware does such pre-processing happen?\n- How does the power consumption vary when using the proposed technique?\n- Can you comment on the applicability of DEX to other DNN tasks, such as object detection or natural language processing, which are also common tasks for tiny devices? Will DEX still be effective in improving inference accuracy for those tasks?", "limitations": "The authors have acknowledged the limitations regarding the exploration of larger models. However, they have not addressed the broader applicability of DEX to other tasks beyond classification. It would be beneficial to include evaluations on other applications such as object detection, segmentation, and more to demonstrate the generalizability of the proposed method.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper identifies the accuracy degradation and resource under-utilization issues for DNN inference on tiny AI accelerators and proposes a simple yet effective solution that can improve inference accuracy without additional inference latency.\n- The evaluations are conducted on real hardware devices.\n- The paper is well-written with the motivation, methodology and results being presented in a clear and logical manner.", "weaknesses": "While the paper has notable strengths, several areas could be improved:\n- End-to-end Performance: The impact of proposed DEX on the end-to-end latency and throughput is not evaluated. Although the authors claim no increase in inference latency, the overhead of the channel expansion from input RGB images (including several preprocessing steps) needs to be studied.\n- Power/Energy Measurement: The paper does not include the analysis of power consumption and energy efficiency with DEX, which are crucial considerations for tiny AI accelerators.\n- Scope of Models: The study is limited to classification DNN models. The impact on other tasks, such as object detection, face recognition, and more complex applications, should be explored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your time and effort in providing us with positive and thoughtful comments. We respond to your question in what follows. Please also refer to the *global response* we posted together.\n\n---\n\n*Question 1 & Weakness 1) What is the overhead of the channel expansion process in terms of computational and memory resources? How does this impact end-to-end inference performance? And on which hardware does such pre-processing happen?*\n\nThank you for the question. While our work currently focuses on AI accelerators—specifically their utilization and inference latency—considering data processing overhead is an important discussion for practical deployment. Note that the overhead and impact of data processing depend on the target application scenario and benchmark setup.\n\n\n**Overhead of the channel expansion and hardware**: The latency of the channel expansion process depends on the processor's computational capability. During our evaluation, we pre-processed data on a powerful server, and thus data processing was negligible. \n\nWe additionally conducted data processing on the ultra-low-power MCU processor on the board (Arm Cortex-M4) to understand the data processing overhead on less-capable devices. We measured the overhead of applying DEX to expand channels from a 3x224x224 image (a typical size for ImageNet) to 64x32x32 (the highest channel expansion used in our accelerators) on the Arm Cortex-M4 (120MHz).\n\nThis process took 2.2 ms on the Arm Cortex-M4. In terms of memory, this addition took the SRAM memory of 62KB (64x32x32 Bytes - 3x32x32 Bytes) on the processor. However, since DEX extends data to a size that the data memory in the AI accelerator can accommodate, this additional memory will not be an issue from the AI accelerator’s perspective.\n\n**Impact on end-to-end inference performance**: Note that the MCU processor and the AI accelerator are independent processing components that run in parallel. This means that if the inference latency on the accelerator is higher than the data processing latency, data can be pre-processed for the next inference during the current inference (and thus **data processing latency can be hidden**). For inference, the inference latency of EfficientNet (11.7ms) is higher than the data processing latency of 2.2ms, and thus the inference throughput remains the same under continuous inference. \n\nHowever, this depends on the scenario. The end-to-end impact of data processing latency depends on the processor's computational capability, the dimension of the data, and the size of channel expansion. For instance, in scenarios where data processing is done and transferred in more capable machines (e.g., cloud servers, smartphones, etc.) than the MCU processor on the tiny AI accelerator, the impact of data processing can be even more negligible. \n\nWe will incorporate this into our final draft.\n\n---\n\n*Question 2  & Weakness 2) How does the power consumption vary when using the proposed technique?*\n\nThank you for the question. **We measured the power by varying the size of the channel extension with a Monsoon Power Monitor**. The results are as follows:\n\n\n|   Model   | Chan=3 | Chan=6 | Chan=18 | Chan=36 | Chan=64 |\n|:---------:|:------:|:------:|:-------:|:-------:|:-------:|\n| SimpleNet |  53.82 |  53.85 |   58.21 |   61.42 |   68.97 |\n|  WideNet  |  60.74 |  61.37 |   63.76 |   67.92 |   77.14 |\n\nAll numbers are in milliwatts (mW).\n\nAs the number of channels increased, power consumption increased accordingly. This is because a higher number of channels uses more processors in the AI accelerator, leading to increased power consumption. \n\nWe will incorporate this into our final draft.\n\n---\n\n*Question 3  & Weakness 3) Can you comment on the applicability of DEX to other DNN tasks, such as object detection or natural language processing, which are also common tasks for tiny devices? Will DEX still be effective in improving inference accuracy for those tasks?*\n\nThank you for the question. The core idea of this work is to utilize additional channels for extra inputs to improve task accuracy. We validated the generalizability of our method across 16 cases (four datasets and four models). We believe this idea would still be beneficial for other tasks that have a low number of input channels, such as RGB images.\n\nDuring the rebuttal, **we conducted an experiment to see if our idea generalizes to a face detection task**. Specifically, we used the VGGFace2 dataset and the Tiny Single-Shot Detection (Tiny SSD) model [r1]. Due to computational complexity and limited time during the rebuttal period, we downsized the dataset by taking the first 100 identities, resulting in 33K training samples and 2K test samples for 50 epochs. We used the Adam optimizer with a fixed learning rate of 0.001.\n\nHere is the result: \n\n| Method       | Channel Size | mAP     |\n|--------------|:--------------:|:---------:|\n| Downsampling | 3            | 0.65347 |\n| DEX          | 18           | 0.68317 |\n| DEX          | 64           | **0.69307** |\n\n\nThe results show that mAP (mean Average Precision) improved with DEX compared to Downsampling, illustrating that DEX works for a face detection task.\n\nNevertheless, we acknowledge that DEX might be less effective in certain tasks where incorporating more pixel information is less beneficial and where detailed pixel-level information might not significantly improve performance. A thorough evaluation is necessary to verify this for different tasks.\n\nWe will incorporate this in the final draft.\n\n[r1] Tiny SSD: A Tiny Single-shot Detection Deep Convolutional Neural Network for Real-time Embedded Object Detection", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eSfFiiwJsO", "reviewer": "Reviewer_Zbte", "summary": "The paper proposes a novel method to address the accuracy degradation caused by downsampling on small AI processors. The authors observed that the input layer often has a small number of channels, leading to underutilization of the processors. To mitigate this issue, they introduce a technique involving patch-wise even sampling and channel-wise stacking. This method incorporates additional spatial information, thereby improving accuracy while efficiently using processing resources that would otherwise be wasted.", "questions": "* How does the proposed method perform on more complex models and larger datasets beyond the scope of the current evaluation?\n* Are there specific scenarios or types of models where this method might be less effective?", "limitations": "The authors discussed the limitations concerning small models and acknowledged the potential negative societal impact due to the increased use of computational resources to improve accuracy. In my opinion, their discussion is sufficient and adequate.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The motivation for the proposed method is clear and well-justified.\n* The idea of sampling and channel-wise stacking is both simple and effective, demonstrating a practical solution to a common problem in small AI processors.\n* The paper provides well-defined baselines, including normal downsampling and CoordConv, and offers comparisons with various data channel extension approaches. \n* The authors conduct a thorough sensitivity analysis, evaluating accuracy, model size, and latency across different channel sizes.\n* The method shows low latency and minimal model size overhead, making it a promising solution for improving accuracy without significant performance trade-offs.", "weaknesses": "* Further comparisons with a broader range of more complex models could strengthen the evaluation.\n* There could be more discussion of potential limitations and scenarios where the method might be less effective.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your time and effort in providing us with positive and thoughtful comments. We respond to your question in what follows. Please also refer to the *global response* we posted together.\n\n---\n\n*Question 1 & Weakness 1) How does the proposed method perform on more complex models and larger datasets beyond the scope of the current evaluation?*\n\nThank you for the suggestion. We agree that evaluating the proposed method on more complex models and larger datasets would be beneficial. However, the tiny AI accelerators currently face memory and architecture limitations that restrict support for complex and larger models. For instance, WideNet already utilizes 70% of the weight memory (432KB). Given that MAX78000/MAX78002 are the only available platforms with disclosed hardware details and open-source tools, we plan to extend our idea to a wider variety of models with more computationally capable tiny AI accelerators in the future.\n\nAdditionally, during the rebuttal period, we conducted an experiment to see if DEX is applicable to another task, face detection, using the VGGFace2 dataset. The results demonstrated the effectiveness of DEX over the downsampling, with mAP improving from 0.65347 to 0.69307. Please refer to the response to Reviewer k2BH, Question 3 for further details.\n\n\n---\n\n*Question 2 & Weakness 2) Are there specific scenarios or types of models where this method might be less effective?*\n\nThank you for the question. We think DEX might be less effective in certain tasks where incorporating more pixel information is not beneficial. For instance, DEX might be less effective in scene categorization where the overall structure and composition of the scene are more important than the detailed textures or pixel-level variations, such as determining if an image is an indoor or outdoor scene. In those cases, alternative data extension strategies might be used instead of patch-wise even sampling to utilize the additional channel budget.\n\nWe will incorporate this discussion in our final draft.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Global Response

Dear Reviewers,

We appreciate all of you for your positive reviews and for highlighting the **strengths of our work**:

**ySD7:** (1) Easy to understand, (2) clearly written, (3) well-organized, and (4) demonstrates the effectiveness of the proposed method.  

**UAVo:** (1) Presents a simple yet compelling idea, (2) clear illustrations, and (3) in-depth analysis.  

**k2BH:** (1) Identifies important issues in tiny AI accelerators, (2) a simple yet effective solution, (3) evaluation with real devices, (4) well-written, and (5) clear and logical motivation and methodology.

**Zbte:** (1) Well-justified motivation, (2) both simple and effective, demonstrating a practical solution, (3) well-defined baselines, (4) thorough sensitivity analysis, and (5) low latency with minimal model size overhead.

We also sincerely thank the reviewers for their **constructive comments** to improve our work. We have addressed all the questions from reviewers with clarifications and new experiments during this rebuttal period. **We summarize how we addressed the reviewers’ main questions** as follows:

**ySD7:** 
- We clarified the assumption and highlighted its importance in tiny AI accelerators.
- We clarified our technical contribution.
- We clarified that we compared with existing baselines.
- We conducted an experiment to compare with patch-wise random sampling.

**UAVo:**  
- We clarified the memory architecture of the tiny AI accelerators.
- We clarified the input size limitation.
- We discussed the possibility of applying DEX to intermediate layers.
- We highlighted the importance of the tiny AI accelerator platforms and the significance of the result.


**k2BH:**  
- We measured the overhead of data processing with Arm Cortex-M4. 
- We measured the power consumption using a Monsoon Power Monitor.
- We conducted an experiment on a face detection task.

**Zbte:**  
- We clarified the scope of the experiments and conducted an experiment on a face detection task.
- We discussed scenarios where DEX might be less effective.

We will carefully incorporate these points and our responses into our final draft. Thank you once again for your valuable feedback and suggestions.

Sincerely,  
Authors

Meta-review: This paper presents a method to improve the efficiency of running CNN models on tiny AI accelerators. Instead of using direct downsampling, the method involves extending the input data channels to incorporate additional image information into unused data memory and processors. The approach was verified on two tiny AI accelerator platforms with popular vision models and benchmarks, showing enhanced accuracy and efficiency.

The paper is well-written and easy to follow. The evaluation is thorough, and the results show the effective of the proposed methods. During the rebuttal, the authors addressed reviewers’ questions in detail, providing additional data and clarifications. The paper received generally favorable reviews from all reviewers. 

However, the reviewers noted that the method is quite simple, and its technical contribution is somewhat limited. Additionally, the technique is mainly applicable to specific tiny accelerators. Nevertheless, considering the method’s effectiveness, the comprehensive analysis and evaluation, and the significance of tiny AI accelerators in applying CNN models, the paper is recommended for borderline acceptance.
Decision: Accept (poster)