# Analog Features Extractor for Ultra-Low Power Embedded AI Listening and Keyword Spotting

Sebastián Marzetti, Valentin Gies, Valentin Barchasz, Hervé Barthélemy, Hervé Glotin

► To cite this version:

Sebastián Marzetti, Valentin Gies, Valentin Barchasz, Hervé Barthélemy, Hervé Glotin. Analog Features Extractor for Ultra-Low Power Embedded AI Listening and Keyword Spotting. 2022 IEEE/ACS 19th International Conference on Computer Systems and Applications (AICCSA), 2024. hal-05005685

HAL Id: hal-05005685

https://hal.science/hal-05005685v1

Submitted on 25 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Analog Features Extractor for Ultra-Low Power Embedded AI Listening and Keyword Spotting

Sebastián Marzetti*‡, Valentin Gies*‡, Valentin Barchasz*‡, Hervé Barthélemy*‡, Hervé Glotin†‡

*Aix Marseille Université, Université de Toulon, CNRS, IM2NP - UMR 7334, France

†Université de Toulon, Aix Marseille Univ, CNRS, LIS, DYNI, Marseille, France

‡Centre d'Intelligence Artificielle en Acoustique Naturelle, cian.lis-lab.fr, Université de Toulon, France

**Abstract**—In this paper, a novel ultra-low power audio system is designed and tested on keyword spotting (KWS). It implements mixed (analog-digital) processing and allows always-on detection with an average power consumption of < 77 µW, detecting one word every 30 seconds. Combining mixed signal processing, it achieves 88% classification accuracy between 8 classes. It is based on an always-on Analog Features Extractor (AFE), which provides spectral information, and main processor wakes-up only when necessary for minimal power consumption. Then, this information is sampled by a low frequency ADC to compose a spectrogram processed with a Convolutional Neural Network (CNN) for classification. It can be used for long term environmental monitoring or intelligent Internet of Things (IoT) using small batteries such as a single CR2032 coin cell to reach up to 1 year of autonomy.

***Index Terms***—Ultra Low-Power, Keyword Spotting, Signal Processing, Embedded System, Mixed signal processing, Analog, Digital, Embedded Artificial Intelligence, Machine Learning, Voice Detection, Long Term Monitoring, Soundscape Monitoring

## I. INTRODUCTION AND RELATED WORKS

Ultra-low power systems at-the-edge are a key for embedding AI in small size devices, having constrained batteries. This feature allows them to last for years of continuous service. Keyword spotting (KWS) is an interesting case as it makes possible an easy interface with humans: for example, a voice control switch for house lights, which avoids cable connections. It can be extended to several vocal command, or even to soundscape monitoring. KWS supports a limited number of commands as large vocabulary models can not be integrated into small microcontrollers. However, it has some advantages such as latency, removal of transmission of sensible data and most important a reduced power consumption.

Most of proposed works are implemented with discrete components based on fully digital processing, such as [1] with a KWS system for controlling a robot where the input audio signal is sampled at 16 kHz and digitally processed for features extraction and deep neural network classification. In [2]–[5] KWS systems are based on Convolutional Recurrent Neural Network implemented with fully digital processing on a Raspberry Pi 3B+ and on microcontrollers. All these works implement Mel-Frequency Cepstrum Coefficient (MFCC) as features extraction, allowing high classification accuracy

(91% for 7 classes [1], 98.05% for 11 classes [2], and in [3] up to 98%). However, even if the implementation is efficient, always-on detection with ultra-low power consumption (< 200 µW) is not possible.

Solid state integrated circuits also implement fully digital processing [6], where the first detection stage consists of an always-on Voice-Activity-Detection (VAD) with a sampling rate of 16 kHz, as well as [7] where voice is sampled at 8 kHz. Other works implement mixed signal processing [8] with an analog features extractor based in a time domain convolutional neural network. [9] demonstrates the relevancy of this choice: analog is more energy efficient than digital processing for frequencies higher than 100 Hz. [10] proposes also an implementation based on an analog front-end using a filter bank for reducing power consumption. The system [10] has been fabricated in a 65 nm CMOS process and achieve > 86% accuracy for 12 class on the Google Speech Command Dataset [11] (GSCD) with a power consumption of 23 µW. Another one [8] in a 28 nm CMOS process achieves > 93.5% accuracy for 2 classes only on the GSCD with a power consumption of 108 nW. Finally, [12] proposes a voltage-domain based analog multiplication architecture for reducing power consumption on Deep Neural Network (DNN) processing. Several works [13]–[15] implement mixed signal processing for features extraction in voice activity recognition applications.

The main objective here is trying to reduce as much as possible power consumption while maintaining good audio classification and using standard discrete electronic parts without building a new integrated circuit. This constraint implies to focus on a good matching of embedded architecture and algorithms without having the possibility to improve power consumption by changing silicon technology. More specifically, separation of the analog and digital parts is defined according to [9] whereas algorithms can be seen as a mix of other presented solutions.

Proposed work implements an always-on analog front-end (AFE) for features extraction and microcontroller wake-up in case of a signal presence. The microcontroller processes a CNN on the information acquired from the AFE and classifies
the inputs into the classes learned from the dataset. This is presented in sections: II: system architecture and its components; III: comparison between input signal spectrum and the output of the AFE; IV: CNN implemented on the microcontroller; V and VI: results and conclusion.

## II. SYSTEM ARCHITECTURE

The blocks diagram of the proposed system (Fig.1) shows its amplified input signal entering to an AFE in charge of a continuous analysis and a microcontroller wake-up. The AFE is composed by a filter bank, each one of them followed by a peak detector, to extract the signal energy of each band. Finally, an ADC converts the outputs of each peak detector. This is done in always-on mode while the CNN is processed only when necessary, by a comparator wake-up interrupt. All components are described below.

<figure><img src="image_2.png" /><figcaption>Fig. 1: System blocks diagram.</figcaption></figure>

1) Analog Features Extractor (AFE): it is composed by a filters bank of 6 filters, each one followed by a peak detector and an extra peak detector for the input signal amplified which is compared with a reference level to wake-up the microcontroller. These blocks are described as follows:

a) Filters bank: it is composed by 6, second order, Band Pass Filters (BPF). The circuit implemented for each one of them is shown Fig.2.

<figure><img src="image_3.png" /><figcaption>Fig. 2: Circuit band pass filter.</figcaption></figure>

The filters are composed by two RC filters (high-pass and low-pass) coupled by two operational amplifiers MAX40007 [16] as voltage followers. This simple schema allows an easy cut-off frequency adjustment. The filters are logarithmically spaced from 220 to 4878 Hz (Fig.3).

<figure><img src="image_4.png" /><figcaption>Fig. 3: Filters bank frequency response.</figcaption></figure>

b) Peak detector: each BPF output is followed by a peak detector to hold and detect the energy on each one of the bands. Then, the input amplified is also followed by a peak detector to hold the signal level. The circuit implemented for the peak detector is in Fig.4.

<figure><img src="image_5.png" /><figcaption>Fig. 4: Circuit Peak Detector.</figcaption></figure>

Peak detectors are passive and are composed by a diode and a RC filter, the ones processing the output of the filters have a cut-off frequency of 5 Hz and the one processing the input is 1 Hz.

c) Comparator: This one is an ultra-low power comparator from Texas Instruments (TLV3691 [17]), with a configurable reference value fixed by the microcontroller. It is in charge of microcontroller wake-up for CNN processing. Reference value is:

$$Reference = \frac{V_{CC}}{2} + \frac{V_{CC}}{18}. \quad (1)$$

2) ADC: the Analog to Digital Converter samples the outputs of each peak detector having as inputs the band pass filters. Therefore, it samples 6 channels with a sampling rate of 88 samples per second.

3) Microcontroller: a Texas Instruments CC2652R1 has been used. When it wakes-up from a comparator interrupt, it forms a tensor with the 32 samples of each ADC channel, resulting into an array of 192x1, which is processed by the CNN and then feeds a serial port interface.

## III. KEYWORD SIGNALS SPECTRUM COMPARISON

In this section, the short term Fourier transform (STFT) and the AFE of a keyword "yes" of the Google Speech Command Dataset (GSCD) [11] are compared. Its temporal signal Fig.5a, shows in the first 20 ms the low frequencies of starting vowel and then the high frequencies of the fricative consonant S.

The frequencies predominance observed in Fig.5a are also the ones into the STFT and the proposed AFE (Fig.5b).
<figure><img src="image_6.png" /><figcaption>Fig. 5: Temporal signal, spectrograms and AFE comparison of the word *Yes*.</figcaption></figure>

The usual STFT spectrogram can be compared with the output of the AFE in Fig.6. It shows the 32 samples acquired for each one of BPF where filter frequency band increases with the number of the filter. In this way, for BPF1, the amplitude for the first 16 samples is important and represents the vowels *Y* and *E*. Then, the BPF6 shows an increasing energy value in respect to the others BPF in the last 16 samples, giving in this way a similar spectrogram (Fig. 5c).

<figure><img src="image_7.png" /><figcaption>Fig. 6: The AFE outputs for word *Yes* (in one dimension).</figcaption></figure>

## IV. CNN IMPLEMENTATION

For processing the output of the AFE, the proposed CNN is:

1. Input layer: Composed by the 32 samples acquired from each channels, which results in a tensor of 192x1 elements.

2. First layer: 1D Convolution of 2 filters with a kernel size of 33 with an activation layer ReLU.

3. Second layer: Max-Pooling 1D with a pool size of 2.

4. Third layer: Flatten layer grouping the tensors in a one dimensional tensor.

5. Layers 4th, 5th and 6th: Fully connected dense layers with 20 neurons each one and an activation function ReLU.

6. Output layer: 8 neurons with a softmax activation function representing the 8 classes.

This CNN has been designed to fit in the microcontroller memory and to maximize classification accuracy. Several tests have been done by changing the number of internal layers and the number of neurons. Finally the CNN presented in Fig. 7 has the best performances because although it has few neurons in each layer, the non-linearity increased. It has been trained using TensorFlow and GSCD using 500 samples of each class.

<figure><img src="image_8.png" /><figcaption>Fig. 7: The diagram of the used Convolutional Neural Network.</figcaption></figure>

## V. RESULTS AND DISCUSSION

In this section, the system processing time and classification accuracy in the GSCD are presented.

### A. CNN processing time

The CNN presented in section IV is processed in an Arm M4F turning with a clock frequency of 48 [MHz]. Thus, for one classification the processing time required is 750 [$\mu$S].

### B. Power consumption

System power consumption depends on the two detection states. First one is the always-on mode where the AFE is continuously processing the input signal. Then, when signal level reaches the threshold fixed, one-shot classification mode is triggered. Power consumption for both modes is:

* Always-on mode: Average power consumption in this mode is 32.4 $\mu$W.

* Classification mode: In this mode, average power consumption is 10.4 mW for 121 mS. This includes the ADC measure reads and the CNN processing.

In this way considering one classification every different period of time, average power consumption is the following:

* One detection every 2 s: 661 $\mu$W,

* One detection every 10 s: 160 $\mu$W,

* One detection every 30 s: 76 $\mu$W,

* One detection every minute: 55 $\mu$W.
### C. Classification accuracy

Three sub-dataset of the GSCD [11] have been used as training and evaluating database to validate the proposed system performances. These ones are described with their classification accuracy:

1) 2-class sub-dataset, where target words are "yes" and "no" → classification accuracy 97%.

2) 4-class sub-dataset, where the target words are "yes", "no", "backward" and "forward" → classification accuracy 94%.

3) 8-class sub-dataset, where target words are "yes", "no", "backward", "forward", "up", "down", "right" and "left" → classification accuracy 88%.

The 8-class confusion matrix sub-dataset is in Fig.8.

<table><caption>Fig. 8: Confusion matrix of the 8 target words.</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="8">PREDICTIONS</th></tr><tr><th>Yes</th><th>No</th><th>Backward</th><th>Forward</th><th>Up</th><th>Down</th><th>Right</th><th>Left</th></tr></thead><tbody><tr><th rowspan="8">TRUE</th><th>Yes</th><td>88</td><td>0</td><td>0</td><td>0</td><td>3</td><td>0</td><td>0</td><td>9</td></tr><tr><th>No</th><td>0</td><td>89</td><td>0</td><td>2</td><td>0</td><td>8</td><td>1</td><td>0</td></tr><tr><th>Backward</th><td>0</td><td>2</td><td>92</td><td>5</td><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><th>Forward</th><td>0</td><td>0</td><td>2</td><td>90</td><td>0</td><td>4</td><td>4</td><td>0</td></tr><tr><th>Up</th><td>3</td><td>2</td><td>0</td><td>0</td><td>89</td><td>0</td><td>4</td><td>2</td></tr><tr><th>Down</th><td>0</td><td>8</td><td>0</td><td>0</td><td>1</td><td>85</td><td>5</td><td>1</td></tr><tr><th>Right</th><td>0</td><td>3</td><td>1</td><td>4</td><td>0</td><td>0</td><td>84</td><td>8</td></tr><tr><th>Left</th><td>10</td><td>1</td><td>0</td><td>0</td><td>1</td><td>0</td><td>5</td><td>83</td></tr></tbody></table>

### D. Discussion

An ultra-low power system for keyword spotting detection using commercially available components has been successfully developed. The key for reducing its power consumption is mixed (analog-digital) signal processing, as long as, the gain-bandwidth product of the components that are adapted to the target frequencies (< 8 kHz in voice detection applications). Authors proposal, following their previous claim [9], is to run an analog processing when high operating frequencies are required, as it is relevant for features extraction, and then these features are digitally processed at lower frequencies.

Due to constrained resources in the used microcontroller, the CNN model size allows an important accuracy classification for two words "yes" and "no", but decreases when adding more words.

In the literature several interesting KWS systems have been developed achieving up to under µW power consumption. In most of these works, mixed signal processing has been implemented. The Tab. I presents a comparison of these works. Tab. I shows that the higher the number of keywords to detect, the lower the classification accuracy and the higher the power consumption. [7] and [8] have really low power consumption, but are focused on only a very few number of keywords: this leads to reduce the need for a complex CNN, thus reducing drastically power consumption. Relevant existing integrated circuits solutions to be compared with the proposed one using discrete components are [12] and [10]. These solutions have a comparable number of spotted keywords (around 10), and achieve comparable classification results (around 90%). Power consumption of existing solutions is lower (23 µW and 52 µW

<table><caption>TABLE I: System measurements compared to state-of-the-art</caption><thead><tr><th></th><th>[7]</th><th>[8]</th><th>[10]</th><th>[12]</th><th>This KWS</th></tr></thead><tbody><tr><td>Year</td><td>2021</td><td>2022</td><td>2022</td><td>2019</td><td>2023</td></tr><tr><td>Techn. [nm]</td><td>28</td><td>28</td><td>65</td><td>22</td><td>Discrete</td></tr><tr><td>ML Algo.</td><td>CNN</td><td>TD-CNN</td><td>RNN</td><td>CNN</td><td>CNN</td></tr><tr><td>Voltage [V]</td><td>0.41</td><td>0.9-0.6</td><td>0.5</td><td>0.55</td><td>1.8</td></tr><tr><td>Keywords</td><td>2-5</td><td>2</td><td>12</td><td>10</td><td>8</td></tr><tr><td>Power [µW]</td><td>0.510</td><td>0.108</td><td>23</td><td>52</td><td>77</td></tr><tr><td>Acc. [%]</td><td>97-91.7</td><td>93.5</td><td>86</td><td>91</td><td>88</td></tr><tr><td>Dataset</td><td>GSCD</td><td>GSCD</td><td>GSCD</td><td>GSCD</td><td>GSCD</td></tr><tr><td>Pow./V. [µA]</td><td>1.24</td><td>0.12-0.18</td><td>46</td><td>94.5</td><td>43</td></tr></tbody></table>

against 77 µW), but this is mainly due to supply voltage of the chip that cannot be reduced when using commercially available chips. Considering current instead of power consumption leads to a different result: proposed solution is equivalent to the best solid-state one with only 43 µA.

These results are showing that it is possible to be very close to the best state-of-the-art results in KWS in terms of classification and energy efficiency using only discrete commercially available circuits. This relies on optimising the match of embedded architecture and algorithms, without implementing a new silicon device.

## VI. CONCLUSION

In this paper, an ultra-low power mixed signal keyword spotting system has been proposed. Even if power consumption level is minimal, an embedded AI has been implemented and processed at-the-edge, this is possible when using mixed signal processing as presented in [9], where it shows that *analog processing can be more than 30 times more efficient in terms of power consumption that digital one*. The system proposed is composed by an Analog Features Extractor (AFE), which processes high-frequency signals and extracts interesting features to be acquired and processed with low sampling rates. Then these features are finally the input of a CNN that achieves in the best case an accuracy of 98%.

Finally, the described average power consumption of the system depends of the number of detection executed. For example, in the case of one detection per minute, average power consumption is 55 µW. Therefore, this system can be embedded at-the-edge for Intelligent Internet of Things (IIoT) autonomous systems working on a small battery. For the case of a single CR2032 coin cell an autonomy of 1 year can be expected, therefore, proposed model opens new perspectives for environmental long-term machine listening.

## VII. ACKNOWLEDGMENT

The authors thank DGA Innovation, AID - RAPID and ANR which provided the grants ANR-20-CHIA-0014 and ANR-21-CE04-0019 received by H.G. and the grants DEMAIN received by V.G..
REFERENCES

[1] M. N. Miah and G. Wang, “Keyword spotting with deep neural network on edge devices,” in 2022 IEEE 12th International Conference on Electronics Information and Emergency Communication (ICEIEC), 2022, pp. 98–102.

[2] Y. Wei, Z. Gong, S. Yang, K. Ye, and Y. Wen, “Edgecrnn: an edge-computing oriented model of acoustic feature enhancement for keyword spotting,” *Journal of Ambient Intelligence and Humanized Computing*, vol. 13, pp. 1–11, 03 2022.

[3] Y. Zhang, N. Suda, L. Lai, and V. Chandra, “Hello edge: Keyword spotting on microcontrollers,” CoRR, vol. abs/1711.07128, 2017. [Online]. Available: http://arxiv.org/abs/1711.07128

[4] J. Wang and S. Li, “Keyword spotting system and evaluation of pruning and quantization methods on low-power edge microcontrollers,” 2022.

[5] M. Shah, S. Arunachalam, J. Wang, D. Blaauw, D. Sylvester, H.-S. Kim, J.-S. Seo, and C. Chakrabarti, “A fixed-point neural network architecture for speech applications on resource constrained hardware,” *Journal of Signal Processing Systems*, vol. 90, 05 2018.

[6] S. Zheng, P. Ouyang, D. Song, X. Li, L. Liu, S. Wei, and S. Yin, “An ultra-low power binarized convolutional neural network-based speech recognition processor with on-chip self-learning,” *IEEE Transactions on Circuits and Systems I: Regular Papers*, vol. 66, no. 12, pp. 4648–4661, 2019.

[7] W. Shan, M. Yang, T. Wang, Y. Lu, H. Cai, L. Zhu, J. Xu, C. Wu, L. Shi, and J. Yang, “A 510-nw wake-up keyword-spotting chip using serial-fft-based mfcc and binarized depthwise separable cnn in 28-nm cmos,” *IEEE Journal of Solid-State Circuits*, vol. 56, no. 1, pp. 151–164, 2021.

[8] F. Chen, K.-F. Un, W.-H. Yu, P.-I. Mak, and R. P. Martins, “A 108-nw 0.8-mm2 analog voice activity detector featuring a time-domain cnn with sparsity-aware computation and sparsified quantization in 28-nm cmos,” *IEEE Journal of Solid-State Circuits*, vol. 57, no. 11, pp. 3288–3297, 2022.

[9] S. Marzetti, V. Gies, V. Barchasz, H. Barthélemy, and H. Glotin, “Comparing analog and digital processing for ultra low-power embedded artificial intelligence,” in 2022 IEEE International Conference on Internet of Things and Intelligence Systems (IoTIS), 2022, pp. 112–116.

[10] K. Kim, C. Gao, R. Graça, I. Kiselev, H.-J. Yoo, T. Delbruck, and S.-C. Liu, “A 23-w keyword spotting ic with ring-oscillator-based time-domain feature extraction,” *IEEE Journal of Solid-State Circuits*, vol. 57, no. 11, pp. 3298–3311, 2022.

[11] P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition,” *arXiv* 1804.03209, 2018.

[12] B. Liu, Z. Wang, W. Zhu, Y. Sun, Z. Shen, L. Huang, Y. Li, Y. Gong, and W. Ge, “An ultra-low power always-on keyword spotting accelerator using quantized convolutional neural network and voltage-domain analog switching network-based approximate computing,” *IEEE Access*, vol. 7, pp. 186 456–186 469, 2019.

[13] M. Yang, C.-H. Chien, T. Delbruck, and S.-C. Liu, “A 0.5 v 55 w 64 x 2 channel binaural silicon cochlea for event-driven stereo-audio sensing,” *IEEE Journal of Solid-State Circuits*, vol. PP, pp. 1–16, 09 2016.

[14] M. Yang, C.-H. Yeh, Y. Zhou, J. P. Cerqueira, A. A. Lazar, and M. Seok, “Design of an always-on deep neural network-based 1- μ w voice activity detector aided with a customized software model for analog feature extraction,” *IEEE Journal of Solid-State Circuits*, vol. 54, no. 6, pp. 1764–1777, 2019.

[15] S. Oh, M. Cho, Z. Shi, J. Lim, Y. Kim, S. Jeong, Y. Chen, R. Rothe, D. Blaauw, H.-S. Kim, and D. Sylvester, “An acoustic signal processing chip with 142-nw voice activity detection using mixer-based sequential frequency scanning and neural network classification,” *IEEE Journal of Solid-State Circuits*, vol. 54, no. 11, pp. 3005–3016, 2019.

[16] NanoPower Op Amp in Ultra-TinyWLP and SOT23 Packages, Maxim Integrated, 2018. [Online]. Available: https://datasheets.maximintegrated.com/en/ds/MAX40007.pdf

[17] TLV3691 0.9-V to 6.5-V, Nanopower Comparator, Texas Instruments, 2013, revised 2015.