# Manipulation of acoustic focusing for multi-target sensing with distributed microphones in smart car cabin

Yuqi Su, Fusang Zhang, Beihong Jin, Daqing Zhang

► To cite this version:

Yuqi Su, Fusang Zhang, Beihong Jin, Daqing Zhang. Manipulation of acoustic focusing for multi-target sensing with distributed microphones in smart car cabin. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2025, 9 (2), pp.1 - 28. 10.1145/3729470. hal-05477383

HAL Id: hal-05477383

https://hal.science/hal-05477383v1

Submitted on 26 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

Latest updates: https://dl.acm.org/doi/10.1145/3729470

RESEARCH-ARTICLE

# Manipulation of Acoustic Focusing for Multi-target Sensing with Distributed Microphones in Smart Car Cabin

YUQI SU, Institute of Software Chinese Academy of Sciences, Beijing, Beijing, China

FUSANG ZHANG, Institute of Software Chinese Academy of Sciences, Beijing, Beijing, China

BEIHONG JIN, Institute of Software Chinese Academy of Sciences, Beijing, Beijing, China

DAQING ZHANG, Polytechnic Institute of Paris, Palaiseau, Ile-de-France, France

Open Access Support provided by:

Polytechnic Institute of Paris

Institute of Software Chinese Academy of Sciences

<figure><img src="image_7.png" /></figure>

PDF Download

3729470.pdf

12 February 2026

Total Citations: 1

Total Downloads: 766

Published: 18 June 2025

Citation in BibTeX format
# Manipulation of Acoustic Focusing for Multi-target Sensing with Distributed Microphones in Smart Car Cabin

YUQI SU, Institute of Software, Chinese Academy of Sciences & University of Chinese Academy of Sciences, China

FUSANG ZHANG, Institute of Software, Chinese Academy of Sciences & University of Chinese Academy of Sciences, China

BEIHONG JIN, Institute of Software, Chinese Academy of Sciences & University of Chinese Academy of Sciences, China

DAQING ZHANG, Peking University, China, & Institut Polytechnique de Paris, France

Smart car cabins have drawn great attention in the past few years to enable a wide range of applications, including in-car entertainment, road noise cancellation, and voice control. These applications have been underpinned by the addition of a very large number of acoustic modules in the cabin, which has further opened up new opportunities for acoustic contactless sensing in smart cabins. Different from the traditional smartphone or smart speaker for acoustic sensing, in this paper, we envision to manipulate the inherently embedded speakers and microphones in car cabin to produce 3D spatial signal focusing for sensing. We propose the new concept of in-car occupancy grids, which enables target identification in 3D cabin space. In order to accomplish signal focusing in a small spot region, we establish the distributed acoustic 3D spatial beamforming model. To further address the phase ambiguity induced by large microphone spacing, we design dual frequency transmitted signals and "virtually" create a low-frequency signal to sense targets, which characterizes the highly sensitive and noise-free properties. We build a distributed acoustic sensing system in car and conduct both benchmark experiments and real-world application studies. The results demonstrate that the proposed system can be applied to various sensing tasks, which has superior performance in terms of accurate occupancy detection, multi-target sensing, very subtle motion detection, and interference resistance.

CCS Concepts: • **Human-centered computing** → **Ubiquitous and mobile computing systems and tools**.

Additional Key Words and Phrases: Acoustic focusing sensing, Smart cabin, Multi-target sensing, Occupancy grids, Multi-path combination

**ACM Reference Format:**

Yuqi Su, Fusang Zhang, Beihong Jin, and Daqing Zhang. 2025. Manipulation of Acoustic Focusing for Multi-target Sensing with Distributed Microphones in Smart Car Cabin. *Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.* 9, 2, Article 49 (June 2025), 28 pages. https://doi.org/10.1145/3729470

## 1 Introduction

The smart car cabin market is witnessing significant growth and innovation in recent years. The market size is valued at USD 100 billion in 2023 and is expected to reach USD 214 billion by 2031, growing at a 9.5% CAGR from 2024 to 2031 [22]. In terms of comfort and enjoyment, the car cabin now provides a superior experience. Over the past few years, there has been a considerable change in the interiors of automobiles. The most significant

Authors' addresses: Yuqi Su, suyuqi23@otcaix.iscas.ac.cn, Institute of Software, Chinese Academy of Sciences; University of Chinese Academy of Sciences, Beijing, China. Fusang Zhang, fusang@iscas.ac.cn, State Key Laboratory of Computer Sciences, Institute of Software, Chinese Academy of Sciences; University of Chinese Academy of Sciences, Beijing, China. Fusang Zhang is also with the Inspur Computing Technology Pty Ltd., P.R China. Beihong Jin, Beihong@iscas.ac.cn, Institute of Software, Chinese Academy of Sciences; University of Chinese Academy of Sciences, Beijing, China. Daqing Zhang, dqzhang@sei.pku.edu.cn, Telecom SudParis, Institut Polytechnique de Paris, Evry, France. Fusang Zhang and Beihong Jin are corresponding authors.

<figure><img src="image_8.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License.
© 2025 Copyright held by the owner/author(s).
ACM 2474-9567/2025/6-ART49
https://doi.org/10.1145/3729470

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 9, No. 2, Article 49. Publication date: June 2025.
<figure><img src="image_9.png" /><figcaption>Fig. 1. An illustration of traditional acoustic signal beam and distributed acoustic signal focusing. The co-located circular array on smart speaker produces an energy beam spreading in the direction of the target, while the distributed in-car microphone array produces an energy spot surrounding the target.</figcaption></figure>

change is the abundance of acoustic modules (i.e., speakers and microphones) found within modern automobiles. It is noted that the Tesla Model S has a new car sound system with over 22 acoustic modules [9]. Mercedes-Benz develops the Burmester audio system [6] with 31 acoustic modules, while BMW creates the Bowers & Wilkins diamond surround sound system [20] with more than 30 acoustic modules. All these audio systems support the rich functions in the smart car cabin: voice control [3], road noise cancellation (RNC) [7], hands-free voice calls [4], and in-car entertainment [11].

We notice that in addition to voice-based smart applications, these acoustic modules in automobile interiors also provide great opportunities to achieve smart sensing capability in cabins. Existing smart sensing solutions in cabin adopt cameras, pressure sensors, RF radars and smartphones. However, camera-based solutions [16][42] are sensitive to lighting conditions and raise privacy issues. The pressure sensors can identify occupation only when the target is on the seat. Recent RF based solution (such as UWB and millimeter wave radar) [60][28] places a radar device on the car ceiling. However, the entire car cabin cannot be covered by a single device, and using several radar systems will be expensive. For the acoustic based sensing solution, existing work [34] utilizes the speaker and microphone in a smartphone mounted in front of the driver to detect dangerous driving. However, device placement and coverage have many constraints, hindering real adoption. Recent work proposes utilizing acoustic modules in the car cabin to achieve child presence detection [46]. This work only focuses on breath detection for a single target and is unable to identify the target location and support multiple target sensing.

In this paper, we envision a novel acoustic focusing based sensing paradigm by utilizing the distributed microphones in car cabin that can provide the two desirable advantages:

* **System function aspect: signal focusing in a small spot region in the 3D space.** As shown in Fig. 1, traditional acoustic beamforming is based on linear or circular arrays, where the energy beam can only achieve a line shape in the direction of the target. Our in-car distributed acoustic system is able to concentrate the signal in a small spot due to the fact that the acoustic modules are placed surrounding the target. So, the granularity for gridded space occupancy is more finely established. As a result, the target is better isolated, providing the possibility of multi-target sensing in the car. Additionally, signal focusing could provide the ability to resist interference, indicating that the system is robust to interference from the environment.

* **System performance aspect: signal focusing significantly enhances the sensing quality.** The distributed acoustic modules can effectively overcome the shortcomings of short acoustic sensing range
and significantly enlarge the sensing coverage. Meanwhile, the superposition of signals from different microphones can significantly increase the signal strength, and the phase alignment can further enhance the capability to detect subtle motion of the target. Thus, both the coverage and the sensing sensitivity can be greatly improved.

Based on the two advantages, we achieve acoustic focusing by leveraging the 3D beamforming technique to combine signals from the distributed multiple microphones. Furthermore, compared to traditional beamforming, distributed acoustic beamforming offers an increased number of microphones and larger apertures. This gives us the opportunity to obtain high resolution, but it also presents a number of practical challenges.

The first challenge is modeling the relationship between distributed microphones to create a signal focusing on a small spot region in 3D car space. Traditional microphone linear and circular arrays are all co-located, and the distance from the target to the microphone array can easily satisfy the far-field conditions [2]. That is, the reflected signals arrive at the same direction to each microphone, and the signals can all be considered as parallel incident. Whereas for the in-car distributed microphone arrays, the directions of reflected signals vary greatly from the target to different positions of microphones. This breaks the requirement for parallel incidence of signals in traditional beamforming models. Thus, it is challenging to adapt the beamforming model for distributed microphone arrays under near-field conditions.

The second challenge is that since the spacing between distributed microphones is much larger than the wavelength of the acoustic signal (e.g., the front and back microphones are typically separated by the distance *D* of 150 cm [17], whereas for 19 kHz ultrasound signals commonly used for sensing, the wavelength $λ = 1.8$ cm, which is much smaller than *D*). The naturally large aperture of distributed microphones enables better angular resolution. At the same time, serious phase ambiguity is also introduced by the large aperture, making multi-target separation challenging. Therefore, the second challenge is how can we achieve fine-grained acoustic focusing without phase ambiguity, and accomplish multi-target sensing.

To address the first challenge, we propose a new concept of occupancy grids in car cabins. We divide 3D cabin space into small grids and identify the various reflection paths in each microphone received signal. Then, we combine the signals from different microphones in terms of signal strength, which helps to achieve target occupancy detection. Next, we propose a distributed acoustic 3D beamforming model to create a signal focusing on a small spot region. To overcome the constraints of traditional beamforming under far-filed conditions, we characterize the signal transformation of distributed microphones in the polar coordinate by quantifying the relations of azimuth and elevation angle from different microphones. This model effectively establishes the acoustic focusing of an in-car distributed microphone array on a target under near-field conditions.

To address the second challenge, we propose a novel dual-frequency signal design and differential frequency based signal processing technique. Specifically, we concurrently send two high-frequency signals with distinct center frequencies as the transmitted signals. After receiving these two high-frequency signals, we employ a differential frequency signal processing method to "virtually" create a low-frequency signal for sensing. The phase ambiguity issue is resolved by this low-frequency signal with a long wavelength. As this is a virtual low-frequency signal, the signal still has the characteristics of being highly sensitive and noise-free. Thus, this technique can successfully realize multi-target sensing without ambiguity issues.

Building upon the model and technique, we establish a distributed acoustic focusing sensing system in the car cabin, which consists of the car's built-in speakers and microphones. We conduct benchmark experiments to verify the capability of 3D-spatial occupancy detection and multiple target sensing. For the evaluation of spatial occupancy detection, the grid division granularity is 2 × 2 × 2 $cm^3$. With such a fine granularity, the detection accuracy between our occupancy grids and the ground-truth region can reach up to 92%. Through real-world case studies in car cabins, we demonstrate the performance of our system in practical applications, including: 1) multi-target breath detection. In the case of four targets in the car (all seats are full), our system is able to
<figure><img src="image_10.png" /><figcaption>Fig. 2. Distributed acoustic focusing sensing system</figcaption></figure>

focus on different locations and detect multiple targets in the car. The average breathing rate estimation error is only 0.85 bpm. 2) driver dangerous driving activity detection (e.g., in-car eye blink, yawn, and nod detection). Because of the enhanced signal quality due to signal focusing, very subtle target movements (e.g., blinking) can be detected. The detection rate of blinking activity is as high as 96.4%. 3) multi-target gesture recognition. The system is capable of recognizing multiple targets while making different hand gestures. The system focuses the signal on the hand and accurately tracks the signal changes caused by different gestures, significantly reducing the interference between the targets. The average recognition accuracy reaches 94.5% on the five gestures.

The main contributions of our work are summarized as follows.

* We are the first to propose the concept of acoustic focusing based sensing paradigm, achieving the 3D-spatial occupancy detection and multiple target sensing in car cabin platform.

* We establish the distributed acoustic 3D beamforming model, which can focus the signal on the target under near-field conditions, and overcome the constraints of the traditional model on the target's far field.

* We propose the dual frequency signal design and differential frequency based beamforming technique, which effectively address the phase ambiguity issue for the acoustic focusing.

* We build a distributed acoustic focusing based sensing system with in-car acoustic modules. We conduct benchmark experiments and implement three practical sensing applications in real cars. Compared to the existing work, we demonstrate the superior performance of the proposed system in terms of spatial occupancy detection, multi-target sensing, very subtle motion detection, and interference resistance.

## 2 Motivation and Problem Description

This work aims to develop a practical acoustic focusing based contactless sensing platform leveraging a large number of inherently embedded acoustic modules in the car cabin. This facilitates real adoption of contactless sensing capabilities, this also greatly enhances sensing performance for a wide range of smart cabin applications, such as driver activity detection, human-car interaction, child presence detection and etc.

We note that a large number of microphones deployed at different positions in the cabin have become essential components of in-car voice recognition and audio-visual entertainment. For example, Infineon's AEC-Q103-qualified MEMS system has 16 microphones [10]. The AAC Technologies company provides in-car microphone modules with 6 multi-position microphone arrays, and each position at least has two microphones [8]. According to studies on the quantity and placement of acoustic modules in different car models, we envision a possible acoustic focusing based sensing system will be deployed in the smart cabin. As shown in Fig. 2, given a typical mid-size car cabin's 3D-space (1×1.8 $m^2$ rectangular area with a 1.5 m height [17]), the speakers and microphones are deployed around the top of the ceiling. If we keep the number of speakers and microphones to a reasonable number (e.g., 4 speakers and 16 mics), the uniform distribution of these acoustic modules enhances sound clarity and quality for all occupants. For contactless sensing, we utilize speakers to transmit inaudible high-frequency
acoustic signals (e.g., ZC-sequence signal with a frequency exceeding 15 kHz).<fnref n="1" /> The transmitted signal arrives at the target, and the reflected signals are captured by microphones located at different positions, naturally forming a distributed microphone array throughout the cabin. Promisingly, this provides opportunities for us to manipulate the in-car distributed microphone array for acoustic sensing, achieving signal focusing in a specific region to address the contactless sensing challenges of multi-target and severe interference.

Note that traditional acoustic sensing systems, such as those in mobile phones and speakers, typically include a small number of microphones (commonly two to six constrained by hardware size). This results in a microphone array aperture of only a few centimeters, leading to wide beam and low directivity. In contrast, distributed microphone arrays naturally have a much larger aperture (up to the meter spacing), and more microphones (usually more than ten), which leads to a narrower signal beam and higher angular resolution. Based on this, we utilize the benefits of such large aperture arrays to provide a unique 3D spatial sensing system for a distributed microphone array in a cabin.

Specifically, we first construct the in-car occupancy grids to restore the energy distribution of sensing targets in the car, which provide an essential capability for target number identification and location estimation. Then, we introduce our proposed 3D beamforming model to focus the signals at each target sequentially, which creates a peak energy at each target by aligning the signals phase from different position microphones, thereby greatly enhancing the sensing signal quality.

The next sections describe in detail the new concept of spatial occupancy grids and the distributed acoustic 3D beamforming model.

# 3 Occupancy Grids Construction in Car Cabin

## 3.1 Concept of Occupancy Grids

Inspired by Occupancy Network [39] developed by Tesla in 2022, we present the concept of occupancy grids. This network is utilized in Autopilot to calculate the probability of detecting targets, drivable regions, and other entities within an outdoor environment. By evaluating the space occupied by different targets within the cabin, we can provide personalized sensing capabilities that enable services and interactions for both drivers and passengers. This method makes it possible to produce the targets occupancy, which includes information on the number and location of targets.

Human activities in car cabin lead to dynamic variations in the energy of acoustic signals. As illustrated in Fig. 3, we first divide the 3D space cabin into smaller grids in order to quantitatively measure the dynamic reflected signals from targets. The embedded speaker transmits continuous acoustic signals for spatial sensing. Reflection from multiple grids with sensing targets causes these signals to scatter in different directions. The multipath signals are received by microphones at different positions. The occupancy of moving targets in each grid is indicated by the strength of the reflected signals inside that grid. Therefore, we utilize the dynamic energy distribution across all grids in the entire 3D space to estimate occupants.

## 3.2 Construction of Occupancy Grids

The construction of our in-car occupancy grids involves two main steps: **Multipath Mapping from Single Microphone** and **Signal Strength Superposition from Multiple Microphones**. As shown in Fig. 3, for the first step, we map the signals from each microphone into 3D cabin grids and achieve multipath signal decomposition. For the second step, we combine multipath signals from all microphones across each grid and calculate the sum of these signals strength. Repeating this process, we derive an in-car spatial occupancy grids, represented as an $M \times N \times Q$ matrix, where each item indicates the energy strength within this grid. This occupancy grids can

<footnote n="1">¹This high-frequency signal is compatible with existing in-car music and voice functions, allowing for integrated communication and sensing.</footnote>
<figure><img src="image_11.png" /><figcaption>Fig. 3. Occupancy grids construction in car cabin</figcaption></figure>

capture fine-grained motion, enabling us to obtain different scales of sensing information generated by multiple targets.

### 3.2.1 Multipath mapping from single microphone.
Given the positions of microphones and speakers within the car, the speaker transmits continuous ZC-sequence signals to enable spatial sensing. These acoustic signals are reflected from different body areas and eventually superimposed on each microphone. A cross-correlation operation is then performed between the received and transmitted signals at each microphone to extract the channel impulse response (CIR) [46]. This CIR comprises both static and dynamic components as the following:

$$ cir(t) = \underbrace{\sum_{i=1}^{P_s} A_i e^{-j2\pi \frac{d_i}{\lambda}}}_{\text{static component}} + \underbrace{\sum_{i=1}^{P_d} A_i e^{-j2\pi \frac{d_i+\Delta d_i(t)}{\lambda}}}_{\text{dynamic component}}, \quad (1) $$

where $A_i$ represents the amplitude of the signal in i-th path, $d_i$ and $\Delta d_i(t)$ denote the reflection path length and the time-varying change in path length, respectively. $P_s$ and $P_d$ indicate the number of static and dynamic multipaths, respectively, and $\lambda$ denotes the wavelength of the acoustic signal.

Since we pay attention to the cabin occupancy detection, the irrelevant static multipath component is filtered out using a loopback filter [24], leaving the channel impulse response to contain only the dynamic multipath component.

### Mapping signal pulses to 3D occupancy grids.
We divide the 3D cabin space into small cubes with the size of $2 \times 2 \times 2 \text{ cm}^3$. Each speaker is sequentially activated to transmit acoustic signals, with each cycle transmitting a single frame of the signal. Given the positions of the speakers and microphones, we calculate the propagation distance of the signal from the speaker, through its reflection target within the j-th grid, and finally to the i-th microphone in the array. The position of each 3D grid is then mapped to a signal pulse of the dynamic multipath component in each microphone's normalized channel impulse response.

### 3.2.2 Signal strength superposition from multiple microphones.
By repeating the mapping process for N microphones, we obtain the reflected signal from j-th grid:

$$ S_{g_j} = [S_{g_j,M_1}, S_{g_j,M_2}, \dots, S_{g_j,M_N}], \quad (2) $$
<figure><img src="image_12.png" /><figcaption>Fig. 4. Occupancy grids of different number of targets</figcaption></figure>

where, $S_{g_j, M_i}$ denotes the signal energy strength of the j-th grid from i-th microphone. Thus, the total energy strength of the reflected signal within each j-th grid is expressed as:

$$E(g_j) = \sum_{i=1}^{N} |S_{g_j, M_i}|. \qquad (3)$$

The energy distribution of reflected signals reveals the occupancy of targets within the car space. By setting an energy strength threshold, the regions corresponding to moving targets can be efficiently identified. The occupancy grids for targets 1, 2, and 3 are illustrated in Fig. 4. Grids with energy strength over the threshold are grouped into clusters, allowing the approximate locations and numbers of the targets to be observed within the energy distribution occupancy grids.

**Selection of the target center.** We define a new metric function to select the center of each target:

$$F_{g_j} = \alpha \cdot \frac{1}{|g_j - c_G|} + \beta \cdot E(g_j), \qquad (4)$$

where, $c_G$ represents the geometric center of the area occupied by the target, $|g_j - c_G|$ denotes the distance between the j-th grid and the target's geometric center, and $E(g_j)$ denotes the dynamic energy strength of the j-th grid. The coefficients $\alpha$ and $\beta$ are weight factors. Therefore, $F_{g_j}$ represents the weight between the distance of the j-th grid to the geometric center and the dynamic signal energy. A higher value of $F_{g_j}$ suggests an higher probability of the grid being the true center of the target. Note that occupancy grids are a target-level identification method, we need to focus the signal of target based on the 3D beamforming, and then extract target motion information (e.g., breathing and gestures).

## 4 Distributed Acoustic 3D Spatial Beamforming Model

Traditional microphone array designs assume that the acoustic signals arrive at each microphone with parallel incidence, and the distance between microphones is typically restricted to no more than half the wavelength of the signals. However, in our car-mounted distributed microphone array, the distances from the target to the microphones are comparable to the microphone spacing, resulting in a near-field condition [2]. As a result, the traditional beamforming model, which operates under far-field conditions, is not appropriate for this near-field scenario. To address this limitation, we propose a 3D spatial beamforming model for car-mounted distributed microphone arrays, overcoming the constraints of traditional beamforming. Specifically, we first model the phase differences between the signals received by each microphone due to different propagation distances. In contrast to traditional methods, we incorporate the target's 3D spatial location to account for propagation distance differences, and introduce azimuth and elevation angle information to construct a phase compensation vector for
<figure><img src="image_13.png" /><figcaption>Fig. 5. Distributed acoustic 3D spatial beamforming model</figcaption></figure>

each microphone in a polar coordinate system. This 3D beamforming technique enables spot-focusing on the target region, facilitating spatially separated sensing of multiple targets.

The following section provides a detailed modeling process of our 3D beamforming for distributed microphone array in car cabin.

First, as illustrated in Fig. 5a, we position this distributed microphone array within a three-dimensional spatial coordinate system. To facilitate the modeling process, the distributed microphone array (e.g., 16 microphones) is placed in the YOZ plane with the same microphone spacing. The lower-left microphone of the array is located at the coordinate origin and serves as the reference microphone.

Specifically, we assume that the i-th microphone $M_i$ is represented as $(x_i, y_i, z_i)$ in cartesian coordinates. To facilitate the conversion of the angular relationship, we transform these coordinates into a polar coordinate system as $(d_i, \theta_i, \varphi_i)$. Since the array is in the YOZ plane, we obtain $x_i = 0, \theta_i = 90^\circ$. The target K is represented in both cartesian coordinates as $(x_K, y_K, z_K)$ and polar coordinates as $(d_K, \theta_K, \varphi_K)$. The transformation between the two coordinate systems can be expressed as follows:

$$ \begin{cases} x_K = d_K \cos \varphi_K \cos \theta_K \\ y_K = d_K \cos \varphi_K \sin \theta_K \\ z_K = d_K \sin \varphi_K \end{cases}. \qquad (5) $$
The distance between the target K and the microphone $M_i$ can be denoted as:

$$ 
\begin{aligned}
d_{KM_i} &= \sqrt{(x_K - x_i)^2 + (y_K - y_i)^2 + (z_K - z_i)^2} \\
&= \sqrt{(x_K^2 + y_K^2 + z_K^2) + (x_i^2 + y_i^2 + z_i^2) - (2x_K x_i + 2y_K y_i + 2z_K z_i)} \\
&= \sqrt{d_K^2 + y_i^2 + z_i^2 - 2d_K(y_i \cos \varphi_K \sin \theta_K + z_i \sin \varphi_K)}.
\end{aligned}
$$

Then, the phase difference of received signals between $M_i$ and the reference microphone is expressed as:

$$ \Delta\phi_i = \frac{2\pi}{\lambda} (d_K - d_{KM_i}) = \frac{2\pi}{\lambda} \left( d_K - \sqrt{d_K^2 + y_i^2 + z_i^2 - 2d_K(y_i \cos \varphi_K \sin \theta_K + z_i \sin \varphi_K)} \right). \quad (6) $$

Therefore, the direction vector of the i-th microphone is denoted as:

$$ a_i = e^{-j\Delta\phi_i} = e^{-j\frac{2\pi}{\lambda}(d_K - \sqrt{d_K^2 + y_i^2 + z_i^2 - 2d_K(y_i \cos \varphi_K \sin \theta_K + z_i \sin \varphi_K)})}. \quad (7) $$

In the far-field condition<fnref n="2" />, the distance $d_K$ from the reference microphone to the target is significantly larger than the spacing of the microphones $d_i$, where $d_i = \sqrt{y_i^2 + z_i^2}$. Then, we have the following approximation:

$$ \sqrt{d_K^2 + y_i^2 + z_i^2 - 2d_K(y_i \cos \varphi_K \sin \theta_K + z_i \sin \varphi_K)} \approx d_K - (y_i \cos \varphi_K \sin \theta_K + z_i \sin \varphi_K), \\ a_i \approx e^{-j\frac{2\pi}{\lambda}(y_i \cos \varphi_K \sin \theta_K + z_i \sin \varphi_K)}. $$

In three-dimensional space (as shown in Fig. 5d), the received signal from microphone has a direction vector along the y-axis as $\vec{v}_y = y_i \cos \varphi_K \sin \theta_K$, and another along the z-axis as $\vec{v}_z = z_i \sin \varphi_K$. Therefore, by rotating the direction vector, we can combine the microphone signals from different positions and cooperatively adjust them to precisely direct to the target.

Assuming that the transmitted signal is $s(t)$, and taking into account the phase delay resulting from the path propagation difference induced by the target's reflection, the reflection signal received by the distributed microphone array can be expressed as:

$$ X(t) = \begin{bmatrix} x_1(t) \\ x_2(t) \\ \vdots \\ x_N(t) \end{bmatrix} = s(t)e^{-j\phi(t)} \begin{bmatrix} a_1 \\ a_2 \\ \vdots \\ a_N \end{bmatrix} = s(t)e^{-j\phi(t)} a(\theta, \varphi), \quad (8) $$

where, $x_i(t)$ denotes the received signal of each microphone, $\phi(t)$ denotes the phase delay of reference microphone at the time $t$, and $a(\theta, \varphi)$ is the direction vector matrix of the microphone array.

Then, the superimposed focusing signal after beamforming is:

$$ y(t) = \sum_{i=1}^{N} w_i \cdot x_i(t) = W \cdot X(t) = s(t)e^{-j\phi(t)} W \cdot a(\theta, \varphi), \quad (9) $$

where $W = [w_1, w_2, \dots, w_N]$ denotes the weight vector of acoustic beamforming, among $w_i = \bar{a}_i = e^{j\Delta\phi_i} = e^{j\frac{2\pi}{\lambda}(d_K - \sqrt{d_K^2 + y_i^2 + z_i^2 - 2d_K(y_i \cos \varphi_i \sin \theta_i + z_i \sin \varphi_i)})}$. Here, $\theta$ and $\varphi$ are traversed within the angle range $[0, \pi]$, when the superimposed signal gain of $y(t)$ is maximized, indicating that the beam is directed toward the target. Note

<footnote n="2">²The target is considered to be in the far-field region when the distance $d$ between the target and the array satisfies the condition $d \ge \frac{2L^2}{\lambda}$, where $L$ is the size of the array and $\lambda$ is the wavelength of the acoustic signal [2]. Under these conditions, the acoustic signal operates as a plane wave with the same angle of incidence to the microphone array.</footnote>
<figure><img src="image_14.png" /><figcaption>Fig. 6. 3D beamforming result with a 16-element microphone array arranged in an area of the same size in an car cabin (1×1.8 m² rectangular area with a 1.5 m height) [13]</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Fig. 7. Impact of four typical layouts of microphone array placement on beamforming performance (with 16 microphones)</figcaption></figure>

that the modeling process of 3D beamforming in a distributed microphone array does not depend on far-field constraint, thus serving as a general model for the beamforming technique.

To demonstrate the modeling results, as shown in Fig. 6, we simulate the signal focusing through 3D beamforming with a 16-element microphone array arranged in an area of the same size in an car cabin. It is clear that the distance between the microphones is larger than half the signal's wavelength, which causes grating lobes to appear at particular angles. These lobes can be confused with the main lobe and interfere with target identification.

Here, we also investigate some impact factors (i.e., microphone layout and number) affecting the 3D beamforming performance. As shown in Fig. 7, we verify the performance with four typical layouts of microphone array in the cabin [8, 10]. The results show that grating lobes have less impact and the main lobe becomes more noticeable when microphone arrays are more evenly spaced apart. This effect occurs because a more uniform distribution of microphone array has no angle coupling between the azimuth and elevation, which leads to a larger effective receiving area and more diverse signals for beamforming, resulting in higher beamforming performance [32]. In addition, we verify the impact of different number of microphones (i.e., 8, 10, 16, and 18), as shown in Fig. 8. The results reveal that an increased number of microphones strengthens the main lobe and concentrates the energy of beam. However, increasing the number of microphones enhances the sharpness and prominence of the main lobe, but it fails to resolve the phase ambiguity issue. To address this limitation, we design dual-frequency signals in Section 5 to suppress grating lobes and mitigate interference issues in multi-target sensing.

In summary, our model for distributed microphone array 3D beamforming provides guidance on the optimal placement of microphones in future smart cabins. To achieve high beamforming performance, it is suggested that
<figure><img src="image_16.png" /><figcaption>Fig. 8. Impact of different number of microphones on beamforming performance (with 4-th layout)</figcaption></figure>

the typical medium-sized four-seater car can be equipped with a minimum of sixteen microphones and evenly spaced apart throughout the car’s ceiling.

## 5 Phase Ambiguity Removal Processing Based on Dual Frequency Signals

Note note that the spacing between microphones exceeds the signal wavelength (e.g., for 19 kHz acoustic signal, $\lambda = 1.8$ cm), resulting in significant phase ambiguity during beamforming. This phase ambiguity resulting in several grating lobes adjacent to the main lobe (as illustrated in Fig. 10a), which interfere the accurate identification of the target. An intuitive solution might be to increase the signal wavelength. However, this solution presents challenges. A longer signal wavelength corresponds to a lower signal frequency, which can lead to acoustic noise interference within the car. Furthermore, low-frequency signals undergo increased signal attenuation, which reduces the signal-to-noise ratio after reflection. To address these issues, we design a dual frequency transmitted signal and a differential frequency based signal processing, which retain the high-sensitive and noise-free characteristics of high-frequency and can be transformed to a “virtual” low-frequency acoustic signal to suppress the interference.

**Dual frequency signal design.** We design a unique transmitted signal by modulating the ZC-sequence signal to two high-frequency bands (i.e., 15–17 kHz and 18–20 kHz). For the received signal, these two high-frequency signals are processed by two corresponding bandpass filters independently, and then conduct cross-correlation operation to obtain CIR signals, respectively. Consequently, the dynamic CIR of the two high-frequency bands can be denoted as follows:

$$ cir_{f_1}(n_0, t) = A_1 e^{-j \frac{2\pi}{\lambda_1} (d_0 + \Delta d(t))} = A_1 e^{-j \frac{2\pi}{c} f_1 (d_0 + \Delta d(t))}, \quad (10) $$

$$ cir_{f_2}(n_0, t) = A_2 e^{-j \frac{2\pi}{\lambda_2} (d_0 + \Delta d(t))} = A_2 e^{-j \frac{2\pi}{c} f_2 (d_0 + \Delta d(t))}, \quad (11) $$

where $f_1$ and $f_2$ represent the center frequencies of the two high-frequency bands, respectively, $A_1$ and $A_2$ are the amplitude of the signals. $n_0$ denotes the range bin of target, and $d_0$ denotes the original distance of target. c denotes the speed of acoustic signal.

**Differential frequency based signal processing.** We then perform a division operation between the CIRs of the two frequency bands, and construct a differential frequency signal to virtually obtain a lower frequency signal with a longer wavelength. Then, the CIR of the differential frequency signal is given by:

$$ cir_{\Delta f}(n_0, t) = \frac{cir_{f_2}(n_0, t)}{cir_{f_1}(n_0, t)} = A' e^{-j \frac{2\pi}{c} \Delta f (d_0 + \Delta d(t))}, \quad (12) $$

where $\Delta f = f_2 - f_1$, $A' = A_2/A_1$. Specifically, for the two signals with frequency bands 15–17 kHz and 18–20 kHz, respectively, $f_1 = 16$ kHz and $f_2 = 19$ kHz, then the differential frequency $\Delta f = 3$ kHz. The wavelength of the
<figure><img src="image_17.png" /><figcaption>Fig. 9. 3D differential frequency based beamforming</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>Fig. 10. Performance of differential frequency based 3D beamforming</figcaption></figure>

differential frequency signal is $\lambda' = c/\Delta f \approx 11.4$ cm, which is nearly five times longer than $\lambda_1$ (2.1 cm) and $\lambda_2$ (1.8 cm).

**3D differential frequency based beamforming (3D-DFBF).** The long wavelength characteristic of the differential frequency signal is beneficial to remove the phase ambiguity. Therefore, we define a new weight vector for the distributed 3D beamforming:

$$w_i^{\Delta f} = e^{j \frac{2\pi}{c} \Delta f (d_K - \sqrt{d_K^2 + y_i^2 + z_i^2 - 2d_K (y_i \cos \varphi \sin \theta + z_i \sin \varphi)})} \quad (13)$$

As shown in Fig. 9a, the phase change of the differential frequency signal is less than a full cycle under the same target displacement. Consequently, there is no phase ambiguity issue, leading to no grating lobe for the differential frequency based beamforming (as shown in Fig. 9b).

According to the target center selected by the highest value of $F_{g_j}$ in occupancy grids, the reflected signal of the target is given by:

$$X_c(t) = [x_{c,M_1}(t), x_{c,M_2}(t), \dots, x_{c,M_N}(t)]^T, \quad (14)$$

where, $x_{c,M_i}(t)$ denotes the target signal from the $i$-th microphone, and $(\cdot)^T$ denotes the transpose operation of the matrix. By combining the azimuth angle $\theta$ and elevation angle $\varphi$, the 3D-DFBF weight vectors are applied to jointly align the microphone phase at different positions in 3D cabin space, achieving signal focusing. Then, the superimposed focusing signal after 3D-DFBF is:

$$y(t) = \sum_{i=1}^{N} w_i^{\Delta f} \cdot x_{c,M_i}(t) = W^{\Delta f} \cdot X_c(t), \quad (15)$$
where $W^{\Delta f} = [w_1^{\Delta f}, w_2^{\Delta f}, \dots, w_N^{\Delta f}]$ denotes the 3D-DFBF weight vector. Note that the 3D-DFBF weights only require traversing within the grids occupied by target for reducing computation. Then, we can obtain the real target center $(\theta_K, \varphi_K)$ by identifying the peak of the angular spectrum (as shown in Fig. 10b).

**Target signal enhancement for sensing.** Since the raw high-frequency signals have higher signal-to-noise ratio than the differential frequency signals, after resolving the ambiguity issue, we then reuse the high-frequency to further enhance the target sensing signal. Specifically, we compensate the phase of high-frequency signals (i.e., $f_2$) from different microphones based on the real target center $(\theta_K, \varphi_K)$ and then superpose them as follows:

$$y_{\text{target}}(t) = \sum_{i=1}^{N} w_{i}^{\Delta f}(\theta_{K}, \varphi_{K}) \cdot x_{f_{2},M_{i}}(t), \quad (16)$$

where, $y_{\text{target}}(t)$ is the sensing signal of target, $x_{f_2,M_i}(t)$ denotes the target sensing signal from the i-th microphone at the second high-frequency band, and $w_i^{\Delta f}(\theta_K, \varphi_K) = e^{j \frac{2\pi}{c} \Delta f (d_K - \sqrt{d_K^2 + y_i^2 + z_i^2 - 2d_K (y_i \cos \varphi_K \sin \theta_K + z_i \sin \varphi_K)})}$.

Fig. 10a illustrates the result of 3D beamforming based on the single high-frequency signal. Despite focusing on the target, several grating lobes appear around the main lobe and interfere with target identification. In contrast, Fig. 10b presents the result of 3D beamforming based on the differential frequency signal, effectively suppressing grating lobes and concentrating energy on the main lobe. Fig. 10c shows the original signal and the beamforming signal, respectively. The signal after beamforming shows the perfect periodic motion of the target (i.e., human breath).

**Difference between traditional beamforming and 3D-DFBF.** Traditional beamforming employs a linear microphone array placed in front of the target (as shown in Fig. 11), with the beam directed along the target toward the receiver. Any interference appearing along the beam path is captured by the receiver. In contrast, our proposed distributed 3D beamforming uses microphones uniformly surrounding the target (as shown in Fig. 12). By applying our novel phase difference compensation method for near-field conditions, we align the phases of all signals from the target. This results in higher energy levels at the target compared to other regions of the propagation space, effectively focusing the signals into a small point region at the target. In summary, our distributed beamforming model overcomes the far-field limitations of traditional beamforming and enables 3D signal focusing under near-field conditions.

<figure><img src="image_19.png" /><figcaption>(a) Layout</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>(b) Beam pattern</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>(a) Layout</figcaption></figure>

<figure><img src="image_22.png" /><figcaption>(b) Beam pattern</figcaption></figure>

<caption>Fig. 11. Acoustic beamforming effect of the traditional beamforming</caption>

<caption>Fig. 12. Acoustic focusing effect of the distributed 3D beamforming</caption>
<figure><img src="image_23.png" /><figcaption>Fig. 13. System overview</figcaption></figure>

## 6 System Implementation

In this section, we present the design of our in-car distributed microphone array sensing system, utilizing the proposed concept and technique (as shown in Fig. 13). The system consists of three main modules:

**Signal Preprocessing:** A continuous dual-frequency<fnref n="3" /> ZC-sequence (Zadoff-Chu) signal is transmitted by the in-car speaker, while the distributed microphone array receives the signals reflected from the target. First, the bandpass filter is used to extract these two high-frequency band signals. Then, these two signals perform a cross-correlation operation with the transmitted signals to obtain the channel impulse response (CIR) [46], respectively. Based on channel impulse response, we use multipath mapping to create the spatial occupancy grids, and we extract reflected signals from the target to enable acoustic signal focusing.

**Occupancy Grids Construction:** First, we perform a division operation between the CIRs of the two frequency bands. Then, the car cabin space is divided into smaller 3D grids to construct occupancy grids. Based on the known positions of speakers and microphones, we map the signal from each microphone into 3D spatial grids. By superimposing the reflected signals within each grid, we can derive the dynamic energy distribution across the car cabin. This occupancy grids facilitate the extraction of multi-target reflected signals, allowing for enhanced signal focusing and providing occupancy information on the number and location of targets in the cabin. Subsequently, we select the center of each target for focusing sensing.

**Spatial Acoustic Focusing for Sensing:** Due to the previous differential frequency signal processing, the target signal we obtained is a “virtual” low-frequency signal with a longer wavelength. The system applies distributed 3D differential frequency based beamforming to align the phase of each microphone with the azimuth and elevation angles. This phased adjustment sequentially directs the main lobe of the beam to each target, enabling energy focusing in the target area and obtaining the real position of the target. Then, the target position based beamforming weight vectors are used to compensate the phase of high-frequency CIR signals (i.e., $CIR_{f_2}$) from different microphones, aiming at enhancing the sensing signal. The software-based digital 3D beamforming technique thus allows for simultaneously sensing multiple targets.

<footnote n="3">³In the experimental setup of this work, we transmit acoustic signals containing the two frequency bands of 15 – 17 kHz and 18 – 20 kHz.</footnote>
<figure><img src="image_24.png" /><figcaption>Fig. 14. Experiment setup of benchmark</figcaption></figure>

# 7 Evaluation

In this section, we perform benchmark experiments to evaluate the effectiveness of our system. We investigate the impact factors on our system performance, such as the size of targets, thickness of targets, number of targets, number of microphones, and layouts of microphone array. Next, we present the performance of our system in practical applications, including 1) multi-target breath detection in car, 2) driver dangerous driving activity detection, and 3) multi-target gesture recognition in car. The proposed system introduces new features, i.e., in-car target location estimation, multi-target identification, and anti-interference capabilities. These advancements are designed to integrate with existing applications and provide technical support for more intelligent and customized interaction experiences.

## 7.1 Benchmark Experiments

**Experiment setup:** We implement our distributed acoustic focusing sensing system based on an audio system embedded in a real car<fnref n="4" />. As shown in Fig. 14a, our system includes sixteen microphone units, four BOSE speakers [19], a sound card (PXUA216MB-DL2-M/XMOS), and a car power amplifier. Fig. 14b shows the simulated car space, designed based on the size of the interior space of the typical mid-size cars according to these studies [13, 17, 18], with a size of 2.05 m × 1.2 m × 1.5 m. We place 16 microphone units with the same spacing (i.e., 0.37 m) on the ceiling, which is the placement of microphone array commonly used for in-car voice interaction [11]. Additionally, four speakers are located at the positions corresponding to car tweeters [12, 14], two at the A-pillar and two at the B-pillar, each aligned with a microphone unit. Our system controls the speakers to transmit dual-frequency ZC-sequence signals in turn by signal frame.<fnref n="5" /> The dual-frequency ZC-sequence signals are generated with parameters $f_1 = 16$ kHz, $f_2 = 19$ kHz, $B_1 = B_2 = 2$ kHz and $T = 0.0083$ s.<fnref n="6" /> The receiver employs at a 48 kHz sampling rate. The microphone arrays then receive the reflected signal and send it to a computer for further processing. We place the target on the sliding track, moving back and forth in

<footnote n="4">⁴In compliance with a non-disclosure agreement, we secure authorization from a prominent automotive manufacturer to utilize the embedded audio system of their car for experimental purposes. We take out the audio system and deploy it in different scenarios.</footnote>

<footnote n="5">⁵Transmitting signals in turn minimizes the energy consumption of the distributed acoustic sensing system while maintaining enough spatial coverage. By switching speakers at signal frame intervals (i.e., 8.3 ms), each speaker signal reaches the sensing target, which serves as a secondary reflective source, independent of the speaker's position.</footnote>

<footnote n="6">⁶Our design includes 120 frames signals per second, with each speaker transmitting 30 frames per second, satisfying the sampling requirements for subsequent experiments and applications.</footnote>
<figure><img src="image_25.png" /><figcaption>Fig. 15. Performance of occupancy grids with different impacts</figcaption></figure>

the experimental space, which allows precise control of motion displacement and round trips. Then, based on software control, the beam directs toward each sensing target individually, enabling multiple target motion tracking. We evaluate system performance with the metrics of occupancy overlap rate, motion displacement estimation accuracy and motion frequency estimation error.

### 7.1.1 Occupancy grids performance.
To comprehensively evaluate the performance of the occupancy grids, we carry out more studies to investigate the performance of different size, thickness and material of targets. The evaluation metric used is the occupancy overlap rate, which quantifies the overlap between the estimated target energy distribution grids and the physical occupancy of the target's actual space.

**Impact of target size:** We first examine the impact of target size on the performance of the occupancy grids, where the targets are five cubic with side lengths gradually increasing from 15 cm to 35 cm. These targets are placed one at a time on a sliding track to move, and corresponding occupancy grids are built for each target. Experimental results (Fig. 15a) demonstrate that as the target size increases, the occupancy overlap rate also increases. The target overlap rate is improved by larger surfaces because they increase the reflected signal energy distribution. However, the benefit in energy distribution is no longer increased when the target size reaches a specific size, and the occupancy overlap rate remains above 90%.

**Impact of target thickness:** To evaluate the impact of target thickness, we conduct experiments using a cardboard box with a front face measuring 40 cm × 30 cm and a thickness varying from 4 cm to 12 cm. Results (Fig. 15b) show that the occupancy overlap rate increases with thickness. This effect can be attributed to the larger reflective surface area resulting from increased thickness, which enhances the reflected signal and increases the signal energy.

**Impact of target material:** To evaluate the impact of target material, we employ targets of the same size made of different materials, including cardboard, plastic, wood, metal, and fabric. According to the results (Fig. 15c), targets made from metal, plastic, and cardboard have high occupancy overlap rates (over 90%) because of their flat surfaces, which allow for specular reflection. On the contrary, wood and fabric show lower occupancy overlap rates due to their rough surfaces, which result in weaker reflected signal energy.

These experimental results demonstrate that the performance of the occupancy grids is influenced by the size, thickness, and surface material of reflective targets. Overall, the performance of the occupancy grids is stable, and the average occupancy overlap rate is 88.3%.

### 7.1.2 Single-target motion estimation.
To evaluate the system motion estimation performance for a single target across different motion displacements (i.e., 5 cm, 10 cm, 12.5 cm, and 15 cm), we conduct an experiment using
<figure><img src="image_26.png" /><figcaption>Fig. 16. Performance of single target</figcaption></figure>

<figure><img src="image_27.png" /><figcaption>Fig. 17. Performance of multiple targets</figcaption></figure>

a slide-controlled target (29 cm × 24 cm × 3 cm) performing periodical movements over a 2-minute period. As shown in Fig. 16a, in terms of motion displacements, the average occupancy overlap rate of spatial estimation reaches 92%, with stable system performance. As shown in Fig. 16b and Fig. 16c, results indicate that the system achieves an average displacement estimation accuracy of 92.5% across four motion displacements, with the motion frequency estimation error not exceeding 0.67 cycles per minute (cpm). These results confirm the system's efficiency and accuracy in estimating the target with different motion displacements.

### 7.1.3 Multi-target motion estimation.
To evaluate the system's performance in tracking motion of multiple targets, we design experiments in which 1-5 targets are respectively controlled to perform movements with different displacements. For multi-target spatial estimation, when three targets are present, the overlap rate remains above 86% (as shown in Fig. 17a), indicating strong performance. However, as the number of targets increases, the overlap rate begins to decrease. To address this, we apply the 3D-DFBF technique to focus the signal on each target, enabling the accurate extraction of target motion information individually. As shown in Fig. 17b and Fig. 17c, results indicate that while an increasing number of targets slightly reduces motion displacement estimation accuracy, this accuracy consistently remains above 90%. Additionally, the motion frequency estimation error for each target remains below 1 cpm. These results demonstrate that the system maintains high estimation accuracy and reliability in complex multi-target motion scenarios.

### 7.1.4 Impact of the number of microphones.
To evaluate the impact of microphone number on system performance without reducing signal coverage, we reduce the microphone number to a minimum of eight. In our experiments, we test motion estimation with 16, 10, and 8 microphones for tracking two moving targets. The results (as shown
<figure><img src="image_28.png" /><figcaption>Fig. 18. Impact for different number of microphones</figcaption></figure>

<figure><img src="image_29.png" /><figcaption>Fig. 20. Different layouts of distributed microphone array</figcaption></figure>

in Fig. 18) indicate that motion displacement estimation accuracy reduces as the microphone number decreases. This decrease in accuracy is due to an increase in grating lobes generated during signal beamforming when fewer microphones are used, matching with simulation results in Section 4. These grating lobes interfere with accurate target identification and signal extraction, thus decaying the system performance.

### 7.1.5 Impact of different layouts of microphone array.
To ensure signal coverage at least four seating areas within the car, including the driver's seat, front passenger seat, and the left and right rear seats, we design four distinct microphone layouts for these key areas. Initial findings indicate that placing microphones too closely with each other, could impact the signal acquisition areas. As shown in Fig. 20, we adjust the microphone layouts on the sides, front, and rear of the test area to achieve closer alignment. The motion estimation performance is then tested for two moving targets across these four layouts. As shown in Fig. 19, results indicate that placing microphones at uniform intervals achieves the highest motion displacement estimation accuracy of 93.5%, with motion frequency estimation error remaining below 1 cpm. This reason behind is that a more uniform distribution of microphone array has no angle coupling between the azimuth and elevation [32], which leads to a larger effective receiving area, improving focusing and beamforming. Both experimental and theoretical results demonstrate that the focusing performance of the car-mounted distributed microphone array depends on the number and layout of microphones. For optimal signal focusing, it is suggested that the typical medium-sized four-seat car can be equipped with a minimum of sixteen microphones and evenly spaced apart throughout the car's ceiling.

### 7.1.6 Impact of w/wo the 3D-DFBF technique.
To compare the performance gains obtained with and without the 3D differential frequency based beamforming (3D-DFBF) technique, we use multi-target motion tracking as an example. In the experiments, one to five targets are controlled to move with varying displacements. Based on the target location information obtained from the occupancy grids, we utilize 3D-DFBF and the multipath superposition enhancement algorithm from the reference [46] to estimate each target's motion displacement and
<figure><img src="image_30.png" /><figcaption>Fig. 21. Performance of multi-target motion tracking with and without 3D-DFBF</figcaption></figure>

frequency. Fig. 21 shows the results, which show that using 3D-DFBF provides stable system performance with an average motion displacement estimate accuracy of 92%. The accuracy of the displacement estimation remains around 90% even when all five targets are present at the same time. On the contrary, without 3D-DFBF, the motion displacement estimation accuracy drops dramatically as the number of targets increases (from 93.4% to 34.9%). This decrease arises because, although the multipath superposition enhancement algorithm improves the signal-to-noise ratio, it cannot reduce interference from other targets. Consequently, the received signals become a composite of reflections from multiple targets. In summary, our 3D-DFBF approach significantly improves multi-target sensing performance, reducing motion frequency estimation error by 1.54 cpm and improving multi-target motion displacement estimation accuracy by 50%.

## 7.2 Applications

**Experiment setup:** We deploy a distributed acoustic system in the car, comprising four Bose speakers and 16 microphones. As illustrated in Fig. 22, we place the microphones uniformly around the car ceiling, with six microphones on each side and two on each end, based on established microphone module locations for in-car voice control [3] and benchmark experimental results. Following standard practice in common smart cabin [12, 14], we place four speakers in the A-pillar (2x) and B-pillar (2x), adjacent to the microphones in the center of each area. A laptop is connected to the entire setup, controlling the four speakers to transmit dual-frequency ZC-sequence signals in turn by signal frame, while the distributed microphone array captures the reflected signals. We have obtained the written consent from all participants before experiments, and our Institutional Review Boards have approved entire research activities.

<figure><img src="image_31.png" /><figcaption>Fig. 22. Experiment setup in the real car</figcaption></figure>

### 7.2.1 Case Study 1: Multi-target breath detection in car.
We recruit ten volunteers (four females and six males, aged between 10 and 65 years) to evaluate the impact of target diversity. For single-target breath detection,
<figure><img src="image_32.png" /><figcaption>Fig. 23. Experiment scenarios of multi-target breath detection</figcaption></figure>

<figure><img src="image_33.png" /><figcaption>Fig. 24. Breathing signal patterns of multiple targets</figcaption></figure>

volunteers are instructed to remain stationary at all five locations<sup>7</sup> within the car, with each individual collecting breath data for 2 minutes at each location. For multi-target breath detection, volunteers are instructed to switch locations in turn for breath detection. Fig. 23 illustrates our test scenario.

We first construct in-car space occupancy grids to estimate the number and locations of targets within the car. Subsequently, we focus the energy sequentially on each target for breath detection using the distributed microphone array 3D-DFBF technique. The breathing rate is calculated using an autocorrelation function (ACF), and the mean absolute error of this rate serves as a metric to validate the system’s effectiveness. Ground truth breathing waves and rates are collected using a belt recorder sensor, NUL-236 [1]. Multi-target breath detection aims to detect the breath of multiple passengers simultaneously. Specifically, we detect the breathing signal for each target, compare it to the groundtruth, and then calculate the average breathing rate estimation error for multiple targets as the system performance metric. Additionally, we compare our system to a baseline acoustic-based breath detection solution [46], which does not realize signal focusing within the car.

First, we compare with the baseline to evaluate the performance of single-target breath detection. Our system utilizes the acoustic focusing technique, and the baseline utilizes the multipath superposition enhancement algorithm proposed in reference [46]. Fig. 25a presents the mean absolute error of breathing rate estimation for a single target at different locations within the car. Compared to the baseline, the mean absolute error in estimating the breathing rate of a single target is reduced to 1/2. The reason is that our technique has the ability to focus the signal on the target, resulting in a higher sensing signal-to-noise ratio. However, the baseline algorithm is incapable of detecting multiple targets’ breath simultaneously. Fig. 24a and Fig. 24b illustrate the breathing signals from our system and the baseline, respectively, in the presence of two targets in the car. The baseline outputs the breathing signal waveforms as a superposition of the two targets’ signals, while our system clearly delineates the breathing waveform of each target.

Second, we compare our system to the baseline with different hardware deployments by evaluating the performance of multi-target breath detection. We deploy eight microphones in accordance with the layout

<footnote n="7">⁷Two in the front seats and three in the rear seats.</footnote>
<figure><img src="image_34.png" /><figcaption>Fig. 25. Overall performance of multi-target breath detection</figcaption></figure>

described in the baseline (two on each side of the B-pillar and four at the front under the rearview mirror), while keeping the speaker positions consistent with the current system (one on each side of the A-pillar and B-pillar). Then, breath detection is performed for 1 – 4 targets simultaneously within the car. The experimental results demonstrate that, as the number of targets increases, the microphone layout in the baseline shows serious interference between the extracted breath signals, leading to a higher mean absolute error in the estimated breathing rate (as shown in Fig. 25b). These results confirm that the application scenario studied in this paper is incompatible with the microphone layout in the baseline. Furthermore, Fig. 25b also shows the mean absolute error of breathing rate estimation for the microphone layout of our system in scenarios involving 1 – 4 targets in the car. The mean absolute error of breathing rate estimation does not exceed 0.85 bpm even with four targets in the car.

### 7.2.2 Case Study 2: Driver dangerous driving activity detection.
Driver dangerous driving activity detection plays a crucial role in monitoring and analyzing driver behaviors, such as fatigue driving and distracted driving, thereby contributing to the reduction of human-induced traffic accidents. Dangerous driving behaviors are often accompanied by abnormal head movements, which diverge from typical driving behaviors. For instance, when a driver is fatigued, one may observe a high frequency of head nodding, yawning, and eye blinking, which often occur unconsciously and repetitively. To facilitate in-car driver activity detection, we employ a distributed acoustic sensing system embedded in the car (as shown in Fig. 26). The evaluation experiments focus on four common dangerous driving activities (as shown in Fig. 27): eye blinking, head nodding, yawning (representing three drowsy driving activities), and looking elsewhere (indicating one distracted driving activity).

<figure><img src="image_35.png" /><figcaption>Fig. 26. Experiment scenario</figcaption></figure>

<figure><img src="image_36.png" /><figcaption>Fig. 27. Signal patterns of different dangerous driving activity</figcaption></figure>
<figure><img src="image_37.png" /><figcaption>Fig. 28. Performance of driver dangerous driving activity detection</figcaption></figure>

We utilize an in-car distributed microphone array sensing model to focus the signal on the driver's head, enabling extraction of head motion characteristics from sensing signals, specifically the magnitude of distance changes and movement duration. Given the distinct and consistent signal patterns generated by different driving activities, we apply a dynamic time warping (DTW) based matching [40] to measure the similarity between the current signals and pre-recorded activity signals stored in a template library, effectively identifying driver activities. In terms of data collection, we follow existing methods for driving data collection [34, 54]. To ensure safety, a moderator seated in the back of the car instructs the driver to simulate fatigued driving state with these four driving activities, including yawning (longer duration and greater head displacement), blinking (longer duration and more frequent), nodding (longer duration and greater head displacement), and looking elsewhere (longer duration). Additionally, a camera is placed in front of the driver to record the entire experiment as groundtruth. A total of ten volunteers participate in our experiment, and over approximately two hours, nearly 500 samples are collected.

Fig. 28a presents a confusion matrix for four activities, and the average detection accuracy exceeds 95.9%. The other detection errors in the confusion matrix can be attributed to several factors. First, the similarity in signal patterns between yawning and looking elsewhere often leads to confusion, with a 3% probability of misclassification between these two activities. Second, the signal fluctuations caused by nodding closely resemble those of yawning, making nodding prone to being misclassified as yawning. However, due to the typically longer duration of yawning compared to nodding, yawning is less likely to be misclassified as nodding. To address these issues, more robust activity classification methods, such as deep learning, can be employed. These methods are capable of adapting to a wider range of signal patterns and more driving activities, thereby enhancing the practical applicability of this system in real-world scenarios. Additionally, we compare our system with DriverSonar [34], a recent smartphone-based method using its speaker and microphone to detect driver activities. DriverSonar's detection performance declines significantly when the phone is not oriented toward the driver and does not recognize blinking, a common sign of fatigue driving. In contrast, our system achieves an average detection accuracy that is 3.4% higher and a detection accuracy of 96.4% for blinking, as shown in Fig. 28b.

### 7.2.3 Case Study 3: Multi-target gesture recognition in car.
Gesture recognition plays an increasingly important role in human-car interaction within smart cabins. By enabling the driver and passengers to interact with the car through intuitive hand gestures rather than physical buttons or touchscreens, gesture recognition enhances the ease and convenience of the driving experience. Based on the distributed microphone array, our system focuses signals on the hand region and detects gesture types by analyzing signal patterns induced by hand movements. As shown in Fig. 29, both the driver and passenger perform different gestures simultaneously, which the system then recognizes the signal pattern. As shown in Fig. 31, we evaluate five gesture types, i.e., pull, push,
push-and-pull, triangle, and knock twice. We instruct volunteers to perform these gestures, resulting in a dataset of 1,000 samples across the five gestures. Each target's gestures are recognized based on a DTW-based matching scheme [40], with accuracy shown in Fig. 30. Our system achieves an average gesture recognition accuracy of 94.5%. In contrast to previous gesture detection [48, 49], our solution allows gesture control according to the user's location. For instance, the passengers can open the window next to them by performing a simple gesture, such as "knock twice". Moreover, location-based authentication improves driver control security by identifying the user (such as the driver or passenger) through location-aware gesture recognition. Based on the new location-based sensing, we can train classifiers using neural networks through supervised learning to recognize more types of gestures and achieve higher recognition accuracy. Furthermore, the proposed acoustic focusing technique successfully addresses problems such as multi-target interference and system instability, providing the basis to enable personalized customization of a variety of applications.

<figure><img src="image_38.png" /><figcaption>Fig. 29. Experiment scenario</figcaption></figure>

<figure><img src="image_39.png" /><figcaption>Fig. 30. Accuracy of multi-target gesture recognition</figcaption></figure>

<figure><img src="image_40.png" /><figcaption>Fig. 31. Signal patterns of five hand gestures</figcaption></figure>

# 8 Related Work

## 8.1 Smart Sensing Techniques in Car Cabin

Smart car cabin has recently gained tremendous attention with various techniques being explored, including camera [31], optical/weight/pressure sensors [26][33][41], RF sensors (Wi-Fi, FMCW radar and UWB radar) and etc. For camera-based solution, the systems are usually based on image information from a camera mounted in front of the driver. They provide fatigue driving monitoring for the driver based on the analysis of his facial expression, eye blinking, or head motions [42]. Other applications include seat occupancy detection [25][44], intrusion detection [29] and child's presence detection [16][53]. However, the camera-based sensing requires good lighting conditions, and also cannot see through seats with blind spot. Traditional cars equip their passenger seats with specialized sensors, such as optical, weight, and pressure sensors. Only passenger occupancy can be
monitored by these sensors [26][33][43]. Additionally, the sensing coverage is primarily restricted to the seats. RF-based solutions have been recently popular. Activity monitoring [61] and child presence detection system [55] based on Wi-Fi sensing have also been proposed. CARIN [28] employs Wi-Fi CSI-based techniques to recognize activity of passenger in the presence of interference. However, due to the limited bandwidth, the sensing accuracy of Wi-Fi is insufficient, and it is easy to be interfered by rich multipath in car cabin. Meanwhile, UWB and FMCW radars have wide bandwidth and high sensing resolution. V2iFi [60] is proposed to use UWB signal to achieve vital sign monitoring in car cabin. UWB radar can also be used for car occupancy detection [38]. Zheng et al. propose hand gesture recognition in-car with Frequency Modulated Continuous Wave (FMCW) Radar and Transformer [59]. BSENSE [47] adopts a hybrid architecture that combines signal processing and a deep learning pipeline. This system can distinguish the breathing signals of adults, children and infants, and monitor their vital signs over time using millimeter-wave radar. However, these solutions require extra specialized hardware and expensive installation as being non-standard offerings in cars, preventing their wide adoption.

## 8.2 Acoustic Sensing in Car Cabin

Acoustic sensing has become an active area in recent years and has produced a wide range of applications, including activity sensing [34][36], gesture recognition [27][35], and vital sign monitoring [45][56]. For acoustic-based sensing in car cabins, most existing work employs the speaker and microphone inside a smartphone. For example, SteerTrack [52] can track the rotation angle of the steering wheel. The system transmits the acoustic signal and then analyzes the echoes reflected from the driver's hands to track the hand motions under different steering maneuvers. ER [51] leverages smartphone audio devices to realize an early warning recognition of inattentive driving events including turning back, fetching forward, eating/drinking, etc. Furthermore, D3-Guard [50] utilizes the Doppler shift of acoustic signals to capture the driver's action patterns to detect drowsy driving. BlinkListener [37] also uses a smartphone-emitted acoustic signal to detect eye blinking and identify drowsy driving. Note that all of these works use smartphone-based acoustic modules, however, the sensing range of smartphones is restricted to the driver's seat, making it difficult to accomplish full car sensing. Vecare [58] employs two speakers and one microphone within the car to detect child presence. However, this system does not consider the practical deployment locations of microphones and speakers in modern car cabin. The most recent work [46] proposes to utilize distributed in-car acoustic modules, which are already embedded into real cars, to monitor a child's body movements and breathing patterns, realizing a full-coverage, high-precision CPD system. Despite significant progress, this work only pays attention to sense vital signs of a single target. For complex multipath in a cabin, the system is unable to identify the target location and further sense multiple targets.

## 9 Limitation and Discussion

**Simultaneous breath and gesture detection.** Considering the scenario that the user performs hand movements (e.g., knock twice) while breathing, it is challenging to accurately identify grids that correspond to chest movements using the threshold-based approach. This is due to the fact that hand movements' high energy strength often obscures the signals associated with chest movements. While occupancy grids method can effectively differentiate between different targets (e.g., one person breathing while another is waving his/her hands), it is unable to differentiate between different parts of a single target. To address this limitation, we use 3D beamforming's focusing ability to separate signals from different body parts and extract sensing signals such as breathing and gesture motion. When a significant hand movement displacement interferes with the propagation path of reflected signals from the chest to the microphone, it leads to abnormalities in the breathing pattern. According to the recent radio-frequency based sensing works Mobi2Sense [57] and MSense [30], this issue can potentially be resolved by using the hand motion signal as a reference and applying a signal ratio technique to mitigate motion interference.
**Impact of microphone numbers.** The minimum number of microphones required for effective acoustic focusing depends on the size of the car cabin. Note that multi-target identification is not an issue for cars with only one seat. A traditional microphone array consisting of four to six microphones placed in front of the driver (e.g., near the rearview mirror) can be used. A smaller number of microphones is enough for two-seat cars, which have approximately half the interior space of four-seat cars [5, 21]. Based on modeling analysis, we suggest employing a minimum of 12 microphones to achieve effective acoustic focusing. In contrast, for larger seven-seat cars with three rows of seats [15, 23], which have significantly larger interior spaces, at least 20 microphones are required to ensure optimal spatial focusing performance.

**Impact of target size and thickness on tracking.** The size and thickness of the target have little impact on the system's motion tracking performance. There are two main causes: 1) the distributed microphone array, which captures target reflection signals from multiple directions, effectively mitigating errors associated with traditional single-direction microphone arrays; and 2) the 3D acoustic focusing technique employed by the system, which effectively separates target signals from interference, ensuring accurate extraction of motion characteristics regardless of target size or thickness. These capabilities contribute to the system's robust and stable tracking performance.

**Impact of hardware layouts on acoustic focusing performance.** To provide manufacturers with optimal guidance on microphone layout, we investigate the impact of alternative microphone layouts. For example, in a layout as reference [46], four microphones are positioned at the front beneath the rearview mirror, and two microphones are placed on each side of the B-pillar. Due to the absence of microphones at the rear of the car, distinguishing signals from specific directions becomes challenging. Another example is a layout where four microphones are evenly distributed above each seat in a four-seat car. In this layout, the reception ranges of the microphones overlap considerably, and there is substantial coupling between the azimuth and elevation angles involved in phase adjustment during 3D beamforming [32]. As a result, phase ambiguity occurs at certain angles, preventing the signal from being accurately focused on the target location.

**Impact of ambient noises.** While there might be some background noise throughout the experiments, including ambient traffic, speaking of the passengers and music playing, it has little impact on how effectively our system performs. The influence of noise is primarily concentrated in the low-frequency range (< 7 kHz). Although traffic noise may influence the higher-frequency range, these signals remain outside our target frequency bands (15 – 17 kHz and 18 – 20 kHz). During signal processing, we apply a bandpass filter to separate the signals within the target frequency bands, ensuring that noise from other frequencies does not affect signal quality.

## 10 Conclusions

In this paper, we present an acoustic focusing based sensing paradigm by leveraging the distributed acoustic modules in a car cabin for the first time. To achieve signal focusing in small spot regions, we mathematically model the distributed acoustic 3D spatial beamforming under near-field conditions. We propose the differential frequency signal processing method to virtually construct the low-frequency signal in order to address the phase ambiguity issue. We implement the proposed acoustic focusing-based sensing system with in-car acoustic modules. Comprehensive experiments demonstrate that the proposed system is capable of achieving accurate occupancy detection, multi-target sensing and is robust against interference. We believe distributed acoustic focusing based sensing is a foundation in the cabin, which can be used to support a variety of applications in smart cabins.
Acknowledgments

This work is partially supported by the National Natural Science Foundation of China (No. 62422213, No.62172394, No. 62072450), the Beijing Natural Science Foundation (L223034), the Beijing Nova Program (20240484641), the Innovation Team 2024 ISCAS (No. 2024-66).

References

[1] 2017. Respiration Monitor Belt logger sensor NUL-236. https://neulog.com/respiration-monitor-belt/

[2] 2018. What are Near Field and Far Field Regions of an Antenna? https://www.everythingrf.com/community/what-are-near-field-and-far-field-regions-of-an-antenna

[3] 2019. Advanced Microphone Applications for Automotive. https://www.kingstate.com.tw/index.php?do=news&id=44

[4] 2019. Hands-free Car Mic Module Design. https://www.kingstate.com.tw/index.php?do=technology&id=62&lang=en

[5] 2021. Toyota C+pod – details of new two-seat electric car. https://mag.toyota.co.uk/what-is-the-toyota-cpod/.

[6] 2023. 2021 Mercedes-Benz S-Class Burmester Sound System: Are 31 Speakers Enough? https://thedailymotor.com/sound-systems/2021-mercedes-benz-s-class-burmester-sound-system-are-31-speakers-enough/

[7] 2023. 4 Audio Trends Transforming the Automotive Industry. https://www.ti.com/document-viewer/lit/html/SSZT278

[8] 2023. AAC Technologies Launches a Complete Set of Automotive MEMS Microphone Modules to Accelerate Its Automotive Business. https://www.aactechnologies.com/post/aac-technologies-launches-a-complete-set-of-automotive-mems-microphone-modules-to-accelerate-its-automotive-business

[9] 2023. Audio Systems for the Tesla Model S and Model X. https://teslatap.com/articles/audio-systems-for-the-tesla-model-s-and-model-x/

[10] 2023. Automotive-qualified MEMS microphone - The car is all ears. https://www.rutronik.com/article/automotive-qualified-mems-microphone-the-car-is-all-ears

[11] 2023. The Best Car Stereo Microphones for 2023 You'll be heard loud and clear with these mics. https://futurism.com/best-car-stereo-microphones

[12] 2023. The BMW sound systems from Harman Kardon and Bowers Wilkins. Be it Harman Kardon or Bowers Wilkins: With these BMW sound systems, no wish remains unfulfilled. https://www.bmw-m.com/en/topics/magazine-article-pool/bmw-m-harman-kardon-bowers-wilkins.html

[13] 2023. Dimensions and Weights of TESLA. https://www.tesla.com/ownersmanual/model3/en_cn/GUID-56562137-FC31-4110-A13C-9A9FC6657BF0.html

[14] 2023. Tesla Model 3 Performance: Detailed review of the EV's music system. https://www.team-bhp.com/news/tesla-model-3-performance-detailed-review-evs-music-system

[15] 2024. 7-SEATER SUVS. https://www.landrover-kazakhstan.com/en/seven-seater-suvs.

[16] 2024. Customize exclusive solutions for your intelligent assisted driving system. https://www.candidelectronics.com/oe-part-oms-occupant-monitor-system-in-car-surveillance-camera_p23.html

[17] 2024. Dimensions of new BMW cars showing length, width and height. https://www.automobiledimension.com/bmw-car-dimensions.html

[18] 2024. Dimensions of new Tesla cars showing length, width and height. https://www.automobiledimension.com/tesla-car-dimensions.html

[19] Every drive is perfectly tuned. https://automotive.bose.com/road-ahead/in-cabin-experiences

[20] 2024. LISTEN DIFFERENTLY. Created by Bowers Wilkins for BMW. https://discover.bmw.co.uk/article/bowers-wilkins

[21] 2024. Porsche 718 Boxster 2016 dimensions, boot space and interior photos. https://www.automobiledimension.com/model/porsche/718-boxster.

[22] 2024. Smart Vehicle Cabins Market Size By Product, By Application, By Geography, Competitive Landscape And Forecast. https://www.marketresearchintellect.com/product/global-smart-vehicle-cabins-market/

[23] 2025. All the details you were looking for Kodiaq SE Drive (7 Seats). https://www.skoda.co.uk/new-cars/kodiaq/se-7-seats/dimensions.

[24] Fadel Adib, Zachary Kabelac, Dina Katabi, and Robert C. Miller. 2014. 3D tracking via body radio reflections. In *Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation* (Seattle, WA) (NSDI’14), 317–329.

[25] Maged S. AL-Quraishi, Syed Saad Azhar Ali, Muhammad AL-Qurishi, Tong Boon Tang, and Sami Elferik. 2024. Technologies for detecting and monitoring drivers’ states: A systematic review. *Heliyon* 10, 20 (2024), e39592. https://doi.org/10.1016/j.heliyon.2024.e39592

[26] Joan Albesa and Manel Gasulla. 2015. Occupancy and Belt Detection in Removable Vehicle Seats Via Inductive Power Transmission. *IEEE Transactions on Vehicular Technology* 64, 8 (2015), 3392–3401. https://doi.org/10.1109/TVT.2014.2356443

[27] Takashi Amesaka, Hiroki Watanabe, Masanori Sugimoto, and Buntarou Shizuki. 2022. Gesture Recognition Method Using Acoustic Sensing on Usual Garment. *Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.* 6, 2, Article 41 (July 2022), 27 pages. https://doi.org/10.1145/3534579

[28] Yunhao Bai and Xiaorui Wang. 2020. CARIN. *Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies* 4, 1 (Mar 2020), 1–28. https://doi.org/10.1145/3380992
[29] Haibin Cai, Donghee Lee, Hwang Joonkoo, Yinfeng Fang, Song Li, and Honghai Liu. 2017. Embedded vision based automotive interior intrusion detection system. In *2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)*. 2909-2914. https://doi.org/10.1109/SMC.2017.8123069

[30] Zhaoxin Chang, Fusang Zhang, Jie Xiong, Weiyan Chen, and Daqing Zhang. 2024. MSense: Boosting Wireless Sensing Capability Under Motion Interference. In *Proceedings of the 30th Annual International Conference on Mobile Computing and Networking* (Washington D.C., DC, USA) (*ACM MobiCom '24*). Association for Computing Machinery, New York, NY, USA, 108-123. https://doi.org/10.1145/3636534.3649350

[31] Meng-Che Chuang, Raja Bala, Edgar A. Bernal, Peter Paul, and Aaron Burry. 2014. Estimating Gaze Direction of Vehicle Drivers Using a Smartphone Camera. In *2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops*. 165-170. https://doi.org/10.1109/CVPRW.2014.30

[32] T. Filik and T. E. Tuncer. 2009. Uniform and nonuniform V-shaped planar arrays for 2-D direction-of-arrival estimation. *Radio Science* 44, 05 (2009), 1-12. https://doi.org/10.1029/2008RS003949

[33] Nik Mohd Zarifie Hashim, Hatijah Basri, Anuar Jaafar, Mohammed Abdul Aziz, Azahari Salleh, and Abd Shukur Ja'afar. 2014. Child in car alarm system using various sensors. https://api.semanticscholar.org/CorpusID:54643207

[34] Hongbo Jiang, Jingyang Hu, Daibo Liu, Jie Xiong, and Mingjie Cai. 2021. DriverSonar: Fine-Grained Dangerous Driving Detection Using Active Sonar. *Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.* 5, 3, Article 108 (Sept. 2021), 22 pages. https://doi.org/10.1145/3478084

[35] Dong Li, Jialin Liu, Sunghoon Ivan Lee, and Jie Xiong. 2023. Room-Scale Hand Gesture Recognition Using Smart Speakers. In *Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems* (Boston, Massachusetts) (*SenSys '22*). Association for Computing Machinery, New York, NY, USA, 462-475. https://doi.org/10.1145/3560905.3568528

[36] Jie Lian, Xu Yuan, Ming Li, and Nian-Feng Tzeng. 2021. Fall Detection via Inaudible Acoustic Sensing. *Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.* 5, 3, Article 114 (Sept. 2021), 21 pages. https://doi.org/10.1145/3478094

[37] Jialin Liu, Dong Li, Lei Wang, and Jie Xiong. 2021. BlinkListener: "Listen" to Your Eye Blink Using Your Smartphone. *Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.* 5, 2, Article 73 (June 2021), 27 pages. https://doi.org/10.1145/3463521

[38] Yongsen Ma, Yunze Zeng, and Vivek Jain. 2020. CarOSense: Car Occupancy Sensing with the Ultra-Wideband Keyless Infrastructure. *Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.* 4, 3, Article 91 (Sept. 2020), 28 pages. https://doi.org/10.1145/3411820

[39] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. 2019. Occupancy Networks: Learning 3D Reconstruction in Function Space. *IEEE Computer Society, Los Alamitos, CA, USA, 4455-4465*. https://doi.org/10.1109/CVPR.2019.00459

[40] P. Muhammad and S. Anjana Devi. 2016. Hand Gesture User Interface for Smart Devices Based on Mems Sensors. *Procedia Computer Science* 93 (2016), 940-946. https://doi.org/10.1016/j.procs.2016.07.279 Proceedings of the 6th International Conference on Advances in Computing and Communications.

[41] Andrea Palazzi, Davide Abati, Simone Calderara, Francesco Solera, and Rita Cucchiara. 2019. Predicting the Driver's Focus of Attention: The DR(eye)VE Project. *IEEE Transactions on Pattern Analysis & Machine Intelligence* 41, 07 (July 2019), 1720-1733. https://doi.org/10.1109/TPAMI.2018.2845370

[42] Yantao Qiao, Kai Zeng, Lina Xu, and Xiaoyu Yin. 2016. A smartphone-based driver fatigue detection using fusion of multiple real-time facial features. In *2016 13th IEEE Annual Consumer Communications Networking Conference (CCNC)*. 230-235. https://doi.org/10.1109/CCNC.2016.7444761

[43] Yao Rong, Chao Han, Christian Hellert, Antje Loyal, and Enkelejda Kasneci. 2022. Artificial Intelligence Methods in In-Cabin Use Cases: A Survey. *IEEE Intelligent Transportation Systems Magazine* 14, 3 (2022), 132-145. https://doi.org/10.1109/MITS.2021.3050883

[44] Leila Sharara, Alexandros Politis, Lubna Alazzawi, and Mohammed Ismail. 2023. An Embedded Real-Time Driver Monitoring System based on AI and Computer Vision. In *2023 International Conference on Microelectronics (ICM)*. 258-263. https://doi.org/10.1109/ICM60448.2023.10378917

[45] Xingzhe Song, Boyuan Yang, Ge Yang, Ruirong Chen, Erick Forno, Wei Chen, and Wei Gao. 2020. SpiroSonic: monitoring human lung function via acoustic sensing on commodity smartphones. In *Proceedings of the 26th Annual International Conference on Mobile Computing and Networking* (London, United Kingdom) (*MobiCom '20*). Association for Computing Machinery, New York, NY, USA, Article 52, 14 pages. https://doi.org/10.1145/3372224.3419209

[46] Yuqi Su, Fusang Zhang, Kai Niu, Tianben Wang, Beihong Jin, Zhi Wang, Yalan Jiang, Daqing Zhang, Lili Qiu, and Jie Xiong. 2024. Embracing Distributed Acoustic Sensing in Car Cabin for Children Presence Detection. *Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.* 8, 1, Article 16 (March 2024), 28 pages. https://doi.org/10.1145/3643548

[47] Mingyue Tang, Pranshu Teckchandani, Jizheng He, Hanbo Guo, and Elahe Soltanaghai. 2024. BSENSE: In-vehicle Child Detection and Vital Sign Monitoring with a Single mmWave Radar and Synthetic Reflectors. In *Proceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems* (Hangzhou, China) (*SenSys '24*). Association for Computing Machinery, New York, NY, USA, 478-492. https://doi.org/10.1145/3666025.3699352

[48] Penghao Wang, Ruobing Jiang, and Chao Liu. 2022. Amaging: Acoustic Hand Imaging for Self-adaptive Gesture Recognition. In *IEEE INFOCOM 2022 - IEEE Conference on Computer Communications*. 80-89. https://doi.org/10.1109/INFOCOM48880.2022.9796906
[49] Yanwen Wang, Jiaxing Shen, and Yuanqing Zheng. 2020. Push the Limit of Acoustic Gesture Recognition. In *IEEE INFOCOM 2020 - IEEE Conference on Computer Communications*. 566-575. https://doi.org/10.1109/INFOCOM41043.2020.9155402

[50] Yadong Xie, Fan Li, Yue Wu, Song Yang, and Yu Wang. 2019. D3-Guard: Acoustic-based Drowsy Driving Detection Using Smartphones. In *IEEE INFOCOM 2019 - IEEE Conference on Computer Communications*. 1225-1233. https://doi.org/10.1109/INFOCOM.2019.8737470

[51] Xiangyu Xu, Hang Gao, Jiadi Yu, Yingying Chen, Yanmin Zhu, Guangtao Xue, and Minglu Li. 2017. ER: Early recognition of inattentive driving leveraging audio devices on smartphones. In *IEEE INFOCOM 2017 - IEEE Conference on Computer Communications*. 1-9. https://doi.org/10.1109/INFOCOM.2017.8057022

[52] Xiangyu Xu, Jiadi Yu, Yingying Chen, Yanmin Zhu, and Minglu Li. 2018. SteerTrack: Acoustic-Based Device-Free Steering Tracking Leveraging Smartphones. In *2018 15th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON)*. 1-9. https://doi.org/10.1109/SAHCN.2018.8397115

[53] Hee Jung Yoon, Ho-Kyeong RA, Can Basaran, Sang Hyuk Son, Taejoon Park, and Jeonggil Ko. 2017. Fuzzy Bin-Based Classification for Detecting Children's Presence with 3D Depth Cameras. *ACM Trans. Sen. Netw.* 13, 3, Article 21 (Aug. 2017), 28 pages. https://doi.org/10.1145/3079764

[54] Lu Yu, Xinyi Yang, Hengjian Wei, Jianguo Liu, and Bo Li. 2024. Driver fatigue detection using PPG signal, facial features, head postures with an LSTM model. *Heliyon* 10, 21 (2024), e39479. https://doi.org/10.1016/j.heliyon.2024.e39479

[55] Xiaolu Zeng, Beibei Wang, Chenshu Wu, Sai Deepika Regani, and K. J. Ray Liu. 2022. WiCPD: Wireless Child Presence Detection System for Smart Cars. *IEEE Internet of Things Journal* 9, 24 (2022), 24866-24881. https://doi.org/10.1109/JIOT.2022.3194873

[56] Fusang Zhang, Zhi Wang, Beihong Jin, Jie Xiong, and Daqing Zhang. 2020. Your Smart Speaker Can "Hear" Your Heartbeat! *Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.* 4, 4, Article 161 (Dec. 2020), 24 pages. https://doi.org/10.1145/3432237

[57] Fusang Zhang, Jie Xiong, Zhaoxin Chang, Junqi Ma, and Daqing Zhang. 2022. Mobi2Sense: empowering wireless sensing with mobility. In *Proceedings of the 28th Annual International Conference on Mobile Computing And Networking* (Sydney, NSW, Australia) (*MobiCom '22*). Association for Computing Machinery, New York, NY, USA, 268-281. https://doi.org/10.1145/3495243.3560518

[58] Yi Zhang, Weiying Hou, Zheng Yang, and Chenshu Wu. 2023. VeCare: Statistical Acoustic Sensing for Automotive In-Cabin Monitoring. In *20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23)*. USENIX Association, Boston, MA, 1185-1200. https://www.usenix.org/conference/nsdi23/presentation/zhang-yi

[59] Lianqing Zheng, Jie Bai, Xichan Zhu, Libo Huang, Chewu Shan, Qiong Wu, and Lei Zhang. 2021. Dynamic Hand Gesture Recognition in In-Vehicle Environment Based on FMCW Radar and Transformer. *Sensors* 21, 19 (2021). https://doi.org/10.3390/s21196368

[60] Tianyue Zheng, Zhe Chen, Chao Cai, Jun Luo, and Xu Zhang. 2020. V2iFi: in-Vehicle Vital Sign Monitoring via Compact RF Sensing. *Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.* 4, 2, Article 70 (June 2020), 27 pages. https://doi.org/10.1145/3397321

[61] Yongpan Zou, Weifeng Liu, Kaishun Wu, and Lionel M. Ni. 2017. Wi-Fi Radar: Recognizing Human Behavior with Commodity Wi-Fi. *IEEE Communications Magazine* 55, 10 (2017), 105-111. https://doi.org/10.1109/MCOM.2017.1700170