VIBEID: A STRUCTURAL VIBRATION-BASED SOFT BIOMETRIC DATASET FOR HUMAN GAIT RECOGNITION

Authors: Mainak Chakraborty, Chandan, Sahil Anchal, Bodhibrata Mukhopadhyay, Subrat Kar
Primary area: datasets and benchmarks
Keywords: Structural vibrations, Gait Recognition, Deep learning, Machine learning

Abstract:
We present VIBeID, a dataset and benchmark designed for advancing non-invasive human gait recognition using structural vibration. Structural vibrations, produced by the rhythmic impact of the toe and heel on the ground, are distinct and can be used as a privacy-preserving and non-cooperative soft-biometric modality. We curated the largest dataset VIBeID consists of footfall generated structural vibrations of 100 subjects. Existing datasets in this field typically include around ten subjects and lack comprehensive exploration of domain adaptation. To thoroughly explore the domain adaptation aspect of this biometric approach, we recorded vibration data on three distinct floor types (wooden, carpet, and cement) and at three distances from the geophone sensor (1.5 m, 2.5 m, and 4.0 m), involving 40
and 30 subjects, respectively. Additionally, we benchmarked our dataset against video recordings from 15 individuals in an outdoor setting. Beyond providing 88 hours of raw vibration data, VIBeID establishes a comprehensive benchmark for a) person identification: where the aim is to recognize individuals through their unique structural vibrations, b) domain adaptation: assessing model performance across different walking surfaces and sensor positions, and c) multi-modal comparison: comparing vibration-based and vision-based identification methods. Our experiments, using both machine learning and deep learning approaches, establish a baseline for future research in this field, and introduce a large-scale dataset for the broader machine learning community.

Decision: reject
Reviewer scores: 6, 6, 6, 5

Reviews and discussion:
<thread>
creation date:
1730220243542

thread_type:
official_review

<thread_object>
creator: reviewer_35gq

<content>
summary:
The paper presents a novel dataset termed VIBEID, designed for human gait recognition using structural vibration data. The dataset includes recordings of 100 subjects across various distances, floors, and environments. Experiments demonstrate that structural vibration can serve as a viable biometric trait across different scenarios.

strengths:
- Vibration-based gait recognition introduces a novel approach for human identification.
- The proposed baseline methods are effective.
- This work establishes the largest-scale vibration-based dataset to date.

weaknesses:
- The dataset has a limited number of subjects, although it includes over 88 hours of recorded data.
- Compared to commonly used vision-based gait recognition, the operating distance remains relatively short.
- The evaluation setup lacks clarity.

questions:
1. To what extent do walking speed and the carrying of objects impact recognition performance?
2. Building on question 1, does abnormal gait pose significant challenges for re-identification?
3. The VIBEID dataset studies an operating distance range of 1.5m to 4m, while vision-based gait recognition typically works at distances over 10m. What is the distance limit for vibration sensors to capture meaningful gait signals?
4. If there are obstructions between subjects and the sensor, is reliable recognition still possible?
5. How does the proposed vibration-based gait recognition handle scenarios with multiple pedestrians walking simultaneously?
6. I recommend replacing Figure 3 with a clearer version.
7. The evaluation protocol should be more detailed.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730493179810

thread_type:
official_review

<thread_object>
creator: reviewer_lqav

<content>
summary:
The presented work introduces a new biometric dataset for human gait recognition based on structural vibrations. The dataset is applied to various tasks such as person identification, domain adaptation, and multi-modal scenarios combining vibration and vision-based identification methods. Experimental analysis includes verification of machine-learning and deep learning approaches.

strengths:
1. The description of the data collection protocol is clearly written with sufficient details and clear explanations of the research motivation
2. The introduction and related work sections contain important background information justifying the motivation for the introduced dataset.

weaknesses:
1. Is there any requirement for using a specific sensor type during the inference if trained on the presented dataset? I'm wondering about the practical implication of the proposed solution. 
2. The work indicates that the concurrent activity was not taken into account, however it's very possible to happen in real-life scenarios. Would the presented dataset be sufficient for handling such scenarios? How should one prepare for additional noise introduced in this way? 
3. It's not clear how filtering of potential noise was performed? Was the assumption that the data collection is performed in an isolated environment without any noise? You mentioned that there was environmental noise present, but how do you quantify its presence? If the assumption is that there is minimal or no noise, it again raises question around the practicality of the solution.
4. It's not clear how the data was split for training and testing? Were the same subjects present in both subsets or did you ensure no overlap?
5. One of the motivation behind introducing a new dataset is that other datasets contains a limited number of subjects. It's mentioned that there are 100 subjects in the proposed dataset but then only 30 and 40 subjects are used for floor types and distance measurements. Why not all 100 subjects were used for all of the scenarios?

questions:
1. What do you mean by events in Table 2?
2. Line 407 - where is table 10? or did you mean 1?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730519738398

thread_type:
official_review

<thread_object>
creator: reviewer_g5tv

<content>
summary:
The authors built a dataset with 100 people using geophone to do multiple experiments based on the human's structural vibration. It consists of multiple covariances including floor types, and distances. The work tries to find a connection between the vibration and identities and builds multiple benchmarks.

strengths:
The paper is clear and easy to follow, and the tables and figures are easy to understand
The proposed question is interesting, trying to build the connection between identity and walking vibration, a fine detail when a human is walking. And authors collect a relatively large dataset in multiple conditions.

weaknesses:
In real-life applications, it is hard to find a good condition to use a geophone to capture a human's gait with little noise.

How to control the noise in outdoor cases.

Compared to a camera, the vibration-based method is restricted by the sensor and distance. 

The protocol is not clear. How is the train and test set defined? For human identification, the identities appearing in the training set will not be present in the test set. It seems these experiments do not follow this setting

questions:
Although the authors define the floor in different classes, the hardness might be a more reliable way to classify. Since different carpets' thicknesses may have different responses. And what is the distance range for the geophone, since 4 m is not far for camera sensor
</content>
</thread_object>

</thread>
<thread>
creation date:
1730983797985

thread_type:
official_review

<thread_object>
creator: reviewer_sqmc

<content>
summary:
This study introduces a benchmark for gait recognition utilizing a novel structural vibration sensing technique, the geophone. and the new benchmark comprised of in total 100 subjects, collected under indoor or outdoor settings. It invesitgated whether the novel sensing modality can encode identity related information, and what is the limitations or sensitivity of this technique. Although this work addresses an interesting topic, it may not yet provide the technical depth or extensive experimental validation expected for broader applicability. There are also several concerns regarding the experimental settings and presentation clarity.

strengths:
This paper presents a well-justified study, especially it clearly identifies current research gap for person identification. Overall it reads very well.

weaknesses:
* [Sample size] Probably for the gait recognition task, we are more interest in how many subjects collected. The subject size, compared with current large dataset, especially the GaitSet, is still not that comparable. 

* [Technical contents] I am concerned that the technical content is somewhat limited, even for a benchmark paper, for ICLR. Please consider adding more experiments and tasks to thoroughly validate the usability of this dataset, such as Re-ID, gait event detection, and generalization across subpopulations... I encourage the authors to refer to established works like GaitSet for inspiration. Gait data is highly complex, influenced by factors such as age, gender, emotion, and health conditions. Reflecting on these factors in your experiments would enhance the depth of the study.

* [Applicability] I am also concerned that this kind of ambient sensor can only be applied indoor or with relatively small distances, which might limit its application, compared to wearable data?

* [Experiment setup] In the experiment settings, I noticed that there were no concurrent human activity when recording the data, this may be another issue that limits the usability of this study. Additionally, will the data be sensitive to the perspective of the sensor, as I know it is quite sensitive for vision based person identification. 

* [Gait event] gait event detection, is this be validated in terms of accuracy?

* [Dataset details] Further elaboration on the dataset’s composition and subject split for person identification would be valuable, particularly for readers unfamiliar with this topic. 

* [Table clarity] Table 5 is not clearly illustrated, what is the performance comparison between structural vibration and camera? Very limited information is given in both the table and the associated texts. Expanding on this comparison would help readers understand the relative strengths of each technique.

* [Minor - clarification] I assume the subjects of A2, A3, A4 are part of A1, correct? Please clarify this. 

* [Minor - citation] When citing a work which actually does not play any role in your sentence, please use (X et al., XXXX), rather than X et al. (XXXX).

questions:
I would appreciate if the authors could address the concerns I raised in the weaknesses section.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces VIBeID, a comprehensive dataset for human gait recognition using structural vibration, addressing the limitations of prior small-scale datasets. VIBeID includes over 88 hours of vibration data from 100 individuals, collected across different floor types (wood, carpet, cement) and distances (1.5m, 2.5m, 4.0m) from a geophone sensor, with additional multi-modal data combining vibrations and video recordings. It establishes benchmarks for person identification, domain adaptation, and multi-modal comparison, showcasing the effectiveness of machine learning and deep learning methods, such as ResNet-18 and ResNet-50, for identifying individuals based on their unique walking-induced structural vibrations. The dataset demonstrates structural vibration as a non-invasive, privacy-preserving biometric modality suitable for diverse environments, with significant potential for applications in security, healthcare, and smart buildings. Future work aims to expand the dataset and explore new applications, further solidifying structural vibration's role in soft biometrics.

The paper is well written and the dataset is composed of quite new modality for human gait recognition. However, some concerns are still not addressed:
- Dataset Scale: Recent gait recognition datasets using emerging sensors like Lidar typically involve thousands of subjects, which makes the size of VIBEID appear limited in comparison. The feasibility of geophone-based gait recognition has already been demonstrated by pioneering works. The small scale of dataset may limit the usage in ML community considering that this data modality is new.
- Feasibility in the real world: The current approach still struggles with simultaneously identifying multiple target subjects. This limitation significantly reduces the system’s practical applicability in real-world scenarios. I do believe single-person also has a value yet the audience may not be ML community (more in ubiquitous computing / mobile computing).

Therefore, I think the paper has novelty but may not be ready for an ICLR publication.