Beyond Hearing: Learning Task-Agnostic ExG Representations from Earphones via Physiology-Informed Tokenization
Authors: Hyungjun Yoon
Seungjoo Lee
Yu Wu
XiaoMeng Chen
Taiting Lu
Freddy Yifei Liu
Taeckyung Lee
Hyeongheon Cha
Haochen Zhao
Gaoteng Zhao
Dongyao Chen
Cecilia Mascolo
Sung-Ju Lee
Lili Qiu
Venue: ICLR 2026 Poster (2026)
Keywords: Physiology-informed multi-band tokenization
ExG
Representation learning
Free-living ExG dataset
Task-agnostic training
Primary area: applications to neuroscience & cognitive science
TL;DR: We collected 50h of free-living ExG data with an earphone-based device and propose PiMT, a physiology-informed multi-band tokenization approach designed for task-agnostic representation learning with reconstruction-based pre-training.

Abstract: Electrophysiological (ExG) signals offer valuable insights into human physiology, yet building foundation models that generalize across everyday tasks remains challenging due to two key limitations: (i) insufficient data diversity, as most ExG recordings are collected in controlled labs with bulky, expensive devices; and (ii) task-specific model designs that require tailored processing (i.e., targeted frequency filters) and architectures, which limit generalization across tasks. To address these challenges, we introduce an approach for scalable, task-agnostic ExG monitoring in the wild. We collected 50 hours of unobtrusive free-living ExG data with an earphone-based hardware prototype to narrow the data diversity gap. At the core of our approach is Physiology-informed Multi-band Tokenization (PiMT), which decomposes ExG signals into 12 physiology-informed tokens, followed by a reconstruction task to learn robust representations. This enables adaptive feature recognition across the full frequency spectrum while capturing task-relevant information. Experiments on our new DailySense dataset—the first to enable ExG-based analysis across five human senses—together with four public ExG benchmarks, demonstrate that PiMT consistently outperforms state-of-the-art methods across diverse tasks.

BEYOND HEARING: LEARNING TASK-AGNOSTIC ExG REPRESENTATIONS FROM EARPHONES VIA MASSIOLOGY-INOORMME TOKENIZATION
Hyungjun Yoon
Seungjoo Lee Yu Yvonne Wu Xiaomeng Chen
Taiting Lu
Freddy Yifei Liu
Taeckyung Lee! Hyeongheon Cha! Haochen Zhao
Gaoteng Zhao
Dongyao Chen
Cecilia Mascolo
Sung-Ju Lee! Lili Qiu
'KAIST CCaneegee Mellon University UUniversity of Cambridge
*Shangaai Jiao Tong University
SPennsylvania State University 'UCA
" Northwest University
*University of Texas Austin "Microsoft Research
ABSTRACT
Electrophysiological (ExG) signals offer valuable insights into human physiology, yet building foundation models that generalize across everyday tasks remains challenging due two key limitations: (i) insufficient data diversity, as most ExG recordings are collected in controlled labs with bulky, expensive devices; and (ii) task-specific model designs that require tailored processing (i.e., targeted frequency filters) and architectures, which limit generalization across tasks. To address these challenges, we introduce an approach for scalable, task-agnostic ExG monitoring in the wild. We collected 50 hours of unobtrusive free-living ExG data with an earphone-basee based hardware prototype to narrow the data diversity gap. At the core of our approach is Plysiology-informeed Multi-hand Tokenization (PiMT), which decomposes ExG signals into 12 physiology-infee -nnooooe-dmmedd tokens, followed by reconstruction task learn robust representations. This enables adaptive feature recognition across the full frequency spectrum while capturing task-relevant information. Experiments on our new DailySense dataset- the first to enable ExG-based analysis across five human senses together with four public ExG benchmarks. demonstrate that PiMT consistently outperforms stae-offtheeaae methods across diverse tasks.
INTRODUCTION
Electrophysiological (ExG) signals, including electroenccepaalograpy (EEG), electromyogra-
phy (EMG), electrooculograhhy (EOG), and electrocardiograply (ECG), provide critical insights into neural, museular, ocular, and cardiovascular activities. They enable wide range physiological
applications, from gaze tracking (Merino et al. (200)) and emotion recognition (Gkintoni et al. 2025
to sleep staging Nguyen et al. 2016) and scizure detection (JW et al., 2016). Recent advances in deep learning have improved ExG analysis by developing data-driven training approaches (Song et al. 2022 Jiang et al. 2024) that capture complex temporal and spectral patterns for various physiological this, foundation models, which have demonstrated remarkable success across domains by leveraging large-scale data to learn general-purpose representations (Narayanswamy
tasks. Building
al. 2025 offer promising opportunity for advancing everyday ExG analysis.
However, ExG foundation models remain underexplored due to two limitations: (i) insufficient dataset diversity and (ii) task-specific model design. First, ExG datasets are typically collected in
controlled environments (Zheng & Lu 20155] Katsigiannis & Ramzan 2018 Wang et al. (023) using bulky, expensive devices (e.g., EEG headsets (Duvinage et al. 2013) This setup restricts both scale
"Equal contribution. The work is done during internship at Microsoft Research. Correspondence to: hyungjun yoon@ka ist. as.k, seungjoc weaggjocleeee eee cmu. edu, yw5730car uc. ajtu_chenxm@sjie juu. edu.en

---

and diversity across tasks, leaving free-living ExG data largely untapped. Second, existing ExG models are highly task-specific, relying on tailored processing pipelines, i.e., architectures optimized for fixed frequency band, which limits their generalization. For example, gaze tracking methods are designed to capture low-frequency bands (0. (0.1-15H1) Hz) (Merino et al. 2010), whereas emotion recognition relies higher EEG bands (8-30 Hz) (Gkintooii et al. 2025). As result, model trained for gaze tracking cannot be directly applied to emotion recognition, highlighting the lack of transferability across tasks.
To address the first challenge, we collected free-living ExG data in unobtrusive settings, constructing the DailySense dataset. For this, we prototyped NeuroBuds, an earphone-basee ExG sensing device. Unlike traditional bulky systems, NeuroBuds is lightweight, low-cost, and portable while still capturing rich physiological signals: near-ear EEG, EMG from facial muscles, and EOG from eye movements. This design enables long-term data collection, overcoming the constraints of lab-based recordings. Leveraging this platform, we collected 50 hours of free-living ExG recordings from 22 participants engaged in unconstrained daily activities. Furthermore, we gathered 20 hours of targeted lask-specific data spanning the five human senses (i.e., sight, hearing, taste, touch, and smell), establishing the first benchmark for evaluating model performance across diverse tasks.
Morcover, we propose Physiology-vve innioology-iformeed Multi-ban Tokenization (PiMT), an approach designed to learn task-agnostic ExG representations. Instead of relying wide-band input, PiMT decomposes ExG data into 12 fixed sub-band tokens, each corresponding to distinct physiological modalities. For instance, the [0.5 4 Hz] band captures EEG delta waves, which are informative for sleep staging (Elsaid & Labanowskij 2017), whereas the [15-45 Hz] band reflects low-frequency EMG activities, relevant for muscle activation and motor tasks (Allison & Fujiwara 2002). These structured tokens provide the encoder with fine-grained access to diverse spectral features, enabling the model to capture task-relevant information while remaining agnostic any specific task. Coupled with self-super reconstruction objectives, we train a robust, transferable representations that generalize effectively across diverse downstream tasks.
task-specific narrow band single
To evaluate our approach, we benchmark PiMT against the state-of-the-ar ExG training approaches. Specifically, we evaluate it on our newly introduced DailySense benchmark, which spans tasks across the five human senses, along with four widely used datasets covering diverse ExG applications, including emotion recognition, sleep staging, and brain-computer interface (BCI) tasks. Extensive experiments demonstrate that PiMT achieves robust performance and strong generalization across both DailySense and public datasets. Our key contributions are as follows:
We identify the key limitations of existing ExG wwwpeeoooks-issfffciine dataset diversity and task-specific model design- that hinder generalization to real-world applications.
We introduce NeuroBuds, an earphone-based prototype for unobtrusive, long-term ExG monitoring. Leveraging NeuroBuds, we curate DailySense, dataset containing 50 hours of free-living recordings and 20 hours of task-specific data spanning the five human senses.
We propose PiMT, a task- task-agnostic ExG training approach that incorporates novel, physiology-informt multi-band tokenization scheme. This enables automatic extraction of task-relevant features across the entire frequency spectrum.
Extensive experiments on DailySense spanning six distinct tasks and four public ExG benchmarks show PiMT achieves state-of-the-art performance, with an average Fl-score of 87,6 over baseline models.
Together, these contributions form integrated co-design in which the hardware, dataset, and training approach are mutually enabling, establishing a scalable foundation for rerl-world ExG analysis that bridges wearable sensing and foundation models for deeper human understanding.
2 RELATED WORK
To enable effective analysis of ExG signals and uncover valuable physiological patterns, recent ap- proaches can be categorized into following three main groups: (i) Conventional deep learning frame- works, such as EEGNet (Lawhern al. 2018) and DeepConvNet (Schirmeister et al. | (017)) leverage temporal and spatial convolutions to extract features direetly from raw ExG signals. (i)) Transformer- based models, which capture local and long-term temporal dependencies, are well-suited for complex,
2

---

Physiology-intormed band Tokenization (PIMT)
ExG signal
ExG filter bank
Multi-band tokens
Channel
EMG-LF -FF--LFI (15-45 Hi)
prp dastnod bands
Patch sag Commenttionn
Learnable tokintzer
Faequancy bands
E0G-overall (0.1 H4]
Channel 2
EEG CEG--tatta (6-8
EEG detta (0.5-4 HA)
Pre-training and fine-tuning
Bidirectional Marbaa
Norm
Multi-band tokens
Pre-training
Projection
(channcl-stacked)
Flatten & pooling
Reconstruction
decoder
emb
E.whh
Encodler
Forward Carry
Boclwerd Carne
Fine-tuning
Forward
Bookiward
Task-specific
Chamnel
classitteriregressor
Benerrs
Projection
+
Figure 1: Overview of PIMT. ExG signals are decomposed into 12 sub-bands via Physiology- informed Multi-band Tokenization (PiMT). A Bidirectional-Mambe encoder processes the tokens, and the model pre-trained with reconstruction tasks before fine-tuning on downstream tasks.
high-dimensional ExG signals. Early efforts such EEGConformer (Song al.. (2022) combine convolution and attention to jointly model local and global patterns. PatchTST (Nie et al.] 2023)
introduces patch-wise attention and independent channel encoding, while Medformer (Wang al.) 2024) enhances feature extraction through multi-scale patching and cross-channel attention. Most recently, Iitirectiinal-Mamtle (Zhu et al.) (024)) applies bidirectional state-space modeling for efficient long-range dynamics. (ili) Self-supervised learning methods aim to learn generalizable
representations from unlabeled ExG signals using proxy tasks such as masked modeling or contrastive learning. BrainBERT (Wang al., (0233) first applied BERT-style masked modeling to intracranial
EEG spectrograms. BIOT (Yang et al. 22023) extends this idea to cross dataset via patch-token trans- formers, and BrainWave (Yuan et al. 2024) further scaled it to foundation models trained large clinical datasets. However, despite these advances, existing approaches typically focus on specific ExG tasks and modalities, and primarily evaluated lab-controlled datasets. This limitation presents an opportunity to develop more generalized and robust representations from free-living ExG data. We address these gaps by introducing unified, frequency-agnostic framework trained real-world data, improving both robustness and practical usability.
3 LEARNING TASK- AGNOSTIC ExG REPRESENTATION
Motivation. Real-world ExG tasks are often associated with distinct physiological frequency bands. For example, gaze tracking with EOG signals typically relies on low-frequency components in 0.1-15 Hz range (Merino et al. 2010), whereas EEG-based emotion recognition depends on higher- frequency bands, such 8-30 Hz (Gkintoni al. 2025). Prior methods either design task-specific models (Gao et al. 2024 Altaheri et al. 2023) or apply harrow-band filters (Farhana et al. 2023 Apicella al. 2021). l). both of which limit generalization across tasks. While wide-band filter (e.g., 0-100 offers broader coverage, it suffers from loss of physiological features and poor task adaptation. Moreover, different hardware configurations may emphasize or attenuate specific frequency components, motivating flexible representation that does not rely prior assumptions about which bands or electrodes are informative. We aim to develop a task- -ggnsticc method that generalizes across tasks without relying on task-specific customization.
We propose training framework that enables NeuroBuds to generalize effectively across diverse tasks. Figure provides the overview. First, sesgiolooy-informee (PiMT) decomposes the input into 12 physiology-in -ifoooeed sub- bub-bands, producing tokens that grant the model fine-grained access task-relevant features across different frequency ranges. Next,
Overview.
Multi-band Tokenization
3

---

Bidirectional-Mann Mamba encoder generates embeddings from the tokenized representations. To leverage unlabeled free-living data, we introduce a reconstruction based pre-training to learn robust ExG representations. The pre-trained encoder is then fine-tuned on downstream tasks.
3.1 PHYSIOLOGYY-NFFRREE MULTI-BAND TOKENIZATION (PiMT)
To enable the task-agnosta framework, we design a two-step tokenization pipeline that converts raw ExG signals into structured embeddings: (i) physiology-infomee frequency decomposition via an ExG Filter Bank, and (ii) Patch Segmentation and Tokenization to generate input tokens.
ExG Filter Bank. Instead of relying task-specific frequency bands, we design fixed filter bank grounded in established physiological knowledge of ExG signals (Niedermeyer & Lopes da Silva
2005 Nunez & Srinivasan 2006; Task Force 1996). Concretely, define 12 canonical sub-band
filters spanning key physiological modalities: EEGidelta (0.5-4 Hz), EEG-theta (4-8 Hz), EFG-alpha (8-13 Hz), EEG-beta (13-30 Hz), HEG-gamma (30-100
(15-45 Hz), EMG-Midd-requenny (45-95 Hz), EMG-Hiig--Frequency (95-100 Hz), EOG-overall (0.1-20) Hz), BCG-Low-Frequency (0.03-0. Hz), ECG-Hiigh-Frqquenny (0. (0.12-0.488 Hz), and the QRS complex (8-50 Hz).
Multi-Band Filtering. ExG signals are decomposed into complementary spectral components by simultancously applying all filters in the bank. This decomposition provides the model with fine- grained. physiologically relevant features that span multiple modalities and tasks, rather than forcing reliance single- single-band representation. Formally, given multi-channel ExG signal X R.?." where X. is from channel с over time steps, we apply the NE filters to obtain band-specific signals X5 Xe e IRT where f E ,l,..., NF Each X retains only the components within band f. serving as the foundation for subsequent tokenization.
Patch Segmentation and Tokenization. The band specifcc signal {,, is segmented into non- overlapping patches, i.e., Pf.e.! E R", where w denotes the patch size and indexes its temporal position. This segmentation improves computational efficiency and facilitates modeling long-range temporal dependencies (Nie et al. 2023). Together, each patch contextualized by three dimensions: frequency f. channel c, and time 7, forming structured 3I) representation of the ExG input. Finally, each patch Pf., is projected into a latent embedding éf,cl E IR through a learnable tokenizer, where d denotes the embedding dimension. Specifically, we use single linear layer shared across all embedding space.
tokens to map each patch into
3.2 ENCODER
We adopt Bidirectional-Mann Mamba for its strong ability capture long-range sequential dependen- cies (Schiff et al., 2024; Shams et al. 2024). A detailed analysis of its effectiveness compared with standard Transformers on ExG data provided in Appendix H Furthermore, since PiMT introduces an additional frequency dimension that increases sequence length. Mamba is especially suitable: it achieves linear-time complexity in sequential modeling, whereas Transformers suffer from quadratic complexity (Gu & Dao 2024).
To fully leverage the rich structure multi-channel ExG signals, we organize the input tokens along three axes, i.e., frequency, channel, and time, in fixed scanning sequence. Based on empirical validation, we adopt a frequency-first (/), channel-second (c), and time-last (l) ordering scheme. To achieve this, the embeddings ffe..l are flattened into sequence following the (J x Z) order and then passed into the encoder. The encoder produces contextualized representations z, which are subsequently fed into the downstream task heads.
3.3 PRE- TRE-TRAINIGG FROM FREE-LIVING ExG DATA
Most existing ExG models are trained on lab-controlled datasets using task-specific designs, limiting their ability to generalize to real-world conditions. In contrast, ExG signals collected in free-living environments provide richer diversity and broader coverage of human activities, enabling models to learn more robust and general-pin -urppse representations. To exploit this free-living unlabeled data, we adopt a self-supervised pre- pre-training based on reconstruction objectives. Our design choice is motivated by prior work showing that reconstruction outperforms alternatives, iee, contrastive
4

---

learning, when training physiological foundation models on unlabeled data (Narayanswamy et al. 2025). To ensure robust feature extraction, we define six distinct reconstruction tasks, each paired with dedicated decoder, which are jointly used to train the encoder Eg.
Given a sequence of patches p generated from a raw signal x, the encoder Eg maps it into latent representation = (p(p). The decoder D& reconstructs the original signal A = DAB(()) (z). This task encourages the encoder to capture temporal features while reducing noise.
Autoencoding:
Masked Reconstruction: To enforce contextual learning, we employ masked reconstruction (Devlin 2018; He et al.) 2022). The patches p are partially masked along time, channel, and frequency dimensions, producing a corrupted version p maskk The encoder processes the masked input to yield mask (p ). The decoder DR recovers the original signal, generating pARR D (2 misk)
Frequency Domain Feature Reconstructions: To capture spectral information, we incorporate two frequency-domain reconstruction tasks. We first apply the Fast Fourier Transform (FFT) to obtain amplitude P* and phase p". Two decoders, D and D reconstruct these features from the encoded representation z, producing أ and p², respectively. Specifically, the two decoders aim recover the original frequency domain signals based on: pa = (z) and pa = D (z), respectively.
Masked Frequency Domain Reconstructions: To enhance the model's s capacity to infer spectral features from incomplete inputs, we apply the same frequency reconstruction tasks to masked input signals. Two additional decoders, DMA and D reconstruct the amplitude and phase, producing PMA and pM from 2 mask, The new decoders aim to recover the original frequency domain signals based on: = p== (z (zmask) and p" = mask), respectively.
MM
To jointly optimize the self-supervised objectives, assign each task independent decoder that reconstructs a specific aspect of the input signal, while sharing the encoder. Training is guided by mean absolute error (MAE) losses between the original and reconstructed signals. We combine these losses into a single objective by weighting each task-specific loss with a coefficient X, which controls its relative contribution. The overall reconstruction loss is thus weighted sum across all tasks. Each decoder is implemented as a lightweight MLP designed to reconstruct the target sequence. The À values were empirically selected, and details are provided in Appendix G
3.4 FINE-TUNING
Building on the representations learned from free-living data, we fine-tune the model to diverse downstream tasks (e.g., sight, hearing, taste, touch, and smell). The pre-trained encoder serves as feature extractor, while task-specific decoders are trained on labeled data. For classification tasks, we aggregate the encoder's patch-wise outputs into a fixed-length feature vector via mean pooling. The vector is then passed through a fully connected classification decoder trained with cross-entropy loss. For continuous regression tasks, such as gaze tracking, we employ linear decoder operating the patch level to generate sequential outputs, which are then aggregated into the final prediction. The model is optimized using standard regression loss.
DAILYSENSE: FREE-LIVING ExG DATA ACROSS FIVE HUMAN SENSES
We built DailySense, an ExG dataset collected through carphones, designed to enhance the ExG dataset diversity beyond traditional lab-controlled settings and to enable benchmarking across broad spectrum of daily life tasks. DailySense includes data from 22 participants, including 50 hours of unlabeled free-living recordings and 20 hours of labeled task-specific data spanning the five fundamental human senses] Compared with existing lab-based ExG benchmarks, which often involve similar number of participants but shorter recording durations (e.g., DREAMER includes 23 participants with approximately 20 hours of data), DailySense provides a more diverse and comprehensive dataset, laying a stronger foundation for generalizable ExG representation learning.
Data Collection Platform. To collect free-living ExG data, we developed NeuroBuds, an carphone- integrated ExG sensing prototype. Unlike traditional head-mounted ExG devices that are bulky and
'Or data collection was approved by the Institutional Review Board (IRB). We are also currently working with our legal team determine the possibility of publicly conditionally sharing the dataset.
5

---

NeuroBuds
Chh
EDG
Free living Gar-ExG
Downstream across five human senses
Right
Sight
Hearing
Taste
Touch
Smell
Representation Modia
ca
Trakoagnosic premaning
Figure 2: Overview of DailySense dataset. Using our earphone-base ExG analysis device, Neu- roBuds, we collect free-living unlabeled data for aagnotgc pre-training and labeled data spanning five human senses, serving as benchmarks for downstream tasks.
expensive (10,0000-500000) NeuroBuds employs an earhook-style form factor that is low-cost and compact, and well-suitable for scalable, long-term daily use. The device integrates amplification, digitization, onboard storage, and wireless transmission into a lightweight PCB (4.2 cm x 2.2 cm, 20 g. $80). During data collection, participants wore earphones with integrated electrodes and carried the PCB as shown in Figure 2 The around-the-ear electrodes can then capture ExG signals, including EEG (sites T7-T10, FT7-FTIO, TP7-TP10), auricular electrodes for EMG, and lateral electrodes for EOG, providing cognitive, muscular, and ocular coverage. We detail hardware design in Appendix A and the physiological rationale behind the design and signal quality for each modality in Appendix B
Data Collection Protocol. DailySense contains (i) unlabeled data of daily life and (ii) labeled data spanning the five human senses. total of 22 participants (ages 23-62, 16 men, women) wore NeuroBuds during daily routines without restrictions, performing natural activities such walking, cating, talking, and facial movements. This produced 50 hours free-living ear-ExG recordings. Furthermore, curated six benchmark tasks covering the five human senses: (i) gaze tracking, (ii) interest inference while watching videos (sight), (iii) interest inference while listening to audio (hearing). (iv) surface texture classification (touch: rough smooth), (v) taste classification (sweet vs. sour), and (vi) smell classification (floral vs. sour). Data were collected in a task-controlled environment with up to seven participants per task, producing 20 hours of labeled recordings. All experimental protocols followed prior brain-computer interface studies (Amini et al. 2022 Iravani
al. 2019 Namazi Kulish 2016
Vo al. 2023
Xia et 2023). Further experimental details
are provided in Appendix C
Data Processing. We adopted a minimal pre- pre-processing pipeline to ensure fair and uniform com- parison across methods. While certain tasks may benefit from additional task-specific filtering or explicit frequency-domáin transforms, introducing such pre-processing would encode task dependetn assumptions that conflict with our goal of task-agnostic modeling. Following established proto- cols (Jiang et al. 2024), each channel's time-domain signal first undergoes second-order IIR notch filtering (quality factor Q = 30) at both 50 Hz and 60 Hz using zero-phase forward-bar backward filtering. The notch-filtered signal is then processed using band-pass filtering implemented with first-order Butterworth low-pass and high-pass filters, again applied with zero-phase filtering. When PiMT is applied, 12 filter banks are constructed as parallel branches from independent copies of the notch-filtered signal. Each branch corresponds serves as an isolated physiological view of the same underlying signal. After filtering, signals are resampled 200 Hz and normalized. The continuous recordings are segmented into ooe-llerlapping 4-second windows. To improve model robustness, we augment the training data with small additive noise. Appendix provides visualizations of the collected signals.
predefined physiological frequency band and
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Baselines. We benchmarked PiMT against baselines including a traditional machine learning model (SVM), ExG-specific neural architectures (DeepConvNet (Schirrmeister et al. 2017). EEGNet (Lawh al. 2018), and EEGConformee (Song al.] (2022). and general purpose time-series models (Time-Series Series Transformer (TST) (Zerveas et al. 2021). PatchTST (Nie et al. (023). and Bidirectional-
general-purpose
Mamba (Zhu et al. 2024)). Among them, emphasize PStchTST strong baseline- advanced

---

Table 1: Performance of PiMT and baselines on DailySense. Classification results are in Fl-score, and gaze tracking performance is in angular error. Best results are highlighted in bold.
Classification (t)
Regression (4)
Method
Video
Audio
Taste
Touch
Smell
Avg.
Gaze
Without pre-triinin
SVM
0.665 = 0:075
0 a
0.556 0.134
0.554 0000
0.510 = 0.065
0.579
6.60" #: 1.27
EEGNee
0.753 0.119
0.712 + 0.34
0.709 0.08
0.643
0007
0.669 :: 0.033
0.697
6.52" à: 1:34
DeepConvNet
0.680 0.113
0.706 t 0.12
0.633 0074
0.638
0.636 :: 0.02.
0.659
7.04" à: 1.31
TST
0.773 0.112
0.705 a 0.100
0.731 no08
0.669
= 0.166
0.667 a: 0.0s
0.709
6.54% 4: 1.30
PatchTST
0.771 +: 0,146
0.749 a: 2.1.3
0.731 0.092
0,686 = 0.119
0.681 +: 0.049
0.724
6.47 4: 1.28"
EEGConformer
0.738 a: 0,127
0.752 à: 3.0444
0.688
0.022
0.678
0.102
0.670 :: 0.047
0.705
6.53 1.28"
Bidirectional-Mann
0,820 d: 0.102
0.858
0.1.13
0.733
0.000
0,762 0010
0.722 :: 0.067
0.779
6.533 1.16"
PIMT (Ours)
0.858 :: 9,084
0.885 0.125
0.790 0u77
0.807 0.113
0.753 0.w9
0.819
6.11 1.30
With pre-training
PTtchTST
0,807 0.146
0.786 0.146
0.697
0.999
0,700 0.131
0,670 0,182
0.732
6.42 1.33"
PIMT (Ours)
0.964 0,025
0.961 0,838
0 0,064
0.860
0.118
0.793
0.069
0.876
6.00 1.13
masked reconstruction model built on Transformer backbone that independently models each ExG channel and excels at capturing long-range temporal modeling.
Benchmark Datasets. We evaluated PiMT on DailySense along with four widely used ExG bench- marks: DREAMER (Katsigiannis & Ramzan 2018) and SEED (Zheng & Lu (015) for emotion recognition, Sleep-EDF (Kemp et al.] 2000) for sleep stage classification, and BCI Competition IV 2b (Leeb et al. 2008) for motor imagery. Dataset details are provided in Appendix E
Implementation Details and Metries. Our implementation consists of two primary stages: (i) pre- training on free-living data and (ii) task-specific fine-tuning. We first pre-train PiMT on 50 hours of free-living data using a masked reconstruction objective, then fine-tune on each downstream dataset using an 8:2 train-test split for each participant. For evaluation, we report the mean squared error (MSE) in angular (9) units for gaze tracking and maerr-averaged Fi-scores for all classifi- cation tasks. All tasks are repeated three times with different random seeds, and we report the corresponding standard deviation. For public benchmarks, data preprocessing follows the official EEGConformer implementation. Additional training and reproducing details, resource specifications, and hyperparameter tuning are provided in Appendix and Appendix G
5.2 EVALUATION ON DAISY
Table shows the Fi-scores of PiMT compared with the baselines DailySense. Overall, the results demonstrate that ear-ExG combined with PiMT can effectively capture five human senses, achieving up 81.9% F1 F1-score and as low 6. "gaze error, even without pre-training. Notably, PiMT outperformed all baselines, achieving a 4% improvement in F1-score and 0.41 "rduuction in PiMT's ability to interpret task-relevant frequency
gaze error. We attribute this generalizability bands, capability essential for handling diverse ExG-based tasks characterized by heterogencous frequency- features. We also observed that the Mamba-based backbone contributed significantly to performance gains; detailed comparisons against Transformer-bart variants are in Appendix H
Effect Pre-Training on Free-Living Data. key advantage of NeuroBuds is its ability facilitate effortless collection of ExG signals, enabling large-scale pre-training. We evaluated the performance of PiMT when pre-trained on free-living data and compared it with PatchTST, which the only baseline with a tailored pre-training strategy. As shown in Table T. pre-training improved the average Fl-score of PiMT from 81.9% to 87.6%. Similarly, PStchTST improved from 72.4% to 73.2%, whereas PiMT demonstrated a substantially larger gain. These results highlight the effectiveness of both our hardware- ennalleee-enbled free-living data collection and our reconstruction-bue based pre-training framework. Further analysis of our pre- pre-training is provided in Appendix[aame Appendix[]
5.3 EVALUATION ON PUBLIC BENCHMARKS
To validate the generalizability of PiMT beyond the DailySense dataset, we evaluated it on four widely used public benchmarks covering diverse ExG tasks. We compared against two strongest
7

---

Table Fl-score on four public ExG benchmarks across various tasks.
Baselines
DREAMER
SEED
Sleep-EDF
BCI Competition IV 2b
PatchIST
0.889 0.83
0.756 0.093
0.810 0.25
0.637 0.008
pitrectiionl-Maamiie
0.875 0.000
0.750
0.796
0.646
PIMT (Ours)
0,910 =
0.820 =: 0.121
0.066
0.693 0,004
Classification (t)
Regression (1)
video
0.95
audio
0.90
touch
MAE
smell
0.80
6.0
75
5.9
1band
2band
4band
12band
1band
2band
4band
12bnnd
Number of Frequency Bands
Number Frequency Bands
Comparison of different ExG tokenization strategies: 1-band (0.1-75 Hz), 2-band (0.1- 15 Hz and 15-75 4-band H0.1-5Hz, 5-15 Hz, 15-35 Hz, and 35-75 Hz), and our 12-band filter bank (described in Section 3.1
Figure 3:
baselines, PatchTST (Nie et al. 2023 and BidirectionalMaamaa (Zhu et al., 2024). As shown in Table 2 PiMT consistently outperformed all baselines across all datasets. Overall, these results demonstrate that our training strategy learns general-purpose ExG representations through PiMT, leading to robust performance across diverse benchmarks. This confirms generalization beyond the self-collected DailySense dataset to real-world benchmarks, which underscores the potential of NeuroBuds as a unified framework for generalizable ExG representation learning.
5.4 EFFECT OF MULTI-BAND TOKENIZATION
To understand the impact of multi-band tokenization, we compared the model performance under different frequency-band tokenization strategies on Daily Sense. As shown in Figure performance consistently improves as the number of bands increases. Our 12-band filter bank approach outperforms the 1-, 2-, and 4-band variants, achieving an average 4.6% Fl-score gain on classification tasks and the lowest gaze-tracking error. These results suggest that fine one-grained decomposition allows NeuroBuds exploit subtle but physiologically meaningful spectral cues. We also compare PiMT with common time-frequency tokenization methods in Appendix K demonstrating its superior performance.
Saliency Analysis. further understand the impact on downstream task, we conducted visual analysis of how different frequency bands contribute to each task. [ saliency maps that token during inference. Importantly, we observed clear task-relevant activation patterns: (i) gaze and video tasks, which closely linked eye movements, exhibited strong activation in low- fwwffreuency bands (Plöchl et al. (2012), and (ii) touch, taste, smell, and auditory interest classification emphasized high-frequency components, consistent with somatosensory beta-low-gamma activity involved in processing external stimuli (Bauer et al. 2006) and peri-auricular EMG spectra reflecting near-ear muscle movements (Goncharova al. 2003). These findings demonstrate that PiMT enables the model to dynamically focus on task- relevant frequency components without explicit supervision. We stress that this property essential for enabling generalizable ExG-based applications in daily-life scenarios using NeuroBuds.
highlight the contribution of each frequency-
5.5 IMPACT OF PRE-TRAINING DATA SCALE
We examine how the scale of pre-training data influences representation quality and downstream task performance. To this end, we randomly split the pre-training corpus into 80% training and 20% held-out test data, and subsampled varying proportions of the training set. F showss the test loss across epochs under varying training data scales. As expected, larger pre-training sets consistently produced lower losses, indicating that PiMT benefits from additional data and scales effectively.
6 presents downstream results on Deile For classification tasks, average performance saturates around 30% of the pre-training data, suggesting diminishing returns beyond this point.
8

---

Frequency Bands [Hz]
05:4 68 412
10
45
66 3.7 0.44
4-90
Video
Touch
Audio
Smell
Gaze
Figure 4: Saliency analysis demonstrating how the model dynamically captures task-relevant fre- quency bands via multi-bant tokenization.
Classification (t)
Regression (1)
Test (lo scale)
0.900
Date Scale
10%
0.875
30%
0.850
100%
0.825
0.800
Epoch
60
100
Data Scale(%)
10
30
50
100
0
Data Scale(%)
10 30
100
Figure 5: Test loss across dif- ferent pre-training data scales.
Figure Downstream performance of PiMT with varying amounts of pre-training data.
In contrast, gaze regression continues to improve up to 50%, highlighting task- dask-depnddent benefits of larger pre-training scales. Overall, these findings suggest that while some tasks quickly reach saturation, others continue benefit from larger-scale pre-training. Importantly, the consistent loss reductions in Figure confirm that PiMT can effectively exploit additional data, underscoring its promise as a seneral-purposs ExG representation model.
5.6 FURTHER ANALYSIS AND ABLATION STUDY
On-Device Deployment and Real-Time Analysis (Appendix(N) N). We evaluated the runtime overhead of PiMT (with Transformer as a backbone) on a commercial smartphone (Samsung Galaxy S24), which serves as a representative companion device for earphones. The model achieved efficient runtime performance with an average inference latency of 25 ms, memory usage of 266 MB (3.6%). and CPU utilization of 20.3%.
-nee-OnnSSuubett-Ouu Evaluation. To fur- ther assess pross-participaat generalization, we conducted suue-one-subjcct-uut (LOSO) exper-
Classification (1)
Regression
0.9
Angular Error (*)
iments on DailySense. In DailySense, some participants contributed to both free-living pre-
Score
training data and specifi data. Therefore, in LOSO, for each target user, their data was ex- cluded from pre-training and used only for test- ing. Despite this constraint, PiMT achieved per- formance comparable full pre-training (Fig-
0.6
Subject excluded
Subjert--verrapper
Avg.
Figure LOSO Performance when the target sub- ject's data is either included or excluded.
ure 7 confirming its ability to generalize effec- tively unseen users.
Ablation Study (Appendix J L). We performed an ablation study to investigate the contribution of each pre-training component to representation learning. As shown in Table 7 the complete model achieved the highest overall performance. We observed consistent performance improvements as additional components were incorporated, indicating the complementary benefits of each module. Furthermore, Table 9 shows that patch size of 0.5 seconds yielded the best downstream performance compared to alternative configurations.
Comparison with Controlled Pre-Treee Training Datasets (Appendix Q). We compare pre-training our 50-hour Deile dataset against larger controlled corpora: TUAR (Hamid et al. 2020) (98.6 hours) and TUAR+TUSZ (Shah et al. 2018) (498.6 hours). Despite its smaller size, DailySense achieves the best classification performance with competitive gaze accuracy, demonstrating that free-living data yields more transferable representations than controlled recordings.

---

6 DISCUSSION
We acknowledge several limitations of our current approach. Like many existing ExG frameworks, the scale of the collected dataset remains relatively limited. Although DailySense includes 50 hours of free-living recordings and 20 hours of task-specific data from 22 participants, comparable to or exceeding many existing lab- based ExG benchmarks, it is still modest by the standards of large-scale wearable sensing studies. In addition, the demographic diversity of the participants is limited, which may restrict the model's is ability to generalize across broader populations. Our LOSO experiments show promising cross- cross-subject generalization; however, performance drops when training and testing on different users (Appendix O highlight the persistent challenge of inter-subject variability (e.g., individual physiology and sensor placement). Addressing full cross-subject generalization is beyond the primary scope of this work, which focuses on task-agnosti representation learning under mixed ExG signals. Nonetheless, by demonstrating the effectiveness of earphone-ders derived tree-living ExG data for representation learning, NeuroBuds provides a scalable path toward broader population-level data collection and establishes the foundation for a more generalizable framework in future work.
7 CONCLUSION
We tackle two long-standing barriers in ExG analysis: (i) the lack of diverse, real-world data and (ii) the reliance on task-specific model designs. To address data diversity, we developed NeuroBuds, an earphone-based sensing prototype, and curated DailySense, the first dataset with 50 hours of free-living and 20 hours of task-specific ExG data spanning all five human senses. To overcome task- specificity, we propose Physiology-informet ExG signals into structured tokens across 12 canonical sub-bands aligned with distinct physiological modalities. Combined with reconstruction-baset pre-training on free-living data, PiMT learns robust representations that generalize across tasks. Evaluations on DailySense and four public benchmarks demonstrate that PiMT consistently outperforms state-of-the-arr baselines. Together, these contri- butions push ExG research beyond narrow, lab-constrained applications toward generalizable and real-world physiological sensing. Looking ahead, this work opens new opportunities in personalized health monitoring, cognitive interfaces, and scalable everyday sensing powered by ExG platforms.
Multi-band Tokenization (PiMT), which decomposes
ACKNOWLEDGMENTS
This work was supported by the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea govermment (MSIT) (No. 22240442-0444866, Non-invasive near-infrared based AI technology for the diagnosis and treatment of brain diseases). We sincerely thank Zilong Wang, Jiazhao Wang, Legend Zhu, Nicholas Yuan, and Qi Zhang for their insightful discussions and strong support throughout this project.
ETHICS STATEMENT
Our data collection was approved by the Institutional Review Board (IRB), ensuring the safety of both the participants and the device prototype used in the study. For the other experiments, used publicly available datasets, which were used in accordance with their intended purposes. There is no ethical issue with this paper.
REPRODUCIBILITY STATEMENT
Our Physiology- ennsiology-informed informed Multi-band Tokenization approach can be reproduced using the filter bank described in Section 3.1 Experimental details are provided in Section S Appendix and G
USAGE OF LARGE LANGUAGE MODELS
Large Language Models (LLMS)) were used solely for polishing the writing of this paper.
10

---

REFERENCES
Tobii
4c eye tracker.
213414285- 11155--peciii Fictiinsss wowpooos------e----oooe haaerhheeeeoiiil-eeeeaaarr-e---- 2016.
Garry T. Allison and Takayuki Fujiwara. The relationship between emg median frequency and low frequency band amplitude changes at different levels of muscle capacity. Cliniial biomechanics, 17 6:464-9, 9, 2002.
sewant wewemmattccccchaaa.. org Corp/oorpuusaaaa :10::2499
Hamdi Altaheri, Ghulam Muhammad, Mansour Alsulaiman, Syed Umar Amin, Ghadir Ali Altwaajrii, Wadood Abdul, Mohamed Bencherif, and Mohammed Faisal. Deep learning techniques for classification of electroenseephalogram (eeg) motor imagery (mi) signals: A review. Neural 2023.
Computing and Applicatiions, 15620)::46811144
Ali Amini, Karim Faez, and Mahmood Amiri. Surface roughness classification in dynamic touch using eeg signals. In 2022 30th International Conference on Electrical Engineering (ICEE), pp. 205-209, 2022. doi: 10.1 11199/CEEE566.22222.22222222
Andrea Apicella, Pasquale Arpaia, Giovanna Mastrati, and Nicola Moccaldi. Eeg-based detection of emotional valence towards a reproducible measurement of emotions. Scientific Reports, 11(1): 21615, 2021.
Markus Bauer, Robert Oostenveld, Maarten Peeters, and Pascal Fries. Tactile spatial attention enhances gamma-ban band activity in somatosensory cortex and reduces low- low-frequency activity in parieto-occipital areas. Journal of Neuroscience, 26(2):490-501, 2006.
Wenhui Cui, Woojae Jeong, Philipp Thölke, Takfarinas Medani, Karim Jerbi, Anand A Joshi, and Richard M Leahy. Neuro-gpt: Towards a foundation model for eeg. In 2024 IEEE International Symposium Biomedical Imaging (ISBI), pp. 1-5. IEEE, 2024.
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXiv:1810.04480 04805, 2018
Matthieu Duvinage, Thierry Castermans, Mathieu Petieau, Thomas Hoellinger, Guy Cheron, and Thierry Dutoit. Performance of the emotiv epoc headset for p300-based applications. Biomedical engineering online, 12:1-15, 2013.
Ola Elsaid and Michael Labanowski. Physiology, sleep stages. 2017. URL https://api
wowmannttcccholrr..r.oo////o :t:p://oooouu::::999:5550 79564518
Cunhang Fan, Jinqin Wang. Wei Huang, Xiaoke Yang. Guangxiong Pei, Taihao Li, and Zhao Lv. Leiht residual convolution-basee capsule network for eeg emotion recognition. Adv. Eng. Inform., (C), August 2024. ISSN 1474-0346. doi: 10. 2..1016f..ei.2224.10233.
URL
httpss///dii. www.oi..orr/iio wtt.r///...1111///... aei 21222.00222
Zitao Fang, Chenxuan Li, Hongting Zhou, Shuyang Yu, Guodong Du, Ashwaq Qasem, Yang Lu, Jing Li, Junsong Zhang, and Sim Kuan Goh. Neuript: Foundation model for neural interfaces. arXiv preprint arXiv:2510.16544 2025.
Iffat Farhana, Jungpil Shin, Shabbir Mahmood, Md Rabiul Islam, and Md Khademul Islam Molla. Emotion recognition using narrowband spatial features of electroeeneephalogaaphy IEEE Access, 11:44019-44033, 2023.
Pengxuan Gao, Tianyu Liu, Jia-Wen Liu, Bao-Liang Lu, and Wei-Long Zheng. Multimodal multi- view wawplll-sspaaaattppprre masked autoencoder for self-supervised emotion recognition. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1926-1930. IEEE, 2024.
Eygenia Gkintoni, Anthimos Aroutzidis, Hera Antonopoulou, and Constantinos Halkiopoulos. From neural networks to emotional networks: systematic review of eeg-based emotion recognition in cognitive neuroscience and real-world applications. Brain Sciences, 15(3):220, 2025.
Irina I Goncharova, Dennis J McFarland, Theresa M Vaughan, and Jonathan R Wolpaw. Emg contamination of eeg: spectral and topographical characteristics. Clinical neurophysiology. 114(9): 1580-1593, 2003.
11

---

Albert Gu and Tri Dao. Mamba:
Linear-tine time sequence modeling with selective state spaces. In First Conference on Language Modeling, 2024.
Ahmed Hamid, Katherine Gagliano, Safwanur Rahman, Nikita Tulin, Vincent Tchiong, Iyad Obeid, and Joseph Picone. The temple university artifact corpus: An annotated corpus of eeg artifacts. In 2020 IEEE Signal Processing in Medicine and Biology Symposium (SPMB), pp. 1-4. IEEE, 2020.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000-16009, 2022.
Behzad Iravani, Artin Arshamian, Kathrin Ohla, Donald A. Wilson, and Johan N. Lundström. Non- invasive recording from the human olfactory bulb. Nature Communications, 11, 2019. URL
https: //api. semant icscho ar. wtggCorpus...ooooooooogggg ID: 195391466
Weibang Jiang, Liming Zhao, and Bao liang Lu. Large brain model for learning generic representa- tions with tremendous EEG data in BCI. In The Twelfth International Conference on Learning Representations, 2024. URL httpt: hnttps::/openteeieww net forum ?ii.umiiiiiee Qid=
Britton JW, Frey LC, and Hopp JLet al. Eleetnoencehallggaapyy (EEG): An Introductory Text and Aslas of Normal and Abnormal Findings in Adults, Children, and Infants. Chicago: American Epilepsy Society, 2016.
Stamos Katsigiannis and Naeem Ramzan. Dreamer: A database for emotion recognition through eeg and ecg signals from wireless low-cost off-the-shelf devices. IEEE Journal of Biomedical and Health Informatics, 22(1):98-100 107, 2018. doi: 10.1 N0.1099/BHH122017268855533
B. Kemp, A.H. Zwinderman, B. Tuk, H.A.C. Kamphuisen, and J.J.I.. Oberye. Analysis of sleep- dependent neuronal feedback loop: the slow-wave microcontinuity of the eeg. IEEE Transactions on Biomedical Engineering, 4799):1185-1199) 2000. doi: 1091109/10.867988.
Daniel R Kramer, Krista Lamorie-Foote. Michael Barbaro, Morgan B Lee, Terrance Peng, Angad Gogia, George Nune, Charles Y Liu, Spencer S Kellis, and Brian Lee. Utility and lower limits of frequency detection surface electrode stimulation for somatosensory brain-computer interface in humans. Neurosurgical focus, 48(2):E2, 2020.
Vernon J Lawhern, Amelia Solon, Nicholas Waytowich, Stephen M Gordon, Chou Hung, and Brent Lance. Eegnet: a compact convolutional neural network for eeg-based brain-computer interfaces. Journal of Neural Engineering, 15(5):056013, jul 2018. doi: URL https://dss http://xx.oo..oo/111.11101 http:///s....aaooooo..../a/////2
Robert Leeb, Clemens Brunner, G Müller-Putz, A Schlögl, and GJGUOT Pfurtscheller. Bei competi- tion 2008-graz data set b. Graz University of Technology, Austria, 16:1-6, 2008
Louise R Manfredi, Hannes P Saal, Kyler J Brown, Mark C Zielinski, John F Dammann III, Vicky Polashock, and Sliman Bensmaia. Natural scenes in tactile texture. Journal of neurophysiology 17209))1722-1802, 2014.
Manuel Mcrino, Octavio Rivera, Isabel Gómez, Alberto Molina, and Enrique Dorronzoro. A method of eog signal processing to detect the direction of eye movements. In 2010 First International Conference on Sensor Device Technologies and Applications, pp. 100-105. IEEE, 2010.
Hamidreza Namazi and Vladimir V. Kulish. Fractal based analysis of the influence of odorants on heart activity. Scientific Reports, 6, 2016. URL htllhhtts///aaae www.::/w...saaaaacc.ceeccemcc org
CorpusID:14622292
Girish Narayanswamy, Xin Liu, Kumar Ayush, Yuzhe Yang, Xuhai Xu, shun liao, Jake Garri- son, Shyam A. Tailor, Jacob Sunshine, Yun Liu, Tim Althoff, Shrikanth Narayanan, Pushmeet Kohli, Jiening Zhan, Mark Malhotra, Shwetak Patel, Samy Abdel-Ghaffar, and Daniel McDuff. Scaling wearable foundation models. In The Thirteenth International Conference on Learning Representations, 2025. /ttp///////////////////////a//an
12

---

Anh Nguyen, Raghda Alqurashi, Zohreh Raghebi, Farnoush Banaei-kashan Ann C. Halbower, and Tam Vu. A lightweight and inexpensive in-ear sensing system for automatic whole-night sleep stage monitoring. In Proceedings of the 14th ACM Conference Embedded Network Sensor Systems CD-ROM, SenSys '16 pp. 230-244, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450342636. doi: 01445//994551129994500 URL
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Confer- Learning Representations, 2023. httpst/:::/ /oppnreviee.eeeeeeoooeeeeeeee..e // wtt.:.eeefffrumtmeeeeee forum?id-
net/
JbacOvTOco1
Emst Niedermeyer and Fernando Lopes da Silva (eds.). Eleetneencephalograppyy. Basic Principles, Clinical Applications, and Related Fields. Lippincott Williams & Wilkins, 5 edition, 2005. ISBN 0781751268.
Paul L. Nunez and Ramesh Srinivasan. Electric Fields of the Brain: The Neurophysics of EEG. Oxford University Press, 2 edition, 2006. ISBN 019505038X.
Iyad Obeid and Joseph Picone. The temple university hospital ceg data corpus. 10:394-398, 2016. doi: 1tt99//ninns211.001111
Michael Plöchl, José P Ossandón, and Peter König. Combining eeg and eye tracking: identifica- tion, characterization, and correction of eye movement artifacts in electtoencephaloorappii data. Frontiers in human neuroscience, 6:278, 2012.
Yair Schiff, Chia-Hsiang Kao, Aaron Gokaslan, Tri Dao, Albert Gu, and Volodymyr Kuleshov. Caduceus: bi-directional equivariant long-range dna sequence modeling. Proceedings of the 41 International Conference Machine Learning, ICML'24. JMLR.org. 2024.
Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for eeg decoding and visualization. Human brain mapping, 38(1 11:3311-424, 2017.
Margitta Seeck, Laurent Koessler, Thomas Bast, Frans Leijten, Christoph Michel, Christoph Baum- gartner, Bin He, and Sándor Beniczky. The standardized eeg electrode array of the ifen. Clinical neurophysiology. 128(10):2070-200) 2017.
Vinit Shah, Eva Von Weltin, Silvia Lopez, James Riley McHugh, Lillian Veloso, Meysam Golmo- hammadi, Iyad Obeid, and Joseph Picone. The temple university hospital seizure detection corpus. Frontiers in neuroinformaties, 12:83, 2018.
Siavash Shams, Sukru Samet Dindar, Xilin Jiang, and Nima Mesgarani. Ssamba: Self-supervised audio representation learning with mamba state space model. In 2024 IEEE Spoken Language Technology Workshop (SLT), pp- 1053-1059, IEEE, 2024.
Yonghao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao. Eeg conformer: Convolutional transformer for eeg decoding and visualization. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:710-719 2022.
ESC/NASPE Task Force. Heart rate variability: Standards of measurement, physiological interpreta- tion, and clinical use. Circulation, 93(5): 1043-106,, 1996. doi:
Hoang- -ea--Thhueeee Vo, Maa-hhu-Quyni Nguyen, Do Duc Cuong, and Tuan Van Huynh. Classification taste-eeg signals using base neural network. In 2023 RIVF International Conference on Computing and Communication Technologies (RIVF), pp. 107-111, 2023. doi: 35.1009/IIVFF001552200
10471803.
Christopher Wang, Vighnesh Subramaniam, Adam Uri Yaari, Gabriel Kreiman, Boris Katz, Ignacio Cases, and Andrei Barbu. Brainbert: Self self-supe representation learning for intracranial recordings. In The Eleventh International Conference Learning Representations, 2023.
13

---

Yihe Wang, Nan Huang, Taida Li, Yujun Yan, and Xiang Zhang. Medformer: A multi- granularity patching transformer for medical time-series classification. A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 36314-36341. Curran Asso- ciates, Inc., 2024. URL UtLLhhtpp:::/prrcceeiinsss.e. ps http:paper_fflespppeeee wonference.ec..ee Copeeeenr pdf
Xiuxin Xia, Yuchao Yang, Yan Shi, Wenbo Zheng, and Hong Men. Decoding taste information in human brain: temporal and spatial reconstruction data augmentation method coupled with taste eeg, 2023. BRLhttss::/ /ttetps://axi...ee. wttp:///wwwvv..r..ooo htt.t:p://a..i..ooraaaa///////// 2307 23377005665
Chaoqi Yang. M Westover, and Jimeng Sun. Biot: Biosignal transformer for cross-data learning in the wild. Advances Neural Information Processing Systems, 36:78240-78260, 2023.
Zhizhang Yuan, Fanqi Shen, Meng Li, Yuguo Yu, Chenhao Tan, and Yang Yang. Brainwave: A brain signal foundation model for clinical applications. arXiv preprint 1xxi::242..105// 2024.
George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery data mining, pp. 2114-2124, 2021.
Wei-Long Zheng and Bao-Liang Lu. Investigating critical frequency bands and channels for EEG- based emotion recognition with deep neural networks. IEEE Transactions on Autonomous Mental Development, 7(3):162-175, 2015. doi: 10. 20.100///MDD2221522331114
Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: efficient visual representation learning with bidirectional state space model. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org. 2024.
14

---

A NEUROBUDS HARDWARE DESIGN
To enable large-scale, in-the-wild ExG data collection, we built an earphone-based sensing platform consisting of two main components:
To adopt an earhook-style form factor, we use commercial earphone (Powerbeats PB123) as the backbone, and wrap conductive tape around the frame to form clectrodes. Each side includes five electrodes: the top ones the left and right act as bias and reference, while the remaining cight serve as signal channels.
Earphone-Shaped Sensing Array:
Lightweight Processing Board: We design custom Printed Circuit Board (PCB) integrating signal amplification, digitization, wireless transmission, and onboard storage:
Amplification: An bio-amplifier chip (ADS1299) and the front-end circuit support 8-channel ExG signal conditioning.
Digitization and Control: An ESP32 microcontroller handles A/D conversion, peripheral control, and real-time streaming.
Wireless Streaming: Microcontroller's built-in Wi-Fi/BLE enables direct transmission to phones or PCs for data collection or real-time on-device inference.
Storage: A microSD slot supports continuous onboard logging.
To minimize size and weight without compromising signal integrity, we adopted highly integrated chips (ADS1299, ESP32), and designed a compact 6-layer PCB with dense layout of components on both sides to further reduce footprint. The resulting design measures just 4.2 cm 2.2 em and weighs only 20 g. which is significantly smaller than existing COTS systems like OpenBCI (6.1 em x 6.1 cm, 80 g) or OpenEarable (5.7 cm x 3 cm, only support ExG channel).
During use, the board is enclosed in 3D-printed case and connected to the sensing array via Dupont wires. Users can wear the platform unobtrusively during daily activities, with the board placed in pocket worn the body, enabling free-living data collection.
B QUALITY OF EEG, EOG, AND EMG SIGNAL
Our electrode placement around the ear was carefully designed to capture EEG, EOG, and EMG signals while maintaining a compact and unobtrusive form factor. Below, we outline the physiological rationale and supporting evidence for each modality.
EEG: The electrodes align with standard around-the-car EEG channels, i.e., T7-T10, FT7-FTIO, and TP7-TP10 in the 10-10 EEG system (Seeck et al.) 2017). The strong classification performance on cognitive tasks demonstrates that our recordings contain reliable EEG activity.
EMG: Electrodes positioned on auricular muscles capture EMG signals linked to facial expressions. We further validated this by recording deliberate facial movements (e.g., blinking, biting), which produced distinct EMG-specific patterns.
Electrodes placed on both sides of the head enable strong horizontal EOG capture, with partial vertical EOG sensitivity due to vertical displacement. Eye movement patterns clearly observed in Appendix D and our gaze tracking accuracy (within 6. degrees as shown in Appendix(] P further supports the presence of robust EOG signals.
EOG:
are
This electrode configuration enables simultaneous acquisition of FEG, EMG, and EOG signals, providing rich multimodal ExG dataset while preserving wearability for daily use.
C DAILY DATA COLLECTION PROTOCOL.
As the first unified earphone-bast based system to explore whether minimally intrusive ear- ear-ExG signals reflect information for human senses, dataset design strictly follows established protocols traditionally collected by EEG headsets which are often binary settings. this section, we describe the detailed protocol of our six task-specific experiments. The experimental tasks included:
15

---

Participants were seated 60 cm from 13.5-inch laptop (model: Surface Book 2, display size: 3000 x 2000 pixels, vertical refresh rate: 59 Hz). This task evaluated whether EXG signals could accurately track gaze positions. The error was quantified as the angular difference between the ExG-based gaze estimation and the ground truth obtained 2016) (model: Tobii 4C Eye Tracker).
Gaze Tracking:
from Tobii eye tracker (tob)
Auditory and Video Interest Inference: Inspired by SEED and DREAMER datasets (Zheng Lu
2015] Katsigiannis Ramzan 2018), this experiment explored the correlation between ExG signals and engagement with visual/auditory stimuli. Partici- pants were asked to watch or listen to video clips. After each session, they rated their interest level. Each participant watched//istened to six stimulus clips, each lasting six minutes. The goal is to classify the participant's 's emotional state every four seconds based on ExG responses.
Surface Texture Classification (Touch Perception): Participants interacted with different textured surfaces to analyze ExG responses to tactile stimuli (Amini et al., 2022). Each participant rubbed either rough or smooth surface for 60 continuous seconds, repeating this process 10 times for each texture. The goal to classify the participant's touch perception every four seconds.
This experiment assessed ExG responses to different taste profiles (sweet vs. sour). Participants sipped liquid and held it in their mouth for 20 seconds (Vo et al. 2023 Xia et al. (203). To prevent cros-contaminationn
Taste Classification:
30-second rest period was enforced between different taste samples, allowing participants to rinse their mouths before four seconds.
proceeding to the next task. The task aims to classify the participant's 's taste perception every
Smell Classification: This task examined ExG signal responses to olfactory stimuli (Iravani jet al., 2019; Namazi & Kulish 2016). Participants inhaled pleasant and unpleasant odors, and the model was evaluated on its ability distinguish between different scent categories.
Table 3 provides a comprehensive summary of the classification labels, stimulus materials, trial durations, number of sessions, and trial structures per participant for each task.
Table 3: Experimental task details.
Task
Labels
Materials
Trial duration time
Total sessions
Total time
Rest time
Taste
Sweet
Chocolate milk
20 seconds
15
300 seconds
30 seconds
Sour
Vinegur
seconds
300 seconds
30 seconds
Touch
Rough
Scent paper
I minute
10
10 minutes
20 seconds
Smooth
Silk
10 minutes
20 seconds
Smell
Lavender
Lavender scent bag
20 seconds
15
300 seconds
30 seconds
Sour
Vinegur
20 seconds
300 seconds
30 seconds
Video
Interesting
Comedy Clips
5 minutes
30 minutes
30 seconds
Not-interesting
Lectures/ LectunesDocumenttary
minutes
30 minutes
30 seconds
Audio
Interesting
Comedy podcast
minutes
30 minutes
30 seconds
Not-interesting
Lectures
minutes
30 minutes
30 seconds
D VISUALIZATION OF EXG SIGNALS
We visualized the raw ExG signals measured using NeuroBuds and illustrated how they are trans- formed into multi-band tokens through bandpass filtering across different frequency ranges. Figure shows the decomposition of the raw signal into twelve frequency bands, each of which is subsequently tokenized as part of our multi-band sequence. For gaze tracking, low-frequency bands (e.g., EOG- overall and ECG-HF bands) exhibit clearer temporal patterns that align with EOG signals (Merino) jet al. 2010). In contrast, for tasks such as touch, wow-frequency activity is less prominent, while informative features emerge in higher-frequency bands (Manfredi et al. 2014 Kramer al. 2020)
16

---

In addition to evaluating ExG quality implicitly through downstream task performance, also conducted a direct quantitative comparison between our earphone- based NeuroBuds prototype and a research-grasd -gade OpenBCI device. Specifically, two participants (approximately one hour of synchronized data) and computed Pearson correlations between NeuroBuds and OpenBCI channels. 'he average cross-system correlation reached 0.71 (statistically significant, p < 0,001), demonstrating that NeuroBuds capture ExG/EOG activity with high consistency relative performed visualization analysis to assess overall signal similarity. Visual comparison of synchronized raw ExG signals shows that NeuroBuds and OpenBCI exhibit closely aligned temporal patterns, with similar waveform shapes, amplitudes, and drift trends throughout the recording as shown in Figure 9 These qualitative observations, together with the correlation analysis, further confirm that NeuroBuds produces ExG signals that closely match those from research-grann devices.
eye-movement tracking experiment with
laboratory-grade system. Beyond quantitative metries, also
Raw ExG signal
EEG-delta (0.5-4 Hz)
EEG LEG-gamma (30-100 Hz)
EMG-HF (95-100 Hz)
EEG-theta (4-8 Hz)
EOG-overall (0.1-20 Hz)
ECG-LF (0.03-0.12 Hz)
MM AM
EEG-alpha (8-13
EMG-LF (15-45 Hz)
ECG- HF (0:12-0.488 Hz)
EEG-beta (13-30 Hz)
EMG-MF (45-95 Hz)
complex
Figure 8: Raw ExG signals from DailySense dataset and their decomposition into twelve physiology- informed frequency bands for PiMT.
Cvvrriyy Compertion
CambOlDD DC
0.5
32.9
12.5
35.0
37.5
200
Figure 9: Visualization of synchronized ExG signals collected from NeuroBuds and research-grade OpenBCI device with the Pearson correlation of 0.7586, demonstrating strong cross-system signal similarity.
17

---

Table 4: Statistics of DailySense and benchmark datasets
Dataset
Device
Setting
Signals
Participants
Hours
Daily
Earphone
Daily free-living
ExG
22
70
DREAMER
Headset
Film watching
EEG
23
23
SEED
Headset
Film watching
EEG
15
45
Sleep-EDF
Polysomn.
Sleep lab
EEG,EOG,EMG
15
45
BCI Competition IV 2b
Headset
Controlled
EEG, EOG
9
6
E BENCHMARK DATASETS
We used four public benchmark datasets to further validate effectiveness PiMT. The statistics of public benchmark compared with DailySense is demonstrated in For all benchmark datasets, we performed random 80/20 split, assigning 80% of the data training and the remaining 20% to testing. We followed established protocols (Jiang et al. 2024 to preprocess the ExG signals.
DREAMER (Katsigiannis Ramzan 2018) is an EEG-based emotion recognition dataset collected from 23 participants while they watched 18 film clips designed to elicit different affective states. The dataset provides signals from electroencephalogram (14 channels 128 Hz) and electrocardiogram (2 channels at 256 Hz). Each trial is annotated with self-reported valence, arousal, and dominance scores on a 5-point scale. We used the EEG recordings for classífication of emotional states, formulating the task as binary classification based on dominance levels, where trials with dominance 2 3 were labeled as high and those with dominance < as low.
SEED Zheng & Lu 2015) is emotion recognition dataset with EEG recordings from 15 subjects. Participants watched 15 film clips (five positive, five neutral, and five negative) across three sessions. EEG was recorded from 62 channels using the ESI NeuroScan system 1000 Hz. We used the downsampled signal at 200 Hz. The dataset provides trial-level emotion labels (positive, neutral, negative).
SDeep-EDF (Kemp et al. 2000) is a dataset used for sleep stage classification. contains 197 whole-night polysomnographic recordings from both healthy subjects and patients with mild sleep difficulties. The EEG signals were recorded from two channels (Fpz-Cz and P7-Oz) at 100 Hz, and the EOG signals were also sampled 100 Hz. We used five-class scoring (W, NI, N2. N3, REM) for classification only using EEG signals.
BCI Competition IV 2b (Lebb from 9 subjects across sessions. Subjects were asked to perform left-hand and right-hand motor imagery tasks. Each session included multiple runs of motor imagery trials, with EEG recorded from three bipolar channels (C3, Cz, C4) at 250 Hz. The dataset defines two classes corresponding to left-hand and right-hand motor imagery.
2008 is motor imagery dataset consisting of EEG recordings
F IMPLEMENTATION DETAILS
For the Bidirectional-Mann Mamba model, we used 8 layers with a hidden state size of 16 and an em- bedding dimension of 64. The decoder consisted of two fully connected layers, each with a hidden dimension of 64. For the baseline implementations, we tuned SVM using grid search over C. (0.1, 1, 10, 100), E (0.01, 0.001, 0.0001), and kernel types (rbf. linear, poly). For the other baselines, we followed their official implementations and performed grid searches tune key hyperparameters, such as learning rate and batch size.
For the train/test split, we first segmented long sequences into 4-second windows and randomly shuffled them. We then applied a standard 80/20 division to construct the training and test sets.
To address potential inconsistencies with prior work, we replicated the exact data processing pipeline and code from the EEGConformer repository, including resampling, normalization, and standard segmentation protocols. We also re-evaluated all methods under this standardized setting, As shown in Table 5J PiMT consistently outperforms all baselines, confirming that our improvements hold across different preprocessing configurations.
18

---

Table 5: Fl-scores on SEED and BCI Competition IV 2b under the standardized EEGConformer preprocessing setting.
Method
SEED
BCI Competition IV 2b
PatchTST
0.945
0,826
Bidirectional-Marn -aabba
0.956
0.823
EEGConformer
0.955
0.855
PiMT
0.971
0.869
G HYPER-PARAMETER TUNING
Our implementation consists of two primary stages: representation learning and task-specific fine- tuning. During the representation learning stage, we pre-trained the model using mask and recon- struction objectives to learn robust representations transferable across various downstream tasks. The representation model trained on the entire hours of free-living data, after which fine-tuned on each specific task before final evaluation on the corresponding test set.
The weighting coefficients (A) for the pre- pre-training objectives were selected heuristically based on empirical observations. We initialized all À values to I and monitored the convergence dynamiic of individual loss terms. We found that the autoencoding loss (CAE) and masked reconstruction loss (LMR) converged more slowly than others; their weights were therefore increased 2 to encourage balanced training. While we did not conduct a full hyperparameter sweep, this adjustment yielded more stable convergence without introducing instability.
To further optimize performance, we performed grid search over key hyperparameters. Throughout the experiment, used AdamW optimizer with 0.01 weight decay. During the pre-training stage, the batch size was fixed at 256, and the learning rate was scheduled from 0.01 to 0.001 using a cosine decay scheduler. For the backbone architecture, we adopted Bidirectional Mamba model with 16 layers and a hidden dimension of 16. The masking ratio for the pre-training was fixed at 50%. During the fine-tuning stage, the batch size was fixed for all tasks, 10 for Gaze and for the remaining tasks. The learning rate followed cosine decay schedule from 0.001 to 0.00001. All experiments were repeated 3 times and the results are reported as the mean and standard deviation. All models were implemented using PyTorch, and the experimental evaluations were conducted on NVIDIA A100-SXM-8000 GPUs.
H BACKBONE COMPARISON: MAMBA VS. TRANSFORMER
We adopt Bidirectional-Mamba ZZhu et al. 2024) as our backbone architecture, which has demon- strated state-of-the-an performance across various time-series tasks (Zerveas et al. 2021 Song et al.)
2022). In addition to Mamba, Transformer based architectures are also widely used as backbones time-series foundation models. In our main evaluation (Scction(5.2) PatchTST (Nie al. 2023) showed strong performance; notably, it represents rating PiMT tokenization. We have shown the effectiveness of PiMT tokenization in Section 5.4 To further demonstrate the effectiveness our system design, we replace the Aidirectional-Mame with Transformer backbone for fair comparison. Specifically, we conduct the experiments with and without pre-training the free-living dataset.
transformer-basee architecture without incorpo-
As shown in Figure T0 the Mamba-based model consistently outperformed the Transformer-basel model under all settings, achieving 6.4% improvement without pre-training and an 8.5% gain with pre-training. These results confirm that Mamba is a strong architectural choice for ExG signal modeling.
I IMPACT OF PRE-TRAINING
To isolate the effect of the reconstruction-bent based pre-training from the contribution of PiMT, evaluated model performance without applying PiMT. We compared (i) Bidirectional-Mamne and (ii) Transformer-basee architecture (PatchTST), each trained with and without pre-training under
19

---

Without pre-training
With pre-training Cassificatioo (1)
Classification (1)
Regression (4)
Regression (4)
1.0
1.0
0.9
0.9
Angular Error
Angular Error
Transtormer
0.6
Transormer
0.5
0.5
Video Audio
Taste
Touch
Smel
Aug.
Gaze
Video Audio
Taste Touch Smell
Avg.
Gaze
Figure 10: Comparison of Mamba and Transformer backbones.
Table 6: Performance of Transformer and Bidirectional-Mann Mambba models with and without pre-training, excluding PiMT. Classification results in F1-score, and gaze tracking performance in angular emro.
Classification (t)
Regression (+)
Method
Video
Audio
Taste
Touch
Smell
Avg.
Gaze
Without pre-truining
Transformer
0.771
0.749 0.113
0.724
6.47
Bidirectional Mamba
0.820 + 0.858 0.113
0.600
0.762
0.779
With pre-training
Transformer
0.807 +
0.786 0.147
0.697
0.670
0.732
6.422
Bidirectional-Manan
0.891 = 0.077
0.901 0.04
0.741 0.106
0,806
0.188
0.726 0.092
0.813
6.23 a
identical settings. All models receive the same minimally processed input and share comparable architectural configurations.
Table the results. Pre-training consistently improves performance across both architectures. For Bidirectional-Manee pre-training increases the average classification F1-score from 0.779 to 0.813 and reduces gaze tracking error from 6.53 to 6.23. Similarly, for the Transformer architecture, pre-training improves the average F1-score from 0.724 to 0.732 and slightly reduces gaze error from 6.47 to 6.42. These results demonstrate that the proposed pre-training strategy provides measurable benefits even without PiMT, indicating that the learned representations from free-living data are intrinsically useful.
J EFFECT OF PRE-TRAINING COMPONENTS
Our pre-training framework on free-living data comprises six reconstruuctin--baeed tasks designed for unlabeled ExG signals: Autoencoding (AE), Masked Reconstruction (MR), (frequency) Ampli- tude Reconstruction (A), (frequency) Phase Reconstruction (P), Masked Amplitude Reconstruction (MA), and Masked Phase Reconstruction (MP). We assessed the contribution of each component by performing ablation experiments.
Table depicts the results. Although most ablation settings still achieve relatively strong perfor- mance, highlighting the overall effectiveness of pre-training, all are consistently lower than the full combination (0.876), confirming the benefit of jointly using all reconstruction tasks. The perfor- mance drops in individual ablations are modest, as complementary tasks help maintain efficacy. However, we observed task-specific sensitivities: for instance, MR, A. MA, and MP are particularly important for gaze tracking, where excluding them led to notable increase in error. Meanwhile, AE, MR, and phase-related reconstructions strongly influence taste classification. where their re- moval caused meaningful performance degradation. These findings suggest that temporal- versus Trequency-focuseee reconstruction tasks contribute differently depending on the task, reflecting the distinct feature requirements of each modality.
20

---

Table Pre-training ablation. Classification results are in Fl-score, and gaze tracking performance is in angular error.
Classification (t)
Regression (L)
Method
Video
Audio
Taste
Touch
Smell
Avg.
Gaze
PIMT (Ours)
0,964 0.0s
0,961
0.08
0.801 a.64
0,860 0.18
0.793 0.009
0.876
6.00" 1.15
w/o AE
0,970
0,959
0.806 0.66
0.852 0.155
0.778 0.085
0.873
6.00" 1.09
w/o MR
0,962
0,956
0.798 0.58
0.859
0.783 0/88
0.872
6.10% :: 1.00
0,967
0,955
0.816
0,850
0.768 0.079
0,871
6.19" # 1.23"
MA
0,965
0,960
0.818 0.062
0,849
0.774
0.873
6.12" # 1.20"
P
0,979
0,961 0.041
0.803
0.856
0.764
0.873
5.98" # 1.15
=: 01000
0,960 0,000
0.798 0.002
0.857 0.12
0.777 0.002
0.873
6.15" +: 1.26
K COMPARISON WITH DIFFERENT TOKENIZATION METHODS
To evaluate the effectiveness of PiMT, we compare it with two widely used Lime-frequenc, tokeniza- tion approaches: Short-Time Fourier Transform (STFT), Continuous Wavelet Transform (CWT), and raw signal. All methods are evaluated on DailySense using the same Bidirectional-Mane backbone and identical training settings to ensure fair comparison.
STFT. We generate frequency-domain domain features by sliding fixed window over time. The frequency vector extracted from each window is treated as token and passed into the same learnable tokenizer used in our framework.
CWT. We compute continuous time- frequeeryyncy coefficients using the Morlet wavelet, defining 12 frequency scales between 0.5 and 75 Hz for fairness. The resulting cene-ffuuunccy representation preserves timestamps without additional segmentation. We then apply our patching scheme, and the frequency-time features within each patch are used single token.
Raw Signal. As a baseline, we also evaluate direct patch-based tokenization on the raw signal without explicit frequency decomposition.
Table 8: Comparison of different tokenization strategies on the DailySense dataset. We report classification Fl scores (1) and gaze regression error in degrees (L).
Classification (t)
Regression (4)
Tokenization
Video
Audio
Taste
Touch
Smell
Avg.
Gaze
Raw Signal
0.820 a 0.101
0.858 a 0.113
0.733 = 0,800
0.762 = 0.100
0,722 0.07
0.779
6.53 + 1.30
STFT
0.800 a 0.101
0.855 a 0.112
0.731
:: 0,666
0.736 = 0.101
0.721 a: 0:47
0,769
9.04 a 0.4
CWT
0. 86863.. 0.10
0.881
1.10
0.765
0087
0.800 umm
0.757 0.64
0.813
6.57 1.30"
PIMT
0,858 0.084
0,885
0.125
0.790
0077
0.807
0.753 9.69
0.819
6.11 1.20
Table 8 shows the results. Frequency-domain approaches (STFT and CWT) provide meaningful representations for certain tasks, particularly Video and Smell, suggesting that explicit spectral structure can be beneficial. However, STFT exhibits degraded gaze performance (9.04)). ). likely due to loss of fine-grained temporal information. CWT improves STFT but remains below PiMT in overall performance.
PiMT achieves the best average Fl-score (0.819) and the lowest gaze error (6.11%), demonstrating strong performance across all tasks. By decomposing the signal into multiple physiology-informee bands while preserving time-domain structure within each band, PiMT balances frequency awareness with temporal fidelity, which is particularly important for temporally sensitive tasks such as gaze tracking.
L IMPACT OF PATCH SIZE
We discuss the impact of different patch sizes. Specifically, selected the patch size empirically based on performance trends across tasks. A sensitivity study illustrating the effect of different patch sizes is presented in Table 9 A smaller patch size provides less contextual information for each
21

---

Table Ablation study on the impact patch size. We report classification Fl scores (1) and gaze regression error in degrees (1). Our method (0.5 sec patch size) achieves the best overall balance across tasks.
Classification (f)
Regression (
Patchsize
Video
Audio
Taste
Touch
Smell
Avg.
Gaze
0.25 sec
0.977 0.200
0.967 0.03
0.776 0.065
0.849 + 0,124
0.741 =: 0,000
0.862
6,23" #: 1.25
0.5 see (Ours)
0.964 1028
0.961
0.801
0.64
0,860 a: 0.18
0,793
:: 0,000
0.876
6.00" LDP
1.0 sec
0,962 0034
0,945
= 0.04
0.821
0.063
0.836 a: 0,130
0.784
+: 0,083
0.870
6.10" a: 109
2.0 sec
0.947 0.299
0,932 9.074
0.776 a: 0.101
0.823 a: 0.125
0.769 +: 0.105
0.850
6.24% LOs"
classification window, which may limit performance. However, it benefits gaze regression, as the participant's is gaze is more likely to remain fixed within a shorter temporal window. In contrast, larger patch sizes offer more temporal context for classification tasks but increase the likelihood of gaze shifts or overlapping signals from multiple classes, potentially degrading both classification and gaze estimation performance. We observed that a patch size of 0.5 seconds provides the best trade-off, yielding strong performance across both classification and regression tasks.
M EVALUATION WITH FROZEN ENCODERS
Table 10: Evaluation under frozen encoder settings. We report classification F1 scores (1) and gaze regression error in degrees (4). PiMT maintains strong performance even when the encoder is frozen.
Classification (t)
Regression (L)
Method
Video
Audio
Taste
Touch
Smell
Avg.
Gaze
PIMT (finetunc)
0.964 0022
0
0,088
0.801 0.04
0.860 2.13
0.793
0.009
0.876
6.00
1.13"
PatchTST (frozen)
0.812 146
0.736 1.14
0.717
0.696
0.666
0.725
7.23"
PiMT (frozen)
0.957
0.944
0.799
0. 0.863 0.10
0.771
0.866
6.36"
To further evaluate the intrinsic quality of the learned representations, we assess model performance under a frozen-encodee setting. After pre-training, the encoder is kept fixed, and only lightweight task- specific decoder layers are trained for each downstream task. This protocol isolates the contribution of the pre-trained representation and reflects the feasibility of parameter-efficien adaptation compared to full end-to-end fine-tuning.
Table TO reports the results. PiMT with full fine-tuning achieves an average F1-score of 0.876 and gaze error of 6.00. When freezing the encoder, PiMT maintains strong performance (0. (0.866 average F1, 6.36 gaze error), indicating that the pre-trained features generalize effectively across diverse tasks even without encoder updates. For comparison, PatchTST under the frozen setting achieves an average F1-score of 0.725 and gaze error of 7.23, consistently below its fully fine-tuned counterpart. Notably, the gaze task exhibits a larger performance drop under frozen settings, suggesting that temporally sensitive tasks benefit more from encoder-level adaptation.
N EFFICIENCY ANALYSIS
Table II: Runtime performance of PiMT on smartphone (Samsung Galaxy S24).
Metric
Value
Inference Latency
25 ms
Memory Usage
266 MB (3.6%)
CPU Usage
20.3%
Model Size (ONNX)
2.0 MB
We evaluated the runtime overhead of our method on a commercial smartphone (Samsung Galaxy S24), which serves as a practical companion device for earphone-based systems. Since NeuroBuds
22

---

Table 12: Performance of PiMT compared PatchTST on the DailySense dataset under three data split settings: within-session, cross-session, and cross-
Classification (t)
Regression (L)
Method
Video
Audio
Taste
Touch
Smell
Avg.
Gaze
Widhii--essson
PatchTST
0.807
0.786 0.146
0.697 amo
0,700 0.31
0.670 0.082
0.732
6.42" 1.3*
PiMT (Ours)
0.964
0,961
0.801
0.860 0.18
0.793 0/000
0.876
6.00% 1.1
Cross-subject
PatchTST
0.654 0.047
0.595
0.561 0.047
0,553
0.539 0.002
0,580
7.07" * 1.25
PiMT (Ours)
0.612 =
0.578 0.092
0.593 0.038
0.577 0.092
0.571
0.586
7.78" * 0.95"
Cross-session
PutchTST
0.658 = 0.202
0,656 = u157
0 0.695 = 0.101
0,639 0.087
0.611 = 0.008
0.652
7.56" # 1.35*
PIMT (Ours)
0.697 = 0.24
0.698 = 8.18
0.704 = 0.106
0.763 = 0.156
0.639 = 0.146
0,700
6,98 =: 1.51"
supports real-time data streaming via BLE, we consider a deployment scenario where inference is offloaded to the smartphone.
Because the Mamba architecture is not yet supported on Android and lacks corresponding hardware acceleration, we substituted Mamba with a Transformer of equivalent architecture and parameter size (e.g., number of layers, dmode)). Prior work has shown that Transformers generally incur higher inference costs under comparable hardware acceleration (Gu & Dao 2024). To preserve the core algorithmic behavior of PiMT, retained PiMT and the 3D positional embeddings.
The resulting models were exported to ONNX and evaluated using 4-second input sliding windows (200 Hz sampling, with the same preprocessing in the main experiments). The measured runtime performance is summarized in Table I1
Overall, the results indicate that inference be executed in real time up 40 Hz with minimal resource consumption. Preprocessing operations such as filtering and windowing can be performed directly on the NeuroBuds board. In addition, given that Mamba has been reported offer 5x higher throughput than Transformers, we anticipate supporting on-device inference with even lower overhead.
O GENERALIZATION TO UNSEEN SESSIONS AND USERS
We also tested generalization to unseen conditions during testing in cross-subject and cross-session scenarios. Cross-subject involves training and testing on different users, while cross-session assumes the model is tested on a different session from the same user, introducing a temporal shift. These domain shifts are open challenges for ExG-based tasks, with prior work (Fan al. 2024) reporting over 30% drop in accuracy. As shown in Table [2] our method also experienced performance drop under the cross-subject setting (58.6%). In the cross-session setting, NeuroBuds showed stronger robustness, achieving 70.0% compared to PatchTST's s 65.2% Generalization to unseen conditions remains an open challenge and is focus of our future research. Nevertheless, we believe that large- scale data collection enabled by the daily usability of NeuroBuds can play key role in improving robustness in wallworldd applications.
P STATISTICAL SIGNIFICANCE
To assess whether our method provides statistically significant improvements over the baselines, we conduct paired Wilcoxon signed-rank tests on DailySense. Each task contains 6-9 participants, and for every participant we train each model with three random seeds. For a given seed, all models share the exact same train-test split; because the split strongly influences performance, constructing pairs at the seed level ensures fair and properly controlled comparison. For each model pair, we therefore form paired samples based on participant- seed combinations (e.g.. participants x 3 seeds 21 paired samples), and the Wilcoxon test is applied to this aggregated set of paired differences. For the classification tasks, we test whether the performance differences are consistently positive (higher F1 is better), and for gaze estimation test whether the differences consistently
23

---

Table 13: Paired Wilcoxon p-values when comparing PiMT each baseline on DailySense. Lower values indicate stronger evidence that PiMT outperforms the baseline.
Classification (t)
Regression (L)
Method
Video
Audio
Taste
Touch
Smell
Avg.
Gaze
Without pre-training
4-77
c.77
7-63
7.45c-09
4.77
9.61 **a
4.41c-05 ..a
SVM
***
...
64*
***
c.21 1.64e-04
1.23
1.49<-08
c.59
c-25
1.01
EEGNet
DeepConvNet
9.82e-05
c.77
c.64
1.42e-07
e.77
4.48
c.03
..e
TST
1.17e-04
1.19e-05
1.92e-02
4.
e.
1.22e-15
e.04
PatchTST
4.32 c-04 ...
3.00
c-49
c-03
c.65
c.14
***
EEGConformer
9.82c-05 6-25 ...
e.23
1.42e-07
c.24 60
c.28
1.38
Bidirectional-
6.14
1.83e-02
1.56e-02
1.12e.03
e.00
1.01 c-07
1.15
PIMT (Ours)
With pre-training
PatchTST
1.17e-04
9.
1.26
7.45 08
e
1.91 e-18
3.29 03
PIMT (Ours)
p S 0.05, pp 0.01, SE* p S 0.001. Entries marked "_" correspond self-comparisons.
Table 14: Performance on DailySense using different pre-training sources, showing our 50-hr free- living dataset outperforms larger controlled datasets.
Classifcationn (f)
Regression (L)
Pro-Training Dataset
Video
Audio
Taste
Touch
Smell
Avg.
Gaze
No PT
0,858
0,885
0.790 OCT
0.807
0.753
0.819
6.11"
TUAR (98.6 hrs) TUAR + TUSZ (498 6 hrs)
0.964 as85
0,950
0.791
ooss
0.824 0.124
0.736 0.02
0,853
5.95*
0.964 0004
0,950
0.04
0.803
0006
0.833 0.10
0.741 000
0,858
6.03² 1.14"
DallySense (Ours, 50 hrs)
0.964 0,025
0 0:838
0.801 6,064
0.860 =: 0.115
0.793 0.09
0.876
6.00 " LI3"
negative (lower angular error is better). For the "Avg." column, we pool paired differences across all five classification tasks before applying the test. As shown in Table 13 PiMT achieves statistically significant improvements over nearly all baselines and modalities, often with extremely small p-values, demonstrating that the gains are consistent across participants and robust to seed-level variation.
Q PRE-TRAINING USING PUBLIC CONTROLLED ExG DATASETS
We further examine how our 50-hour free-living DailySense dataset compares to pre-training on larger publicly available ExG corpora collected in controlled settings. Specifically, we evaluate models pre-trained on TUAR (Hamid al. 2020) and TUSZ (Shah et al. 2018), two widely used pre-training datasets in recent EEG foundation models (Jiang et al. 2024 Cui et al. 2024 Fang al. 2025). TUAR, a curated subset of TUEG (Obeid & Picone 2016), contains annotations for five artifact types-including eye movements, chewing, and muscle activity-makinn it relevant to our downstream tasks such as gaze tracking. TUSZ provides extensive seizure annotations and is among the largest publicly available EEG corpora. Since these datasets use different electrode montages, we select electrodes with closest spatial matches -F,, F8, T3, T4, T5, T6, O1, and 02 in the 10-20 system-for pre-training. We consider two pre-training configurations: (1) TUAR alone (98.6 hours) and (2) TUAR combined with a subset of TUSZ (498.6 hours total), representing moderate- and large-scale controlled EEG datasets, respectively. As shown Table 14 pre-training on DailySense achieves the best average performance across all five classification tasks and yields competitive gaze estimation accuracy, despite its substantially smaller size. This highlights the power of learning more generalizable and robust representations from free-living data, which better capture the natural variability present in sal-world human behavior than controlled laboratory recordings. Given the ease of free-living data collection, DailySense can be scaled more readily, which would further strengthen these advantages.
24


Reviews:
[{"review_id": "IkyUn10O6J", "reviewer": "Reviewer_mTe5", "summary": "This work built an earphone-based prototype for ExG data collection. The paper built a dataset consisting of 50 hours of free-living data, and 20 hours of task-specific data based on this new prototype. The paper further proposed a machine learning method called physiology-informed multi-band tokenization (PiMT) that learn representations from the ExG signals. The method is experimentally validated on both the collected dataset, and also four public ExG benchmarks.", "questions": "Have the author considered submitting the dataset to a journal instead?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- Solid prototype building and dataset collection efforts. The dataset is gonna be highly valuable to the community if made public, especially considering its multimodality nature and the study design that covers a wide range of tasks of interest.\n- Solid experimental efforts. The method is applied on a wide range of tasks, including both private datasets and public datasets, and showed good performance.\n- Very interesting saliency analysis, showing both how different frequency components are being effective differently on different tasks, and also shows how the multimodal ExG dataset is being effective such that it allows researchers to analyze such frequency components.\n- Interesting experiments in section 5.5, showing increasing pre-training data at scale can improve performance.", "weaknesses": "1. It is unclear if the self-collected dataset is superior comparing to existing non free-living datasets. Specifically:\n- The paper lacks experiments showing how the free-living dataset compares to existing larger-scale pre-training datasets, for example, the multimodal sleep datasets that contains thousands of hours of data (You snooze you win challenge, or TU datasets, as used in [1, 2, 3]). The paper hypothesize the potential benefits of collecting free-living ExG data, but the experimental results do not demonstrate as such. To demonstrate the self-collected datasets are effective, the authors should consider comparing the pre-training benefits based on the new dataset, and compare against the pre-training benefits given by using (1) 50 hours of existing datasets; (2) >>50 hours of existing datasets, and compare performance differences.\n\n[1] Chien, H. Y. S., Goh, H., Sandino, C. M., & Cheng, J. Y. (2022). Maeeg: Masked auto-encoder for eeg representation learning. arXiv preprint arXiv:2211.02625.\n\n[2] Liu, Ran, Ellen L. Zippi, Hadi Pouransari, Chris Sandino, Jingping Nie, Hanlin Goh, Erdrin Azemi, and Ali Moin. \"Frequency-aware masked autoencoders for multimodal pretraining on biosignals.\" arXiv preprint arXiv:2309.05927 (2023).\n\n[3] Fang, Ching, Christopher Sandino, Behrooz Mahasseni, Juri Minxha, Hadi Pouransari, Erdrin Azemi, Ali Moin, and Ellen Zippi. \"Promoting cross-modal representations to improve multimodal foundation models for physiological signals.\" arXiv preprint arXiv:2410.16424 (2024).\n\n2. The technical details of the proposed method are unclear, and the reported numbers are way off the range of existing numbers, leading to doubts and questions about their experimental settings. For example, in papers like [4], they report around 40% balanced accuracy on SEED-V dataset and around 54% balanced accuracy on BCI competition, which is the state-of-the-art performance that is commonly reported. Yet in this paper, there is 82% accuracy on SEED dataset (which one?) and 69% accuracy on BCI competition, which leads to concerns - have the authors used the common metrics like balanced accuracies? Also, the performance reported on SleepEDF seems to be quite low, see [2, 5], which both reported 83%-85% accuracy on SleepEDF. The experimental results inconsistency makes it hard to evaluate if the machine learning method is effective.\n\n[4] Wang, Jiquan, Sha Zhao, Zhiling Luo, Yangxuan Zhou, Haiteng Jiang, Shijian Li, Tao Li, and Gang Pan. \"Cbramod: A criss-cross brain foundation model for eeg decoding.\" arXiv preprint arXiv:2412.07236 (2024).\n\n[5] Eldele, Emadeldeen, Mohamed Ragab, Zhenghua Chen, Min Wu, Chee Keong Kwoh, Xiaoli Li, and Cuntai Guan. \"Time-series representation learning via temporal and contextual contrasting.\" arXiv preprint arXiv:2106.14112 (2021).\n\n\nOverall, the contribution of the prototype and the self-collected dataset is a bit disconnected from the machine learning perspective. I’d suggest the authors to design the experiments differently, to either showcase the effectiveness of the dataset, or the effectiveness of the machine learning approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "F22HUAEibz", "reviewer": "Reviewer_ejoF", "summary": "This paper introduces a new framework for scalable, task-agnostic Electrophysiological (ExG) signal representation learning from earphone-based sensors. To address data diversity and task specificity limitations of existing approaches, the authors develop a wearable hardware platform (“NeuroBuds”) and collect the DailySense dataset, comprising 50 hours of free-living recordings and 20 hours of labeled, multi-sensory data spanning the five human senses. The proposed Physiology-informed Multi-band Tokenization (PiMT) method decomposes ExG signals into 12 pre-defined, physiologically meaningful sub-bands, enabling generalizable tokenization. The model is pre-trained using self-supervised multi-task reconstruction and then fine-tuned for various downstream tasks. Comprehensive experiments benchmark performance against standard baselines across both the new DailySense dataset and four public ExG datasets, with results indicating significant improvements in generalization and accuracy.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Ambitious Data Collection and New Benchmark: The DailySense dataset, comprising the largest known free-living ExG recordings across diverse human activities and all five senses, marks a significant step toward real-world applicability for physiological sensing. The use of an earphone-based device (NeuroBuds) demonstrates strong engineering innovation, promising unobtrusive and scalable physiological monitoring.\n\n2. Principled Tokenization Approach: The PiMT framework’s multi-band tokenization sits on a solid physiological basis, decomposing signals into 12 canonical sub-bands. This overcomes the usual artificial rigidity of task-specific band choices and supports more generic, transferable representations. The explicit mapping of ExG modalities to sub-bands is clearly visualized and justified in both Figure 1 and the accompanying methodology.\n\n3. Thorough Experimental Validation: The paper conducts comprehensive comparisons against traditional and state-of-the-art baselines (SVM, DeepConvNet, EEGNet, PatchTST, EEGConformer, Bidirectional-Mamba), with both within-dataset and cross-benchmark evaluations. Ablations and analyses deepen understanding of where the gains come from and how PiMT generalizes.\n\n4. Foundation Model Potential: By combining self-supervised pre-training on free-living data with a flexible multi-band tokenization scheme, PiMT takes a meaningful step toward a foundation model for ExG. The method is not tied to any specific task or sensor configuration, and its performance gains across diverse tasks and datasets suggest that it learns robust, general-purpose representations. This is a notable advance in a field often dominated by narrow, task-specific models.", "weaknesses": "1. Limited Generalization to Unseen Subjects and Modest Cohort Size: While the participant count (N=22) is comparable to prior lab-based ExG studies, it remains insufficient to support strong claims of robust population-level generalization. This limitation is clearly evidenced by the significant performance drop in the cross-subject setting (Table 7), where the average F1-score falls to ~58%. Although the authors acknowledge this challenge and provide Leave-One-Subject-Out (LOSO) results (Figure 7), the sharp decline underscores that user-independent modeling remains a substantial hurdle. The current work is a robust proof-of-concept rather than a fully generalizable solution. Future work would benefit from larger-scale recruitment to better address subject variability.\n\n2. Ambiguity in Filter Bank Implementation and Saliency Analysis: The physiological basis for the 12 frequency bands is well-motivated, but the practical implementation lacks critical details, which may hinder reproducibility. The manuscript does not specify key filter parameters (e.g., filter type, order, transition bandwidth, or handling of overlapping bands like EMG-Low and EEG-Beta/Gamma). These details are crucial for replicating the tokenization process.\n\n3. Lack of Statistical Significance Testing: The reporting of mean performance with standard deviations is standard practice, but it is not sufficient to firmly establish the superiority of a proposed method over multiple strong baselines. The absence of statistical significance tests makes it difficult to assess whether the observed improvements (e.g., the 4% F1-score gain in Table 1) are statistically reliable, especially given the variability inherent in physiological data. Incorporating such tests would greatly strengthen the quantitative claims.\n\n4. Insufficient Validation of NeuroBuds Signal Quality Against Gold Standards: A key selling point of the work is the novel NeuroBuds hardware. However, the validation of signal quality is primarily indirect, relying on downstream task performance. To fully establish the device's credibility for research use, a more direct, quantitative comparison with a clinical-grade or research-standard ExG system (even on a small subset of participants) would be highly valuable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8bhdbeFfep", "reviewer": "Reviewer_UWPo", "summary": "This paper introduces pre-training methods and a new tokenization scheme for electrophysiological (ExG) signal modeling.  Their proposed pre-training methods and Physiology-informed Multi-band Tokenization scheme aim to address the challenge of ExG model generalization.  Evaluated on common ExG benchmark datasets, the introduced models outperform existing baselines.  The authors additionally collect the DailySense dataset, a free-living dataset of ExG data collected while participants perform tasks targeting different human senses, to further benchmark the generality of their proposed model.", "questions": "- What is the performance of the model without PiMT but with a proposed pre-training strategy?  Can other model architectures benefit from a similar pre-training strategy? If I am understanding correctly, table 1 shows model performance with and without pre-training but always uses PiMT. \n- The results of this work primarily tackle task-generalization.  Do you expect that this physiologically informed tokenization scheme can also help extract features that are beneficial for cross-subject generalization?\n- After fine-tuning, how well do the learned features generalize to the evaluation tasks of interest when the encoder is completely frozen (as opposed to enabling task-specific model updates during fine-tuning)?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The authors tackle the important challenge of model generalization in ExG modeling.  In many cases in this modeling domain, generalization is largely accounted for with dataset scale, requiring more resources.  However, the novel tokenization scheme proposed in this work achieves improvements in generalization by developing an encoding scheme that seeks to explicitly account for known physiological principals of ExG signals.\n- Proposed strategies (pre-training and PiMT) consistently outperform baseline models across a variety of common (and new) benchmarks. \n- Ablation studies are provided to verify the relevance of different pre-training strategies, frequency bands, backbones, and other design choices.", "weaknesses": "- The physiologically informed aspect of PiMT assumes electrode configurations that can yield signals of interest at each frequency band and does not account for mixing of these signals, potentially limiting generality of the scheme to different hardware configurations.  For instance, computing EOG signal features from occipital electrodes may not have much physiological meaning (the model would still likely learn features relevant to the training task, but the physiological intuition diminishes). \n- Baseline models evaluated in Table 1 are all evaluated on data collected from NeuroBuds.  These baselines may not be readily applicable to minimally pre-processed data and performance improvements may be observed when using data that has been processed further.  PiMT explicitly performs filtering across various frequency bands of interest, so implicitly includes additional preprocessing that other methods may not benefit from.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GaEB54nclV", "reviewer": "Reviewer_ZFuK", "summary": "The author introduces a new design of earphone based wearable device incorporating several electrode sensors to capture EEG, EOG, and EMG. A new benchmark dataset DailySense, containing 50 hours of data from 22 participants, is proposed. A paradigm pre-trained model is presented and indicates leading performance across several downstream human sensing tasks against baselines.", "questions": "Most of my suggestions are comprised in the weaknesses above. Overall the presentation of the paper is clear and all the notions are explained comprehensively.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- From the hardware design perspective, the design of the proposed wearable is ingenious. The earbuds based device is able to capture brain activity, heart activity, and facial muscle activity simultaneously. The device is lightweight and low-cost. \n- From the modeling perspective, the proposed Physiology informed Multi-band Tokenization (PiFT) is novel. The design of this tokenization process is reasonable in handling the sensing data with multiple channels and varied frequency-band of interest. \n- The experimental results are comprehensive, where varied baselines are included in comparison. Ablation studies are conducted to justify the efficiency of the proposed components.", "weaknesses": "## The modeling approach requires some additional justification. \n- If the emphasis is on the proposed PiFT mechanism, then it would be better to show that the gain in performance is consistent and independent of the backbone. The experimental setting could either be:\n    - 1) With PiFT fixed, other than just modeling with Bidirectional Mamba, also model with other backbone such as Transformers. \n    - 2) With the same backbone, compare PiFT performance against other different tokenization mechanisms, for example short term Fourier transform and continuous wavelet transform which are widely used in signal processing applications. \n- Recognizing the relatively higher uncertainty (due to more hyper-parameters and configuration) and timely cost on evaluate models’ representation with finetuning, I would suggest evaluate with linear probing, and compare against several recent and open-sourced (i.e. can be used off-the-shelf) models that are pretrained on wearable signals and the design of the model are intend to be channel agnostic, such as CBraMod [1] and NormWear [2]. \n\n## There are some limitations on the collected data. \nSince collecting data often contain difficulties, given the thoughtful device design and reasonable modeling scheme, this issue is not a major concern, but it still deserves to be mentioned. \n- The size of the collected data is a bit limited, where 50 hours of wearable signals from 22 participants is considered a small dataset in the wearable sensing domain. Also the demographic diversity of the dataset is also a limitation. \n- The downstream tasks constructed from the collected dataset tend to be relatively straightforward, with the majority formulated as binary classification problems.\n- A well-rounded data sheet reporting the statistics of the collected data and the 4 public datasets will improve the presentation a lot. For example, the total hours of data, participants information, and distribution of different sensor signals, etc. \n\n## Reference\n[1] Wang, Jiquan, et al. \"Cbramod: A criss-cross brain foundation model for eeg decoding.\" 2024. \n\n[2] Luo, Yunfei, et al. \"Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals.\" 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)