Reviewerly/exHarmony huggingface dataset:
Learning Outlier-Aware Representation with Synthetic Boundary Samples

Out-of-distribution (OOD) detection provides an essential scheme to build a reliable deep learning system. Some of the previous works additionally acquired OOD samples for model training or hyperparameter tuning. However, OOD samples are usually missing in real-world scenarios and the prior knowledge of these samples is commonly unavailable. To cope with these issues, this paper presents a novel approach to synthesizing the informative samples near the boundary between in-distribution and OOD which sufficiently reflect OOD samples in training stage. These synthetic boundary samples are located right outside in-distribution (ID) where the latent variable is distributed by a multivariate Gaussian. Accordingly, this paper presents the outlier-aware representation learning which utilizes the synthetic boundary samples to train an OOD detector by only using the unlabeled ID data. The model is then trained to learn a compact decision boundary between ID and OOD samples. The experiments demonstrate that the proposed methods outperform state-of-the-art performance in presence of different OOD data.
Listen to the Theatre! Exploring Florentine Performative Spaces

A music performance space constitutes the frame as well as the content of the listeners’ experience. The acoustic environment forces continuous negotiations that differ according to a listener’s role and position as conductor, performer or audience member. The aim of this research is to investigate the acoustics of a performative space, the Teatro del Maggio Musicale Fiorentino in Florence, following two complementary paths, both based on an interactive model. The first one offers an impulse-response experience: the user can virtually explore the opera hall by choosing between the binaural reproductions of 13 different listening positions whether off-site (using an audiovisual web app) or on-site, through bone conduction headphones.The second one is about the aural and visual perception of a rehearsal of the romance "Una furtiva lagrima" from Donizetti’s opera L’elisir d’amore. Through the use of ambisonics recordings, 360 degrees videos and a virtual reality headset, the user can experience the performance from three different points of the theatre alternatively: on stage, in the orchestra pit and in the audience area. The three different perspectives can be switched instantaneously using a remote controller.
Decentralized Stochastic Projection-Free Learning with Compressed Push-Sum

We consider decentralized stochastic learning methods with data being distributed among multiple nodes. The nodes communicate gradients with their connected neighbors. To reduce communication overhead, several compression techniques have been proposed for solving unconstrained optimization problems in a decentralized setting. In this paper, we focus on decentralized stochastic learning with convex constraints. Specifically, we propose a novel communication-efficient decentralized stochastic Frank-Wolfe algorithm to solve finite-sum constrained minimization problems by communicating only a compressed version of the gradient. The proposed method guarantees a convergence rate of about $O(n^{-1/2}k^{-1/3})$ for convex objectives, and for non-convex objectives, it guarantees a convergence rate of about $O(n^{-1/2}k^{-2/9})$, where n is the number of nodes in the network. We empirically validate the efficacy of the proposed algorithm in terms of communication overhead and suboptimality gap on several benchmark machine learning tasks.
AdaEE: Adaptive Early-Exit DNN Inference Through Multi-Armed Bandits

Deep Neural Networks (DNNs) are widely used to solve a growing number of tasks, such as image classification. However, their deployment at resource-constrained devices still poses challenges related to energy consumption and delay over-heads. Early-Exit DNNs (EE-DNNs) address the challenges by adding side branches through their architecture. Under an edge-cloud co-inference, if the confidence at a side branch is larger than a fixed confidence threshold, the inference is performed completely at the edge device, saving computation for more difficult observations. Otherwise, the edge device offloads the inference task to the cloud, incurring overhead. Despite its success, EE-DNNs for image classification have to cope with distorted images. The baseline distortion level depends on the environmental context, e.g., time of the day, lighting, and weather conditions. To cope with varying distortion, we propose Adaptive Early-Exit in Deep Neural Networks (AdaEE), a novel algorithm to dynamically adjust the confidence threshold based on context, leveraging the Upper Confidence Bound (UCB) for that matter. AdaEE provably achieves logarithmic regret under mild conditions. We experimentally verify that 1) convergence occurs after collecting a few thousand observations for images with different distortion levels and overhead values, and 2) AdaEE obtains a lower cumulative regret when compared against alternatives using the Caltech-256 dataset subject to varying distortion.
The Distributed Fusion Filtering Problem Of Tessarine Signals From Multisensor Observations Affected With Packet Dropouts

The distributed fusion architecture is used to estimate a tessarine signal from the observations provided by several sensors, assuming that they are affected by packet dropouts. By modeling the uncertainty by means of Bernoulli random variables, an algorithm is derived to calculate recursively the distributed fusion filter. Due to the computational advantages of the $\mathbb{T}_{k}$-properness assumption, the fusion filtering problem has been studied over the reduced-dimensional state-space model under these properness conditions. A numerical simulation example illustrates, on the one hand, the better performance of the distributed filter over the local ones and, on the other, that the processing in the tessarine field yields to better estimations than in the quaternion one, under $\mathbb{T}_{k}$-properness conditions.
Detection of Online Hate Comments Based on Feature Embedding and Deep Learning

Nowadays, anti-social behavior is common on social media. One of these behaviors is d disseminating hate-based postings. Authors of hate speech have targeted specific groups of people based on their identities, such as race, religion, country, ethnicity, and gender. They are more commonly referred to as Anti-black, Anti-Muslim, Anti-Asian, xenophobic, misogynistic, and homophobic. The hate-biased content in social media is noisy, uses emojis and contains grammatical errors, making the word meaning using statistical feature extraction and machine learning difficult. Due to the success of pre-trained models in computer vision, this paper presented the utilization of Global Vector (GLoVe), feature embedding model to capture the global context, specifically for race and religion class from social media data and modeling using two deep learning approaches named Convolutional Neural Network (CNN) and Bidirectional Long Short Term Memory (BiLSTM). The results demonstrate that CNN classifier outperforms BiLSTM in terms of F1 score using employing GloVe 100 and Glove 300 dimensions.
GapReplay: A High-Accuracy Packet Replayer

Network traffic has become increasingly complicated and diverse with the continuing development of the Internet. It is challenging for the traffic generators to construct a large variety of traffic while maintaining the similarity to the real traffic. Given that real traffic from the production network is handy to retrieve, replaying traces from the production network can maximize the authenticity of traffic, contributing to experiments to evaluate the effectiveness of algorithms. Recently, many pieces of research in the networking field, especially algorithms based on machine learning (ML), are extremely sensitive to statistical characteristics of the traffic, such as ML-based traffic classification, traffic detection, traffic prediction, etc. However, there are only a few available statistics for the growing encrypted traffic, such as packet size and inter-arrival time (IAT). To replay traffic accurately, in this paper, we propose the GapReplay, a packet replayer that can remain identical with the original nanosecond-precision pcap trace in packet contents and achieve high accuracy in timestamps. GapReplay reaches line rate while transmitting packets by appending and extending packets, which will be dropped and truncated by a programmable device, to guarantee satisfying accuracy. We implement our mechanism on top of DPDK and P4. The evaluation results demonstrate that GapReplay can achieve a nanosecond-level accuracy, much better than the state-of-the-art such as MoonGen and tcpreplay, where the best of them is at least 1000 times less accurate than our mechanism.
A Novel Darknet Traffic Classification Method Based on Knowledge Graph with Dynamic Embedding Learning

Darknet is described as an individual encrypted part of the Internet that can only be accessed with specific anonymity tools. Achieving accurate classification of darknet traffic is crucial for identifying anonymous network applications and combating cybercrimes. Machine learning-based and deep learning-based classifiers have achieved decent results in darknet traffic classification. However, these methods can not learn global and distinctive darknet flow embedding representations, resulting in limited classification performance. To tackle these issues, we propose Dark-DKGC, a novel darknet traffic classification method based on Knowledge Graph (KG) with Dynamic Knowledge Graph (DKG) embedding learning. Dark-DKGC first constructs Darknet Traffic Dynamic Knowledge Graph (Dark-DKG). Then Dark-DKGC utilizes the DKG embedding method to effectively learn the embedding representations of all flows. Finally, machine learning-based classifiers are trained based on the embedding representations of flows to identify darknet traffic. Our experimental studies suggest that Dark-DKGC can effectively capture distinguishable embedding representations for darknet flows. In multiclass classification scenario, its average accuracy is about 7%-13% higher than state-of-the-art methods and 1% higher than the static KG embedding-based classifier. Besides, compared to the static KG embedding method, Dark-DKGC takes advantage of its online embedding learning to improve test efficiency significantly. Moreover, the visualization of Dark-DKG allows a certain degree of interpretability for the classification results.
PalmEcho: Multimodal Authentication for Smartwatch via Beating Gestures

With the popularity of smartwatches, users can access private information stored in the device by simply touching the watch screen. However, smartwatches also expose users to the risk of information leakage because they lack proper authentication schemes. This paper proposes PalmEcho, a multimodal authentication scheme for smartwatches. The basic idea of PalmEcho is to capture the vibration and the sound generated from user’s beating gestures, and then fuse multimodal signals to extract unique features for user authentication. However, signals of beating gestures are short in the time domain, which makes it hard to extract effective features. To address this challenge, our work reveals that the spectral energy distribution of the generated sound is unique to each user and provides rich information for authentication. Moreover, conventional classification networks require large amounts of user data for training, which is not convenient in the user authentication scenario. To address this challenge, we design a prototypical network called BeatNet, which allows users to register with a few samples. Experimental results show that PalmEcho can reach an average F1-score of 94%.
ATTAA: Active Text Traffic Analysis Attacks on Secure Messaging Applications

Popular Secure Instant Messaging (SIM) applications like Telegram and WhatsApp have deployed state-of-the-art encryption schemes in recent years to protect the security of user communications. However, SIM applications are still not completely secure. Governments can surveil and censor users who participate in groups on sensitive topics based on the leaked information of their SIM clients. In this paper, we find two types of padding flaws in SIM applications where the padding length is not long enough, thereby exposing users' encrypted traffic characteristics. Furthermore, we first present an Active Text Traffic Analysis Attack (ATTAA) that enables the adversary to obtain sensitive information about target users' clients by merely monitoring their encrypted SIM traffic. Specifically, the adversary can quickly identify the participants of target SIM groups with high accuracy. Our study demonstrates a significant, real-world threat to SIM users due to increasing government regulation on social media. We demonstrate the practicality of our ATTAA through extensive experiments on real-world SIM communications. Although SIM applications have various restrictions on message sending, our results show that only ten text messages in 10 seconds are enough to successfully attack Telegram and WhatsApp with an accuracy of 99.94% and 98.66%, and a false positive rate of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$4.3\times 10^{-3}$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1.5\times 10^{-4}$</tex> .
Communication-Efficient Federated Learning with Heterogeneous Devices

The conventional model aggregation-based federated learning (FL) approaches require all local models to have the same architecture and fail to support practical scenarios with heterogeneous local models. Moreover, the frequent model exchange is costly for resource-limited wireless networks since modern deep neural networks usually have over-million parameters. To tackle these challenges, we first propose a novel knowledge-aided FL (KFL) framework, which aggregates light high-level data features, namely knowledge, in the per-round learning process. The KFL allows devices to design their machine learning models independently and reduces the communication overhead in the training process. We then experimentally show that different temporal device scheduling patterns lead to considerably different learning performance. With this insight, we formulate a stochastic optimization problem for joint device scheduling and bandwidth allocation under limited devices' energy budgets and develop an efficient online algorithm to achieve an energy-learning trade-off in the learning process. Experimental results on the CIFAR-10 dataset show that the proposed KFL can reduce over 87% communication overhead while achieving better learning performance than the baselines. In addition, the proposed device scheduling algorithm converges faster than benchmark scheduling schemes.
PACLASS: A Lightweight Classification Framework on DNS-Over-HTTPS

DNS-over-HTTPS is becoming the main protocol of DNS during the development of the Internet. The method of detecting hidden tunnels by using traditional DNS protocols is no longer adaptable. The existing methods of using machine learning have many problems, such as large amount of data and limited applicable data types. To solve the cost problem, we designed a lightweight encrypted DoH traffic classification framework called PACLASS. Different from other studies, PA-CLASS only uses the feature of packet size, does not need the time dimension of data, and supports data sampling and log data. As a result, it requires less computing and storage resources and applies to a wider range type of data. We verify the effectiveness of our framework with six machine learning methods, and evaluate the improvement of the performance of the framework through experiments, such as data sampling ratio, setting the minimum threshold of communication, pre-training model. PACLASS can use less data to train the model. When using the sampled training data, PACLASS can reduce the amount of training data by 69.7% and the F1-score can reach 98.51%.
Boosting Physical Layer Black-Box Attacks with Semantic Adversaries in Semantic Communications

End-to-end semantic communication (ESC) system is able to improve communication efficiency by only transmitting the semantics of the input rather than raw bits. Although promising, ESC has also been shown susceptible to the crafted physical layer adversarial perturbations due to the openness of wireless channels and the sensitivity of neural models. Previous works focus more on the physical layer white-box attacks, while the challenging black-box ones, as more practical adversaries in real-world cases, are still largely under-explored. To this end, we present SemBLK, a novel method that can learn to generate destructive physical layer semantic attacks for an ESC system under the black-box setting, where the adversaries are imperceptible to humans. Specifically, 1) we first introduce a surrogate semantic encoder and train its parameters by exploring a limited number of queries to an existing ESC system. 2) Equipped with such a surrogate encoder, we then propose a novel semantic perturbation generation method to learn to boost the physical layer attacks with semantic adversaries. Experiments on two public datasets show the effectiveness of our proposed SemBLK in attacking the ESC system under the black-box setting. Finally, we provide case studies to visually justify the superiority of our physical layer semantic perturbations.
SynFMPL: A Federated Meta Pseudo Labeling Framework with Synergetic Strategy

Recently, Google proposed a privacy-preserving framework, Federated Learning (FL), to realize the era of edge intelligence. However, the limited labeled data that can only be labeled by the data owner with domain know-how threatens the feasibility of FL. In fact, Semi-supervised Learning (SSL) is a key to leveraging the unlabeled data on the client side while the confirmation bias issue triggered by incorrect predictions in the old-fashion Teacher-Student structure drops the model accuracy significantly. In this paper, we introduce the state-of-the-art SSL technique called Meta Pseudo Labels (MPL), which advocates a feedback strategy in the Teacher-Student architecture to mitigate the performance degradation. Unfortunately, in FL the heterogeneous data without sharing among clients are non-Independent and Identically Distributed (non-IID). The knowledge that shuttles between the Teacher and Student model may be partial and undermine the effectiveness of MPL. To make a performance breakthrough, we proposed a novel FL framework called Federated Meta Learning with Synergetic Strategy (SynFMPL), which combines the advantages of both Teacher and Student models. To our best knowledge, we are the first work that integrates MPL technique in FL. Moreover, we specifically focus on the client side, which fully leverages the unlabeled data for model generalization. The experiment shows that SynFPML outperforms all baselines in various non-IID scenarios. The Student model in SynFMPL even exceeds the upper bound in all baselines by 4% and maintains a virtuous cycle in the learning process.
Federated Cooperative 3D Object Detection for Autonomous Driving

Federated learning has shown great potential in improving the accuracy of models designed for connected autonomous vehicles (CAVs). However, existing approaches only focus on data collected by CAVs, ignoring the valuable insights provided by other types of clients, such as road-side units (RSUs). In this paper, we propose an approach that combines federated learning with cooperative perception to create a more comprehensive and robust global model. Our approach adopts a multi-layered structure that partitions CAVs and RSUs into local clusters. By incorporating the data captured by both CAVs and RSUs, this novel approach can lead to a more accurate and comprehensive global model that reflects the collective knowledge of all agents. We evaluate the proposed approach on the V2X-Set benchmark. The overall average precision of our approach using RSUs reaches 68.62% at Intersection-over-Union threshold of 0.5, significantly outperforming traditional CAV-based federated learning.
rePurpose: A Case for Versatile Network Measurement

Network throughput tests, commonly known as “speed tests” are widely used by consumers, regulators, and ISPs to measure and diagnose network performance. However, the tools used to conduct these tests are often costly in terms of data consumption. Moreover, the speed tests rely on data that is transferred to clients for the sole purpose of measuring throughput with the data being discarded and serving no other purpose. In this paper, we present rePurpose, a system that moves useful content (ads) to enable periodic speed tests by significantly offsetting the cost of network measurement, thereby avoiding harming user QoE. rePurpose can work within the existing ad ecosystem to pre-stage ads needed by users ahead of time. We evaluate the efficacy of rePurpose by emulating a common scenario where users watch videos and ads. Our evaluation shows that rePurpose can reduce the data cost of periodic speed tests by up to 90%. Moreover, by virtue of the time-shifted delivery courtesy of the periodic speed tests moving useful data, rePurpose improves video and ad QoE by reducing or eliminating start-up delay by up to five seconds.
Immersive spatialized live music composition with performers: a case study, Le vent qui hurle

This article presents reflections on theoretical and practical aspects of the composition of immersive spatialized live music with performers through a case study, Le vent qui hurle.This is a piece written for the Ensemble d'oscillateurs founded by Nicolas Bernier at the Faculty of Music of the University of Montreal, semi-modular analog synthesizers, metal sheets and sound spatialization.Initially, I present the research context, discussing the concepts of spatialization and immersion within the frame of live music.I then examine how compositional intentions can be related to the composition of the sound space.I later discuss the relationship between sound and space by introducing spatial attributes and spatialization strategies.Then I illustrate the context in which the piece was created, covering the rehearsal period and the writing of the score.I then present the immersive, spatial and musical composition strategies I used in writing the piece, illustrating techniques that explore the relationship between analog synthesis and the composition of sound space and the relationship between the spatialization created by speakers with the spatialization created with acoustic sources.Finally, I outline future developments.The perspective of this article is mainly oriented toward the compositional dimension and not the technological dimension.
A Comparative Analysis of Speaker and Headphone-Based Immersive Audio in VR and Gaming Applications

Precise and accurate audio spatialization is crucial for gaming and Virtual Reality (VR) applications, as it is an essential aspect of the immersion experienced by users. When the audio of a virtual environment is synchronized with its visual sources and localized accurately, users are more likely to feel deeply engaged and suspend their disbelief. Audio Middleware software is commonly used in cooperation with game engines to meet the audio requirements of virtual environments created by game developers. This research paper presents a comparative analysis of speaker-and headphone-based immersive audio systems within the context of VR. The research findings indicate that, despite the headphone-based system offering more accurate audio localization and a higher ‘perceived audio quality’, the speaker-based system resulted in a greater sense of ‘immersion.’