Reviewerly/exHarmony huggingface dataset:
Semantic-Discriminative Mixup for Generalizable Sensor-based Cross-domain Activity Recognition

It is expensive and time-consuming to collect sufficient labeled data to build human activity recognition (HAR) models. Training on existing data often makes the model biased towards the distribution of the training data, thus the model might perform terribly on test data with different distributions. Although existing efforts on transfer learning and domain adaptation try to solve the above problem, they still need access to unlabeled data on the target domain, which may not be possible in real scenarios. Few works pay attention to training a model that can generalize well to unseen target domains for HAR. In this paper, we propose a novel method called Semantic-Discriminative Mixup (SDMix) for generalizable cross-domain HAR. Firstly, we introduce semantic-aware Mixup that considers the activity semantic ranges to overcome the semantic inconsistency brought by domain differences. Secondly, we introduce the large margin loss to enhance the discrimination of Mixup to prevent misclassification brought by noisy virtual labels. Comprehensive generalization experiments on five public datasets demonstrate that our SDMix substantially outperforms the state-of-the-art approaches with 6% average accuracy improvement on cross-person, cross-dataset, and cross-position HAR.
Designing Guided User Tasks in VR Embodied Experiences

Virtual reality (VR) offers extraordinary opportunities in user behavior research to study and observe how people interact in immersive 3D environments. A major challenge of designing these 3D experiences and user tasks, however, lies in bridging the inter-relational gaps of perception between the designer, the user, and the 3D scene. Paul Dourish identified three gaps of perception: ontology between the scene representation and the user and designer interpretation, intersubjectivity of task communication between designer and user, and intentionality between the user's intentions and designer's interpretations. We present the GUsT-3D framework for designing Guided User Tasks in embodied VR experiences, i.e., tasks that require the user to carry out a series of interactions guided by the constraints of the 3D scene. GUsT-3D is implemented as a set of tools that support a 4-step workflow to (1) annotate entities in the scene with navigation and interaction possibilities, (2) define user tasks with interactive and timing constraints, (3) manage interactions, task validation, and user logging in real-time, and (4) conduct post-scenario analysis through spatio-temporal queries using ontology definitions. To illustrate the diverse possibilities enabled by our framework, we present two case studies with an indoor scene and an outdoor scene, and conducted a formative evaluation involving six expert interviews to assess the framework and the implemented workflow. Analysis of the responses show that the GUsT-3D framework fits well into a designer's creative process, providing a necessary workflow to create, manage, and understand VR embodied experiences.
k-Means Clustering with Optimal Centroid: An Optimization Insisted Model for Removing Outliers

In data cleaning, the process of detecting and correcting corrupt, inaccurate or irrelevant records from the record set is a tedious task. Particularly, the process of “outlier detection” occupies a significant role in data cleaning that removes or eliminates the outlier’s that exist in data. Traditionally, more efforts have been taken to remove the outliers, and one of the promising ways is customizing clustering models. In this manner, this paper intends to propose a new outlier detection model via enhanced k-means with outlier removal (E-KMOR), which assigns all outliers into a group naturally during the clustering process. For assigning the point to be outliers, a new intra-cluster based distance evaluation is employed. The main contribution of this paper is to select cluster centroid optimally through a newly proposed hybrid optimization algorithm termed particle updated lion algorithm (PU-LA), which hybrids the concepts of LA and particle swarm optimization (PSO), respectively. Thereby, the proposed work is named as E-KMOR-PU-LA. Finally, the efficacy of the proposed E-KMOR-PU-LA model is proved through a comparative analysis over conventional models by concerning runtime and accuracy.
Hyperparameter Importance of Quantum Neural Networks Across Small Datasets

As restricted quantum computers are slowly becoming a reality, the search for meaningful first applications intensifies. In this domain, one of the more investigated approaches is the use of a special type of quantum circuit – a so-called quantum neural network – to serve as a basis for a machine learning model. Roughly speaking, as the name suggests, a quantum neural network can play a similar role to a neural network. However, specifically for applications in machine learning contexts, very little is known about suitable circuit architectures, or model hyperparameters one should use to achieve good learning performance. In this work, we apply the functional ANOVA framework to quantum neural networks to analyze which of the hyperparameters were most influential for their predictive performance. We analyze one of the most typically used quantum neural network architectures. We then apply this to 7 open-source datasets from the OpenML-CC18 classification benchmark whose number of features is small enough to fit on quantum hardware with less than 20 qubits. Three main levels of importance were detected from the ranking of hyperparameters obtained with functional ANOVA. Our experiment both confirmed expected patterns and revealed new insights. For instance, setting well the learning rate is deemed the most critical hyperparameter in terms of marginal contribution on all datasets, whereas the particular choice of entangling gates used is considered the least important except on one dataset. This work introduces new methodologies to study quantum machine learning models and provides new insights toward quantum model selection.
Modeling and generating synthetic anomalies for energy and power time series

With the development of the smart grid, the number of recorded energy and power times series increases noticeably. This increase allows for the automation of smart grid applications such as load forecasting and load management. This automation, however, requires data that only represents the typical behavior of the system. To ensure that such data is available, detecting the anomalies often present in recorded data is important. As a result, anomaly detection methods are a recent research topic. However, their development is often limited by undefined anomaly characteristics and a lack of labeled anomalous data. To overcome this challenge, we propose a method that generates synthetic anomalies based on real-world anomalies that can be inserted into energy and power time series. For this, we analyze real energy and power time series to identify four types of commonly occurring anomalies. Given the identified anomaly types, we formally model each type and use these models to insert synthetic anomalies of each type into arbitrary energy or power time series. We show that our method is not only capable of generating synthetic anomalies with real-world properties, but also beneficial for training supervised anomaly detection methods.
Cache Allocation in Multi-Tenant Edge Computing via online Reinforcement Learning

We consider in this work Edge Computing (EC) in a multi-tenant environment: the resource owner, i.e., the Network Operator (NO), virtualizes the resources and lets third party Service Providers (SPs - tenants) run their services, which can be diverse and with heterogeneous requirements. Due to confidentiality guarantees, the NO cannot observe the nature of the traffic of SPs, which is encrypted. This makes resource allocation decisions challenging, since they must be taken based solely on observed monitoring information.We focus on one specific resource, i.e., cache space, deployed in some edge node, e.g., a base station. We study the decision of the NO about how to partition cache among several SPs in order to minimize the upstream traffic. Our goal is to optimize cache allocation using purely data-driven, model-free Reinforcement Learning (RL). Differently from most applications of RL, in which the decision policy is learned offline on a simulator, we assume no previous knowledge is available to build such a simulator. We thus apply RL in an online fashion, i.e., the policy is learned by directly perturbing the actual system and monitoring how its performance changes. Since perturbations generate spurious traffic, we also limit them. We show in simulation that our method rapidly converges toward the theoretical optimum, we study its fairness, its sensitivity to several scenario characteristics and compare it with a method from the state-of-the-art. Our code to reproduce the results is available as open source. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Flexible Order Aware Sequential Recommendation

Sequential recommendations can dynamically model user interests, which has great value since users' interests may change rapidly with time. Traditional sequential recommendation methods assume that the user behaviors are rigidly ordered and sequentially dependent. However, some user behaviors have flexible orders, meaning the behaviors may occur in any order and are not sequentially dependent. Therefore, traditional methods may capture inaccurate user interests based on wrong dependencies. Motivated by this, several methods identify flexible orders by continuity or similarity. However, these methods fail to comprehensively understand the nature of flexible orders since continuity or similarity do not determine order flexibilities. Therefore, these methods may misidentify flexible orders, leading to inappropriate recommendations. To address these issues, we propose a Flexible Order aware Sequential Recommendation (FOSR) method to identify flexible orders comprehensively. We argue that orders' flexibilities are highly related to the frequencies of item pair co-occurrences. In light of this, FOSR employs a probabilistic based flexible order evaluation module to simulate item pair frequencies and infer accurate order flexibilities. The frequency labeling module extracts labels from the real item pair frequencies to guide the order flexibility measurement. Given the measured order flexibilities, we develop a flexible order aware self-attention module to model dependencies from flexible orders comprehensively and learn dynamic user interests effectively. Extensive experiments on four benchmark datasets show that our model outperforms various state-of-the-art sequential recommendation methods.
Accelerometer-based elderly fall detection system using edge artificial intelligence architecture

Falls have long been one of the most serious threats to elderly people's health. Detecting falls in real-time can reduce the time the elderly remains on the floor after a fall, hence avoiding fall-related medical conditions. Recently, the fall detection problem has been extensively researched. However, the fall detection systems that use a traditional internet of things (IoT) architecture have some limitations such as latency, high power consumption, and poor performance in areas with unstable internet. This paper intends to show the efficacy of detecting falls in a resource-constrained microcontroller at the edge of the network using a wearable accelerometer. Since the hardware resources of microcontrollers are limited, a lightweight fall detection deep learning model was developed to be deployed on a microcontroller with only a few kilobytes of memory. The microcontroller was installed in a low-power wide-area network based on long range (LoRa) communication technology. Through comparative testing of different lightweight neural networks and traditional machine learning algorithms, the convolutional neural network (CNN) has been shown to be the most suited, with 95.55% accuracy. The CNN model reached inference times lower than 37.84 ms with 61.084 kilobytes storage requirements, which implies the capability to detect fall event in real-time in low-power microcontrollers.
A Comparison Between Single- and Multi-Scale Approaches for Classification of Histopathology Images

Whole slide images (WSIs) are digitized histopathology images. WSIs are stored in a pyramidal data structure that contains the same images at multiple magnification levels. In digital pathology, most algorithmic approaches to analyze WSIs use a single magnification level. However, images at different magnification levels may reveal relevant and distinct properties in the image, such as global context or detailed spatial arrangement. Given their high resolution, WSIs cannot be processed as a whole and are broken down into smaller pieces called tiles. Then, a prediction at the tile-level is made for each tile in the larger image. As many classification problems require a prediction at a slide-level, there exist common strategies to integrate the tile-level insights into a slide-level prediction. We explore two approaches to tackle this problem, namely a multiple instance learning framework and a representation learning algorithm (the so-called “barcode approach”) based on clustering. In this work, we apply both approaches in a single- and multi-scale setting and compare the results in a multi-label histopathology classification task to show the promises and pitfalls of multi-scale analysis. Our work shows a consistent improvement in performance of the multi-scale models over single-scale ones. Using multiple instance learning and the barcode approach we achieved a 0.06 and 0.06 improvement in F1 score, respectively, highlighting the importance of combining multiple scales to integrate contextual and detailed information.
EtinyNet: Extremely Tiny Network for TinyML

There are many AI applications in high-income countries because their implementation depends on expensive GPU cards (~2000$) and reliable power supply (~200W). To deploy AI in resource-poor settings on cheaper (~20$) and low-power devices (
Efficient Compact Bilinear Pooling via Kronecker Product

Bilinear pooling has achieved excellent performance in fine-grained recognition tasks. Nevertheless, high-dimensional bilinear features suffer from over-fitting and inefficiency. To alleviate these issues, compact bilinear pooling (CBP) methods were developed to generate low-dimensional features. Although the low-dimensional features from existing CBP methods enable high efficiency in subsequent classification, CBP methods themselves are inefficient. Thus, the inefficiency issue of the bilinear pooling is still unsolved. In this work, we propose an efficient compact bilinear pooling method to solve the inefficiency problem inherited in bilinear pooling thoroughly. It decomposes the huge-scale projection matrix into a two-level Kronecker product of several small-scale matrices. By exploiting the ``vec trick'' and the tensor modal product, we can obtain the compact bilinear feature through the decomposed projection matrices in a speedy manner. Systematic experiments on four public benchmarks using two backbones demonstrate the efficiency and effectiveness of the proposed method in fine-grained recognition.
ReMoNet: Recurrent Multi-Output Network for Efficient Video Denoising

While deep neural network-based video denoising methods have achieved promising results, it is still hard to deploy them on mobile devices due to their high computational cost and memory demands. This paper aims to develop a lightweight deep video denoising method that is friendly to resource-constrained mobile devices. Inspired by the facts that 1) consecutive video frames usually contain redundant temporal coherency, and 2) neural networks are usually over-parameterized, we propose a multi-input multi-output (MIMO) paradigm to process consecutive video frames within one-forward-pass. The basic idea is concretized to a novel architecture termed Recurrent Multi-output Network (ReMoNet), which consists of recurrent temporal fusion and temporal aggregation blocks and is further reinforced by similarity-based mutual distillation. We conduct extensive experiments on NVIDIA GPU and Qualcomm Snapdragon 888 mobile platform with Gaussian noise and simulated Image-Signal-Processor (ISP) noise. The experimental results show that ReMoNet is both effective and efficient on video denoising. Moreover, we show that ReMoNet is more robust under higher noise level scenarios.
Animating Images to Transfer CLIP for Video-Text Retrieval

Recent works show the possibility of transferring the CLIP (Contrastive Language-Image Pretraining) model for video-text retrieval with promising performance. However, due to the domain gap between static images and videos, CLIP-based video-text retrieval models with interaction-based matching perform far worse than models with representation-based matching. In this paper, we propose a novel image animation strategy to transfer the image-text CLIP model to video-text retrieval effectively. By imitating the video shooting components, we convert widely used image-language corpus to synthesized video-text data for pretraining. To reduce the time complexity of interaction matching, we further propose a coarse to fine framework which consists of dual encoders for fast candidates searching and a cross-modality interaction module for fine-grained re-ranking. The coarse to fine framework with the synthesized video-text pretraining provides significant gains in retrieval accuracy while preserving efficiency. Comprehensive experiments conducted on MSR-VTT, MSVD, and VATEX datasets demonstrate the effectiveness of our approach.
A corpus-based approach to author’s idiolect study: lexicological aspect

The aim of this paper is to present the application of a multimodal approach to the analysis of the writer’s lexicon. The versatility of the anthropocentric paradigm of modern linguistics and the approaches applied in our research are mindful of the choice of relevant research methods. The features of a writer’s lexicon (or idiolect in a broader sense) have been of great interest among scholars though they have been mostly explored by literary critics. Nowadays, more linguists are also interested in thorough research on the subject by applying more rigorous scientific methods like quantitative and statistical ones. The paper herewith focuses on a few of them as a preliminary attempt to showcase some results. The primary concern is to make this study corpus-based. Hence it is crucial to create a corpus of works based on the generally acknowledged rules and principles. Therefore, such a corpus may contribute to the development of corpus linguistics and be applicable to other research regarding studying the writer’s idiolect. Another point of interest is that such research may either support or somehow disprove established views regarding the writing style of a certain author (that is very useful when considering author attribution issues or even forensic linguistics, though that is not the case of the given research). Finally, lexical semantics plays one of the leading roles in defining dominant concepts of an author’s writing style corresponding to the cognitive linguistics domain. As a result of such an approach, it may help to construct the lexicon of the author into an ideographic entity. Furthermore, similar findings or comparative analysis of corresponding objects of study may contribute to displaying the author’s outlook and subsequently that of the whole nation.
The HW-TSC’s Offline Speech Translation System for IWSLT 2022 Evaluation

This paper describes the HW-TSC's designation of the Offline Speech Translation System submitted for IWSLT 2022 Evaluation.We explored both cascade and end-to-end system on three language tracks (en-de, en-zh and en-ja), and we chose the cascade one as our primary submission.For the automatic speech recognition (ASR) model of cascade system, there are three ASR models including Conformer, S2T-Transformer and U2 trained on the mixture of five datasets.During inference, transcripts are generated with the help of domain controlled generation strategy.Context-aware reranking and ensemble based robustness enhancement strategy are proposed to produce better ASR outputs.For machine translation part, we pretrained three translation models on WMT21 dataset and fine-tuned them on indomain corpora.Our cascade system shows more competitive performance than the known offline systems in the industry and academia.
Double FCOS: A Two-Stage Model Utilizing FCOS for Vehicle Detection in Various Remote Sensing Scenes

Vehicle detection in various remote sensing scenes is a challenging task. Various remote sensing scenes are mixed up with images of multi-scene, multi-quality, multi-scale and multi-class. Vehicle detection models suffer from inadequate candidate boxes, weak positive proposal sampling and poor classification performance, resulting in a detection performance degradation when they are applied in various scenes. What is worse, there is no such a dataset covering various scenes which is for vehicle detection. This paper proposes a vehicle detection model called Double FCOS and a vehicle dataset called 4MVD for vehicle detection in various remote sensing scenes. Double FCOS is a two-stage detection model based on fully convolution one-stage object detection (FCOS). FCOS is exploited in the RPN stage to generate candidate boxes in various scenes. The two-stage positive and negative sample model (TPNSM) is carefully designed to enhance the positive proposal sampling effects, particularly the tiny or weak vehicles, which are ignored in FCOS. A two-step classification model (TSCM) has been designed in the RCNN stage with a proposal classification branch and point classification branch to enhance the classification performance between the various types of vehicles. 4MVD (multi-scene, multi-quality, multi-scale and multi-class vehicle dataset) is collected from various remote sensing scenes to evaluate the performance of Double FCOS. A mean average accuracy of 78.3 percentage for vehicle detection on 5 categories has been received by Double FCOS on 4MVD. Extensive experiments demonstrate that Double FCOS significantly improves the performance of vehicle detection in various remote sensing scenes.
Stylistic Response Generation by Controlling Personality Traits and Intent

Personality traits influence human actions and thoughts, which is manifested in day to day conversations.Although glimpses of personality traits are observable in existing open domain conversation corpora, leveraging generic language modelling for response generation overlooks the interlocutor idiosyncrasies, resulting in non-customizable personality agnostic responses.With the motivation of enabling stylistically configurable response generators, in this paper we experiment with end-to-end mechanisms to ground neural response generators based on both (i) interlocutor Big-5 personality traits, and (ii) discourse intent as stylistic control codes.Since most of the existing large scale open domain chat corpora do not include Big-5 personality traits and discourse intent, we employ automatic annotation schemes to enrich the corpora with noisy estimates of personality and intent annotations, and further assess the impact of using such features as control codes for response generation using automatic evaluation metrics, ablation studies and human judgement.Our experiments illustrate the effectiveness of this strategy resulting in improvements to existing benchmarks.Additionally, we yield two silver standard annotated corpora with intents and personality traits annotated, which can be of use to the research community.