Reviewerly/exHarmony huggingface dataset:
A Non-Targeted Attack Approach for the Coarse Misclassification Problem

The evaluation of classifiers' robustness against adversarial attacks is typically performed through metrics based on the minimal perturbation required for misclassification. The conventional method of generating these perturbations relies on setting a limit on the maximum allowed perturbation (restricted attack method) and a non-targeted attack formulation. This approach, however, disregards any relationships between classes. Our paper introduces a novel, non-targeted, bound-restricted method for achieving coarse misclassification, so that the perturbed feature is classified outside its true coarse class. We present an efficient, single-step solution to the coarse misclassification problem and analyze its computational requirements. Our experiments showcase the superiority of our method, surpassing state-of-the-art in terms of both the perceptibility of adversarial examples and runtime.
Measuring the Impact of Domain Factors in Self-Supervised Pre-Training

Human speech data comprises a rich set of domain factors such as accent, syntactic and semantic variety, or acoustic environment. Previous work explores the effect of domain mismatch in automatic speech recognition between pre-training and fine-tuning as a whole [1] but does not dissect the contribution of individual factors. In this paper, we present a controlled study to better understand the effect of such factors on the performance of pre-trained representations on automatic speech recognition. To do so, we pre-train models either on modified natural speech or synthesized audio, with a single domain factor modified, and then measure performance after fine-tuning. Results show that phonetic domain factors play an important role during pre-training while grammatical and syntactic factors are far less important. To our knowledge, this is the first study to better understand the domain characteristics of pre-trained sets in self-supervised pre-training for speech.
You Can’t Always Check What You Wanted: : Selective Checking and Trusted Execution to Prevent False Actuations in Real-Time Internet-of-Things

Modern Internet-of-Things devices are vulnerable to attacks targeting outgoing actuation commands that modify their physical behaviors. We present a “selective checking” mechanism that uses game-theoretic modeling to identify the suitable subset of commands to be checked in order to deter an adversary. This mechanism is coupled with a “delay-aware” trusted execution environment to ensure that only verified actuation commands are ever sent to the physical system, thus maintaining the safety and integrity of the system. Our proposed selective checking and trusted execution (SCATE) framework is implemented on an off-the-shelf ARM platform running embedded Linux and tested on four realistic IoT-specific cyber-physical systems (a ground rover, a flight controller, a robotic arm and an automated syringe pump).
A Novel Framework for Multi-Source Domain Adaptation with Discriminative Feature Learning

Unsupervised domain adaptation deals with the scenario where there is scarcity of labeled data for the training of machine learning models. Traditionally, the knowledge of a learner is transferred from a source domain to a target domain in a conventional setting. In this paper, we propose a novel framework for Multi-Source Domain Adaptation which can train the model using multiple source domains and can generalize well on a target dataset. It mitigates the domain shift caused by different domains by (1) coordinating the statistical moments for all combinations of the source and target domain distributions and (2) by collaboratively extracting the deep features from the shared representation space. Extensive experiments have been performed for the task of image classification using different datasets widely varying in size and complexity in order to demonstrate the effectiveness of our approach compared to the naive source-combined domain adaptation setting.
Entity Linking based on RoFormer-Sim for Chinese Short Texts

Entity linking is an important means to identify named entities in text and a key technology for constructing knowledge graphs, playing an important role in fields such as intelligent question answering and information retrieval. However, existing entity linking methods for short texts have low accuracy due to the lack of rich contextual information, informal expression, and incomplete grammar structures. Therefore, this paper proposes a short-text entity linking model based on the RoFormer-Sim pre-training model. Firstly, entity context features are extracted by the RoFormer-Sim pre-training model, and then text similarity calculation and sorting are performed with candidate entity description texts to obtain the corresponding entity in the knowledge base with the disambiguated entity. The experimental results show that the RoFormer-Sim model can provide prior knowledge for entity linking, and the proposed model in this paper has an F1 value of 0.8851, which is better than other entity linking models based on other pre-training models.
Efficient Moving Target Detection Using Resource-Constrained Neural Networks

In recent years, the widespread use of autonomous vehicles, such as aerial and automotive, has enhanced our abilities to perform target tracking, dispensing our over-reliance on visual features. With the development of computer vision and deep learning techniques, vision-based classification and recognition have recently received special attention in the scientific community. Moreover, recent advances in the field of neural networks with quantized weights and activations down to single bit precision have allowed the development of models that can be deployed in resource-constrained settings, where a trade-off between task performance and efficiency is accepted. In this work we design an efficient single stage object detector based on CenterNet containing a combination of full precision and binary layers. Our model is easy to train and achieves comparable results with a full precision network trained from scratch while requiring an order of magnitude less FLOP. This opens the possibility of deploying an object detector in applications where time is of the essence and a graphical processing unit (GPU) is absent. We train our model and evaluate its performance by comparing with state-of-the-art techniques, obtaining higher accurate results and provide an insight into the design process of resource constrained neural networks involving trade-offs.
Supervised Image Segmentation for High Dynamic Range Imaging

Regular cameras and cell phones are able to capture limited luminosity. In terms of quality, most of the produced images by such devices are not similar to the real world. Various methods, which fall under the name of High Dynamic Range (HDR) Imaging, can be utilised to cope with this problem and produce an image with more details. However, most methods for generating an HDR image from Multi-Exposure images only focus on how to combine different exposures and do not consider the choice the best details of each image. By convers, in this research it is strived to detect the most visible areas of each image with the help of image segmentation. Two methods of producing the Ground Truth are considered, as manual and Otsu thresholding, and two similar neural networks are used to train segment these areas. Finally, it is shown that the neural network is able to segment the visible parts of pictures acceptably.
Explaining Cyberbullying Trait Detection Through High Accuracy Transformer Ensemble

Cyberbullying remains a pernicious concern within social media which can have an outsized impact on teens and children who are spending more and more of their social engagements online. The recent COVID-19 pandemic physically isolated young students during school and other public closures which served to increase usage of social media. These events heighten the need for careful, explainable, and ethical tools for monitoring and identifying cyberbullying with explaining classification to specific traits. Classical machine learning methods which form a large part of the existing domain provide better introspection into how the ML system arrives at its inference. These traditional stochastic based methodologies are hampered by the complex task of feature engineering and generally lag behind newer deep neural network (DNN) architectures in accuracy. The tradeoff is that DNNs are often opaque with respect to explainability of the model, the inference, and the impact of vast training sets toward ethics and bias. In this work, we propose an advanced architecture that incorporates second and third generation transformer-based models to provide focused and highly accurate cyberbullying trait detection and identification allowing for human verification and validation based on these inferences. This architecture uses the IEEE Fine-Grained Cyberbullying Dataset (FGCD) exceeding current SOA.
Financial Time Series Data Prediction by Combination Model Adaboost-KNN-LSTM

Financial time series is the main research direction of financial researchers due to its important economic value. However, owing to stochastic nature of financial markets, the data used by financial time series models need to have a series of features. This limits the use of financial forecasting models. A model for small datasets is proposed to address the above limitations. This model combines Adaptive Boosting(AdaBoost), K-Nearest Neighbors(KNN) and Long Short-Term Memory(LSTM) to accurately predict financial time series data by using a small number of features and data. Compared with classical machine learning models, the proposed model has the best prediction accuracy.
MIRec: Neural News Recommendation with Multi-Interest and Popularity-Aware Modeling

News recommendation is critical for online news services. How to precisely match news content with users’ interests lies in the core of personalized news recommendation. Existing methods mainly learn a unified embedding vector for each user to represent his/her interests. However, users’ diverse interests can not be expressed adequately by a single embedding representation because of the lack of expressiveness. Additionally, incorporating news popularity into news recommendation can effectively improve accuracy since users with different interests are drawn to current popular news. In this paper, we propose a news recommendation method with multi-interest and popularity-aware modeling, named MIRec. We propose a novel news encoder with attentive learning to obtain unified representations of clicked news from the content and popularity. Furthermore, we exploit a popularity-aware multi-interest extractor to generate multi-interest representations of users and eliminate the bias of news popularity in preference modeling. Besides, we design a popularity predictor for candidate news, which measures its popularity based on the popularity features, recency, and interaction rate. Finally, we adopt a gated mechanism based on user multi-interest representation and the popularity of candidate news to make recommendations. Extensive experiments on large-scale benchmark dataset demonstrate that our approach significantly outperforms existing state-of-the-art methods.
MTSNet: Deep Probabilistic Cross-multivariate Time Series Modeling with External Factors for COVID-19

Complex intelligent systems such as for tackling the COVID-19 pandemic involve multiple multivariate time series (MTSs), where both target variables (such as COVID-19 infected, confirmed, and recovered cases) and external factors (such as virus mutation and infectivity, vaccination, and government intervention influence) are coupled. Forecasting such MTSs with multiple external MTS factors needs to model both within and between MTS interactions and handle their uncertainty, heterogeneity, and dynamics. Existing shallow to deep MTS modelers, including regressors, deep recurrent neural networks such as DeepAR, deep state space models, and deep factor models, do not jointly characterize these issues in a probabilistic manner across MTSs. We propose an end-to-end deep probabilistic cross-MTS learning network MTSNet. MTSNet incorporates a tensor input with scaled target and external MTSs. It then vertically and horizontally stacks long-short memory networks for encoding and decoding target MTSs and enhances uncertainty modeling, generalization and forecasting robustness by residual connection, variational zoneout, and probabilistic forecasting. The tensor input is projected to a probability distribution for target MTS forecasting. MTSNet outperforms the SOTA deep probabilistic MTS networks in forecasting COVID-19 confirmed cases and ICU patient numbers for six countries by involving virus mutation, vaccination, government interventions, and infectivity.
Color Deconvolution Applied to Domain Adaptation in HER2 Histopathological Images

Breast cancer early detection is crucial for improving patient outcomes. The Institut Catala de la Salut (ICS) has launched‘ the DigiPatICS project to develop and implement artificial intelligence algorithms to assist with the diagnosis of cancer. In this paper, we propose a new approach for facing the color normalization problem in HER2-stained histopathological images of breast cancer tissue, posed as an style transfer problem. We combine the Color Deconvolution technique with the Pix2Pix GAN network to present a novel approach to correct the color variations between different HER2 stain brands. Our approach focuses on maintaining the HER2 score of the cells in the transformed images, which is crucial for the HER2 analysis. Results demonstrate that our final model outperforms the state-of-the-art image style transfer methods in maintaining the cell classes in the transformed images and is as effective as them in generating realistic images.
Symbiotic Artificial Intelligence: Order Picking And Ambient Sensing

Using egocentric video and head motion data from 67 order picking tasks (244 picks;149 orders), we learn visual models of the 10 objects picked to fulfill the orders. Boundary segmentations of the four actions (pick, carry, place, carry empty) of order picking had an average test RMSE of 1.11 seconds using computer vision and 5.53 seconds using only head motion $( \approx 39.8$ seconds/task). The 10 objects were clustered with 93.8% accuracy using weak supervision provided by the picks (which could occur in any order) specified in the tasks. We apply the 10 resulting models on independent test data to recognize three objects involving 50 tasks (185 picks;98 orders) and 10 objects involving 10 tasks (35 picks;24 orders). Accuracy was up to 90.3% and 69.1%, respectively. We propose order picking as a practical use case of egocentric Symbiotic AI, where ambient sensing is used without explicit supervision to train an agent which can then help the user improve task speed and accuracy. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Demystifying Complex Treatment Recommendations: A Hierarchical Cooperative Multi-Agent RL Approach

Reinforcement Learning (RL) techniques are widely adopted in healthcare research to learn effective policies for complex sequential decision-making tasks, such as treatment recommendation to achieve better clinical outcomes. However, existing RL based solutions suffer from two major limitations. (1) Burdens a single agent to make complex treatment recommendations: existing RL algorithms do not explicitly support the common hierarchical decision-making processes used in practise. That is, they consider treatment recommendation as a singular decision and the treatment space as a flat search space, and expect a single RL agent to select the best treatment outright. (2) Sub optimal patient state representation techniques: accurate representation of patient states and their dynamics is essential to make effective clinical decisions. However, there are three main limitations in current state representation techniques in the clinical environment: (i) lack of clear structure and high variability of input features; (ii) lack of a temporal context, which makes it difficult to fully capture patient conditions and their dynamics at a given time; (iii) and the use of imputation techniques, which undermines personalization, introduce biases and errors that can cause harm to patients. To address the complexities of treatment recommendation and generate superior treatment policies, we propose a consolidated RL solution that incorporates a hierarchical multi-agent architecture and advanced patient state representation technique. In our proposed architecture, the complex task is decomposed into a hierarchy of subtasks, each handled by a specialized agent that coordinates skills learned by lower level agents to determine a high-level treatment policy. The patient state representation technique is designed to effectively encode the changing states of patients throughout their hospital stay, addressing the three primary limitations of current approaches. Our experiments using the sepsis treatment task shows that the proposed solution consistently outperforms state-of-the-art baselines and improves patient survival.
Online Continual Learning for Control of Mobile Robots

This work presents a novel approach which integrates deep learning, online learning and continual learning paradigms for adaptive control for robotic systems. Deep learning allows generalising knowledge about the robot, while online learning can adapt to variable operating conditions, and continual learning enables remembering previous knowledge. The proposed method approximates the inverse dynamics of the robot, which is formulated as a regression problem. With a minimum knowledge of the robot's dynamics, the proposed method shows its capability to reduce tracking errors online by continuously learning and compensating for internal and external changing conditions. Furthermore, the simulation results show that the proposed approach with online continual learning improves the control performance of ground and aerial mobile robots.
Domain Translation via Latent Space Mapping

In this paper, we study the problem of multi-domain translation: given an element (a) of domain A, we wish to generate a corresponding element (b) in another domain B, and vice versa. Acquiring supervision in multiple domains can be a tedious task, also we propose to learn this translation from one domain to another when supervision is available as a pair (a, b), and leverage possible unpaired data when only (a) or only (b) is available. We introduce a new unified framework called Latent Space Mapping (LSM), which exploits the manifold assumption to learn a latent space from each domain. Unlike existing approaches, we propose to further regularize each latent space using available domains by learning each dependency between pairs of domains. We evaluate our approach on three tasks performing i) a synthetic dataset with image translation, ii) a real-world task of semantic segmentation for medical images, and iii) a real-world task of facial landmark detection.
End to End Sign Language Translation via Multitask Learning

Sign language translation (SLT) is usually seen as a two-step process of continuous sign language recognition (CSLR) and gloss-to-text translation. We propose a novel, Transformer-based architecture to jointly perform CSLR and sign-translation in an end-to-end fashion. We extend the ordinary Transformer decoder with two channels to support multitasking, where each channel is devoted to solving a particular problem. To control the memory footprint of our model, channels are designed to share most of their parameters with each other. However, each channel still has a dedicated set of parameters that is fine-tuned with respect to the channel's task. In order to evaluate the proposed architecture, we focus on translating German signs into English sequences and use the RWTH-PHOENIX-Weather 2014 T corpus in our experiments. Evaluation results along with detailed quantitative and qualitative analyses indicate that the mixture of information provided by the multitask decoder was successful and enabled us to achieve superior performance in comparison to other SLT models.
RulePG: Syntactic Rule-enhanced Paraphrase Generation

The deep neural network (DNN) has become prevailing for paraphrase generation (PG), a widely studied task in natural language processing. Even though the DNN models have shown impressive performances, they are still far from generating desired paraphrases that require sufficient diversity in wording while keeping the correct semantics. Meanwhile, the large DNN models suffer from significant cost of training, the difficulty of reproducibility, and the black-box training process, and thus make it hard to generalize and implement a novel model in other works. Therefore, we propose a simple but effective alternative: a post-hoc method called RulePG, which is based on syntactic rules for PG. It has the advantages of improving wording diversities while preserving semantics, and is designed post-hoc manner so that it is model-agnostic and would not require extra training when utilized. Moreover, instructed by the rules, the generation process is more interpretable with explicit control of sentence forms. To gain and make full use of rules, we construct SRPG (syntactic rules for paraphrase generation), the first syntactic rule set for PG. We novelly build semantic representations for SRPG and endow the RulePG with the capacity of semantic understanding. Automatic evaluation on various metrics and human evaluations demonstrate the effectiveness of our method and show that the RulePG reaches new state-of-the-art performances.
Zero-Shot Text-to-Speech Synthesis Conditioned Using Self-Supervised Speech Representation Model

This paper proposes a zero-shot text-to-speech (TTS) conditioned by a self-supervised speech-representation model acquired through self-supervised learning (SSL). Conventional methods with embedding vectors from x-vector or global style tokens still have a gap in reproducing the speaker characteristics of unseen speakers. A novel point of the proposed method is the direct use of the SSL model to obtain embedding vectors from speech representations trained with a large amount of data. We also introduce the separate conditioning of acoustic features and a phoneme duration predictor to obtain the disentangled embeddings between rhythm-based speaker characteristics and acoustic-feature-based ones. The disentangled embeddings will enable us to achieve better reproduction performance for unseen speakers and rhythm transfer conditioned by different speeches. Objective and subjective evaluations showed that the proposed method can synthesize speech with improved similarity and achieve speech-rhythm transfer.
Glove-Ing Attention: A Multi-Modal Neural Learning Approach to Image Captioning

Articulating pictures using natural language is a complex undertaking within the realm of computer vision. The process of generating image captions involves producing depictions of images which can be achieved through advanced learning frameworks utilizing convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Yet, conventional RNNs face challenges such as gradient explosion and vanishing gradients, resulting in inferior outcomes when producing non-evocative sentences. In this paper, we proposed an encoder-decoder deep neural network to generate image captions using state-of-the-art backbone architecture EfficientNet as the encoder network. We used multimodal gated recurrent units (GrU) for the decoder, which incorporate GloVe word embeddings for the text data and visual attention for the image data. The network is trained on three different datasets, Indiana Chest X-ray, COCO and WIT, and the results are evaluated on the standard performance metrics of BLEU and METEOR. The quantitative results show that the network achieves promising results compared to the state-of-the-art models. The source code is publically available at https://bitbucket.org/larswise/imagecaptioning/src/master/wit_pipeline/.