Reviewerly/exHarmony huggingface dataset:
Bilingual End-to-End ASR with Byte-Level Subwords

In this paper, we investigate how the output representation of an end-to-end neural network affects multilingual automatic speech recognition (ASR). We study different representations including character-level, byte-level, byte pair encoding (BPE), and byte-level byte pair encoding (BBPE) representations, and analyze their strengths and weaknesses. We focus on developing a single end-to-end model to support utterance-based bilingual ASR, where speakers do not alternate between two languages in a single utterance but may change languages across utterances. We conduct our experiments on English and Mandarin dictation tasks, and we find that BBPE with penalty schemes can improve utterance-based bilingual ASR performance by 2% to 5% relative even with smaller number of outputs and fewer parameters. We conclude with analysis that indicates directions for further improving multilingual ASR.
Eco-Fedsplit: Federated Learning with Error-Compensated Compression

Federated learning is an emerging framework for collaborative machine-learning on devices which do not want to share local data. State-of-the art methods in federated learning reduce the communication frequency, but are not guaranteed to converge to the optimal model parameters. These methods also experience a communication bottleneck, especially when the devices are power-constrained and communicate over a shared medium. This paper presents ECO-FedSplit, an algorithm that increases the communication efficiency of federated learning without sacrificing solution accuracy. The key is to compress inter-device communication and to compensate for information losses in a theoretically justified manner. We prove strong convergence properties of ECO-FedSplit on strongly convex optimization problems and show that the algorithm yields a highly accurate solution with dramatically reduced communication. Extensive numerical experiments validate our theoretical result on real data sets.
Multiple Instance Learning with Task-Specific Multi-Level Features for Weakly Annotated Histopathological Image Classification

Pathological examination is regarded as the gold standard for cancer diagnosis in clinical medicine. Gigapixel histopathological images play an important role in developing automatic pathology diagnosis systems. Because of the expensive cost of pixel-level annotation, deep multiple instance learning (MIL) has become the main approach to classify histopatho-logical images with only slide-level annotation. However, three major challenges including lack of data efficiency because MIL approaches rely on task-agnostic feature extractor, overfitting challenges caused by high data imbalance between tumor and normal tissues, and the similarity between tumor and normal patches, are to be tackled. We proposed a three-stage deep MIL approach to address these challenges. The first stage generated pseudo instance-level labels by utilizing max-max ranking loss. The second stage refined the feature extractor into a task-specific feature extractor with pseudo labels. Focal loss was used to tackle the data imbalance challenge. Partial decoder component was adopted to better distinguish tumor tissues. We re-trained the MIL network with task-specific multi-level features in the third stage. Our model was able to predict both instance-level and bag-level labels in the inference stage. Experiment results showed our model outperformed baseline models in both accuracy and AUC metrics.
An Effective Steganalysis for Robust Steganography with Repetitive JPEG Compression

With the development of social networks, traditional covert communication requires more consideration of lossy processes of Social Network Platforms (SNPs), which is called robust steganography. Since JPEG compression is a universal processing of SNPs, a method using repeated JPEG compression to fit transport channel matching is recently proposed and shows strong compression-resist performance. However, the repeated JPEG compression will inevitably introduce other artifacts into the stego image. Using only traditional steganalysis methods does not work well towards such robust steganography under low payload. In this paper, we propose a simple and effective method to detect the mentioned steganography by chasing both steganographic perturbations as well as continuous compression artifacts. We introduce compression-forensic features as a complement to steganalysis features, and then use the ensemble classifier for detection. Experiments demonstrate that this method owns a similar and better performance with respect to both traditional and neural-network-based steganalysis.
When Does Backdoor Attack Succeed in Image Reconstruction? A Study of Heuristics vs. Bi-Level Solution

Recent studies have demonstrated the lack of robustness of image reconstruction networks to test-time evasion attacks, posing security risks and potential for misdiagnoses. In this paper, we evaluate how vulnerable such networks are to training-time poisoning attacks for the first time. In contrast to image classification, we find that trigger-embedded basic backdoor attacks on these models executed using heuristics lead to poor attack performance. Thus, it is non-trivial to generate backdoor attacks for image reconstruction. To tackle the problem, we propose a bi-level optimization (BLO)-based attack generation method and investigate its effectiveness on image reconstruction. We show that BLO-generated back-door attacks can yield a significant improvement over the heuristics-based attack strategy.
U-GAT-VC: Unsupervised Generative Attentional Networks for Non-Parallel Voice Conversion

Non-parallel voice conversion (VC) is a technique of transfer-ring voice from one style to another without using a parallel corpus in model training. Various methods are proposed to approach non-parallel VC using deep neural networks. Among them, CycleGAN-VC and its variants have been widely accepted as benchmark methods. However, there is still a gap to bridge between the real target and converted voice and an increased number of parameters leads to slow convergence in training process. Inspired by recent advancements in unsupervised image translation, we propose a new end-to-end unsupervised framework U-GAT-VC that adopts a novel inter- and intra-attention mechanism to guide the voice conversion to focus on more important regions in spectrograms. We also introduce disentangle perceptual loss in our model to capture high-level spectral features. Subjective and objective evaluations shows our proposed model outperforms CycleGAN-VC2/3 in terms of conversion quality and voice naturalness.
Multiview Long-Short Spatial Contrastive Learning For 3D Medical Image Analysis

The success of supervised deep learning heavily depends on large labeled datasets whose construction is often challenging in medical image analysis. Contrastive learning, a variant of self-supervised learning, is a potential solution to alleviate the strong demand for data annotation. In this work, we extend the contrastive learning framework to 3D volumetric medical imaging. Specifically, we propose (1) multiview contrasting strategy to maximize the mutual information between three views of 3D image to learn global representations and (2) long-short spatial contrasting strategy to learn local representations by matching a short spatial clip to a long spatial clip in the latent space. To combine these two strategies, we propose multiview long-short spatial contrastive learning (MLSSCL) framework, which can effectively learn generic 3D representations. Our extensive experiments on two brain Magnetic Resonance Imaging (MRI) datasets demonstrate that MLSSCL significantly outperforms learning from scratch and other self-supervised learning methods on both classification and segmentation tasks.
Binary Dense Predictors for Human Pose Estimation Based on Dynamic Thresholds and Filtering

Binary neural networks (BNNs) contribute a lot to the efficiency of image classification models. However, in dense predication tasks such as human pose estimation, predictions in different locations are coupled and rely on the extraction of features across entire images. As a result, more robust and adaptive binarization is required to bridge the performance gap between binarized and full precision models. We propose two approaches to conduct image-aware and pixel-aware dynamic binarization in a model for human pose estimation. Firstly, a simplified dynamic thresholding is leveraged in the backbone to determine unique binarization thresholds for each image. Secondly, in the decoder, we decouple binarization for each pixel according to the activations surrounding the pixel. Dynamic filtering modules are proposed to determine a different binarization strategy for each pixel. Compared with the strong baselines, the proposed framework improves 5.2% and 3.6% mAP on the COCO test-dev benchmark for ResNet-18/34 architectures respectively.
Position-Invariant Adversarial Attacks on Neural Modulation Recognition

Deep neural networks (DNNs) are widely used for neural modulation recognition (NMR) in the electronic field and have been shown to be vulnerable to adversarial examples for NMR. In the physical signal communication scenario, the adversarial signal transmitted by the adversary is affected by the channel, resulting in a random time delay with the original signal and causing decay on the attack performance. To ad-dress this issue, we propose the Position-Invariant adversarial attack Method (PIM) that generates the position-invariant adversarial signal by averaging the adversarial signals generated by shifted input signals to mitigate the channel effect on time delay. Our PIM can be easily integrated with other methods to achieve better results. Extensive experiments demonstrate that the proposed method could outperform all baselines for adversarial attacks on NMR under the time delay setting.
Pyramid Fusion Attention Network For Single Image Super-Resolution

Recently, convolutional neural network (CNN) has made a mighty advance in image super-resolution (SR). Most recent models exploit attention mechanism (AM) to focus on high-frequency information. However, these methods exclusively consider interdependencies among channels or spatials, leading to equal treatment of channel-wise or spatial-wise features thus hindering the power of AM. In this paper, we propose a pyramid fusion attention network (PFAN) to tackle this problem. Specifically, a novel pyramid fusion attention (PFA) is developed where stacked residual blocks are employed to model the relationship between pixels among all channels, and pyramid fusion structure is adopted to expand receptive field. Besides, a progressive backward fusion strat-egy is introduced to make full use of hierarchical features, which are beneficial to obtaining more contextual representations. Comprehensive experiments demonstrate the superiority of our proposed PFAN against state-of-the-art methods.
TalkingFlow: Talking Facial Landmark Generation with Multi-Scale Normalizing Flow Network

Deterministic models dominate the field of talking facial land-mark generation by directly mapping speech signals to a certain lip-sync facial landmark sequence, which often suffer from regression to the mean face. In contrast, probability generative models are more beneficial to handle complex data space and generate diverse samples. In this work, we pro-pose a flow-based probabilistic network named TalkingFlow to generate natural talking facial landmark with head movements from speech data. It is implemented by a weighted multi-scale architecture to improve model representation capability and a conditional Temporal Convolutional Network module to fuse speech data. Extensive experiments results show that it can effectively generate diverse and natural facial landmark from speech data. All code will be made publicly available online.
Block-Sparse Adversarial Attack to Fool Transformer-Based Text Classifiers

Recently, it has been shown that, in spite of the significant performance of deep neural networks in different fields, those are vulnerable to adversarial examples. In this pa-per, we propose a gradient-based adversarial attack against transformer-based text classifiers. The adversarial perturbation in our method is imposed to be block-sparse so that the resultant adversarial example differs from the original sentence in only a few words. Due to the discrete nature of textual data, we perform gradient projection to find the minimizer of our proposed optimization problem. Experimental results demonstrate that, while our adversarial attack maintains the semantics of the sentence, it can reduce the accuracy of GPT-2 to less than 5% on different datasets (AG News, MNLI, and Yelp Reviews). Furthermore, the block-sparsity constraint of the proposed optimization problem results in small perturbations in the adversarial example. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Adversary Distillation for One-Shot Attacks on 3D Target Tracking

Considering the vulnerability of existing deep models in the adversarial scenario, the robustness of 3D target tracking is not guaranteed. In this paper, we present an efficient generation based adversarial attack, termed Adversary Distillation Network (AD-Net), which is able to distract a victim tracker in a single shot. In contrast to existing adversarial attacks derived from point perturbations, the proposed method designs a generative network to distill an adversarial example from a tracking template through point-wise filtration. A binary distribution encoding layer is specialized to filter points, which is modeled as a Bernoulli distribution and approximated in a differentiable formulation. To boost the performance of adversarial example generation, a feature extraction module is deployed, which leverages the PointNet++ architecture to learn hierarchical features for the template points as well as similarities with the search areas. Experimental results on the KITTI vision benchmark show that the proposed adversarial attack can effectively mislead popular deep 3D trackers.
Natural-Looking Adversarial Examples from Freehand Sketches

Deep neural networks (DNNs) have achieved great success in image classification and recognition compared to previous methods. However, recent works have reported that DNNs are very vulnerable to adversarial examples that are intentionally generated to mislead the predictions of the DNNs. Here, we present a novel freehand sketch-based natural-looking adversarial example generator that we call SketchAdv. To generate a natural-looking adversarial example from a sketch, we force the encoded edge information (i.e., the visual attributes) to be close to the latent random vector fed to the edge generator and adversarial example generator. This preserves the spatial consistency of the adversarial example generated from the random vector with the edge information. In addition, by employing a sketch-edge encoder with a novel sketch-edge matching loss, we reduce the gap between edges and sketches. We evaluate the proposed method on several dominant classes of SketchyCOCO, the benchmark dataset for sketch to image translation. Our experiments show that our SketchAdv produces visually plausible adversarial examples while remaining competitive with other adversarial attack methods.
Neural-FST Class Language Model for End-to-End Speech Recognition

We propose Neural-FST Class Language Model (NFCLM) for end-to-end speech recognition, a novel method that combines neural network language models (NNLMs) and finite state transducers (FSTs) in a mathematically consistent framework. Our method utilizes a background NNLM which models generic background text together with a collection of domain-specific entities modeled as individual FSTs. Each output token is generated by a mixture of these components; the mixture weights are estimated with a separately trained neural decider. We show that NFCLM significantly outperforms NNLM by 15.8% relative in terms of Word Error Rate. NFCLM achieves similar performance as traditional NNLM and FST shallow fusion while being less prone to overbiasing and 12 times more compact, making it more suitable for on-device usage.
Interpretable Image Classification Using Sparse Oblique Decision Trees

Interpreting the image datasets is a difficult task, as each image contains a lot of irrelevant data. This paper presents a simple yet effective method to interpret the image datasets. We achieve this by using sparse oblique trees as a tool to select features from the dataset. These trees are not only accurate but also very interpretable. The hierarchical structure of the tree helps to visualize the underlying patterns in the dataset. By studying the weights of the nodes, we can determine what set of features differentiate between classes or groups of classes. We effectively demonstrate our results in multiple image datasets.
Developing Digital Game Based Learning to Support Numeracy of Elementary School Teacher Education Students

Digital Game-Based Learning (DGBL) is a form of learning innovation that facilitates students to learn game-based. This study aims to produce digital game-based learning products that can improve the numeracy skills of elementary school teacher education students. The game will be tested for validity and practicality before being applied in learning. This study indicates that the resulting product, “CAKAP NUMERASI” game, is suitable for use in learning and is practically used in learning. The validation test results from learning experts in the field of elementary school teacher education showed a percentage of 95.9%, which was in the very valid criteria. The results of the practicality test are in the practical criteria with a percentage of 80.4%. Based on the notes on the validity and practicality of the media, several ways can be done to improve the quality of the media before being tested on a wider range of students.
Predicting stress levels of automobile drivers using classical machine learning classifiers

Traffic congestion has been found to be a substantial source of stress for many people. To put it another way, driving under the influence of high amounts of stress could result in accidents and other long-term health issues. As a result, there is a pressing need to measure and anticipate the key reasons (features or classes) that raise stress levels in drivers. Car drivers’ stress levels while driving are investigated using four different data mining algorithms: K-Nearest Neighbor (KNN), Decision Tree, PART, and Naive Bayes. A wearable biomedical device attached to the driver’s body was used in an experiment in Amman, Jordan, where 12 drivers traveled different routes. According to the obtained data (dataset), ‘Yes’ indicates the presence of stress whereas ‘No’ indicates the absence of stress. Naïve Bayes surpasses the PART, KNN, and Decision Tree in terms of AUC and Precision, but the KNN is superior in terms of Recall and F1 for the other classifiers. Furthermore, the results show that the level of stress differs depending on the gender, age, driving skills, distraction, and driving-concentration features.
SEOSS-Queries - a software engineering dataset for text-to-SQL and question answering tasks

Stakeholders of software development projects have various information needs for making rational decisions during their daily work. Satisfying these needs requires substantial knowledge of where and how the relevant information is stored and consumes valuable time that is often not available. Easing the need for this knowledge is an ideal text-to-SQL benchmark problem, a field where public datasets are scarce and needed. We propose the SEOSS-Queries dataset consisting of natural language utterances and accompanying SQL queries extracted from previous studies, software projects, issue tracking tools, and through expert surveys to cover a large variety of information need perspectives. Our dataset consists of 1,162 English utterances translating into 166 SQL queries; each query has four precise utterances and three more general ones. Furthermore, the dataset contains 393,086 labeled utterances extracted from issue tracker comments. We provide pre-trained SQLNet and RatSQL baseline models for benchmark comparisons, a replication package facilitating a seamless application, and discuss various other tasks that may be solved and evaluated using the dataset. The whole dataset with paraphrased natural language utterances and SQL queries is hosted at figshare.com/s/75ed49ef01ac2f83b3e2.
Conceptual Framework for Adversarial Thinking Adoption in Experiential Learning Model for Robotics Learning

In this high-technological era, cybersecurity is well-known due to various cybercrimes that happen from time to time. Adversarial thinking, which can be explained as one ability of strategic thinking commonly applied by the hacker, needs to be disclosed to society as one of cybersecurity awareness. Therefore, this study highlights the usage of adversarial thinking in education by adopting the adversarial thinking elements in the experiential learning model. The conceptual framework of adopting adversarial thinking has been developed by using the Matsuo-Nagata learning model 2020. The Matsuo-Nagata learning model has only been applied in the training programs; therefore, to apply the current learning model to the education and cybersecurity field, a new learning model has been revised and adopted with the adversarial thinking aspect. Robotic learning has been chosen as the learning tool for the revised experiential learning model. This study involves three elements of adversarial thinking: analytical, creative, and practical. By analyzing the previous study and the survey results from the expert, the findings show that adversarial thinking is suitable and essential to be adopted in the experiential learning model.
Mapping of Agile Processes into Project Management Knowledge Areas and Processes

In the world of Information Technology to keep up with fast pace and constant changes the updated knowledge of processes and techniques is the essential requirement. For constant changes and successful delivery of projects the software project management techniques and framework were introduced. Currently many different process models exist which are selected and implemented according to the needs of project and organizations. To bring uniformity in project management processes PMI introduced Project management framework consisting of 10 knowledge areas and 5 processes. The framework consists of all aspects of the project management and processes which are required to be followed for the delivery of project from initiation of the project till the closure of the project. The other most popular software project management is Agile project management which emphasis on delivery in phases and releases known as sprints. The agile processes consist of Scrum, XP, Lean, Kanban, Crystal; Agile unified Process and many more. The agile process flow consists of six processes but the main lacking found in these processes was the unified framework to be followed by Agile Masters and Project managers. In this research, the Project management 10 knowledge areas and 5 processes have been mapped with the processes of Agile project management to propose Agile project management framework. A survey was also to get insight view of the individuals regarding agile project management and where the agile project management lacks focus in comparison with traditional project management. Agile Project management is an extensive and vast topic on which further research is required to be carried out to further polish these processes according the needs of the project. In future, more extensive research can be carried out on the artifacts required for agile project management, their template.