Reviewerly/exHarmony huggingface dataset:
Optimizing distributed training deployment in heterogeneous GPU clusters

This paper proposes HeteroG, an automatic module to accelerate deep neural network training in heterogeneous GPU clusters. To train a deep learning model with large amounts of data, distributed training using data or model parallelism has been widely adopted, mostly over homogeneous devices (GPUs, network bandwidth). Heterogeneous training environments may often exist in shared clusters with GPUs of different models purchased in different batches and network connections of different bandwidth availability (e.g., due to contention). Classic data parallelism does not work well in a heterogeneous cluster, while model-parallel training is hard to plan. HeteroG enables highly-efficient distributed training over heterogeneous devices, by automatically converting a single-GPU training model to a distributed one according to the deep learning graph and available resources. HeteroG embraces operation-level hybrid parallelism, communication architecture selection and execution scheduling, based on a carefully designed strategy framework exploiting both GNN-based learning and combinatorial optimization. We compare HeteroG with existing parallelism schemes and show that it achieves up-to 222% training speed-up. HeteroG also enables efficient training of large models over a set of heterogeneous devices where simple parallelism is infeasible.
The Carabela Project and Manuscript Collection: Large-Scale Probabilistic Indexing and Content-based Classification

The main aim of the Carabela project was to develop and apply techniques that allow textual searching on massive Spanish collections of 15th-19th century manuscripts. The project focused on a relatively small subset of 125 000 images of collections of interest to underwater archaeology. For this type of manuscripts, state-of-the-art automatic transcription techniques, generally fail to achieve usable transcription accuracy. Therefore, rather than insisting in actual transcription, methodologies for probabilistic indexing of handwritten text images have been adopted. This has allowed us to effectively cope with the intrinsically high degree of uncertainty of the text contained in most historical manuscripts, leading to highly effective systems for textual search and retrieval. Carabela has gone one step further by developing new techniques to classify probabilistically indexed, but otherwise untranscribed, text images according to their textual content. These techniques have been successfully used to automatically classify Carabela bundels (each containing hundreds or thousands of pages) according to their “level of risk” of public exposure, in order to control their access and avoid as much as possible the plundering of Spanish underwater heritage.
Understanding type changes in Java

Developers frequently change the type of a program element and update all its references for performance, security, concurrency,library migration, or better maintainability. Despite type changes being a common program transformation, it is the least automated and the least studied. With this knowledge gap, researchers miss opportunities to improve the state of the art in automation for software evolution, tool builders do not invest resources where automation is most needed, language and library designers can-not make informed decisions when introducing new types, and developers fail to use common practices when changing types. To fill this gap, we present the first large-scale and most fine-grained empirical study on type changes in Java. We develop state-of-the-art tools to statically mine 297,543 type changes and their subsequent code adaptations from a diverse corpus of 129 Java projects containing 416,652 commits. With this rich data set we answer research questions about the practice of type changes. Among others, we found that type changes are actually more common than renaming,but the current research and tools for type changes are inadequate.Based on our extensive and reliable data, we present actionable,empirically-justified implications.
Mitigating Embedding and Class Assignment Mismatch in Unsupervised Image Classification

Unsupervised image classification is a challenging computer vision task. Deep learning-based algorithms have achieved superb results, where the latest approach adopts unified losses from embedding and class assignment processes. Since these processes inherently have different goals, jointly optimizing them may lead to a suboptimal solution. To address this limitation, we propose a novel two-stage algorithm in which an embedding module for pretraining precedes a refining module that concurrently performs embedding and class assignment. Our model outperforms SOTA when tested with multiple datasets, by substantially high accuracy of 81.0% for the CIFAR-10 dataset (i.e., increased by 19.3 percent points), 35.3% accuracy for CIFAR-100-20 (9.6 pp) and 66.5% accuracy for STL-10 (6.9 pp) in unsupervised tasks.
Automatic and efficient variability-aware lifting of functional programs

A software analysis is a computer program that takes some representation of a software product as input and produces some useful information about that product as output. A software product line encompasses many software product variants, and thus existing analyses can be applied to each of the product variations individually, but not to the entire product line as a whole. Enumerating all product variants and analyzing them one by one is usually intractable due to the combinatorial explosion of the number of product variants with respect to product line features. Several software analyses (e.g., type checkers, model checkers, data flow analyses) have been redesigned/re-implemented to support variability. This usually requires a lot of time and effort, and the variability-aware version of the analysis might have new errors/bugs that do not exist in the original one. Given an analysis program written in a functional language based on PCF, in this paper we present two approaches to transforming (lifting) it into a semantically equivalent variability-aware analysis. A light-weight approach (referred to as shallow lifting ) wraps the analysis program into a variability-aware version, exploring all combinations of its input arguments. Deep lifting, on the other hand, is a program rewriting mechanism where the syntactic constructs of the input program are rewritten into their variability-aware counterparts. Compositionally this results in an efficient program semantically equivalent to the input program, modulo variability. We present the correctness criteria for functional program lifting, together with correctness proof sketches of shallow lifting. We evaluate our approach on a set of program analyses applied to the BusyBox C-language product line.
Towards Part-Aware Monocular 3D Human Pose Estimation: An Architecture Search Approach

Even though most existing monocular 3D pose estimation approaches achieve very competitive results, they ignore the heterogeneity among human body parts by estimating them with the same network architecture. To accurately estimate 3D poses of different body parts, we attempt to build a part-aware 3D pose estimator by searching a set of network architectures. Consequently, our model automatically learns to select a suitable architecture to estimate each body part. Compared to models built on the commonly used ResNet-50 backbone, it reduces 62% parameters and achieves better performance. With roughly the same computational complexity as previous models, our approach achieves state-of-the-art results on both the single-person and multi-person 3D pose estimation benchmarks.
Motion vector modification distortion analysis-based payload allocation for video steganography

Video steganography forms a covert communication channel by data embedding in cover elements. To consider inter-frame mutual embedding impacts, this paper proposes a payload allocation strategy in video steganography based on motion vector modification distortion analysis. Firstly, the motion vector modification distortion caused by data embedding is analyzed. Then, a rate–distortion model reflecting the residue deviation propagation in successive inter-coded frames is derived. According to this model, the residue deviation propagation weight of each inter-coded frame can be computed. Finally, an inter-frame payload allocation strategy is designed in order to restrain the residue deviation propagation. Experimental results demonstrate that the proposed payload allocation strategy can enhance existing motion vector-based video steganographic methods in terms of undetectability and video coding performance. Besides, the lower computational complexity can be achieved.
IoD swarms collision avoidance via improved particle swarm optimization

Drones flights have been investigated widely. In the presence of high density and complex missions, collision avoidance among swarm of drones and with environment obstacles becomes a challenging task and indispensable. This paper aims to enhance the optimality and rapidity of three dimensional IoD path generation by improving the particle swarm optimization (PSO) algorithm. The improvements include using chaos map logic to initialize the population of PSO. Also, adaptive mutation is utilized to balance local and global search. Then, the inactive particles are replaced by new fresh particles to push the solution toward global optimal. Furthermore, Monte Carlo simulation is carried out and the results are compared with slandered PSO and with recent work CIPSO. The results exhibit significant improvement in convergence speed as well as optimal solution which prove the ability of proposed method to generate safety path for IoD formation without collision with terrain obstacle and among drones.
Recognition of Drivers’ Activity Based on 1D Convolutional Neural Network

Background and objective: Driving a car is a complex activity which involves movements of the whole body. Many studies on drivers’ behavior are conducted to improve road traffic safety. Such studies involve the registration and processing of multiple signals, such as electroencephalography (EEG), electrooculography (EOG) and the images of the driver’s face. In our research, we attempt to develop a classifier of scenarios related to learning to drive based on the data obtained in real road traffic conditions via smart glasses. In our approach, we try to minimize the number of signals which can be used to recognize the activities performed while driving a car. Material and methods: We attempt to evaluate the drivers’ activities using both electrooculography (EOG) and a deep learning approach. To acquire data we used JINS MEME smart glasses furnished with 3-point EOG electrodes, 3-axial accelerometer and 3-axial gyroscope. Sensor data were acquired on 20 drivers (ten experienced and ten learner drivers) on the same 28.7 km route under real road conditions in southern Poland. The drivers performed several tasks while wearing the smart glasses and the tasks were linked to the signal during the drive. For the recognition of four activities (parking, driving through a roundabout, city traffic and driving through an intersection), we used one-dimensional convolutional neural network (1D CNN). Results: The maximum accuracy was 95.6% on validation set and 99.8% on training set. The results prove that the model based on 1D CNN can classify the actions performed by drivers accurately. Conclusions: We have proved the feasibility of recognizing drivers’ activity based solely on EOG data, regardless of the driving experience and style. Our findings may be useful in the objective assessment of driving skills and thus, improving driving safety.
Significant symptoms and nonsymptom-related factors for malaria diagnosis in endemic regions of Indonesia

ObjectivesThis study aims to identify significant symptoms and nonsymptom-related factors for malaria diagnosis in endemic regions of Indonesia.MethodsMedical records are collected from patients suffering from malaria and other febrile diseases from public hospitals in endemic regions of Indonesia. Interviews with eight Indonesian medical doctors are conducted. Feature selection and machine learning techniques are used to develop malaria classifiers for identifying significant symptoms and nonsymptom-related factors.ResultsSeven significant symptoms (duration of fever, headache, nausea and vomiting, heartburn, severe symptom, dizziness, and joint pain) and patients’ history of malaria as a nonsymptom-related factor contribute most to malaria diagnosis. As a symptom, fever duration is more significant than temperature or fever for distinguishing malaria from other febrile diseases. Shivering, fever, and sweating (known to indicate malaria presence in Indonesia) are shown to be less significant than other symptoms in endemic regions.ConclusionsThree most suitable malaria classifiers have been developed to identify the significant features that can be used to predict malaria as distinct from other febrile diseases. With extensive experiments on the classifiers, the significant features identified can help medical doctors in the clinical diagnosis of malaria and raise public awareness of significant malaria symptoms at early stages.
A Systematic Study of Elastic Objects in Kernel Exploitation

Recent research has proposed various methods to perform kernel exploitation and bypass kernel protection. For example, security researchers have demonstrated an exploitation method that utilizes the characteristic of elastic kernel objects to bypass KASLR, disclose stack/heap cookies, and even perform arbitrary read in the kernel. While this exploitation method is considered a commonly adopted approach to disclosing critical kernel information, there is no evidence indicating a strong need for developing a new defense mechanism to limit this exploitation method. It is because the effectiveness of this exploitation method is demonstrated only on anecdotal kernel vulnerabilities. It is unclear whether such a method is useful for a majority of kernel vulnerabilities.
Four-pattern, phase-step non-sensitive phase shifting method based on Carré algorithm

Carré algorithm is a classic phase shifting measurement method due to its advantage in resisting the error caused by inaccurate phase step. The conventional Carré algorithm only calculates the wrapped phase, so multiple frequencies are used to unwrap the phase so as to obtain the absolute phase, which increases the number of patterns and decreases the measuring speed. We propose a high-accuracy and 4-pattern phase retrieval method based on Carré algorithm. The phase step in Carré algorithm is encoded as a linear variable. With the linear phase step, we obtain both the wrapped phase and absolute phase simultaneously from only 4 patterns. The proposed method is non-sensitive to the errors in phase step, the same as the conventional Carré algorithm. In addition, because only 4 patterns are required for obtaining the wrapped phase and absolute phase, the proposed method is practical in high-speed 3D measurement. Experimental results demonstrate its performance in high accuracy (absolute accuracy is higher than 0.02 mm and relative accuracy is higher than 0.03%), efficiency (requires only 4 patterns) and robustness.
QoE-Aware 3D Video Streaming via Deep Reinforcement Learning in Software Defined Networking Enabled Mobile Edge Computing

With the advancements of wireless network transmission technology, 2D video is hard to satisfy people's requirement for multimedia. Therefore, the high-definition 3D video that can bring a whole new viewing experience is starting to enter people's vision. However, when a tremendously large number of users play 3D video, it puts enormous computational pressure on the cloud server, which incurs high transmission latency. To release the tension, in this articl we consider a promising computing and networking architecture by incorporating Mobile Edge Computing (MEC) and Software-defined Networking (SDN) and propose a novel resource allocation model (RAM) to allocate resources and reduce delay. At the same time, we introduce the Quality of Experience (QoE) Model (QoEM), which uses information collected during 3D video playback to adaptively allocate the rate of future tiles. The model addresses the problem of assigning the best transmission speed to the block in the case of time-varying characteristicfactors during transmission. We propose an Actor-Critic-based deep reinforcement learning algorithm for viewport prediction and QoE optimization, called QoE-AC. For the differential transmission in the playback phase, we use the LSTM network for bandwidth and viewport prediction, while combining the historical information of the blocks into the Actor-Critic network as observations. The network can be adaptively assigned the best transmission speed for future tiles based on observations to maximize QoE. Finally, the experimental results show that the actual performance of the model is much better than other existing 3D video network models. Under different QoE targets, our proposed system can be adapted to all situations and has a 10%-15% performance improvement.
Differential Advising in Multiagent Reinforcement Learning

Agent advising is one of the main approaches to improve agent learning performance by enabling agents to share advice. Existing advising methods have a common limitation that an adviser agent can offer advice to an advisee agent only if the advice is created in the same state as the advisee's state. However, in complex environments, it is a very strong requirement that two states are the same, because a state may consist of multiple dimensions and two states being the same means that all these dimensions in the two states are correspondingly identical. Therefore, this requirement may limit the applicability of existing advising methods to complex environments. In this article, inspired by the differential privacy scheme, we propose a differential advising method that relaxes this requirement by enabling agents to use advice in a state even if the advice is created in a slightly different state. Compared with the existing methods, agents using the proposed method have more opportunity to take advice from others. This article is the first to adopt the concept of differential privacy on advising to improve agent learning performance instead of addressing security issues. The experimental results demonstrate that the proposed method is more efficient in complex environments than the existing methods.
Betrayed by Motion: Camouflaged Object Discovery via Motion Segmentation

The objective of this paper is to design a computational architecture that discovers camouflaged objects in videos, specifically by exploiting motion information to perform object segmentation. We make the following three contributions: (i) We propose a novel architecture that consists of two essential components for breaking camouflage, namely, a differentiable registration module to align consecutive frames based on the background, which effectively emphasises the object boundary in the difference image, and a motion segmentation module with memory that discovers the moving objects, while maintaining the object permanence even when motion is absent at some point. (ii) We collect the first large-scale Moving Camouflaged Animals (MoCA) video dataset, which consists of over 140 clips across a diverse range of animals (67 categories). (iii) We demonstrate the effectiveness of the proposed model on MoCA, and achieve competitive performance on the unsupervised segmentation protocol on DAVIS2016 by only relying on motion.
Multi-agent Embodied Question Answering in Interactive Environments

We investigate a new AI task—Multi-Agent Interactive Question Answering—where several agents explore the scene jointly in interactive environments to answer a question. To cooperate efficiently and answer accurately, agents must be well-organized to have balanced work division and share knowledge about the objects involved. We address this new problem in two stages: Multi-Agent 3D Reconstruction in Interactive Environments and Question Answering. Our proposed framework features multi-layer structural and semantic memories shared by all agents, as well as a question answering model built upon a 3D-CNN network to encode the scene memories. During the reconstruction, agents simultaneously explore and scan the scene with a clear division of work, organized by next viewpoints planning. We evaluate our framework on the IQuADv1 dataset and outperform the IQA baseline in a single-agent scenario. In multi-agent scenarios, our framework shows favorable speedups while remaining high accuracy.
Effective Deep Learning Models for Automatic Diacritization of Arabic Text

While building a text-to-speech system for the Arabic language, we found that the system synthesized speeches with many pronunciation errors.The primary source of these errors is the lack of diacritics in modern standard Arabic writing.These diacritics are small strokes that appear above or below each letter to provide pronunciation and grammatical information.We propose three deep learning models to recover Arabic text diacritics based on our work in a text-to-speech synthesis system using deep learning.The first model is a baseline model used to test how a simple deep learning model performs on the corpora.The second model is based on an encoder-decoder architecture, which resembles our text-to-speech synthesis model with many modifications to suit this problem.The last model is based on the encoder part of the textto-speech model, which achieves state-of-the-art performances in both word error rate and diacritic error rate metrics.These models will benefit a wide range of natural language processing applications such as text-to-speech, part-of-speech tagging, and machine translation.
Recognition and Classification of Dynamic Hand Gestures by a Wearable Data-Glove

This paper introduces a complete work-flow for the translation of dynamic isolated signs based on data acquired from a data-glove. A sign language translation system based on a wearable device represents indeed a more efficient solution with respect to cameras or position trackers for helping speech-impaired people on a daily basis. The paper describes the different steps required for a sign language translation, namely segmentation, feature extraction and classification, together with the custom data-glove used for data-acquisition. The paper presents also experimental results, comparing different machine learning classifiers and discussing their performances both in terms of translation accuracy and computational time. The proposed work-flow has been tested both on data acquired from a custom data-glove and on a public database, and it outperforms those of other works in literature. The reported analysis suggests a multi-layer perceptron neural network as the most suitable classifier for the realization of a wearable sign language translation system.
Effective heterogeneous ensemble classification: An alternative approach for selecting base classifiers

In this paper, an alternative approach to select base classifiers forming a parallel heterogeneous ensemble is proposed. The fundamental concept is to trim poorly performing classifiers; thus, a more effective heterogeneous ensemble can be generated. More specifically, the proposed trimming approach finds an optimal subset of classifiers to form the desired heterogeneous ensemble. The main challenge is how to detect poor performance classifiers. To address this issue, the differences in effectiveness between base classifiers forming the ensemble are utilized to spot weak classifiers. For evaluating the proposed approach, eighteen benchmark datasets are used for generating the heterogeneous ensemble classification and comparisons with the state-of-the-art methods are conducted. The experimental analysis demonstrated the effectiveness and superiority of the proposed approach when compared to other state-of-the-art approaches.