Reviewerly/exHarmony huggingface dataset:
Didispeech: A Large Scale Mandarin Speech Corpus

This paper introduces a new open-sourced Mandarin speech corpus, called DiDiSpeech. It consists of about 800 hours of speech data at 48kHz sampling rate from 6000 speakers and the corresponding texts. All speech data in the corpus is recorded in quiet environment and is suitable for various speech processing tasks, such as voice conversion, multi-speaker text-to-speech and automatic speech recognition. We conduct experiments with multiple speech tasks and evaluate the performance, showing that it is promising to use the corpus for both academic research and practical application. The corpus is available at https://outreach.didichuxing.com/research/opendata/.
Three-way recommendation model based on shadowed set with uncertainty invariance

Recommender systems are an effective tool to resolve information overload by enabling the selection of the subsets of items from a universal set based on user preferences. The operation of most of recommender systems depends on the prediction ratings, which may introduce a degree of uncertainty into the process of recommendation. However, systems equipped with only two strategies lack the flexibility to address such uncertain decision-making problems. Thus, the presence of far-fetched recommendations accompanied by uncertainties often decreases recommendation quality. To resolve this issue, a three-way recommendation model based on a novel shadowed set is proposed in this paper to reduce decision-making risk and improve quality. To this end, a neighborhood rough set model is first introduced into three-way recommendation to determine similar user to active users with respect to the original rating decision system. This helps to avoid the uncertainty generated during the assignment of prediction rating. Subsequently, the optimal neighborhood radius is defined to overcome the subjectivity associated with the construction of the aforementioned neighborhood with a subjective parameter. Following this, a novel shadowed set model, based on neighborhood memberships of boundary users, is proposed to partition all users into positive region, negativity region and boundary region. This facilitates the adoption of different decisions by recommender systems for users in different regions. Finally, the effectiveness and reliability of the proposed model are verified on two Movielens datasets via comparison analyses.
A mutual information based federated learning framework for edge computing networks

With the application of artificial intelligence in all field of life, people pay more attention to user privacy and data security. Under the condition of protecting user privacy, the federated learning model has become a popular research technology to solve the data islands problems. The edge computing network can be applied to smart city, Internet of vehicles and so on. Federated learning is a framework in which multiple hosts jointly learn a machine learning model. Each work device maintains the local model of its local training dataset, while the master device maintains the global model by aggregating the local models from the work devices. However, it cannot ensure that every local work device is an honest user because of a phenomenon that the hosts has been operated by attacker interferes in the process of local model training. In this paper, we assume that malicious nodes upload unreal learning parameters in the federated learning framework, which the global model will have high error rate. We propose a federated learning parameter aggregating algorithm based on mutual information. We introduced the relevance of model training learning rate to determine the consistency of the training direction of the local and central models at coarse granularity. We aggregated the parameters of the models at fine granularity based on the correlation of the gradients based on the mutual information. The mutual information method is used to calculate the similarity of the gradient trend between the local training model and overall model. We set the trust weight of each work device to reduce the negative impact of malicious nodes. The evaluation results show that the classification accuracy of the MIFL model is improved as compared with the average federated learning without malicious node. Especially, in the case of existing malicious nodes, the proposed algorithm can defend against malicious node attacks and sustain the robustness of Federated learning.
Reducing DNN Properties to Enable Falsification with Adversarial Attacks

Deep Neural Networks (DNN) are increasingly being deployed in safety-critical domains, from autonomous vehicles to medical devices, where the consequences of errors demand techniques that can provide stronger guarantees about behavior than just high test accuracy. This paper explores broadening the application of existing adversarial attack techniques for the falsification of DNN safety properties. We contend and later show that such attacks provide a powerful repertoire of scalable algorithms for property falsification. To enable the broad application of falsification, we introduce a semantics-preserving reduction of multiple safety property types, which subsume prior work, into a set of equivalid correctness problems amenable to adversarial attacks. We evaluate our reduction approach as an enabler of falsification on a range of DNN correctness problems and show its cost-effectiveness and scalability.
Role of technology in COVID-19 pandemic

Novel Corona Virus is the most recent pandemic, which has struck more than 210 countries and territories all over the world placing states in a perilous position. Enormous research are being done on the virus detection, providing treatments to relief symptoms and developing its vaccine, which, according to an estimate, might take one to two more years. Therefore, WHO has laid stress upon the governments worldwide to guarantee competent surveillance and identification of infected individuals to control severity of COVID-19 pandemic effects. Latest technologies, such as IoMT, drones, robots, UVs, GPS, and Bluetooth, can play a primary role in such circumstances to mitigate the impact of COVID-19 outbreak. Therefore, our study highlights numerous technological solutions, which are of great help in controlling disease spread and facing challenges caused by it.
Mixup Regularized Adversarial Networks for Multi-Domain Text Classification

Using the shared-private paradigm and adversarial training can significantly improve the performance of multi-domain text classification (MDTC) models. However, there are two issues for the existing methods: First, instances from the multiple domains are not sufficient for domain-invariant feature extraction. Second, aligning on the marginal distributions may lead to a fatal mismatch. In this paper, we propose mixup regularized adversarial networks (MRANs) to address these two issues. More specifically, the domain and category mixup regularizations are introduced to enrich the intrinsic features in the shared latent space and enforce consistent predictions in-between training instances such that the learned features can be more domain-invariant and discriminative. We conduct experiments on two benchmarks: The Amazon review dataset and the FDU-MTL dataset. Our approach on these two datasets yields average accuracies of 87.64% and 89.0% respectively, outperforming all relevant baselines.
Synthesis of New Words for Improved Dysarthric Speech Recognition on an Expanded Vocabulary

Dysarthria is a condition where people experience a reduction in speech intelligibility due to a neuromotor disorder. Previous works in dysarthric speech recognition have focused on accurate recognition of words encountered in training data. Due to the rarity of dysarthria in the general population, a relatively small amount of publicly-available training data exists for dysarthric speech. The number of unique words in these datasets is small, so ASR systems trained with existing dysarthric speech data are limited to recognition of those words. In this paper, we propose a data augmentation method using voice conversion that allows dysarthric ASR systems to accurately recognize words outside of the training set vocabulary. We demonstrate that a small amount of dysarthric speech data can be used to capture the relevant vocal characteristics of a speaker with dysarthria through a parallel voice conversion system. We show that it’s possible to synthesize utterances of new words that were never recorded by speakers with dysarthria, and that these synthesized utterances can be used to train a dysarthric ASR system.
Itsy-Bits: Fabrication and Recognition of 3D-Printed Tangibles with Small Footprints on Capacitive Touchscreens

Tangibles on capacitive touchscreens are a promising approach to overcome the limited expressiveness of touch input. While research has suggested many approaches to detect tangibles, the corresponding tangibles are either costly or have a considerable minimal size. This makes them bulky and unattractive for many applications. At the same time, they obscure valuable display space for interaction.
Analysis of Edge-Optimized Deep Learning Classifiers for Radar-Based Gesture Recognition

The increasing significance of technology in daily lives led to the need for the development of convenient methods of human-computer interaction (HCI). Given that the existing HCI approaches exhibit various limitations, hand gesture recognition-based HCI may serve as a more intuitive mode of human-machine interaction in many situations. In addition, the system has to be deployable on low-power devices for applicability in broadly defined Internet of Things (IoT) and smart home solutions. Recent advances exhibit the potential of deep learning models for gesture classification, whereas they are still limited to high-performance hardware. Embedded neural network accelerators are constrained in terms of available memory, central processing unit (CPU) clock speed, graphics processing unit (GPU) performance, and a number of supported operations. The aforementioned problems are addressed in this paper by namely two approaches - simplifying the signal processing pipeline to avoid recurrent structures and efficient topological design. This paper employs an intuitive scheme allowing for the generation of the data in the compressed form from the sequence of range-Doppler images (RDI). Thus, it allows for the design of a neural classifier avoiding the usage of recurrent layers. The proposed framework has been optimized for Intel <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">®</sup> Neural Compute Stick 2 (Intel <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">®</sup> NCS 2), at the same time achieving promising classification accuracy of 97.57%. To confirm the robustness of the proposed algorithm, five independent persons have been involved in the algorithm testing process.
Real-time operation of a multi-rate, multi-protocol quantum key distribution transmitter

Quantum key distribution (QKD) is the best candidate for securing communications against attackers, who may in the future exploit quantum-enhanced computational powers to break classical encryption. As such, new challenges are arising from our need for large-scale deployment of QKD systems. In a realistic scenario, transmitting and receiving devices from different vendors should be able to communicate with each other without the need for matching hardware. Therefore, practical deployment of QKD would require hardware capable of adapting to different protocols and clock rates. Here, we address this challenge by presenting a multi-rate, multi-protocol QKD transmitter linked to a correspondingly adaptable QKD receiver. The flexibility of the transmitter, achieved by optical injection locking, allows us to connect it with two receivers with inherently different clock rates. Furthermore, we demonstrate the multi-protocol operation of our transmitter, communicating with receiving parties employing different decoding circuits.
Stacked Multimodal Attention Network for Context-Aware Video Captioning

Recent neural models for video captioning usually employ an attention-based encoder-decoder framework. However, current approaches mainly attend to the motion features and object features of the video when generating the caption, but ignore the potential but useful historical information. Besides, exposure bias and vanishing gradients problems always exist in current caption generation models. In this paper, we propose a novel video captioning framework, named Stacked Multimodal Attention Network (SMAN). It adopts additional visual and textual historical information during caption generation as context features, employs a stacked architecture to process different features gradually, and utilizes the Reinforcement Learning method and coarse-to-fine training strategy to further improve the generated results. Both quantitative and qualitative experiments on the benchmark datasets of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MSVD</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MSR-VTT</i> show the effectiveness and feasibility of our framework. The codes are available on <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/zhengyi123456/SMAN</uri> .
Classifier-adaptation knowledge distillation framework for relation extraction and event detection with imbalanced data

Fundamental information extraction tasks, such as relation extraction and event detection, suffer from a data imbalance problem. To alleviate this problem, existing methods rely mostly on well-designed loss functions to reduce the negative influence of imbalanced data. However, this approach requires additional hyper-parameters and limits scalability. Furthermore, these methods can only benefit specific tasks and do not provide a unified framework across relation extraction and event detection. In this paper, a Classifier-Adaptation Knowledge Distillation (CAKD) framework is proposed to address these issues, thus improving relation extraction and event detection performance. The first step is to exploit sentence-level identification information across relation extraction and event detection, which can reduce identification errors caused by the data imbalance problem without relying on additional hyper-parameters. Moreover, this sentence-level identification information is used by a teacher network to guide the baseline model’s training by sharing its classifier. Like an instructor, the classifier improves the baseline model’s ability to extract this sentence-level identification information from raw texts, thus benefiting overall performance. Experiments were conducted on both relation extraction and event detection using the Text Analysis Conference Relation Extraction Dataset (TACRED) and Automatic Content Extraction (ACE) 2005 English datasets, respectively. The results demonstrate the effectiveness of the proposed framework.
Entity summarization: State of the art and future challenges

The increasing availability of semantic data has substantially enhanced Web applications. Semantic data such as RDF data is commonly represented as entity-property-value triples. The magnitude of semantic data, in particular the large number of triples describing an entity, could overload users with excessive amounts of information. This has motivated fruitful research on automated generation of summaries for entity descriptions to satisfy users’ information needs efficiently and effectively. We focus on this prominent topic of entity summarization, and our research objective is to present the first comprehensive survey of entity summarization research. Rather than separately reviewing each method, our contributions include (1) identifying and classifying technical features of existing methods to form a high-level overview, (2) identifying and classifying frameworks for combining multiple technical features adopted by existing methods, (3) collecting known benchmarks for intrinsic evaluation and efforts for extrinsic evaluation, and (4) suggesting research directions for future work. By investigating the literature, we synthesized two hierarchies of techniques. The first hierarchy categories generic technical features into several perspectives: frequency and centrality, informativeness, and diversity and coverage. In the second hierarchy we present domain-specific and task-specific technical features, including the use of domain knowledge, context awareness, and personalization. Our review demonstrated that existing methods are mainly unsupervised and they combine multiple technical features using various frameworks: random surfer models, similarity-based grouping, MMR-like re-ranking, or combinatorial optimization. We also found a few deep learning based methods in recent research. Current evaluation results and our case study showed that the problem of entity summarization is still far from being solved. Based on the limitations of existing methods revealed in the review, we identified several future directions: the use of semantics, human factors, machine and deep learning, non-extractive methods, and interactive methods.
Multi-Objective Software Effort Estimation: A Replication Study

Replication studies increase our confidence in previous results when the findings are similar each time, and help mature our knowledge by addressing both internal and external validity aspects. However, these studies are still rare in certain software engineering fields. In this paper, we replicate and extend a previous study, which denotes the current state-of-the-art for multi-objective software effort estimation, namely CoGEE. We investigate the original research questions with an independent implementation and the inclusion of a more robust baseline (LP4EE), carried out by the first author, who was not involved in the original study. Through this replication, we strengthen both the internal and external validity of the original study. We also answer two new research questions investigating the effectiveness of CoGEE by using four additional evolutionary algorithms (i.e., IBEA, MOCell, NSGA-III, SPEA2) and a well-known Java framework for evolutionary computation, namely JMetal (rather than the previously used R software), which allows us to strengthen the external validity of the original study. The results of our replication confirm that: (1) CoGEE outperforms both baseline and state-of-the-art benchmarks statistically significantly ( <inline-formula><tex-math notation="LaTeX">$p &lt;0.001$</tex-math></inline-formula> ); (2) CoGEE’s multi-objective nature makes it able to reach such a good performance; (3) CoGEE’s estimation errors lie within claimed industrial human-expert-based thresholds. Moreover, our new results show that the effectiveness of CoGEE is generally not limited to nor dependent on the choice of the multi-objective algorithm. Using CoGEE with either NSGA-II, NSGA-III, or MOCell produces human competitive results in less than a minute. The Java version of CoGEE has decreased the running time by over 99.8 percent with respect to its R counterpart. We have made publicly available the Java code of CoGEE to ease its adoption, as well as, the data used in this study in order to allow for future replication and extension of our work.
Attack to Fool and Explain Deep Networks

Deep visual models are susceptible to adversarial perturbations to inputs. Although these signals are carefully crafted, they still appear noise-like patterns to humans. This observation has led to the argument that deep visual representation is misaligned with human perception. We counter-argue by providing evidence of human-meaningful patterns in adversarial perturbations. We first propose an attack that fools a network to confuse a whole category of objects (source class) with a target label. Our attack also limits the unintended fooling by samples from non-sources classes, thereby circumscribing human-defined semantic notions for network fooling. We show that the proposed attack not only leads to the emergence of regular geometric patterns in the perturbations, but also reveals insightful information about the decision boundaries of deep models. Exploring this phenomenon further, we alter the 'adversarial' objective of our attack to use it as a tool to 'explain' deep visual representation. We show that by careful channeling and projection of the perturbations computed by our method, we can visualize a model's understanding of human-defined semantic notions. Finally, we exploit the explanability properties of our perturbations to perform image generation, inpainting and interactive image manipulation by attacking adversarialy robust 'classifiers'. In all, our major contribution is a novel pragmatic adversarial attack that is subsequently transformed into a tool to interpret the visual models. The article also makes secondary contributions in terms of establishing the utility of our attack beyond the adversarial objective with multiple interesting applications.
AutoAblation

Ablation studies provide insights into the relative contribution of different architectural and regularization components to machine learning models' performance. In this paper, we introduce AutoAblation, a new framework for the design and parallel execution of ablation experiments. AutoAblation provides a declarative approach to defining ablation experiments on model architectures and training datasets, and enables the parallel execution of ablation trials. This reduces the execution time and allows more comprehensive experiments by exploiting larger amounts of computational resources. We show that AutoAblation can provide near-linear scalability by performing an ablation study on the modules of the Inception-v3 network trained on the TenGeoPSAR dataset.
Detection of Polyps in Colonoscopic Videos Using Saliency Map-Based Modified Particle Filter

In this article, an automatic polyp detection system for endoscopic video frames is proposed. Manual inspection of each frame for polyp localization in the colonoscopic video has many adversaries. This work proposes a real-time tracking framework for polyp region segmentation in hugely acquired colonoscopic video frames. In our work, the polyp region in the frame is roughly detected by a saliency map at first, followed by a modified tracking mechanism for localization. The work suggests the use of a visual saliency map as the measurement model for tracking. The saliency map is composed of four probability maps generated by incorporating the characteristics associated with the polyps. The elliptical shape of the polyps is used by the particles for final refinement using an active contour (AC) model. The tracking efficiency and the segmentation score achieved using the proposed method suggest that our method can be used for polyp detection and localization. The proposed method achieves an average dice score of 66.06% in the CVC clinic Database. Our method can be employed in both online as well as off-line endoscopic video sequences. A GUI is also designed using the proposed method as an automatic polyp detection system.