Reviewerly/exHarmony huggingface dataset:
Continuous Streaming Multi-Talker ASR with Dual-Path Transducers

Streaming recognition of multi-talker conversations has so far been evaluated only for 2-speaker single-turn sessions. In this paper, we investigate it for multi-turn meetings containing multiple speakers using the Streaming Unmixing and Recognition Transducer (SURT) model, and show that naively extending the single-turn model to this harder setting incurs a performance penalty. As a solution, we propose the dual-path (DP) modeling strategy first used for time-domain speech separation. We experiment with LSTM and Transformer based DP models, and show that they improve word error rate (WER) performance while yielding faster convergence. We also explore training strategies such as chunk width randomization and curriculum learning for these models, and demonstrate their importance through ablation studies. Finally, we evaluate our models on the LibriCSS meeting data, where they perform competitively with offline separation-based methods.
Generalization Self-distillation with Epoch-wise Regularization

Recent advances in deep neural network have achieved remarkable successes in various computer vision tasks. However, deep neural network with millions of parameters may suffer from poor generalization due to overfitting. To improve the generalization performance, many methods have been proposed such as data augmentation, label smoothing and knowledge distillation. In this paper, we extend self-knowledge distillation to enhance the model generalization performance without incurring extra computation cost, called EWR-KD, which is a simple yet effective method to progressively distill knowledge from the model itself. Concretely, it consists of two components: 1) the self-distillation scheme that progressively softens the learning targets by using the past model prediction; 2) the sample-reweighting scheme that dynamically decides the trust degree to transfer more informative knowledge by introducing uncertainty estimation. With the two components, EWR-KD is robust to both corrupt noises and adversarial noises, and can be easily combined with current advanced regularization techniques. We theoretically show that EWR-KD minimizes cross-entropy by adding an epoch-wise regularization, which measures the difference between the past model prediction at ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$t-1$</tex> )-th epoch and the current prediction at <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$t$</tex> -th epoch. Finally, Extensive experimental results on clean datasets and noisy datasets empirically demonstrate that EWR-KD not only improves the performance of the state-of-the-art baseline but also yields well calibration.
Deep Embedded Clustering with ResNets

Clustering is an AI technique that has been successfully applied to the abundance of unlabelled real-world data for revealing hidden patterns and knowledge extraction. Deep Embedded Clustering (DEC) is a deep Autoencoder (AE) based model that learns feature representations and cluster assignments simultaneously. DEC learns the mapping from input data to a low-dimensional embedded space through joint optimization of feature transformation and clustering. Our previous work demonstrates how adding residual connections to deep AEs (RAEs) reduces the performance degradation of learned features when performing downstream classification on learned features. Further, it evidenced that RAE has improved unsupervised feature learning capability compared to AE. In this paper, we are evaluating the effect of residual connections in the context of Deep Embedded Clustering (DEC), which we refer to as RDEC. RDEC was compared against regular DEC. We considered various numbers of hidden layers and several bench-mark datasets: MNIST, Fashion MNIST, Reuters, and Human activity recognition. When increasing the depth of the neural network gradually, the presented RDEC showed up to 56% of less performance degradation compared to DEC. Further, the distribution of clustering accuracies showed that the presented RDEC outperforms DEC when comparing the accuracy variance and mean accuracy.
Adaptive Adversarial Training for Meta Reinforcement Learning

Meta Reinforcement Learning (MRL) enables an agent to learn from a limited number of past trajectories and extrapolate to a new task. In this paper, we attempt to improve the robustness of MRL. We build upon model-agnostic meta-learning (MAML) and propose a novel method to generate adversarial samples for MRL by using Generative Adversarial Network (GAN). That allows us to enhance the robustness of MRL to adversal attacks by leveraging these attacks during meta training process.
Explaining Image Misclassification in Deep Learning via Adversarial Examples

With the increasing use of convolutional neural networks (CNNs) for computer vision and other artificial intelligence tasks, the need arises to interpret their predictions. In this work, we tackle the problem of explaining CNN misclassification of images. We propose to construct adversarial examples that allow identifying the regions of the input images that had the largest impact on the CNN wrong predictions. More specifically, for each image that was incorrectly classified by the CNN, we implemented an inverted adversarial attack consisting on modifying the input image as little as possible so that it becomes correctly classified. The changes made to the image to fix classification errors explain the causes of misclassification and allow adjusting the model and the data set to obtain more accurate models. We present two methods, of which the first one employs the gradients from the CNN itself to create the adversarial examples and is meant for model developers. However, end users only have access to the CNN model as a black box. Our second method is intended for end users and employs a surrogate model to estimate the gradients of the original CNN model, which are then used to create the adversarial examples. In our experiments, the first method achieved 99.67% success rate at finding the misclassification explanations and needed on average 1.96 queries per misclassified image to build the corresponding adversarial example. The second method achieved 73.08% success rate at finding the explanations with 8.73 queries per image on average.
An Efficient Channel-level Pruning for CNNs without Fine-tuning

The success of CNNs in various applications is accompanied by a significant increase in the computation and parameter storage costs. Model compression techniques are able to remove a significant fraction of network parameters to reduce the costs. Channel pruning is among the predominant approaches to compress networks. The typical framework of existing channel pruning methods consists of three steps: training a dense network, pruning redundant parameters, and fine-tuning to increase model accuracy. However, the fine-tuning step usually takes a long time, which is even close to the training costs. Besides, the decoupling of training and pruning leads to that many unimportant parameters that will be removed later still need to be trained. To address these issues, we propose the Dynamic Mask-based Channel Pruning (DMCP) method in this study. The algorithm zeros out unimportant channels with mask vectors and prunes the redundant weights during training. It achieves comparable model accuracy with existing methods, without the fine-tuning step. Moreover, DMCP prunes parameters and reconfigures models during training, so that the number of operations for training useless parameters is reduced. In our evaluations, DMCP removes up to 82% parameters for VGG16, VGG19, and ResNet18 on CIFAR10 dataset, and it reduces up to 58% floating point operation (FLOPs) of training.
Unsupervised Pretraining of Echo State Networks for Onset Detection

Note onset detection – the detection of the beginning of new note events – is a fundamental task for music analysis that can help to improve Automatic Music Transcription (AMT). The method for onset detection always follows a similar outline: An audio signal is transformed into an Onset Detection Function (ODF), which should have rather low values (i.e. close to zero) for most of the time, and pronounced peaks at onset times, which can then be extracted by applying peak picking algorithms on the ODF. Currently, Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs) define the state of the art. In this paper, we build upon previous work about onset detection using Echo State Networks (ESNs) that have achieved comparable results to CNNs. We show that unsupervised pre-training of the ESN leads to similar results whilst reducing the model complexity.
Empirical Analysis of Performance of MT Systems and Its Metrics for English to Bengali: A Black Box-Based Approach

There are numerous use cases of machine translation (MT) systems. Therefore, it has become very important to evaluate the performance of MT which can help researchers design a robust and reliable machine translation system. Although there are number of automatic MT evaluation metrics available nowadays, most of these fail to produce correct score. In this paper, we shall describe the most recent framework in MT industry that is Neural Machine Translation, and our approach is to test the performances of most popular translators—Google Translate and Microsoft’s Bing translator. We use language pairs English and Bengali for our detail analysis. Experiments are performed to translate English to Bengali. Bengali is a resource poor and one of the most widely spoken language in Indian subcontinent. Unlike glass box approach, where the performance of the MT systems are tried to enhance by adjusting various hyper parameters, we will be using black box approach, i.e., evaluating the performance of the already built systems. Beside performance analysis of the two translators, our main focus is to evaluate the performance of one of the very popular automatic evaluators Bilingual Under Study (BLEU) and some other automatic evaluation metrics in our work. This paper aims to measure the performance of BLEU and other automatic metrics during English to Bengali translation process by conducting survey with the help of questionnaires distributed among twenty people having moderate to high linguistic expertise on both the languages. Their responses are collected, and mean score is calculated which we have considered as human score (human judgement). BLEU and scores generated by other automatic metrics are used and compared their performance with human-generated score. A correlation is measured between human score and other metrics with Pearson correlation coefficient. Finally, some important observations are reported for this language pairs.
Underwater Image Enhancement Using Deep Learning

Over the last few years, academicians all over the world have been studying about the underwater images and the ability to obtain crystal clear images. Along with this, the entire process of restoring the obtained images is a tedious task. Due to the scientific phenomena of absorption along with scattering, the obtained underwater images suffer from few defects. The notable problems in these images are: a distortion in the original color, blurriness, and the effects of low contrast. Overcoming these shortfalls is a herculean task for researchers in image processing domain. Light travels in a constricted pathway, when it is gone through water. Here, the bigger frequencies are influenced more, when compared with the more limited frequencies; hence, pictures get submerged and becomes greenish blue as they come up short on certain frequency parts. For example, if a picture obtained at a profundity of around 4–5 m submerged, then it will need red frequency on the grounds that the more extended frequency segments of the apparent range are weakened first. With additional increment, other frequency segments will likewise begin to lose importance. The pictures accordingly experience the ill effects of restricted perceivability range, lopsided lighting, and presence of splendid antiquities. To overcome this, the proposed research work uses deep learning model to enhance the underwater images.
Wavelength-tuning interferometry with suppression of dispersive error

In measurements of the variations in the thickness and surface profile of optical flats using wavelength-tuning fringe analysis, an interference signal can be detected based on the modulation frequency of the signal. However, the modulation frequency can deviate from the predetermined value through the refractive-index dispersion of the optical flat. In this study, two types of noble 25-frame phase-detection algorithms that can compensate for the effect of refractive-index dispersion were developed using Fourier analysis. The behaviors of the 25-frame algorithms were visualized on both the frequency domain and complex plane. The dispersion-resistance ability and signal-to-noise ration of the 25-frame algorithm were estimated by calculating the PV errors and using the frequency-transfer-function paradigm, respectively. Finally, the variations in the thickness and surface profile of the optical flat were measured using a wavelength-tuning Fizeau interferometer and the proposed 25-frame algorithms. The standard deviations of the thickness and surface measurements were 1.654 nm and 8.368 nm, respectively, which were much smaller than those of the conventional phase-detection algorithms.
Robust Image Hashing Using Chromatic Channel

Image security remains a challenging issue in modern age of multimedia communication. In recent days, image hashing technique has gained a lot attentions because of the availability of different multimedia editing tools. Color image hashing has been explored limited. In this work, chromatic channel information has been used to construct hashing. The ring partition techniques are generally designed on the intensity channel. Chromatic information of an images are also added along with intensity channel information to create an image hash that is sensitive to global as well as local color changes. Experiment has been conducted on different standard database and results illustrate capability of the proposed method for detection of color forgery, while maintaining robustness against operations which preserve content and good discrimination. The efficacy of the proposed method is shown using receiver operating characteristics.
New customizable digital data container for optical cryptosystems

Quick response (QR) codes have recently become a popular data container for optical encryption systems. Their effectiveness is based on the error-correction codes, that allow them to decode data even when the media itself is significantly damaged. However, QR codes were developed for specific applications and have a significant number of elements that are superfluous for optical encryption. In this paper we present a new customizable digital data container (CDDC), designed to replace QR codes in optical encryption. CDDC is based on the error-correction codes too, but has a wide range of configurable parameters and can be specialized for a particular optical setup. The results of noise robustness analysis, numerical and optical experiments on application of the new container in a spatially incoherent optical encryption system demonstrate the supremacy of CDDC in comparison to QR codes in terms of error correction capabilities and data density.
Challenges and Implications of the WHO’s Digital Cross-Border COVID-19 Vaccine Passport Recognition Pilot

As the development and distribution of COVID-19 vaccinations continues, the concept of a digital Yellow Card or smart vaccine passport has been discussed. Verifying an individual’s COVID-19 vaccination would improve cross-border mobility and limit the potential spread of the virus. Recently, the WHO signed a Memorandum of Understanding with the Government of Estonia for providing expertise and technical solutions like the interoperable cross-border data exchange ecosystem, X-Road, for piloting a smart vaccine passport trust architecture. Subsequently, the WHO is in the beginning stages of undergoing a process of digital transformation to accommodate the pilot (DT). In this paper, the authors present an exploratory case study using qualitative workshop methods and data triangulation to investigate the technical, political and organizational DT processes occurring in the pilot project. The results suggest the proposed cross-border data exchange trust architecture would exchange non-sensitive personal health information. Some WHO countries are willing to participate and engage in the pilot, but political and legal barriers remain for large-scale implementation. Ultimately, the potential establishment of a trusted global architecture health framework, where not only COVID-19 vaccine data can be exchanged, but new cross-border health services can be created, warrants further investigation.
HRotatE: Hybrid Relational Rotation Embedding for Knowledge Graph

Knowledge Graph represents the real world's information in the form of triplets (head, relation, and tail). However, most Knowledge Graphs are highly incomplete. The goal of a Knowledge-Graph Completion task is to predict missing links in a given Knowledge Graph. Various approaches exist to predict a missing link in a Knowledge Graph, but the most prominent approaches are based on tensor factorization and Knowledge-Graph embeddings, such as RotatE and SimplE. The RotatE model depicts each relation as a rotation from the source entity (Head) to the target entity (Tail) via a complex vector space. In RotatE, the head and tail entities are derived from one embedding-generation class, resulting in a relatively low prediction score. SimplE is primarily based on a Canonical Polyadic (CP) decomposition. SimplE enhances the CP approach by adding the inverse relation where head embedding and tail embedding are taken from the different embedding-generation class, but they are still dependent on each other. However, SimplE is not able to predict composition patterns. This paper presents a new, hybridized variant (HRotatE) of the existent RotatE approach. Essentially, HRotatE is hybridized from RotatE and SimplE. We have used the principle of inverse embedding (from the SimplE model) in a bid to improve the prediction scores of HRotatE. Hence, our results have proven to be better than the native RotatE. Also, HRotatE outperforms several state-of-the-art models on different datasets. Conclusively, our proposed approach (HRotatE) is relatively efficient such that it utilizes half the number of training steps required by RotatE, and it generates approximately the same result as RotatE.
Towards Lower Precision Quantization for Pedestrian Detection in Crowded Scenario

Automatic pedestrian detection in real-world un-cooperative scenarios is a well-known problem in computer vision, which has again gained in visibility last year due to distancing requirements. This remains a very challenging task, especially in crowded areas. Due to diverse technical and privacy issues, embedded systems such as smart cameras and smaller drones are becoming ubiquitous. Those complex detection models are not designed for on-edge processing in resource-constrained environments. Therefore, quantization techniques are required, in order to reduce the weights of a model to low-precision and not only effectively compress the model, but also allow to use low bitwidth arithmetic, which in term can be accelerated from specialized hardware. However, using an effective quantization scheme while maintaining accuracy is challenging. In this work we first establish a Quantization-aware training (QAT) and Post-training Quantization (PTQ) baseline for 8-bit uniform quantization to RetinaNet for person detection on the extremely challenging PANDA dataset. Those achieve near lossless performance in terms of accuracy by about 5× speed-up of the CPU inference and 4× model size reduction for 8-bit PTQ quantized model. Further experiments with aggressive quantization scheme in 4- and 2-bit show diverse challenges resulting in severe instabilities. We apply both uniform and non-uniform quantization to overcome those and provide insights and strategies to fully quantize in 4- and 2-bit. Through this process we systematically evaluate the sensibility of individual parts of RetinaNet for quantization in very low precision. Finally, we show the resistance of quantization for limited amount of data.
SINETStream: Enabling Research IoT Applications with Portability, Security and Performance Requirements

Demands for Internet of Things (IoT) platforms are increasing along with the expansion of data-driven sciences, and academic network infrastructure such as national research and education networks (NRENs) are now being encouraged to aggressively support diverse IoT research projects. In Japan, the National Institute of Informatics (NII) operates an NREN called "SINET5," which is an academic backbone network linking more than 900 universities and research institutions. In addition to SINET5's high-speed 100 Gbps backbone network, SINET5 also provides a mobile network called "Mobile SINET" as IoT infrastructure. To provide crucial support to IoT research projects and to facilitate the development and deployment of IoT applications, we present herein our experience with our software library, "SINETStream." In this paper, we provide an overview of SINETStream and discuss lessons learned from examples of application deployment over Mobile SINET. SINETStream provides a common and simple application program interface (API) for various message brokers, security functions, and performance tuning support features. These functions improve the portability of applications and help application developers to remove hindrances to the development of secure and efficient IoT applications. The experimental results provided herein show that application developers can use SINETStream functions within a reasonable overhead. We also show how the combination of SINETStream and Mobile SINET enables users to develop and deploy highly confidential and efficient IoT applications.
Fundamental limits of over-the-air optimization: Are analog schemes optimal?

We consider convex optimization on a <tex>$d$</tex> dimensional space where coded gradients are sent over an additive Gaussian noise channel with variance <tex>$\sigma^{2}$</tex>. The codewords satisfy an average power constraint <tex>$P$</tex>, resulting in the signal-to-noise ratio (SNR) of <tex>$P/\sigma^{2}$</tex>. Many schemes have been proposed for this problem, termed over-the-air optimization, in recent years. We present lower and upper bounds for the convergence rates for over-the-air optimization. Our first result is a lower bound for the convergence rate showing that any code must slowdown the convergence rate by a factor of roughly <tex>$\sqrt{d/\log(1+\text{SNR})}$</tex>. Next, we consider a popular class of schemes called analog coding, where a linear function of the gradient is sent. We show that a simple scaled transmission analog coding scheme results in a slowdown in convergence rate by a factor of <tex>$\sqrt{d(1+1/\text{SNR})}$</tex>. This matches the previous lower bound up to constant factors for low SNR, making the scaled transmission scheme optimal at low SNR. However, we show that this slowdown is necessary for any analog coding scheme. In particular, a slowdown in convergence by a factor of <tex>$\sqrt{d}$</tex> remains even when SNR tends to infinity, a clear shortcoming of analog coding schemes at high SNR. Remarkably, we present a simple quantize-and-modulate scheme that uses Amplitude Shift Keying and almost attains the optimal convergence rate at all SNRs.