Reviewerly/exHarmony huggingface dataset:
Leveraging Language ID in Multilingual End-to-End Speech Recognition

Recent advances in end-to-end speech recognition have made it possible to build multilingual models, capable of recognizing speech in multiple languages. Multilingual models can outperform their monolingual counterparts, depending on the amount of training data and the relatedness of languages. However, in some cases, these models rely on having perfect knowledge of the language being spoken; that is, they expect to be provided with an external language ID that augments the input features or modulates internal layers of the network. In this paper, we introduce a novel technique for inferring the language ID in a streaming fashion using RNN-T, and a novel loss function that pressures the model to identify the language after as few frames as possible. The output of this streaming language-ID model is used in training and inference of a multilingual recognition model. We show the effectiveness of our approach through experiments on two sets of languages, one consisting of different dialects of Arabic, and the other consisting of Nordic languages, Finnish and Dutch.
Optimised robust watermarking technique using CKGSA in DCT‐SVD domain

Digital watermarking embeds a watermark to minimise the problem of illegal copying and disseminating multimedia contents. However, the existing techniques do not maintain the imperceptibility and robustness simultaneously. To achieve the same, this study proposes an optimised robust watermarking technique using chaotic kbest gravitational search algorithm. The chaotic kbest gravitational search algorithm is used to obtain the optimal values of embedding factors. The efficacy of the proposed technique has been experimented on a standard images and compared with the six recent state-of-the-art techniques in terms of imperceptibility and robustness. The experimental results validate that the proposed technique outperforms the other considered techniques.
Are Accelerometers for Activity Recognition a Dead-end?

Accelerometer-based (and by extension other inertial sensors) research for Human Activity Recognition (HAR) is a dead-end. This sensor does not offer enough information for us to progress in the core domain of HAR - to recognize everyday activities from sensor data. Despite continued and prolonged efforts in improving feature engineering and machine learning models, the activities that we can recognize reliably have only expanded slightly and many of the same flaws of early models are still present today. Instead of relying on acceleration data, we should instead consider modalities with much richer information - a logical choice are images. With the rapid advance in image sensing hardware and modelling techniques, we believe that a widespread adoption of image sensors will open many opportunities for accurate and robust inference across a wide spectrum of human activities. In this paper, we make the case for imagers in place of accelerometers as the default sensor for human activity recognition. Our review of past works has led to the observation that progress in HAR had stalled, caused by our reliance on accelerometers. We further argue for the suitability of images for activity recognition by illustrating their richness of information and the marked progress in computer vision. Through a feasibility analysis, we find that deploying imagers and CNNs on device poses no substantial burden on modern mobile hardware. Overall, our work highlights the need to move away from accelerometers and calls for further exploration of using imagers for activity recognition.
Seamless Virtual Whole Slide Image Synthesis and Validation Using Perceptual Embedding Consistency

Stain virtualization is an application with growing interest in digital pathology allowing simulation of stained tissue images thus saving lab and tissue resources. Thanks to the success of Generative Adversarial Networks (GANs) and the progress of unsupervised learning, unsupervised style transfer GANs have been successfully used to generate realistic, clinically meaningful and interpretable images. The large size of high resolution Whole Slide Images (WSIs) presents an additional computational challenge. This makes tilewise processing necessary during training and inference of deep learning networks. Instance normalization has a substantial positive effect in style transfer GAN applications but with tilewise inference, it has the tendency to cause a tiling artifact in reconstructed WSIs. In this paper we propose a novel perceptual embedding consistency (PEC) loss forcing the network to learn color, contrast and brightness invariant features in the latent space and hence substantially reducing the aforementioned tiling artifact. Our approach results in more seamless reconstruction of the virtual WSIs. We validate our method quantitatively by comparing the virtually generated images to their corresponding consecutive real stained images. We compare our results to state-of-the-art unsupervised style transfer methods and to the measures obtained from consecutive real stained tissue slide images. We demonstrate our hypothesis about the effect of the PEC loss by comparing model robustness to color, contrast and brightness perturbations and visualizing bottleneck embeddings. We validate the robustness of the bottleneck feature maps by measuring their sensitivity to the different perturbations and using them in a tumor segmentation task. Additionally, we propose a preliminary validation of the virtual staining application by comparing interpretation of 2 pathologists on real and virtual tiles and inter-pathologist agreement.
New methodology for calculating cost-efficiency of different ways of voting: is internet voting cheaper?

New ways of voting in elections are being sought by electoral administrations worldwide who want to reverse declining voter turnouts without increasing electoral budgets. This paper presents a novel approach to cost accounting for multi-channel elections based on local elections in Estonia. By doing so, it addresses an important gap in the academic literature in this field. The authors confirm that internet voting was most cost-efficient voting channel offered to Estonian voters.
Cryptocurrency Mechanisms for Blockchains: Models, Characteristics, Challenges, and Applications

In recent years, blockchains have drawn considerable attention from engineers, researchers, and institutions alike; and their implementation has affected fields like e-healthcare, Internet of Things, smart contracts, e-finance, logistics, and so forth. The focus of this chapter shall be the cryptocurrency mechanisms for blockchains. Cryptocurrencies are a new form of virtual currency, first introduced to the masses with the creation of bitcoin, developed by Satoshi Nakamoto. They are a more secure and verified form of transaction, which uses cryptographic protocols in a peer-to-peer system generating a distributed ledger. With the growing popularity of blockchains, a large number of cryptocurrencies have been introduced, greatly changing the prospects of blockchains in the domain of finance. Greater attention is now being paid by stakeholders around the world to various mechanisms of using blockchains in cryptocurrencies. In these blockchain systems, all the committed transactions are stored in a list of blocks with several transactional properties including irreversibility, pseudonymous, security, and global scalability. As both permission and permissionless blockchain platforms are being challenged to meet rigorous real-world application requirements, such as immediate transaction finality, low latencies, good scalability, and high-performance, the limitations of existing consensus models are being recognized. To achieve the necessary agreement in a network among distributed processes, consensus mechanisms are used as fault-tolerant mechanisms in blockchains systems. Many consensus models have been designed and new ones are still emerging to combat the limitations of their predecessors. Models like Proof-of-Work, Proof-of-Stake, Proof-of-Elapsed Time, and many others that follow open-ended or closed-ended participation have their own advantages and limitations. The consensus mechanism maintains the sanctity of the data and the key properties of the blockchain; the security of the consensus model is perhaps an important measure of the quality of the model. Poor choice of consensus mechanism could lead to blockchain fork, consensus failure, dominance, and poor performance thereby compromising the data recorded on the blockchain. Despite all these challenges, blockchains show immense potential in improving cryptocurrency by bringing numerous opportunities. This chapter provides a primer on the basics of cryptocurrencies using blockchain technology and discusses the existing narratives about the technology's potential to facilitate efficient mechanisms for financial transactions among others. The principal disciples of the technology are discussed in great detail while also flagging the potential points of concern and conflict surrounding the technology. It concludes with suggestions for future research.
The ASSIN 2 Shared Task: A Quick Overview

This paper offers a brief overview on the ASSIN 2, an evaluation shared task collocated with STIL 2019. ASSIN 2 covered two different but related tasks: Recognizing Textual Entailment (RTE), also known as Natural Language Inference (NLI), and Semantic Textual Similarity (STS). The ASSIN 2 collection was made of pairs of sentences annotated with human judgments for NLI and STS. Participating teams could take part in any of the tasks or both: nine teams participated in the STS task and eight in the NLI task.
U-Net Conditional GANs for Photo-Realistic and Identity-Preserving Facial Expression Synthesis

Facial expression synthesis (FES) is a challenging task since the expression changes are highly non-linear and depend on the facial appearance. Person identity should also be well preserved in the synthesized face. In this article, we present a novel U-Net Conditional Generative Adversarial Network for FES. U-Net helps retain the property of the input face, including the identity information and facial details. Category condition is added to the U-Net model so that one-to-many expression synthesis can be achieved simultaneously. We also design constraints for identity preservation during FES to further guarantee that the identity of the input face can be well preserved in the generated face image. Specifically, we pair the generated output with condition image of other identities for the discriminator, so as to encourage it to learn the distinctions between the synthesized and natural images, as well as between input and other identities, which can help improve its discriminating ability. Additionally, we utilize the triplet loss to maintain the generated face images closer to the same identity person by imposing a margin between the positive pairs and negative pairs in feature space. Both qualitative and quantitative evaluations are conducted on the Oulu-CASIA NIR8VIS facial expression database, the Radboud Faces Database, and the Karolinska Directed Emotional Faces database, and the experimental results show that our method can generate faces with natural and realistic expressions while preserving identity information.
ARC: Anomaly-aware Robust Cloud-integrated IoT service composition based on uncertainty in advertised quality of service values

From the IoT perspective, each intelligent device can be considered as a potential source of service. Since several services perform the same function, albeit with different quality of service (QoS) parameters, service composition becomes a crucial problem to find an optimal set of services to automate a typical business process. The majority of prior research has investigated the service composition problem with the assumption that advertised QoS values are deterministic and do not change over time. However, factors like sensors failure and network topology changes cause uncertainty in the advertised QoS values. To address this challenge, we propose a novel Anomaly-aware Robust service Composition (ARC) to deal with the problem of uncertainty of QoS values in a dynamic environment of Cloud and IoT. The proposed approach uses Bertsimas and Sim mathematical robust optimization method, which is independent of the statistical distribution of QoS values, to compose services. Moreover, our approach exploits a machine learning-based anomaly detection technique to improve the stability of the solution with a fine-grained identification of abnormal QoS records. The results demonstrate that our approach achieves 14.55% of the average improvement in finding optimal solutions compared to the previous works, such as information theory-based and clustering-based methods.
Cumulative Rain Density Sensing Network for Single Image Derain

This paper focuses on single image derain, which aims to restore clear image from single rain image. Through full consideration of different frequency information preservation and the complicated interactions between rain-streaks and background, a novel end-to-end cumulative rain-density sensing network (CRDNet) is proposed for adaptive rain-streaks removal. An effective W-Net with powerful learning ability is proposed as a key component to recover rain-invariant low-frequency signals. A cumulative rain-density classifier with a novel cost-sensitive label encoding strategy is proposed as an auxiliary network to improve discriminative power of extracted high-frequency rain-streaks through multi-task training. The proposed CRDNet has been compared with state-of-the-art methods on two public datasets. The quantitative and visual experimental results demonstrate that it can achieve excellent performance with great improvement. Related source code and models are available on github https://github.com/peylnog/CRDNet.
Privacy-Sensitive Parallel Split Learning

Mobile devices and medical centers have access to rich data that is suitable for training deep learning models. However, these highly distributed datasets are privacy sensitive making privacy issues for applying deep learning techniques to the problem at hand. Split Learning can solve these data privacy problems, but the possibility of overfitting exists because each node doesn't train in parallel but in a sequential manner. In this paper, we propose a parallel split learning method that prevents overfitting due to differences in a training order and data size by the node. Our method selects mini-batch size considering the amount of local data on each node and synchronizes the layers that nodes have during the training process so that all nodes can use the equivalent deep learning model when the training is complete.
Stochastic Optimization of Contextual Neural Networks with RMSprop

The paper presents modified version of Generalized Error Backpropagation algorithm (GBP) merged with RMSprop optimizer. This solution is compared with analogous method based on Stochastic Gradient Descent. Both algorithms are used to train MLP and CxNN neural networks solving selected benchmark and real–life classification problems. Results indicate that usage of GBP-RMSprop can be beneficial in terms of increasing classification accuracy as well as decreasing activity of neurons’ connections and length of training. This suggests that RMSprop can effectively solve optimization problems of variable dimensionality. In the effect, merging GBP with RMSprop as well as with other optimizers such as Adam and AdaGrad can lead to construction of better algorithms for training of contextual neural networks.
End-to-End Speech Recognition in Agglutinative Languages

This paper considers end-to-end speech recognition systems based on deep neural networks (DNN). The studies used different types of neural networks, CTC model and attention-based encoder-decoder models. As a result of the study, it was proved that the CTC model works without language models directly for agglutinative languages, but the best is ResNet with 11.52% of CER and 19.57% of WER of using the language model. An experiment with the BLSTM neural network using the attention-based encoder-decoder models showed 8.01% of CER of and 17.91% of WER. Using the experiment, it was proved that without integrating language models, good results can be achieved. The best result showed ResNet.
Differentiable Scene Graphs

Reasoning about complex visual scenes involves perception of entities and their relations. Scene Graphs (SGs) provide a natural representation for reasoning tasks, by assigning labels to both entities (nodes) and relations (edges). Reasoning systems based on SGs are typically trained in a two-step procedure: first, a model is trained to predict SGs from images, and next a separate model is trained to reason based on the predicted SGs. However, it would seem preferable to train such systems in an end-to-end manner. The challenge, which we address here is that scene-graph representations are non-differentiable and therefore it isn't clear how to use them as intermediate components. Here we propose Differentiable Scene Graphs (DSGs), an image representation that is amenable to differentiable end-to-end optimization, and requires supervision only from the downstream tasks. DSGs provide a dense representation for all regions and pairs of regions, and do not spend modelling capacity on regions of the image that do not contain objects or relations of interest. We evaluate our model on the challenging task of identifying referring relationships (RR) in three benchmark datasets: Visual Genome, VRD and CLEVR. Using DSGs as an intermediate representation leads to new state-of-the-art performance. The full code is available at https://github.com/shikorab/DSG.
New Secure Approach to Backup Cryptocurrency Wallets

Bitcoin and other cryptocurrencies have become popular and motivate more hackers to steal digital funds. Users protect their private keys using crypto wallets to keep their funds safe from hackers. While the most secure option is hardware wallet, it suffers from lack of a secure and convenient backup and recovery process. Almost all existing wallets use mnemonics to back up the private keys, and a user must write down these words on a piece of paper. This approach is not only inconvenient but also problematic since the paper could be lost or stolen, resulting in a hacker recovering the keys. In this paper, we propose a new digital scheme to securely back up a hardware wallet relying on the side-channel human visual verification enabled by display screen on a hardware wallet. Using this method, we transfer the root of private keys from one hardware wallet to another wallet securely even via an untrusted terminal, such as a smartphone. At the end of this process, the user has two hardware wallets with the same private keys while she may use one of them as the main wallet and another one as a backup wallet.
Unpaired Image-to-Speech Synthesis With Multimodal Information Bottleneck

Deep generative models have led to significant advances in cross-modal generation such as text-to-image synthesis. Training these models typically requires paired data with direct correspondence between modalities. We introduce the novel problem of translating instances from one modality to another without paired data by leveraging an intermediate modality shared by the two other modalities. To demonstrate this, we take the problem of translating images to speech. In this case, one could leverage disjoint datasets with one shared modality, e.g., image-text pairs and text-speech pairs, with text as the shared modality. We call this problem “skip-modal generation” because the shared modality is skipped during the generation process. We propose a multimodal information bottleneck approach that learns the correspondence between modalities from unpaired data (image and speech) by leveraging the shared modality (text). We address fundamental challenges of skip-modal generation: 1) learning multimodal representations using a single model, 2) bridging the domain gap between two unrelated datasets, and 3) learning the correspondence between modalities from unpaired data. We show qualitative results on image-to-speech synthesis; this is the first time such results have been reported in the literature. We also show that our approach improves performance on traditional cross-modal generation, suggesting that it improves data efficiency in solving individual tasks.
DAVID: Dual-Attentional Video Deblurring

Blind video deblurring restores sharp frames from a blurry sequence without any prior. It is a challenging task because the blur due to camera shake, object movement and defocusing is heterogeneous in both temporal and spatial dimensions. Traditional methods train on datasets synthesized with a single level of blur, and thus do not generalize well across levels of blurriness. To address this challenge, we propose a dual attention mechanism to dynamically aggregate temporal cues for deblurring with an end-to-end trainable network structure. Specifically, an internal attention module adaptively selects the optimal temporal scales for restoring the sharp center frame. An external attention module adaptively aggregates and refines multiple sharp frame estimates, from several internal attention modules designed for different blur levels. To train and evaluate on more diverse blur severity levels, we propose a Challenging DVD dataset generated from the raw DVD video set by pooling frames with different temporal windows. Our framework achieves consistently better performance on this more challenging dataset while obtaining strongly competitive results on the original DVD benchmark. Extensive ablative studies and qualitative visualizations further demonstrate the advantage of our method in handling real video blur.
Efficient Foreign Object Detection Between PSDs and Metro Doors via Deep Neural Networks

Platform Screen Doors (PSDs) have been widely used in modern Asian and European metro systems due to the advantages of safety, comfort for passengers. Unfortunately, someone or something will be caught by PSDs and metro doors occasionally, which may lead to serious accidents. Therefore, the foreign object detection between PSDs and metro doors is a burning problem. Moreover, this problem is a challenging and still largely under-explored topic. In recent years, we have seen significant improvements in generic object detection built on deep learning techniques. Accordingly, this paper adopts deep learning technologies to address the problem of foreign object detection between PSDs and metro doors. To the best of our knowledge, this is the first attempt to use deep learning to solve the problem. To realize this, a dataset including 984 real-world images (with 600 × 480 pixels) labeled for six types of foreign objects (bag, bottle, person, plastic bag, umbrella, other) is developed. Then, we compared the performance of some state-of-the-art object detection algorithms (such as You Only Look Once -YOLOv3, Single Shot MultiBox Detector -SSD, and CenterNet) on the dataset. Experimental results demonstrate that the foreign object detection algorithms based on deep neural networks have achieved excellent results, which not only improves the accuracy of detection but also give the categories of foreign objects. YOLOv3 - tiny can achieve the fastest detection speed, up to 200 Frame Per Second (FPS); CenterNet can achieve the best detection results, up to 99.7% mean Average Precision (mAP).