Reviewerly/exHarmony huggingface dataset:
Improving Entity Recall in Automatic Speech Recognition with Neural Embeddings

Automatic speech recognition (ASR) systems often have difficulty recognizing long-tail entities such as contact names and local restaurant names, which usually do not occur, or occur infrequently, in the system’s training data. In this work, we present a method which uses learned text embeddings and nearest neighbor retrieval within a large database of entity embeddings to correct misrecognitions. Our text embeddings are produced by a neural network trained so that the embeddings of acoustically confusable phrases have low cosine distances. Given the embedding of the text of a potential entity misrecognition and a precomputed database containing entities and their corresponding embeddings, we use fast, scalable nearest neighbor retrieval algorithms to find candidate corrections within the database. The inserted candidates are then scored using a function of the original text’s cost in the lattice and the distance between the embedding of the original text and the embedding of the candidate correction. Using this lattice augmentation techique, we demonstrate a 46% reduction in word error rate (WER) and 46% reduction in oracle word error rate (OWER) on an evaluation set with popular film queries.
Integrity promised: leveraging blockchain technology for medical image sharing

Medical images are important for any healthcare organization. Several cloud solutions have been suggested for sharing medical images between organizations, but a third-party centralized cloud service cannot be trusted completely. On the other hand, blockchain is a technology that is decentralized and completely trustworthy. It enables a distributed system with no centralized authority. This technology allows us to share images in a more secure manner, thereby achieving integrity. The results are far better when we combine the merits of both cloud storage and blockchain. In this chapter, a new methodology is proposed that is a combination of both cloud storage and blockchain.
Adaptive Feature Weight Learning For Robust Clustering Problem with Sparse Constraint

Clustering task has been greatly developed in recent years like partition-based and graph-based methods. However, in terms of improving robustness, most existing algorithms only focus on noise and outliers between data, while ignoring the noise in feature space. To deal with this situation, we propose a novel weight learning mechanism to adaptively reweight each feature in the data. Combining with the clustering task, we further propose a robust fuzzy K-Means model based on the auto-weighted feature learning, which can effectively reduce the proportion of noisy features. Besides, a regularization term is introduced into our model to make the sample-to-clusters memberships of each sample have suitable sparsity. Specifically, we design an effective strategy to determine the value of the regularization parameter. The experimental results on both synthetic and real-world datasets demonstrate that our model has better performance than other classical algorithms.
What And Where To Focus In Person Search

Person search aims to locate and identify the query person from a gallery of original scene images. Almost all previous methods only consider single high-level semantic information, ignoring that the essence of identification task is to learn rich and expressive features. Additionally, large pose variations and occlusions of the target person significantly increase the difficulty of search task. For these two findings, we first propose multilevel semantic aggregation algorithm for more discriminative feature descriptors. Then, a pose-assisted attention module is designed to highlight fine-grained area of the target and simultaneously capture valuable clues for identification. Extensive experiments confirm that our framework can coordinate multilevel semantics of persons and effectively alleviate the adverse effects of occlusion and various pose. We also achieve state-of-the-art performance on two challenging datasets CUHK-SYSU and PRW.
Customization of open-source solutions on the example of the LMS Moodle distance learning platform

Abstract Open-source software has been a vast topic of thought and discussion since its inception. Many companies and developers did not understand for what purpose open-source software was developed, and some even opposed the existence of such solutions. Others, on the contrary, supported and still support the creation and distribution of this kind of programs, despite their unprofitableness. One of these programs is Moodle. It is a free web application that provides the ability to create websites for online learning, created in the early 2000s. Anyone can easily create or modify their own learning platform for free and without problems with copyright and licenses. The platform supports a huge number of languages, is constantly updated, and is also easy and convenient to customize. The user can add and customize his plugins and addons. The system has found wide application in connection with the emergence of COVID-19, since most schools and universities have switched to distance learning. Moodle functionality allows you to collect more than 50 users in one session without losing productivity. The platform has a built-in mechanism for authorizing users, including a login and password, which will not allow intruders to carry out unauthorized access to personal data of users. Teachers can leave the necessary files and recordings of classes on the platform for learners to access them. This article discusses the goals and issues of creating open-source software, its customization and application for solving problems of various kinds of activities, including the organization of multi-user educational platforms using the Moodle platform.
Stable and Effective One-Step Method for Person Search

Person search, which requires both pedestrian detection and person re-identification, is a challenging computer vision task applied to real-world scenarios. The challenges faced by detection and re-identification, such as occlusion, poor illumination, confusing background, are still urgent for person search. In addition, one-step methods for person search need to deal with the divergence between two tasks. In this work, we propose an end-to-end model containing the feature extractor, the region proposal network, and the multi-task learning module. In order to process divergence between detection and re-identification, we introduce switchable normalization and gradient centralization to improve the stability of the model. To solve the imbalance problem of hard examples, we introduce focal loss as a classification loss in the multi-task learning module. The experimental results on two bench-marks, i.e., CUHK-SYSU and PRW, well demonstrate that our method outperforms the state-of-the-art one-step methods.
Advances in Morphological Neural Networks: Training, Pruning and Enforcing Shape Constraints

In this paper, we study an emerging class of neural networks, the Morphological Neural networks, from some modern perspectives. Our approach utilizes ideas from tropical geometry and mathematical morphology. First, we state the training of a binary morphological classifier as a Difference-of-Convex optimization problem and extend this method to multiclass tasks. We then focus on general morphological networks trained with gradient descent variants and show, quantitatively via pruning schemes as well as qualitatively, the sparsity of the resulted representations compared to FeedForward networks with ReLU activations as well as the effect the training optimizer has on such compression techniques. Finally, we show how morphological networks can be employed to guarantee monotonicity and present a softened version of a known architecture, based on Maslov Dequantization, which alleviates issues of gradient propagation associated with its "hard" counterparts and moderately improves performance.
Self-Avatars in Virtual Reality: A Study Protocol for Investigating the Impact of the Deliberateness of Choice and the Context-Match

The illusion of virtual body ownership (VBO) plays a critical role in virtual reality (VR). VR applications provide a broad design space which includes contextual aspects of the virtual surroundings as well as user-driven deliberate choices of their appearance in VR potentially influencing VBO and other well-known effects of VR. We propose a protocol for an experiment to investigate the influence of deliberateness and context-match on VBO and presence. In a first study, we found significant interactions with the environment. Based on our results we derive recommendations for future experiments.
Movement-induced Priors for Deep Stereo

We propose a method for fusing stereo disparity estimation with movement-induced prior information. Instead of independent inference frame-by-frame, we formulate the problem as a non-parametric learning task in terms of a temporal Gaussian process prior with a movement-driven kernel for inter-frame reasoning. We present a hierarchy of three Gaussian process kernels depending on the availability of motion information, where our main focus is on a new gyroscope-driven kernel for handheld devices with low-quality MEMS sensors, thus also relaxing the requirement of having full 6D camera poses available. We show how our method can be combined with two state-of-the-art deep stereo methods. The method either work in a plug-and-play fashion with pre-trained deep stereo networks, or further improved by jointly training the kernels together with encoder-decoder architectures, leading to consistent improvement.
Degraded Document Image Binarization using Novel Background Estimation Technique

Over the past few decades, the use of scanned historical document images has increased dramatically, especially with the emergence of online libraries and standard benchmark datasets like DIBCO. The historical documents are usually in very-poor conditions containing noises like large ink stains, bleed-through, liquid spills, uneven-background, spots, faded-ink, weak/thin text that makes the task of binarization very difficult. In this paper, we propose an effective degraded document image binarization algorithm that performs accurate text segmentation. Our method first estimates the background utilizing information from neighboring pixels and filter smoothening. The next step is background subtraction that helps in the compensation of background distortions. The document is segmented using Otsu thresholding, and then we process the image to remove the remaining noise and maximize text content using labelled connected components. Our method outperforms several existing and widely used binarization algorithms on F-measure, PSNR, DRD, and pseudo F-measure when evaluated on H-DIBCO 2016 and H-DIBCO 2018 datasets and can very effectively detect faint characters from a document image.
RW-GCN: Training Graph Convolution Networks with Biased Random Walk for Semi-supervised Classification

Graph convolution networks (GCN) have recently been one of the most powerful methods in various tasks such as node classification and graph clustering. In the present study, we propose RW-GCN which utilizes biased random walk to assist in feature aggregation and GCN training process. RW-GCN employs biased random walks to generate node pairs. These pairs can be utilized to build a symmetric matrix to replace the adjacent matrix for GCN training. With these pairs generated above, we train the latent representation vectors by skip-gram. Our experiments demonstrate that compared to GCN, our model generates better results on node classification tasks performed on multiple datasets. In this way, both homophily and structural equivalence can be considered. Results of experiments on three datasets are presented to prove the availability of our method.
Conditional Descriptor Revision and Its Modelling by a CSP

Descriptor revision is a belief change framework that was introduced by Hansson as an alternative to the currently prevailing AGM paradigm. One central idea of descriptor revision is to describe the desired outcome of a belief change. Thus, descriptor revision allows expressing different kinds of belief change operations like revision or contraction in a structured and combined way. In this paper, we investigate the framework of conditional descriptor revision. Conditional descriptor revision is a variation of descriptor revision aimed at the revision of ranking functions in the context of conditional logic. It is obtained by applying descriptor revision to conditional logic and additionally requiring the belief changes to fulfil the principle of conditional preservation. We show how conditional descriptor revision can be characterized by a constraint satisfaction problem (CSP). In contrast to previous work, we cover the full descriptor language over conditionals closed under conjunction, disjunction, and negation. We also line out an implementation of conditional descriptor revision based on its CSP representation. Since propositional logic can be embedded into conditional logic, our approach also provides descriptor revision for propositional logic.
Advances in Code Summarization

Several studies have suggested that comments describing source code can help mitigate the burden of program understanding. However, software systems usually lack adequate comments and even when present, the comments may be obsolete or unhelpful. Researchers have addressed this issue by automatically generating comments from source code, a task referred to as Code Summarization. In this technical presentation, we take a deeper look at some of the significant, recent works in the area of code summarization and how each of them attempts to take a new perspective of this task including methods leveraging RNNs, Transformers, Graph neural networks and Reinforcement learning.
Coarse to Fine: Progressive and Multi-Task Learning for Salient Object Detection

Most deep learning-based salient object detection (SOD) methods tried to manipulate the convolution block to effectively capture the context of object. In this paper, we propose a novel method, called progressive and mutl-task learning scheme, to extract the context of object by only manipulating the learning scheme without changing the network architecture. The progressive learning scheme is a method to grow the decoder progressively in the train phase. In other words, starting from easier low-resolution layers, it gradually adds high-resolution layers. Although the progressive learning successfully captures the context of object, its output boundary tends to be rough. To solve this problem, we also propose a multi-task learning (MTL) scheme that processes the object saliency map and contour in a single network jointly. The proposed MTL scheme trains the network in an edge-preserved direction through an auxiliary branch that learns contours. The proposed a learning scheme can be combined with other convolution block manipulation methods. Extensive experiments on five datasets show that the proposed method performs best compared with state-of-the-art methods in most cases.
A practical framework of multi-person 3D human pose estimation with a single RGB camera

We propose a practical framework named 'DN-2DPN-3DPN' for multi-person 3D pose estimation with a single RGB camera. Our framework performs three-stages tasks on the input video: our DetectNet(DN) firstly detects the people's bounding box individually for each frame of the video, while our 2DPoseNet(2DPN) estimates the 2D poses for each person in the second stage, and our 3DPoseNet(3DPN) is finally applied to obtain the 3D poses of the people. Experiments validate that our method can achieve state-of-the-art performance for multi-person 3D human pose estimation on the Human3.6M dataset.
Abacus: A Tool for Precise Side-Channel Analysis

Side-channel vulnerabilities can leak sensitive information unconsciously. In this paper, we introduce the usage of Abacus. Abacus is a tool that can analyze secret-dependent control-flow and secret-dependent data-access leakages in binary programs. Unlike previous tools that can only identify leakages, it can also estimate the amount of leaked information for each leakage site. Severe vulnerabilities usually leak more information, allowing developers to triage the patching effort for side-channel vulnerabilities. This paper is to help users make use of Abacus and reproduce our previousresults. Abacus is available at https://github.com/s3team/Abacus.
Contrastive Separative Coding for Self-Supervised Representation Learning

To extract robust deep representations from long sequential modeling of speech data, we propose a self-supervised learning approach, namely Contrastive Separative Coding (CSC). Our key finding is to learn such representations by separating the target signal from contrastive interfering signals. First, a multi-task separative encoder is built to extract shared separable and discriminative embedding; secondly, we propose a powerful cross-attention mechanism performed over speaker representations across various interfering conditions, allowing the model to focus on and globally aggregate the most critical information to answer the "query" (current bottom-up embedding) while paying less attention to interfering, noisy, or irrelevant parts; lastly, we form a new probabilistic contrastive loss which estimates and maximizes the mutual information between the representations and the global speaker vector. While most prior unsupervised methods have focused on predicting the future, neighboring, or missing samples, we take a different perspective of predicting the interfered samples. Moreover, our contrastive separative loss is free from negative sampling. The experiment demonstrates that our approach can learn useful representations achieving a strong speaker verification performance in adverse conditions.
FFT based Horizontal SCA Attack against ECC

In this paper we introduce our fast Fourier transform (FFT) based horizontal SCA attack performed against an ASIC implementation of the Montgomery kP algorithm for the NIST elliptic curve B-233. In contrast to the recently reported correctness of 77% for the scalar k revealed analysing the trace using the comparison to the mean in the time domain we achieved a correctness of 96% for the key revealed analysing the trace in the frequency domain, based on FFT.
Deep learning based adaptive bit allocation for heterogeneous interference channels

This paper proposes an adaptive bit allocation scheme by using a fully connected (FC) deep neural network (DNN) considering imperfect channel state information (CSI) for heterogeneous networks. Achieving an accurate CSI has a crucial role on the system performance of the heterogeneous networks. Different quantization techniques have been employed to reduce the feedback overhead. However, the system performance cannot increase linearly with the number of bits increasing exponentially. Since optimizing the total number of bits is too complex for the entire network, an initial step is performed to distribute the bits to each cell in the conventional method. Then, the distributed bits are further allocated to each channel optimally. In order to enable direct allocation for the entire network, a FC-DNN based method is presented in this study. The optimized number of bits can be directly obtained for a different number of bits and scenarios by the proposed approach. The simulations are performed by using various scenarios with different allocation schemes. The performance results show that the DNN based method achieves a closer performance to the conventional approach.
Collaborative Design of Augmented Flashcards for Design History Class

This case study looks at augmented reality (AR) flashcards used in a university-level classroom. The collaborative design effort used RtD and pulled from the outcomes of three prior case studies using AR in the classroom. This poster focuses on the latest case study. Using research through design (RtD) to develop various AR user interface (UI) for educators to use as a framework to augment classroom content aiming to improve learning outcomes for all students. The case study shows that more research is warranted, and there is a viable need for using AR in higher education humanities courses.
Representative Local Feature Mining for Few-Shot Learning

Few-shot learning aims to recognize unseen images of new classes with only a few training examples. While great progress has been made with deep learning technology, most metric-based works rely on the measurement based on global feature representation of images, which is sensitive to background factors due to the scarcity of training data. Given this, we propose a novel method that chooses representative local features to facilitate few-shot learning. Specifically, we propose a "task-specific guided" strategy to mine local features that are task-specific and discriminative. For each task, we first mine representative local features for labeled images by a loss guided mechanism. Then these local features are used to guide a classifier to mine representative local features for unlabeled images. In this way, task-specific representative local features can be selected for better classification. We empirically show our method can effectively alleviate the negative effect introduced by background factors. Extensive experiments on two few-shot benchmarks show the effectiveness of the proposed method.
Human-Aware Coarse-to-Fine Online Action Detection

In this work, we propose a two-stage framework to efficiently and effectively detect actions on-the-fly. An action location network (ALN) is developed in the first stage to judge whether the current frame is action-related, while the second stage involves an action classification network (ACN) to further identify the action category. In this way, irrelevant negative frames are quickly discarded and actions are detected as early as they occur. Moreover, we highlight human areas at both the stages by respectively incorporating a human detector and a human mask layer. As a result, more accurate spatial-temporal windows of actions are detected, based on which more robust features are extracted for classification. Experimental results on two popular benchmarks demonstrate the superior performance of the proposed approach.
Unsupervised Learning of Facial Landmarks based on Inter-Intra Subject Consistencies

We present a novel unsupervised learning approach to image landmark discovery by incorporating the inter-subject landmark consistencies on facial images. This is achieved via an inter-subject mapping module that transforms original subject landmarks based on an auxiliary subject-related structure. To recover from the transformed images back to the original subject, the landmark detector is forced to learn spatial locations that contain the consistent semantic meanings both for the paired intra-subject images and between the paired inter-subject images. Our proposed method is extensively evaluated on two public facial image datasets (MAFL, AFLW) with various settings. Experimental results indicate that our method can extract the consistent landmarks for both datasets and achieve better performances compared to the previous state-of-the-art methods quantitatively and qualitatively.