Reviewerly/exHarmony huggingface dataset:
Wlinker: Modeling Relational Triplet Extraction As Word Linking

Relational triplet extraction (RTE) is a fundamental task for automatically extracting information from unstructured text, which has attracted growing interest in recent years. However, it remains challenging due to the difficulty in extracting the overlapping relational triplets. Existing approaches for overlapping RTE, either suffer from exposure bias or designing complex tagging scheme. In light of these limitations, we take an innovative perspective on RTE by modeling it as a word linking problem that learns to link from subject words to object words for each relation type. To this end, we propose a simple but effective multi-task learning model, WLinker, which can extract overlapping relational triplets in an end-to-end fashion. Specifically, we perform word link prediction based on multi-level biaffine attention for leaning the word-level correlations under each relation type. Additionally, our model joint entity detection and word link prediction tasks by a multi-task framework, which combines the local sequential and global dependency structures of words in sentence and captures the implicit interactions between the two tasks. Extensive experiments are conducted on two benchmark datasets NYT and WebNLG. The results demonstrate the effectiveness of WLinker, in comparison with a range of previous state-of-the-art baselines.
Exploring Transferability Measures and Domain Selection in Cross-Domain Slot Filling

As an essential task for natural language understanding, slot filling aims to identify the contiguous spans of specific slots in an utterance. In real-world applications, the labeling costs of utterances may be expensive, and transfer learning techniques have been developed to ease this problem. However, cross-domain slot filling could significantly suffer from negative transfer due to non-targeted or zero-shot slots. Originally, this paper explores several ways to measure transferability across slot filling domains and finds that the shared slot number could serve as an efficient and effective estimator. First, this frustratingly easy measure requires no training data and is efficient to calculate. Second, it guides us heuristically select source domains that contain more shared slots with the target domain, which obtains SOTA results on Snips benchmark. Third, a dynamic transfer procedure based on this estimator clearly shows the negative transfer in cross-domain slot filling. We finally explore a source-free scene that we could only obtain black-box source models and propose to weight source domains based on prediction entropy.
The impact of cross language on acoustic-to-articulatory inversion and its influence on articulatory speech synthesis

Estimating articulatory representations (ARs) from acoustic features is known as acoustic-to-articulatory inversion (AAI). Various factors of input acoustic features impact the performance of AAI. In this work, we investigate the effect of unseen language on the AAI performance in both seen and unseen speaker conditions. We further perform experiments to analyze how these AAI predictions in unseen language and unseen speaker conditions, in turn, impact the articulatory speech synthesis, i.e., articulatory-to-acoustic forward mapping (AAF). We hypothesize that this investigation enables the exploration of alternative approaches to voice conversion across unseen languages using ARs. Experiments are performed on the AAF model trained using English ARs and evaluated on ARs from unseen speakers speaking different native Indian languages, namely, Hindi, Kannada, Telugu, and Tamil. Experiments reveal that, for AAI, there is a drop in performance due to the mismatch in language in both seen and unseen speaker evaluations. For AAF, subjective evaluations reveal that the synthesized speech quality of non-native (mismatched language) speech is comparable with that of English (matched language).
Clustering and Separating Similarities for Deep Unsupervised Hashing

The lack of supervised information is the pivotal problem in unsupervised hashing. Most methods leverage deep features extracted from pre-trained models to generate semantic similarities as supervised information. These fixed features are, however, neither designed originally for retrieval nor updated adaptively during training. In this paper, we propose a novel deep Unsupervised Cluster and Separate Hashing (UCSH) to address these issues. Specifically, we introduce a fully end-to-end deep hashing network with a binary latent Variational AutoEncoder (VAE), which enables hash codes capable of reconstructing deep features as well as preserving semantic relations. Moreover, a ‘Cluster and Separate’ scheme is proposed to jointly cluster deep features and separate semantic similarities. Both the implicit feature clustering and the explicit similarity separating loss encourage the separation of similar and dissimilar pairs, enabling the iteratively updated similarities to better excavate semantic relations. Experiments conducted on three benchmarks show the superiority of UCSH.
Sampling Set Selection for Graph Signals under Arbitrary Signal Priors

We propose a sampling set selection method for graph signals under arbitrary signal priors. Most approaches of graph signal sampling assume that signals are bandlimited. However, in practical situations, there exist many full-band graph signals like piecewise smooth/constant signals. Our sampling set selection method allows for arbitrary graph signal models as long as they are linear. This can be derived from a generalized sampling framework. In contrast to existing works, we focus on the direct sum condition between sampling and reconstruction subspaces where the direct sum condition plays a key role for the best possible recovery of sampled signals. We also design a fast sampling set selection algorithm based on the proposed method with the Neumann series approximation. In sampling and recovery experiments, we validate the effectiveness of the proposed method for several graph signal models.
Acoustic Comparison of Physical Vocal Tract Models with Hard and Soft Walls

This study explored how the frequencies and bandwidths of the acoustic resonances of physical tube models of the vocal tract differ when they have hard versus soft walls. For each of 10 tube shapes representing different vowels, two physical models were made: one with rigid plastic walls, and one with soft silicone walls. For all models, the acoustic transfer functions were measured and the bandwidths and frequencies of the first three resonances were determined. For the models with soft walls, the resonance frequencies and bandwidths below 2 kHz were consistently higher than for the models with hard walls. These results confirm the general predictions of mathematical lumped-element models for the mechanical impedance of the vocal tract walls. The resonance bandwidths of the silicone models were well within the range of 50–90 Hz measured for humans, while they were too small for the rigid tubes below 2 kHz.
Joint Global-Local Alignment for Domain Adaptive Semantic Segmentation

Unsupervised domain adaptation has shown promising results in leveraging synthetic (source) images for semantic segmentation of real (target) images. One key issue is how to align data distributions between the source and target domains. Adversarial learning has been applied to align these distributions. However, most existing approaches focus on aligning the output distributions related to image (global) segmentation. Such global alignment may not result in effective alignment due to the inherent high dimensionality feature space involved in the alignment. Moreover, global alignment might be hindered by the noisy outputs corresponding to background pixels in the source domain. To address this limitation, we propose a local output alignment. Such an approach can also mitigate the influences of noisy background pixels from the source domain when performing the local alignment. Our experiments show that by adding local output alignment into various global alignment based domain adaptation, our joint global-local alignment methods improves semantic segmentation. Code is available at https://github.com/skrya/globallocal.
Multimodal Depression Classification using Articulatory Coordination Features and Hierarchical Attention Based text Embeddings

Multimodal depression classification has gained immense popularity over the recent years. We develop a multimodal depression classification system using articulatory coordination features extracted from vocal tract variables and text transcriptions obtained from an automatic speech recognition tool that yields improvements of area under the receiver operating characteristics curve compared to unimodal classifiers (7.5% and 13.7% for audio and text respectively). We show that in the case of limited training data, a segment-level classifier can first be trained to then obtain a session-wise prediction without hindering the performance, using a multi-stage convolutional recurrent neural network. A text model is trained using a Hierarchical Attention Network (HAN). The multimodal system is developed by combining embeddings from the session-level audio model and the HAN text model.
Unsupervised Model Adaptation for End-to-End ASR

End-to-end (E2E) Automatic Speech Recognition (ASR) systems are widely applied in various devices and communication domains. However, state-of-the-art ASR systems are known to underperform when there is a mismatch in the training and test domains. As a result, acoustic models deployed in production are often adapted to the target domain to improve accuracy. This paper proposes a method to perform unsupervised model adaptation for E2E ASR using first-pass transcriptions of adaptation data produced by the baseline ASR model itself. The paper proposes two transcription confidence measures that can be used to select an optimal in-domain adaptation set. Experiments were performed using the Quartznet ASR architecture on the HarperValleyBank corpus. Results show that the unsupervised adaptation technique with the confidence measure based data selection results in a 8% absolute reduction in word error rate on the HarperValleyBank test set. The proposed method can be applied to any E2E ASR system and is suitable for model adaptation on call center audio with little to no manual transcription.
TH-Net: A Method Of Single 3d Object Tracking Based On Transformers And Hausdorff Distance

3D object tracking is the key of automatic driving. We propose a new 3D object tracking method called Transformer-Hausdorff Net (TH-Net). It contains three main modules: Feature Extraction, Feature Fusion, and Proposal Generation. The Feature Extraction module extracts features from the template and search area, where the permutation-invariable Transformers is leveraged to deal with the point cloud’s dis-order and sparsity. The features of template and search area are then fused by the Feature Fusion module to generate the tracking clues. Based on the tracking clues, we further generate 3D target proposal and execute verification in Proposal Generation module. TH-Net achieves a performance improvement in contrast to the state-of-the-art work on KITTI and NuScenes dataset.
Towards Better Meta-Initialization with Task Augmentation for Kindergarten-Aged Speech Recognition

Children’s automatic speech recognition (ASR) is always difficult due to, in part, the data scarcity problem, especially for kindergarten-aged kids. When data are scarce, the model might overfit to the training data, and hence good starting points for training are essential. Recently, meta-learning was proposed to learn model initialization (MI) for ASR tasks of different languages. This method leads to good performance when the model is adapted to an unseen language. How-ever, MI is vulnerable to overfitting on training tasks (learner overfitting). It is also unknown whether MI generalizes to other low-resource tasks. In this paper, we validate the effectiveness of MI in children’s ASR and attempt to alleviate the problem of learner overfitting. To achieve model-agnostic meta-learning (MAML), we regard children’s speech at each age as a different task. In terms of learner overfitting, we propose a task-level augmentation method by simulating new ages using frequency warping techniques. Detailed experiments are conducted to show the impact of task augmentation on each age for kindergarten-aged speech. As a result, our approach achieves a relative word error rate (WER) improvement of 51% over the baseline system with no augmentation or initialization.
Multi-Level Relation Aware Network for Person Re-Identification

Person attribute or pose information has improved person re-identification performance, however, inaccurate pose or attribute module will damage the final identification performance. Based on this, we propose a multi-scale relation aware network (MSRA) for person re-identification. Specifically, we design an attribute relation mining module to construct an attribute map through constraint loss to learn the correlation between different attributes. Besides, we construct a multi-level Pose Pyramid based on the physical structure of the human body, so as to model the internal relationship between pose points. Finally, we designed a cross-scale graph convolution to infer the cooperative structural relation between different layers of components and fused it with the attribute relation module to reinforce the feature. Many experiments on three large-scale datasets verify the effectiveness and state-of-the-art performance of the proposed method.
Signal Compression via Neural Implicit Representations

Existing end-to-end signal compression schemes using neural networks are largely based on an autoencoder-like structure, where a universal encoding function creates a compact latent space and the signal representation in this space is quantized and stored. Recently, advances from the field of 3D graphics have shown the possibility of building implicit representation networks, i.e., neural networks returning the value of a signal at a given query coordinate. In this paper, we propose using neural implicit representations as a novel paradigm for signal compression with neural networks, where the compact representation of the signal is defined by the very weights of the network. We discuss how this compression framework works, how to include priors in the design, and highlight interesting connections with transform coding. While the framework is general, and still lacks maturity, we already show very competitive performance on the task of compressing point cloud attributes, which is notoriously challenging due to the irregularity of the domain, but becomes trivial in the proposed framework.
Byzantine-Robust and Communication-Efficient Distributed Non-Convex Learning Over Non-IID Data

Motivated by the emerging federated learning applications, we jointly consider the problems of Byzantine-robustness and communication efficiency in distributed non-convex learning over non-IID data. We propose a compressed robust stochastic model aggregation (CRSA) method, which applies the idea of robust stochastic model aggregation to achieve Byzantine-robustness over non-IID data, while compresses the transmitted messages so as to achieve communication efficiency. Utilizing the tools of Moreau envelope and proximal point projection, we establish the convergence of C-RSA for distributed non-convex learning problems. Numerical experiments on training a large-scale neural network demonstrate the effectiveness of the proposed C-RSA method.
Federated Self-Training for Data-Efficient Audio Recognition

Federated learning is a distributed machine learning paradigm dealing with decentralized and personal datasets. Since data reside on devices like smartphones, labeling is entrusted to the clients or labels are extracted in an automated way. Specifically, in the case of audio data, acquiring semantic annotations can be prohibitively expensive and time-consuming. As a result, an abundance of audio data remains unlabeled and unexploited on users’ devices. Existing federated learning approaches largely focus on supervised learning without harnessing the unlabeled data. Here, we study the problem of semi-supervised learning of audio models in conjunction with federated learning. We propose FedSTAR, a self-training approach to exploit large-scale on-device unlabeled data to improve the generalization of audio recognition models. We conduct experiments on diverse public audio classification datasets and investigate the performance of our models under varying percentages of labeled data and show that with as little as 3% labeled data, FedSTAR on average can improve the recognition rate by 13.28% compared to the fully-supervised federated model.
Contrastive Translation Learning For Medical Image Segmentation

Unsupervised domain adaptation commonly uses cycle generative networks to produce synthesis data from source to target domains. Unfortunately, translated samples cannot effectively preserve semantic information from input sources, resulting in bad or low adaptability of the network to segment target data. This work proposes an advantageous domain translation mechanism to improve the perceptual ability of the network for accurate unlabeled target data segmentation. Our domain translation employs patchwise contrastive learning to improve the semantic content consistency between input and translated images. Our approach was applied to unsupervised domain adaptation based abdominal organ segmentation. The experimental results demonstrate the effectiveness of our framework that outperforms other methods.
Learning Common Dependency Structure for Unsupervised Cross-Domain Ner

Unsupervised cross-domain NER task aims to solve the issues when data in a new domain are fully-unlabeled. It leverages labeled data from source domain to predict entities in unlabeled target domain. Since training models on large domain corpus is time-consuming, in this paper, we consider an alternative way by introducing syntactic dependency structure. Such information is more accessible and can be shared between sentences from different domains. We propose a novel framework with dependency-aware GNN (DGNN) to learn these common structures from source domain and adapt them to target domain, alleviating the data scarcity issue and bridging the domain gap. Experimental results show that our method outperforms state-of-the-art methods.
Patch Steganalysis: A Sampling Based Defense Against Adversarial Steganography

In recent years, the classification accuracy of CNN (convolutional neural network) steganalyzers has rapidly improved. However, as general CNN classifiers will misclassify adversarial samples, CNN steganalyzers can hardly detect adversarial steganography, which combines adversarial samples and steganography. Adversarial training and preprocessing are two effective methods to defend against adversarial samples. But literature shows adversarial training is ineffective for adversarial steganography. Steganographic modifications will also be destroyed by preprocessing, which aims to wipe out adversarial perturbations. In this paper, we propose a novel sampling based defense method for steganalysis. Specifically, by sampling image patches, CNN steganalyzers can bypass the sparse adversarial perturbations and extract effective features. Additionally, by calculating statistical vectors and regrouping deep features, the impact on the classification accuracy of common samples is effectively compressed. The experiments show that the proposed method can significantly improve the robustness against adversarial steganography without adversarial training.
Hierarchical Feature Aggregation Network for Deep Image Compression

Existing CNN-based methods for image compression extract features through serially connected high-to-low (encoder) or low-to-high (decoder) resolution stages, leading to insufficient utilization of hierarchical features. To solve this problem, we present a hierarchical feature aggregation network (HFAN) for generating more informative latent representations. In detail, we propose two strategies, namely inter-stage feature aggregation and intra-stage feature aggregation. The inter-stage feature aggregation integrates multi-scale information thereby producing more contextual features. The intra-stage aggregation fuses features within the same stage to enrich representations of one specific resolution. Besides, we incorporate a lightweight pixel-wise attention mechanism to further enhance the discriminative ability of our network. Extensive experiments demonstrate that our HFAN achieves superior performance over state-of-the-art methods without a hyperprior variational autoencoder.
A Novel Part Feature Integration and Fusion Method for Fine-Grained Vehicle Recognition

In this paper, we propose a novel light-weight feature integration and fusion method to enhance the discriminative ability of deep convolutional features for the task of fine-grained vehicle recognition. The proposed method is built on the deep convolutional layers from which the discriminative part features could be integrated and fused accordingly. More specifically, a basic feature integration module is adopted to integrate the feature maps of deep convolutional layers into groups in each of which the related discriminative parts are assembled together. Then a fusion module follows to model the coarse-to-fine relationship of the part features and further ensure the integrity and effectiveness of the part features. We conduct comparison experiments on public dataset, and the results show that the proposed method achieves comparable performance with state-of-the-art algorithms.
Two Strategies Toward Lightweight Image Super-Resolution

Recent convolution neural networks (CNNs) have achieved remarkable success in lightweight image super-resolution (LISR). The goal of LISR is to restore more accurate details with less model capacity. However, we observe two phenomena in current micro-architectures, one is the lack of consistent learning ability of high-frequency components, the other is large residual problem which does harm to the stability of residual learning. To tackle the two issues, we propose two strategies, namely global-guided attention strategy (GGAS) and channel-wise scaling strategy (CWSS), which can significantly improve the performance of the state-of-the-arts with negligible overheads.
Importance Sampling Cams For Weakly-Supervised Segmentation

Classification networks can be used to localize and segment objects in images by means of class activation maps (CAMs). However, without pixel-level annotations, classification networks are known to (1) mainly focus on discriminative regions, and (2) to produce diffuse CAMs without well-defined prediction contours. In this work, we approach both problems with two contributions for improving CAM learning. First, we incorporate importance sampling based on the class-wise probability mass function induced by the CAMs to produce stochastic image-level class predictions. This results in CAMs which activate over a larger extent of objects. Second, we formulate a feature similarity loss term which aims to match the prediction contours with edges in the image. As a third contribution, we conduct experiments on the PASCAL VOC 2012 benchmark dataset to demonstrate that these modifications significantly increase the performance in terms of contour accuracy, while being comparable to current state-of-the-art methods in terms of region similarity.