Reviewerly/exHarmony huggingface dataset:
Finer-Grained Decomposition for Parallel Quantum Mimo Processing

Exploiting (near-)optimal MIMO signal processing algorithms in the next generation (NextG) cellular systems holds great promise in achieving significant wireless performance gains in spectral efficiency and device connectivity, to name a few. However, it is extremely difficult to enable optimal processing methods in the systems, since the required computational amount increases exponentially with more users and higher data rates, while available processing time is strictly limited. In this regard, quantum signal processing has been recently identified as a promising potential enabler of the (near-)optimal algorithms in the systems, since quantum computing could dramatically speed up the computation via non-conventional effects based on quantum mechanics. Given existing quantum decoherence and noise on quantum hardware, parallel quantum optimization could accelerate the process even further at the expense of more qubit usage. In this paper, we discuss the parallelization of quantum MIMO processing and investigate a spin-level preprocessing method for relatively finer-grained decomposition that can support more flexible parallel quantum signal processing, compared to the recently reported symbol-level decomposition method. We evaluate the method on the state-of-the-art analog D-Wave Advantage quantum processor.
End-to-End Non-Autoregressive Image Captioning

Most of the existing image captioning models use the autoregressive approach to generate captions, which leads to high latency in the inference process. Non-autoregressive decoding generates words in parallel, which greatly improves the model inference speed. However, non-autoregressive decoding usually leads to performance loss due to the loss of word input. In this paper, we propose a semantic retrieval module that uses image features to retrieve semantic information as input of the non-autoregressive decoder, narrowing the performance gap between the non-autoregressive and the autoregressive model. Furthermore, we adopt Swin-Transformer instead of Faster R-CNN to extract image features, thus building an end-to-end image caption model. Experiments conducted on the MSCOCO dataset show that our model achieves new state-of-the-art performances of 122.6% CIDEr score on the ’Karpathy’ offline test split with 37× inference speedup.
Identifying Entrainment in Task-Oriented Conversations

Human interlocutors adapt their behavior to each other in a conversation through entrainment. While entrainment has been found in long chit-chat conversations, much less research has been conducted on task-oriented dialogs. In this paper, we investigate short task-oriented Wizard-of-Oz conversations for acoustic-prosodic and lexical entrainment. We conduct significance tests that reveal changes in speech pitch and frequent words as important indicators of entrainment. Our findings will guide user-entraining dialog systems to improve the quality of conversations.
Semantics-Guided Object Removal for Facial Images: with Broad Applicability and Robust Style Preservation

Object removal and image inpainting in facial images is a task in which objects that occlude a facial image are specifically targeted, removed, and replaced by a properly reconstructed facial image. Two different approaches utilize U-net-based generator and modulated approach, and they respectively have been widely endorsed but notwithstanding each method’s disadvantages of low generative capability and low reconstruction power. Here, we propose a Semantics-Guided Inpainting Network (SGIN), which is the invention of a desirable trade-off between those two methods that can be applied to any form of occluding mask while maintaining a consistent style and preserving high-fidelity details of the original image. By using the guidance of a semantic map, our model is capable of manipulating facial features and styles which grants direction to the one-to-many problem for further practicability.
Mabnet: Master Assistant Buddy Network With Hybrid Learning for Image Retrieval

Image retrieval has garnered a growing interest in recent times. The current approaches are either supervised or self-supervised. These methods do not exploit the benefits of hybrid learning using both supervision and self-supervision. We present a novel Master Assistant Buddy Network (MAB-Net) for image retrieval which incorporates both the learning mechanisms. MABNet consists of master and assistant block, both learning independently through supervision and collectively via self-supervision. The master guides the assistant by providing its knowledge base as a reference for self-supervision and the assistant reports its knowledge back to the master by weight transfer. We perform extensive experiments on the public datasets with and without post-processing.
Multi-Output RNN-T Joint Networks for Multi-Task Learning of ASR and Auxiliary Tasks

We propose a multi-output joint network architecture for RNN-T transducer, for multi-task modeling of ASR and auxiliary tasks that rely on ASR outputs. Each output of the joint network predicts tar-get labels with disjoint vocabularies for each task, while sharing the same audio features by the encoder and language model features by the prediction network. Each task is trained with an RNN-T loss that marginalizes over all possible paths, and we allow multiple tasks to share the blank logit so that they are synchronized. We demonstrate our method on two auxiliary tasks, namely capitalization and pause prediction, and discuss different considerations for modeling and inference procedures. For capitalization, we successfully distill capitalization labels from a standalone text normalization model, and achieve competitive Uppercase Error Rate (UER) while offering streaming capability and improved inference efficiency. In addition, our model has similar capitalization accuracy compared to a mixed-case ASR model, but obtains improved WERs if integrated with external language models. For pause prediction, we achieve the same performance as the previous two-step approach while providing a simpler training recipe without affecting ASR accuracy.
Test Your Samples Jointly: Pseudo-Reference for Image Quality Evaluation

In this paper, we address the well-known image quality assessment problem but in contrast from existing approaches that predict image quality independently for every images, we propose to jointly model different images depicting the same content to improve the precision of quality estimation. This proposal is motivated by the idea that multiple distorted images can provide information to disambiguate image features related to content and quality. To this aim, we combine the feature representations from the different images to estimate a pseudo-reference that we use to enhance score prediction. Our experiments show that at test-time, our method successfully combines the features from multiple images depicting the same new content, improving estimation quality.
Real-Time MRI Video Synthesis from Time Aligned Phonemes with Sequence-to-Sequence Networks

Real-Time Magnetic resonance imaging (rtMRI) of the midsagittal plane of the mouth is of interest for speech production research. In this work, we focus on estimating utterance level rtMRI video from the spoken phoneme sequence. We obtain time-aligned phonemes from forced alignment, to obtain frame-level phoneme sequences which are aligned with rtMRI frames. We propose a sequence-tosequence learning model with a transformer phoneme encoder and convolutional frame decoder. We then modify the learning by using intermediary features obtained from sampling from a pretrained phoneme-conditioned variational autoencoder (CVAE). We train on 8 subjects in a subject-specific manner and demonstrate the performance with a subjective test. We also use an auxiliary task of air tissue boundary (ATB) segmentation to obtain the objective scores on the proposed models. We show that the proposed method is able to generate realistic rtMRI video for unseen utterances, and adding CVAE is beneficial for learning the sequence-to-sequence mapping for subjects where the mapping is hard to learn.
A Novel Transformer-Based Pipeline for Lung Cytopathological Whole Slide Image Classification

We propose a novel three-stage Transformer-based methodology for entire cytopathological whole slide image (WSI) classification. The key idea is to leverage Transformer to extract the fine-grained lesion-level features and then progressively aggregate them into intermediate-grained patch-level features and coarse-grained WSI-level features for classification. Specifically, we first extract multi-scale lesion features from each patch image via Transformer-based lesion detection, and then adaptively aggregate the extracted lesion features into the corresponding patch feature with an MLP-Mixer. Finally, we select the most representative patch features and feed them into the Vision Transformer (ViT) for the final WSI-level classification. We collect a dataset consisting of 961 lung cytopathological WSIs of pleural effusions cytology specimens and conduct extensive experiments on it. The experimental results demonstrate that the proposed method outperforms existing state-of-the-art (SOTA) methods for cytopathological WSI classification.
Think Before You Speak: Concept-Guided Explicit Persona Reasoning for Personalized Dialogue Generation

It is a critical challenge for open-domain dialogue agents to generate context-coherent responses which can present a consistent personality. However, existing methods mainly focus on the penalty of the persona-inconsistent responses, leaving out considering the context-incoherence problem caused by wrong persona selection. In this paper, we propose the Think-Before-You-Speak (TBYS) model, consisting of Concept-guided Persona Reasoning module and Consistent Dialogue Generation module, to explicitly select persona sentences semantically relevant to the current turn and generate responses based on the selection results. The experimental results on Persona-Chat show that TBYS can generate coherent and consistent responses, outperforming state-of-the-art baselines in both automatic and human evaluations.
Low in Resolution, High in Precision: UAV Detection with Super-Resolution and Motion Information Extraction

The rapid development of unmanned aerial vehicle (UAV) market presents potential threats to public security and personal privacy, and the vision sensors are widely deployed to detect the invasive UAVs because of the intuitivity and accessibility of the video. However, the small pixel area and weak morphological characteristics of distant invasive UAVs pose a considerable challenge to the detection precision. Prior work on UAV detection simply focuses on the information fusion between different feature layers, but ignoring the feature information inside each layer. In addition, to detect small UAV in video streams, the motion information of the target is also a noteworthy feature. In this regard, we propose a feature super-resolution-based UAV detector with motion information extractor. The proposed network fully utilizes the motion information of UAVs between temporal frames and the spatial invariant features between different resolution frames to pursue a high-accuracy small UAV detection performance. Experiments on Drone vs Birds dataset are carried out, and it is demonstrated that a higher detection accuracy on small UAVs is achieved compared with the baseline.
Towards Realizing the Value of Labeled Target Samples: A Two-Stage Approach for Semi-Supervised Domain Adaptation

Semi-Supervised Domain Adaptation (SSDA) is a recently emerging research topic that extends from the widely-investigated Unsupervised Domain Adaptation (UDA) by further having a few target samples labeled, i.e., the model is trained with labeled source samples, unlabeled target samples as well as a few labeled target samples. Compared with UDA, the key to SSDA lies how to most effectively utilize the few labeled target samples. Existing SSDA approaches simply merge the few precious labeled target samples into vast labeled source samples or further align them, which dilutes the value of labeled target samples and thus still obtains a biased model. To remedy this, in this paper, we propose to decouple SSDA as an UDA problem and a semi-supervised learning problem where we first learn an UDA model using labeled source and unlabeled target samples and then adapt the learned UDA model in a semi-supervised way using labeled and unlabeled target samples. By utilizing the labeled source samples and target samples separately, the bias problem can be well mitigated. We further propose a consistency learning based mean teacher model to effectively adapt the learned UDA model using labeled and unlabeled target samples. Experiments show our approach outperforms existing methods.
Implicit Bayes Adaptation: A Collaborative Transport Approach

The power and flexibility of Optimal Transport (OT) have pervaded a wide spectrum of problems, including recent Machine Learning challenges such as unsupervised domain adaptation. Its essence of quantitatively relating two probability distributions by some optimal metric, has been creatively exploited and shown to hold promise for many real-world data challenges. In a related theme in the present work, we posit that domain adaptation robustness is rooted in the intrinsic (latent) representations of the respective data, which are inherently lying in a non-linear submanifold embedded in a higher dimensional Euclidean space. We account for the geometric properties by refining the l <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Euclidean metric to better reflect the geodesic distance between two distinct representations. We integrate a metric correction term as well as a prior cluster structure in the source data of the OT-driven adaptation. We show that this is tantamount to an implicit Bayesian framework, which we demonstrate to be viable for a more robust and better-performing approach to domain adaptation. Substantiating experiments are also included for validation purposes.
Assessing the Robustness of Deep Learning-Assisted Pathological Image Analysis Under Practical Variables of Imaging System

With the advancement of deep learning, computer-assisted clinical diagnosis, such as liquid-based cervical cytology, has attracted more attention. However, the fragile robustness of deep learning models has a non-negligible impact on their classification accuracy and reliability. To be more specific, various scanner parameters will be used depending on the pathologist’s preferences during the clinical diagnosis process (e.g., field source brightness, contrast, saturation, etc.), and this variation will lead to the unstable performance of the model. In this paper, we construct an evaluation pathway to assess the stability and consistency of deep learning models under various customized scanner parameters. Specifically, a multi-scanned dataset consists of 4200 whole slide images (WSIs) is generated by scanning 200 stained slices using various scanner parameters. Moreover, we conducted a large number of experiments to investigate the robustness of numerous models, including convolution-based and transformer-based models concerning various scanner parameter settings. Furthermore, we introduce several indicators to analyze the prediction accuracy, consistency and robustness of the model on the constructed dataset. The experimental results indicate that the deep learning models are sensitive to luminance-related scanner parameters. In addition, transformer-based models have better robustness than traditional convolutional neural networks. Our code has been made available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Select The Best: Enhancing Graph Representation with Adaptive Negative Sample Selection

Graph contrastive learning (GCL) has emerged as a powerful tool to address real-world widespread label scarcity problems and has achieved impressive success in the graph learning domain. Albeit their remarkable performance, most current works mainly focus on designing sample augmentation methods, while the effect of negative sample selection strategy is largely ignored by previous works but rather practical and significant for graph contrastive learning. In this paper, we study the impact of negative samples on learning graph-level representations, and innovatively propose a Reinforcement Graph Contrastive Learning (ReinGCL) for negative sample selection. To be concrete, our model consists of two major components: a graph contrastive learning framework (GCLF), and a selection distribution generator (SDG) for producing the selection probabilities based on RL. The key insight is that Re-inGCL attempts to leverage SDG to guide GCLF and narrow the divergence between the augmented positive pairs, so as to further improve graph representation learning. Extensive experiments demonstrate that our approach significantly yields superior performance compared to the state-of-the-art.
Data Augmentation Based On Invariant Shape Blending For Deep Learning Classification

In this article, we introduce a new technique for augmenting 2D shape datasets based on a planar blending. In particular, a recently introduced shape blending method is applied to numerous pairs of shapes extracted from a given class. Several in-between data belonging to the same category are therefore generated. While traditional data augmentation approaches mainly apply simple transformations to the original data, the proposed technique allows the generation of non-linear variations of the input shapes by covering significantly the shape space. To demonstrate its interest, our augmentation technique is applied to the task of 2D shape classification. Experiments are performed on two benchmarks, namely KIMIA’99 and MPEG-7 CE using two different Convolutional Neural Network (CNN) architectures. The results show the superiority of our method over traditional augmentation techniques.
An End-to-End Neural Network for Image-to-Audio Transformation

This paper describes an end-to-end (E2E) neural architecture for the audio rendering of small portions of display content on low resource personal computing devices. It is intended to address the problem of accessibility for vision-impaired or vision-distracted users at the hardware level. Neural image-to-text (ITT) and text-to-speech (TTS) approaches are reviewed and a new technique is introduced to efficiently integrate them in a way that is both efficient and back-propagate-able, leading to a non-autoregressive E2E image-to-speech (ITS) neural network that is efficient and trainable. Experimental results are presented showing that, compared with the non-E2E approach, the proposed E2E system is 29% faster and uses 19% fewer parameters with a 2% reduction in phone accuracy. A future direction to address accuracy is presented.
Robust Content-Variant Reference Image Quality Assessment Via Similar Patch Matching

Although image quality assessment (IQA) methods have achieved remarkable success in the past decades, full-reference IQA is limited to reference images, while no-reference IQA has relatively poor performance. To boost the performance of IQA models in the no-reference scenario, a new class of IQA methods using content-variant high-quality images as references have emerged. However, the existing approaches do not take advantage of the content information of the content-variant reference (CVR) images, resulting in the insufficient use of high-quality reference information and the unsatisfactory robustness of the algorithm performance. To effectively utilize CVR images and make the algorithm more robust, we propose a CVR IQA scheme based on similar patch matching. For each image patch to be evaluated, the patch with the most similar content is first searched in the CVR image as the reference patch. Since the two patches are more similar, more useful reference information can be extracted. A similarity calculation module based on cross-attention is designed to find content-similar patches. Extensive experimental results show that the proposed algorithm has good performance and robustness.
Healthcall Corpus and Transformer Embeddings from Healthcare Customer-Agent Conversations

We present the corpus called HealthCall which was recorded in real-life conditions in the call center of Malakoff Humanis a health insurance company. The records include two separate audio channels, the first one for the customer and the second one for the agent. This corpus includes a transcription of the spoken conversations and was divided into three sets: training (Train), development (Dev) and test (Test) sets. Two customer relationship management tasks were assessed on the HealthCall corpus: the classification of user requests and the detection of complaints. For this purpose, we have investigated 18 feature sets: 12 linguistic and 6 audio sets. We have used BERT models for the linguistic features and Wav2Vec models for the audio features. The results show that the linguistic features give always the best results (92.7% for the Request task and 69.0% for the Complaint task) but the concatenation of acoustic and linguistic features allows a slight improvement for the Complaint task (69.4% versus 69%).
Eigen-Decomposition-Free Directed Graph Sampling via Gershgorin Disc Alignment

Graph sampling is the problem of choosing a node subset via sampling matrix H ∈ {0, 1} <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K×N</sup> to collect samples y = Hx ∈ℝ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</sup> , K < N, so that the target signal x ∈ ℝ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sup> can be reconstructed in high fidelity. While sampling on undirected graphs is well studied, we propose the first eigen-decomposition-free sampling scheme tailored specifically for directed graphs, leveraging a previous undirected graph sampling method based on Gershgorin disc alignment (GDAS). Concretely, given a directed positive graph ${{\mathcal{G}}^d}$ specified by random-walk graph Laplacian matrix L <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">rw</inf> , we first define reconstruction of a smooth signal x <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∗</sup> from samples y using graph shift variation (GSV) $\left\| {{{\mathbf{L}}_{rw}}{\mathbf{x}}} \right\|_2^2$ as a signal prior. To minimize the worst-case reconstruction error of the linear system solution x <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∗</sup> = C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−1</sup> H <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">⊤</sup> y with symmetric coefficient matrix${\mathbf{C}} = {{\mathbf{H}}^ \top }{\mathbf{H}} + \mu {\mathbf{L}}_{rw}^ \top {{\mathbf{L}}_{rw}}$, the E-optimality sampling objective is to choose H to maximize the smallest eigenvalue λ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</inf> (C) of C. To circumvent eigen-decomposition, we maximize instead a lower bound $\lambda _{\min }^ - \left( {{\mathbf{SC}}{{\mathbf{S}}^{ - 1}}} \right)$ of λ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</inf> (C)—smallest Gershgorin disc left-end of a similarity transform of C—via a variant of GDAS based on Gershgorin circle theorem (GCT). Experimental results show that our sampling method yielded smaller signal reconstruction errors at a faster speed compared to competing schemes.