Reviewerly/exHarmony huggingface dataset:
Helical phase steering via four-wave mixing in a closely cycled double-ladder atomic system

This work presents a theoretical analysis of the helical phase steering via a four-wave mixing (FWM) process in a closely cycled double-ladder atomic system. Our focus is to understand the contribution of the two-photon resonant process governed by a probe field and a driving laser in the control of the helical phase. The theoretical analysis allows us to understand the helical phase can be effectively steered when the two-photon resonant condition Δ=−δ is satisfied. Moreover, we investigate the superposition modes of the FWM field and a same-frequency Gaussian beam, which testify the two-photon resonant condition Δ=−δ plays an important role in the steering of the helical phase.
Generative Model based Highly Efficient Semantic Communication Approach for Image Transmission

Deep learning (DL) based semantic communication methods have been explored to transmit images efficiently in recent years. In this paper, we propose a generative model based semantic communication to further improve the efficiency of image transmission and protect private information. In particular, the transmitter extracts the interpretable latent representation from the original image by a generative model exploiting the GAN inversion method. We also employ a privacy filter and a knowledge base to erase private information and replace it with natural features in the knowledge base. The simulation results indicate that our proposed method achieves comparable quality of received images while significantly reducing communication costs compared to the existing methods.
WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks

Filter pruning has attracted increasing attention in recent years for its capacity in compressing and accelerating convolutional neural networks. Various data-independent criteria, including norm-based and relationship-based ones, were proposed to prune the most unimportant filters. However, these state-of-the-art criteria fail to fully consider the dissimilarity of filters, and thus might lead to performance degradation. In this paper, we first analyze the limitation of relationship-based criteria with examples, and then introduce a new data-independent criterion, Weighted Hybrid Criterion (WHC), to tackle the problems of both norm-based and relationship-based criteria. By taking the magnitude of each filter and the linear dependence between filters into consideration, WHC can robustly recognize the most redundant filters, which can be safely pruned without introducing severe performance degradation to networks. Extensive pruning experiments in a simple one-shot manner demonstrate the effectiveness of the proposed WHC. In particular, WHC can prune ResNet-50 on ImageNet with more than 42% of floating point operations reduced without any performance loss in top-5 accuracy.
Phonetic RNN-Transducer for Mispronunciation Diagnosis

Non-autoregressive models and, in particular, Connectionist Temporal Classification (CTC) models have been the most popular approaches towards mispronunciation detection and diagnosis (MDD) task. In this paper, we identify two important knowledge gaps in MDD that have not been well studied in existing MDD research. First, CTC-based MDD models often assume conditional independence in the predicted phonemes, and therefore prominent mispronunciation patterns are underutilized. Second, existing MDD approaches are constrained to use training data from a language-specific phoneme set, and therefore cannot distinguish accented sounds that are not present in the predefined phoneme set. In this paper, we propose a set of autoregressive phonetic Recurrent Neural Network Transducer (RNN-T) MDD models that are capable of capturing temporal dependence of mispronunciation patterns. We further devise an extended phoneme set and weakly supervised training strategy to allow the model to distinguish similar sounding phonemes from different languages. We evaluate the proposed method on the public L2-ARCTIC dataset. Results have shown the proposed phonetic RNN-T model achieves significant improvements in false acceptance rate compared to state-of-the-art methods.
TEA-PSE 3.0: Tencent-Ethereal-Audio-Lab Personalized Speech Enhancement System For ICASSP 2023 Dns-Challenge

This paper introduces the Unbeatable Team’s submission to the ICASSP 2023 Deep Noise Suppression (DNS) Challenge. We expand our previous work, TEA-PSE, to its upgraded version – TEA-PSE 3.0. Specifically, TEA-PSE 3.0 incorporates a residual LSTM after squeezed temporal convolution network (S-TCN) to enhance sequence modeling capabilities. Additionally, the local-global representation (LGR) structure is introduced to boost speaker information extraction, and multi-STFT resolution loss is used to effectively capture the time-frequency characteristics of the speech signals. Moreover, retraining methods are employed based on the freeze training strategy to fine-tune the system. According to the official results, TEA-PSE 3.0 ranks 1st in both ICASSP 2023 DNS-Challenge track 1 and track 2.
Personalizing Federated Learning with Over-The-Air Computations

Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner. Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server. But the training efficiency is often hindered by challenges arising from limited communication and data heterogeneity. In this paper, we present a distributed training paradigm that employs analog over-the-air computation to alleviate the communication bottleneck. Additionally, we leverage a bi-level optimization framework to personalize the federated learning model so as to cope with the data heterogeneity issue. As a result, it enhances the generalization and robustness of each client’s local model. We elaborate on the model training procedure and its advantages over conventional frameworks. We provide a convergence analysis that theoretically demonstrates the training efficiency. We also conduct extensive experiments to validate the efficacy of the proposed framework.
Any-to-Any Voice Conversion with F<sub>0</sub> and Timbre Disentanglement and Novel Timbre Conditioning

Despite recent advances in voice conversion (VC), it is still challenging to do real-time one-shot voice conversion with good control over timbre and F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</inf> . In this work, we present a PPG-based VC model that directly decodes waveforms. We designed a speaker conditioned decoder based on HiFi-GAN[1], along with a new discriminator that produces high quality audio. Using an F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</inf> prenet and F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</inf> augmented speaker encoder, we are able to control F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</inf> and timbre independently with high fidelity. Our objective and subjective evaluations show that our method is preferred over others in terms of audio quality, timbre similarity and prosody retention.
Multi-Rate Adaptive Transform Coding for Video Compression

Contemporary lossy image and video coding standards rely on transform coding, the process through which pixels are mapped to an alternative representation to facilitate efficient data compression. Despite impressive performance of end-to-end optimized compression with deep neural networks, the high computational and space demands of these models has prevented them from superseding the relatively simple transform coding found in conventional video codecs. In this study, we propose learned transforms and entropy coding that may either serve as (non)linear drop-in replacements, or enhancements for linear transforms in existing codecs. These transforms can be multi-rate, allowing a single model to operate along the entire rate-distortion curve. To demonstrate the utility of our framework, we augmented the DCT with learned quantization matrices and adaptive entropy coding to compress intra-frame AV1 block prediction residuals. We report substantial BD-rate and perceptual quality improvements over more complex nonlinear transforms at a fraction of the computational cost.
Textless Direct Speech-to-Speech Translation with Discrete Speech Representation

Research on speech-to-speech translation (S2ST) has progressed rapidly in recent years. Many end-to-end systems have been proposed and show advantages over conventional cascade systems, which are often composed of recognition, translation and synthesis sub-systems. However, most of end-to-end systems still rely on intermediate textual supervision during training, which makes it infeasible to work for languages without written forms. In this work, we propose a novel model, Textless Translatotron, which is based on Translatotron 2 [1], for training an end-to-end direct S2ST model without any textual supervision. Instead of jointly training with an auxiliary task predicting target phonemes as in Translatotron 2, the proposed model uses an auxiliary task predicting discrete speech representations which are obtained from learned or random speech quantizers. When a speech encoder pre-trained with unsupervised speech data is used for both models, the proposed model obtains translation quality nearly on-par with Translatotron 2 on the multilingual CVSS-C corpus [2] as well as the bilingual Fisher Spanish-English corpus [3]. On the latter, it outperforms the prior state-of-the-art textless model by +18.5 BLEU.
Analyzing research trends of sentiment analysis and its applications for Coronavirus disease (COVID-19): A systematic review

COVID-19 epidemic is one of the worst disaster which affected people worldwide. It has impacted whole civilization physically, monetarily, and also emotionally. Sentiment analysis is an important step to handle pandemic effectively. In this work, systematic literature review of sentiment analysis of Indian population towards COVID-19 and its vaccination is presented. Recent exiting works are considered from four primary databases including ACM, Web of Science, IEEE Explore, and Scopus. Total 40 publications from January 2020 to August 2022 are selected for systematic review after applying inclusion and exclusion algorithm. Existing works are analyzed in terms of various challenges encountered by the existing authors with collected datasets. It is analyzed that mainly three techniques namely lexical, machine and deep learning are used by various authors for sentiment analysis. Performance of various applied techniques are comparative analyzed. Direction of future research works with recommendations are highlighted.
Contrastive Speech Mixup for Low-Resource Keyword Spotting

Most of the existing neural-based models for keyword spotting (KWS) in smart devices require thousands of training samples to learn a decent audio representation. However, with the rising demand for smart devices to become more person-alized, KWS models need to adapt quickly to smaller user samples. To tackle this challenge, we propose a contrastive speech mixup (CosMix) learning algorithm for low-resource KWS. CosMix introduces an auxiliary contrastive loss to the existing mixup augmentation technique to maximize the relative similarity between the original pre-mixed samples and the augmented samples. The goal is to inject enhancing constraints to guide the model towards simpler but richer content-based speech representations from two augmented views (i.e. noisy mixed and clean pre-mixed utterances). We conduct our experiments on the Google Speech Command dataset, where we trim the size of the training set to as small as 2.5 mins per keyword to simulate a low-resource condition. Our experimental results show a consistent improvement in the performance of multiple models, which exhibits the effectiveness of our method.
Animal Re-Identification Algorithm for Posture Diversity

Re-identification (Re-ID) technology is important for wildlife conservation and intelligent farm management. With the development of deep learning, the performance of animal ReID based on computer vision has been improved. However, variations in animal pose push a negative impact on recognition performance. In this paper, a Multi-pose Feature Fusion Network (MPFNet) is proposed to improve the performance of the Re-ID. First, we construct three pose modules for the three postures, that is, standing, sitting, and lying, respectively. In each pose module, there are two parallel branches, one is a global branch for extracting global features, and the other is a local branch for extracting local features. In addition, to obtain more effective feature representations, we weighted fusion for the global branching of the three pose modules. We validate the efficiency of MPFNet on both the self-built MPDD dog dataset and the public ATRW Amur Tiger dataset. Experimental results show that MPFNet can obtain better recognition performance than other state-of-the-art Re-ID methods. The source of code will be public available at https://github.com/hezhimin7028/MPFNet.
Neural Diarization with Non-Autoregressive Intermediate Attractors

End-to-end neural diarization (EEND) with encoder-decoder-based attractors (EDA) is a promising method to handle the whole speaker diarization problem simultaneously with a single neural network. While the EEND model can produce all frame-level speaker labels simultaneously, it disregards output label dependency. In this work, we propose a novel EEND model that introduces the label dependency between frames. The proposed method generates non-autoregressive intermediate attractors to produce speaker labels at the lower layers and conditions the subsequent layers with these labels. While the proposed model works in a non-autoregressive manner, the speaker labels are refined by referring to the whole sequence of intermediate labels. The experiments with the two-speaker CALLHOME dataset show that the intermediate labels with the proposed non-autoregressive intermediate attractors boost the diarization performance. The proposed method with the deeper net-work benefits more from the intermediate labels, resulting in better performance and training throughput than EEND-EDA.
Learned Kalman Filtering in Latent Space with High-Dimensional Data

The Kalman filter (KF) is a widely-used algorithm for tracking dynamical systems that can be faithfully captured by state space (SS) models. The need to fully describe an SS model limits its applicability under complex settings, e.g., when tracking based on visual or graphical data. This challenge can be treated by mapping the measurements into latent features obeying some postulated closed-form SS model, and applying the KF in the latent space. However, the validity of this approximated SS model may constitute a limiting factor. In this work we tackle the challenges associated with tracking from high-dimensional measurements by jointly learning the KF along with the latent space mapping. Our proposed approach combines a learned encoder while tracking in the latent space using the recently proposed data-driven Kalman-Net, and having both modules jointly tuned from data. Our empirical results demonstrate that the proposed approach achieves improved performance over both model-based and data-driven techniques, by learning a surrogate latent representation that most facilitates tracking.
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices

Federated Learning (FL) is a technique to train models on distributed edge devices with local data samples. Differential Privacy (DP) can be applied with FL to provide a formal privacy guarantee for sensitive data on device. Our goal is to train a large neural network language model (NNLM) on compute-constrained devices while preserving privacy using FL and DP. However, the noise required to guarantee differential privacy increases as the model size grows, which often prevents convergence. We propose Partial Embedding Updates (PEU), a novel technique to reduce the impact of DP-noise by decreasing payload size. Furthermore, we adopt Low Rank Adaptation (LoRA) and Noise Contrastive Estimation (NCE) to reduce the memory demands of large models on compute-constrained devices. We demonstrate in simulation and with real devices that this combination of techniques makes it possible to train large-vocabulary language models while preserving accuracy and privacy.
FANet: An Arbitrary Direction Remote Sensing Object Detection Network Based on Feature Fusion and Angle Classification

High-precision remote sensing image object detection has broad application prospects in military defense, disaster emergency, urban planning, and other fields. However, the arbitrary orientation, dense arrangement, and small size of objects in remote sensing images lead to poor detection accuracy of existing methods. To achieve accurate detection, this paper proposes an arbitrary directional remote sensing object detection method, called FANet, based on feature fusion and angle classification. Initially, the angle prediction branch is introduced, and the circular smooth label method is used to transform the angle regression problem into a classification problem, which solves the difficult problem of abrupt changes in the boundaries of the rotating frame while realizing the object frame rotation. Subsequently, to extract robust remote sensing objects, innovative introduce pure convolutional model as a backbone network, while Conv is replaced by GSConv to reduce the number of parameters in the model along with ensuring detection accuracy. Finally, the strengthen connection feature pyramid network (SC-FPN) is proposed to redesign the lateral connection part for deep and shallow layer feature fusion, and add jump connections between the input and output of the same level feature map to enrich the feature semantic information. In addition, add a variable parameter to the original localization loss function to satisfy the bounding box regression accuracy under different IoU thresholds, and thus obtain more accurate object detection. The comprehensive experimental results on two public datasets for rotated object detection DOTA and HRSC2016 demonstrate the effectiveness of our method.
A Virtual Reality Mining Training Simulator for Proximity Detection

Many applications in industrial mining rely on large manually operated trucks to transport materials around the mine. These trucks are often enormous, with very limited visibility for the driver. The combination of limited visibility and a truck with a substantial amount of weight is a recipe for accidents resulting in severe property destruction or even loss of human life. To solve these issues, we implement a simulation of a LIDAR-based proximity collision detection system to notify drivers of an imminent collision with something (or someone) outside of their field of view. We have developed a virtual reality training simulation to help train miners how to use this new system. Our simulation focuses on two main participant types: truck drivers and ground workers. This separation allows both parties to gain experience operating around the other in a low-risk virtual environment. Our final result aggregated useful features from a variety of past works and enhanced them with more immersive input and output devices.
A Process to Support Heuristic Evaluation and Tree Testing from a UX Integrated Perspective

This paper presents a proposal for an integral user experience evaluation process, as well as the methodology followed to carry it out, and the validation achieved. In previous works, two processes were proposed: the first one, was to perform Heuristic Evaluations, and the second one, was to perform Tree Testing Evaluations. Both processes are aligned with User Experience improvement. Heuristic Evaluation is a method that allows UX and HCI specialists to evaluate a software product through a set of heuristics to find usability problems. Tree Testing or Reverse Card Sorting is a method based on the evaluation by end users of the information architecture of a software product so that the hypotheses that led to the proposed design can be validated early and improvements in the design can be made. This paper presents how, through a collaborative and continuous improvement approach, an integrated process was developed to perform the aforementioned methods, both in face-to-face and remote schemes, with an integral approach. In addition, this proposal lays the groundwork for adding other UX evaluation methods and techniques in the future. To carry out this, an AS IS was considered, a remote collaborative workshop was held with the stakeholders involved, and with the identified improvement points, the integral process was elaborated, which was captured in BPMN diagrams. Finally, it was verified that the proposal covered all the improvement points identified in the workshop, and the proposal was validated through expert judgment. The collaborative experience and the feedback obtained by both workshop participants and experts led us to conclude the importance of involving users throughout the improvement experience, with a user-centered focus. This proposal is important because this process can be implemented in any organization that performs UX evaluations. Likewise, a software platform can be implemented, so that the automation and standardization of this type of process can be achieved.
Structural and Chemical Complexity of Minerals: The Information-Based Approach

A general overview of the theory of complexity of minerals is given based on the approximation of minerals as mostly ideal crystals with idealized chemical compositions (but taking into account chemical substitutions). The procedures for the calculation of chemical and structural complexities from information-theoretic arguments are given, and their relations are analyzed. The concept of information landscapes is introduced that are complementary to energy landscapes. The relations between complexity and symmetry, complexity and entropy, complexity and thermodynamic parameters, complexity and metastable crystallization are reviewed. The complexity of minerals is increasing in the course of geological history, both in structural and chemical terms. Historically, more and more complex minerals come to the attention of mineralogists, which is demonstrated by the analysis of temporal dynamics of mineralogical discoveries. The analysis of relative complexity of living organisms and minerals from the information-theoretic viewpoint reveal their drastic difference, even not considering functional complexity of living beings.
The Evolution of Mineral Evolution

The diversity and distribution of Earth’s minerals have evolved through almost 4.57 billion years as a consequence of changing physical, chemical, and ultimately biological processes. “Mineral evolution,” the study of planetary mineralogy through deep time, has undergone its own evolution, from a qualitative narrative of successive stages of change, to a more quantitative exploration of mineral occurrences as a function of age, and ultimately as catalyst for the newly emerging field of mineral informatics. A key feature of this evolution is the recognition that every mineral specimen is an information-rich “time capsule,” waiting to be opened. Each sample reveals its history through hundreds of attributes: trace and minor elements, stable and radioactive isotopes, solid and fluid inclusions, and myriad non-ideal structural, optical, magnetic, electrical, elastic, and other properties. Increasing realization of the data-rich character of minerals has pointed the way to an “evolutionary system of mineralogy,” which complements the official IMA classification scheme based on the Dana system.