pwc-archive/papers-with-abstracts huggingface dataset:
Masked Modeling Duo for Speech: Specializing General-Purpose Audio Representation to Speech using Denoising Distillation
Authors: Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi, Noboru Harada, Kunio Kashino
Date: 2023-05-23 00:00:00
Tasks: Denoising, Knowledge Distillation, Self-Supervised Learning

Self-supervised learning general-purpose audio representations have demonstrated high performance in a variety of tasks. Although they can be optimized for application by fine-tuning, even higher performance can be expected if they can be specialized to pre-train for an application. This paper explores the challenges and solutions in specializing general-purpose audio representations for a specific application using speech, a highly demanding field, as an example. We enhance Masked Modeling Duo (M2D), a general-purpose model, to close the performance gap with state-of-the-art (SOTA) speech models. To do so, we propose a new task, denoising distillation, to learn from fine-grained clustered features, and M2D for Speech (M2D-S), which jointly learns the denoising distillation task and M2D masked prediction task. Experimental results show that M2D-S performs comparably to or outperforms SOTA speech models on the SUPERB benchmark, demonstrating that M2D can specialize in a demanding field. Our code is available at: https://github.com/nttcslab/m2d/tree/master/speech
Multi-Stream Extension of Variational Bayesian HMM Clustering (MS-VBx) for Combined End-to-End and Vector Clustering-based Diarization
Authors: Marc Delcroix, Naohiro Tawara, Mireia Diez, Federico Landini, Anna Silnova, Atsunori Ogawa, Tomohiro Nakatani, Lukas Burget, Shoko Araki
Date: 2023-05-23 00:00:00
Tasks: Clustering, speaker-diarization, Speaker Diarization

Combining end-to-end neural speaker diarization (EEND) with vector clustering (VC), known as EEND-VC, has gained interest for leveraging the strengths of both methods. EEND-VC estimates activities and speaker embeddings for all speakers within an audio chunk and uses VC to associate these activities with speaker identities across different chunks. EEND-VC generates thus multiple streams of embeddings, one for each speaker in a chunk. We can cluster these embeddings using constrained agglomerative hierarchical clustering (cAHC), ensuring embeddings from the same chunk belong to different clusters. This paper introduces an alternative clustering approach, a multi-stream extension of the successful Bayesian HMM clustering of x-vectors (VBx), called MS-VBx. Experiments on three datasets demonstrate that MS-VBx outperforms cAHC in diarization and speaker counting performance.
Conditional Online Learning for Keyword Spotting
Authors: Michel Meneses, Bruno Iwami
Date: 2023-05-19 00:00:00
Tasks: Continual Learning, Keyword Spotting

Modern approaches for keyword spotting rely on training deep neural networks on large static datasets with i.i.d. distributions. However, the resulting models tend to underperform when presented with changing data regimes in real-life applications. This work investigates a simple but effective online continual learning method that updates a keyword spotter on-device via SGD as new data becomes available. Contrary to previous research, this work focuses on learning the same KWS task, which covers most commercial applications. During experiments with dynamic audio streams in different scenarios, that method improves the performance of a pre-trained small-footprint model by 34%. Moreover, experiments demonstrate that, compared to a naive online learning implementation, conditional model updates based on its performance in a small hold-out set drawn from the training distribution mitigate catastrophic forgetting.
Unsupervised ASR via Cross-Lingual Pseudo-Labeling
Authors: Tatiana Likhomanenko, Loren Lugosch, Ronan Collobert
Date: 2023-05-19 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Recent work has shown that it is possible to train an $\textit{unsupervised}$ automatic speech recognition (ASR) system using only unpaired audio and text. Existing unsupervised ASR methods assume that no labeled data can be used for training. We argue that even if one does not have any labeled audio for a given language, there is $\textit{always}$ labeled data available for other languages. We show that it is possible to use character-level acoustic models (AMs) from other languages to bootstrap an $\textit{unsupervised}$ AM in a new language. Here, "unsupervised" means no labeled audio is available for the $\textit{target}$ language. Our approach is based on two key ingredients: (i) generating pseudo-labels (PLs) of the $\textit{target}$ language using some $\textit{other}$ language AM and (ii) constraining these PLs with a $\textit{target language model}$. Our approach is effective on Common Voice: e.g. transfer of English AM to Swahili achieves 18% WER. It also outperforms character-based wav2vec-U 2.0 by 15% absolute WER on LJSpeech with 800h of labeled German data instead of 60k hours of unlabeled English data.
Distribution-aware Fairness Test Generation
Authors: Sai Sathiesh Rajan, Ezekiel Soremekun, Yves Le Traon, Sudipta Chattopadhyay
Date: 2023-05-08 00:00:00
Tasks: Autonomous Driving, Fairness, Object

Ensuring that all classes of objects are detected with equal accuracy is essential in AI systems. For instance, being unable to identify any one class of objects could have fatal consequences in autonomous driving systems. Hence, ensuring the reliability of image recognition systems is crucial. This work addresses how to validate group fairness in image recognition software. We propose a distribution-aware fairness testing approach (called DistroFair) that systematically exposes class-level fairness violations in image classifiers via a synergistic combination of out-of-distribution (OOD) testing and semantic-preserving image mutation. DistroFair automatically learns the distribution (e.g., number/orientation) of objects in a set of images. Then it systematically mutates objects in the images to become OOD using three semantic-preserving image mutations - object deletion, object insertion and object rotation. We evaluate DistroFair using two well-known datasets (CityScapes and MS-COCO) and three major, commercial image recognition software (namely, Amazon Rekognition, Google Cloud Vision and Azure Computer Vision). Results show that about 21% of images generated by DistroFair reveal class-level fairness violations using either ground truth or metamorphic oracles. DistroFair is up to 2.3x more effective than two main baselines, i.e., (a) an approach which focuses on generating images only within the distribution (ID) and (b) fairness analysis using only the original image dataset. We further observed that DistroFair is efficient, it generates 460 images per hour, on average. Finally, we evaluate the semantic validity of our approach via a user study with 81 participants, using 30 real images and 30 corresponding mutated images generated by DistroFair. We found that images generated by DistroFair are 80% as realistic as real-world images.
Robust Fault Estimators for Nonlinear Systems: An Ultra-Local Model Design
Authors: Farhad Ghanipoor, Carlos Murguia, Peyman Mohajerin Esfahani, Nathan van de Wouw
Date: 2023-05-23 00:00:00

This paper proposes a nonlinear estimator for the robust reconstruction of process and sensor faults for a class of uncertain nonlinear systems. The proposed fault estimation method augments the system dynamics with an ultra-local (in time) internal state-space representation (a finite chain of integrators) of the fault vector. Next, a nonlinear state observer is designed based on the known parts of the augmented dynamics. This nonlinear filter (observer) reconstructs the fault signal as well as the states of the augmented system. We provide sufficient conditions that guarantee stability of the estimation error dynamics: firstly, asymptotic stability (i.e., exact fault estimation) in the absence of perturbations induced by the fault model mismatch (mismatch between internal ultra-local model for the fault and the actual fault dynamics), uncertainty, external disturbances, and measurement noise and, secondly, Input-to-State Stability (ISS) of the estimation error dynamics is guaranteed in the presence of these perturbations. In addition, to support performance-based estimator design, we provide Linear Matrix Inequality (LMI) conditions for $\mathcal{L}_2$-gain and $\mathcal{L}_2-\mathcal{L}_\infty$ induced norm and cast the synthesis of the estimator gains as a semi-definite program where the effect of model mismatch and external disturbances on the fault estimation error is minimized in the sense of $\mathcal{L}_2$-gain, for an acceptable $\mathcal{L}_2-\mathcal{L}_\infty$ induced norm with respect to measurement noise. The latter result facilitates a design that explicitly addresses the performance trade-off between noise sensitivity and robustness against model mismatch and external disturbances. Finally, numerical results for a benchmark system illustrate the performance of the proposed methodologies.
Darwin: A DRAM-based Multi-level Processing-in-Memory Architecture for Data Analytics
Authors: Donghyuk Kim, Jae-Young Kim, Wontak Han, Jongsoon Won, Haerang Choi, Yongkee Kwon, Joo-Young Kim
Date: 2023-05-23 00:00:00
Tasks: CPU

Processing-in-memory (PIM) architecture is an inherent match for data analytics application, but we observe major challenges to address when accelerating it using PIM. In this paper, we propose Darwin, a practical LRDIMM-based multi-level PIM architecture for data analytics, which fully exploits the internal bandwidth of DRAM using the bank-, bank group-, chip-, and rank-level parallelisms. Considering the properties of data analytics operators and DRAM's area constraints, Darwin maximizes the internal data bandwidth by placing the PIM processing units, buffers, and control circuits across the hierarchy of DRAM. More specifically, it introduces the bank processing unit for each bank in which a single instruction multiple data (SIMD) unit handles regular data analytics operators and bank group processing unit for each bank group to handle workload imbalance in the condition-oriented data analytics operators. Furthermore, Darwin supports a novel PIM instruction architecture that concatenates instructions for multiple thread executions on bank group processing entities, addressing the command bottleneck by enabling separate control of up to 512 different in-memory processing units simultaneously. We build a cycle-accurate simulation framework to evaluate Darwin with various DRAM configurations, optimization schemes and workloads. Darwin achieves up to 14.7x speedup over the non-optimized version. Finally, the proposed Darwin architecture achieves 4.0x-43.9x higher throughput and reduces energy consumption by 85.7% than the baseline CPU system (Intel Xeon Gold 6226 + 4 channels of DDR4-2933). Compared to the state-of-the-art PIM, Darwin achieves up to 7.5x and 7.1x in the basic query operators and TPC-H queries, respectively. Darwin is based on the latest GDDR6 and requires only 5.6% area overhead, suggesting a promising PIM solution for the future main memory system.
Descent & Landing Trajectory and Guidance Algorithms with Divert Capabilities for Moon Landing
Authors: Francesco Capolupo, Antonio Rinalducci
Date: 2023-05-23 00:00:00

This paper presents the preliminary design of the descent and landing trajectory of the ESA Argonaut lunar lander. The mission scenario and driving system constraints are presented and accounted for in the design of a fuel-optimal trajectory that includes divert capabilities, as required to achieve a safe landing. A sub-optimal descent and landing trajectory is then presented and computed from the optimal one, and the related on-board guidance algorithms are derived. The proposed end-to-end guidance solution represents an easily implementable alternative to on-board optimization, minimizing the verification & validation effort, computational footprint, and programmatic risk in the development of the related GN&C capabilities. A dedicated off-line optimization process is also outlined, and exploited to optimize the propellant consumption of the sub-optimal trajectory and to ensure the fulfillment of system constraints despite the use of simple algorithms on-board. The sub-optimal trajectory is compared to the optimal baseline, and conclusions are drawn on the applicability of the proposed approach to the Argonaut mission.
Hardware Trojans in Power Conversion Circuits
Authors: Jacob Sillman, Ajay Suresh
Date: 2023-05-23 00:00:00

This report investigates the potential impact of a Trojan attack on power conversion circuits, specifically a switching signal attack designed to trigger a locking of the pulse width modulation (PWM) signal that goes to a power field-effect transistor (FET). The first simulation shows that this type of attack can cause severe overvoltage, potentially leading to functional failure. The report proposes a solution using a large bypass capacitor to force signal parity, effectively negating the Trojan circuit. The simulation results demonstrate that the proposed solution can effectively thwart the Trojan attack. However, several caveats must be considered, such as the size of the capacitor, possible current leakage, and the possibility that the solution can be circumvented by an adversary with knowledge of the protection strategy. Overall, the findings suggest that proper protection mechanisms, such as the proposed signal-parity solution, must be considered when designing power conversion circuits to mitigate the risk of Trojan attacks.
Encoding Carbon Emission Flow in Energy Management: A Compact Constraint Learning Approach
Authors: Linwei Sang, Yinliang Xu, Hongbin Sun
Date: 2023-05-22 00:00:00
Tasks: energy management, Management

Decarbonizing the energy supply is essential and urgent to mitigate the increasingly visible climate change. Its basis is identifying emission responsibility during power allocation by the carbon emission flow (CEF) model. However, the main challenge of CEF application is the intractable nonlinear relationship between carbon emission and power allocation. So this paper leverages the high approximation capability and the mixed-integer linear programming (MILP) representability of the deep neural networks to tackle the complex CEF model in carbon-electricity coordinated optimization. The compact constraint learning approach is proposed to learn the mapping from power injection to bus emission with sparse neural networks (SNNs). Then the trained SNNs are transformed equivalently as MILP constraints in the downstream optimization. In light of the ``high emission with high price'' principle, the blocked carbon price mechanism is designed to price emissions from the demand side. Based on the constraint learning and mechanism design, this paper proposes the carbon-aware energy management model in the tractable MILP form to unlock the carbon reduction potential from the demand side. The case study verifies the approximation accuracy and sparsity of SNN with fewer parameters for accelerating optimization solution and reduction effectiveness of demand-side capability for mitigating emission.
Demographic effects of aggregation in the presence of a component Allee effect
Authors: Daniel Cardoso Pereira Jorge, Ricardo Martinez-Garcia
Date: 2023-05-22 00:00:00

The component Allee effect (AE) is the positive correlation between an organism's fitness component and population density. Depending on the population spatial structure, which determines the interactions between organisms, a component AE might lead to positive density-dependence in the population per capita growth rate and establish a demographic AE. However, existing spatial models impose a fixed population spatial structure, which limits the understanding of how a component AE and spatial dynamics jointly determine the existence of demographic AEs. We introduce a spatially explicit theoretical framework where spatial structure and population dynamics are emergent properties of the individual-level demographic and movement rates. This framework predicts various spatial patterns depending on its specific parameterization, including evenly spaced aggregates of organisms, that determine the demographic-level by-products of the component AE. We find that aggregation increases population abundance and allows population survival in harsher environments and at lower global population densities when compared with uniformly distributed organisms. Moreover, aggregation can prevent the component AE from manifesting at the population level or restrict it to the level of each independent aggregate. These results provide a mechanistic understanding of how component AEs might operate for different spatial structures and manifest at larger scales.
Cyto- and bio-compatibility assessment of plasma-treated polyvinylidene fluoride scaffolds for cardiac tissue engineering
Authors: Maria Kitsara, Gaelle Revet, Jean-Sebastien, Vartanian-Grimaldi, Alexandre Simon, Mathilde Minguy, Antoine Miche, Vincent Humblot, Thierry Dufour, Onnik Agbulut
Date: 2023-05-16 00:00:00

As part of applications dealing with cardiovascular tissue engineering, drop-cast polyvinylidene fluoride (PVDF) scaffolds have been treated by cold plasma to enhance their adherence to cardiac cells. The scaffolds were treated in a dielectric barrier device where cold plasma was generated in a gaseous environment combining a carrier gas (helium or argon) with/without a reactive gas (molecular nitrogen). We show that an Ar-N2 plasma treatment of 10 min results in significant hydrophilization of the scaffolds, with contact angles as low as 52.4{\deg} instead of 132.2{\deg} for native PVDF scaffolds. Correlation between optical emission spectroscopy and X-ray photoelectron spectroscopy shows that OH radicals from the plasma phase can functionalize the surface scaffolds, resulting in improved wettability. For all plasma-treated PVDF scaffolds, the adhesion and maturation of primary cardiomyocytes is increased, showing a well-organized sarcomeric structure ({\alpha}-actinin immunostaining). The efficacy of plasma treatment was also supported by real-time PCR analysis to demonstrate an increased expression of the genes related to adhesion and cardiomyocyte function. Finally, the biocompatibility of the PVDF scaffolds was studied in a cardiac environment, after implantation of acellular scaffolds on the surface of the heart of healthy mice. Seven and 28 days after implantation, no exuberant fibrosis and no multinucleated giant cells were visible in the grafted area, hence demonstrating the absence of foreign body reaction and the biocompatibility of these scaffolds.
KineticNet: Deep learning a transferable kinetic energy functional for orbital-free density functional theory
Authors: Roman Remme, Tobias Kaczun, Maximilian Scheurer, Andreas Dreuw, Fred A. Hamprecht
Date: 2023-05-08 00:00:00
Tasks: GPU

Orbital-free density functional theory (OF-DFT) holds the promise to compute ground state molecular properties at minimal cost. However, it has been held back by our inability to compute the kinetic energy as a functional of the electron density only. We here set out to learn the kinetic energy functional from ground truth provided by the more expensive Kohn-Sham density functional theory. Such learning is confronted with two key challenges: Giving the model sufficient expressivity and spatial context while limiting the memory footprint to afford computations on a GPU; and creating a sufficiently broad distribution of training data to enable iterative density optimization even when starting from a poor initial guess. In response, we introduce KineticNet, an equivariant deep neural network architecture based on point convolutions adapted to the prediction of quantities on molecular quadrature grids. Important contributions include convolution filters with sufficient spatial resolution in the vicinity of the nuclear cusp, an atom-centric sparse but expressive architecture that relays information across multiple bond lengths; and a new strategy to generate varied training data by finding ground state densities in the face of perturbations by a random external potential. KineticNet achieves, for the first time, chemical accuracy of the learned functionals across input densities and geometries of tiny molecules. For two electron systems, we additionally demonstrate OF-DFT density optimization with chemical accuracy.
3D Molecular Geometry Analysis with 2D Graphs
Authors: Zhao Xu, Yaochen Xie, Youzhi Luo, Xuan Zhang, Xinyi Xu, Meng Liu, Kaleb Dickerson, Cheng Deng, Maho Nakata, Shuiwang Ji
Date: 2023-05-01 00:00:00
Tasks: Deep Learning, Property Prediction, Self-Supervised Learning

Ground-state 3D geometries of molecules are essential for many molecular analysis tasks. Modern quantum mechanical methods can compute accurate 3D geometries but are computationally prohibitive. Currently, an efficient alternative to computing ground-state 3D molecular geometries from 2D graphs is lacking. Here, we propose a novel deep learning framework to predict 3D geometries from molecular graphs. To this end, we develop an equilibrium message passing neural network (EMPNN) to better capture ground-state geometries from molecular graphs. To provide a testbed for 3D molecular geometry analysis, we develop a benchmark that includes a large-scale molecular geometry dataset, data splits, and evaluation protocols. Experimental results show that EMPNN can efficiently predict more accurate ground-state 3D geometries than RDKit and other deep learning methods. Results also show that the proposed framework outperforms self-supervised learning methods on property prediction tasks.
A stable deep adversarial learning approach for geological facies generation
Authors: Ferdinand Bhavsar, Nicolas Desassis, Fabien Ors, Thomas Romary
Date: 2023-05-12 00:00:00
Tasks: Variational Inference

The simulation of geological facies in an unobservable volume is essential in various geoscience applications. Given the complexity of the problem, deep generative learning is a promising approach to overcome the limitations of traditional geostatistical simulation models, in particular their lack of physical realism. This research aims to investigate the application of generative adversarial networks and deep variational inference for conditionally simulating meandering channels in underground volumes. In this paper, we review the generative deep learning approaches, in particular the adversarial ones and the stabilization techniques that aim to facilitate their training. The proposed approach is tested on 2D and 3D simulations generated by the stochastic process-based model Flumy. Morphological metrics are utilized to compare our proposed method with earlier iterations of generative adversarial networks. The results indicate that by utilizing recent stabilization techniques, generative adversarial networks can efficiently sample from target data distributions. Moreover, we demonstrate the ability to simulate conditioned simulations through the latent variable model property of the proposed approach.