pwc-archive/papers-with-abstracts huggingface dataset:
TAPLoss: A Temporal Acoustic Parameter Loss for Speech Enhancement
Authors: Yunyang Zeng, Joseph Konan, Shuo Han, David Bick, Muqiao Yang, Anurag Kumar, Shinji Watanabe, Bhiksha Raj
Date: 2023-02-16 00:00:00
Tasks: Speaker Recognition, Speech Enhancement

Speech enhancement models have greatly progressed in recent years, but still show limits in perceptual quality of their speech outputs. We propose an objective for perceptual quality based on temporal acoustic parameters. These are fundamental speech features that play an essential role in various applications, including speaker recognition and paralinguistic analysis. We provide a differentiable estimator for four categories of low-level acoustic descriptors involving: frequency-related parameters, energy or amplitude-related parameters, spectral balance parameters, and temporal features. Unlike prior work that looks at aggregated acoustic parameters or a few categories of acoustic parameters, our temporal acoustic parameter (TAP) loss enables auxiliary optimization and improvement of many fine-grain speech characteristics in enhancement workflows. We show that adding TAPLoss as an auxiliary objective in speech enhancement produces speech with improved perceptual quality and intelligibility. We use data from the Deep Noise Suppression 2020 Challenge to demonstrate that both time-domain models and time-frequency domain models can benefit from our method.
Document Flattening: Beyond Concatenating Context for Document-Level Neural Machine Translation
Authors: Minghao Wu, George Foster, Lizhen Qu, Gholamreza Haffari
Date: 2023-02-16 00:00:00
Tasks: Machine Translation, Translation

Existing work in document-level neural machine translation commonly concatenates several consecutive sentences as a pseudo-document, and then learns inter-sentential dependencies. This strategy limits the model's ability to leverage information from distant context. We overcome this limitation with a novel Document Flattening (DocFlat) technique that integrates Flat-Batch Attention (FBA) and Neural Context Gate (NCG) into Transformer model to utilize information beyond the pseudo-document boundaries. FBA allows the model to attend to all the positions in the batch and learns the relationships between positions explicitly and NCG identifies the useful information from the distant context. We conduct comprehensive experiments and analyses on three benchmark datasets for English-German translation, and validate the effectiveness of two variants of DocFlat. Empirical results show that our approach outperforms strong baselines with statistical significance on BLEU, COMET and accuracy on the contrastive test set. The analyses highlight that DocFlat is highly effective in capturing the long-range information.
Tree-Based Representation and Generation of Natural and Mathematical Language
Authors: Alexander Scarlatos, Andrew Lan
Date: 2023-02-15 00:00:00
Tasks: Math, Mathematical Reasoning, valid

Mathematical language in scientific communications and educational scenarios is important yet relatively understudied compared to natural languages. Recent works on mathematical language focus either on representing stand-alone mathematical expressions, especially in their natural tree format, or mathematical reasoning in pre-trained natural language models. Existing works on jointly modeling and generating natural and mathematical languages simply treat mathematical expressions as text, without accounting for the rigid structural properties of mathematical expressions. In this paper, we propose a series of modifications to existing language models to jointly represent and generate text and math: representing mathematical expressions as sequences of node tokens in their operator tree format, using math symbol and tree position embeddings to preserve the semantic and structural properties of mathematical expressions, and using a constrained decoding method to generate mathematically valid expressions. We ground our modifications in GPT-2, resulting in a model MathGPT, and demonstrate that it outperforms baselines on mathematical expression generation tasks.
Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Models
Authors: Abteen Ebrahimi, Arya D. McCarthy, Arturo Oncevay, Luis Chiruzzo, John E. Ortega, Gustavo A. Giménez-Lugo, Rolando Coto-Solano, Katharina Kann
Date: 2023-02-15 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Part-Of-Speech Tagging, Word Alignment

Large multilingual models have inspired a new class of word alignment methods, which work well for the model's pretraining languages. However, the languages most in need of automatic alignment are low-resource and, thus, not typically included in the pretraining data. In this work, we ask: How do modern aligners perform on unseen languages, and are they better than traditional methods? We contribute gold-standard alignments for Bribri--Spanish, Guarani--Spanish, Quechua--Spanish, and Shipibo-Konibo--Spanish. With these, we evaluate state-of-the-art aligners with and without model adaptation to the target language. Finally, we also evaluate the resulting alignments extrinsically through two downstream tasks: named entity recognition and part-of-speech tagging. We find that although transformer-based methods generally outperform traditional models, the two classes of approach remain competitive with each other.
Photonic Neural Networks: A Compact Review
Authors: Mohammad Ahmadi, Hamidreza Bolhasani
Date: 2023-02-16 00:00:00
Tasks: Articles, valid

It has long been known that photonic science and especially photonic communications can raise the speed of technologies and producing manufacturing. More recently, photonic science has also been interested in its capabilities to implement low-precision linear operations, such as matrix multiplications, fast and effciently. For a long time most scientists taught that Electronics is the end of science but after many years and about 35 years ago had been understood that electronics do not answer alone and should have a new science. Today we face modern ways and instruments for doing tasks as soon as possible in proportion to many decays before. The velocity of progress in science is very fast. All our progress in science area is dependent on modern knowledge about new methods. In this research, we want to review the concept of a photonic neural network. For this research was selected 18 main articles were among the main 30 articles on this subject from 2015 to the 2022 year. These articles noticed three principles: 1- Experimental concepts, 2- Theoretical concepts, and, finally 3- Mathematic concepts. We should be careful with this research because mathematics has a very important and constructive role in our topics! One of the topics that are very valid and also new, is simulation. We used to work with simulation in some parts of this research. First, briefly, we start by introducing photonics and neural networks. In the second we explain the advantages and disadvantages of a combination of both in the science world and industries and technologies about them. Also, we are talking about the achievements of a thin modern science. Third, we try to introduce some important and valid parameters in neural networks. In this manner, we use many mathematic tools in some portions of this article.
Fourier Analysis Meets Runtime Analysis: Precise Runtimes on Plateaus
Authors: Benjamin Doerr, Andrew James Kelley
Date: 2023-02-16 00:00:00
Tasks: Evolutionary Algorithms

We propose a new method based on discrete Fourier analysis to analyze the time evolutionary algorithms spend on plateaus. This immediately gives a concise proof of the classic estimate of the expected runtime of the $(1+1)$ evolutionary algorithm on the Needle problem due to Garnier, Kallel, and Schoenauer (1999). We also use this method to analyze the runtime of the $(1+1)$ evolutionary algorithm on a new benchmark consisting of $n/\ell$ plateaus of effective size $2^\ell-1$ which have to be optimized sequentially in a LeadingOnes fashion. Using our new method, we determine the precise expected runtime both for static and fitness-dependent mutation rates. We also determine the asymptotically optimal static and fitness-dependent mutation rates. For $\ell = o(n)$, the optimal static mutation rate is approximately $1.59/n$. The optimal fitness dependent mutation rate, when the first $k$ fitness-relevant bits have been found, is asymptotically $1/(k+1)$. These results, so far only proven for the single-instance problem LeadingOnes, thus hold for a much broader class of problems. We expect similar extensions to be true for other important results on LeadingOnes. We are also optimistic that our Fourier analysis approach can be applied to other plateau problems as well.
Ultrafast single-channel machine vision based on neuro-inspired photonic computing
Authors: Tomoya Yamaguchi, Kohei Arai, Tomoaki Niiyama, Atsushi Uchida, Satoshi Sunada
Date: 2023-02-15 00:00:00
Tasks: Anomaly Detection

High-speed machine vision is increasing its importance in both scientific and technological applications. Neuro-inspired photonic computing is a promising approach to speed-up machine vision processing with ultralow latency. However, the processing rate is fundamentally limited by the low frame rate of image sensors, typically operating at tens of hertz. Here, we propose an image-sensor-free machine vision framework, which optically processes real-world visual information with only a single input channel, based on a random temporal encoding technique. This approach allows for compressive acquisitions of visual information with a single channel at gigahertz rates, outperforming conventional approaches, and enables its direct photonic processing using a photonic reservoir computer in a time domain. We experimentally demonstrate that the proposed approach is capable of high-speed image recognition and anomaly detection, and furthermore, it can be used for high-speed imaging. The proposed approach is multipurpose and can be extended for a wide range of applications, including tracking, controlling, and capturing sub-nanosecond phenomena.
Motivation literally. Construction and expression of educational aspirations on Parcoursup
Authors: Marie-Paule Couto, Marion Valarcher
Date: 2023-02-16 00:00:00
Tasks: Sociology

This paper analyses the framing and expression of French high school students' aspirations. It sheds new light on the inequalities in tracking between academic versus technological and vocational track. Through the analysis of a national survey and a corpus of cover letters written by applicants for a sociology degree, it shows that, due to the lack of means, teachers mainly have two types of guidance support strategies.Teachers use to target and concentrate their supporting on ``good students'' in vocational tracks, while, in academic tracks, they delegate some steps of the tracking procedures to families. These different strategies have effects on the way high school students internalise school prescriptions and restitute them in cover letters. Through the close support they benefit from teachers, ``good students'' in vocational tracks strongly internalise the instructions and their place in the school hierarchy. In academic tracks, students' expression of the aspirations is much more dependent of their familial capital.
Reimagining Demand-Side Management with Mean Field Learning
Authors: Bianca Marin Moreno, Margaux Brégère, Pierre Gaillard, Nadia Oudjane
Date: 2023-02-16 00:00:00
Tasks: Management

Integrating renewable energy into the power grid while balancing supply and demand is a complex issue, given its intermittent nature. Demand side management (DSM) offers solutions to this challenge. We propose a new method for DSM, in particular the problem of controlling a large population of electrical devices to follow a desired consumption signal. We model it as a finite horizon Markovian mean field control problem. We develop a new algorithm, MD-MFC, which provides theoretical guarantees for convex and Lipschitz objective functions. What distinguishes MD-MFC from the existing load control literature is its effectiveness in directly solving the target tracking problem without resorting to regularization techniques on the main problem. A non-standard Bregman divergence on a mirror descent scheme allows dynamic programming to be used to obtain simple closed-form solutions. In addition, we show that general mean-field game algorithms can be applied to this problem, which expands the possibilities for addressing load control problems. We illustrate our claims with experiments on a realistic data set.
New $\sqrt{n}$-consistent, numerically stable higher-order influence function estimators
Authors: Lin Liu, Chang Li
Date: 2023-02-16 00:00:00
Tasks: Epidemiology

Higher-Order Influence Functions (HOIFs) provide a unified theory for constructing rate-optimal estimators for a large class of low-dimensional (smooth) statistical functionals/parameters (and sometimes even infinite-dimensional functions) that arise in substantive fields including epidemiology, economics, and the social sciences. Since the introduction of HOIFs by Robins et al. (2008), they have been viewed mostly as a theoretical benchmark rather than a useful tool for statistical practice. Works aimed to flip the script are scant, but a few recent papers Liu et al. (2017, 2021b) make some partial progress. In this paper, we take a fresh attempt at achieving this goal by constructing new, numerically stable HOIF estimators (or sHOIF estimators for short with ``s'' standing for ``stable'') with provable statistical, numerical, and computational guarantees. This new class of sHOIF estimators (up to the 2nd order) was foreshadowed in synthetic experiments conducted by Liu et al. (2020a).
A Geometric Reduction Approach for Identity Testing of Reversible Markov Chains
Authors: Geoffrey Wolfer, Shun Watanabe
Date: 2023-02-16 00:00:00

We consider the problem of testing the identity of a reversible Markov chain against a reference from a single trajectory of observations. Employing the recently introduced notion of a lumping-congruent Markov embedding, we show that, at least in a mildly restricted setting, testing identity to a reversible chain reduces to testing to a symmetric chain over a larger state space and recover state-of-the-art sample complexity for the problem.
Improved Discretization Analysis for Underdamped Langevin Monte Carlo
Authors: Matthew Zhang, Sinho Chewi, Mufan Bill Li, Krishnakumar Balasubramanian, Murat A. Erdogdu
Date: 2023-02-16 00:00:00

Underdamped Langevin Monte Carlo (ULMC) is an algorithm used to sample from unnormalized densities by leveraging the momentum of a particle moving in a potential well. We provide a novel analysis of ULMC, motivated by two central questions: (1) Can we obtain improved sampling guarantees beyond strong log-concavity? (2) Can we achieve acceleration for sampling? For (1), prior results for ULMC only hold under a log-Sobolev inequality together with a restrictive Hessian smoothness condition. Here, we relax these assumptions by removing the Hessian smoothness condition and by considering distributions satisfying a Poincar\'e inequality. Our analysis achieves the state of art dimension dependence, and is also flexible enough to handle weakly smooth potentials. As a byproduct, we also obtain the first KL divergence guarantees for ULMC without Hessian smoothness under strong log-concavity, which is based on a new result on the log-Sobolev constant along the underdamped Langevin diffusion. For (2), the recent breakthrough of Cao, Lu, and Wang (2020) established the first accelerated result for sampling in continuous time via PDE methods. Our discretization analysis translates their result into an algorithmic guarantee, which indeed enjoys better condition number dependence than prior works on ULMC, although we leave open the question of full acceleration in discrete time. Both (1) and (2) necessitate R\'enyi discretization bounds, which are more challenging than the typically used Wasserstein coupling arguments. We address this using a flexible discretization analysis based on Girsanov's theorem that easily extends to more general settings.
Fairly Adaptive Negative Sampling for Recommendations
Authors: Xiao Chen, Wenqi Fan, Jingfan Chen, Haochen Liu, Zitao Liu, Zhaoxiang Zhang, Qing Li
Date: 2023-02-16 00:00:00
Tasks: Attribute, Fairness

Pairwise learning strategies are prevalent for optimizing recommendation models on implicit feedback data, which usually learns user preference by discriminating between positive (i.e., clicked by a user) and negative items (i.e., obtained by negative sampling). However, the size of different item groups (specified by item attribute) is usually unevenly distributed. We empirically find that the commonly used uniform negative sampling strategy for pairwise algorithms (e.g., BPR) can inherit such data bias and oversample the majority item group as negative instances, severely countering group fairness on the item side. In this paper, we propose a Fairly adaptive Negative sampling approach (FairNeg), which improves item group fairness via adaptively adjusting the group-level negative sampling distribution in the training process. In particular, it first perceives the model's unfairness status at each step and then adjusts the group-wise sampling distribution with an adaptive momentum update strategy for better facilitating fairness optimization. Moreover, a negative sampling distribution Mixup mechanism is proposed, which gracefully incorporates existing importance-aware sampling techniques intended for mining informative negative samples, thus allowing for achieving multiple optimization purposes. Extensive experiments on four public datasets show our proposed method's superiority in group fairness enhancement and fairness-utility tradeoff.
M3E-Yolo: A New Lightweight Network for Traffic Sign Recognition
Authors: Guo Haoran, Li Fan, Kuang Ping, Xiong Gang
Date: 2023-01-19 00:00:00
Proceeding: 19th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) 2023 1
Tasks: Traffic Sign Detection, Traffic Sign Recognition

Traffic sign recognition is committed to ensuring the safety of automatic driving. Inspired by YOLOv5, this paper proposes a new model to solve the problem of poor balance between the accuracy and efficiency of existing algorithms in traffic sign recognition. Firstly, the lightweight network MobileNetV3 is introduced for feature extraction to reduce the number of parameters. Secondly, attention mechanism module is introduced to enhance channel features, which makes up for the reduced accuracy caused by the simplified model. Experiments show that the mAP value trained by our model on the Chinese traffic sign dataset reaches 93.6%, which is similar to the level of YOLOv5, and the number of parameters is less than a quarter of YOLOv5.
Elastic-Link for Binarized Neural Network
Authors: Jie Hu, Ziheng Wu, Vince Tan, Zhilin Lu, Mengze Zeng, Enhua Wu
Date: 2021-12-19 00:00:00
Tasks: Binarization

Recent work has shown that Binarized Neural Networks (BNNs) are able to greatly reduce computational costs and memory footprints, facilitating model deployment on resource-constrained devices. However, in comparison to their full-precision counterparts, BNNs suffer from severe accuracy degradation. Research aiming to reduce this accuracy gap has thus far largely focused on specific network architectures with few or no 1x1 convolutional layers, for which standard binarization methods do not work well. Because 1x1 convolutions are common in the design of modern architectures (e.g. GoogleNet, ResNet, DenseNet), it is crucial to develop a method to binarize them effectively for BNNs to be more widely adopted. In this work, we propose an "Elastic-Link" (EL) module to enrich information flow within a BNN by adaptively adding real-valued input features to the subsequent convolutional output features. The proposed EL module is easily implemented and can be used in conjunction with other methods for BNNs. We demonstrate that adding EL to BNNs produces a significant improvement on the challenging large-scale ImageNet dataset. For example, we raise the top-1 accuracy of binarized ResNet26 from 57.9% to 64.0%. EL also aids convergence in the training of binarized MobileNet, for which a top-1 accuracy of 56.4% is achieved. Finally, with the integration of ReActNet, it yields a new state-of-the-art result of 71.9% top-1 accuracy.
Automated Heartbeat Classification Using 3-D Inputs Based on Convolutional Neural Network With Multi-Fields of View
Authors: FEITENG LI 1, YIN XU 1, ZHIJIAN CHEN 1, AND ZHENYAN LIU2
Date: 2019-06-10 00:00:00
Proceeding: IEEEXplore 2019 6
Tasks: Heartbeat Classification

A high-performance method of automated heartbeat classification based on Convolutional
Neural Network (CNN) is proposed in this paper. To make full use of the electrocardiogram information
acquired from different parts of the human body, we present a novel 3-D data structure as the input of
the CNN. The 3-D structure consists of multiple feature maps, each of which indicates the information
collected from one lead and contains morphological characteristic, RR-interval, and beat-to-beat correlation
feature. Besides, atrous spatial pyramid pooling (ASPP) module which uses filters with different resolutions
is adopted to extract deep features in multi-fields of view. Validated on the MIT-BIH arrhythmia database,
the proposed method yields an overall accuracy of 91.44% in the inter-patient practice. In particular, this
method achieves 89.05% and 95.15% in the sensitivities of the supraventricular ectopic beat (SVEB) and
ventricular ectopic beat (VEB) classes, respectively. With the high performance in detecting these two
pathological classes, this method has potential clinical application.
Unboxing Tree Ensembles for interpretability: a hierarchical visualization tool and a multivariate optimal re-built tree
Authors: Giulia Di Teodoro, Marta Monaci, Laura Palagi
Date: 2023-02-15 00:00:00
Tasks: feature selection

The interpretability of models has become a crucial issue in Machine Learning because of algorithmic decisions' growing impact on real-world applications. Tree ensemble methods, such as Random Forests or XgBoost, are powerful learning tools for classification tasks. However, while combining multiple trees may provide higher prediction quality than a single one, it sacrifices the interpretability property resulting in "black-box" models. In light of this, we aim to develop an interpretable representation of a tree-ensemble model that can provide valuable insights into its behavior. First, given a target tree-ensemble model, we develop a hierarchical visualization tool based on a heatmap representation of the forest's feature use, considering the frequency of a feature and the level at which it is selected as an indicator of importance. Next, we propose a mixed-integer linear programming (MILP) formulation for constructing a single optimal multivariate tree that accurately mimics the target model predictions. The goal is to provide an interpretable surrogate model based on oblique hyperplane splits, which uses only the most relevant features according to the defined forest's importance indicators. The MILP model includes a penalty on feature selection based on their frequency in the forest to further induce sparsity of the splits. The natural formulation has been strengthened to improve the computational performance of {mixed-integer} software. Computational experience is carried out on benchmark datasets from the UCI repository using a state-of-the-art off-the-shelf solver. Results show that the proposed model is effective in yielding a shallow interpretable tree approximating the tree-ensemble decision function.