pwc-archive/papers-with-abstracts huggingface dataset:
Algebra of L-banded Matrices
Authors: Shunqi Huang, Lei Liu, Brian M. Kurkoski
Date: 2022-12-20 00:00:00

Convergence is a crucial issue in iterative algorithms. Damping is commonly employed to ensure the convergence of iterative algorithms. The conventional ways of damping are scalar-wise, and either heuristic or empirical. Recently, an analytically optimized vector damping was proposed for memory message-passing (iterative) algorithms. As a result, it yields a special class of covariance matrices called L-banded matrices. In this paper, we show these matrices have broad algebraic properties arising from their L-banded structure. In particular, compact analytic expressions for the LDL decomposition, the Cholesky decomposition, the determinant after a column substitution, minors, and cofactors are derived. Furthermore, necessary and sufficient conditions for an L-banded matrix to be definite, a recurrence to obtain the characteristic polynomial, and some other properties are given. In addition, we give new derivations of the determinant and the inverse. (It's crucial to emphasize that some works have independently studied matrices with this special structure, named as L-matrices. Specifically, L-banded matrices are regarded as L-matrices with real and finite entries.)
Quantifying different modeling frameworks using topological data analysis: a case study with zebrafish patterns
Authors: Electa Cleveland, Angela Zhu, Bjorn Sandstede, Alexandria Volkening
Date: 2022-12-22 00:00:00
Tasks: Topological Data Analysis

Mathematical models come in many forms across biological applications. In the case of complex, spatial dynamics and pattern formation, stochastic models also face two main challenges: pattern data is largely qualitative, and model realizations may vary significantly. Together these issues make it difficult to relate models and empirical data -- or even models and models -- limiting how different approaches can be combined to offer new insights into biology. These challenges also raise mathematical questions about how models are related, since alternative approaches to the same problem -- e.g., cellular Potts models; off-lattice, agent-based models; on-lattice, cellular automaton models; and continuum approaches -- treat uncertainty and implement cell behavior in different ways. To help open the door to future work on questions like these, here we adapt methods from topological data analysis and computational geometry to quantitatively relate two different models of the same biological process in a fair, comparable way. To center our work and illustrate concrete challenges, we focus on the example of zebrafish-skin pattern formation, and we relate patterns that arise from agent-based and cellular automaton models.
How different are self and nonself?
Authors: Andreas Mayer, Jonathan A. Levine, Christopher J. Russo, Quentin Marcou, William Bialek, Benjamin D. Greenbaum
Date: 2022-12-22 00:00:00

Biological and artificial networks routinely make reliable distinctions between similar inputs, and the rules for making these distinctions are learned. In some ways, self/nonself discrimination in the immune system is similar, being both reliable and (partly) learned through thymic selection. In contrast to other examples, we show that the distributions of self and nonself peptides are nearly identical but strongly inhomogeneous. Reliable discrimination is possible only because self-peptides are a particular finite sample drawn out of this distribution, and T cells can target the spaces in between these samples. In conventional learning problems, this would constitute overfitting and lead to disaster. Here, the strong inhomogeneities imply instead that the immune system gains by targeting peptides which are similar to self, with maximum sensitivity for sequences just one or two substitutions away. This prediction from the structure of the underlying distribution in sequence space agrees, for example, with the observed responses to mutation derived cancer neoantigens.
Combining mutation and recombination statistics to infer clonal families in antibody repertoires
Authors: Natanael Spisak, Gabriel Athènes, Thomas Dupic, Thierry Mora, Aleksandra M. Walczak
Date: 2022-12-22 00:00:00

B-cell repertoires are characterized by a diverse set of receptors of distinct specificities generated through two processes of somatic diversification: V(D)J recombination and somatic hypermutations. B cell clonal families stem from the same V(D)J recombination event, but differ in their hypermutations. Clonal families identification is key to understanding B-cell repertoire function, evolution and dynamics. We present HILARy (High-precision Inference of Lineages in Antibody Repertoires), an efficient, fast and precise method to identify clonal families from single- or paired-chain repertoire sequencing datasets. HILARy combines probabilistic models that capture the receptor generation and selection statistics with adapted clustering methods to achieve consistently high inference accuracy. It automatically leverages the phylogenetic signal of shared mutations in difficult repertoire subsets. Exploiting the high sensitivity of the method, we find the statistics of evolutionary properties such as the site frequency spectrum and dN/dS ratio do not depend on the junction length. We also identify a broad range of selection pressures spanning two orders of magnitude.
CDO calibration via Magnus Expansion and Deep Learning
Authors: Marco Di Francesco, Kevin Kamm
Date: 2022-12-23 00:00:00
Tasks: Deep Learning, GPU

In this paper, we improve the performance of the large basket approximation developed by Reisinger et al. to calibrate Collateralized Debt Obligations (CDO) to iTraxx market data. The iTraxx tranches and index are computed using a basket of size $K= 125$. In the context of the large basket approximation, it is assumed that this is sufficiently large to approximate it by a limit SPDE describing the portfolio loss of a basket with size $K\rightarrow \infty$. For the resulting SPDE, we show four different numerical methods and demonstrate how the Magnus expansion can be applied to efficiently solve the large basket SPDE with high accuracy. Moreover, we will calibrate a structural model to the available market data. For this, it is important to efficiently infer the so-called initial distances to default from the Credit Default Swap (CDS) quotes of the constituents of the iTraxx for the large basket approximation. We will show how Deep Learning techniques can help us to improve the performance of this step significantly. We will see in the end a good fit to the market data and develop a highly parallelizable numerical scheme using GPU and multithreading techniques.
Approaching Globally Optimal Energy Efficiency in Interference Networks via Machine Learning
Authors: Bile Peng, Karl-Ludwig Besser, Ramprasad Raghunath, Eduard A. Jorswieck
Date: 2022-11-25 00:00:00

This work presents a machine learning approach to optimize the energy efficiency (EE) in a multi-cell wireless network. This optimization problem is non-convex and its global optimum is difficult to find. In the literature, either simple but suboptimal approaches or optimal methods with high complexity and poor scalability are proposed. In contrast, we propose a machine learning framework to approach the global optimum. While the neural network (NN) training takes moderate time, application with the trained model requires very low computational complexity. In particular, we introduce a novel objective function based on stochastic actions to solve the non-convex optimization problem. Besides, we design a dedicated NN architecture for the multi-cell network optimization problems that is permutation-equivariant. It classifies channels according to their roles in the EE computation. In this way, we encode our domain knowledge into the NN design and shed light into the black box of machine learning. Training and testing results show that the proposed method without supervision and with reasonable computational effort achieves an EE close to the global optimum found by the branch-and-bound algorithm. Hence, the proposed approach balances between computational complexity and performance.
On Orthogonal and Symplectic Matrix Ensembles
Authors: Craig A. Tracy, Harold Widom
Date: 1995-09-17 00:00:00

The focus of this paper is on the probability, $E_\beta(0;J)$, that a set $J$ consisting of a finite union of intervals contains no eigenvalues for the finite $N$ Gaussian Orthogonal ($\beta=1$) and Gaussian Symplectic ($\beta=4$) Ensembles and their respective scaling limits both in the bulk and at the edge of the spectrum. We show how these probabilities can be expressed in terms of quantities arising in the corresponding unitary ($\beta=2$) ensembles. Our most explicit new results concern the distribution of the largest eigenvalue in each of these ensembles. In the edge scaling limit we show that these largest eigenvalue distributions are given in terms of a particular Painlev\'e II function.
MAViC: Multimodal Active Learning for Video Captioning
Authors: Gyanendra Das, Xavier Thomas, Anant Raj, Vikram Gupta
Date: 2022-12-11 00:00:00
Tasks: Active Learning, Decoder, Semantic Similarity, Semantic Textual Similarity, valid, Video Captioning

A large number of annotated video-caption pairs are required for training video captioning models, resulting in high annotation costs. Active learning can be instrumental in reducing these annotation requirements. However, active learning for video captioning is challenging because multiple semantically similar captions are valid for a video, resulting in high entropy outputs even for less-informative samples. Moreover, video captioning algorithms are multimodal in nature with a visual encoder and language decoder. Further, the sequential and combinatorial nature of the output makes the problem even more challenging. In this paper, we introduce MAViC which leverages our proposed Multimodal Semantics Aware Sequential Entropy (M-SASE) based acquisition function to address the challenges of active learning approaches for video captioning. Our approach integrates semantic similarity and uncertainty of both visual and language dimensions in the acquisition function. Our detailed experiments empirically demonstrate the efficacy of M-SASE for active learning for video captioning and improve on the baselines by a large margin.
Revisiting Orthogonality Regularization: A Study for Convolutional Neural Networks in Image Classification
Authors: Taehyeon Kim, Se-Young Yun
Date: 2022-06-23 00:00:00
Proceeding: IEEE Access 2022 6
Tasks: image-classification, Image Classification

Recent research in deep Convolutional Neural Networks(CNN) faces the challenges of vanishing/exploding gradient issues, training instability, and feature redundancy. Orthogonality Regularization(OR), which introduces a penalty function considering the orthogonality of neural networks, could be a remedy to these challenges but is surprisingly not popular in the literature. This work revisits the OR approaches and empirically answer the question: Even when comparing various regularizations like weight decay and spectral norm regularization, which is the most powerful OR technique? We begin by introducing the improvements of various regularization techniques, specifically focusing on OR approaches over a variety of architectures. After that, we disentangle the benefits of OR in the comparison of other regularization approaches with a connection on how they affect norm preservation effects and feature redundancy in the forward and backward propagation. Our investigations show that Kernel Orthogonality Regularization(KOR) approaches, which directly penalize the orthogonality of convolutional kernel matrices, consistently outperform other techniques. We propose a simple KOR method considering both row- and column- orthogonality, of which empirical performance is the most effective in mitigating the aforementioned challenges. We further discuss several circumstances in the recent CNN models on various benchmark datasets, wherein KOR gains more effectiveness.
Nothing Stands Alone: Relational Fake News Detection with Hypergraph Neural Networks
Authors: Ujun Jeong, Kaize Ding, Lu Cheng, Ruocheng Guo, Kai Shu, Huan Liu
Date: 2022-12-24 00:00:00
Tasks: Fake News Detection

Nowadays, fake news easily propagates through online social networks and becomes a grand threat to individuals and society. Assessing the authenticity of news is challenging due to its elaborately fabricated contents, making it difficult to obtain large-scale annotations for fake news data. Due to such data scarcity issues, detecting fake news tends to fail and overfit in the supervised setting. Recently, graph neural networks (GNNs) have been adopted to leverage the richer relational information among both labeled and unlabeled instances. Despite their promising results, they are inherently focused on pairwise relations between news, which can limit the expressive power for capturing fake news that spreads in a group-level. For example, detecting fake news can be more effective when we better understand relations between news pieces shared among susceptible users. To address those issues, we propose to leverage a hypergraph to represent group-wise interaction among news, while focusing on important news relations with its dual-level attention mechanism. Experiments based on two benchmark datasets show that our approach yields remarkable performance and maintains the high performance even with a small subset of labeled news data.
Streaming Traffic Flow Prediction Based on Continuous Reinforcement Learning
Authors: Yanan Xiao, Minyu Liu, Zichen Zhang, Lu Jiang, Minghao Yin, Jianan Wang
Date: 2022-12-24 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Time Series, Time Series Analysis

Traffic flow prediction is an important part of smart transportation. The goal is to predict future traffic conditions based on historical data recorded by sensors and the traffic network. As the city continues to build, parts of the transportation network will be added or modified. How to accurately predict expanding and evolving long-term streaming networks is of great significance. To this end, we propose a new simulation-based criterion that considers teaching autonomous agents to mimic sensor patterns, planning their next visit based on the sensor's profile (e.g., traffic, speed, occupancy). The data recorded by the sensor is most accurate when the agent can perfectly simulate the sensor's activity pattern. We propose to formulate the problem as a continuous reinforcement learning task, where the agent is the next flow value predictor, the action is the next time-series flow value in the sensor, and the environment state is a dynamically fused representation of the sensor and transportation network. Actions taken by the agent change the environment, which in turn forces the agent's mode to update, while the agent further explores changes in the dynamic traffic network, which helps the agent predict its next visit more accurately. Therefore, we develop a strategy in which sensors and traffic networks update each other and incorporate temporal context to quantify state representations evolving over time.
MURPHY: Relations Matter in Surgical Workflow Analysis
Authors: Shang Zhao, Yanzhe Liu, Qiyuan Wang, Dai Sun, Rong Liu, S. Kevin Zhou
Date: 2022-12-24 00:00:00
Tasks: Relation, Triplet

Autonomous robotic surgery has advanced significantly based on analysis of visual and temporal cues in surgical workflow, but relational cues from domain knowledge remain under investigation. Complex relations in surgical annotations can be divided into intra- and inter-relations, both valuable to autonomous systems to comprehend surgical workflows. Intra- and inter-relations describe the relevance of various categories within a particular annotation type and the relevance of different annotation types, respectively. This paper aims to systematically investigate the importance of relational cues in surgery. First, we contribute the RLLS12M dataset, a large-scale collection of robotic left lateral sectionectomy (RLLS), by curating 50 videos of 50 patients operated by 5 surgeons and annotating a hierarchical workflow, which consists of 3 inter- and 6 intra-relations, 6 steps, 15 tasks, and 38 activities represented as the triplet of 11 instruments, 8 actions, and 16 objects, totaling 2,113,510 video frames and 12,681,060 annotation entities. Correspondingly, we propose a multi-relation purification hybrid network (MURPHY), which aptly incorporates novel relation modules to augment the feature representation by purifying relational features using the intra- and inter-relations embodied in annotations. The intra-relation module leverages a R-GCN to implant visual features in different graph relations, which are aggregated using a targeted relation purification with affinity information measuring label consistency and feature similarity. The inter-relation module is motivated by attention mechanisms to regularize the influence of relational features based on the hierarchy of annotation types from the domain knowledge. Extensive experimental results on the curated RLLS dataset confirm the effectiveness of our approach, demonstrating that relations matter in surgical workflow analysis.
Robust computation of optimal transport by $β$-potential regularization
Authors: Shintaro Nakamura, Han Bao, Masashi Sugiyama
Date: 2022-12-26 00:00:00

Optimal transport (OT) has become a widely used tool in the machine learning field to measure the discrepancy between probability distributions. For instance, OT is a popular loss function that quantifies the discrepancy between an empirical distribution and a parametric model. Recently, an entropic penalty term and the celebrated Sinkhorn algorithm have been commonly used to approximate the original OT in a computationally efficient way. However, since the Sinkhorn algorithm runs a projection associated with the Kullback-Leibler divergence, it is often vulnerable to outliers. To overcome this problem, we propose regularizing OT with the \beta-potential term associated with the so-called $\beta$-divergence, which was developed in robust statistics. Our theoretical analysis reveals that the $\beta$-potential can prevent the mass from being transported to outliers. We experimentally demonstrate that the transport matrix computed with our algorithm helps estimate a probability distribution robustly even in the presence of outliers. In addition, our proposed method can successfully detect outliers from a contaminated dataset
Off-Policy Reinforcement Learning with Loss Function Weighted by Temporal Difference Error
Authors: Bumgeun Park, TaeYoung Kim, Woohyeon Moon, Luiz Felipe Vecchietti, Dongsoo Har
Date: 2022-12-26 00:00:00
Tasks: Deep Reinforcement Learning, OpenAI Gym, reinforcement-learning, Reinforcement Learning (RL)

Training agents via off-policy deep reinforcement learning (RL) requires a large memory, named replay memory, that stores past experiences used for learning. These experiences are sampled, uniformly or non-uniformly, to create the batches used for training. When calculating the loss function, off-policy algorithms assume that all samples are of the same importance. In this paper, we hypothesize that training can be enhanced by assigning different importance for each experience based on their temporal-difference (TD) error directly in the training objective. We propose a novel method that introduces a weighting factor for each experience when calculating the loss function at the learning stage. In addition to improving convergence speed when used with uniform sampling, the method can be combined with prioritization methods for non-uniform sampling. Combining the proposed method with prioritization methods improves sampling efficiency while increasing the performance of TD-based off-policy RL algorithms. The effectiveness of the proposed method is demonstrated by experiments in six environments of the OpenAI Gym suite. The experimental results demonstrate that the proposed method achieves a 33%~76% reduction of convergence speed in three environments and an 11% increase in returns and a 3%~10% increase in success rate for other three environments.
Improving Complex Knowledge Base Question Answering via Question-to-Action and Question-to-Question Alignment
Authors: Yechun Tang, Xiaoxia Cheng, Weiming Lu
Date: 2022-12-26 00:00:00
Tasks: Knowledge Base Question Answering, Language Modeling, Language Modelling, Question Answering, Question Rewriting

Complex knowledge base question answering can be achieved by converting questions into sequences of predefined actions. However, there is a significant semantic and structural gap between natural language and action sequences, which makes this conversion difficult. In this paper, we introduce an alignment-enhanced complex question answering framework, called ALCQA, which mitigates this gap through question-to-action alignment and question-to-question alignment. We train a question rewriting model to align the question and each action, and utilize a pretrained language model to implicitly align the question and KG artifacts. Moreover, considering that similar questions correspond to similar action sequences, we retrieve top-k similar question-answer pairs at the inference stage through question-to-question alignment and propose a novel reward-guided action sequence selection strategy to select from candidate action sequences. We conduct experiments on CQA and WQSP datasets, and the results show that our approach outperforms state-of-the-art methods and obtains a 9.88\% improvements in the F1 metric on CQA dataset. Our source code is available at https://github.com/TTTTTTTTy/ALCQA.
Kidney and Kidney Tumour Segmentation in CT Images
Authors: Qi Ming How, Hoi Leong Lee
Date: 2022-12-26 00:00:00
Tasks: Computed Tomography (CT), Segmentation

Automatic segmentation of kidney and kidney tumour in Computed Tomography (CT) images is essential, as it uses less time as compared to the current gold standard of manual segmentation. However, many hospitals are still reliant on manual study and segmentation of CT images by medical practitioners because of its higher accuracy. Thus, this study focuses on the development of an approach for automatic kidney and kidney tumour segmentation in contrast-enhanced CT images. A method based on Convolutional Neural Network (CNN) was proposed, where a 3D U-Net segmentation model was developed and trained to delineate the kidney and kidney tumour from CT scans. Each CT image was pre-processed before inputting to the CNN, and the effect of down-sampled and patch-wise input images on the model performance was analysed. The proposed method was evaluated on the publicly available 2021 Kidney and Kidney Tumour Segmentation Challenge (KiTS21) dataset. The method with the best performing model recorded an average training Dice score of 0.6129, with the kidney and kidney tumour Dice scores of 0.7923 and 0.4344, respectively. For testing, the model obtained a kidney Dice score of 0.8034, and a kidney tumour Dice score of 0.4713, with an average Dice score of 0.6374.
Modeling Nonlinear Dynamics in Continuous Time with Inductive Biases on Decay Rates and/or Frequencies
Authors: Tomoharu Iwata, Yoshinobu Kawahara
Date: 2022-12-26 00:00:00
Tasks: Time Series, Time Series Analysis

We propose a neural network-based model for nonlinear dynamics in continuous time that can impose inductive biases on decay rates and/or frequencies. Inductive biases are helpful for training neural networks especially when training data are small. The proposed model is based on the Koopman operator theory, where the decay rate and frequency information is used by restricting the eigenvalues of the Koopman operator that describe linear evolution in a Koopman space. We use neural networks to find an appropriate Koopman space, which are trained by minimizing multi-step forecasting and backcasting errors using irregularly sampled time-series data. Experiments on various time-series datasets demonstrate that the proposed method achieves higher forecasting performance given a single short training sequence than the existing methods.