pwc-archive/papers-with-abstracts huggingface dataset:
Optimal Sampled-Data Control of a Nonlinear System
Authors: Yasuaki Oishi, Noboru Sakamoto
Date: 2021-12-29 00:00:00

Optimal sampled-data control of a nonlinear system is considered with the stable-manifold approach and extensive use of numerical techniques. The idea is to notice the Hamiltonian system associated with the considered optimal control problem and to compute trajectories on its stable manifold. Since the control input accompanied with those trajectories is proved to be optimal, the optimal control law can be obtained through interpolation. The stable-manifold approach was originally proposed for continuous-time optimal control and here it is adapted for sampled-data control based on the works of Navasca. In the case of sampled-data control, the approach requires the state transition of the controlled plant during one sampling period together with its derivatives with respect to the state and the input. Their computation is achieved by numerical techniques. Moreover, a shooting method is proposed for systematic generation of the trajectories and extension is considered for the intersample behavior to be taken into account. The proposed method is applied to tracking control of a wheeled mobile robot. It works successfully with a rather long sampling period.
Probabilistic models, compressible interactions, and neural coding
Authors: Luisa Ramirez, William Bialek, Stephanie E. Palmer, David J. Schwab
Date: 2021-12-28 00:00:00

In physics we often use very simple models to describe systems with many degrees of freedom, but it is not clear why or how this success can be transferred to the more complex biological context. We consider models for the joint distribution of many variables, as with the combinations of spiking and silence in large networks of neurons. In this probabilistic framework, we argue that simple models are possible if the mutual information between two halves of the system is consistently sub--extensive, and if this shared information is compressible. These conditions are not met generically, but they are met by real world data such as natural images and the activity in a population of retinal output neurons. We introduce compression strategies that combine the information bottleneck with an iteration scheme inspired by the renormalization group, and find that the number of parameters needed to describe the distribution of joint activity scales with the square of the number of neurons, even though the interactions are not well approximated as pairwise. Our results also show that this shared information is essentially equal to the information that individual neurons carry about natural visual inputs, which has surprising implications for the neural code.
Spatial Distribution Patterns of Clownfish in Recirculating Aquaculture Systems
Authors: Fahad Aljehani, Ibrahima N'Doye, Micaela S. Justo, John E. Majoris, Michael L. Berumen, Taous-Meriem Laleg-Kirati
Date: 2021-12-29 00:00:00
Tasks: Optical Flow Estimation

Successful aquaculture systems can reduce the pressure and help secure the most diverse and productive Red Sea coral reef ecosystem to maintain a healthy and functional ecosystem within a sustainable blue economy. Interestingly, recirculating aquaculture systems are currently emerging in fish farm production practices. On the other hand, monitoring and detecting fish behaviors provide essential information on fish welfare and contribute to an intelligent production in global aquaculture. This work proposes an efficient approach to analyze the spatial distribution status and motion patterns of juvenile clownfish \textit{(Amphiprion bicinctus)} maintained in aquaria at three stocking densities (1, 5, and 10 individuals/aquarium). The estimated displacement is crucial in assessing the dispersion and velocity to express the clownfish's spatial distribution and movement behavior in a recirculating aquaculture system. Indeed, we aim to compute the velocity, magnitude, and turning angle using an optical flow method to assist aquaculturists in efficiently monitoring and identifying fish behavior. We test the system design on a database containing two days of video streams of juvenile clownfish maintained in aquaria. The proposed displacement estimation reveals good performance in measuring clownfish's motion and dispersion characteristics leading to assessing the potential signs of stress behaviors. We demonstrate the effectiveness of the proposed technique for quantifying variation in clownfish activity levels between recordings taken in the morning and afternoon at different stocking densities. It provides practical baseline support for online predicting and monitoring feeding behavior in ornamental fish aquaculture.
Literature Review of the Pioneering Approaches in Cloud-based Search Engines Powered by LETOR Techniques
Authors: Gizem Gezici
Date: 2021-12-29 00:00:00
Tasks: Retrieval

Search engines play an essential role in our daily lives. Nonetheless, they are also very crucial in enterprise domain to access documents from various information sources. Since traditional search systems index the documents mainly by looking at the frequency of the occurring words in these documents, they are barely able to support natural language search, but rather keyword search. It seems that keyword based search will not be sufficient for enterprise data which is growing extremely fast. Thus, enterprise search becomes increasingly critical in corporate domain. In this report, we present an overview of the state-of-the-art technologies in literature for three main purposes: i) to increase the retrieval performance of a search engine, ii) to deploy a search platform to a cloud environment, and iii) to select the best terms in expanding queries for achieving even a higher retrieval performance as well as to provide good query suggestions to its users for a better user experience.
Multivariate Trend Filtering for Lattice Data
Authors: Veeranjaneyulu Sadhanala, Yu-Xiang Wang, Addison J. Hu, Ryan J. Tibshirani
Date: 2021-12-29 00:00:00

We study a multivariate version of trend filtering, called Kronecker trend filtering or KTF, for the case in which the design points form a lattice in $d$ dimensions. KTF is a natural extension of univariate trend filtering (Steidl et al., 2006; Kim et al., 2009; Tibshirani, 2014), and is defined by minimizing a penalized least squares problem whose penalty term sums the absolute (higher-order) differences of the parameter to be estimated along each of the coordinate directions. The corresponding penalty operator can be written in terms of Kronecker products of univariate trend filtering penalty operators, hence the name Kronecker trend filtering. Equivalently, one can view KTF in terms of an $\ell_1$-penalized basis regression problem where the basis functions are tensor products of falling factorial functions, a piecewise polynomial (discrete spline) basis that underlies univariate trend filtering. This paper is a unification and extension of the results in Sadhanala et al. (2016, 2017). We develop a complete set of theoretical results that describe the behavior of $k^{\mathrm{th}}$ order Kronecker trend filtering in $d$ dimensions, for every $k \geq 0$ and $d \geq 1$. This reveals a number of interesting phenomena, including the dominance of KTF over linear smoothers in estimating heterogeneously smooth functions, and a phase transition at $d=2(k+1)$, a boundary past which (on the high dimension-to-smoothness side) linear smoothers fail to be consistent entirely. We also leverage recent results on discrete splines from Tibshirani (2020), in particular, discrete spline interpolation results that enable us to extend the KTF estimate to any off-lattice location in constant-time (independent of the size of the lattice $n$).
Universal Transformer Hawkes Process with Adaptive Recursive Iteration
Authors: Lu-ning Zhang, Jian-wei Liu, Zhi-yan Song, Xin Zuo
Date: 2021-12-29 00:00:00

Asynchronous events sequences are widely distributed in the natural world and human activities, such as earthquakes records, users activities in social media and so on. How to distill the information from these seemingly disorganized data is a persistent topic that researchers focus on. The one of the most useful model is the point process model, and on the basis, the researchers obtain many noticeable results. Moreover, in recent years, point process models on the foundation of neural networks, especially recurrent neural networks (RNN) are proposed and compare with the traditional models, their performance are greatly improved. Enlighten by transformer model, which can learning sequence data efficiently without recurrent and convolutional structure, transformer Hawkes process is come out, and achieves state-of-the-art performance. However, there is some research proving that the re-introduction of recursive calculations in transformer can further improve transformers performance. Thus, we come out with a new kind of transformer Hawkes process model, universal transformer Hawkes process (UTHP), which contains both recursive mechanism and self-attention mechanism, and to improve the local perception ability of the model, we also introduce convolutional neural network (CNN) in the position-wise-feed-forward part. We conduct experiments on several datasets to validate the effectiveness of UTHP and explore the changes after the introduction of the recursive mechanism. These experiments on multiple datasets demonstrate that the performance of our proposed new model has a certain improvement compared with the previous state-of-the-art models.
Bayesian Neural Hawkes Process for Event Uncertainty Prediction
Authors: Manisha Dubey, Ragja Palakkadavath, P. K. Srijith
Date: 2021-12-29 00:00:00
Tasks: Decision Making, Point Processes, Prediction, Uncertainty Quantification

Event data consisting of time of occurrence of the events arises in several real-world applications. Recent works have introduced neural network based point processes for modeling event-times, and were shown to provide state-of-the-art performance in predicting event-times. However, neural point process models lack a good uncertainty quantification capability on predictions. A proper uncertainty quantification over event modeling will help in better decision making for many practical applications. Therefore, we propose a novel point process model, Bayesian Neural Hawkes process (BNHP) which leverages uncertainty modelling capability of Bayesian models and generalization capability of the neural networks to model event occurrence times. We augment the model with spatio-temporal modeling capability where it can consider uncertainty over predicted time and location of the events. Experiments on simulated and real-world datasets show that BNHP significantly improves prediction performance and uncertainty quantification for modelling events.
Adaptive Transit Design: Optimizing Fixed and Demand Responsive Multi-Modal Transportation via Continuous Approximation
Authors: Giovanni Calabro', Andrea Araldo, Simon Oh, Ravi Seshadri, Giuseppe Inturri, Moshe Ben-Akiva
Date: 2021-12-29 00:00:00

In most cities, transit consists solely of fixed-route transportation, whence the inherent limited Quality of Service for travellers in suburban areas and during off-peak periods. On the other hand, completely replacing fixed-route (FR) with demand-responsive (DR) transit would imply a huge operational cost. It is still unclear how to integrate DR transportation into current transit systems to take full advantage of it. We propose a Continuous Approximation model of a transit system that gets the best from fixed-route and DR transportation. Our model allows deciding whether to deploy a FR or a DR feeder, in each sub-region of an urban conurbation and each time of day, and to redesign the line frequencies and the stop spacing of the main trunk service. Since such a transit design can adapt to the spatial and temporal variation of the demand, we call it Adaptive Transit. Numerical results show that, with respect to conventional transit, Adaptive Transit significantly improves user-related cost, by drastically reducing access time to the main trunk service. Such benefits are particularly remarkable in the suburbs. Moreover, the generalized cost, including agency and user cost, is also reduced. These findings are also confirmed in scenarios with automated vehicles. Our model can assist in planning future-generation transit systems, able to improve urban mobility by appropriately combining fixed and DR transportation.
Shallow decision trees for explainable $k$-means clustering
Authors: Eduardo Laber, Lucas Murtinho, Felipe Oliveira
Date: 2021-12-29 00:00:00
Tasks: Clustering

A number of recent works have employed decision trees for the construction of explainable partitions that aim to minimize the $k$-means cost function. These works, however, largely ignore metrics related to the depths of the leaves in the resulting tree, which is perhaps surprising considering how the explainability of a decision tree depends on these depths. To fill this gap in the literature, we propose an efficient algorithm that takes into account these metrics. In experiments on 16 datasets, our algorithm yields better results than decision-tree clustering algorithms such as the ones presented in \cite{dasgupta2020explainable}, \cite{frost2020exkmc}, \cite{laber2021price} and \cite{DBLP:conf/icml/MakarychevS21}, typically achieving lower or equivalent costs with considerably shallower trees. We also show, through a simple adaptation of existing techniques, that the problem of building explainable partitions induced by binary trees for the $k$-means cost function does not admit an $(1+\epsilon)$-approximation in polynomial time unless $P=NP$, which justifies the quest for approximation algorithms and/or heuristics.
An additive graphical model for discrete data
Authors: Jun Tao, Bing Li, Lingzhou Xue
Date: 2021-12-29 00:00:00
Tasks: model, Relation

We introduce a nonparametric graphical model for discrete node variables based on additive conditional independence. Additive conditional independence is a three way statistical relation that shares similar properties with conditional independence by satisfying the semi-graphoid axioms. Based on this relation we build an additive graphical model for discrete variables that does not suffer from the restriction of a parametric model such as the Ising model. We develop an estimator of the new graphical model via the penalized estimation of the discrete version of the additive precision operator and establish the consistency of the estimator under the ultrahigh-dimensional setting. Along with these methodological developments, we also exploit the properties of discrete random variables to uncover a deeper relation between additive conditional independence and conditional independence than previously known. The new graphical model reduces to a conditional independence graphical model under certain sparsity conditions. We conduct simulation experiments and analysis of an HIV antiretroviral therapy data set to compare the new method with existing ones.
Active Learning of Quantum System Hamiltonians yields Query Advantage
Authors: Arkopal Dutt, Edwin Pednault, Chai Wah Wu, Sarah Sheldon, John Smolin, Lev Bishop, Isaac L. Chuang
Date: 2021-12-29 00:00:00
Tasks: Active Learning

Hamiltonian learning is an important procedure in quantum system identification, calibration, and successful operation of quantum computers. Through queries to the quantum system, this procedure seeks to obtain the parameters of a given Hamiltonian model and description of noise sources. Standard techniques for Hamiltonian learning require careful design of queries and $O(\epsilon^{-2})$ queries in achieving learning error $\epsilon$ due to the standard quantum limit. With the goal of efficiently and accurately estimating the Hamiltonian parameters within learning error $\epsilon$ through minimal queries, we introduce an active learner that is given an initial set of training examples and the ability to interactively query the quantum system to generate new training data. We formally specify and experimentally assess the performance of this Hamiltonian active learning (HAL) algorithm for learning the six parameters of a two-qubit cross-resonance Hamiltonian on four different superconducting IBM Quantum devices. Compared with standard techniques for the same problem and a specified learning error, HAL achieves up to a $99.8\%$ reduction in queries required, and a $99.1\%$ reduction over the comparable non-adaptive learning algorithm. Moreover, with access to prior information on a subset of Hamiltonian parameters and given the ability to select queries with linearly (or exponentially) longer system interaction times during learning, HAL can exceed the standard quantum limit and achieve Heisenberg (or super-Heisenberg) limited convergence rates during learning.
Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective
Authors: Lanning Wei, Huan Zhao, Zhiqiang He
Date: 2021-12-29 00:00:00
Tasks: feature selection, Neural Architecture Search

In recent years, Graph Neural Networks (GNNs) have shown superior performance on diverse real-world applications. To improve the model capacity, besides designing aggregation operations, GNN topology design is also very important. In general, there are two mainstream GNN topology design manners. The first one is to stack aggregation operations to obtain the higher-level features but easily got performance drop as the network goes deeper. Secondly, the multiple aggregation operations are utilized in each layer which provides adequate and independent feature extraction stage on local neighbors while are costly to obtain the higher-level information. To enjoy the benefits while alleviating the corresponding deficiencies of these two manners, we learn to design the topology of GNNs in a novel feature fusion perspective which is dubbed F$^2$GNN. To be specific, we provide a feature fusion perspective in designing GNN topology and propose a novel framework to unify the existing topology designs with feature selection and fusion strategies. Then we develop a neural architecture search method on top of the unified framework which contains a set of selection and fusion operations in the search space and an improved differentiable search algorithm. The performance gains on eight real-world datasets demonstrate the effectiveness of F$^2$GNN. We further conduct experiments to show that F$^2$GNN can improve the model capacity while alleviating the deficiencies of existing GNN topology design manners, especially alleviating the over-smoothing problem, by utilizing different levels of features adaptively.
Feature-context driven Federated Meta-Learning for Rare Disease Prediction
Authors: Bingyang Chen, Tao Chen, Xingjie Zeng, Weishan Zhang, Qinghua Lu, Zhaoxiang Hou, Jiehan Zhou, Sumi Helal
Date: 2021-12-29 00:00:00
Tasks: Disease Prediction, Federated Learning, Meta-Learning, Prediction

Millions of patients suffer from rare diseases around the world. However, the samples of rare diseases are much smaller than those of common diseases. In addition, due to the sensitivity of medical data, hospitals are usually reluctant to share patient information for data fusion citing privacy concerns. These challenges make it difficult for traditional AI models to extract rare disease features for the purpose of disease prediction. In this paper, we overcome this limitation by proposing a novel approach for rare disease prediction based on federated meta-learning. To improve the prediction accuracy of rare diseases, we design an attention-based meta-learning (ATML) approach which dynamically adjusts the attention to different tasks according to the measured training effect of base learners. Additionally, a dynamic-weight based fusion strategy is proposed to further improve the accuracy of federated learning, which dynamically selects clients based on the accuracy of each local model. Experiments show that with as few as five shots, our approach out-performs the original federated meta-learning algorithm in accuracy and speed. Compared with each hospital's local model, the proposed model's average prediction accuracy increased by 13.28%.
Federated Learning for Cross-block Oil-water Layer Identification
Authors: Bingyang Chena, Xingjie Zenga, Weishan Zhang
Date: 2021-12-29 00:00:00
Tasks: Federated Learning

Cross-block oil-water layer(OWL) identification is essential for petroleum development. Traditional methods are greatly affected by subjective factors due to depending mainly on the human experience. AI-based methods have promoted the development of OWL identification. However, because of the significant geological differences across blocks and the severe long-tailed distribution(class imbalanced), the identification effects of existing artificial intelligence(AI) models are limited. In this paper, we address this limitation by proposing a dynamic fusion-based federated learning(FL) for OWL identification. To overcome geological differences, we propose a dynamic weighted strategy to fuse models and train a general OWL identification model. In addition, an F1 score-based re-weighting scheme is designed and a novel loss function is derived theoretically to solve the data long-tailed problem. Further, a geological knowledge-based mask-attention mechanism is proposed to enhance model feature extraction. To our best knowledge, this is the first work to identify OWL using FL. We evaluate the proposed approach with an actual well logging dataset from the oil field and a public 3W dataset. Experimental results demonstrate that our approach significantly out-performs other AI methods.
APRIL: Finding the Achilles' Heel on Privacy for Vision Transformers
Authors: Jiahao Lu, Xi Sheryl Zhang, Tianli Zhao, Xiangyu He, Jian Cheng
Date: 2021-12-28 00:00:00
Proceeding: CVPR 2022 1
Tasks: Federated Learning

Federated learning frameworks typically require collaborators to share their local gradient updates of a common model instead of sharing training data to preserve privacy. However, prior works on Gradient Leakage Attacks showed that private training data can be revealed from gradients. So far almost all relevant works base their attacks on fully-connected or convolutional neural networks. Given the recent overwhelmingly rising trend of adapting Transformers to solve multifarious vision tasks, it is highly valuable to investigate the privacy risk of vision transformers. In this paper, we analyse the gradient leakage risk of self-attention based mechanism in both theoretical and practical manners. Particularly, we propose APRIL - Attention PRIvacy Leakage, which poses a strong threat to self-attention inspired models such as ViT. Showing how vision Transformers are at the risk of privacy leakage via gradients, we urge the significance of designing privacy-safer Transformer models and defending schemes.