bluuebunny/crossref_metadata_2025 huggingface dataset:
SCAN: A Spatial Context Attentive Network for Joint Multi-Agent Intent Prediction
Authors: Sekhon Jasmine , Fleming Cody
Published: September 2022

Safe navigation of autonomous agents in human centric environments requires the ability to understand and predict motion of neighboring pedestrians. However, predicting pedestrian intent is a complex problem. Pedestrian motion is governed by complex social navigation norms, is dependent on neighbors' trajectories and is multimodal in nature. In this work, we propose SCAN, a Spatial Context Attentive Network that can jointly predict socially-acceptable multiple future trajectories for all pedestrians in a scene. SCAN encodes the influence of spatially close neighbors using a novel spatial attention mechanism in a manner that relies on fewer assumptions, is parameter efficient, and is more interpretable compared to state-of-the-art spatial attention approaches. Through experiments on several datasets we demonstrate that our approach can also quantitatively outperform state of the art trajectory prediction methods in terms of accuracy of predicted intent.
A Deep Reinforcement Learning Approach to First-Order Logic Theorem Proving
Authors: Crouse Maxwell , Abdelaziz Ibrahim , Makni Bassem , Whitehead Spencer , Cornelio Cristina , Kapanipathi Pavan , Srinivas Kavitha , Thost Veronika , Witbrock Michael , Fokoue Achille
Published: September 2022

Automated theorem provers have traditionally relied on manually tuned heuristics to guide how they perform proof search. Deep reinforcement learning has been proposed as a way to obviate the need for such heuristics, however, its deployment in automated theorem proving remains a challenge. In this paper we introduce TRAIL, a system that applies deep reinforcement learning to saturation-based theorem proving. TRAIL leverages (a) a novel neural representation of the state of a theorem prover and (b) a novel characterization of the inference selection process in terms of an attention-based action policy. We show through systematic analysis that these mechanisms allow TRAIL to significantly outperform previous reinforcement-learning-based theorem provers on two benchmark datasets for first-order logic automated theorem proving (proving around 15% more theorems).
Knowledge-Base Degrees of Inconsistency: Complexity and Counting
Authors: Fichte Johannes K. , Hecher Markus , Meier Arne
Published: September 2022

Description logics (DLs) are knowledge representation languages that are used in the field of artificial intelligence (AI).
A common technique is to query DL knowledge bases, e.g., by Boolean Datalog queries, and ask for entailment.
But real world knowledge-bases are often obtained by combining data from various sources.
This, inherently, might result in certain inconsistencies (with respect to a given query) and requires to estimate a degree of inconsistency before using a knowledge-base.
In this paper, we provide a complexity analysis of fixed-domain non-entailment (NE) on Datalog programs for well-established families of knowledge bases (KBs).
We exhibit a detailed complexity map for the decision cases, counting and projected counting, which may serve as a quantitative measure for inconsistency of a KB with respect to a query.
Our results show that NE is natural for the second, third, and fourth level of the polynomial (counting) hierarchy depending on the type of the studied query (stratified, normal, disjunctive) and one level higher for the projected versions.
Further, we show fixed-parameter tractability by bounding the treewidth, provide a constructive algorithm, and show its theoretical limitation in terms of conditional lower bounds.
REM-Net: Recursive Erasure Memory Network for Commonsense Evidence Refinement
Authors: Huang Yinya , Fang Meng , Zhan Xunlin , Cao Qingxing , Liang Xiaodan
Published: September 2022

When answering a question, people often draw upon their rich world knowledge in addition to the particular context. While recent works retrieve supporting facts/evidence from commonsense knowledge bases to supply additional information to each question, there is still ample opportunity to advance it on the quality of the evidence. It is crucial since the quality of the evidence is the key to answering common- sense questions, and even determines the upper bound on the QA systems’ performance. In this paper, we propose a recursive erasure memory network (REM-Net) to cope with the quality improvement of evidence. To address this, REM-Net is equipped with a module to refine the evidence by recursively erasing the low-quality evidence that does not explain the question answering. Besides, instead of retrieving evidence from existing knowledge bases, REM-Net leverages a pre-trained generative model to generate candidate evidence customized for the question. We conduct experiments on two commonsense question answering datasets, WIQA and CosmosQA. The results demonstrate the performance of REM- Net and show that the refined evidence is explainable.
Interpreting Neural Networks as Quantitative Argumentation Frameworks
Authors: Potyka Nico
Published: September 2022

We show that an interesting class of feed-forward neural networks can be understood as quantitative argumentation frameworks.
This connection creates a bridge between research in Formal Argumentation and Machine Learning.
We generalize the semantics of feed-forward neural networks to acyclic graphs and study the resulting
computational and semantical properties in argumentation
graphs. As it turns out, the semantics gives stronger guarantees than existing semantics
that have been tailor-made for the argumentation setting.
From a machine-learning perspective, the connection does not seem immediately helpful. While it gives intuitive meaning
to some feed-forward-neural networks, they remain difficult to understand due to their size and density.
However, the connection seems helpful for combining background knowledge in form of
sparse argumentation networks with dense neural networks that have been trained for complementary purposes
and for learning the parameters of quantitative argumentation frameworks in an end-to-end fashion from data.
Stratified Negation in Datalog with Metric Temporal Operators
Authors: Tena Cucala David J , Wałęga Przemysław A , Cuenca Grau Bernardo , Kostylev Egor
Published: September 2022

We extend DatalogMTL—Datalog with operators from metric temporal logic—by adding stratified negation as failure. The new language provides additional expressive power for representing and reasoning about temporal data and knowledge in a wide range of applications. We consider models over the rational timeline, study their properties, and establish the computational complexity of reasoning. We show that, as in negation-free DatalogMTL, fact entailment in our language is PSPACE-complete in data and EXPSPACE-complete in combined complexity. Thus, the extension with stratified negation does not lead to higher complexity.
On the Tractability of SHAP Explanations
Authors: Van den Broeck Guy , Lykov Anton , Schleich Maximilian , Suciu Dan
Published: September 2022

SHAP explanations are a popular feature-attribution mechanism for explainable AI. They use game-theoretic notions to measure the influence of individual features on the prediction of a machine learning model. Despite a lot of recent interest from both academia and industry, it is not known whether SHAP explanations of common machine learning models can be computed efficiently. In this paper, we establish the complexity of computing the SHAP explanation in three important settings. First, we consider fully-factorized data distributions, and show that the complexity of computing the SHAP explanation is the same as the complexity of computing the expected value of the model. This fully-factorized setting is often used to simplify the SHAP computation, yet our results show that the computation can be intractable for commonly used models such as logistic regression. Going beyond fully-factorized distributions, we show that computing SHAP explanations is already intractable for a very simple setting: computing SHAP explanations of trivial classifiers over naive Bayes distributions. Finally, we show that even computing SHAP over the empirical distribution is #P-hard.
Semi-supervised Sequence Classification through Change Point Detection
Authors: Ahad Nauman , Davenport Mark A.
Published: September 2022

Sequential sensor data is generated in a wide variety of real-world applications. A fundamental machine learning challenge involves learning effective classifiers for such sequential data. While deep learning has led to impressive performance gains in recent years within domains such as speech, this has relied on the availability of large datasets of sequences with high-quality labels. In many applications, however, the associated class labels are often extremely limited, with precise labelling/segmentation being too expensive to perform in a high volume. However, large amounts of unlabelled data may still be available. In this paper we propose a novel framework for semi-supervised learning in such contexts. In an unsupervised manner, change-point detection methods can be used to identify instances where classes change within in a sequence. We show that change points provide examples of similar/dissimilar pairs of sequences which, when coupled with class labels, can be used in a semi-supervised classification setting. Pairs from labels and change points are used by a neural network to learn improved representations for classification. We provide extensive synthetic simulations and show that the learned representations are better than those learned through an autoencoder and obtain improved results on simulations and human activity recognition datasets.
Does Explainable Artificial Intelligence Improve Human Decision-Making?
Authors: Alufaisan Yasmeen , Marusich Laura R. , Bakdash Jonathan Z. , Zhou Yan , Kantarcioglu Murat
Published: September 2022

Explainable AI provides insights to users into the why for
model predictions, offering potential for users to better understand
and trust a model, and to recognize and correct AI
predictions that are incorrect. Prior research on human and
explainable AI interactions has focused on measures such as
interpretability, trust, and usability of the explanation. There
are mixed findings whether explainable AI can improve actual
human decision-making and the ability to identify the
problems with the underlying model. Using real datasets, we
compare objective human decision accuracy without AI (control),
with an AI prediction (no explanation), and AI prediction
with explanation. We find providing any kind of AI prediction
tends to improve user decision accuracy, but no conclusive
evidence that explainable AI has a meaningful impact.
Moreover, we observed the strongest predictor for human decision
accuracy was AI accuracy and that users were somewhat
able to detect when the AI was correct vs. incorrect, but
this was not significantly affected by including an explanation.
Our results indicate that, at least in some situations, the
why information provided in explainable AI may not enhance
user decision-making, and further research may be needed to
understand how to integrate explainable AI into real systems.
TabNet: Attentive Interpretable Tabular Learning
Authors: Arik Sercan Ö. , Pfister Tomas
Published: September 2022

We propose a novel high-performance and interpretable canonical deep tabular data learning architecture, TabNet. TabNet uses sequential attention to choose which features to reason from at each decision step, enabling interpretability and more efficient learning as the learning capacity is used for the most salient features. We demonstrate that TabNet outperforms other variants on a wide range of non-performance-saturated tabular datasets and yields interpretable feature attributions plus insights into its global behavior. Finally, we demonstrate self-supervised learning for tabular data, significantly improving performance when unlabeled data is abundant.
Deterministic Mini-batch Sequencing for Training Deep Neural Networks
Authors: Banerjee Subhankar , Chakraborty Shayok
Published: September 2022

Recent advancements in the field of deep learning have dramatically improved the performance of machine learning models in a variety of applications, including computer vision, text mining, speech processing and fraud detection among others. Mini-batch gradient descent is the standard algorithm to train deep models, where mini-batches of a fixed size are sampled randomly from the training data and passed through the network sequentially. In this paper, we present a novel algorithm to generate a deterministic sequence of mini-batches to train a deep neural network (rather than a random sequence). Our rationale is to select a mini-batch by minimizing the Maximum Mean Discrepancy (MMD) between the already selected mini-batches and the unselected training samples. We pose the mini-batch selection as a constrained optimization problem and derive a linear programming relaxation to determine the sequence of mini-batches. To the best of our knowledge, this is the first research effort that uses the MMD criterion to determine a sequence of mini-batches to train a deep neural network. The proposed mini-batch sequencing strategy is deterministic and independent of the underlying network architecture and prediction task. Our extensive empirical analyses on three challenging datasets corroborate the merit of our framework over competing baselines. We further study the performance of our framework on two other applications besides classification (regression and semantic segmentation) to validate its generalizability.
ExGAN: Adversarial Generation of Extreme Samples
Authors: Bhatia Siddharth , Jain Arjit , Hooi Bryan
Published: September 2022

Mitigating the risk arising from extreme events is a fundamental goal with many applications, such as the modelling of natural disasters, financial crashes, epidemics, and many others. To manage this risk, a vital step is to be able to understand or generate a wide range of extreme scenarios. Existing approaches based on Generative Adversarial Networks (GANs) excel at generating realistic samples, but seek to generate typical samples, rather than extreme samples. Hence, in this work, we propose ExGAN, a GAN-based approach to generate realistic and extreme samples. To model the extremes of the training distribution in a principled way, our work draws from Extreme Value Theory (EVT), a probabilistic approach for modelling the extreme tails of distributions. For practical utility, our framework allows the user to specify both the desired extremeness measure, as well as the desired extremeness probability they wish to sample at. Experiments on real US Precipitation data show that our method generates realistic samples, based on visual inspection and quantitative measures, in an efficient manner. Moreover, generating increasingly extreme examples using ExGAN can be done in constant time (with respect to the extremeness probability τ), as opposed to the O(1/τ) time required by the baseline approach.
Fairness, Semi-Supervised Learning, and More: A General Framework for Clustering with Stochastic Pairwise Constraints
Authors: Brubach Brian , Chakrabarti Darshan , Dickerson John P. , Srinivasan Aravind , Tsepenekas Leonidas
Published: September 2022

Metric clustering is fundamental in areas ranging from Combinatorial Optimization and Data Mining, to Machine Learning and Operations Research. However, in a variety of situations we may have additional requirements or knowledge, distinct from the underlying metric, regarding which pairs of points should be clustered together. To capture and analyze such scenarios, we introduce a novel family of stochastic pairwise constraints, which we incorporate into several essential clustering objectives (radius/median/means). Moreover, we demonstrate that these constraints can succinctly model an intriguing collection of applications, including among others Individual Fairness in clustering and Must-link constraints in semi-supervised learning. Our main result consists of a general framework that yields approximation algorithms with provable guarantees for important clustering objectives, while at the same time producing solutions that respect the stochastic pairwise constraints. Furthermore, for certain objectives we devise improved results in the case of Must-link constraints, which are also the best possible from a theoretical perspective. Finally, we present experimental evidence that validates the effectiveness of our algorithms.
Counterfactual Explanations for Oblique Decision Trees:Exact, Efficient Algorithms
Authors: Carreira-Perpiñán Miguel  Á. , Hada Suryabhan Singh
Published: September 2022

We consider counterfactual explanations, the problem of minimally adjusting features in a source input instance so that it is classified as a target class under a given classifier. This has become a topic of recent interest as a way to query a trained model and suggest possible actions to overturn its decision. Mathematically, the problem is formally equivalent to that of finding adversarial examples, which also has attracted significant attention recently. Most work on either counterfactual explanations or adversarial examples has focused on differentiable classifiers, such as neural nets. We focus on classification trees, both axis-aligned and oblique (having hyperplane splits). Although here the counterfactual optimization problem is nonconvex and nondifferentiable, we show that an exact solution can be computed very efficiently, even with high-dimensional feature vectors and with both continuous and categorical features, and demonstrate it in different datasets and settings. The results are particularly relevant for finance, medicine or legal applications, where interpretability and counterfactual explanations are particularly important.
Automated Clustering of High-dimensional Data with a Feature Weighted Mean Shift Algorithm
Authors: Chakraborty Saptarshi , Paul Debolina , Das Swagatam
Published: September 2022

Mean shift is a simple interactive procedure that gradually shifts data points towards the mode which denotes the highest density of data points in the region. Mean shift algorithms have been effectively used for data denoising, mode seeking, and finding the number of clusters in a dataset in an automated fashion. However, the merits of mean shift quickly fade away as the data dimensions increase and only a handful of features contain useful information about the cluster structure of the data. We propose a simple yet elegant feature-weighted variant of mean shift to efficiently learn the feature importance and thus, extending the merits of mean shift to high-dimensional data. The resulting algorithm not only outperforms the conventional mean shift clustering procedure but also preserves its computational simplicity. In addition, the proposed method comes with rigorous theoretical convergence guarantees and a convergence rate of at least a cubic order. The efficacy of our proposal is thoroughly assessed through experimental comparison against baseline and state-of-the-art clustering methods on synthetic as well as real-world datasets.
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks
Authors: Chang Xiangyu , Li Yingcong , Oymak Samet , Thrampoulidis Christos
Published: September 2022

Deep networks are typically trained with many more parameters than the size of the training dataset. Recent empirical evidence indicates that the practice of overparameterization not only benefits training large models, but also assists – perhaps counterintuitively – building lightweight models. Specifically, it suggests that overparameterization benefits model pruning / sparsification. This paper sheds light on these empirical findings by theoretically characterizing the high-dimensional asymptotics of model pruning in the overparameterized regime. The theory presented addresses the following core question: ``should one train a small model from the beginning, or first train a large model and then prune?''. We analytically identify regimes in which, even if the location of the most informative features is known, we are better off fitting a large model and then pruning rather than simply training with the known informative features. This leads to a new double descent in the training of sparse models: growing the original model, while preserving the target sparsity, improves the test accuracy as one moves beyond the overparameterization threshold. Our analysis further reveals the benefit of retraining by relating it to feature correlations. We find that the above phenomena are already present in linear and random-features models. Our technical approach advances the toolset of high-dimensional analysis and precisely characterizes the asymptotic distribution of over-parameterized least-squares. The intuition gained by analytically studying simpler models is numerically verified on neural networks.
Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
Authors: Che Tong , Liu Xiaofeng , Li Site , Ge Yubin , Zhang Ruixiang , Xiong Caiming , Bengio Yoshua
Published: September 2022

AI Safety is a major concern in many deep learning applications such as autonomous driving. Given a trained deep learning model, an important natural problem is how to reliably verify the model's prediction. In this paper, we propose a novel framework --- deep verifier networks (DVN) to detect unreliable inputs or predictions of deep discriminative models, using separately trained deep generative models. Our proposed model is based on conditional variational auto-encoders with disentanglement constraints to separate the label information from the latent representation. We give both intuitive and theoretical justifications for the model. Our verifier network is trained independently with the prediction model, which eliminates the need of retraining the verifier network for a new model. We test the verifier network on both out-of-distribution detection and adversarial example detection problems, as well as anomaly detection problems in structured prediction tasks such as image caption generation. We achieve state-of-the-art results in all of these problems.
Continuous-Time Attention for Sequential Learning
Authors: Chien Jen-Tzung , Chen Yi-Hsiang
Published: September 2022

Attention mechanism is crucial for sequential learning where a wide range of applications have been successfully developed. This mechanism is basically trained to spotlight on the region of interest in hidden states of sequence data. Most of the attention methods compute the attention score through relating between a query and a sequence where the discrete-time state trajectory is represented. Such a discrete-time attention could not directly attend the continuous-time trajectory which is represented via neural differential equation (NDE) combined with recurrent neural network. This paper presents a new continuous-time attention method for sequential learning which is tightly integrated with NDE to construct an attentive continuous-time state machine. The continuous-time attention is performed at all times over the hidden states for different kinds of irregular time signals. The missing information in sequence data due to sampling loss, especially in presence of long sequence, can be seamlessly compensated and attended in learning representation. The experiments on irregular sequence samples from human activities, dialogue sentences and medical features show the merits of the proposed continuous-time attention for activity recognition, sentiment classification and mortality prediction, respectively.
SHOT-VAE: Semi-supervised Deep Generative Models With Label-aware ELBO Approximations
Authors: Feng Hao-Zhe , Kong Kezhi , Chen Minghao , Zhang Tianye , Zhu Minfeng , Chen Wei
Published: September 2022

Semi-supervised variational autoencoders (VAEs) have obtained strong results, but have also encountered the challenge that good ELBO values do not always imply accurate inference results.In this paper, we investigate and propose two causes of this problem: (1) The ELBO objective cannot utilize the label information directly. (2) A bottleneck value exists, and continuing to optimize ELBO after this value will not improve inference accuracy. On the basis of the experiment results, we propose SHOT-VAE to address these problems without introducing additional prior knowledge. The SHOT-VAE offers two contributions: (1) A new ELBO approximation named smooth-ELBO that integrates the label predictive loss into ELBO. (2) An approximation based on optimal interpolation that breaks the ELBO value bottleneck by reducing the margin between ELBO and the data likelihood. The SHOT-VAE achieves good performance with 25.30% error rate on CIFAR-100 with 10k labels and reduces the error rate to 6.11% on CIFAR-10 with 4k labels.
Uncertainty-Aware Multi-View Representation Learning
Authors: Geng Yu , Han Zongbo , Zhang Changqing , Hu Qinghua
Published: September 2022

Learning from different data views by exploring the underlying complementary information among them can endow the representation with stronger expressive ability. However, high-dimensional features tend to contain noise, and furthermore, quality of data usually varies for different samples (even for different views), i.e., one view may be informative for one sample but not the case for another. Therefore, it is quite challenging to integrate multi-view noisy data under unsupervised setting. Traditional multi-view methods either simply treat each view with equal importance or tune the weights of different views to fixed values, which are insufficient to capture the dynamic noise in multi-view data. In this work, we devise a novel unsupervised multi-view learning approach, termed as Dynamic Uncertainty-Aware Networks (DUA-Nets). Guided by the uncertainty of data estimated from the generation perspective, intrinsic information from multiple views is integrated to obtain noise-free representations. Under the help of uncertainty estimation, DUA-Nets weigh each view of individual sample according to data quality so that the high-quality samples (or views) can be fully exploited while the effects from the noisy samples (or views) will be alleviated. Our model achieves superior performance in extensive experiments and shows the robustness to noisy data.
The Importance of Modeling Data Missingness in Algorithmic Fairness: A Causal Perspective
Authors: Goel Naman , Amayuelas Alfonso , Deshpande Amit , Sharma Amit
Published: September 2022

Training datasets for machine learning often have some form of missingness. For example, to learn a model for deciding whom to give a loan, the available training data includes individuals who were given a loan in the past, but not those who were not. This missingness, if ignored, nullifies any fairness guarantee of the training procedure when the model is deployed. Using causal graphs, we characterize the missingness mechanisms in different real-world scenarios. We show conditions under which various distributions, used in popular fairness algorithms, can or can not be recovered from the training data. Our theoretical results imply that many of these algorithms can not guarantee fairness in practice. Modeling missingness also helps to identify correct design principles for fair algorithms. For example, in multi-stage settings where decisions are made in multiple screening rounds, we use our framework to derive the minimal distributions required to design a fair algorithm. Our proposed algorithm also decentralizes the decision-making process and still achieves similar performance to the optimal algorithm that requires centralization and non-recoverable distributions.
Towards Reusable Network Components by Learning Compatible Representations
Authors: Gygli Michael , Uijlings Jasper , Ferrari Vittorio
Published: September 2022

This paper proposes to make a first step towards compatible and hence reusable network components. Rather than training networks for different tasks independently, we adapt the training process to produce network components that are compatible across tasks. In particular, we split a network into two components, a features extractor and a target task head, and propose various approaches to accomplish compatibility between them. We systematically analyse these approaches on the task of image classification on standard datasets.
We demonstrate that we can produce components which are directly compatible without any fine-tuning or compromising accuracy on the original tasks. Afterwards, we demonstrate the use of compatible components on three applications: Unsupervised domain adaptation, transferring classifiers across feature extractors with different architectures, and increasing the computational efficiency of transfer learning.
ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization
Authors: Huang Xunpeng , Xu Runxin , Zhou Hao , Wang Zhe , Liu Zhengyang , Li Lei
Published: September 2022

Stochastic gradient descent (SGD) is a widely used method for its outstanding generalization ability and simplicity. Adaptive gradient methods have been proposed to further accelerate the optimization process. In this paper, we revisit existing adaptive gradient optimization methods with a new interpretation. Such new perspective leads to a refreshed understanding of the roles of second moments in stochastic optimization. Based on this, we propose Angle-Calibration Moment method (ACMo), a novel stochastic optimization method. It enjoys the benefits of second moments with only first moment updates. Theoretical analysis shows that ACMo is able to achieve the same convergence rate as mainstream adaptive methods. Experiments on a variety of CV and NLP tasks demonstrate that ACMo has a comparable convergence to state-of-the-art Adam-type optimizers, and even a better generalization performance in most cases. The code is available at https://github.com/Xunpeng746/ACMo.
IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks
Authors: Jeon Insu , Lee Wonkwang , Pyeon Myeongjang , Kim Gunhee
Published: September 2022

We propose a new GAN-based unsupervised model for disentangled representation learning. The new model is discovered in an attempt to utilize the Information Bottleneck (IB) framework to the optimization of GAN, thereby named IB-GAN.
The architecture of IB-GAN is partially similar to that of InfoGAN but has a critical difference; an intermediate layer of the generator is leveraged to constrain the mutual information between the input and the generated output. The intermediate stochastic layer can serve as a learnable latent distribution that is trained with the generator jointly in an end-to-end fashion. As a result, the generator of IB-GAN can harness the latent space in a disentangled and interpretable manner. With the experiments on dSprites and Color-dSprites dataset, we demonstrate that IB-GAN achieves competitive disentanglement scores to those of state-of-the-art β-VAEs and outperforms InfoGAN. Moreover, the visual quality and the diversity of samples generated by IB-GAN are often better than those by β-VAEs and Info-GAN in terms of FID score on CelebA and 3D Chairs dataset.
LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification
Authors: Jiang Ting , Wang Deqing , Sun Leilei , Yang Huayi , Zhao Zhengyang , Zhuang Fuzhen
Published: September 2022

Extreme multi-label text classification(XMC) is a task for finding the most relevant labels from a large label set. Nowadays deep learning-based methods have shown significant success in XMC. However, the existing methods (e.g., AttentionXML and X-Transformer etc) still suffer from 1) combining several models to train and predict for one dataset, and 2) sampling negative labels statically during the process of training label ranking model, which will harm the performance and accuracy of model. To address the above problems, we propose LightXML, which adopts end-to-end training and dynamical negative labels sampling. In LightXML, we use GAN like networks to recall and rank labels. The label recalling part will generate negative and positive labels, and the label ranking part will distinguish positive labels from these labels. Based on these networks, negative labels are sampled dynamically during label ranking part training. With feeding both label recalling and ranking parts with the same text representation, LightXML can reach high performance. Extensive experiments show that LightXML outperforms state-of-the-art methods in five extreme multi-label datasets with much smaller model size and lower computational complexity. In particular, on the Amazon dataset with 670K labels, LightXML can reduce the model size up to 72% compared to AttentionXML. Our code is available at http://github.com/kongds/LightXML.
Deep Probabilistic Canonical Correlation Analysis
Authors: Karami Mahdi , Schuurmans Dale
Published: September 2022

We propose a deep generative framework for multi-view learning
based on a probabilistic interpretation of canonical correlation
analysis (CCA). The model combines a linear multi-view
layer in the latent space with deep generative networks as
observation models, to decompose the variability in multiple
views into a shared latent representation that describes the
common underlying sources of variation and a set of viewspecific
components. To approximate the posterior distribution
of the latent multi-view layer, an efficient variational inference
procedure is developed based on the solution of probabilistic
CCA. The model is then generalized to an arbitrary number of
views. An empirical analysis confirms that the proposed deep
multi-view model can discover subtle relationships between
multiple views and recover rich representations.
Bayesian Dynamic Mode Decomposition with Variational Matrix Factorization
Authors: Kawashima Takahiro , Shouno Hayaru , Hino Hideitsu
Published: September 2022

Dynamic mode decomposition (DMD) and its extensions are data-driven methods that have substantially contributed to our understanding of dynamical systems. However, because DMD and most of its extensions are deterministic, it is difficult to treat probabilistic representations of parameters and predictions. In this work, we propose a novel formulation of a Bayesian DMD model. Our Bayesian DMD model is consistent with the procedure of standard DMD, which is to first determine the subspace of observations, and then compute the modes on that subspace. Variational matrix factorization makes it possible to realize a fully-Bayesian scheme of DMD. Moreover, we derive a Bayesian DMD model for incomplete data, which demonstrates the advantage of probabilistic modeling. Finally, both of nonlinear simulated and real-world datasets are used to illustrate the potential of the proposed method.
A Flexible Framework for Communication-Efficient Machine Learning
Authors: Khirirat Sarit , Magnússon Sindri , Aytekin Arda , Johansson Mikael
Published: September 2022

With the increasing scale of machine learning tasks, it has become essential to reduce the communication between computing nodes. Early work on gradient compression focused on the bottleneck between CPUs and GPUs, but communication-efficiency is now needed in a variety of different system architectures, from high-performance clusters to energy-constrained IoT devices. In the current practice, compression levels are typically chosen before training and settings that work well for one task may be vastly suboptimal for another dataset on another architecture. In this paper, we propose a flexible framework which adapts the compression level to the true gradient at each iteration, maximizing the improvement in the objective function that is achieved per communicated bit. Our framework is easy to adapt from one technology to the next by modeling how the communication cost depends on the compression level for the specific technology. Theoretical results and practical experiments indicate that the automatic tuning strategies significantly increase communication efficiency on several state-of-the-art compression schemes.
Understanding Catastrophic Overfitting in Single-step Adversarial Training
Authors: Kim Hoki , Lee Woojin , Lee Jaewook
Published: September 2022

Although fast adversarial training has demonstrated both robustness and efficiency, the problem of "catastrophic overfitting" has been observed. This is a phenomenon in which, during single-step adversarial training, the robust accuracy against projected gradient descent (PGD) suddenly decreases to 0% after a few epochs, whereas the robust accuracy against fast gradient sign method (FGSM) increases to 100%. In this paper, we demonstrate that catastrophic overfitting is very closely related to the characteristic of single-step adversarial training which uses only adversarial examples with the maximum perturbation, and not all adversarial examples in the adversarial direction, which leads to decision boundary distortion and a highly curved loss surface. Based on this observation, we propose a simple method that not only prevents catastrophic overfitting, but also overrides the belief that it is difficult to prevent multi-step adversarial attacks with single-step adversarial training.
Counterfactual Fairness with Disentangled Causal Effect Variational Autoencoder
Authors: Kim Hyemi , Shin Seungjae , Jang JoonHo , Song Kyungwoo , Joo Weonyoung , Kang Wanmo , Moon Il-Chul
Published: September 2022

The problem of fair classification can be mollified if we develop a method to remove the embedded sensitive information from the classification features. This line of separating the sensitive information is developed through the causal inference, and the causal inference enables the counterfactual generations to contrast the what-if case of the opposite sensitive attribute. Along with this separation with the causality, a frequent assumption in the deep latent causal model defines a single latent variable to absorb the entire exogenous uncertainty of the causal graph. However, we claim that such structure cannot distinguish the 1) information caused by the intervention (i.e., sensitive variable) and 2) information correlated with the intervention from the data. Therefore, this paper proposes Disentangled Causal Effect Variational Autoencoder (DCEVAE) to resolve this limitation by disentangling the exogenous uncertainty into two latent variables: either 1) independent to interventions or 2) correlated to interventions without causality. Particularly, our disentangling approach preserves the latent variable correlated to interventions in generating counterfactual examples. We show that our method estimates the total effect and the counterfactual effect without a complete causal graph. By adding a fairness regularization, DCEVAE generates a counterfactual fair dataset while losing less original information. Also, DCEVAE generates natural counterfactual images by only flipping sensitive information. Additionally, we theoretically show the differences in the covariance structures of DCEVAE and prior works from the perspective of the latent disentanglement.
Positions, Channels, and Layers: Fully Generalized Non-Local Network for Singer Identification
Authors: Kuo I-Yuan , Wei Wen-Li , Lin Jen-Chun
Published: September 2022

Recently, a non-local (NL) operation has been designed as the central building block for deep-net models to capture long-range dependencies (Wang et al. 2018). Despite its excellent performance, it does not consider the interaction between positions across channels and layers, which is crucial in fine-grained classification tasks. To address the limitation, we target at singer identification (SID) task and present a fully generalized non-local (FGNL) module to help identify fine-grained vocals. Specifically, we first propose a FGNL operation, which extends the NL operation to explore the correlations between positions across channels and layers. Secondly, we further apply a depth-wise convolution with Gaussian kernel in the FGNL operation to smooth feature maps for better generalization. More, we modify the squeeze-and-excitation (SE) scheme into the FGNL module to adaptively emphasize correlated feature channels to help uncover relevant feature responses and eventually the target singer. Evaluating results on the benchmark artist20 dataset shows that the FGNL module significantly improves the accuracy of the deep-net models in SID. Codes are available at https://github.com/ian-k-1217/Fully-Generalized-Non-Local-Network.
Hypothesis Disparity Regularized Mutual Information Maximization
Authors: Lao Qicheng , Jiang Xiang , Havaei Mohammad
Published: September 2022

We propose a hypothesis disparity regularized mutual information maximization (HDMI) approach to tackle unsupervised hypothesis transfer---as an effort towards unifying hypothesis transfer learning (HTL) and unsupervised domain adaptation (UDA)---where the knowledge from a source domain is transferred solely through hypotheses and adapted to the target domain in an unsupervised manner. In contrast to the prevalent HTL and UDA approaches that typically use a single hypothesis, HDMI employs multiple hypotheses to leverage the underlying distributions of the source and target hypotheses. To better utilize the crucial relationship among different hypotheses---as opposed to unconstrained optimization of each hypothesis independently---while adapting to the unlabeled target domain through mutual information maximization, HDMI incorporates a hypothesis disparity regularization that coordinates the target hypotheses jointly learn better target representations while preserving more transferable source knowledge with better-calibrated prediction uncertainty. HDMI achieves state-of-the-art adaptation performance on benchmark datasets for UDA in the context of HTL, without the need to access the source data during the adaptation.
Learnable Dynamic Temporal Pooling for Time Series Classification
Authors: Lee Dongha , Lee Seonghyeon , Yu Hwanjo
Published: September 2022

With the increase of available time series data, predicting their class labels has been one of the most important challenges in a wide range of disciplines. Recent studies on time series classification show that convolutional neural networks (CNN) achieved the state-of-the-art performance as a single classifier. In this work, pointing out that the global pooling layer that is usually adopted by existing CNN classifiers discards the temporal information of high-level features, we present a dynamic temporal pooling (DTP) technique that reduces the temporal size of hidden representations by aggregating the features at the segment-level. For the partition of a whole series into multiple segments, we utilize dynamic time warping (DTW) to align each time point in a temporal order with the prototypical features of the segments, which can be optimized simultaneously with the network parameters of CNN classifiers. The DTP layer combined with a fully-connected layer helps to extract further discriminative features considering their temporal position within an input time series. Extensive experiments on both univariate and multivariate time series datasets show that our proposed pooling significantly improves the classification performance.
Interpretable Embedding Procedure Knowledge Transfer via Stacked Principal Component Analysis and Graph Neural Network
Authors: Lee Seunghyun , Song Byung Cheol
Published: September 2022

Knowledge distillation (KD) is one of the most useful techniques for light-weight neural networks. Although neural networks have a clear purpose of embedding datasets into the low-dimensional space, the existing knowledge was quite far
from this purpose and provided only limited information. We argue that good knowledge should be able to interpret the embedding procedure. This paper proposes a method of generating interpretable embedding procedure (IEP) knowledge based on principal component analysis, and distilling it based on a message passing neural network. Experimental results show that the student network trained by the proposed KD method improves 2.28% in the CIFAR100 dataset, which is a higher performance than the state-of-the-art (SOTA) method. We also demonstrate that the embedding procedure knowledge is interpretable via visualization of the proposed KD process. The implemented code is available at https://github.com/sseung0703/IEPKT.
Study of the mechanical properties of polylactide composites with jute reinforcements
Authors: Lascano Diego , Aljaro Cristobal , Fages Eduardo , Rojas-Lema Sandra , Ivorra-Martinez Juan , Montanes Nestor
Published: September 2022

Concern has arisen in recent years over the excessive use of oil-based materials, making the development of new materials with a low environmental impact all the more urgent. In this study, environmentally friendly composites were obtained using a thermoplastic polylactide (PLA) matrix with jute fibers (fabrics and non-woven mats) as reinforcement. PLA/jute biocomposites were manufactured by thermocompression. The effect of the amount of jute fiber reinforcement (in the 30–50 wt.% range) on the tensile and flexural properties of these composites was analyzed, and the fiber–matrix interaction was assessed by scanning electron microscopy. The results show that thermocompression molding is a simple technique for obtaining high-environmental-efficiency biocomposites with high reinforcement loadings (up to 50 wt.%). As expected, the tensile properties are directly related to the amount of fiber loading, as well as the directionality of these fibers in the composite. Mechanical performance is also highly dependent on fiber–matrix interactions. These biocomposites could be attractive as lightweight interior panels in the automotive industry, cases/covers in electric/electronic applications, and turbine blade components in the wind energy industry, among others, due to their balanced mechanical properties and the rather complex shapes that could be obtained by thermocompression.
CONVERSION OF ADSORPTION PROPERTIES OF BENTONITE CLAYS UNDER COMPLEX TECHNOGENIC IMPACT
Authors:   Perm National Research Polytechnic University, Anyukhina A. V. , Seredin V. V. ,   Perm State University, Mironova A. A. ,   Perm State University
Published: September 2022

The results of study of the change in the sorption index by chemical, thermal and mechanical treatment of bentonite clay have been obtained. They showed that during the complex processing of samples with mechanical pressure, temperature and cationic solutions, there is a general tendency to an increase in the sorption index. It was found that the greatest effect on the activation of sorption properties is exerted by vertical pressure, heat treatment of samples at 200 оС and saturation with a cationic solution of ferric chloride, which allows us to consider this processing of samples as the most effective for the samples of this study
FORECASTING AND SEARCHING FOR GOLD MINERALIZATION WITHIN THE AMGUEMO-KANCHALAN METALLOGENIC ZONE (CHUKOTKA)
Authors:   Perm State University, Goldyrev V. N.
Published: September 2022

Despite the existence of a well-known set of the forecast and search criteria, only three gold and silver epithermal deposits were discovered on the territory of the Amguem-Kanchalan metallogenic zone, which has significant resource potential. The resource base of the enterprises developing them is extremely limited. The aim of the study is to clarify the regional and local conditions for localization of epithermal gold-silver mineralization, forecasting and searching for promising structures and areas of its manifestation within the AmGuemo-Kanchalan metallogenic zone. The airborne geological studies of the territory were carried out. The proposed areas potential for gold mineralization are identified. Recommendations for conducting search and evaluation work are given.