bluuebunny/crossref_metadata_2025 huggingface dataset:
Combating Mode Collapse via Offline Manifold Entropy Estimation
Authors: Liu Haozhe , Li Bing , Wu Haoqian , Liang Hanbang , Huang Yawen , Li Yuexiang , Ghanem Bernard , Zheng Yefeng
Published: June 2023

Generative Adversarial Networks (GANs) have shown compelling results in various tasks and applications in recent years. However, mode collapse remains a critical problem in GANs. In this paper, we propose a novel training pipeline to address the mode collapse issue of GANs. Different from existing methods, we propose to generalize the discriminator as feature embedding and maximize the entropy of distributions in the embedding space learned by the discriminator. Specifically, two regularization terms, i.e., Deep Local Linear Embedding (DLLE) and Deep Isometric feature Mapping (DIsoMap), are introduced to encourage the discriminator to learn the structural information embedded in the data, such that the embedding space learned by the discriminator can be well-formed. Based on the well-learned embedding space supported by the discriminator, a non-parametric entropy estimator is designed to efficiently maximize the entropy of embedding vectors, playing as an approximation of maximizing the entropy of the generated distribution. By improving the discriminator and maximizing the distance of the most similar samples in the embedding space, our pipeline effectively reduces the mode collapse without sacrificing the quality of generated samples. Extensive experimental results show the effectiveness of our method which outperforms the GAN baseline, MaF-GAN on CelebA (9.13 vs. 12.43 in FID) and surpasses the recent state-of-the-art energy-based model on the ANIMEFACE dataset (2.80 vs. 2.26 in Inception score).
Hard Sample Aware Network for Contrastive Deep Graph Clustering
Authors: Liu Yue , Yang Xihong , Zhou Sihang , Liu Xinwang , Wang Zhen , Liang Ke , Tu Wenxuan , Li Liang , Duan Jingcan , Chen Cancan
Published: June 2023

Contrastive deep graph clustering, which aims to divide nodes into disjoint groups via contrastive mechanisms, is a challenging research spot. Among the recent works, hard sample mining-based algorithms have achieved great attention for their promising performance. However, we find that the existing hard sample mining methods have two problems as follows. 1) In the hardness measurement, the important structural information is overlooked for similarity calculation, degrading the representativeness of the selected hard negative samples. 2) Previous works merely focus on the hard negative sample pairs while neglecting the hard positive sample pairs. Nevertheless, samples within the same cluster but with low similarity should also be carefully learned. To solve the problems, we propose a novel contrastive deep graph clustering method dubbed Hard Sample Aware Network (HSAN) by introducing a comprehensive similarity measure criterion and a general dynamic sample weighing strategy. Concretely, in our algorithm, the similarities between samples are calculated by considering both the attribute embeddings and the structure embeddings, better revealing sample relationships and assisting hardness measurement. Moreover, under the guidance of the carefully collected high-confidence clustering information, our proposed weight modulating function will first recognize the positive and negative samples and then dynamically up-weight the hard sample pairs while down-weighting the easy ones. In this way, our method can mine not only the hard negative samples but also the hard positive sample, thus improving the discriminative capability of the samples further. Extensive experiments and analyses demonstrate the superiority and effectiveness of our proposed method. The source code of HSAN is shared at https://github.com/yueliu1999/HSAN and a collection (papers, codes and, datasets) of deep graph clustering is shared at https://github.com/yueliu1999/Awesome-Deep-Graph-Clustering on Github.
Multi-View Domain Adaptive Object Detection on Camera Networks
Authors: Lu Yan , Zhong Zhun , Shu Yuanchao
Published: June 2023

In this paper, we study a new domain adaptation setting on camera networks, namely Multi-View Domain Adaptive Object Detection (MVDA-OD), in which labeled source data is unavailable in the target adaptation process and target data is captured from multiple overlapping cameras. In such a challenging context, existing methods including adversarial training and self-training fall short due to multi-domain data shift and the lack of source data. To tackle this problem, we propose a novel training framework consisting of two stages. First, we pre-train the backbone using self-supervised learning, in which a multi-view association is developed to construct an effective pretext task. Second, we fine-tune the detection head using robust self-training, where a tracking-based single-view augmentation is introduced to achieve weak-hard consistency learning. By doing so, an object detection model can take advantage of informative samples generated by multi-view association and single-view augmentation to learn discriminative backbones as well as robust detection classifiers. Experiments on two real-world multi-camera datasets demonstrate significant advantages of our approach over the state-of-the-art domain adaptive object detection methods.
Weight Predictor Network with Feature Selection for Small Sample Tabular Biomedical Data
Authors: Margeloiu Andrei , Simidjievski Nikola , Liò Pietro , Jamnik Mateja
Published: June 2023

Tabular biomedical data is often high-dimensional but with a very small number of samples. Although recent work showed that well-regularised simple neural networks could outperform more sophisticated architectures on tabular data, they are still prone to overfitting on tiny datasets with many potentially irrelevant features. To combat these issues, we propose Weight Predictor Network with Feature Selection (WPFS) for learning neural networks from high-dimensional and small sample data by reducing the number of learnable parameters and simultaneously performing feature selection. In addition to the classification network, WPFS uses two small auxiliary networks that together output the weights of the first layer of the classification model. We evaluate on nine real-world biomedical datasets and demonstrate that WPFS outperforms other standard as well as more recent methods typically applied to tabular data. Furthermore, we investigate the proposed feature selection mechanism and show that it improves performance while providing useful insights into the learning task.
The Unreasonable Effectiveness of Deep Evidential Regression
Authors: Meinert Nis , Gawlikowski Jakob , Lavin Alexander
Published: June 2023

There is a significant need for principled uncertainty reasoning in machine learning systems as they are increasingly deployed in safety-critical domains.
A new approach with uncertainty-aware regression-based neural networks (NNs), based on learning evidential distributions for aleatoric and epistemic uncertainties, shows promise over traditional deterministic methods and typical Bayesian NNs, notably with the capabilities to disentangle aleatoric and epistemic uncertainties.
Despite some empirical success of Deep Evidential Regression (DER), there are important gaps in the mathematical foundation that raise the question of why the proposed technique seemingly works.
We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a heuristic rather than an exact uncertainty quantification.
We go on to discuss corrections and redefinitions of how aleatoric and epistemic uncertainties should be extracted from NNs.
Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization
Authors: Mutti Mirco , De Santi Riccardo , Rossi Emanuele , Calderon Juan Felipe , Bronstein Michael , Restelli Marcello
Published: June 2023

In the sequential decision making setting, an agent aims to achieve systematic generalization over a large, possibly infinite, set of environments. Such environments are modeled as discrete Markov decision processes with both states and actions represented through a feature vector. The underlying structure of the environments allows the transition dynamics to be factored into two components: one that is environment-specific and another that is shared. Consider a set of environments that share the laws of motion as an example. In this setting, the agent can take a finite amount of reward-free interactions from a subset of these environments. The agent then must be able to approximately solve any planning task defined over any environment in the original set, relying on the above interactions only. Can we design a provably efficient algorithm that achieves this ambitious goal of systematic generalization? In this paper, we give a partially positive answer to this question. First, we provide a tractable formulation of systematic generalization by employing a causal viewpoint. Then, under specific structural assumptions, we provide a simple learning algorithm that guarantees any desired planning error up to an unavoidable sub-optimality term, while showcasing a polynomial sample complexity.
Behavioral Learning in Security Games: Threat of Multi-Step Manipulative Attacks
Authors: Nguyen Thanh H. , Sinha Arunesh
Published: June 2023

This paper studies the problem of multi-step manipulative attacks in Stackelberg security games, in which a clever attacker attempts to orchestrate its attacks over multiple time steps to mislead the defender's learning of the attacker's behavior. This attack manipulation eventually influences the defender's patrol strategy towards the attacker's benefit. Previous work along this line of research only focuses on one-shot games in which the defender learns the attacker's behavior and then designs a corresponding strategy only once. Our work, on the other hand, investigates the long-term impact of the attacker's manipulation in which current attack and defense choices of players determine the future learning and patrol planning of the defender. This paper has three key contributions. First, we introduce a new multi-step manipulative attack game model that captures the impact of sequential manipulative attacks carried out by the attacker over the entire time horizon. Second, we propose a new algorithm to compute an optimal manipulative attack plan for the attacker, which tackles the challenge of multiple connected optimization components involved in the computation across multiple time steps. Finally, we present extensive experimental results on the impact of such misleading attacks, showing a significant benefit for the attacker and loss for the defender.
Ising-Traffic: Using Ising Machine Learning to Predict Traffic Congestion under Uncertainty
Authors: Pan Zhenyu , Sharma Anshujit , Hu Jerry Yao-Chieh , Liu Zhuo , Li Ang , Liu Han , Huang Michael , Geng Tony
Published: June 2023

This paper addresses the challenges in accurate and real-time traffic congestion prediction under uncertainty by proposing Ising-Traffic, a dual-model Ising-based traffic prediction framework that delivers higher accuracy and lower latency than SOTA solutions. While traditional solutions face the dilemma from the trade-off between algorithm complexity and computational efficiency, our Ising-based method breaks away from the trade-off leveraging the Ising model's strong expressivity and the Ising machine's strong computation power. In particular, Ising-Traffic formulates traffic prediction under uncertainty into two Ising models: Reconstruct-Ising and Predict-Ising. Reconstruct-Ising is mapped onto modern Ising machines and handles uncertainty in traffic accurately with negligible latency and energy consumption, while Predict-Ising is mapped onto traditional processors and predicts future congestion precisely with only at most 1.8% computational demands of existing solutions. Our evaluation shows Ising-Traffic delivers on average 98X speedups and 5% accuracy improvement over SOTA.
Balanced Column-Wise Block Pruning for Maximizing GPU Parallelism
Authors: Park Cheonjun , Park Mincheol , Oh Hyun Jae , Kim Minkyu , Yoon Myung Kuk , Kim Suhyun , Ro Won Woo
Published: June 2023

Pruning has been an effective solution to reduce the number of computations and the memory requirement in deep learning.
The pruning unit plays an important role in exploiting the GPU resources efficiently.
The filter is proposed as a simple pruning unit of structured pruning.
However, since the filter is quite large as pruning unit, the accuracy drop is considerable with a high pruning ratio.
GPU rearranges the weight and input tensors into tiles (blocks) for efficient computation.
To fully utilize GPU resources, this tile structure should be considered, which is the goal of block pruning.
However, previous block pruning prunes both row vectors and column vectors.
Pruning of row vectors in a tile corresponds to filter pruning, and it also interferes with column-wise block pruning of the following layer.
In contrast, column vectors are much smaller than row vectors and can achieve lower accuracy drop.
Additionally, if the pruning ratio for each tile is different,
GPU utilization can be limited by imbalanced workloads by irregular-sized blocks.
The same pruning ratio for the weight tiles processed in parallel enables the actual inference process to fully utilize the resources without idle time.
This paper proposes balanced column-wise block pruning, named BCBP, to satisfy two conditions: the column-wise minimal size of the pruning unit and balanced workloads.
We demonstrate that BCBP is superior to previous pruning methods through comprehensive experiments.
Dynamic Structure Pruning for Compressing CNNs
Authors: Park Jun-Hyung , Kim Yeachan , Kim Junho , Choi Joon-Young , Lee SangKeun
Published: June 2023

Structure pruning is an effective method to compress and accelerate neural networks. While filter and channel pruning are preferable to other structure pruning methods in terms of realistic acceleration and hardware compatibility, pruning methods with a finer granularity, such as intra-channel pruning, are expected to be capable of yielding more compact and computationally efficient networks. Typical intra-channel pruning methods utilize a static and hand-crafted pruning granularity due to a large search space, which leaves room for improvement in their pruning performance. In this work, we introduce a novel structure pruning method, termed as dynamic structure pruning, to identify optimal pruning granularities for intra-channel pruning. In contrast to existing intra-channel pruning methods, the proposed method automatically optimizes dynamic pruning granularities in each layer while training deep neural networks. To achieve this, we propose a differentiable group learning method designed to efficiently learn a pruning granularity based on gradient-based learning of filter groups. The experimental results show that dynamic structure pruning achieves state-of-the-art pruning performance and better realistic acceleration on a GPU compared with channel pruning. In particular, it reduces the FLOPs of ResNet50 by 71.85% without accuracy degradation on the ImageNet dataset. Our code is available at https://github.com/irishev/DSP.
Self-Supervised Audio-Visual Representation Learning with Relaxed Cross-Modal Synchronicity
Authors: Sarkar Pritam , Etemad Ali
Published: June 2023

We present CrissCross, a self-supervised framework for learning audio-visual representations. A novel notion is introduced in our framework whereby in addition to learning the intra-modal and standard 'synchronous' cross-modal relations, CrissCross also learns 'asynchronous' cross-modal relationships. We perform in-depth studies showing that by relaxing the temporal synchronicity between the audio and visual modalities, the network learns strong generalized representations useful for a variety of downstream tasks. To pretrain our proposed solution, we use 3 different datasets with varying sizes, Kinetics-Sound, Kinetics400, and AudioSet. The learned representations are evaluated on a number of downstream tasks namely action recognition, sound classification, and action retrieval. Our experiments show that CrissCross either outperforms or achieves performances on par with the current state-of-the-art self-supervised methods on action recognition and action retrieval with UCF101 and HMDB51, as well as sound classification with ESC50 and DCASE. Moreover, CrissCross outperforms fully-supervised pretraining while pretrained on Kinetics-Sound.
A Fair Generative Model Using LeCam Divergence
Authors: Um Soobin , Suh Changho
Published: June 2023

We explore a fairness-related challenge that arises in generative models. The challenge is that biased training data with imbalanced demographics may yield a high asymmetry in size of generated samples across distinct groups. We focus on practically-relevant scenarios wherein demographic labels are not available and therefore the design of a fair generative model is non-straightforward. In this paper, we propose an optimization framework that regulates the unfairness under such practical settings via one statistical measure, LeCam (LC)-divergence. Specifically to quantify the degree of unfairness, we employ a balanced-yet-small reference dataset and then measure its distance with generated samples using the LC-divergence, which is shown to be particularly instrumental to a small size of the reference dataset. We take a variational optimization approach to implement the LC-based measure. Experiments on benchmark real datasets demonstrate that the proposed framework can significantly improve the fairness performance while maintaining realistic sample quality for a wide range of the reference set size all the way down to 1% relative to training set.
Machines of Finite Depth: Towards a Formalization of Neural Networks
Authors: Vertechi Pietro , Bergomi Mattia G.
Published: June 2023

We provide a unifying framework where artificial neural networks and their architectures can be formally described as particular cases of a general mathematical construction---machines of finite depth. Unlike neural networks, machines have a precise definition, from which several properties follow naturally. Machines of finite depth are modular (they can be combined), efficiently computable, and differentiable. The backward pass of a machine is again a machine and can be computed without overhead using the same procedure as the forward pass. We prove this statement theoretically and practically via a unified implementation that generalizes several classical architectures---dense, convolutional, and recurrent neural networks with a rich shortcut structure---and their respective backpropagation rules.
Stability-Based Generalization Analysis for Mixtures of Pointwise and Pairwise Learning
Authors: Wang Jiahuan , Chen Jun , Chen Hong , Gu Bin , Li Weifu , Tang Xin
Published: June 2023

Recently, some mixture algorithms of pointwise and pairwise learning (PPL) have been formulated by employing the hybrid error metric of “pointwise loss + pairwise loss” and have shown empirical effectiveness on feature selection, ranking and recommendation tasks. However, to the best of our knowledge, the learning theory foundation of PPL has not been touched in the existing works. In this paper, we try to fill this theoretical gap by investigating the generalization properties of PPL. After extending the definitions of algorithmic stability to the PPL setting, we establish the high-probability generalization bounds for uniformly stable PPL algorithms. Moreover, explicit convergence rates of stochastic gradient descent (SGD) and regularized risk minimization (RRM) for PPL are stated by developing the stability analysis technique of pairwise learning. In addition, the refined generalization bounds of PPL are obtained by replacing uniform stability with on-average stability.
Feature Distribution Fitting with Direction-Driven Weighting for Few-Shot Images Classification
Authors: Wei Xin , Du Wei , Wan Huan , Min Weidong
Published: June 2023

Few-shot learning has received increasing attention and witnessed significant advances in recent years. However, most of the few-shot learning methods focus on the optimization of training process, and the learning of metric and sample generating networks. They ignore the importance of learning the ground-truth feature distributions of few-shot classes. This paper proposes a direction-driven weighting method to make the feature distributions of few-shot classes precisely fit the ground-truth distributions. The learned feature distributions can generate an unlimited number of training samples for the few-shot classes to avoid overfitting. Specifically, the proposed method consists of two optimization strategies. The direction-driven strategy is for capturing more complete direction information that can describe the feature distributions. The similarity-weighting strategy is proposed to estimate the impact of different classes in the fitting procedure and assign corresponding weights. Our method outperforms the current state-of-the-art performance by an average of 3% for 1-shot on standard few-shot learning benchmarks like miniImageNet, CIFAR-FS, and CUB. The excellent performance and compelling visualization show that our method can more accurately estimate the ground-truth distributions.
Decentralized Stochastic Multi-Player Multi-Armed Walking Bandits
Authors: Xiong Guojun , Li Jian
Published: June 2023

Multi-player multi-armed bandit is an increasingly relevant decision-making problem, motivated by applications to cognitive radio systems. Most research for this problem focuses exclusively on the settings that players have full access to all arms and receive no reward when pulling the same arm. Hence all players solve the same bandit problem with the goal of maximizing their cumulative reward. However, these settings neglect several important factors in many real-world applications, where players have limited access to a dynamic local subset of arms (i.e., an arm could sometimes be ``walking'' and not accessible to the player). To this end, this paper proposes a multi-player multi-armed walking bandits model, aiming to address aforementioned modeling issues. The goal now is to maximize the reward, however, players can only pull arms from the local subset and only collect a full reward if no other players pull the same arm. We adopt Upper Confidence Bound (UCB) to deal with the exploration-exploitation tradeoff and employ distributed optimization techniques to properly handle collisions. By carefully integrating these two techniques, we propose a decentralized algorithm with near-optimal guarantee on the regret, and can be easily implemented to obtain competitive empirical performance.
AdaTask: A Task-Aware Adaptive Learning Rate Approach to Multi-Task Learning
Authors: Yang Enneng , Pan Junwei , Wang Ximei , Yu Haibin , Shen Li , Chen Xihua , Xiao Lei , Jiang Jie , Guo Guibing
Published: June 2023

Multi-task learning (MTL) models have demonstrated impressive results in computer vision, natural language processing, and recommender systems. Even though many approaches have been proposed, how well these approaches balance different tasks on each parameter still remains unclear. In this paper, we propose to measure the task dominance degree of a parameter by the total updates of each task on this parameter. Specifically, we compute the total updates by the exponentially decaying Average of the squared Updates (AU) on a parameter from the corresponding task. Based on this novel metric, we observe that many parameters in existing MTL methods, especially those in the higher shared layers, are still dominated by one or several tasks. The dominance of AU is mainly due to the dominance of accumulative gradients from one or several tasks. Motivated by this, we propose a Task-wise Adaptive learning rate approach, AdaTask in short, to separate the accumulative gradients and hence the learning rate of each task for each parameter in adaptive learning rate approaches (e.g., AdaGrad, RMSProp, and Adam). Comprehensive experiments on computer vision and recommender system MTL datasets demonstrate that AdaTask significantly improves the performance of dominated tasks, resulting SOTA average task-wise performance. Analysis on both synthetic and real-world datasets shows AdaTask balance parameters in every shared layer well.
Generalized Semantic Segmentation by Self-Supervised Source Domain Projection and Multi-Level Contrastive Learning
Authors: Yang Liwei , Gu Xiang , Sun Jian
Published: June 2023

Deep networks trained on the source domain show degraded performance when tested on unseen target domain data. To enhance the model's generalization ability, most existing domain generalization methods learn domain invariant features by suppressing domain sensitive features. Different from them, we propose a Domain Projection and Contrastive Learning (DPCL) approach for generalized semantic segmentation, which includes two modules: Self-supervised Source Domain Projection (SSDP) and Multi-Level Contrastive Learning (MLCL). SSDP aims to reduce domain gap by projecting data to the source domain, while MLCL is a learning scheme to learn discriminative and generalizable features on the projected data. During test time, we first project the target data by SSDP to mitigate domain shift, then generate the segmentation results by the learned segmentation network based on MLCL. At test time, we can update the projected data by minimizing our proposed pixel-to-pixel contrastive loss to obtain better results. Extensive experiments for semantic segmentation demonstrate the favorable generalization capability of our method on benchmark datasets.
Continual Variational Autoencoder via Continual Generative Knowledge Distillation
Authors: Ye Fei , Bors Adrian G.
Published: June 2023

Humans and other living beings have the ability of short and long-term memorization during their entire lifespan. However, most existing Continual Learning (CL) methods can only account for short-term information when training on infinite streams of data. In this paper, we develop a new unsupervised continual learning framework consisting of two memory systems using Variational Autoencoders (VAEs). We develop a Short-Term Memory (STM), and a parameterised scalable memory implemented by a Teacher model aiming to preserve the long-term information. To incrementally enrich the Teacher's knowledge during training, we propose the Knowledge Incremental Assimilation Mechanism (KIAM), which evaluates the knowledge similarity between the STM and the already accumulated information as signals to expand the Teacher's capacity. Then we train a VAE as a Student module and propose a new Knowledge Distillation (KD) approach that gradually transfers generative knowledge from the Teacher to the Student module. To ensure the quality and diversity of knowledge in KD, we propose a new expert pruning approach that selectively removes the Teacher's redundant parameters, associated with unnecessary experts which have learnt overlapping information with other experts. This mechanism further reduces the complexity of the Teacher's module while ensuring the diversity of knowledge for the KD procedure. We show theoretically and empirically that the proposed framework can train a statistically diversified Teacher module for continual VAE learning which is applicable to learning infinite data streams.
Priori Anchor Labels Supervised Scalable Multi-View Bipartite Graph Clustering
Authors: You Jiali , Ren Zhenwen , You Xiaojian , Li Haoran , Yao Yuancheng
Published: June 2023

Although multi-view clustering (MVC) has achieved remarkable performance by integrating the complementary information of views, it is inefficient when facing scalable data. Proverbially, anchor strategy can mitigate such a challenge a certain extent. However, the unsupervised dynamic strategy usually cannot obtain the optimal anchors for MVC. The main reasons are that it does not consider the fairness of different views and lacks the priori supervised guidance. To completely solve these problems, we first propose the priori anchor graph regularization (PAGG) for scalable multi-view bipartite graph clustering, dubbed as SMGC method. Specifically, SMGC learns a few representative consensus anchors to simulate the numerous view data well, and constructs a bipartite graph to bridge the affinities between the anchors and original data points. In order to largely improve the quality of anchors, PAGG predefines prior anchor labels to constrain the anchors with discriminative cluster structure and fair view allocation, such that a better bipartite graph can be obtained for fast clustering. Experimentally, abundant of experiments are accomplished on six scalable benchmark datasets, and the experimental results fully demonstrate the effectiveness and efficiency of our SMGC.
STARS: Spatial-Temporal Active Re-sampling for Label-Efficient Learning from Noisy Annotations
Authors: Yu Dayou , Shi Weishi , Yu Qi
Published: June 2023

Active learning (AL) aims to sample the most informative data instances for labeling, which makes the model fitting data efficient while significantly reducing the annotation cost. However, most existing AL models make a strong assumption that the annotated data instances are always assigned correct labels, which may not hold true in many practical settings. In this paper, we develop a theoretical framework to formally analyze the impact of noisy annotations and show that systematically re-sampling guarantees to reduce the noise rate, which can lead to improved generalization capability. More importantly, the theoretical framework demonstrates the key benefit of conducting active re-sampling on label-efficient learning, which is critical for AL. The theoretical results also suggest essential properties of an active re-sampling function with a fast convergence speed and guaranteed error reduction. This inspires us to design a novel spatial-temporal active re-sampling function by leveraging the important spatial and temporal properties of maximum-margin classifiers. Extensive experiments conducted on both synthetic and real-world data clearly demonstrate the effectiveness of the proposed active re-sampling function.
Are Transformers Effective for Time Series Forecasting?
Authors: Zeng Ailing , Chen Muxi , Zhang Lei , Xu Qiang
Published: June 2023

Recently, there has been a surge of Transformer-based solutions for the long-term time series forecasting (LTSF) task. Despite the growing performance over the past few years, we question the validity of this line of research in this work. Specifically, Transformers is arguably the most successful solution to extract the semantic correlations among the elements in a long sequence. However, in time series modeling, we are to extract the temporal relations in an ordered set of continuous points. While employing positional encoding and using tokens to embed sub-series in Transformers facilitate preserving some ordering information, the nature of the permutation-invariant self-attention mechanism inevitably results in temporal information loss.
To validate our claim, we introduce a set of embarrassingly simple one-layer linear models named LTSF-Linear for comparison. Experimental results on nine real-life datasets show that LTSF-Linear surprisingly outperforms existing sophisticated Transformer-based LTSF models in all cases, and often by a large margin. Moreover, we conduct comprehensive empirical studies to explore the impacts of various design elements of LTSF models on their temporal relation extraction capability. We hope this surprising finding opens up new research directions for the LTSF task. We also advocate revisiting the validity of Transformer-based solutions for other time series analysis tasks (e.g., anomaly detection) in the future.
Behavior Estimation from Multi-Source Data for Offline Reinforcement Learning
Authors: Zhang Guoxi , Kashima Hisashi
Published: June 2023

Offline reinforcement learning (RL) have received rising interest due to its appealing data efficiency. The present study addresses behavior estimation, a task that aims at estimating the data-generating policy. In particular, this work considers a scenario where data are collected from multiple sources. Neglecting data heterogeneity, existing approaches cannot provide good estimates and impede policy learning. To overcome this drawback, the present study proposes a latent variable model and a model-learning algorithm to infer a set of policies from data, which allows an agent to use as behavior policy the policy that best describes a particular trajectory. To illustrate the benefit of such a fine-grained characterization for multi-source data, this work showcases how the proposed model can be incorporated into an existing offline RL algorithm. Lastly, with extensive empirical evaluation this work confirms the risks of neglecting data heterogeneity and the efficacy of the proposed model.
ContraFeat: Contrasting Deep Features for Semantic Discovery
Authors: Zhu Xinqi , Xu Chang , Tao Dacheng
Published: June 2023

StyleGAN has shown strong potential for disentangled semantic control, thanks to its special design of multi-layer intermediate latent variables. However, existing semantic discovery methods on StyleGAN rely on manual selection of modified latent layers to obtain satisfactory manipulation results, which is tedious and demanding. In this paper, we propose a model that automates this process and achieves state-of-the-art semantic discovery performance. The model consists of an attention-equipped navigator module and losses contrasting deep-feature changes. We propose two model variants, with one contrasting samples in a binary manner, and another one contrasting samples with learned prototype variation patterns. The proposed losses are computed with pretrained deep features, based on our assumption that the features implicitly possess the desired semantic variation structure including consistency and orthogonality. Additionally, we design two metrics to quantitatively evaluate the performance of semantic discovery methods on FFHQ dataset, and also show that disentangled representations can be derived via a simple training process. Experimentally, we show that our models achieve state-of-the-art semantic discovery results without relying on layer-wise manual selection, and these discovered semantics can be used to manipulate real-world images.
Mixed-Variable Black-Box Optimisation Using Value Proposal Trees
Authors: Zuo Yan , Nguyen Vu , Dezfouli Amir , Alexander David , Muir Benjamin Ward , Chades Iadine
Published: June 2023

Many real-world optimisation problems are defined over both categorical and continuous variables, yet efficient optimisation methods such as Bayesian Optimisation (BO) are ill-equipped to handle such mixed-variable search spaces. The optimisation breadth introduced by categorical variables in the mixed-input setting has seen recent approaches operating on local trust regions, but these methods can be greedy in suboptimal regions of the search space. In this paper, we adopt a holistic view and aim to consolidate optimisation of the categorical and continuous sub-spaces under a single acquisition metric. We develop a tree-based method which retains a global view of the optimisation spaces by identifying regions in the search space with high potential candidates which we call value proposals. Our method uses these proposals to make selections on both the categorical and continuous components of the input. We show that this approach significantly outperforms existing mixed-variable optimisation approaches across several mixed-variable black-box optimisation tasks.
Reading-With: A Collaborative Method for Inclusive Architectural Histories
Authors: Hultzsch Anne , Pérez Martínez Sol
Published: June 2023

This article describes reading-with, a new collaborative reading method developed as part of the ERC-funded project WoWA (Women Writing Architecture 1700-1900). It offers architectural and other historians a starting point to produce inclusive histories based on the writings of those so far invisible within disciplinary canons. Arguing that historiography can be made more inclusive if the historian’s approach to their sources changes, reading-with introduces a set of precise and timed reading tasks organised in layers to focus the researcher’s attention on different aspects of the narration as well as their response to the text. Based on methods developed in the fields of psychology and feminist oral history, it aims to disrupt learned, canonical reading methods, paying attention to the relationship between reader, text, narrator, and context. Performed by several groups of researchers and master students in Switzerland and Chile in 2022-23, the method has been tested with texts authored by women in the 18th and 19th centuries. It foregrounds their architectural agency and influence on architectural cultures, even if their links to the professional sphere have so far been regarded as marginal. Reading-with is an invitation to read otherwise to centre the experiences and agency of those not yet acknowledged.
བཀྲ་ཤིས་སྒང་རྫོང་ཁག་ཨུ་རྫོ་རོང་རྒེད་འོག་ནང་ལུ་ས་ཉེན་གྱིི་ཉིནམ་བརྩི་སྲུང་ཞུ་སྲོལ།
Authors: བཟང་པོ། བསོད་ནམས་ , ཆོས་སྒྲོན། ལས་སྐྱིད་
Published: June 2023

ས་ཉེན་གྱི་ཉིནམ་འདི་ བཀྲིས་སྒང་རྫོང་ཁག་ཨུ་རྫོ་རོང་རྒེད་འོག་གི་གཡུས་ཚན་ཁག་ཚུ་ནང་ ལོ་བསྟར་རང་ཟླ་ ༦ པའི་ཚེས་ ༡༨ ལུ་བརྩི་སྲུང་ཞུ་སྲོལ་འདུག། དེ་ཡང་ གནམ་བྱཱར་གྱི་སྐབས་ འབུཔ་སྲིན་འཐོན་པའི་དུས་ཚོད་འབདཝ་ ལས་ ཞིང་ནང་ལཱ་མ་འབད་བར་ བརྩི་སྲུང་འབད་བའི་ཉིནམ་ཅིག་ཨིན་པས། ཨིན་རུང་ འདིའི་སྐོར་ལས་ ད་ཚུན་གྱི་ བར་ན་ ཞིབ་འཚོལ་འབད་དེ་ ཡིག་ཐོག་ལུ་ཆེནམོ་ཅིག་བཀོད་དེ་མིན་འདུག། ཞིབ་འཚོལ་ཐབས་ལམ་ཡང་ ཁུངས་ བཙན་ཐབས་ལམ་ལས་ རིམ་སྒྲིག་ཆ་ཙམ་གྱི་དྲི་བ་དྲིས་ལན་ལག་ལེན་འཐབ་སྟེ་ ཞིབ་འཚོལ་འབད་ཡི། ཞིབ་ འཚོལ་གྱི་གྲུབ་འབྲས་དང་བསྟུནམ་ད་ ཉིནམ་འདི་གནམ་བྱཱར་ལོ་ཐོག་སྐྱེ་བའི་དུས་ཚོད་ལུ་ སྡིག་པ་ལུ་འཛེམས་ཏེ་ ཉིནམ་གཅིག་གི་རིང་ལུ་ སོ་ནམ་གྱི་ལཱ་ ས་བརྐོ་རྡོ་སློག་ཚུ་མ་འབད་བར་ འཛེམས་སྡོད་པའི་ཚུལ་ཨིན་པས། དེ་སྦེ་ སྡོད་དགོ་མི་དེ་ཡང་ ལོ་ཐོག་ལེགས་ནིའི་དོན་ལུ་ བརྩི་སྲུང་ཞུཝ་ཨིན་པའི་མངོན་གསལ་བྱུང་ཡོད། ཞིབ་འཚོལ་འདི་ གཡུས་སྒོའི་ལམ་སྲོལ་ཉམས་སྲུང་གི་དོན་ལུ་དམིགས་ཏེ་ ཞིབ་འཚོལ་འབད་དེ་ཡོད། ཤུལ་ལས་ ཞིབ་འཚོལ་གྱི་དོན་ ཚན་འདིའི་ཐོག་ལུ་ གཡུས་ཚན་གཞན་ཁར་བརྩི་སྲུང་ཞུ་སྲོལ་ཡོད་མི་དང་ ག་བསྡུར་འབད་དེ་ ཞིབ་འཚོལ་འབད་ ནིའི་གོ་སྐབས་ཡོདཔ་ཨིན།
Consistency of Mycorrhizal Effectiveness on Maize Growth and P Uptake in Two Generations of Pot Culture Using Andisol-Based Media
Authors: Cahyani Vita Ratri , Suryanti  , Minardi Slamet , Dewi Widyatmani Sih
Published: June 2023

The functional roles of Arbuscular Mycorrhiza (AM) in soils with P limitations are well documented. However, the protocol to produce effective AM inocula was still limited. This research aims at obtaining the effective AM culture in handling P constraints for maize growth on Andisols. The first experiment of pot culture I was to propagate and examine the effectiveness of AM isolates by using a factorial completely randomized design with two factors, A=media (A0: zeolite; A1: representative media=Andisol Tengaran; A2: typical media=a mixture of Andisol Tengaran+Tawangmangu; A3: typical media+Bio-RP nutrition; A4: Inceptisol) and I = AM inoculum source (I0: no inoculum; I1: AM from Andisol Tengaran; I2: AM from Andisol Tengaran + Tawangmangu; I3: AM from 8 soil types), with six replications. The second experiment investigated the consistency of mycorrhizal effectiveness by reculturing AM cultures generation I to pot cultures generation II with the same composition of the respective media. The combination treatments of A1I3, A2I2, A0I3, and A4I3 (AM cultures generation I), continued by A1I32, A2I22, A0I32, and A4I32 (AM cultures generation II) showed consistently the highest AM infectivity and effectiveness on maize growth and P uptake on Andisol-based media, and on the comparison media of zeolite and Inceptisol media, respectively.
Brucellosis with multisystemic manifestations: Case report and literature review
Authors: A Miguel , NH Domínguez , LG Moreno
Published: June 2023

Brucellosis is an infection that affects mammals’ cells caused by a species from the genus Brucella, a gram-negative, immobile, facultative, intracellular, and microaerophilic coccobacillus. In humans, the pathogen species are Brucella melitensis, B. abortus, B. suis and B. canis. We are presenting the case of a 62-year-old man with confirmed brucellosis and his consequent cardiovascular, cutaneous, gastrointestinal, hepatic, osteomuscular, renal and hematological complications. It is necessary to be highly suspicious of this infection, since its non-specific nature can be challenging for the physician. Keywords: Brucella melitensis; mammals cells; brucellosis; Mexico
EVOLUTION OF THORACOSCOPIC TREATMENT OF ATRIAL FIBRILLATION: FROM INCEPTION TO CONTEMPORARY APPROACHES
Authors: Revishvili Amiran S. , Artyukhina Elena A. , Strebkova Elizaveta D. , Malyshenko Egor S. , Kadyrova Madina
Published: June 2023

Highlights

The development and implementation of video-assisted thoracoscopic atrial fibrillation ablation since 2005 by R. Wolf and colleagues have undergone a number of modifications for improvement and simplification. A study of the evolutionary development of thoracoscopic ablation of atrial fibrillation is necessary to optimize ablation recruitment and surgical technique for different forms of atrial fibrillation.

 

Abstract

There is a variety of treatments for atrial fibrillation (AF): therapeutic, interventional and surgical. The effectiveness of these methods varies significantly depending on the form of AF. For a long time, surgery was considered the only highly effective treatment for AF. In modern arrhythmology, «Maze IV» and its modifications are considered only as a simultaneous procedure in cardiac surgery. The work of M. Haïssaguerre et al. has established that the main focus of catheter ablation (CA) in AF should be the pulmonary veins, but their isolation in non-paroxysmal AF is questionable. This discovery has stimulated the development of alternative treatment modalities for resistant AF. Recently, video-assisted thoracoscopic epicardial ablation (TSA), which does not require sternotomy, cardiopulmonary bypass and cardioplegia, has been increasingly preferred. TSA of AF is an alternative surgical treatment for isolated forms of AF on the working heart. This procedure may include: pulmonary vein ablation, posterior left atrial wall and pulmonary vein ablation and left atrial appendage exclusion, but its effectiveness varies considerably due to the heterogeneity of patients, surgical technique and follow-up period. The aim of this review is to provide data on the evolutionary development of thoracoscopic ablation of AF depending on the choice of surgical access, ablation line set and ablation device.
IN-HOSPITAL OUTCOMES OF BILATERAL INTERNAL MAMMARY ARTERY GRAFTING
Authors: Frolov Alexey V. , Zagorodnikov Nikita I. , Ivanov Sergey V. , Tarasov Roman S.
Published: June 2023

Highlights

The present study was conducted to evaluate in-hospital outcomes of one of the most effective and simultaneously controversial surgical techniques for myocardial revascularization using two internal thoracic arteries (ITA).

To this date, this is the most complete analysis that has been carried out at the Research Institute on the topic of studying bilateral internal thoracic artery grafting (BITA). The results obtained are of great importance for the subsequent evaluation of long-term results.

 

Abstract

Aim. To analyze in-hospital outcomes of BITA.

Methods. The study included 232 patients who were admitted to the Cardiac Surgery Department for coronary artery bypass grafting (CABG). All patients were divided into 2 groups depending on the use of either single or bilateral ITA conduits. Both groups included equal number of patients – 116. In-hospital outcomes were analyzed.

Results. BITA and CABG with single ITA were proven to be comparable regarding myocardial infarction, stroke, the need for percutaneous coronary intervention, death, and composite endpoint. At the same time BITA patients significantly more often had longer hospital stay and required inotropic drugs.

Conclusion. BITA is a safe procedure comparable to classic CABG in terms of in-hospital complications, but at the same time, this intervention is associated with longer hospital stay, and extended inotropic drugs use.
PULMONARY ARTERY INTIMAL SARCOMA MIMICKING PULMONARY EMBOLISM: SURGICAL TREATMENT RESULTS
Authors: Fedorov Sergey A. , Medvedev Alexander P. , Tselousova Lada M. , Zhurko Sergey A. , Chiginev Vladimir A. , Krasnikova Olga V. , Rogulina Natalia V. , Pichugin Vladimir V. , Brichkin Yuri D. , Gordetsov Alexander S. , Deryabin Roman A. , Kondratieva Anastasia R.
Published: June 2023

Highlights

The article describes a rare case of surgical treatment of pulmonary artery intimal sarcoma. The authors analyzed the causes for incorrect diagnosis of pulmonary embolism in detail, and proposed alternative options for distinguishing the diseases under consideration. Moreover, the authors analyzed different surgical treatment modalities of intracardial sarcomas, and demonstrated the prospect of using molecular hydrogen as a component of an anesthetics during surgical interventions with cardiopulmonary bypass.

 

Abstract

The article describes a rare clinical case of a patient with pulmonary artery sarcoma. The absence of specific symptoms of sarcoma, as well as features of developing cardiovascular and respiratory failure, characteristic of pulmonary embolism, contributed to the incorrect diagnosis. The article analyzes the main problems of primary diagnostics, and proposes new parameters for evaluating manifesting symptoms and the results of additional examination, which should improve the results of surgical treatment of malignant tumors.
ADVANTAGES AND DISADVANTAGES OF HEART VALVE PROSTHESES WITH FLEXIBLE SUPPORTING FRAME COMPARED WITH THE CLASSIC STENTED PROSTHESES: EVALUATION OF THE HYDRODYNAMIC INDICATORS OF THE “UNILINE” AND “TIARA” BIOPROSTHESES
Authors: Klyshnikov Kirill Yu. , Ovcharenko Evgeny A. , Barbarash Leonid S.
Published: June 2023

Highlights

The «TiAra» bioprosthetic heart valves have better hemodynamic characteristics, such as higher effective orifice area and a lower mean pressure gradient.

The «UniLine» bioprosthetic heart valve demonstrated better closing dynamic, expressed in a smaller regurgitation volume.

 

Aim. To assess hydrodynamic characteristics of the «TiAra» bioprosthetic heart valve with flexible supporting frame compared with the classic stented «UniLine» bioprosthetic aortic valve.

Methods. Using the Vivitro Pulse Duplicator (Vivitro Labs, Canada), we simulated the function of the heart via generating pulsatile flow to analyze bioprosthetic heart valves. To comprehensively assess the bioprosthesis function, three valves of each standard size (21, 23, 25 mm) were submitted to hydrodynamic testing, thus making a sample of nine bioprostheses of each model. The article provides the analysis of  the effective orifice area, mean pressure gradient, regurgitation volume, and assessment of the statistical sensitivity of the parameters between groups at p = 0.05.

Results. The assessment revealed that the «TiAra» bioprosthesis has bigger effective orifice area (p = 0.006) and lower mean pressure gradient (p = 0.02): 1.6–2.2 cm2 and 3.6–6.3 mmHg versus 1.08–1.73 cm2 and 4.8–12.1 mmHg, respectively. The regurgitation volume, however, was lower in the «UniLine» bioprostheses 0.8–4.1 mL/cycle versus 6.2–9.0 mL/cycle (p = 0.0004).

Conclusion. Despite the fact that both studied models showed good hydrodynamic performance, the prosthesis with the flexible supporting frame («TiAra») showed better results regarding its effectiveness in vitro via presenting with bigger effective orifice area and lower mean pressure gradient. At the same time, the «UniLine» stented bioprosthesis had lower regurgitation volume, i.e. better closing dynamics.